ForkRecord

Description:

This processor allows the user to fork a record into multiple records. The user must specify at least one Record Path, as a dynamic property, pointing to a field of type ARRAY containing RECORD objects. The processor accepts two modes: 'split' and 'extract'. In both modes, there is one record generated per element contained in the designated array. In the 'split' mode, each generated record will preserve the same schema as given in the input but the array will contain only one element. In the 'extract' mode, the element of the array must be of record type and will be the generated record. Additionally, in the 'extract' mode, it is possible to specify if each generated record should contain all the fields of the parent records from the root level to the extracted record. This assumes that the fields to add in the record are defined in the schema of the Record Writer controller service. See examples in the additional details documentation of this processor.

Additional Details...

Tags:

fork, record, content, array, stream, event

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values.

NameDefault ValueAllowable ValuesDescription
Record ReaderController Service API:
RecordReaderFactory
Implementations: ScriptedReader
GrokReader
JsonPathReader
JsonTreeReader
AvroReader
XMLReader
CSVReader
Specifies the Controller Service to use for reading incoming data
Record WriterController Service API:
RecordSetWriterFactory
Implementations: AvroRecordSetWriter
CSVRecordSetWriter
FreeFormTextRecordSetWriter
JsonRecordSetWriter
ScriptedRecordSetWriter
XMLRecordSetWriter
Specifies the Controller Service to use for writing out the records
Modesplit
  • Extract Generated records will be the elements of the array
  • Split Generated records will preserve the input schema and will contain a one-element array
Specifies the forking mode of the processor
Include Parent Fieldsfalse
  • true
  • false
This parameter is only valid with the 'extract' mode. If set to true, all the fields from the root level to the given array will be added as fields of each element of the array to fork.

Relationships:

NameDescription
failureIn case a FlowFile generates an error during the fork operation, it will be routed to this relationship
originalThe original FlowFiles will be routed to this relationship
forkThe FlowFiles containing the forked records will be routed to this relationship

Reads Attributes:

None specified.

Writes Attributes:

NameDescription
record.countThe generated FlowFile will have a 'record.count' attribute indicating the number of records that were written to the FlowFile.
mime.typeThe MIME Type indicated by the Record Writer
<Attributes from Record Writer>Any Attribute that the configured Record Writer returns will be added to the FlowFile.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship.

System Resource Considerations:

None specified.