ForkRecord

Description:

This processor allows the user to fork a record into multiple records. The user must specify at least one Record Path, as a dynamic property, pointing to a field of type ARRAY containing RECORD objects. The processor accepts two modes: 'split' and 'extract'. In both modes, there is one record generated per element contained in the designated array. In the 'split' mode, each generated record will preserve the same schema as given in the input but the array will contain only one element. In the 'extract' mode, the element of the array must be of record type and will be the generated record. Additionally, in the 'extract' mode, it is possible to specify if each generated record should contain all the fields of the parent records from the root level to the extracted record. This assumes that the fields to add in the record are defined in the schema of the Record Writer controller service. See examples in the additional details documentation of this processor.

Additional Details...

Tags:

fork, record, content, array, stream, event

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values.

Display NameAPI NameDefault ValueAllowable ValuesDescription
Record Readerrecord-readerController Service API:
RecordReaderFactory
Implementations: GrokReader
JsonTreeReader
WindowsEventLogReader
ReaderLookup
ParquetReader
CSVReader
Syslog5424Reader
ExcelReader
CEFReader
XMLReader
ScriptedReader
SyslogReader
JsonPathReader
AvroReader
YamlTreeReader
Specifies the Controller Service to use for reading incoming data
Record Writerrecord-writerController Service API:
RecordSetWriterFactory
Implementations: FreeFormTextRecordSetWriter
CSVRecordSetWriter
ParquetRecordSetWriter
RecordSetWriterLookup
ScriptedRecordSetWriter
XMLRecordSetWriter
JsonRecordSetWriter
AvroRecordSetWriter
Specifies the Controller Service to use for writing out the records
Modefork-modeSplit
  • Extract Generated records will be the elements of the array
  • Split Generated records will preserve the input schema and will contain a one-element array
Specifies the forking mode of the processor
Include Parent Fieldsinclude-parent-fieldsfalse
  • true
  • false
This parameter is only valid with the 'extract' mode. If set to true, all the fields from the root level to the given array will be added as fields of each element of the array to fork.

Dynamic Properties:

Supports Sensitive Dynamic Properties: No

Dynamic Properties allow the user to specify both the name and value of a property.

NameValueDescription
Record Path propertyThe Record Path valueA Record Path value, pointing to a field of type ARRAY containing RECORD objects
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)

Relationships:

NameDescription
failureIn case a FlowFile generates an error during the fork operation, it will be routed to this relationship
originalThe original FlowFiles will be routed to this relationship
forkThe FlowFiles containing the forked records will be routed to this relationship

Reads Attributes:

None specified.

Writes Attributes:

NameDescription
record.countThe generated FlowFile will have a 'record.count' attribute indicating the number of records that were written to the FlowFile.
mime.typeThe MIME Type indicated by the Record Writer
<Attributes from Record Writer>Any Attribute that the configured Record Writer returns will be added to the FlowFile.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship.

System Resource Considerations:

None specified.