PartitionRecord 2.0.0

Bundle
org.apache.nifi | nifi-standard-nar
Description
Splits, or partitions, record-oriented data based on the configured fields in the data. One or more properties must be added. The name of the property is the name of an attribute to add. The value of the property is a RecordPath to evaluate against each Record. Two records will go to the same outbound FlowFile only if they have the same value for each of the given RecordPaths. Because we know that all records in a given output FlowFile have the same value for the fields that are specified by the RecordPath, an attribute is added for each field. See Additional Details on the Usage page for more information and examples.
Tags
bin, group, organize, partition, record, recordpath, rpath, segment, split
Input Requirement
REQUIRED
Supports Sensitive Dynamic Properties
false
Properties
Dynamic Properties
Relationships
Name Description
success FlowFiles that are successfully partitioned will be routed to this relationship
failure If a FlowFile cannot be partitioned from the configured input format to the configured output format, the unchanged FlowFile will be routed to this relationship
original Once all records in an incoming FlowFile have been partitioned, the original FlowFile is routed to this relationship.
Writes Attributes
Name Description
record.count The number of records in an outgoing FlowFile
mime.type The MIME Type that the configured Record Writer indicates is appropriate
fragment.identifier All partitioned FlowFiles produced from the same parent FlowFile will have the same randomly generated UUID added for this attribute
fragment.index A one-up number that indicates the ordering of the partitioned FlowFiles that were created from a single parent FlowFile
fragment.count The number of partitioned FlowFiles generated from the parent FlowFile
segment.original.filename The filename of the parent FlowFile
<dynamic property name> For each dynamic property that is added, an attribute may be added to the FlowFile. See the description for Dynamic Properties for more information.
Use Cases
  • Separate records into separate FlowFiles so that all of the records in a FlowFile have the same value for a given field or set of fields.
    Description
    Separate records into separate FlowFiles so that all of the records in a FlowFile have the same value for a given field or set of fields.
    Keywords
    separate, split, partition, break apart, colocate, segregate, record, field, recordpath
    Configuration
    Choose a RecordReader that is appropriate based on the format of the incoming data.
Choose a RecordWriter that writes the data in the desired output format.

Add a single additional property. The name of the property should describe the type of data that is being used to partition the data. The property's value should be a RecordPath that specifies which output FlowFile the Record belongs to.

For example, if we want to separate records based on their `transactionType` field, we could add a new property named `transactionType`. The value of the property might be `/transaction/type`. An input FlowFile will then be separated into as few FlowFiles as possible such that each output FlowFile has the same value for the `transactionType` field.
  • Separate records based on whether or not they adhere to a specific criteria
    Description
    Separate records based on whether or not they adhere to a specific criteria
    Keywords
    separate, split, partition, break apart, segregate, record, field, recordpath, criteria
    Configuration
    Choose a RecordReader that is appropriate based on the format of the incoming data.
Choose a RecordWriter that writes the data in the desired output format.

Add a single additional property. The name of the property should describe the criteria. The property's value should be a RecordPath that returns `true` if the Record meets the criteria or `false` otherwise.

For example, if we want to separate records based on whether or not they have a transaction total of more than $1,000 we could add a new property named `largeTransaction` with a value of `/transaction/total > 1000`. This will create two FlowFiles. In the first, all records will have a total over `1000`. In the second, all records will have a transaction less than or equal to 1000. Each FlowFile will have an attribute named `largeTransaction` with a value of `true` or `false`.
See Also