PackageFlowFile

Description:

This processor will package FlowFile attributes and content into an output FlowFile that can be exported from NiFi and imported back into NiFi, preserving the original attributes and content.

Tags:

flowfile, flowfile-stream, flowfile-stream-v3, package, attributes

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values.

Display NameAPI NameDefault ValueAllowable ValuesDescription
Maximum Batch Sizemax-batch-size1Maximum number of FlowFiles to package into one output FlowFile using a best effort, non guaranteed approach. Multiple input queues can produce unexpected batching behavior.

Relationships:

NameDescription
originalThe FlowFiles that were used to create the package are sent to this relationship
successThe packaged FlowFile is sent to this relationship

Reads Attributes:

None specified.

Writes Attributes:

NameDescription
mime.typeThe mime.type will be changed to application/flowfile-v3

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship.

Example Use Cases Involving Other Components:

Use Case:

Send FlowFile content and attributes from one NiFi instance to another NiFi instance.

Notes:

A Remote Process Group is preferred to send FlowFiles between two NiFi instances, but an alternative is to use PackageFlowFile then InvokeHTTP sending to ListenHTTP.

Keywords:

flowfile, attributes, content, ffv3, flowfile-stream-v3, transfer

Components involved:

Component Type: org.apache.nifi.processors.standard.PackageFlowFile

Configuration:

"Maximum Batch Size" > 1 can help improve performance by batching many flowfiles together into 1 larger file that is transmitted by InvokeHTTP.

Connect the success relationship of PackageFlowFile to the input of InvokeHTTP.



Component Type: org.apache.nifi.processors.standard.InvokeHTTP

Configuration:

"HTTP Method" = "POST" to send data to ListenHTTP.

"HTTP URL" should include the hostname, port, and path to the ListenHTTP.

"Request Content-Type" = "${mime.type}" because PackageFlowFile output files have attribute mime.type=application/flowfile-v3.



Component Type: org.apache.nifi.processors.standard.ListenHTTP

Configuration:

"Listening Port" = a unique port number.

ListenHTTP automatically unpacks files that have attribute mime.type=application/flowfile-v3.

If PackageFlowFile batches 99 FlowFiles into 1 file that InvokeHTTP sends, then the original 99 FlowFiles will be output by ListenHTTP.





Use Case:

Export FlowFile content and attributes from NiFi to external storage and reimport.

Keywords:

flowfile, attributes, content, ffv3, flowfile-stream-v3, offline, storage

Components involved:

Component Type: org.apache.nifi.processors.standard.PackageFlowFile

Configuration:

"Maximum Batch Size" > 1 can improve storage efficiency by batching many FlowFiles together into 1 larger file that is stored.

Connect the success relationship to the input of any NiFi egress processor for offline storage.



Component Type: org.apache.nifi.processors.standard.UnpackContent

Configuration:

"Packaging Format" = "application/flowfile-v3".

Connect the output of a NiFi ingress processor that reads files stored offline to the input of UnpackContent.

If PackageFlowFile batches 99 FlowFiles into 1 file that is read from storage, then the original 99 FlowFiles will be output by UnpackContent.





System Resource Considerations:

None specified.

See Also:

UnpackContent, MergeContent