JoltTransformRecord

Description:

Applies a JOLT specification to each record in the FlowFile payload. A new FlowFile is created with transformed content and is routed to the 'success' relationship. If the transform fails, the original FlowFile is routed to the 'failure' relationship.

Tags:

record, jolt, transform, shiftr, chainr, defaultr, removr, cardinality, sort

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

Display NameAPI NameDefault ValueAllowable ValuesDescription
Record Readerjolt-record-record-readerController Service API:
RecordReaderFactory
Implementations: JsonTreeReader
GrokReader
AvroReader
SyslogReader
ExcelReader
XMLReader
Syslog5424Reader
CSVReader
CEFReader
JsonPathReader
ReaderLookup
ScriptedReader
ParquetReader
WindowsEventLogReader		Specifies the Controller Service to use for parsing incoming data and determining the data's schema.
Record Writerjolt-record-record-writerController Service API:
RecordSetWriterFactory
Implementations: CSVRecordSetWriter
ParquetRecordSetWriter
FreeFormTextRecordSetWriter
XMLRecordSetWriter
ScriptedRecordSetWriter
RecordSetWriterLookup
JsonRecordSetWriter
AvroRecordSetWriter		Specifies the Controller Service to use for writing out the records
Jolt Transformation DSLjolt-record-transformChain
  • Cardinality Change the cardinality of input elements to create the output.
  • Chain Execute list of Jolt transformations.
  • Default Apply default values to the output.
  • Modify - Default Writes when field name is missing or value is null
  • Modify - Define Writes when key is missing
  • Modify - Overwrite Always overwrite value
  • Remove Remove values from input data to create the output.
  • Shift Shift input data to create the output.
  • Sort Sort input field name values alphabetically. Any specification set is ignored.
  • Custom Custom Transformation. Requires Custom Transformation Class Name
Specifies the Jolt Transformation that should be used with the provided specification.
Custom Transformation Class Namejolt-record-custom-classFully Qualified Class Name for Custom Transformation
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)

This Property is only considered if the [Jolt Specification] Property
Custom Module Directoryjolt-record-custom-modulesComma-separated list of paths to files and/or directories which contain modules containing custom transformations (that are not included on NiFi's classpath).

This property expects a comma-separated list of resources. Each of the resources may be of any of the following types: file, directory.

Supports Expression Language: true (will be evaluated using variable registry only)

This Property is only considered if the [Jolt Specification] Property
Jolt Specificationjolt-record-specJolt Specification for transform of record data. The value for this property may be the text of a JOLT specification or the path to a file containing a JOLT specification. This value is ignored if the Jolt Sort Transformation is selected.

This property requires exactly one resource to be provided. That resource may be any of the following types: file, text.

Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Transform Cache Sizejolt-record-transform-cache-size1Compiling a Jolt Transform can be fairly expensive. Ideally, this will be done only once. However, if the Expression Language is used in the transform, we may need a new Transform for each FlowFile. This value controls how many of those Transforms we cache in memory in order to avoid having to compile the Transform each time.

Relationships:

NameDescription
successThe FlowFile with transformed content will be routed to this relationship
failureIf a FlowFile fails processing for any reason (for example, the FlowFile records cannot be parsed), it will be routed to this relationship
originalThe original FlowFile that was transformed. If the FlowFile fails processing, nothing will be sent to this relationship

Reads Attributes:

None specified.

Writes Attributes:

NameDescription
record.countThe number of records in an outgoing FlowFile
mime.typeThe MIME Type that the configured Record Writer indicates is appropriate

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship.

System Resource Considerations:

None specified.