JoltTransformRecord

Description:

Applies a JOLT specification to each record in the FlowFile payload. A new FlowFile is created with transformed content and is routed to the 'success' relationship. If the transform fails, the original FlowFile is routed to the 'failure' relationship.

Tags:

record, jolt, transform, shiftr, chainr, defaultr, removr, cardinality, sort

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

Display NameAPI NameDefault ValueAllowable ValuesDescription
Jolt TransformJolt TransformChain
  • Shift Shift input JSON/data to create the output JSON.
  • Chain Execute list of Jolt transformations.
  • Default Apply default values to the output JSON.
  • Remove Remove values from input data to create the output JSON.
  • Cardinality Change the cardinality of input elements to create the output JSON.
  • Sort Sort input json key values alphabetically. Any specification set is ignored.
  • Custom Custom Transformation. Requires Custom Transformation Class Name
  • Modify - Default Writes when key is missing or value is null
  • Modify - Overwrite Always overwrite value
  • Modify - Define Writes when key is missing
Specifies the Jolt Transformation that should be used with the provided specification.
Jolt SpecificationJolt SpecificationJolt Specification for transformation of JSON data. The value for this property may be the text of a Jolt specification or the path to a file containing a Jolt specification. 'Jolt Specification' must be set, or the value is ignored if the Jolt Sort Transformation is selected.

This property requires exactly one resource to be provided. That resource may be any of the following types: file, text.

Supports Expression Language: true (will be evaluated using flow file attributes and Environment variables)
Custom Transformation Class NameCustom Transformation Class NameFully Qualified Class Name for Custom Transformation
Supports Expression Language: true (will be evaluated using flow file attributes and Environment variables)

This Property is only considered if the [Jolt Transform] Property has a value of "Custom".
Custom Module DirectoryCustom Module DirectoryComma-separated list of paths to files and/or directories which contain modules containing custom transformations (that are not included on NiFi's classpath).

This property expects a comma-separated list of resources. Each of the resources may be of any of the following types: directory, file.

Supports Expression Language: true (will be evaluated using Environment variables only)

This Property is only considered if the [Jolt Transform] Property has a value of "Custom".
Transform Cache SizeTransform Cache Size1Compiling a Jolt Transform can be fairly expensive. Ideally, this will be done only once. However, if the Expression Language is used in the transform, we may need a new Transform for each FlowFile. This value controls how many of those Transforms we cache in memory in order to avoid having to compile the Transform each time.
Record Readerjolt-record-record-readerController Service API:
RecordReaderFactory
Implementations: CSVReader
JsonPathReader
AvroReader
CEFReader
Syslog5424Reader
JsonTreeReader
WindowsEventLogReader
XMLReader
SyslogReader
JASN1Reader
ReaderLookup
ParquetReader
GrokReader
ScriptedReader
YamlTreeReader
ExcelReader		Specifies the Controller Service to use for parsing incoming data and determining the data's schema.
Record Writerjolt-record-record-writerController Service API:
RecordSetWriterFactory
Implementations: XMLRecordSetWriter
FreeFormTextRecordSetWriter
AvroRecordSetWriter
ScriptedRecordSetWriter
JsonRecordSetWriter
ParquetRecordSetWriter
RecordSetWriterLookup
CSVRecordSetWriter		Specifies the Controller Service to use for writing out the records

Relationships:

NameDescription
originalThe original FlowFile that was transformed. If the FlowFile fails processing, nothing will be sent to this relationship
successThe FlowFile with transformed content will be routed to this relationship
failureIf a FlowFile fails processing for any reason (for example, the FlowFile records cannot be parsed), it will be routed to this relationship

Reads Attributes:

None specified.

Writes Attributes:

NameDescription
record.countThe number of records in an outgoing FlowFile
mime.typeThe MIME Type that the configured Record Writer indicates is appropriate

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship.

System Resource Considerations:

None specified.