Modifies the contents of a FlowFile that contains Record-oriented data (i.e. data that can be read via a RecordReader and written by a RecordWriter) by removing selected fields. This Processor requires that at least one user-defined Property be added. The name of the property is ignored by the processor, but could be a meaningful identifier for the user. The value of the property should indicate a RecordPath that determines the field to be removed. The processor executes the removal in the order in which these properties are added to the processor. Set the "Record Writer" to "Inherit Record Schema" in order to use the updated Record Schema modified when removing Fields.
update, record, generic, schema, json, csv, avro, freeform, text, remove, delete
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values.
| Display Name | API Name | Default Value | Allowable Values | Description |
|---|---|---|---|---|
| Record Reader | Record Reader | Controller Service API: RecordReaderFactory Implementations: CSVReader JsonPathReader AvroReader CEFReader Syslog5424Reader JsonTreeReader WindowsEventLogReader XMLReader SyslogReader JASN1Reader ReaderLookup ParquetReader GrokReader ScriptedReader YamlTreeReader ExcelReader | Specifies the Controller Service to use for reading incoming data | |
| Record Writer | Record Writer | Controller Service API: RecordSetWriterFactory Implementations: XMLRecordSetWriter FreeFormTextRecordSetWriter AvroRecordSetWriter ScriptedRecordSetWriter JsonRecordSetWriter ParquetRecordSetWriter RecordSetWriterLookup CSVRecordSetWriter | Specifies the Controller Service to use for writing out the records |
Supports Sensitive Dynamic Properties: No
Dynamic Properties allow the user to specify both the name and value of a property.
| Name | Value | Description |
|---|---|---|
| A description of the field to remove | A RecordPath to the field to be removed. | Any field that matches the RecordPath set as the value will be removed. Supports Expression Language: true (will be evaluated using flow file attributes and Environment variables) |
| Name | Description |
|---|---|
| failure | If a FlowFile cannot be transformed from the configured input format to the configured output format, the unchanged FlowFile will be routed to this relationship |
| success | FlowFiles that are successfully transformed will be routed to this relationship |
| Name | Description |
|---|---|
| record.error.message | This attribute provides on failure the error message encountered by the Reader or Writer. |
Remove one or more fields from a Record, where the names of the fields to remove are known.
Configure the Record Reader according to the incoming data format.
Configure the Record Writer according to the desired output format.
For each field that you want to remove, add a single new property to the Processor.
The name of the property can be anything but it's recommended to use a brief description of the field.
The value of the property is a RecordPath that matches the field to remove.
For example, to remove the name and email fields, add two Properties:
name = /name
email = /email