PutElasticsearchRecord

Description:

A record-aware Elasticsearch put processor that uses the official Elastic REST client libraries.

Additional Details...

Tags:

json, elasticsearch, elasticsearch5, elasticsearch6, put, index, record

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

NameDefault ValueAllowable ValuesDescription
Index Operationindex
  • create
  • delete
  • index
  • update
  • upsert
The type of the operation used to index (create, delete, index, update, upsert)
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
IndexThe name of the index to use.
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
TypeThe type of this document (used by Elasticsearch for indexing and searching).
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Client ServiceController Service API:
ElasticSearchClientService
Implementation: ElasticSearchClientServiceImpl
An Elasticsearch client service to use for running queries.
Record ReaderController Service API:
RecordReaderFactory
Implementations: ScriptedReader
XMLReader
GrokReader
AvroReader
WindowsEventLogReader
JsonPathReader
ReaderLookup
CSVReader
Syslog5424Reader
SyslogReader
JsonTreeReader
ParquetReader
The record reader to use for reading incoming records from flowfiles.
Batch Size100The number of records to send over in a single batch.
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
ID Record PathA record path expression to retrieve the ID field for use with Elasticsearch. If left blank the ID will be automatically generated by Elasticsearch.
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Index Operation Record PathA record path expression to retrieve the Index Operation field for use with Elasticsearch. If left blank the Index Operation will be determined using the main Index Operation property.
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Index Record PathA record path expression to retrieve the index field for use with Elasticsearch. If left blank the index will be determined using the main index property.
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Type Record PathA record path expression to retrieve the type field for use with Elasticsearch. If left blank the type will be determined using the main type property.
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Log Error Responsesfalse
  • true
  • false
If this is enabled, errors will be logged to the NiFi logs at the error log level. Otherwise, they will only be logged if debug logging is enabled on NiFi as a whole. The purpose of this option is to give the user the ability to debug failed operations without having to turn on debug logging.
Error Record WriterController Service API:
RecordSetWriterFactory
Implementations: FreeFormTextRecordSetWriter
AvroRecordSetWriter
ParquetRecordSetWriter
ScriptedRecordSetWriter
JsonRecordSetWriter
RecordSetWriterLookup
CSVRecordSetWriter
XMLRecordSetWriter
If this configuration property is set, the response from Elasticsearch will be examined for failed records and the failed records will be written to a record set with this record writer service and sent to the "errors" relationship.

Relationships:

NameDescription
retryAll flowfiles that fail due to server/cluster availability go to this relationship.
successAll flowfiles that succeed in being transferred into Elasticsearch go here.
failureAll flowfiles that fail for reasons unrelated to server availability go to this relationship.
errorsIf an output record write is set, any record that failed to process the way it was configured will be sent to this relationship as part of a failed record record set.

Reads Attributes:

None specified.

Writes Attributes:

None specified.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship.

System Resource Considerations:

None specified.