ListenTCPRecord

Description:

Listens for incoming TCP connections and reads data from each connection using a configured record reader, and writes the records to a flow file using a configured record writer. The type of record reader selected will determine how clients are expected to send data. For example, when using a Grok reader to read logs, a client can keep an open connection and continuously stream data, but when using an JSON reader, the client cannot send an array of JSON documents and then send another array on the same connection, as the reader would be in a bad state at that point. Records will be read from the connection in blocking mode, and will timeout according to the Read Timeout specified in the processor. If the read times out, or if any other error is encountered when reading, the connection will be closed, and any records read up to that point will be handled according to the configured Read Error Strategy (Discard or Transfer). In cases where clients are keeping a connection open, the concurrent tasks for the processor should be adjusted to match the Max Number of TCP Connections allowed, so that there is a task processing each connection.

Tags:

listen, tcp, record, tls, ssl

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

NameDefault ValueAllowable ValuesDescription
Local Network InterfaceThe name of a local network interface to be used to restrict listening to a specific LAN.
Supports Expression Language: true (will be evaluated using variable registry only)
PortThe port to listen on for communication.
Supports Expression Language: true (will be evaluated using variable registry only)
Max Size of Socket Buffer1 MBThe maximum size of the socket buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.
Max Number of TCP Connections2The maximum number of concurrent TCP connections to accept. In cases where clients are keeping a connection open, the concurrent tasks for the processor should be adjusted to match the Max Number of TCP Connections allowed, so that there is a task processing each connection.
Read Timeout10 secondsThe amount of time to wait before timing out when reading from a connection.
Record ReaderController Service API:
RecordReaderFactory
Implementations: Syslog5424Reader
AvroReader
JsonPathReader
ScriptedReader
XMLReader
GrokReader
JsonTreeReader
SyslogReader
CSVReader
The Record Reader to use for incoming FlowFiles
Record WriterController Service API:
RecordSetWriterFactory
Implementations: ScriptedRecordSetWriter
JsonRecordSetWriter
CSVRecordSetWriter
AvroRecordSetWriter
XMLRecordSetWriter
FreeFormTextRecordSetWriter
The Record Writer to use in order to serialize the data before writing to a FlowFile
Read Error StrategyTransfer
  • Transfer Transfers any records already received and closes the connection.
  • Discard Discards any records already received and closes the connection.
Indicates how to deal with an error while reading the next record from a connection, when previous records have already been read from the connection.
Record Batch Size1000The maximum number of records to write to a single FlowFile.
SSL Context ServiceController Service API:
RestrictedSSLContextService
Implementation: StandardRestrictedSSLContextService
The Controller Service to use in order to obtain an SSL Context. If this property is set, messages will be received over a secure connection.
Client AuthREQUIRED
  • WANT
  • REQUIRED
  • NONE
The client authentication policy to use for the SSL Context. Only used if an SSL Context Service is provided.

Relationships:

NameDescription
successMessages received successfully will be sent out this relationship.

Reads Attributes:

None specified.

Writes Attributes:

NameDescription
tcp.senderThe host that sent the data.
tcp.portThe port that the processor accepted the connection on.
record.countThe number of records written to the flow file.
mime.typeThe mime-type of the writer used to write the records to the flow file.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component does not allow an incoming relationship.

System Resource Considerations:

None specified.