ListenBeats

Description:

Listens for messages sent by libbeat compatible clients (e.g. filebeats, metricbeats, etc) using Libbeat's 'output.logstash', writing its JSON formatted payload to the content of a FlowFile.This processor replaces the now deprecated ListenLumberjack

Tags:

listen, beats, tcp, logs

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

NameDefault ValueAllowable ValuesDescription
Local Network InterfaceThe name of a local network interface to be used to restrict listening to a specific LAN.
Supports Expression Language: true (will be evaluated using variable registry only)
PortThe port to listen on for communication.
Supports Expression Language: true (will be evaluated using variable registry only)
Receive Buffer Size65507 BThe size of each buffer used to receive messages. Adjust this value appropriately based on the expected size of the incoming messages.
Max Size of Message Queue10000The maximum size of the internal queue used to buffer messages being transferred from the underlying channel to the processor. Setting this value higher allows more messages to be buffered in memory during surges of incoming messages, but increases the total memory used by the processor.
Max Size of Socket Buffer1 MBThe maximum size of the socket buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.
Character SetUTF-8Specifies the character set of the received data.
Max Batch Size1The maximum number of messages to add to a single FlowFile. If multiple messages are available, they will be concatenated along with the <Message Delimiter> up to this configured maximum number of messages
Batching Message Delimiter\nSpecifies the delimiter to place between messages when multiple messages are bundled together (see <Max Batch Size> property).
Max Number of TCP Connections2The maximum number of concurrent TCP connections to accept.
SSL Context ServiceController Service API:
RestrictedSSLContextService
Implementation: StandardRestrictedSSLContextService
The Controller Service to use in order to obtain an SSL Context. If this property is set, messages will be received over a secure connection.

Relationships:

NameDescription
successMessages received successfully will be sent out this relationship.

Reads Attributes:

None specified.

Writes Attributes:

NameDescription
beats.senderThe sending host of the messages.
beats.portThe sending port the messages were received over.
beats.sequencenumberThe sequence number of the message. Only included if <Batch Size> is 1.
mime.typeThe mime.type of the content which is application/json

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component does not allow an incoming relationship.

System Resource Considerations:

None specified.

See Also:

ParseSyslog