Execute Spark Code over a Livy-managed HTTP session to a live Spark context. Supports cached RDD sharing.


spark, livy, http, execute


In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

NameDefault ValueAllowable ValuesDescription
Livy Controller ServiceController Service API:
Implementation: LivySessionController
The controller service to use for Livy-managed session(s).
CodeThe code to execute in the session. This property can be empty, a constant value, or built from attributes using Expression Language. If this property is specified, it will be used regardless of the content of incoming flowfiles. If this property is empty, the content of the incoming flow file is expected to contain valid code to be issued by the processor to the session. Note that Expression Language is not evaluated for flow file contents.
Supports Expression Language: true
Character SetUTF-8The character set encoding for the incoming flow file.
Supports Expression Language: true
Status Check Interval1 secThe amount of time to wait between checking the status of an operation.
Supports Expression Language: true


successFlowFiles that are successfully processed are sent to this relationship
waitFlowFiles that are waiting on an available Spark session will be sent to this relationship
failureFlowFiles are routed to this relationship when they cannot be parsed

Reads Attributes:

None specified.

Writes Attributes:

None specified.

State management:

This component does not store state.


This component is not restricted.

Input requirement:

This component requires an incoming relationship.