AvroReader

Description:

Parses Avro data and returns each Avro record as an separate Record object. The Avro data may contain the schema itself, or the schema can be externalized and accessed by one of the methods offered by the 'Schema Access Strategy' property.

Tags:

avro, parse, record, row, reader, delimited, comma, separated, values

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

NameDefault ValueAllowable ValuesDescription
Schema Access Strategyembedded-avro-schema
  • Use 'Schema Name' Property The name of the Schema to use is specified by the 'Schema Name' Property. The value of this property is used to lookup the Schema in the configured Schema Registry service.
  • Use 'Schema Text' Property The text of the Schema itself is specified by the 'Schema Text' Property. The value of this property must be a valid Avro Schema. If Expression Language is used, the value of the 'Schema Text' property must be valid after substituting the expressions.
  • HWX Schema Reference Attributes The FlowFile contains 3 Attributes that will be used to lookup a Schema from the configured Schema Registry: 'schema.identifier', 'schema.version', and 'schema.protocol.version'
  • HWX Content-Encoded Schema Reference The content of the FlowFile contains a reference to a schema in the Schema Registry service. The reference is encoded as a single byte indicating the 'protocol version', followed by 8 bytes indicating the schema identifier, and finally 4 bytes indicating the schema version, as per the Hortonworks Schema Registry serializers and deserializers, found at https://github.com/hortonworks/registry
  • Confluent Content-Encoded Schema Reference The content of the FlowFile contains a reference to a schema in the Schema Registry service. The reference is encoded as a single 'Magic Byte' followed by 4 bytes representing the identifier of the schema, as outlined at http://docs.confluent.io/current/schema-registry/docs/serializer-formatter.html. This is based on version 3.2.x of the Confluent Schema Registry.
  • Use Embedded Avro Schema The FlowFile has the Avro Schema embedded within the content, and this schema will be used.
Specifies how to obtain the schema that is to be used for interpreting the data.
Schema RegistryController Service API:
SchemaRegistry
Implementations: HortonworksSchemaRegistry
AvroSchemaRegistry
ConfluentSchemaRegistry		Specifies the Controller Service to use for the Schema Registry
Schema Name${schema.name}Specifies the name of the schema to lookup in the Schema Registry property
Supports Expression Language: true
Schema Text${avro.schema}The text of an Avro-formatted Schema
Supports Expression Language: true

State management:

This component does not store state.

Restricted:

This component is not restricted.