AvroRecordSetWriter

AvroRecordSetWriter 2.0.0

Bundle: org.apache.nifi | nifi-record-serialization-services-nar
Description: Writes the contents of a RecordSet in Binary Avro format.
Tags: avro, record, recordset, result, row, serializer, set, writer
Input Requirement
Supports Sensitive Dynamic Properties: false

Properties

Cache Size
Specifies how many Schemas should be cached

Display Name

Cache Size

Description

Specifies how many Schemas should be cached

API Name

cache-size

Default Value

1000

Expression Language Scope

Not Supported

Sensitive

false

Required

true
Compression Format
Compression type to use when writing Avro files. Default is None.
Display Name

Compression Format

Description

Compression type to use when writing Avro files. Default is None.

API Name

compression-format

Default Value

NONE

Allowable Values
- BZIP2
- DEFLATE
- NONE
- SNAPPY
- LZO
Expression Language Scope

Not Supported

Sensitive

false

Required

true
Encoder Pool Size
Avro Writers require the use of an Encoder. Creation of Encoders is expensive, but once created, they can be reused. This property controls the maximum number of Encoders that can be pooled and reused. Setting this value too small can result in degraded performance, but setting it higher can result in more heap being used. This property is ignored if the Avro Writer is configured with a Schema Write Strategy of 'Embed Avro Schema'.

Display Name

Encoder Pool Size

Description

Avro Writers require the use of an Encoder. Creation of Encoders is expensive, but once created, they can be reused. This property controls the maximum number of Encoders that can be pooled and reused. Setting this value too small can result in degraded performance, but setting it higher can result in more heap being used. This property is ignored if the Avro Writer is configured with a Schema Write Strategy of 'Embed Avro Schema'.

API Name

encoder-pool-size

Default Value

32

Expression Language Scope

Environment variables defined at JVM level and system properties

Sensitive

false

Required

true
Schema Reference Writer
Service implementation responsible for writing FlowFile attributes or content header with Schema reference information
Display Name

Schema Reference Writer

Description

Service implementation responsible for writing FlowFile attributes or content header with Schema reference information

API Name

Schema Reference Writer

Service Interface

org.apache.nifi.schemaregistry.services.SchemaReferenceWriter

Service Implementations

org.apache.nifi.confluent.schemaregistry.ConfluentEncodedSchemaReferenceWriter

Expression Language Scope

Not Supported

Sensitive

false

Required

true

Dependencies
- Schema Write Strategy is set to any of [schema-reference-writer]
Schema Write Strategy
Specifies how the schema for a Record should be added to the data.
Display Name

Schema Write Strategy

Description

Specifies how the schema for a Record should be added to the data.

API Name

Schema Write Strategy

Default Value

avro-embedded

Allowable Values
- Embed Avro Schema
- Do Not Write Schema
- Set 'schema.name' Attribute
- Set 'avro.schema' Attribute
- Schema Reference Writer
Expression Language Scope

Not Supported

Sensitive

false

Required

true
Schema Access Strategy
Specifies how to obtain the schema that is to be used for interpreting the data.
Display Name

Schema Access Strategy

Description

Specifies how to obtain the schema that is to be used for interpreting the data.

API Name

schema-access-strategy

Default Value

inherit-record-schema

Allowable Values
- Inherit Record Schema
- Use 'Schema Name' Property
- Use 'Schema Text' Property
Expression Language Scope

Not Supported

Sensitive

false

Required

true
Schema Branch
Specifies the name of the branch to use when looking up the schema in the Schema Registry property. If the chosen Schema Registry does not support branching, this value will be ignored.
Display Name

Schema Branch

Description

Specifies the name of the branch to use when looking up the schema in the Schema Registry property. If the chosen Schema Registry does not support branching, this value will be ignored.

API Name

schema-branch

Expression Language Scope

Environment variables and FlowFile Attributes

Sensitive

false

Required

false

Dependencies
- Schema Access Strategy is set to any of [schema-name]
Schema Cache
Specifies a Schema Cache to add the Record Schema to so that Record Readers can quickly lookup the schema.

Display Name

Schema Cache

Description

Specifies a Schema Cache to add the Record Schema to so that Record Readers can quickly lookup the schema.

API Name

schema-cache

Service Interface

org.apache.nifi.serialization.RecordSchemaCacheService

Service Implementations

org.apache.nifi.schema.inference.VolatileSchemaCache

Expression Language Scope

Not Supported

Sensitive

false

Required

false
Schema Name
Specifies the name of the schema to lookup in the Schema Registry property
Display Name

Schema Name

Description

Specifies the name of the schema to lookup in the Schema Registry property

API Name

schema-name

Default Value

${schema.name}

Expression Language Scope

Environment variables and FlowFile Attributes

Sensitive

false

Required

false

Dependencies
- Schema Access Strategy is set to any of [schema-name]
Schema Reference Reader
Service implementation responsible for reading FlowFile attributes or content to determine the Schema Reference Identifier
Display Name

Schema Reference Reader

Description

Service implementation responsible for reading FlowFile attributes or content to determine the Schema Reference Identifier

API Name

schema-reference-reader

Service Interface

org.apache.nifi.schemaregistry.services.SchemaReferenceReader

Service Implementations

org.apache.nifi.confluent.schemaregistry.ConfluentEncodedSchemaReferenceReader

Expression Language Scope

Not Supported

Sensitive

false

Required

true

Dependencies
- Schema Access Strategy is set to any of [schema-reference-reader]
Schema Registry
Specifies the Controller Service to use for the Schema Registry
Display Name

Schema Registry

Description

Specifies the Controller Service to use for the Schema Registry

API Name

schema-registry

Service Interface

org.apache.nifi.schemaregistry.services.SchemaRegistry

Service Implementations

org.apache.nifi.aws.schemaregistry.AmazonGlueSchemaRegistry

org.apache.nifi.apicurio.schemaregistry.ApicurioSchemaRegistry

org.apache.nifi.schemaregistry.services.AvroSchemaRegistry

org.apache.nifi.confluent.schemaregistry.ConfluentSchemaRegistry

org.apache.nifi.db.schemaregistry.DatabaseTableSchemaRegistry

Expression Language Scope

Not Supported

Sensitive

false

Required

false

Dependencies
- Schema Access Strategy is set to any of [schema-reference-reader, schema-name]
Schema Text
The text of an Avro-formatted Schema
Display Name

Schema Text

Description

The text of an Avro-formatted Schema

API Name

schema-text

Default Value

${avro.schema}

Expression Language Scope

Environment variables and FlowFile Attributes

Sensitive

false

Required

false

Dependencies
- Schema Access Strategy is set to any of [schema-text-property]
Schema Version
Specifies the version of the schema to lookup in the Schema Registry. If not specified then the latest version of the schema will be retrieved.
Display Name

Schema Version

Description

Specifies the version of the schema to lookup in the Schema Registry. If not specified then the latest version of the schema will be retrieved.

API Name

schema-version

Expression Language Scope

Environment variables and FlowFile Attributes

Sensitive

false

Required

false

Dependencies
- Schema Access Strategy is set to any of [schema-name]