ValidateCsv

ValidateCsv 2.0.0

Bundle: org.apache.nifi | nifi-standard-nar
Description: Validates the contents of FlowFiles against a user-specified CSV schema. Take a look at the additional documentation of this processor for some schema examples.
Tags: csv, schema, validation
Input Requirement: REQUIRED
Supports Sensitive Dynamic Properties: false

Properties

Delimiter character
Character used as 'delimiter' in the incoming data. Example: ,

Display Name

Delimiter character

Description

Character used as 'delimiter' in the incoming data. Example: ,

API Name

validate-csv-delimiter

Default Value

,

Expression Language Scope

Environment variables and FlowFile Attributes

Sensitive

false

Required

true
End of line symbols
Symbols used as 'end of line' in the incoming data. Example: \n

Display Name

End of line symbols

Description

Symbols used as 'end of line' in the incoming data. Example: \n

API Name

validate-csv-eol

Default Value

\n

Expression Language Scope

Environment variables and FlowFile Attributes

Sensitive

false

Required

true
Header
True if the incoming flow file contains a header to ignore, false otherwise.
Display Name

Header

Description

True if the incoming flow file contains a header to ignore, false otherwise.

API Name

validate-csv-header

Default Value

true

Allowable Values
- true
- false
Expression Language Scope

Not Supported

Sensitive

false

Required

true
Quote character
Character used as 'quote' in the incoming data. Example: "

Display Name

Quote character

Description

Character used as 'quote' in the incoming data. Example: "

API Name

validate-csv-quote

Default Value

"

Expression Language Scope

Environment variables and FlowFile Attributes

Sensitive

false

Required

true
Schema
The schema to be used for validation. Is expected a comma-delimited string representing the cell processors to apply. The following cell processors are allowed in the schema definition: [ParseBigDecimal, ParseBool, ParseChar, ParseDate, ParseDouble, ParseInt, ParseLong, Optional, DMinMax, Equals, ForbidSubStr, LMinMax, NotNull, Null, RequireHashCode, RequireSubStr, Strlen, StrMinMax, StrNotNullOrEmpty, StrRegEx, Unique, UniqueHashCode, IsIncludedIn]. Note: cell processors cannot be nested except with Optional.

Display Name

Schema

Description

The schema to be used for validation. Is expected a comma-delimited string representing the cell processors to apply. The following cell processors are allowed in the schema definition: [ParseBigDecimal, ParseBool, ParseChar, ParseDate, ParseDouble, ParseInt, ParseLong, Optional, DMinMax, Equals, ForbidSubStr, LMinMax, NotNull, Null, RequireHashCode, RequireSubStr, Strlen, StrMinMax, StrNotNullOrEmpty, StrRegEx, Unique, UniqueHashCode, IsIncludedIn]. Note: cell processors cannot be nested except with Optional.

API Name

validate-csv-schema

Expression Language Scope

Environment variables and FlowFile Attributes

Sensitive

false

Required

true
Validation strategy
Strategy to apply when routing input files to output relationships.
Display Name

Validation strategy

Description

Strategy to apply when routing input files to output relationships.

API Name

validate-csv-strategy

Default Value

FlowFile validation

Allowable Values
- Line by line validation
- FlowFile validation
Expression Language Scope

Not Supported

Sensitive

false

Required

true
Include all violations
If true, the validation.error.message attribute would include the list of all the violations for the first invalid line. Note that setting this property to true would slightly decrease the performances as all columns would be validated. If false, a line is invalid as soon as a column is found violating the specified constraint and only this violation for the first invalid line will be included in the validation.error.message attribute.
Display Name

Include all violations

Description

If true, the validation.error.message attribute would include the list of all the violations for the first invalid line. Note that setting this property to true would slightly decrease the performances as all columns would be validated. If false, a line is invalid as soon as a column is found violating the specified constraint and only this violation for the first invalid line will be included in the validation.error.message attribute.

API Name

validate-csv-violations

Default Value

false

Allowable Values
- true
- false
Expression Language Scope

Not Supported

Sensitive

false

Required

true

Relationships

Name	Description
invalid	FlowFiles that are not valid according to the specified schema are routed to this relationship
valid	FlowFiles that are successfully validated against the schema are routed to this relationship

Writes Attributes

Name	Description
count.valid.lines	If line by line validation, number of valid lines extracted from the source data
count.invalid.lines	If line by line validation, number of invalid lines extracted from the source data
count.total.lines	If line by line validation, total number of lines in the source data
validation.error.message	For flow files routed to invalid, message of the first validation error