CalculateRecordStats 2.0.0

Bundle
org.apache.nifi | nifi-standard-nar
Description
Counts the number of Records in a record set, optionally counting the number of elements per category, where the categories are defined by user-defined properties.
Tags
metrics, record, stats
Input Requirement
REQUIRED
Supports Sensitive Dynamic Properties
false
  • Additional Details for CalculateRecordStats 2.0.0

    CalculateRecordStats

    This processor takes in a record set and counts both the overall count and counts that are defined as dynamic properties that map a property name to a record path. Record path counts are provided at two levels:

    • The overall count of all records that successfully evaluated a record path.
    • A breakdown of counts of unique values that matched the record path operation.

    Consider the following record structure:

    {
  "sport": "Soccer",
  "name": "John Smith"
}

    A valid mapping here would be sport => /sport.

    For a record set with JSON like that, five entries and 3 instances of soccer and two instances of football, it would set the following attributes:

    • record_count: 5
    • sport: 5
    • sport.Soccer: 3
    • sport.Football: 2
Properties
Dynamic Properties
Relationships
Name Description
failure If a FlowFile cannot be processed for any reason, it is routed to this Relationship.
success All FlowFiles that are successfully processed, are routed to this Relationship.
Writes Attributes
Name Description
record.count A count of the records in the record set in the FlowFile.
recordStats.<User Defined Property Name>.count A count of the records that contain a value for the user defined property.
recordStats.<User Defined Property Name>.<value>.count Each value discovered for the user defined property will have its own count attribute. Total number of top N value counts to be added is defined by the limit configuration.