PutAzureDataExplorer

Description:

Acts as an Azure Data Explorer sink which sends FlowFiles to the provided endpoint. Data can be sent through queued ingestion or streaming ingestion to the Azure Data Explorer cluster.

Tags:

Azure, Kusto, ADX, Explorer, Data

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

Display NameAPI NameDefault ValueAllowable ValuesDescription
Kusto Ingest ServiceKusto Ingest ServiceController Service API:
KustoIngestService
Implementation: StandardKustoIngestService
Azure Data Explorer Kusto Ingest Service
Database NameDatabase NameAzure Data Explorer Database Name for ingesting data
Supports Expression Language: true (will be evaluated using flow file attributes and Environment variables)
Table NameTable NameAzure Data Explorer Table Name for ingesting data
Supports Expression Language: true (will be evaluated using flow file attributes and Environment variables)
Ingest Mapping NameIngest Mapping NameThe name of the mapping responsible for storing the data in the appropriate columns.
Supports Expression Language: true (will be evaluated using flow file attributes and Environment variables)
Data FormatData Format
  • avro An Avro format with support for logical types and for the snappy compression codec
  • apacheavro An Avro format with support for logical types and for the snappy compression codec.
  • csv A text file with comma-separated values (,). For more information, see RFC 4180: Common Format and MIME Type for Comma-Separated Values (CSV) Files.
  • json A text file containing JSON objects separated by \n or \r\n. For more information, see JSON Lines (JSONL).
  • multijson A text file containing a JSON array of property containers (each representing a record) or any number of property containers separated by spaces, \n or \r\n. Each property container may be spread across multiple lines. This format is preferable to JSON unless the data is not property containers.
  • orc An ORC file.
  • parquet A parquet file.
  • psv A text file with values separated by vertical bars (|).
  • scsv A text file with values separated by semicolons (;).
  • sohsv A text file with SOH-separated values. (SOH is the ASCII code point 1. This format is used by Hive in HDInsight).
  • tsv A text file with tab delimited values (\t).
  • tsve A text file with tab-delimited values (\t). A backslash (\) is used as escape character.
  • txt A text file with lines separated by \n. Empty lines are skipped
The format of the data that is sent to Azure Data Explorer. Supported formats include: avro, csv, json
Supports Expression Language: true (will be evaluated using flow file attributes and Environment variables)
Partially Succeeded Routing StrategyPartially Succeeded Routing Strategyfailure
  • success
  • failure
Defines where to route FlowFiles that resulted in a partially succeeded status.
Streaming EnabledStreaming Enabledfalse
  • true
  • false
Whether to stream data to Azure Data Explorer.
Ingestion Ignore First RecordIngestion Ignore First Recordfalse
  • true
  • false
Defines whether ignore first record while ingestion.
Poll for Ingest StatusPoll for Ingest Statusfalse
  • true
  • false
Determines whether to poll on ingestion status after an ingestion to Azure Data Explorer is completed
Ingest Status Polling TimeoutIngest Status Polling Timeout5 mDefines the total amount time to poll for ingestion status

This Property is only considered if the [Poll for Ingest Status] Property has a value of "true".
Ingest Status Polling IntervalIngest Status Polling Interval5 sDefines the value of interval of time to poll for ingestion status

This Property is only considered if the [Poll for Ingest Status] Property has a value of "true".

Relationships:

NameDescription
failureIngest processing failed
successIngest processing succeeded

Reads Attributes:

None specified.

Writes Attributes:

None specified.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship.

System Resource Considerations:

None specified.