The processor is capable to trigger different type of events in the HiveMetaStore and generate notifications. The metastore action to be executed is determined from the incoming path and event type attributes. The supported event type values are 'put' in case of data insertion or 'delete' in case of data removal. The notifications should be enabled in the metastore configuration to generate them e.g.: the 'hive.metastore.transactional.event.listeners' should have a proper listener configured, for instance 'org.apache.hive.hcatalog.listener.DbNotificationListener'.
hive, metastore, notification, insert, delete, partition, event
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
| Display Name | API Name | Default Value | Allowable Values | Description |
|---|---|---|---|---|
| Hive Metastore URI | hive-metastore-uri | The URI location(s) for the Hive metastore. This is a comma-separated list of Hive metastore URIs; note that this is not the location of the Hive Server. The default port for the Hive metastore is 9043. If this field is not set, then the 'hive.metastore.uris' property from any provided configuration resources will be used, and if none are provided, then the default value from a default hive-site.xml will be used (usually thrift://localhost:9083). Supports Expression Language: true (will be evaluated using flow file attributes and Environment variables) | ||
| Hive Configuration Resources | hive-config-resources | A file or comma separated list of files which contains the Hive configuration (hive-site.xml, e.g.). Without this, Hadoop will search the classpath for a 'hive-site.xml' file or will revert to a default configuration. Note that to enable authentication with Kerberos e.g., the appropriate properties must be set in the configuration files. Please see the Hive documentation for more details. This property expects a comma-separated list of file resources. Supports Expression Language: true (will be evaluated using Environment variables only) | ||
| Event Type | event-type | ${event.type} | The type of the event. The acceptable values are 'put' in case of data insert or 'delete' in case of data removal. Supports Expression Language: true (will be evaluated using flow file attributes and Environment variables) | |
| Path | path | ${path} | The path of the file or folder located in the file system. Supports Expression Language: true (will be evaluated using flow file attributes and Environment variables) | |
| Catalog Name | catalog-name | The name of the catalog. Supports Expression Language: true (will be evaluated using flow file attributes and Environment variables) | ||
| Database Name | database-name | The name of the database. Supports Expression Language: true (will be evaluated using flow file attributes and Environment variables) | ||
| Table Name | table-name | The name of the table. Supports Expression Language: true (will be evaluated using flow file attributes and Environment variables) | ||
| Kerberos User Service | kerberos-user-service | Controller Service API: KerberosUserService Implementations: KerberosPasswordUserService KerberosKeytabUserService KerberosTicketCacheUserService | Specifies the Kerberos User Controller Service that should be used for authenticating with Kerberos. |
| Name | Description |
|---|---|
| success | A FlowFile is routed to this relationship after the data ingestion was successful. |
| failure | A FlowFile is routed to this relationship if the data ingestion failed and retrying the operation will also fail, such as an invalid data or schema. |
| Name | Description |
|---|---|
| metastore.notification.event | The event type of the triggered notification. |