ListDatabaseTables 2.0.0

Bundle
org.apache.nifi | nifi-standard-nar
Description
Generates a set of flow files, each containing attributes corresponding to metadata about a table from a database connection. Once metadata about a table has been fetched, it will not be fetched again until the Refresh Interval (if set) has elapsed, or until state has been manually cleared.
Tags
database, jdbc, list, sql, table
Input Requirement
FORBIDDEN
Supports Sensitive Dynamic Properties
false
Properties
State Management
Scopes Description
CLUSTER After performing a listing of tables, the timestamp of the query is stored. This allows the Processor to not re-list tables the next time that the Processor is run. Specifying the refresh interval in the processor properties will indicate that when the processor detects the interval has elapsed, the state will be reset and tables will be re-listed as a result. This processor is meant to be run on the primary node only.
Relationships
Name Description
success All FlowFiles that are received are routed to success
Writes Attributes
Name Description
db.table.name Contains the name of a database table from the connection
db.table.catalog Contains the name of the catalog to which the table belongs (may be null)
db.table.schema Contains the name of the schema to which the table belongs (may be null)
db.table.fullname Contains the fully-qualifed table name (possibly including catalog, schema, etc.)
db.table.type Contains the type of the database table from the connection. Typical types are "TABLE", "VIEW", "SYSTEM TABLE", "GLOBAL TEMPORARY", "LOCAL TEMPORARY", "ALIAS", "SYNONYM"
db.table.remarks Contains the name of a database table from the connection
db.table.count Contains the number of rows in the table
Use Cases Involving Other Components
  • Perform a full load of a database, retrieving all rows from all tables, or a specific set of tables.
    Description
    Perform a full load of a database, retrieving all rows from all tables, or a specific set of tables.
    Keywords
    full load, rdbms, jdbc, database
    Processor Configurations
    org.apache.nifi.processors.standard.ListDatabaseTables
    Configure the "Database Connection Pooling Service" property to specify a Connection Pool that is applicable for interacting with your database.
Leave the RecordWriter property unset.

Set the "Catalog" property to the name of the database Catalog; leave it empty to include all catalogs.
Set the "Schema Pattern" property to a Java Regular Expression that matches all database Schemas that should be included; leave it empty to include all Schemas.
Set the "Table Name Pattern" property to a Java Regular Expression that matches the names of all tables that should be included; leave it empty to include all Tables.

Connect the "success" relationship to GenerateTableFetch.
    org.apache.nifi.processors.standard.GenerateTableFetch
    Configure the "Database Connection Pooling Service" property to specify the same Connection Pool that was used in ListDatabaseTables.
Set the "Database Type" property to match the appropriate value for your RDBMS vendor.
Set "Table Name" to `${db.table.fullname}`
Leave the RecordWriter property unset.

Connect the "success" relationship to ExecuteSQLRecord.
    org.apache.nifi.processors.standard.ExecuteSQLRecord
    Configure the "Database Connection Pooling Service" property to specify the same Connection Pool that was used in ListDatabaseTables.
Configure the "Record Writer" property to specify a Record Writer that is appropriate for the desired output data type.
Leave the "SQL select query" unset.

Connect the "success" relationship to the next Processor in the flow.