StartGcpVisionAnnotateFilesOperation 2.0.0

Bundle
org.apache.nifi | nifi-gcp-nar
Description
Trigger a Vision operation on file input. It should be followed by GetGcpVisionAnnotateFilesOperationStatus processor in order to monitor operation status.
Tags
Cloud, Google, Machine Learning, Vision
Input Requirement
Supports Sensitive Dynamic Properties
false
  • Additional Details for StartGcpVisionAnnotateFilesOperation 2.0.0

    Google Vision

    Google Cloud Vision - Start Annotate Files Operation

    Prerequisites

    • Make sure Vision API is enabled and the account you are using has the right to use it
    • Make sure the input file(s) are available in a GCS bucket

    Usage

    StartGcpVisionAnnotateFilesOperation is designed to trigger file annotation operations. This processor should be used in pair with the GetGcpVisionAnnotateFilesOperationStatus Processor. Outgoing FlowFiles contain the raw response to the request returned by the Vision server. The response is in JSON format and contains the result and additional metadata as written in the Google Vision API Reference documents.

    Payload

    The JSON Payload is a request in JSON format as documented in the Google Vision REST API reference document. Payload can be fed to the processor via the JSON Payload property or as a FlowFile content. The property has higher precedence over FlowFile content. Please make sure to delete the default value of the property if you want to use FlowFile content payload. A JSON payload template example:

    {
  "requests": [
    {
      "inputConfig": {
        "gcsSource": {
          "uri": "gs://${gcs.bucket}/${filename}"
        },
        "mimeType": "application/pdf"
      },
      "features": [
        {
          "type": "${vision-feature-type}",
          "maxResults": 4
        }
      ],
      "outputConfig": {
        "gcsDestination": {
          "uri": "gs://${output-bucket}/${filename}/"
        },
        "batchSize": 2
      }
    }
  ]
}

    Features types

    • TEXT_DETECTION: Optical character recognition (OCR) for an image; text recognition and conversion to machine-coded text. Identifies and extracts UTF-8 text in an image.
    • DOCUMENT_TEXT_DETECTION: Optical character recognition (OCR) for a file (PDF/TIFF) or dense text image; dense text recognition and conversion to machine-coded text.

    You can find more details at Google Vision Feature List

    Example: How to set up a simple Annotate Image Flow

    Prerequisites

    • Create an input and output bucket
    • Input files should be available in a GCS bucket
    • This bucket must not contain anything else but the input files
    • Set the bucket property of ListGCSBucket processor to your input bucket name
    • Keep the default value of JSON PAYLOAD property in StartGcpVisionAnnotateFilesOperation
    • Set the Output Bucket property to your output bucket name in StartGcpVisionAnnotateFilesOperation
    • Setup GCP Credentials Provider Service for all GCP related processor

    Execution steps:

    • ListGCSBucket processor will return a list of files in the bucket at the first run.
    • ListGCSBucket will return only new items at subsequent runs.
    • StartGcpVisionAnnotateFilesOperation processor will trigger GCP Vision file annotation jobs based on the JSON payload.
    • StartGcpVisionAnnotateFilesOperation processor will populate the operationKey flow file attribute.
    • GetGcpVisionAnnotateFilesOperationStatus processor will periodically query status of the job.
Properties
Relationships
Name Description
success FlowFiles are routed to success relationship
failure FlowFiles are routed to failure relationship
Writes Attributes
Name Description
operationKey A unique identifier of the operation returned by the Vision server.
See Also