FetchGoogleDrive 2.0.0

Bundle
org.apache.nifi | nifi-gcp-nar
Description
Fetches files from a Google Drive Folder. Designed to be used in tandem with ListGoogleDrive. Please see Additional Details to set up access to Google Drive.
Tags
drive, fetch, google, storage
Input Requirement
REQUIRED
Supports Sensitive Dynamic Properties
false
  • Additional Details for FetchGoogleDrive 2.0.0

    FetchGoogleDrive

    Accessing Google Drive from NiFi

    This processor uses Google Cloud credentials for authentication to access Google Drive. The following steps are required to prepare the Google Cloud and Google Drive accounts for the processors:

    1. Enable Google Drive API in Google Cloud
    2. Grant access to Google Drive folder
      • In Google Cloud Console navigate to IAM & Admin -> Service Accounts.
      • Take a note of the email of the service account you are going to use.
      • Navigate to the folder to be listed in Google Drive.
      • Right-click on the Folder -> Share.
      • Enter the service account email.
    3. Find File ID
      Usually FetchGoogleDrive is used with ListGoogleDrive and ‘drive.id’ is set.
      In case ‘drive.id’ is not available, you can find the Drive ID of the file in the following way:
      • Right-click on the file and select “Get Link”.
      • In the pop-up window click on “Copy Link”.
      • You can obtain the file ID from the URL copied to clipboard. For example, if the URL were https://drive.google.com/file/d/16ALV9KIU_KKeNG557zyctqy2Fmzyqtq/view?usp=share_link,
        the File ID would be 16ALV9KIU_KKeNG557zyctqy2Fmzyqtq
    4. Set File ID in ‘File ID’ property
Properties
Relationships
Name Description
failure A FlowFile will be routed here for each File for which fetch was attempted but failed.
success A FlowFile will be routed here for each successfully fetched File.
Reads Attributes
Name Description
drive.id The id of the file
Writes Attributes
Name Description
drive.id The id of the file
filename The name of the file
mime.type The MIME type of the file
drive.size The size of the file
drive.timestamp The last modified time or created time (whichever is greater) of the file. The reason for this is that the original modified date of a file is preserved when uploaded to Google Drive. 'Created time' takes the time when the upload occurs. However uploaded files can still be modified later.
error.code The error code returned by Google Drive
error.message The error message returned by Google Drive
Use Cases Involving Other Components
  • Retrieve all files in a Google Drive folder
    Description
    Retrieve all files in a Google Drive folder
    Keywords
    google, drive, google cloud, state, retrieve, fetch, all, stream
    Processor Configurations
    org.apache.nifi.processors.gcp.drive.ListGoogleDrive
    The "Folder ID" property should be set to the ID of the Google Drive folder that files reside in.     See processor documentation / additional details for more information on how to determine a Google Drive folder's ID.
    If the flow being built is to be reused elsewhere, it's a good idea to parameterize     this property by setting it to something like `#{GOOGLE_DRIVE_FOLDER_ID}`.

The "GCP Credentials Provider Service" property should specify an instance of the GCPCredentialsService in order to provide credentials for accessing the folder.

The 'success' Relationship of this Processor is then connected to FetchGoogleDrive.
    org.apache.nifi.processors.gcp.drive.FetchGoogleDrive
    "File ID" = "${drive.id}"

The "GCP Credentials Provider Service" property should specify an instance of the GCPCredentialsService in order to provide credentials for accessing the bucket.
See Also