FetchAzureBlobStorage

Deprecation notice:

Processor depends on legacy Microsoft Azure SDK

Please consider using one the following alternatives: FetchAzureBlobStorage_v12

Description:

Retrieves contents of an Azure Storage Blob, writing the contents to the content of the FlowFile

Tags:

azure, microsoft, cloud, storage, blob

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

Display NameAPI NameDefault ValueAllowable ValuesDescription
Container Namecontainer-nameName of the Azure storage container. In case of PutAzureBlobStorage processor, container can be created if it does not exist.
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Storage Credentialsstorage-credentials-serviceController Service API:
AzureStorageCredentialsService
Implementations: AzureStorageCredentialsControllerServiceLookup
AzureStorageCredentialsControllerService
AzureStorageEmulatorCredentialsControllerService
The Controller Service used to obtain Azure Storage Credentials. Instead of the processor level properties, the credentials can be configured here through a common/shared controller service, which is the preferred way. The 'Lookup' version of the service can also be used to select the credentials dynamically at runtime based on a FlowFile attribute (if the processor has FlowFile input).
Storage Account Namestorage-account-nameThe storage account name. There are certain risks in allowing the account name to be stored as a flowfile attribute. While it does provide for a more flexible flow by allowing the account name to be fetched dynamically from a flowfile attribute, care must be taken to restrict access to the event provenance data (e.g., by strictly controlling the policies governing provenance for this processor). In addition, the provenance repositories may be put on encrypted disk partitions. Instead of defining the Storage Account Name, Storage Account Key and SAS Token properties directly on the processor, the preferred way is to configure them through a controller service specified in the Storage Credentials property. The controller service can provide a common/shared configuration for multiple/all Azure processors. Furthermore, the credentials can also be looked up dynamically with the 'Lookup' version of the service.
Sensitive Property: true
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Storage Account Keystorage-account-keyThe storage account key. This is an admin-like password providing access to every container in this account. It is recommended one uses Shared Access Signature (SAS) token instead for fine-grained control with policies. There are certain risks in allowing the account key to be stored as a flowfile attribute. While it does provide for a more flexible flow by allowing the account key to be fetched dynamically from a flowfile attribute, care must be taken to restrict access to the event provenance data (e.g., by strictly controlling the policies governing provenance for this processor). In addition, the provenance repositories may be put on encrypted disk partitions.
Sensitive Property: true
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
SAS Tokenstorage-sas-tokenShared Access Signature token, including the leading '?'. Specify either SAS token (recommended) or Account Key. There are certain risks in allowing the SAS token to be stored as a flowfile attribute. While it does provide for a more flexible flow by allowing the SAS token to be fetched dynamically from a flowfile attribute, care must be taken to restrict access to the event provenance data (e.g., by strictly controlling the policies governing provenance for this processor). In addition, the provenance repositories may be put on encrypted disk partitions.
Sensitive Property: true
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Common Storage Account Endpoint Suffixstorage-endpoint-suffixStorage accounts in public Azure always use a common FQDN suffix. Override this endpoint suffix with a different suffix in certain circumstances (like Azure Stack or non-public Azure regions). The preferred way is to configure them through a controller service specified in the Storage Credentials property. The controller service can provide a common/shared configuration for multiple/all Azure processors. Furthermore, the credentials can also be looked up dynamically with the 'Lookup' version of the service.
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Blobblob${azure.blobname}The filename of the blob
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Proxy Configuration Serviceproxy-configuration-serviceController Service API:
ProxyConfigurationService
Implementation: StandardProxyConfigurationService
Specifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP, SOCKS In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.
Range Startrange-startThe byte position at which to start reading from the blob. An empty value or a value of zero will start reading at the beginning of the blob.
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Range Lengthrange-lengthThe number of bytes to download from the blob, starting from the Range Start. An empty value or a value that extends beyond the end of the blob will read to the end of the blob.
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Client-Side Encryption Key Typecse-key-typeNONE
  • NONE The blobs sent to Azure are not encrypted.
  • SYMMETRIC The blobs sent to Azure are encrypted using a symmetric algorithm.
Specifies the key type to use for client-side encryption.
Client-Side Encryption Key IDcse-key-idSpecifies the ID of the key to use for client-side encryption.
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)

This Property is only considered if the [Client-Side Encryption Key Type] Property has a value of "SYMMETRIC".
Symmetric Keycse-symmetric-key-hexWhen using symmetric client-side encryption, this is the raw key, encoded in hexadecimal
Sensitive Property: true

This Property is only considered if the [Client-Side Encryption Key Type] Property has a value of "SYMMETRIC".

Relationships:

NameDescription
successAll successfully processed FlowFiles are routed to this relationship
failureUnsuccessful operations will be transferred to the failure relationship.

Reads Attributes:

None specified.

Writes Attributes:

NameDescription
azure.lengthThe length of the blob fetched

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship.

System Resource Considerations:

None specified.

See Also:

ListAzureBlobStorage, PutAzureBlobStorage, DeleteAzureBlobStorage