FetchFTP

Description:

Fetches the content of a file from a remote FTP server and overwrites the contents of an incoming FlowFile with the content of the remote file.

Tags:

ftp, get, retrieve, files, fetch, remote, ingest, source, input

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

Display NameAPI NameDefault ValueAllowable ValuesDescription
HostnameHostnameThe fully-qualified hostname or IP address of the host to fetch the data from
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
PortPort21The port to connect to on the remote host to fetch the data from
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
UsernameUsernameUsername
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
PasswordPasswordPassword for the user account
Sensitive Property: true
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Remote FileRemote FileThe fully qualified filename on the remote system
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Completion StrategyCompletion StrategyNone
  • None Leave the file as-is
  • Move File Move the file to the directory specified by the <Move Destination Directory> property
  • Delete File Deletes the original file from the remote system
Specifies what to do with the original file on the server once it has been pulled into NiFi. If the Completion Strategy fails, a warning will be logged but the data will still be transferred.
Move Destination DirectoryMove Destination DirectoryThe directory on the remote server to move the original file to once it has been ingested into NiFi. This property is ignored unless the Completion Strategy is set to 'Move File'. The specified directory must already exist on the remote system if 'Create Directory' is disabled, or the rename will fail.
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Create DirectoryCreate Directoryfalse
  • true
  • false
Used when 'Completion Strategy' is 'Move File'. Specifies whether or not the remote directory should be created if it does not exist.
Connection TimeoutConnection Timeout30 secAmount of time to wait before timing out while creating a connection
Data TimeoutData Timeout30 secWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems
Use CompressionUse Compressionfalse
  • true
  • false
Indicates whether or not ZLIB compression should be used when transferring files
Connection ModeConnection ModePassive
  • Active
  • Passive
The FTP Connection Mode
Transfer ModeTransfer ModeBinary
  • Binary
  • ASCII
The FTP Transfer Mode
Proxy Configuration Serviceproxy-configuration-serviceController Service API:
ProxyConfigurationService
Implementation: StandardProxyConfigurationService
Specifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthN
Proxy TypeProxy TypeDIRECT
  • DIRECT
  • HTTP
  • SOCKS
Proxy type used for file transfers
Proxy HostProxy HostThe fully qualified hostname or IP address of the proxy server
Supports Expression Language: true (will be evaluated using variable registry only)
Proxy PortProxy PortThe port of the proxy server
Supports Expression Language: true (will be evaluated using variable registry only)
Http Proxy UsernameHttp Proxy UsernameHttp Proxy Username
Supports Expression Language: true (will be evaluated using variable registry only)
Http Proxy PasswordHttp Proxy PasswordHttp Proxy Password
Sensitive Property: true
Supports Expression Language: true (will be evaluated using variable registry only)
Internal Buffer SizeInternal Buffer Size16KBSet the internal buffer size for buffered data streams
Log level when file not foundfetchfiletransfer-notfound-loglevelERROR
  • TRACE
  • DEBUG
  • INFO
  • WARN
  • ERROR
  • FATAL
  • NONE
Log level to use in case the file does not exist when the processor is triggered
Use UTF-8 Encodingftp-use-utf8false
  • true
  • false
Tells the client to use UTF-8 encoding when processing files and filenames. If set to true, the server must also support UTF-8 encoding.

Relationships:

NameDescription
successAll FlowFiles that are received are routed to success
comms.failureAny FlowFile that could not be fetched from the remote server due to a communications failure will be transferred to this Relationship.
not.foundAny FlowFile for which we receive a 'Not Found' message from the remote server will be transferred to this Relationship.
permission.deniedAny FlowFile that could not be fetched from the remote server due to insufficient permissions will be transferred to this Relationship.

Reads Attributes:

None specified.

Writes Attributes:

NameDescription
ftp.remote.hostThe hostname or IP address from which the file was pulled
ftp.remote.portThe port that was used to communicate with the remote FTP server
ftp.remote.filenameThe name of the remote file that was pulled
filenameThe filename is updated to point to the filename fo the remote file
pathIf the Remote File contains a directory name, that directory name will be added to the FlowFile using the 'path' attribute
fetch.failure.reasonThe name of the failure relationship applied when routing to any failure relationship

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship.

System Resource Considerations:

None specified.

See Also:

GetSFTP, PutSFTP, GetFTP, PutFTP