CompressContent

Description:

Compresses or decompresses the contents of FlowFiles using a user-specified compression algorithm and updates the mime.type attribute as appropriate. A common idiom is to precede CompressContent with IdentifyMimeType and configure Mode='decompress' AND Compression Format='use mime.type attribute'. When used in this manner, the MIME type is automatically detected and the data is decompressed, if necessary. If decompression is unnecessary, the data is passed through to the 'success' relationship. This processor operates in a very memory efficient way so very large objects well beyond the heap size are generally fine to process.

Tags:

content, compress, decompress, gzip, bzip2, lzma, xz-lzma2, snappy, snappy-hadoop, snappy framed, lz4-framed, deflate, zstd, brotli

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values.

Display NameAPI NameDefault ValueAllowable ValuesDescription
ModeModecompress
  • compress
  • decompress
Indicates whether the processor should compress content or decompress content. Must be either 'compress' or 'decompress'
Compression FormatCompression Formatuse mime.type attribute
  • use mime.type attribute
  • gzip
  • deflate
  • bzip2
  • xz-lzma2
  • lzma
  • snappy
  • snappy-hadoop
  • snappy framed
  • lz4-framed
  • zstd
  • brotli
The compression format to use. Valid values are: GZIP, Deflate, ZSTD, BZIP2, XZ-LZMA2, LZMA, Brotli, Snappy, Snappy Hadoop, Snappy Framed, and LZ4-Framed
Compression LevelCompression Level1
  • 0
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
The compression level to use; this is valid only when using gzip, deflate or xz-lzma2 compression. A lower value results in faster processing but less compression; a value of 0 indicates no (that is, simple archiving) for gzip or minimal for xz-lzma2 compression. Higher levels can mean much larger memory usage such as the case with levels 7-9 for xz-lzma/2 so be careful relative to heap size.

This Property is only considered if all of the following conditions are met:
  • The [Compression Format] Property is set to one of the following values: [zstd], [use mime.type attribute], [deflate], [brotli], [gzip], [xz-lzma2]
  • The [Mode] Property has a value of "compress".
Update FilenameUpdate Filenamefalse
  • true
  • false
If true, will remove the filename extension when decompressing data (only if the extension indicates the appropriate compression format) and add the appropriate extension when compressing data

Relationships:

NameDescription
successFlowFiles will be transferred to the success relationship after successfully being compressed or decompressed
failureFlowFiles will be transferred to the failure relationship if they fail to compress/decompress

Reads Attributes:

NameDescription
mime.typeIf the Compression Format is set to use mime.type attribute, this attribute is used to determine the compression type. Otherwise, this attribute is ignored.

Writes Attributes:

NameDescription
mime.typeIf the Mode property is set to compress, the appropriate MIME Type is set. If the Mode property is set to decompress and the file is successfully decompressed, this attribute is removed, as the MIME Type is no longer known.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship.

System Resource Considerations:

ResourceDescription
CPUAn instance of this component can cause high usage of this system resource. Multiple instances or high concurrency settings may result a degradation of performance.
MEMORYAn instance of this component can cause high usage of this system resource. Multiple instances or high concurrency settings may result a degradation of performance.