Apache NiFi Releases

This document describes the steps required to build and release an official version of the project.

Objective

Produce an official Apache NiFi release from a current source branch.

Background Information

Following the release guide requires understanding several important terms and procedures.

Reference Documentation

Glossary of Terms

Variable References

The release guide references names and values that vary for each version. These values have been written as shell variable references.

For example, when referencing a variable named ${NIFI_VERSION} in a tag such as nifi-${NIFI_VERSION}, the variable reference should be replaced with the current value for the Release Candidate build. When preparing a release of Apache NiFi 0.7.0, the tag would be rendered as nifi-0.7.0.

The example values reflect the release details for Apache NiFi 0.7.0 Release Candidate 2.

Variable Example Value Description
${BRANCH} main Source development branch on which the release is based
${NIFI_VERSION} 0.7.0 Version number targeted for release
${NEXT_VERSION} 0.8.0-SNAPSHOT Future version number for development after the release
${JIRA_TICKET} NIFI-2112 Jira issue number for tracking release tasks
${RC} 2 Release Candidate number starting with 1 for the first build
${RC_TAG_COMMIT_ID} Hexadecimal Git commit hash of the Release Candidate tag
${STAGING_REPO_ID} orgapachenifi-1000 Nexus Repository identifier for staged Maven artifacts
${RM_USERID} username Apache account identifier of the Release Manager
${RELEASE_TAG} rel/nifi-0.7.0 Git repository tag associated with the source to be released
${VOTE_THREAD_URL} URL for the Apache Mailing List archive of the release vote thread

Release Overview

The release process includes steps performed by the Release Manager as well as the project community.

Process Summary

  1. Community member suggests a release timeline and initiates a discussion email thread
  2. PMC member volunteers to act as the Release Manager for the version
  3. RM validates the source branch and stages the code, Maven artifacts, and distributable files for an RC build
  4. RM sends a vote email thread for the RC build
  5. PMC members and community contributors vote to approve or reject the RC build
    1. RM cancels a vote thread for a rejected RC build
    2. RM identifies issues for resolution and prepares a new RC build
  6. RM publishes build artifacts for an approved RC build

Release Candidate Preparation

The Release Manager is responsible for creating, signing, and staging artifacts for a Release Candidate build.

Configure Environment

<profile>
    <id>signing</id>
    <properties>
        <mavenExecutorId>forked-path</mavenExecutorId>
        <gpg.keyname>${RM_USERID}@apache.org</gpg.keyname>
        <gpg.passphrase>REPLACE-WITH-ENCRYPTED-GPG-PASSPHRASE</gpg.passphrase>
    </properties>
</profile>

<servers>
    <server>
        <id>repository.apache.org</id>
        <username>${RM_USERID}</username>
        <password>REPLACE-WITH-ENCRYPTED-REPOSITORY-PASSWORD</password>
    </server>
</servers>
git remote -v

upstream https://gitbox.apache.org/repos/asf/nifi.git (fetch)
upstream https://gitbox.apache.org/repos/asf/nifi.git (push)
origin   https://github.com/${RM_USERID}/nifi.git (fetch)
origin   https://github.com/${RM_USERID}/nifi.git (push)

Update Tracking

Build Artifacts

git checkout -b ${JIRA_TICKET}-RC${RC} ${BRANCH}
./mvnw -T 2C -P contrib-check clean install

Stage Artifacts

export GPG_TTY=$(tty)
./mvnw release:prepare -Psigning \
-DscmCommentPrefix="${JIRA_TICKET}-RC${RC} " \
-Dtag="nifi-${NIFI_VERSION}-RC${RC}" \
-DreleaseVersion="${NIFI_VERSION}" \
-DdevelopmentVersion="${NEXT_VERSION}" \
-Darguments="-DskipTests"
./mvnw release:perform -Psigning \
-DscmCommentPrefix="${JIRA_TICKET}-RC${RC} " \
-Darguments="-DskipTests"

Validate Artifacts

STAGING_DIR=~/staging
mkdir ${STAGING_DIR}
cd ${STAGING_DIR}
SOURCE_RELEASE_ZIP="https://repository.apache.org/service/local/repositories/${STAGING_REPO_ID}/content/org/apache/nifi/nifi/${NIFI_VERSION}/nifi-${NIFI_VERSION}-source-release.zip"
wget ${SOURCE_RELEASE_ZIP} 
wget ${SOURCE_RELEASE_ZIP}.asc
SOURCE_RELEASE_ZIP_FILE="nifi-${NIFI_VERSION}-source-release.zip"
gpg --verify ${SOURCE_RELEASE_ZIP_FILE}.asc ${SOURCE_RELEASE_ZIP_FILE}
unzip ${SOURCE_RELEASE_ZIP_FILE}
cd nifi-${NIFI_VERSION}
./mvnw package -pl \
:minifi-assembly,\
:minifi-tookit-assembly,\
:nifi-assembly,\
:nifi-kafka-connector-assembly,\
:nifi-toolkit-assembly,\
:nifi-registry-assembly,\
:nifi-registry-toolkit-assembly,\
:nifi-stateless-assembly
ARTIFACTS_DIR=~/staging/artifacts
mkdir ${ARTIFACTS_DIR}
cp minifi/minifi-assembly/target/minifi-${NIFI_VERSION}-bin.zip ${ARTIFACTS_DIR}
cp minifi/minifi-toolkit/minifi-toolkit-assembly/target/minifi-toolkit-${NIFI_VERSION}-bin.zip ${ARTIFACTS_DIR}
cp nifi-assembly/target/nifi-${NIFI_VERSION}-bin.zip ${ARTIFACTS_DIR}
cp nifi-external/nifi-kafka-connector-assembly/target/nifi-kafka-connector-assembly-${NIFI_VERSION}.zip ${ARTIFACTS_DIR}
cp nifi-registry/nifi-registry-assembly/target/nifi-registry-${NIFI_VERSION}-bin.zip ${ARTIFACTS_DIR}
cp nifi-registry/nifi-registry-toolkit/nifi-registry-toolkit-assembly/target/nifi-registry-toolkit-${NIFI_VERSION}-bin.zip ${ARTIFACTS_DIR}
cp nifi-stateless/nifi-stateless-assembly/target/nifi-stateless-${NIFI_VERSION}-bin.zip ${ARTIFACTS_DIR}
cp nifi-toolkit/nifi-toolkit-assembly/target/nifi-toolkit-${NIFI_VERSION}-bin.zip ${ARTIFACTS_DIR}
cp ${STAGING_DIR}/nifi-${NIFI_VERSION}-source-release.zip ${ARTIFACTS_DIR}
cp ${STAGING_DIR}/nifi-${NIFI_VERSION}-source-release.zip.asc ${ARTIFACTS_DIR}
cd ${ARTIFACTS_DIR}
gpg -a -b --digest-algo=SHA512 minifi-${NIFI_VERSION}-bin.zip
gpg -a -b --digest-algo=SHA512 minifi-toolkit-${NIFI_VERSION}-bin.zip
gpg -a -b --digest-algo=SHA512 nifi-${NIFI_VERSION}-bin.zip
gpg -a -b --digest-algo=SHA512 nifi-kafka-connector-assembly-${NIFI_VERSION}.zip
gpg -a -b --digest-algo=SHA512 nifi-registry-${NIFI_VERSION}-bin.zip
gpg -a -b --digest-algo=SHA512 nifi-registry-toolkit-${NIFI_VERSION}-bin.zip
gpg -a -b --digest-algo=SHA512 nifi-stateless-${NIFI_VERSION}-bin.zip
gpg -a -b --digest-algo=SHA512 nifi-toolkit-${NIFI_VERSION}-bin.zip
sh -c 'sha256sum $1 | cut -d " " -f 1 > $1.sha256' -- minifi-${NIFI_VERSION}-bin.zip
sh -c 'sha256sum $1 | cut -d " " -f 1 > $1.sha256' -- minifi-toolkit-${NIFI_VERSION}-bin.zip
sh -c 'sha256sum $1 | cut -d " " -f 1 > $1.sha256' -- nifi-${NIFI_VERSION}-source-release.zip
sh -c 'sha256sum $1 | cut -d " " -f 1 > $1.sha256' -- nifi-${NIFI_VERSION}-bin.zip
sh -c 'sha256sum $1 | cut -d " " -f 1 > $1.sha256' -- nifi-kafka-connector-assembly-${NIFI_VERSION}.zip
sh -c 'sha256sum $1 | cut -d " " -f 1 > $1.sha256' -- nifi-registry-${NIFI_VERSION}-bin.zip
sh -c 'sha256sum $1 | cut -d " " -f 1 > $1.sha256' -- nifi-registry-toolkit-${NIFI_VERSION}-bin.zip
sh -c 'sha256sum $1 | cut -d " " -f 1 > $1.sha256' -- nifi-stateless-${NIFI_VERSION}-bin.zip
sh -c 'sha256sum $1 | cut -d " " -f 1 > $1.sha256' -- nifi-toolkit-${NIFI_VERSION}-bin.zip
sh -c 'sha512sum $1 | cut -d " " -f 1 > $1.sha256' -- minifi-${NIFI_VERSION}-bin.zip
sh -c 'sha512sum $1 | cut -d " " -f 1 > $1.sha256' -- minifi-toolkit-${NIFI_VERSION}-bin.zip
sh -c 'sha512sum $1 | cut -d " " -f 1 > $1.sha256' -- nifi-${NIFI_VERSION}-source-release.zip
sh -c 'sha512sum $1 | cut -d " " -f 1 > $1.sha256' -- nifi-${NIFI_VERSION}-bin.zip
sh -c 'sha512sum $1 | cut -d " " -f 1 > $1.sha256' -- nifi-kafka-connector-assembly-${NIFI_VERSION}.zip
sh -c 'sha512sum $1 | cut -d " " -f 1 > $1.sha256' -- nifi-registry-${NIFI_VERSION}-bin.zip
sh -c 'sha512sum $1 | cut -d " " -f 1 > $1.sha256' -- nifi-registry-toolkit-${NIFI_VERSION}-bin.zip
sh -c 'sha512sum $1 | cut -d " " -f 1 > $1.sha256' -- nifi-stateless-${NIFI_VERSION}-bin.zip
sh -c 'sha512sum $1 | cut -d " " -f 1 > $1.sha256' -- nifi-toolkit-${NIFI_VERSION}-bin.zip

Publish Artifacts

git push upstream ${JIRA_TICKET}-RC${RC}
git push upstream nifi-${NIFI_VERSION}-RC${RC}
cd ${STAGING_DIR}
svn checkout https://dist.apache.org/repos/dist/dev/nifi
cd nifi
cp -r ${ARTIFACTS_DIR} nifi-${NIFI_VERSION}
svn add nifi-${NIFI_VERSION}
svn commit -m "${JIRA_TICKET} Uploaded NiFi ${NIFI_VERSION}-RC${RC} artifacts"

Release Candidate Voting

The release vote process should take place for 72 hours under standard circumstances.

The Release Manager is responsible for sending the initial vote thread and tabulating results.

The Release Manager sends an email to the NiFi Developers Mailing List calling for a vote on the Release Candidate.

TO: dev@nifi.apache.org
SUBJECT: [VOTE] Release Apache NiFi ${NIFI_VERSION}
Team,

I am pleased to be calling this vote for the source release of Apache NiFi ${NIFI_VERSION}.

Please review the following guide for how to verify a release candidate build:

https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification

The source being voted on and the convenicen binaries are available in the Apache Repository:

https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}

The build artifacts are available in the Apache Nexus Repository:

https://repository.apache.org/content/repositories/${STAGING_REPO_ID}

Git Tag: nifi-${NIFI_VERSION}-RC${RC}
Git Commit ID: ${RC_TAG_COMMIT_ID}
GitHub Commit: https://github.com/apache/nifi/commit/${RC_TAG_COMMIT_ID}

Checksums of nifi-${NIFI_VERSION}-source-release.zip:

SHA256: ${SHA256_HASH}
SHA512: ${SHA512_HASH}

Release artifacts are signed with the following key:

https://people.apache.org/keys/committer/${RM_USERID}.asc

KEYS file is available in the Apache Repository:

https://dist.apache.org/repos/dist/release/nifi/KEYS

Issues resolved in this version: ${ISSUES_RESOLVED}

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12329307

Release note highlights can be found on the project wiki:

https://cwiki.apache.org/confluence/display/NIFI/Release+Notes

The vote will be open for 72 hours.

Please download the release candidate and evaluate the necessary items including checking
hashes, signatures, build from source, and test.

Please vote:

[ ] +1 Release this package as nifi-${NIFI_VERSION}
[ ] +0 no opinion
[ ] -1 Do not release this package because...

PMC members can cast binding votes. Committers and community members can cast non-binding votes.

The Release Manager can cancel a vote in response to negative findings. Canceling a vote requires sending an email with [CANCEL] in the subject line as follows:

SUBJECT: [CANCEL][VOTE] Release Apache NiFi ${NIFI_VERSION}-RC${RC}

Approving a Release Candidate build requires at least 3 binding positive votes from project PMC members and more positive votes than negative votes.

The Release Manager sends an email to the NiFi Developers Mailing List with the vote results.

TO: dev@nifi.apache.org
SUBJECT: [RESULT][VOTE] Release Apache NiFi ${NIFI_VERSION}-RC${RC}
Apache NiFi Community,

I am pleased to announce that the ${NIFI_VERSION} release of Apache NiFi passes:

    X +1 (binding) votes
    Y +1 (non-binding) votes
    0 0 votes
    0 -1 votes

Thanks to all who helped make this release possible!

Here is the vote thread: ${VOTE_THREAD_URL}

Release Artifacts

DIST_DEV_URL=https://dist.apache.org/repos/dist/dev/nifi/nifi-${NIFI_VERSION}
DIST_RELEASE_URL=https://dist.apache.org/repos/dist/release/nifi/${NIFI_VERSION}
svn move -m "${JIRA_TICKET}" ${DIST_DEV_URL} ${DIST_RELEASE_URL}
git checkout ${BRANCH}
git merge --no-ff ${JIRA_TICKET}-RC${RC}
git push upstream ${BRANCH}
export GPG_TTY=$(tty)
COMMIT_ID=`git rev-list -n 1 nifi-${NIFI_VERSION}-RC${RC}`
RELEASE_TAG="rel/nifi-${NIFI_VERSION}"
git tag -s ${RELEASE_TAG} -m "${JIRA_TICKET} Tagged NiFi ${NIFI_VERSION} ${COMMIT_ID}"
git push upstream ${RELEASE_TAG}
PREVIOUS_VERSION="0.6.0"
PREVIOUS_RELEASE_URL="https://dist.apache.org/repos/dist/release/nifi/${PREVIOUS_VERSION}"
svn delete -m "${JIRA_TICKET}" ${PREVIOUS_RELEASE_URL}
git commit -m "${JIRA_TICKET} Updated Docker version to ${NEXT_VERSION}"
git push upstream ${BRANCH}

Update Documentation

git clone https://gitbox.apache.org/repos/asf/nifi-site.git
cd nifi-site
git push origin main

Update Notes

Announce Release

The Release Manager sends an email to the Apache Announcements List as well as project mailing lists.

TO: announce@apache.org, users@nifi.apache.org, dev@nifi.apache.org
SUBJECT: [ANNOUNCE] Apache NiFi ${NIFI_VERSION} Released
The Apache NiFi Team is pleased to announce the release of Apache NiFi ${NIFI_VERSION}.

Apache NiFi is an easy to use, powerful, and reliable system to process and distribute
data.

https://nifi.apache.org

The release artifacts can be downloaded from the project website.

https://nifi.apache.org/download/

Maven artifacts have been released and mirrored according to Apache distribution processes.

Issues resolved in Apache NiFi ${NIFI_VERSION} are listed in Jira Release Notes.

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12329373

Highlights of the release are available on the project wiki.

https://cwiki.apache.org/confluence/display/NIFI/Release+Notes

Thank you,
The Apache NiFi Team