Apache NiFi Flow Design System Releases

The purpose of this document is to capture and describe the steps involved in producing an official release of Apache NiFi Flow Design System. It is written specifically to someone acting in the capacity of a Release Manager (RM).

The objective

Our aim is to produce an official Apache release from an existing release branch.

Background Material

Terms

High level flow of a release

Variable reference substitutions

Throughout this guide, references must be made to names and values that will vary from release to release. For clarity those variable values have been written like Bash variable references. When a term like “/tmp/src/nifi-fds-${NIFI_FDS_VERSION}” is seen in an instruction or email template it should be replaced with “/tmp/src/nifi-fds-0.1.0” when working the release of “Apache NiFi Flow Design System 0.1.0”.

NOTE: The next version should be the next minor version if the release is based on a major version development branch (e.g main or 0.x). The next version should be the next incremental version if the release is based on a minor version development branch (e.g support/nifi-fds-1.1.x or support/nifi-fds-0.7.4). If this is the first incremental release (e.g. 1.2.1) for a minor release line the support branch may need to be created.

What to validate and how to validate a release

The following is a list of the sorts of things that will be validated and are the basics to check when evaluating a release for a vote.

This list is reflected in the Release Vote and Release Helper Guide emails that are sent once the release has been staged in the Git and Nexus repositories.

The Release Process

The release process includes steps to be performed by the Release Manager as well as the Apache NiFi Flow Design System developer community.

Step 1. Configure the build environment (RM and community)

  1. Confirm that the local Git workspace is configured with an origin remote pointing to the RM’s personal fork of the NiFi source and an “ASF” remote pointing to the Apache Git Repository for NiFi. 
    $ git remote -v
asf	https://gitbox.apache.org/repos/asf/nifi-fds.git (fetch)
asf	https://gitbox.apache.org/repos/asf/nifi-fds.git (push)
origin	https://github.com/${RM_USERID}/nifi-fds.git (fetch)
origin	https://github.com/${RM_USERID}/nifi-fds.git (push)
    Additional remotes will not cause a problem if these two are correct. Other configurations are perfectly acceptable but the appropriate adjustments to the steps in this guide must be made by the release manager.
  2. Confirm that source code can be checked out for the branch being released. 
    git checkout ${BRANCH}
  3. Confirm that the entire application builds correctly in the build environment.

Step 2. Prepare and stage the release (RM)

  1. Create a JIRA ticket for the release tasks for version ${NIFI_FDS_VERSION}.
    The resulting JIRA ticket number is referred to as ${JIRA_TICKET} in this guide.
  2. Create the next version in JIRA, if it doesn’t already exist, so work can continue towards that release.
  3. Create meaningful release notes for this version if not already created. Enter them here on the NiFi wiki.
  4. Create a new branch off ‘main’ named after the JIRA ticket. 
    $ git checkout -b NIFI-FDS-${NIFI_FDS_VERSION}-RC${RC} ${BRANCH}
  5. Ensure the the full application builds and all tests work by executing a build. 
    $ npm run clean:install
  6. Startup and test the demo application from the target source folder. After a few seconds, NiFi FDS should be up and running at http://localhost:8080. 
    $ cd target/frontend-working-directory
$ npm start
  7. Evaluate and ensure the appropriate license headers are present on all source files.
  8. Ensure LICENSE and NOTICE files are complete and accurate. (Developers should always be keeping these up to date as they go along adding source and modifying dependencies to keep this burden manageable.)
  9. Verify that no vulnerabilities exist in any javascript module dependencies (requires npm 6+). 
    $ npm audit

Step 3. Perform the release (RM)

  1. Now it’s time to prepare the release. Manually update the version number in

    • root package.json
    • root package-lock.json
    • src/platform/core/package.json

  2. Zip up the source.

    $ git archive --format=zip --prefix=nifi-fds-${NIFI_FDS_VERSION}/ NIFI-FDS-${NIFI_FDS_VERSION}-RC${RC} > nifi-fds-0.2.0-source-release.zip

  3. Create the tag.

    $ git tag NIFI-FDS-${NIFI_FDS_VERSION}-RC${RC}

  4. Push the release branch to the ASF repository.

    $ git push asf NIFI-FDS-${NIFI_FDS_VERSION}-RC${RC} --tags

    From this branch, the ${RC_TAG_COMMIT_ID} will be the 40 byte commit hash with the comment NIFI-FDS-${NIFI_FDS_VERSION}-RC${RC} prepare release nifi-fds-${NIFI_FDS_VERSION}-RC${RC}

  5. Create the signature and hashes for the source release.

    1. ASCII armored GPG signatures (--digest-algo=SHA512 select the SHA512 hash algorithm). Configure GPG to always prefer stronger hashes. 
      $ gpg -a -b --digest-algo=SHA512 nifi-fds-${NIFI_FDS_VERSION}-source-release.zip  # produces nifi-fds-${NIFI_FDS_VERSION}-source-release.zip.asc
    2. Generate SHA256 hash summaries. 
      $ shasum -a 256 nifi-fds-${NIFI_FDS_VERSION}-source-release.zip | cut -d" " -f1 >  nifi-fds-${NIFI_FDS_VERSION}-source-release.zip.sha256
    3. Generate SHA512 hash summaries. 
      $ shasum -a 512 nifi-fds-${NIFI_FDS_VERSION}-source-release.zip | cut -d" " -f1 >  nifi-fds-${NIFI_FDS_VERSION}-source-release.zip.sha512

  6. Commit the source release along with their hashes and signatures to https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-${NIFI_FDS_VERSION}.

    1. Checkout the Apache dist dev svn repo locally using your Apache credentials 
      $ svn checkout https://dist.apache.org/repos/dist/dev/nifi
    2. Create the directory for nifi-fds-${NIFI_FDS_VERSION}.
    3. Copy the zipped source release and its corresponding signatures and hashes into the new directory.
    4. Add the commit. 
      $ svn add ./nifi-fds-${NIFI_FDS_VERSION}/*
    5. Stage the artifacts by committing to svn. 
      $ svn commit -m 'Staging artifacts for nifi-fds-${NIFI_FDS_VERSION}'

Step 4. Error recovery (RM)

If anything isn’t correct about the staged artifacts you can delete the local tag in git. If you also delete the local branch then it is as if the release never happened. Before doing that though try to figure out what went wrong so the Release Guide can be updated or corrected if necessary.

Step 5. Release Vote (RM and community)

After the release source is staged in the repositories it’s time for the RM to send a release vote to the NiFi community.

Once the release vote is called for, members of the NiFi FDS developer community have 72 hours to evaluate the RC and cast their vote by replying to the “[VOTE] Release …” email sent by the RM.

NOTE: The release vote is majority rule vote that must include at least 3 binding +1 votes Apache NiFi PMC members and more positive than negative binding votes.

  1. RM sends a vote request email to the NiFi Developers Mailing List.

    • TO: dev@nifi.apache.org
    • FROM: ${RM_USERID}@apache.org
    • SUBJECT: [VOTE] Release Apache NiFi Flow Design System ${NIFI_FDS_VERSION}
    Hello,

I am pleased to be calling this vote for the source release of Apache NiFi Flow Design System nifi-fds-${NIFI_FDS_VERSION}.

The source zip, including signatures, etc. can be found at:
https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-${NIFI_FDS_VERSION}

The Git tag is nifi-${NIFI_FDS_VERSION}-RC${RC}
The Git commit ID is ${RC_TAG_COMMIT_ID}
https://gitbox.apache.org/repos/asf?p=nifi-fds.git;a=commit;h=${RC_TAG_COMMIT_ID}

Checksums of nifi-fds-x.y.z-source-release.zip:
SHA1: <40-BYTE-SHA1SUM-HASH>
SHA256: <64-CHAR-SHA256SUM-HASH>
SHA512: <128-CHAR-SHA512SUM-HASH>

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/${RM_USERID}.asc

KEYS file available here:
https://dist.apache.org/repos/dist/release/nifi/KEYS

8 issues were closed/resolved for this release:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12329307

Release note highlights can be found here:
https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version${NIFI_FDS_VERSION}

The vote will be open for 72 hours.
Please download the release candidate and evaluate the necessary items including checking hashes, signatures, build
from source, and test.  The please vote:

[ ] +1 Release this package as nifi-fds-${NIFI_FDS_VERSION}
[ ] +0 no opinion
[ ] -1 Do not release this package because...

  2. RM sends the following helper email to the NiFi Developers Mailing List.

    • TO: dev@nifi.apache.org
    • FROM: ${RM_USERID}@apache.org
    • SUBJECT: Apache NiFi ${NIFI_FDS_VERSION} RC${RC} Release Helper Guide
    Hello Apache NiFi community,

Please find the associated guidance to help those interested in validating/verifying the release so they can vote.

# Download latest KEYS file:
https://dist.apache.org/repos/dist/dev/nifi/KEYS

# Import keys file:
gpg --import KEYS

# Pull down nifi-fds-${NIFI_FDS_VERSION} source release artifacts for review:
wget https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-${NIFI_FDS_VERSION}/nifi-fds-${NIFI_FDS_VERSION}-source-release.zip
wget https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-${NIFI_FDS_VERSION}/nifi-fds-${NIFI_FDS_VERSION}-source-release.zip.asc
wget https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-${NIFI_FDS_VERSION}/nifi-fds-${NIFI_FDS_VERSION}-source-release.zip.sha1
wget https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-${NIFI_FDS_VERSION}/nifi-fds-${NIFI_FDS_VERSION}-source-release.zip.sha256
wget https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-${NIFI_FDS_VERSION}/nifi-fds-${NIFI_FDS_VERSION}-source-release.zip.sha512

# Verify the signature
gpg --verify nifi-fds-${NIFI_FDS_VERSION}-source-release.zip.asc

# Verify the hashes (sha1, sha256, sha512) match the source and what was provided in the vote email thread
sha1sum nifi-fds-${NIFI_FDS_VERSION}-source-release.zip
sha256sum nifi-fds-${NIFI_FDS_VERSION}-source-release.zip
sha512sum nifi-fds-${NIFI_FDS_VERSION}-source-release.zip

# Unzip nifi-fds-${NIFI_FDS_VERSION}-source-release.zip

# Verify the build works
cd nifi-fds-${NIFI_FDS_VERSION}
npm run clean:install

# Verify the contents contain a good README, NOTICE, and LICENSE.

# Verify the git commit ID is correct

# Verify the RC was branched off the correct git commit ID

# Run the demo-app and make sure it works as expected
cd target/frontend-working-directory
npm run watch

# Make sure the README, NOTICE, and LICENSE are present and correct 
cd node_modules/@nifi-fds/core

# Send a response to the vote thread indicating a +1, 0, -1 based on your findings.

Thank you for your time and effort to validate the release!

  3. Developers in the community review the release candidate and reply to the vote email with their vote.

  4. After 72 hours if

    • at least 3 binding (PMC members) cast +1 votes, and
    • the positive binding votes out number any negative binding votes

    the vote passes and the release candidate is officially released. If the vote does not pass, corrections are made on the release branch and a new release candidate is put forward for a new vote.

  5. RM sends vote result email.

    • TO: dev@nifi.apache.org
    • FROM: ${RM_USERID}@apache.org
    • SUBJECT: [RESULT][VOTE] Release Apache NiFi Flow Design System ${NIFI_FDS_VERSION}
    Apache NiFi Community,

I am pleased to announce that the ${NIFI_FDS_VERSION} release of Apache NiFi Flow Design System passes with
    X +1 (binding) votes
    Y +1 (non-binding) votes
    0 0 votes
    0 -1 votes

Thanks to all who helped make this release possible.

Here is the PMC vote thread: ${VOTE_THREAD_URL}

Step 6. Finalize the Release

After the vote is complete and the release is approved, these steps complete the release process.

  1. Move the source code and related artifacts from dist/dev to dist/release:

    $ svn move -m "NIFI-FDS-${NIFI_FDS_VERSION}" https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-${NIFI_FDS_VERSION} https://dist.apache.org/repos/dist/release/nifi/nifi-fds/nifi-fds-${NIFI_FDS_VERSION}

  2. Manually update the version number to the next snapshot

    $ git checkout NIFI-FDS-${NIFI_FDS_VERSION}-RC${RC}

    Update the version number to the next ${NIFI_FDS_VERSION}-SNAPSHOT in

    • root package.json
    • root package-lock.json
    • src/platform/core/package.json
    $ git add -A .
$ git commit -m 'NIFI-FDS-${NIFI_FDS_VERSION} finalize RC${RC}'

  3. Merge the release branch into main.

    $ git checkout main
$ git merge --no-ff NIFI-FDS-${NIFI_FDS_VERSION}-RC${RC}
$ git push asf main

  4. Publish to the npm registry

    $ cd nifi-fds/target/frontend-working-directory/platform/core
$ cp -f ../../README.md ./README.md
$ npm login
$ npm publish

  5. Deploy the gh-pages demo

    First, switch to gh-pages branch:

    $   git remote update
$   git checkout gh-pages
$   git reset --hard asf/gh-pages

    Next, simulate npm install of @nifi-fds/core:

    $   rm -rf ./node_modules/
$   mv ./target/frontend-working-directory/node_modules/ ./
$   rm -rf ./webapp/
$   mv ./target/frontend-working-directory/webapp/ ./

    Next, update the bundles, index.html, and README for the github.io page:

    $   cp ./target/frontend-working-directory/fds-demo.* .
$   cp -f ./target/frontend-working-directory/index.html ./index.html
$   cp -f ./target/frontend-working-directory/README.md ./README.md

    Next, start the demo:

    $   ./node_modules/http-server/bin/http-server

    Now, test that the demo application runs with the latest updates by visiting http://127.0.0.1:8080.

    Next, edit the index.html and change the base href to <base href="/nifi-fds/">:

    $   nano ./index.html

    and change the base href to <base href="/nifi-fds/">.

    Finally, commit and push the changes:

    $   git add -A .
$   git commit -m 'NIFI-FDS-${NIFI_FDS_VERSION} gh-pages update demo application to run NiFi FDS ${NIFI_FDS_VERSION}'
$   git push asf gh-pages:gh-pages

  6. Update the NiFi Web Page to indicate NEWS of the release as appropriate

  7. In JIRA resolve the NIFI-FDS-${JIRA_TICKET} and then mark the release version as ‘Released’ and ‘Archived’ through ‘version’ management in the ‘administration’ console. Then create a new version name for the next NiFi FDS release.

  8. Create a proper signed tag of the released codebase based on the RC Tag created during the release process.

    $ git tag -s rel/nifi-fds-${NIFI_FDS_VERSION} -m "NIFI-FDS-${JIRA_TICKET} signed release tag for approved release of NiFi Flow Design System ${NIFI_FDS_VERSION}" ${RC_TAG_COMMIT_ID}

    For instructions on setting up to sign your tag see here.

  9. Push the release tag to the official ASF repository.

    $ git push asf rel/nifi-fds-${NIFI_FDS_VERSION}

  10. Update the release notes with the final date of the release.

  11. After the release has been complete for 24 hours send the release announcement.