Apache NiFi source code is version controlled using Git version control (browse|checkout) and mirrored to GitHub.
The NiFi website source code is available (browse|checkout) and mirrored to GitHub.
The NiFi NAR maven plugin source code is available (browse|checkout) and mirrored to GitHub.
Track issues on the "NIFI" Project on the Apache JIRA (browse).
We recommend running the following git config commands in order to ensure
that git checks out the repository in a consistent manner. These changes
are particularly important if running on Windows, as the git client has
trouble with long filenames otherwise. Additionally, in Windows, the
default behavior of the git client, when installed, is to set the
core.autocrlf configuration option to
true, which can cause some of
the unit tests to fail.
git config --global core.longpaths true git config --global core.autocrlf false
To check out the code:
git clone https://gitbox.apache.org/repos/asf/nifi.git
Then checkout the
master branch (which should be the default)
git checkout master
master branch currently represents the next major release line (1.x). Substantial work here is underway to improve the NiFi's core framework and UI. If you are interested in building the 0.x baseline to include new components and bug fixes please checkout the 0.x branch.
git checkout -b 0.x origin/0.x
This will checkout the 0.x branch locally and set it to track the 0.x branch in the remote named
origin (the default name).
NOTE: If you are building on Linux, consider these best practices. Typical Linux defaults are not necessarily well tuned for the needs of an IO intensive application like NiFi.
For all of these areas, your distribution's requirements may vary. Use these sections as advice, but consult your distribution-specific documentation for how best to achieve these recommendations.
NiFi will at any one time potentially have a very large number of file handles open. Increase the limits by
editing /etc/security/limits.conf to add something like:
* hard nofile 50000 * soft nofile 50000
NiFi may be configured to generate a significant number of threads. Increase the allowable number by editing /etc/security/limits.conf to add something like:
* hard nproc 10000
* soft nproc 10000
Your distribution may require an edit to /etc/security/limits.d/90-nproc.conf by adding:
* soft nproc 10000
This is particularly important if your flow will be setting up and tearing down a large number of sockets in small period of time.
sudo sysctl -w net.ipv4.ip_local_port_range="10000 65000"
You don't want your sockets to sit and linger too long given that you want to be able to quickly setup and teardown new sockets. It is a good idea to read more about
it but to adjust do something like
sudo sysctl -w net.ipv4.netfilter.ip_conntrack_tcp_timeout_time_wait="1"
Swapping is fantastic for some applications. It isn't good for something like
NiFi that always wants to be running. To tell Linux you'd like swapping off, you
can edit /etc/sysctl.conf to add the following line
vm.swappiness = 0
For the partitions handling the various NiFi repos, turn off things like
Doing so can cause a surprising bump in throughput. Edit the /etc/fstab file
and for the partition(s) of interest add the
Additional information on system administration and settings can be located in our Administrator's Guide.
MAVEN_OPTS="-Xms1024m -Xmx3076m -XX:MaxPermSize=256m"
rootdir of the source tree run
mvn -T C2.0 clean install -Pinclude-grpc
-Pinclude-grpcif you're not on a platform that it supports.
-DskipTeststo compile tests, but skip running them.
Now you should have a fully functioning build off the latest codebase.
NOTE: During development it is not necessary to build the entire code base. Typically, in case of a NAR bundle change you can build only the particular bundle followed by running
mvn -T C2.0 clean install -Ddir-only in
Without any configuration, the application will run on port
8080 and does not require any credentials to modify
the flow. This means of running Apache NiFi should be used only for development/testing and in an environment where only
connections from trusted computers and users can connect to port
8080. Using iptables to allow only localhost connections
8080 is a good start, but on systems with multiple (potentially untrusted) users, also not a sufficient protection.
Running the above build will create a tar.gz (and zip) file in
./nifi-assembly/target. This tar.gz should
contain the full application. Decompressing the tar.gz should make a directory for you containing several other
conf contains application configuration,
bin contains scripts
for launching the application. On Linux and OSX, NiFi can be run using
bin/nifi.sh <command> where
<command> is one of:
start: starts NiFi in the background
stop: stops NiFi that is running in the background
status: provides the current status of NiFi
run: runs NiFi in the foreground and waits to receive a Ctrl-C, which then shuts down NiFi.
install: (available in Linux only, not OSX): installs NiFi as a service that can then be controlled
service nifi start,
service nifi stop,
service nifi status.
For Windows users, the following scripts exist in the
run-nifi.bat: runs NiFi in the foreground and waits to receive a Ctrl-C, which then shuts down NiFi
status-nifi.bat: provides the current status of NiFi
The configuration that is to be used when launching NiFi, such as Java heap size, the user
to run as, which Java command to use, etc. are configurable via the conf/bootstrap.conf file.
The entire concept of how the application will integrate to a given OS and run as an
enduring service is something we're working hard on and would appreciate ideas for. The user experience needs to
With the default settings you can point a web browser at
Logging is configured by default to log to ./logs/nifi-app.log. The following log message should indicate the web UI
is ready for use:
2014-12-09 00:42:03,540 INFO [main] org.apache.nifi.web.server.JettyServer NiFi has started. The UI is available at the following URLs: