Run Apache NiFi Docker on Mac M1

In my recent articles on Apache NiFi, I explained how to run Apache NiFi on Docker and Docker Compose. Recently I got a Mac Book M1 Pro machine from work and surprisingly I couldn't run Apache NiFi on Docker in Mac Book M1. Though the Apache NiFi binary deployment works fine on Mac M1 architecture, the official Apache NiFi Docker image does not support Mac M1 yet (at the time of writing this article). However, Chris Sampson a NiFi committer provided a script to build NiFI docker image that is compatible with Mac M1. This article explains, how to build Apache NiFi docker image on your Mac Book M1 and how to run it.

Disclaimer: Since this is not an official image, I recommend this method only for pipeline development and testing purposes.

Step 01:
Create a shell script named build-nifi.sh with the following content:

nifi_version="${1:?'[error] Must provide paramter for NiFi version'}"
# check nifi_version is correct format
if [[ ! "${nifi_version}" =~ ^[[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+$ ]]; then
    echo "NiFi Version must be in format 'x.y.z' (where x, y and z are digits)"
    exit 1
fi

image_tag="apache/nifi:${nifi_version}-arm64"

# check this is an arm64 machine (e.g. Mac M1/2)
arch_name="$(uname -m)"
if [ "${arch_name}" = "arm64" ]; then
    echo "Running on ARM, buildingarm64 image: ${image_tag}"
else
    echo "Not running on arm64, skipping image build"
    exit 1
fi
echo

# build from nifi source nifi-docker/dockerhub/
nifi_repo="https://github.com/apache/nifi.git"
release_tag="rel/nifi-${nifi_version}"
echo "Cloning NiFi release tag ${release_tag} from ${nifi_repo}"
if [ -d nifi ]; then
    echo "Removing existing nifi directory"
    rm -rf nifi/
fi
git clone -b "${release_tag}" --single-branch "${nifi_repo}" --depth 1

# enable buildkit
export DOCKER_BUILDKIT=1
builder_name=qemu
echo "Creating buildx builder: ${builder_name}"
docker buildx create --use --bootstrap --name="${builder_name}"

echo; echo; echo "Building image with buildx builder: ${builder_name}"
pushd nifi/nifi-docker/dockerhub
docker buildx build --platform linux/arm64 --tag "${image_tag}" --output type=docker .
popd

echo; echo; echo "Removing buildx builder: ${builder_name}"
docker buildx rm -f --builder "${builder_name}"

Step 02:
Run the shell script with the latest NiFi version you want to run as shown below:

sh build-nifi.sh 1.16.3

It will take some time and create a new Docker image with the name: apache/nifi:1.16.3-arm64

Step 03:
List the Docker images to make sure that the arm64 image is there.

docker image ls

Step 04:
Now you can run NiFi using Docker command as explained in Run Apache NiFi in Docker. One difference thouh is you need to change the docker image name to apache/nifi:1.16.3-arm64 and add an extra paramter --platform=linux/arm64.

For example, the following command will start Apache NiFi 1.16.3 on port 8443 using the arm64 image in Mac M1.

docker run --platform=linux/arm64 --name nifi -p 8443:8443 -e SINGLE_USER_CREDENTIALS_USERNAME=admin -e SINGLE_USER_CREDENTIALS_PASSWORD=ctsBtRBKHRAx69EqUghvvgEvjnaLjFEB -d apache/nifi:1.16.3-arm64

Once NiFi is up and running, visit https://localhost:8443/nifi to access the dashboard. For more details about this command, please refer the Run Apache NiFi in Docker article.

 

Similarly you can run NiFi using Docker Compose as explained in Run Apache NiFi Cluster inDocker. Again make sure that you are using the arm64 image you built in Step 02 and define the platform as given below:

version: "3"
services:
  zookeeper:
    hostname: zookeeper
    container_name: zookeeper
    image: 'zookeeper:latest'
    ports:
      - 2181
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
  nifi:
    image: apache/nifi:1.16.3-arm64
    platform: linux/amd64
    ports:
      - 8080
    environment:
      - NIFI_WEB_HTTP_PORT=8080
      - NIFI_CLUSTER_IS_NODE=true
      - NIFI_CLUSTER_NODE_PROTOCOL_PORT=8082
      - NIFI_ZK_CONNECT_STRING=zookeeper:2181
      - NIFI_ELECTION_MAX_WAIT=1 min
      - NIFI_SENSITIVE_PROPS_KEY=xxxxxxxxxxxx

Run the cluster using the following docker-compose command:

docker-compose up

For more details, on how to run NiFi as a cluster using Docker Compose, please refer the Run Apache NiFi Clusterarticle.

If you find this article helpful and/or if you have any questions, please leave a comment below. I will try my best to answer your questions.

• Twitter
• Facebook
• LinkedIn
• Tumblr
• Pinterest