Mastering Fluent Bit: Installing and Configuring Fluent Bit on Kubernetes

This series is a general purpose getting started guide for those of us wanting to learn about the Cloud Native Computing Foundation (CNCF) project Fluent Bit. 

Each article in this series addresses a single topic by providing insights into what the topic is, why we are interested in exploring that topic, where to get started with the topic, and how to get hands-on with learning about the topic as it relates to the Fluent Bit project.

The idea is that each article can stand on its own, but that they also lead down a path that slowly increases our abilities to implement solutions with Fluent Bit telemetry pipelines.

Let's take a look at the topic of this article, installing and configuring Fluent Bit on a Kubernetes cluster.

In case you missed the previous article, I'm providing a short introduction to Fluent Bit before sharing how to install and use the Fluent Bit telemetry pipeline on our own local machine with container images.

What is Fluent Bit?

Before diving into Fluent Bit, let's step back and look at the position of this project within the Fluent organization. If we look at the Fluent organization on Github, we find Fluentd and Fluent Bit project hosted there. The back story is that the project started with log parsing project Fluentd joining the CNCF in 2026 and reaching Graduated status in 2019. 

Once it became apparent that the world was heading into cloud native Kubernetes environments, the solution was not designed for the flexible and light weight requirements that Kubernetes solutions demanded. Fluent bit was born from the need to have a low resource using, high throughput, and highly scalable log management solution for cloud native Kubernetes environments. The project was started within the Fluent organization as a sub-project in 2017 and the rest is now 10 years of history with the release of v4 last week!

Fluent Bit has become so much more than a flexible and lightweight log pipeline solution, now able to process metrics and traces and becoming a telemetry pipeline collection tool of choice for those looking to put the control over their telemetry data right at the source where it's being collected.

Let's get started with Fluent Bit and see what we can do for ourselves!

Why install on Kubernetes?

When you dive into the cloud native world this means you are deploying containers on Kubernetes. The complexities increase dramatically as your applications and microservices interact in this complex and dynamic infrastructure landscape. 

Deployments can auto-scale, pods spin up and are taken down as the need arises, and underlying all of this are the various Kubernetes controlling components. All of these things are generating telemetry data and Fluent Bit is a wonderfully simple way to manage them across a Kubernetes cluster. It provides a way of collecting everything as you go while providing the pipeline parsing, filtering, and routing to handle all your telemetry data.

For developers, this article will demonstrate installing and configuring Fluent Bit as a single point of log collection on a development Kubernetes cluster with a deployed workload.

Where to get started

Before getting started there will be some minimum requirements needed to run all the software and explore this demo project. The first is the ability to run container images with Podman tooling. While it is always best to be running the latest versions of most software, let's look at the minimum you need to work with the examples shown in this article.

It is assumed you can install this on your local machine prior to reading this article. To test this you can run the following from a terminal console on your machine:

$ podman -v

podman version 5.4.1

If you prefer, you can install the Podman Desktop project and it will provide all the needed CLI tooling you see used in the rest of this article. Be aware, I won't spend any time focusing on the desktop version.

Also note that if you want to use Docker, feel free, it's pretty similar in commands and usage that you see here, but again, I will not reference that tooling in this article.

Next you will be using Kind to run a Kubernetes cluster on your local machine, so ensure the version is at least as shown:

$ kind version

kind v0.27.0 ...

To control the cluster and deployments you need the tooling kubectl, with a minimum version as shown:

$ kubectl version

Client Version: v1.32.2

Last but not least, Helm charts are leveraged to control your Fluent Bit deployment on the cluster, so ensure it is at least the following:

$ helm version

version.BuildInfo{Version:"v3.16.4" ...

Finally, all examples in this article have been done on OSX and are assuming the reader is able to convert the actions shown here to their own local machines.

How to install and configure on Kubernetes

The first installation of Fluent Bit on a Kubernetes cluster is done in several steps, but the foundation is ensuring your Podman virtual machine is running. The following assumes you have already initialized your Podman machine, so you can start it as follows:

$ podman machine start

Starting machine "podman-machine-default"
WARN[0000] podman helper is installed, but was not able to claim the global 

docker sock

[SNIPPED OUTPUT]

Another process was listening on the default Docker API socket address.
You can still connect Docker API clients by setting DOCKER_HOST using the
following command in your terminal session:

export DOCKER_HOST='unix:///var/folders/6t/podman/podman-machine-default-api.sock'

Machine "podman-machine-default" started successfully

If you see something like this, then there are issues with connecting to the API socket, so Podman provides a variable to export that will work for this console session. You just need to copy that export line into your console and execute it as follows:

$ export DOCKER_HOST='unix:///var/folders/6t/podman/podman-machine-default-api.sock'

Now that you have Podman ready, you can start the process that takes a few steps in order to install the following:

1. Install a Kubernetes two node cluster with Kind.
2. Install Ghost CMS to generate workload logs.
3. Install and configure Fluent Bit to collect Kubernetes logs.

To get started, create a directory structure for your Kubernetes cluster. You need one for the control node and one for the worker node, so run the following to create your setup:

$ mkdir -p target

$ mkdir -p target/ctrlnode

$ mkdir -p target/wrkrnode1

The next step is to run the Kind install command with a few configuration flags explained below. The first command is to remove any existing cluster you might have of the same name, clearing the way for our installation:

$ KIND_EXPERIMENTAL_PROVIDER=podman kind --name=2node delete cluster

using podman due to KIND_EXPERIMENTAL_PROVIDER
enabling experimental podman provider
Deleting cluster "2node" ...

You need a Kind configuration to define our Kubernetes cluster and pointing it to the directories you created, so create the file 2nodekindconfig.yaml with the following :

kind: Cluster

apiVersion: kind.x-k8s.io/v1alpha4
name: 2nodecluster
nodes:
- role: control-plane
  extraMounts:
    - hostPath: target/ctrlnode
      containerPath: /ghostdir
- role: worker
  extraMounts:
    - hostPath: target/wrkrnode1
      containerPath: /ghostlier

With this file you can create a new cluster with the following definitions and configuration to spin up a two node Kubernetes cluster called 2node: 

$ KIND_EXPERIMENTAL_PROVIDER=podman kind create cluster --name=2node --config="2nodekindconfig.yaml" --retain

using podman due to KIND_EXPERIMENTAL_PROVIDER
enabling experimental podman provider
Creating cluster "2node" ...
 ✓ Ensuring node image (kindest/node:v1.32.2) 🖼
 ✓ Preparing nodes 📦 📦
 ✓ Writing configuration 📜
 ✓ Starting control-plane 🕹️
 ✓ Installing CNI 🔌
 ✓ Installing StorageClass 💾
 ✓ Joining worker nodes 🚜
Set kubectl context to "kind-2node"
You can now use your cluster with:

kubectl cluster-info --context kind-2node

Have a nice day! 👋

The Kubernetes cluster spins up and you can view it with kubectl tooling as follows:

$ kubectl config view

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://127.0.0.1:58599
  name: kind-2node
contexts:
- context:
    cluster: kind-2node
    user: kind-2node
  name: kind-2node
current-context: kind-2node
kind: Config
preferences: {}
users:
- name: kind-2node
  user:
    client-certificate-data: DATA+OMITTED
    client-key-data: DATA+OMITTED

To make use of this cluster you can set the context for your kubectl tooling as follows:

$ kubectl config use-context kind-2node

Switched to context "kind-2node".

Time to deploy a workload on this cluster to start generating real telemetry data for Fluent Bit. To prepare for this installation we need to create the persistent volume storage for our workload, a Ghost CMS. The following needs to be put into the file ghost-static-pvs.yaml:

---

apiVersion: v1
kind: PersistentVolume
metadata:
  name: ghost-content-volume
  labels:
    type: local
spec:
  storageClassName: ""
  claimRef:
    name: data-my-ghost-mysql-0
    namespace: ghost
  capacity:
    storage: 8Gi
  accessModes:
    - ReadWriteMany
  hostPath:
    path: "/ghostdir"
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: ghost-database-volume
  labels:
    type: local
spec:
  storageClassName: ""
  claimRef:
    name: my-ghost
    namespace: ghost
  capacity:
    storage: 8Gi
  accessModes:
    - ReadWriteMany
  hostPath:
    path: "/ghostdir"

With this file you can now use kubectl to create it on your cluster as follows:

$ kubectl create -f ghost-static-pvs.yaml --validate=false

persistentvolume/ghost-content-volume created
persistentvolume/ghost-database-volume created

With the foundations laid for using Ghost CMS as our workload, we need to add the Helm chart to our local repository before using it to install anything:

$ helm repo add bitnami https://charts.bitnami.com/bitnami

"bitnami" has been added to your repositories

The next step is to use this repository to install Ghost CMS, configuring it by supplying parameters as follows:

$ helm upgrade --install ghost-dep bitnami/ghost --version "21.1.15" --namespace=ghost --create-namespace --set ghostUsername="adminuser" --set ghostEmail="admin@example.com" --set service.type=ClusterIP --set service.ports.http=2368

Release "ghost-dep" does not exist. Installing it now.
NAME: ghost-dep
LAST DEPLOYED: Thu May  1 16:28:26 2025
NAMESPACE: ghost
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: ghost
CHART VERSION: 21.1.15
APP VERSION: 5.86.2

** Please be patient while the chart is being deployed **

1. Get the Ghost URL by running:

  echo Blog URL  : http://127.0.0.1:2368/
  echo Admin URL : http://127.0.0.1:2368/ghost
  kubectl port-forward --namespace ghost svc/ghost-dep 2368:2368

2. Get your Ghost login credentials by running:

  echo Email:    admin@example.com
  echo Password: $(kubectl get secret --namespace ghost ghost-dep -o jsonpath="{.data.ghost-password}" | base64 -d)

This command completes pretty quickly, but in the background your cluster is spinning up the Ghost CMS nodes and this takes some time. To ensure your installation is ready to proceed, run the following command that waits for the workload to finish spinning up before proceeding:

$ kubectl wait --for=condition=Ready pod --all --timeout=200s --namespace ghost

pod/ghost-dep-74f8f646b-96d59 condition met
pod/ghost-dep-mysql-0 condition met

If this command times out due to your local machine taking too long, just restart it until it finishes with the two condition met statements. This means your Ghost CMS is up and running, but needs a bit of configuration to reach it on your cluster from the local machine. Run the following commands, noting the first it put into the background with the & sign:

$ kubectl port-forward --namespace ghost svc/ghost-dep 2368:2368 &

Forwarding from 127.0.0.1:2368 -> 2368
Forwarding from [::1]:2368 -> 2368

[1] 6997  

This completes the installation and configuration of our workload, which you can validate is up and running at http://localhost:2368. This should show you a Users Blog landing page on your Ghost CMS instance, nothing more is needed for this article than to have it running.

The final step is to install Fluent Bit and start collecting cluster logs. Start by adding the Fluent Bit Helm chart to your local repository as follows:

$ helm repo add fluent https://fluent.github.io/helm-charts

"fluent" has been added to your repositories

The installation will need some configuration parameters that you need to put into a file passed to the helm chart during installation. Add the following to the file fluentbit-helm.yaml:

args:
  - --workdir=/fluent-bit/etc
  - --config=/fluent-bit/etc/conf/fluent-bit.yaml

config:
  extraFiles:
    fluent-bit.yaml: |
      service:
       flush: 1
       log_level: info
       http_server: true
       http_listen: 0.0.0.0
       http_port: 2020
      pipeline:
        inputs:
          - name: tail
            tag: kube.*
            read_from_head: true
            path: /var/log/containers/*.log
            multiline.parser: docker, cri
        filters:
          - name: grep
            match: '*'
        outputs:
          - name: stdout
            match: '*'

With this file you can now install Fluent Bit on your cluster as follows:

$ helm upgrade --install fluent-bit fluent/fluent-bit --set image.tag="4.0.0" --namespace=logging --create-namespace --values="support/fluentbit-helm.yaml"

Release "fluent-bit" does not exist. Installing it now.
NAME: fluent-bit
LAST DEPLOYED: Thu May  1 16:50:04 2025
NAMESPACE: logging
STATUS: deployed
REVISION: 1
NOTES:
Get Fluent Bit build information by running these commands:

export POD_NAME=$(kubectl get pods --namespace logging -l "app.kubernetes.io/name=fluent-bit,app.kubernetes.io/instance=fluent-bit" -o jsonpath="{.items[0].metadata.name}")
kubectl --namespace logging port-forward $POD_NAME 2020:2020
curl http://127.0.0.1:2020

This starts the installation of Fluent Bit, and again you will need to wait until it completes with the help of the following commands:

$ kubectl wait --for=condition=Ready pod --all --timeout=100s --namespace logging

pod/fluent-bit-58vs8 condition met

Now you can verify that your Fluent Bit instance is running and collecting all Kubernetes cluster logs, from the control node, the worker node, and from the workloads on the cluster with the following:

$ kubectl config set-context --current --namespace logging

Context "kind-2node" modified.

$ kubectl get pods

NAME               READY   STATUS    RESTARTS   AGE
fluent-bit-58vs8   1/1     Running   0          6m56s

$ kubectl logs fluent-bit-58vs8

[DUMPS-ALL-CLUSTER-LOGS-TO-CONSOLE]

Now you have a fully running Kubernetes cluster, with two nodes, a workload in the form of a Ghost CMS, and finally you've installed Fluent Bit as your telemetry pipeline configured to collect all cluster logs.

If you want to do this without each step done manually, I've provided a Logs Control Easy Install project repository that you can download, unzip, and run with one command to automate the above setup on your local machine.

More in the series

In this article you learned how to install and configure Fluent Bit on a Kubernetes cluster to collect telemetry from the cluster. This article is based on this online free workshop.

There will be more in this series as you continue to learn how to configure, run, manage, and master the use of Fluent Bit in the wild. Next up, controlling your logs with Fluent Bit on a Kubernetes cluster.