This series is a general purpose getting started guide for those of us wanting to learn about the Cloud Native Computing Foundation (CNCF) project Fluent Bit.
Each article in this series addresses a single topic by providing insights into what the topic is, why we are interested in exploring that topic, where to get started with the topic, and how to get hands-on with learning about the topic as it relates to the Fluent Bit project.
The idea is that each article can stand on its own, but that they also lead down a path that slowly increases our abilities to implement solutions with Fluent Bit telemetry pipelines.
Let's take a look at the topic of this article, using Fluent Bit tips for developers. In case you missed the previous article, check out the developers guide to service section configuration where you get tips on making the most of your developer inner loop with Fluent Bit.
This article will be a hands-on tour of the things that help you as a developer testing out your Fluent Bit pipelines. We'll take a look at the input plugin section of a developers telemetry pipeline configuration.
All examples in this article have been done on OSX and are assuming the reader is able to convert the actions shown here to their own local machines.
Where to get started
You should have explored the previous articles in this series to install and get started with Fluent Bit on your developer local machine, either using the source code or container images. Links at the end of this article will point you to a free hands-on workshop that lets you explore more of Fluent Bit in detail.
You can verify that you have a functioning installation by testing your Fluent Bit, either using a source installation or a container installation as shown below:
# For source installation.$ fluent-bit -i dummy -o stdout
# For container installation.$ podman run -ti ghcr.io/fluent/fluent-bit:4.0.8 -i dummy -o stdout
...
[0] dummy.0: [[1753105021.031338000, {}], {"message"=>"dummy"}]
[0] dummy.0: [[1753105022.033205000, {}], {"message"=>"dummy"}]
[0] dummy.0: [[1753105023.032600000, {}], {"message"=>"dummy"}]
[0] dummy.0: [[1753105024.033517000, {}], {"message"=>"dummy"}]...Let's look at a few tips and tricks to help you with your local development testing with regards to Fluent Bit input plugins.
Pipeline input tricks
See the previous article for details about the service section of the configurations used in the rest of this article, but for now we plan to focus on our Fluent Bit pipeline and specifically the input plugins that can be of great help in our developer inner loop during testing.
1. Dummy input plugin
When trying to narrow down a problem in our telemetry pipeline filtering or parsing actions, a developer's best friend is the ability to reduce the environment down to a controllable log input to process. Once the filter or parsing action has been validated to work, we can then proceed in our development environment with further testing of our project.
The best pipeline input plugin helping us when trying to sort out that complex, pesky regular expression or Lua script in a filter is the dummy input plugin. It's also a quick and easy plugin to use for exploring a chain of filters that you want to verify before setting loose on your organization's telemetry data.
We see below the ability to generate any sort of telemetry data using the key field dummy, and tagging it with the key field tag. It can be as simple or complex as we need, including pasting in a copy of our production telemetry data if needed to complete our testing. Note that this is a simple example configuration and contains a simple modify filter to illustrate the input plugin usage.
service: flush: 1
log_level: info http_server: on
http_listen: 0.0.0.0
http_port: 2020
hot_reload: on
pipeline:
inputs:
# This entry generates a successful message.
- name: dummy
tag: event.success
dummy: '{"message":"true 200 success"}'
# This entry generates a failure message.
- name: dummy
tag: event.error
dummy: '{"message":"false 500 error"}'
filters:
# Example testing filter to modify events.
- name: modify
match: '*'
condition:
- Key_Value_Equals message 'true 200 success'
remove: message
add:
- valid_message true
- code 200
- type success
outputs:
- name: stdout
match: '*'
format: json_linesRunning the above configuration file allows us to tinker with the filter until we are satisfied that it works before we install it on our developer testing environments. For completeness, we run this configuration to see the output as follows:
# For source installation.$ fluent-bit --config fluent-bit.yaml
# For container installation after building new image with your # configuration using a Buildfile as follows:## FROM ghcr.io/fluent/fluent-bit:4.0.8
# COPY ./fluent-bit.yaml /fluent-bit/etc/fluent-bit.yaml# CMD [ "fluent-bit", "-c", "/fluent-bit/etc/fluent-bit.yaml" ]#$ podman build -t fb -f Buildfile
$ podman run --rm fb
...{"date":1756813283.411961,"valid_message":"true","code":"200","type":"success"}
{"date":1756813283.413117,"message":"false 500 error"}
{"date":1756813284.410205,"valid_message":"true","code":"200","type":"success"}
{"date":1756813284.41048,"message":"false 500 error"}
{"date":1756813285.410716,"valid_message":"true","code":"200","type":"success"}
{"date":1756813285.410987,"message":"false 500 error"}...
As you can see, for developers the options here are endless to quickly verify or tune your telemetry pipelines in a testing environment. Let's look at another handy plugin for developers, the tail input plugin.
2. Tail input plugin
Much of our development work and testing takes place in cloud native environments, therefore, it means we are dealing with dynamic logging and streams of log telemetry data. In UNIX based operating systems there is a very handy tool for looking at large files, specifically connecting to expanding files while displaying the incoming data. This tool is called tail and with a '-f' flag added it provides a file name to attach to while dumping all incoming data to the standard output.
The tail input plugin has been developed with that same idea in mind. In a cloud native environment where Kubernetes is creating dynamic pods and containers, the log telemetry data is collected in a specific location. By using wildcard path names, you can ensure that you are connecting and sending incoming telemetry data to your Fluent Bit pipeline.
service: flush: 1
log_level: infohttp_server: on http_listen: 0.0.0.0 http_port: 2020 hot_reload: on pipeline: inputs:
- name: tail
tag: kube.*
read_from_head: true path: /var/log/containers/*.log
multiline.parser: docker, cri
outputs:
- name: stdout match: '*' format: json_lines json_date_format: java_sql_timestampThis was detailed in a previous article from this series, Controlling Logs with Fluent Bit on Kubernetes, where logs were collected and sent to Fluent Bit. Instead of rehashing the details of this article, we will reiterate a few of the important tips for developers here:
- When attaching to a location, such as the log collection path for a Kubernetes cluster, the tail plugin only collects telemetry data from that moment forward. It's missing previously logged telemetry.
- Using this adjustment to the tail input plugin configuration ensures we see the entire log telemetry data from your testing application:
- read_from_head: true
- Narrowing the log telemetry data down from all running containers on a cluster is shown below, first item being all container logs, the second being your specific app only. This tail input plugin configuration modification is very handy when testing our applications:
- path: /var/log/containers/*.log
- path: /var/log/containers/*APP_BEING_TESTED*
- Finally, we need to ensure we are making use of filters in our pipeline configuration to reduce the telemetry noise when testing applications. This helps us to narrow the focus to the debugging telemetry data that is relevant.
Another plugin worth mentioning in the same breath is the head input plugin. This works the same as the UNIX command head where we are giving it a number of lines to read out of the top of a file (the first N number of lines). Below is an example configuration:
service: flush: 1
log_level: infohttp_server: on http_listen: 0.0.0.0 http_port: 2020 hot_reload: on pipeline: inputs:
- name: head
tag: head.kube.*
path: /var/log/containers/*.log lines: 30 multiline.parser: docker, cri
outputs:
- name: stdout match: '*' format: json_lines json_date_format: java_sql_timestampWith these two input plugins, we now have the flexibility to collect the telemetry data from large and dynamic sets of files. Our final plugin for developers covered in the next section gives us the ability to run almost anything while capturing its telemetry data.
3. Exec input plugin
The final input plugin to be mentioned in our top three listing is the exec input plugin. This powerful plugin gives us the ability to execute any command and process the telemetry data output into our pipeline. Below is a simple example of the configuration executing a shell command and that command is the input for our Fluent Bit pipeline to process:
service: flush: 1
log_level: info
hot_reload: on
pipeline:
inputs:
- name: exec
tag: exec_demo
command: 'for s in $(seq 1 10); do echo "The count is: $s"; done;'
oneshot: true
exit_after_oneshot: true
outputs:
- name: stdout
match: '*'Now let's run this and see the output as follows:
# For source installation.$ fluent-bit --config fluent-bit.yaml# For container installation after building new image with your# configuration using a Buildfile as follows:## FROM ghcr.io/fluent/fluent-bit:4.0.8# COPY ./fluent-bit.yaml /fluent-bit/etc/fluent-bit.yaml# CMD [ "fluent-bit", "-c", "/fluent-bit/etc/fluent-bit.yaml" ]#$ podman build -t fb -f Buildfile$ podman run --rm fb
...[0] exec_demo: [[1757023157.090932000, {}], {"exec"=>"The count is: 1"}]
[1] exec_demo: [[1757023157.090968000, {}], {"exec"=>"The count is: 2"}]
[2] exec_demo: [[1757023157.090974000, {}], {"exec"=>"The count is: 3"}]
[3] exec_demo: [[1757023157.090978000, {}], {"exec"=>"The count is: 4"}]
[4] exec_demo: [[1757023157.090982000, {}], {"exec"=>"The count is: 5"}]
[5] exec_demo: [[1757023157.090986000, {}], {"exec"=>"The count is: 6"}]
[6] exec_demo: [[1757023157.090990000, {}], {"exec"=>"The count is: 7"}]
[7] exec_demo: [[1757023157.090993000, {}], {"exec"=>"The count is: 8"}]
[8] exec_demo: [[1757023157.090997000, {}], {"exec"=>"The count is: 9"}]
[9] exec_demo: [[1757023157.091001000, {}], {"exec"=>"The count is: 10"}]
[2025/09/04 14:59:18] [ info] [engine] service has stopped (0 pending tasks)
[2025/09/04 14:59:18] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2025/09/04 14:59:18] [ info] [output:stdout:stdout.0] thread worker #0 stopped...
Another developer example is while testing our Java service metrics instrumentation, we'd like to capture the exposed metrics as input to our telemetry pipeline to verify in once location that all is well. With the Java service running, the following exec input plugin configuration will do just that:
service: flush: 1
log_level: info
http_server: on
http_listen: 0.0.0.0
http_port: 2020
hot_reload: on
pipeline:
inputs:
- name: exec
tag: exec_metrics_demo
command: 'curl curl http://localhost:7777/metrics'
oneshot: true
exit_after_oneshot: true
propagate_exit_code: true
outputs:
- name: stdout
match: '*'When we run this configuration we see that the online exposed metrics URL is dumped just one time (for this example, but you can remove the oneshot part of the configuration once verified to work) to the telemetry pipeline for processing:
# For source installation.$ fluent-bit --config fluent-bit.yaml# For container installation after building new image with your# configuration using a Buildfile as follows:## FROM ghcr.io/fluent/fluent-bit:4.0.8# COPY ./fluent-bit.yaml /fluent-bit/etc/fluent-bit.yaml# CMD [ "fluent-bit", "-c", "/fluent-bit/etc/fluent-bit.yaml" ]#$ podman build -t fb -f Buildfile$ podman run --rm fb...
[0] exec_demo: [[1757023682.745158000, {}], {"exec"=>"# HELP java_app_c_total example counter"}]
[1] exec_demo: [[1757023682.745209000, {}], {"exec"=>"# TYPE java_app_c_total counter"}]
[2] exec_demo: [[1757023682.745215000, {}], {"exec"=>"java_app_c_total{status="error"} 29.0"}]
[3] exec_demo: [[1757023682.745219000, {}], {"exec"=>"java_app_c_total{status="ok"} 58.0"}]
[4] exec_demo: [[1757023682.745223000, {}], {"exec"=>"# HELP java_app_g_seconds is a gauge metric"}]
[5] exec_demo: [[1757023682.745227000, {}], {"exec"=>"# TYPE java_app_g_seconds gauge"}]
[6] exec_demo: [[1757023682.745230000, {}], {"exec"=>"java_app_g_seconds{value="value"} -0.04967858477018794"}]
[7] exec_demo: [[1757023682.745234000, {}], {"exec"=>"# HELP java_app_h_seconds is a histogram metric"}]
[8] exec_demo: [[1757023682.745238000, {}], {"exec"=>"# TYPE java_app_h_seconds histogram"}]
[9] exec_demo: [[1757023682.745242000, {}], {"exec"=>"java_app_h_seconds_bucket{method="GET",path="/",status_code="200",le="0.005"} 0"}]
[10] exec_demo: [[1757023682.745246000, {}], {"exec"=>"java_app_h_seconds_bucket{method="GET",path="/",status_code="200",le="0.01"} 1"}]
[11] exec_demo: [[1757023682.745250000, {}], {"exec"=>"java_app_h_seconds_bucket{method="GET",path="/",status_code="200",le="0.025"} 1"}]
[12] exec_demo: [[1757023682.745253000, {}], {"exec"=>"java_app_h_seconds_bucket{method="GET",path="/",status_code="200",le="0.05"} 1"}]
[13] exec_demo: [[1757023682.745257000, {}], {"exec"=>"java_app_h_seconds_bucket{method="GET",path="/",status_code="200",le="0.1"} 1"}]
[14] exec_demo: [[1757023682.745261000, {}], {"exec"=>"java_app_h_seconds_bucket{method="GET",path="/",status_code="200",le="0.25"} 1"}]
[15] exec_demo: [[1757023682.745265000, {}], {"exec"=>"java_app_h_seconds_bucket{method="GET",path="/",status_code="200",le="0.5"} 1"}]
[16] exec_demo: [[1757023682.745269000, {}], {"exec"=>"java_app_h_seconds_bucket{method="GET",path="/",status_code="200",le="1.0"} 1"}]
[17] exec_demo: [[1757023682.745273000, {}], {"exec"=>"java_app_h_seconds_bucket{method="GET",path="/",status_code="200",le="2.5"} 3"}]
[18] exec_demo: [[1757023682.745277000, {}], {"exec"=>"java_app_h_seconds_bucket{method="GET",path="/",status_code="200",le="5.0"} 5"}]
[19] exec_demo: [[1757023682.745280000, {}], {"exec"=>"java_app_h_seconds_bucket{method="GET",path="/",status_code="200",le="10.0"} 10"}]
[20] exec_demo: [[1757023682.745284000, {}], {"exec"=>"java_app_h_seconds_bucket{method="GET",path="/",status_code="200",le="+Inf"} 29"}]
[21] exec_demo: [[1757023682.745288000, {}], {"exec"=>"java_app_h_seconds_count{method="GET",path="/",status_code="200"} 29"}]
[22] exec_demo: [[1757023682.745292000, {}], {"exec"=>"java_app_h_seconds_sum{method="GET",path="/",status_code="200"} 407.937109325"}]
[23] exec_demo: [[1757023682.745296000, {}], {"exec"=>"# HELP java_app_s_seconds is summary metric (request latency in seconds)"}]
[24] exec_demo: [[1757023682.745299000, {}], {"exec"=>"# TYPE java_app_s_seconds summary"}]
[25] exec_demo: [[1757023682.745303000, {}], {"exec"=>"java_app_s_seconds{status="ok",quantile="0.5"} 2.3566848312406656"}]
[26] exec_demo: [[1757023682.745307000, {}], {"exec"=>"java_app_s_seconds{status="ok",quantile="0.95"} 4.522227972308204"}]
[27] exec_demo: [[1757023682.745311000, {}], {"exec"=>"java_app_s_seconds{status="ok",quantile="0.99"} 4.781636377835897"}]
[28] exec_demo: [[1757023682.745315000, {}], {"exec"=>"java_app_s_seconds_count{status="ok"} 29"}]
[29] exec_demo: [[1757023682.745318000, {}], {"exec"=>"java_app_s_seconds_sum{status="ok"} 70.511430859743"}]...This has ingested our external Java service metrics instrumentation into our telemetry pipeline for automation and processing during our development testing.
This wraps up a few handy tips and tricks for developers getting started with Fluent Bit input plugins. The ability to set up and leverage these top input plugins is a big help in speeding up your inner development loop experience.
More in the series
In this article you learned a few handy tricks for using Fluent Bit service section in the configuration to improve the inner developer loop experience. This article is based on this online free workshop.
There will be more in this series as you continue to learn how to configure, run, manage, and master the use of Fluent Bit in the wild. Next up, exploring some of the more interesting Fluent Bit output plugins for developers.
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.