OpenTelemetry with Fluent Bit

Fluent Bit is a vendor-agnostic Telemetry agent with full support for OpenTelemetry Protocol (OTLP) and overall processing capabilities.

OpenTelemetry-native data plane

Fluent Bit is built for modern OpenTelemetry pipelines. It ingests and emits OTLP natively, processes telemetry efficiently at the edge or in-cluster, and routes data across hybrid environments without the operational weight of a larger collector deployment.

Use Fluent Bit when you need a compact, high-performance telemetry data plane for logs, metrics, and traces.

OpenTelemetry


OpenTelemetry provides an open-source, vendor-neutral standard for Logs, Metrics, and Traces. It defines an open source schema for these telemetry data types, collectively known as signals, ensuring consistency and compatibility across diverse implementations. Because of its vendor-neutral approach, OpenTelemetry has been widely adopted and supported by numerous platforms and tools in the observability ecosystem. Fluent Bit is one such implementation, offering a lightweight and efficient solution for processing and forwarding telemetry data in line with OpenTelemetry standards.

Each telemetry type in OpenTelemetry has a specific schema that defines the structure and format of the data. This schema ensures that telemetry data is consistent and interoperable across different systems, enabling seamless integration:

/images/opentelemetry/otel_schema.png

While schemas ensure consistency and interoperability of telemetry data, the OTLP (OpenTelemetry Protocol) handles the transport.

OTLP - Transport Protocol

A key component of OpenTelemetry is OTLP (OpenTelemetry Protocol), a gRPC-based transport protocol designed for the structured transmission and reception of telemetry data like logs, metrics, and traces. OTLP ensures data consistency and interoperability across different systems:

/images/opentelemetry/otel_otlp.png


Instrumentation Libraries

OpenTelemetry offers instrumentation libraries for a wide range of programming languages and frameworks. These libraries make it easy for developers to instrument their applications, collecting telemetry data in the OpenTelemetry format and seamlessly sending it to the desired backend using the OTLP protocol:

/images/opentelemetry/otel_sdk.png

While instrumentation libraries are essential for collecting telemetry data within applications, a telemetry agent is needed to process and forward this data to the desired backend: Fluent Bit.

Fluent Bit

In the OpenTelemetry ecosystem Fluent Bit is a high-performance, vendor-agnostic telemetry agent that fully supports the OpenTelemetry Protocol (OTLP). As an alternative to the OpenTelemetry Collector, Fluent Bit offers robust processing capabilities, efficient resource utilization, and seamless integration with various backends.

/images/opentelemetry/otel_fluentbit.png


Why Choose Fluent Bit for OpenTelemetry Collector ?


  • High Performance: Designed for high throughput and low resource usage, Fluent Bit can handle large volumes of data with minimal overhead.

  • Extensive Plugin Ecosystem: With a rich set of input, processors and output plugins, Fluent Bit can easily integrate with diverse data sources and destinations.

  • Flexible and Scalable: Fluent Bit is lightweight and can be deployed in various environments, from edge devices to cloud infrastructures.


Getting Started with Fluent Bit and OpenTelemetry

To get started with Fluent Bit and OpenTelemetry, follow these steps:

  1. Install Fluent Bit: Follow the installation guide for your platform.

  2. Configure OpenTelemetry: Set up Fluent Bit to use the OpenTelemetry Protocol by configuring the appropriate input (source) and output (destination) plugins.

  3. Run Fluent Bit: Start Fluent Bit and verify that it’s collecting and forwarding telemetry data as expected.

For detailed configuration instructions, visit the Fluent Bit documentation.

Community and Support

Join our community to stay updated, share your experiences, and get support from other Fluent Bit users:

  • CNCF Slack: Join the CNCF Slack workspace and connect in the #fluent-bit channel.
  • Twitter: Follow us for the latest news and announcements.
  • Newsletter: Sign up for updates on product information, events, and contributions.