Skip to main content

Bloblang

Introduction​

Bloblang is a powerful, concise, and expressive language designed for data transformation, filtering, and mapping, primarily used within the Benthos stream processor. It enables users to manipulate, enrich, and restructure data as it flows through pipelines, making it an essential tool for modern data engineering and integration tasks.

Definition​

Bloblang is a domain-specific language (DSL) created to simplify the process of transforming structured data formats such as JSON, YAML, and XML. It provides a simple syntax for extracting, modifying, and constructing data objects, allowing users to write complex transformations in a readable and maintainable way.

Usage​

Bloblang is most commonly used within Benthos configuration files to define how incoming data should be processed. However, it can also be used as a standalone tool for ad-hoc data manipulation. Typical usage involves:

  • Mapping fields from one structure to another
  • Filtering out unwanted data
  • Enriching messages with new fields or values
  • Performing conditional logic and type conversions

A simple example of Bloblang usage:

root = {
  "user_id": this.id,
  "email": this.contact.email,
  "is_active": this.status == "active"
}

Use Cases​

  • Data Integration: Transforming data between different formats or schemas when integrating multiple systems.
  • ETL Pipelines: Cleaning, filtering, and enriching data as part of Extract, Transform, Load (ETL) processes.
  • Event Processing: Modifying event payloads in real-time for analytics or monitoring.
  • API Mediation: Restructuring API responses to match client expectations or internal standards.
  • Log Processing: Parsing and reformatting logs for storage, analysis, or alerting.