Skip to main content
Version: v2.0

Kafka event

A Kafka event is specified as {datasourceName}.{topic_name}.{group_id} in the Kafka event specification.

Within the Kafka event structure, the content of the message is captured and made accessible as inputs.body, facilitating its integration into the handler workflow for processing.

Example spec for Kafka event​

 # event for consume data from Topic
Kafka.publish-producer1.kafka_proj: // event key
  id: kafka_consumer
  fn: kafka_consume
  body: #same body structure for all the events

Example workflow consuming a Kafka event​

 # function for consume data
id: kafka_consumer
summary: consumer
tasks:
  - id: set_consume
    fn: com.gs.return
    args: <% inputs.body %>

On validation error handler​

summary: Handle json scehma validation error
id: error_handler
tasks:
  - id: error_step1
    fn: datasource.kafka.publish
    args:
      data: # publish the event and validation error to kafka on a topic
        value:
          event: <% inputs.event %>
          validation_error: <% inputs.validation_error %>