Integrating Grafbase and Redpanda for seamless real-time features

Federated GraphQL APIs are becoming the default for modern application backends. They give teams the flexibility to build and evolve services independently, while still exposing a unified API to consumers. But adding real-time features like live dashboards, notifications, and alerts often requires additional infrastructure.

Now, that complexity is gone.

Grafbase just launched native support for Apache Kafka^® through Grafbase Extensions and Redpanda is the ideal data streaming engine to power it. You can now declaratively integrate Kafka topics into your federated GraphQL API as a virtual subgraph. 

No subgraph servers, no schema stitching, and no glue code. It’s all built into the Grafbase platform.

This post shows how to use the new Kafka Extension with Redpanda to publish and subscribe to event streams from your GraphQL API with just a few lines of config. 

Before we get started, lets cover what Grafbase Extensions are.

What are Grafbase Extensions?

Grafbase Extensions let you extend your federated GraphQL API without deploying additional subgraph infrastructure. You can integrate services like databases, authentication providers, and streaming platforms directly into your schema using declarative configuration and GraphQL directives.

Extensions simplify GraphQL Federation adoption and give you a managed developer experience across your stack. Previously released extensions include Postgres, gRPC, NATS, and Snowflake. 

Why Redpanda + Grafbase?

Redpanda is a drop-in replacement for Kafka that’s built for performance, security, and simplicity:

• Kafka API compatible, so it works out of the box with the Kafka Extension.
• Runs in containers, self-hosted, BYOC, or even fully managed with Redpanda Serverless (which we’re using in this tutorial).

With Redpanda behind your Kafka Extension, you can:

• Publish real-time events from your GraphQL mutations.
• Deliver push-based updates to frontend apps using subscriptions.
• Store all messages durably for historical replay or batch export.

How the Grafbase Kafka extension works

The Grafbase Kafka extension supports two primary operations via GraphQL directives:

• Publishing messages to Kafka topics.
• Subscribing to topics for real-time updates.

It abstracts the complexity of Kafka client configuration, connection pooling, and message serialization, so you can focus on your application logic.

Getting started with Redpanda + Grafbase

Redpanda Serverless gives you a fully managed Kafka API endpoint you can connect to in seconds.

Step 1. Get started with Redpanda Serverless

First, create a free Redpanda Serverless account. Then, create a cluster, choose your cloud region.

Select the Kafka API tab. Configure or grab the connection details and credentials. You’ll use these in your grafbase.toml.

Step 2. Configure Grafbase Kafka extension

Next, make sure you have the Grafbase CLI installed

curl -fsSL https://grafbase.com/downloads/cli | bash

Then add the Kafka extension to your grafbase.toml

[extensions.kafka]
version = "0.1"
[[extensions.kafka.config.endpoint]]
bootstrap_servers = ["<your-redpanda-url>"]
[subgraphs.kafka]
schema_path = "subgraph.graphql"

If you’re using Redpanda Serverless, you can use SASL authentication and TLS

[extensions.kafka.config.endpoint.authentication]
type = "sasl_scram"
username = "your-user"
password = "your-password"
mechanism = "sha512"
[extensions.kafka.config.endpoint.tls]
system_ca = true

Now define your GraphQL schema in

extend schema
  @link(
    url: "https://grafbase.com/extensions/kafka/0.1"
    import: ["@kafkaProducer", "@kafkaPublish", "@kafkaSubscription"]
  )
  @kafkaProducer(
    name: "eventProducer"
    topic: "user-events"
  )
type Mutation {
  publishUserEvent(userId: String!, input: UserEventInput!): Boolean!
    @kafkaPublish(producer: "eventProducer", key: "user-{{args.userId}}")
}
type Subscription {
  userEvents(userId: String!): UserEvent!
    @kafkaSubscription(
      topic: "user-events"
      keyFilter: "user-{{args.userId}}"
    )
}
input UserEventInput {
  action: String!
  metadata: JSON
}
type UserEvent {
  action: String!
  metadata: JSON
  timestamp: String!
}

Step 3. Start and test 

Start your development server to test

grafbase dev

After testing locally, publish your schema as a virtual subgraph

grafbase login
grafbase publish \
  --name kafka \
  my-org/my-graph@branch \
  -m "add kafka integration" \
  --virtual

Start the gateway

export GRAFBASE_ACCESS_TOKEN=<your_access_token>
grafbase-gateway --graph my-graph@branch --config grafbase.toml

The gateway handles all connections internally. Your virtual subgraph doesn't need a URL because the extension manages all communication with your cluster.

Using GraphQL subscriptions for real-time notifications

The @kafkaSubscription directive is designed for push-based updates in frontend applications. Think of it like a notification stream. It’s great for dashboards, alerts, or live status updates.

It’s not meant to replace a full Kafka consumer. There’s no offset tracking, no consumer group coordination, and no delivery guarantees. Each subscription opens a lightweight, ephemeral consumer.

Use it where real-time UX matters and occasional message loss is acceptable.

Example use case perfect for GraphQL subscriptions:

type Subscription {
  # Real-time order status updates
  myOrderUpdates(customerId: String!): OrderUpdate!
    @kafkaSubscription(
      topic: "order-updates"
      keyFilter: "customer-{{args.customerId}}"
    )
  # Threshold-based alerts
  highValueTransactions(threshold: Float!): Transaction!
    @kafkaSubscription(
      topic: "transactions"
      selection: "select(.amount > {{args.threshold}})"
    )
}

Grafbase has a complete example in their GitHub repository.

Try Redpanda and Grafbase

With Grafbase and Redpanda, you can add Kafka-powered features to your API in minutes and no extra subgraph infrastructure is required.

Whether you’re building live dashboards, notifications, or streaming pipelines, Redpanda gives you the speed and reliability of Kafka with none of the operational burden. If you’re ready to keep exploring this powerful pair, check out these links:

• Try Redpanda Serverless
• Explore the Kafka Extension on Grafbase

If you have questions about this tutorial or want to chat with Redpanda engineers, hop into the Redpanda Community on Slack.