Migrating from NATS to Redpanda: Evolve from messaging to streaming

For many engineering teams, NATS is where the journey into real-time messaging began. Its simplicity, speed, and light footprint make it good for messaging between microservices or building reactive systems. With JetStream, NATS extended its capabilities to include persistence and message replay, helping teams go beyond transient pub/sub.

But as organizations scale and real-time data becomes more central to business strategy, those teams often hit a wall. Turns out that what once worked perfectly as a messaging layer is now struggling as a data backbone.

Enter Redpanda, an Apache Kafka^® API-compatible streaming platform. It’s designed to deliver messages while also serving as a durable, high-performance event log that underpins event-driven applications, data pipelines, and business-critical workflows.

So, as event streams become the lifeblood of modern data platforms, organizations are evolving from simple messaging to streaming. To help you with this shift, this post covers the differences between messaging and streaming, outlines a few considerations for migrating to Redpanda Streaming, and explains how Redpanda Connect significantly simplifies the process.

Messaging vs. Streaming

NATS and JetStream effectively solve the messaging problem, delivering data quickly and reliably between services. But modern architectures increasingly require more:

• Durability: Events must be stored and reprocessed long after they’re produced
• Replay and auditability: Historical events are often as important as real-time ones
• Stream processing: Real-time transformations and aggregations are business-critical
• Data platform integration: Event streams must feed data warehouses, ML pipelines, and SaaS platforms.

These requirements enable real-time personalization, predictive analytics, and data-driven decisions. They need a system that treats events as a source of truth, rather than transient messages.

Why Redpanda 

To make it easier, here’s a comparison table between NATS Core, NATS Jetstream, and Redpanda.

Moving from NATS to Redpanda doesn’t just benefit speed or throughput, but also how your infrastructure aligns with better business outcomes.

• Faster product iteration: Replayable event logs let teams test new ML models, analytics pipelines, or downstream services without changing upstream producers
• Improved customer experience: Streaming unlocks personalization, fraud detection, and anomaly detection in real time
• Lower operational overhead: A unified streaming backbone reduces the number of bespoke services teams must build and maintain
• Ecosystem connectivity: Easily connecting different source and sink systems with declarative pipelines 

The comparison makes one thing clear: NATS is good for low-latency, ephemeral messaging, but Redpanda’s capabilities position it as a strategic data platform, one that can scale with your business needs. In short, the move evolves from using a tactical tool to a strategic platform.

A structured approach to migration

Moving from NATS/JetStream to Redpanda is an opportunity to re-architect around event streaming principles. Here’s the framework we recommend:

1. Audit and classify your existing workloads

Assess and begin with a clear inventory, and make sure to identify all the components involved.

• Subjects and streams
• Message payload schemas
• Producers and consumers
• Retention and replay needs

Classify them into categories like transient messaging, durable event streams, and analytics-driven pipelines. Durable event streams and analytics-driven pipelines are strong candidates for Redpanda.

2. Redesign for topics and partitions

Map NATS subjects or JetStream streams to Redpanda topics. Use partitioning strategies to guarantee ordering and scale consumption. Designing your topics carefully now prevents bottlenecks later.

3. Migrate from NATS to Redpanda using Redpanda Connect

Redpanda Connect is a data connectivity framework that makes it easy to integrate different systems. It puts an extensive library of pre-built connectors, processors, and sinks at your fingertips, and is designed to be a more flexible and resilient alternative to traditional data integration frameworks.

Below is an example of this pipeline. This assumes you have NATS, a Redpanda cluster, and Redpanda Connect up and running. In this example, these are running on a local laptop.

Here’s the pipeline to migrate messages NATS Core to the Redpanda cluster:

input:
  nats:
    urls: [ "nats://127.0.0.1:4222" ]
    subject: foo  # This will match foo subject

output:
   kafka_franz:
    seed_brokers: [ "localhost:19092" ]
    topic: foo-topic
    key: ${! meta("nats_subject") }
    tls:
      enabled: false
    sasl: []

Now here’s the pipeline to migrate messages NATS Jetstream to the Redpanda cluster:

input:
  nats_jetstream:
    urls: [ "nats://127.0.0.1:4222" ]
    subject: foo  # This will match foo subject

output:
    kafka_franz:
    seed_brokers: [ "localhost:19092" ]
    topic: foo-topic
    key: ${! meta("nats_subject") }
    tls:
      enabled: false
    sasl: []

It’s that simple! No complex intertwined code.  

‍Note: While this pipeline is a good starting point, you’ll want to include error handling via dead letter queues to make your pipeline production-ready.

4.  Validation checks

Before you migrate consumers and producers to point to Redpanda, it’s good practice to validate that the messages on NATS are equivalent in Redpanda. This ensures that applications and/or pipelines will work exactly as before after migration. 

A few points to consider in validation checks are: message count parity, message content equivalence, ordering and delivery semantics, and end-to-end application behavior.

5. Migrate consumers 

Once events are produced into Redpanda and they’ve been validated, you would have to update downstream services to use Kafka APIs, which are fully compatible with Redpanda. You can read the best practices for consumer offset management. Redpanda has a built-in schema registry that you can use for evolution and validation wherever possible.

6. Final cutover and decommissioning

After validation, redirect producers that have been updated to use Kafka APIs to write to Redpanda and decommission NATS. Our docs have some helpful pointers when configuring producers for Redpanda, so definitely give those a read.

Future-proof your data infrastructure

The bottom line is: for lightweight messaging, NATS Core and NATS JetStream are still good to use. However, as event streams become the backbone of modern data platforms, organizations need something more durable, scalable, and deeply integrated with their analytics, ML, and business intelligence ecosystems.

Redpanda goes beyond this. It enables organizations to transition from message delivery to data streaming, unlocking new capabilities and improving business value.

So, if you’re building for the future, it’s worth exploring how Redpanda can power the next generation of your data infrastructure. Give Redpanda a try, and hop into our Redpanda Community on Slack if you have questions for the team.