Controlling load instead of bending systems: Rate limiting with Apache Pulsar

Why is my ERP suddenly becoming a bottleneck?

In integration projects with Apache Pulsar, sooner or later an uncomfortable question arises:

Why is a single external system enough to bring an otherwise stable ERP to its knees?

The basic scenario is usually simple. An ERP system consumes a topic through which operational transactions are triggered. In normal operation, these messages come from internal workflows - controlled, predictable, well-sized.

Problems only arise when external systems use the same topic, for example for:

• Correction postings
• Bulk imports
• subsequent data cleansing

The ERP itself often does not have a strong backlog or prioritization concept. The result: As soon as an external load arrives without braking, the system is overwhelmed.

Why an ERP ticket misses the real problem

The classic solution is quickly outlined:

• Open a ticket with the ERP manufacturer
• Describe the load scenario
• hope for improvement

Even if this were successful, structural questions remain unanswered:

• Are external systems allowed to intervene unfiltered in internal process chains?
• Can we even distinguish between internal and external use?
• Do we have control and observation options?

Change of perspective: Integration as an infrastructure issue

Apache Pulsar is more than a transport mechanism. Through namespaces, ACLs and Pulsar Functions, Pulsar can be used as an active data hub.

The central idea of ​​this article is:

If load is problematic, it needs to be checked before the ERP - not in the ERP.

Test setup: Make the problem reproducible

This section describes the complete test setup so that the scenario is reproducible 1:1. No assumptions are deliberately made about existing infrastructure.

1. Start Pulsar locally

A single local Pulsar broker in standalone mode is used for testing. This means that Broker, BookKeeper and ZooKeeper are bundled in one process.

podman run -it --rm \
  -p 6650:6650 \
  -p 8080:8080 \
  docker.io/apachepulsar/pulsar:4.0.8 \
  bin/pulsar standalone

• Port 6650: Pulsar Binary Protocol (Clients)
• Port 8080: Admin API, Functions Worker

2. Create ERP namespace

After starting, we switch to the Pulsar shell:

pulsar-shell

First, the namespace for the ERP is created:

admin namespaces create public/erp

3. Definition of the topic

A topic is used in the test:

• ERP input topic (internal)
  persistent://public/erp/command

Topics do not have to be created explicitly in Pulsar - they are created implicitly the first time they are accessed.

4. Simulation of the ERP consumer

The ERP is simulated by a simple Pulsar CLI consumer that outputs all received messages to STDOUT.

export PULSAR_URL=pulsar://127.0.0.1:6650

pulsar-cli consumer \
  -t persistent://public/erp/command \
  -s erp

• Subscription name: erp
• Subscription type: default (Shared)

Every message received is output immediately.

5. Internal producer (normal operation)

The internal producer simulates regular ERP workflows with low, constant load.

seq 1 10000000 | while read i; do
  echo "erp-client-message-$i" \
    | pulsar-cli producer \
        -t persistent://public/erp/command
  sleep 5
done

Characteristics:

• one message every 5 seconds
• constant load
• no burst phases

The consumer receives accordingly:

erp-client-message-1
erp-client-message-2
erp-client-message-3

6. External producer (incident)

The external producer simulates a third-party system that generates messages unhindered.

seq 1 10000000 | while read i; do
  echo "external-client-message-$i" \
    | pulsar-cli producer \
        -t persistent://public/erp/command
done

Characteristics:

• no breaks
• maximum sending speed
• no consideration for the consumer

The result for the ERP consumer:

external-client-message-433
erp-client-message-9
external-client-message-434
external-client-message-435
external-client-message-436
external-client-message-437

Internal messages are almost lost in the flow of external load.

Dispatch rate as a control experiment

For comparison, a dispatch limit is set directly on the ERP topic:

admin topics set-dispatch-rate \
  persistent://public/erp/command \
  --dispatch-rate-period=1 \
  --msg-dispatch-rate=1

The consumer now only receives one message per second - regardless of the source.

The limit is then removed again:

admin topics set-dispatch-rate \
  persistent://public/erp/command \
  --dispatch-rate-period=1 \
  --msg-dispatch-rate=-1

Architectural decision: separation of input channels

Basic idea: External producers are controlled by Pulsar, not by the ERP

External systems publish their messages in their own Pulsar topic (persistent://public/external/command) and not directly into the ERP topic. This topic is consumed exclusively by a Pulsar Function, who is the only authorized producer to write to the internal ERP topic. Rate Limiting is applied on the Dispatch of the Function and thus limits exactly the message rate, with which external events reach the ERP.

A dedicated namespace for the external system

We introduce a second namespace:

admin namespaces create public/external

This creates two explicit channels:

Routing with a Pulsar Function

External messages should continue to reach the ERP - but in a controlled manner. To do this, we use a pass-through pulsar function that forwards messages unchanged.

public final class Copy implements Function<byte[], byte[]> {
    @Override
    public byte[] process(byte[] input, Context context) {
        if (input == null || input.length == 0) {
            return null;
        }
        return input;
    }
}

Deployment:

admin functions create \
  --tenant public \
  --namespace external \
  --name route.copy \
  --jar /abs/path/route-copy.jar \
  --classname net.d2ux.stream.generic.route.Copy \
  --inputs persistent://public/external/command \
  --output persistent://public/erp/command

Rate limiting where it makes sense

Now we set the limit exclusively on the external topic:

admin topics set-dispatch-rate \
  persistent://public/external/command \
  --dispatch-rate-period=5 \
  --msg-dispatch-rate=1

Result:

• internal messages run unchecked
• external load is metered
• the ERP remains stable

external-client-message-433
erp-client-message-9
external-client-message-434
erp-client-message-10
external-client-message-435
erp-client-message-11
external-client-message-436
erp-client-message-12
external-client-message-437

Additional benefit: observability

Pulsar Functions provide metrics:

pulsar_function_received_total{tenant="public",namespace="external",name="route.copy"} 2919

Conclusion

The approach shown illustrates the added value that an architectural change of perspective can offer. Instead of addressing overload situations exclusively at the application level, the integration layer itself becomes the control location.

Apache Pulsar makes it possible to explicitly structure data streams, load behavior and access paths.
By separating internal and external channels and targeted rate limiting, external influences can be controlled without changing existing core systems.

The focus of this article was deliberately on the load and runtime behavior of the described architecture. However, in productive environments, the mechanisms presented must be supplemented by appropriate security measures. In particular, this includes clearly defined ACLs at the namespace level to specify which systems are allowed to publish or consume topics.

Only the interaction of structure, governance and security turns a technical solution into a viable integration architecture.

Image credit: The Pulsar logo is a registered trademark of the Apache Software Foundation. Use in editorial reporting in accordance with official guidelines.