Skip to content
Docs
Event-driven
Security

Security in EDA: who can read what (and who should not) 🔐

Security in event-driven systems is one of those topics we “add later” - until a GDPR review, a security audit, or worse, an incident arrives. This guide covers the choices to make before, not after.

The EDA threat model 🎯

In an event-driven system, the main attack surfaces are:

  • unauthorized broker access: an unauthorized consumer reads events from other teams or domains;
  • unauthorized publishing: a producer publishes fake events or writes to other teams’ topics;
  • exposed payloads: events containing plaintext PII in logs, storage, or monitoring systems;
  • tampering: modification of events in transit or at rest;
  • excessive data retention: events with PII retained far longer than necessary.

Broker authentication 🔑

The broker must know who is talking to it before deciding what it can do.

mTLS (Mutual TLS)

Both the client (producer or consumer) and the broker authenticate each other with X.509 certificates.

  • Pros: strong, mutual authentication without application credentials.
  • Cons: PKI management (certificate rotation and revocation) is operationally expensive.
  • When: on-premises or multi-cloud environments with PKI already in use.

SASL / SCRAM

Username and password authentication with challenge-response. Kafka supports SASL/PLAIN, SASL/SCRAM-SHA-256, and SASL/SCRAM-SHA-512.

  • Pros: easier to manage than mTLS.
  • Cons: credentials must be explicitly protected and rotated.
  • When: less critical environments or teams that do not want to manage PKI.

OAuth 2.0 / OIDC

Clients obtain a JWT token from an identity provider (Keycloak, Okta, AWS IAM Identity Center) and use it to authenticate with the broker. Kafka supports SASL/OAUTHBEARER.

  • Pros: integration with centralized enterprise IAM and automatic token expiry.
  • When: enterprise environments with an identity provider already in use.

Cloud-native IAM

On AWS MSK, Kinesis, SNS/SQS, or Azure Service Bus, the broker uses the cloud provider’s native IAM policies. The client assumes a role with the permissions it needs.

  • Pros: no credentials to manage, native audit trail, and automatic rotation.
  • When: cloud-only environments with an IAM-first architecture.

Authorization: who can do what on which topics 🛡️ 

    flowchart LR
  C[Consumer A] -->|ACL: CONSUME ok| T1[(Topic: orders)]
  C2[Consumer B] -->|ACL: DENIED| T1
  P[Producer X] -->|ACL: PRODUCE ok| T1
  P2[Producer Y] -->|ACL: DENIED| T1

Kafka ACLs

Kafka supports ACLs (Access Control Lists) to define which principals (users or clients) can perform which operations on which resources (topics, consumer groups, or clusters).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# allow orders-consumer to read from the orders-created topic
kafka-acls.sh --add \
  --allow-principal User:orders-consumer \
  --operation Read \
  --topic orders-created

# allow orders-producer to write to the topic
kafka-acls.sh --add \
  --allow-principal User:orders-producer \
  --operation Write \
  --topic orders-created

The principle of least privilege

Each service should access only the topics it uses and perform only the operations it needs. A consumer should not have production rights; a producer should not be able to perform administrative operations.

Practical advice: manage ACLs through IaC (Terraform or Ansible), assign them by service name rather than person, and review them during every security review.

Encryption in transit and at rest 🔒

In transit

TLS is mandatory outside fully private networks. Configure:

  • TLS on broker listeners exposed to external clients;
  • automatic certificate rotation (Let’s Encrypt, Vault, or cert-manager);
  • TLS 1.2 as a minimum, preferably TLS 1.3.

At rest

It depends on the broker and environment:

  • Cloud Kafka (MSK or Confluent Cloud): encryption at rest is managed by the cloud provider (AES-256) and enabled by default.
  • Self-hosted Kafka: it depends on the cluster’s filesystem or storage configuration and must be configured explicitly.
  • Highly sensitive data: consider application-level payload encryption before publishing (envelope encryption with KMS), so data remains encrypted even for people who can access the broker.

PII in events: the practical problem 🧬

Events tend to “accumulate” sensitive data over time. A UserRegistered event containing an email address, name, and date of birth will be retained in the Kafka log for months or years. This creates non-trivial GDPR issues.

Strategies for managing PII in events

1. Payload minimization

Publish only what is strictly necessary. If the consumer needs the email address, it should probably call the profile service instead of receiving it in the default event.

2. Tokenization / pseudonymization

Replace the PII value with an opaque token (UUID) that can be resolved only by someone with access to the lookup service.

1
2
3
4

{
  "order_id": "ord-123",
  "customer_ref": "tok-7f3a9c"
}

The customer_ref field is a token, not direct PII. Only authorized services can resolve the token to the real data.

3. Field-level encryption (crypto-shredding)

Encrypt PII fields in the payload with a per-user key managed by KMS. When the user requests deletion (the GDPR right to erasure), delete the key in KMS: encrypted data remains in the log but becomes permanently unreadable.

    flowchart LR
  EV[Event with PII] -->|encrypt field with user key| ENC[Event with encrypted field]
  ENC --> KAFKA[(Kafka / Event Store)]
  KMS[(KMS)] -->|key per user_id| ENC
  KMS -->|delete key| DEL[Right to erasure - GDPR]

This is the only practical way to satisfy the right to erasure in an Event Sourcing system, where events are immutable by design.

4. Separating data tiers

Distinguish notification events (minimal payload, no PII) from data stores (with PII, a dedicated retention policy, and access control). Not every event must carry everything; it is often a design choice, not a technical constraint.

GDPR and event retention ⚖️

Kafka is not designed for selectively deleting messages. Practical compliance strategies include:

  • Retention policy: configure retention.ms or retention.bytes on topics with PII (for example, 30 days instead of “forever”).
  • Log compaction + tombstone: for compacted topics, a delete tombstone (a message with a null value for a key) removes the record with that key; useful for data keyed by user.
  • Crypto-shredding (see above): delete the KMS key instead of the physical data.

None of these comes for free: plan retention during event design, not as an afterthought.

Audit trail 📋

In EDA, the audit trail is often a natural side effect of Event Sourcing. Even without ES, explicitly logging the following is worthwhile:

  • who produces an event (the producer identity from authentication);
  • when and from where (timestamp plus service mesh identity or source IP);
  • what changed (for update events, at least the sensitive fields involved).

Tools:

  • AWS CloudTrail / CloudWatch for AWS-managed brokers;
  • Kafka audit log for Confluent Platform Enterprise;
  • OpenTelemetry for distributed tracing with identity propagated in the trace context.

EDA security checklist ✅

  • Authentication: every client has its own identity (no credentials shared between services).
  • Authorization: ACLs or IAM policies per topic, following the principle of least privilege.
  • Encryption in transit: TLS enabled on every exposed listener, with automatic rotation.
  • Encryption at rest: verified for the broker’s storage tier.
  • PII inventory: every event containing PII identified during design.
  • Retention policy: configured for every topic containing PII before go-live.
  • Right-to-erasure strategy: crypto-shredding or tokenization defined for GDPR-sensitive data.
  • Audit trail: who publishes what is traced (identity, timestamp, and topic).
  • Security review: event schemas reviewed for data minimization before deployment.

Next steps 🚀

Last updated on
TestingTools and ecosystem