Skip to content

Protocol Codec Architecture ​

This document describes the internal src/protocol crate. It is an implementation reference for contributors; user-facing protocol behavior belongs in the MQTT, Kafka, NATS, and AMQP documentation.

Responsibilities ​

The crate owns wire-level data types and codecs for:

  • MQTT 3.1, 3.1.1, and 5.0
  • Kafka requests and responses
  • AMQP 0-9-1 frames
  • NATS Core commands and messages
  • the internal Storage Engine protocol

Protocol business behavior remains in the corresponding broker crates. The codec crate does not perform topic routing, authentication decisions, subscription management, or storage operations.

Unified Dispatch ​

RobustMQCodec in src/protocol/src/codec.rs wraps the protocol-specific codecs and exposes one tokio_util::codec::{Decoder, Encoder} interface. Decoded values are returned as RobustMQCodecWrapper variants:

VariantPayload
MQTTMqttPacketWrapper, including the negotiated MQTT version
KAFKAKafkaPacketWrapper
AMQPone AMQPFrame
NATSNatsPacket
StorageEngineStorageEnginePacket

Network acceptors for a known protocol construct the codec with RobustMQCodec::new_with_protocol. MQTT listeners use a codec that starts without a version so the first CONNECT packet can select MQTT 3, 4, or 5. A codec created with no preset protocol does not perform general-purpose detection for Kafka, AMQP, NATS, or Storage Engine traffic.

Protocol Identifiers ​

RobustMQProtocol is an internal routing identifier, not a wire-protocol registry. Its numeric values are used by connection metadata and internal packet wrappers:

ProtocolInternal value
MQTT3 / MQTT4 / MQTT53 / 4 / 5
Kafka0
Storage Engine10
AMQP11
NATS12

from_u8 is intentionally a compatibility fallback: it recognizes MQTT4, MQTT5, and NATS, and maps all other values to MQTT3. It must not be treated as a reversible conversion for every value returned by to_u8.

Framing and Partial Reads ​

All codecs preserve incomplete bytes in the input buffer and return Ok(None) until a complete frame or command is available. AMQP and Kafka use length-bearing binary frames. NATS uses CRLF-terminated control lines followed by a payload and trailing CRLF for PUB, HPUB, MSG, and HMSG. MQTT uses its fixed header and variable-length remaining-length field.

Error Handling ​

Protocol-specific codecs return their own errors when used directly. Through RobustMQCodec, MQTT, Kafka, AMQP, and Storage Engine decode failures are currently treated as no decoded packet in the dispatch path, while NATS decode failures are converted to CommonError and propagated so the connection can be closed instead of waiting indefinitely for more bytes. Callers that need precise protocol errors should use the protocol-specific codec directly.

Scope Boundary ​

This crate defines wire representations and serialization. The broker crates decide what a decoded packet means and whether it is accepted, rejected, persisted, acknowledged, or delivered. Changes to protocol semantics should update the relevant protocol compatibility documentation and tests in addition to this internal reference.

🎉 既然都登录了 GitHub,不如顺手给我们点个 Star 吧!⭐ 你的支持是我们最大的动力 🚀