Thinking While Building: A Note on RobustMQ Writing

I've written a fair amount lately about RobustMQ's storage layer design. A friend asked: How much code have you written? I had to admit: quite a few articles, but the code is still in progress.

That might seem backward. Usually you build the system first, validate it, then write to share. I went the opposite way: write the thinking first, discuss it, then implement. Why?

RobustMQ Is an Exploration

First, RobustMQ isn't a commercial project—at least not yet. No KPIs, no funding pressure, no hard delivery dates. That lets us explore more calmly: How should the next generation of message platforms be designed?

That exploration is a big part of RobustMQ. I've been thinking: Where are the pain points in message queues? What are the pros and cons of Kafka, Pulsar, RocketMQ? Can we find a better balance between simplicity and flexibility by combining technologies sensibly? Is compute-storage separation inevitable? When is ISR right vs Quorum?

This thinking has value on its own; it doesn't have to wait for implementation. And honestly, many ideas are vague until written down. Writing forces me to organize them, find logical gaps, think more clearly.

Writing Is Thinking

For me, writing isn't just sharing—it's a thinking tool.

When I wrote "Why choose ISR over Quorum," I had to analyze both thoroughly. I felt ISR was better, but why? What exactly is wrong with Quorum? While writing I realized: Quorum's main issue is the complexity of background repair, and that complexity is easy to underestimate at design time.

When I wrote about the value of Active/Sealed Segment separation, I had to spell it out: going from 100K Leaders to 1K—what do we actually save? How much does "Sealed Segments readable from any replica" help load balancing? In writing I saw: this design doesn't just cut management cost; it also simplifies tiered storage because Sealed Segments are immutable.

Much of that deeper thinking happened while writing. If I only think in my head, I easily stay superficial. Putting it on paper clarifies the logic and exposes the problems.

Why Show the Thinking Process

I hope these articles show RobustMQ's thinking process, not just the final conclusions.

These articles aren't the final form; the architecture may keep changing. I initially leaned toward Quorum, then understood the repair complexity and switched to ISR. At first there was no Active/Sealed concept; later we saw Leader count was an issue and added it. These changes come from thinking.

I think showing that process has value. Let people see how a message queue goes from idea to design—what problems came up, how we traded off and chose. That's more honest than only showing the polished result, and it might help others doing similar exploration.

These articles are also an invitation to discuss. My thinking might be off; the design might have flaws. Sharing it lets people point out "you didn't consider this" or "you missed that scenario"—valuable feedback. Working in isolation easily creates blind spots; exposing ideas and accepting criticism can improve the design.

The Rhythm of Writing and Building

RobustMQ's development rhythm: code, think, write, iterate—all in parallel.

One week we implement the I/O Pool and find batch processing works better than expected—write an article. Next week we design indexes and wrestle with sync vs async—write out both options, and in writing decide on sync. The week after we compare ISR and Quorum; while writing we see Quorum's pitfalls and choose ISR.

Code and articles run in parallel and reinforce each other. Problems in implementation push us to rethink design. Organizing ideas in writing guides implementation. That cycle keeps the project evolving.

Not Braggadocio, But Record

These articles aren't saying "look how great our design is." They're more like "here's where we're at right now."

Months from now we might look back and find some ideas naive or some design flawed. That's exploration—not every step is correct; what matters is moving forward through trial and error. Recording the process gives us a trail: what we tried, what we learned, what mistakes we made.

For an open-source project, transparency matters. Let people know what we're thinking, what we're doing, why we chose as we did. Even if the project doesn't succeed, this record might still help future explorers.

The Point of Exploration

RobustMQ is an exploration—of how the next generation of message platforms should be designed.

That exploration may succeed or fail. We may find some designs don't work and have to restart. Implementation may be harder than expected and force compromises. The market may not want such a system at all—the whole thing might be wishful thinking.

But exploration has value. Even if we never build a successful product, we'll have thought these problems through, validated these ideas, and learned along the way. And the record of that thinking might help others on similar paths.

An Invitation

I write these articles to invite others to explore with us.

If you're also thinking about message queue design, if you care about compute-storage separation, consistency protocols, tiered storage—if you have different ideas or see flaws in our design—let's talk. These articles aren't conclusions; they're a starting point. We want to explore this process together.

Foundation software may not be sexy or have AI's imagination. But it matters. It has technical depth. It deserves to be taken seriously. I'm doing this not because it's sexy, but because I think it matters.