Outbox Pattern — Save it Reliably

When you start using messaging, you will notice that your application oftentimes updates the database and then needs to publish an event or message to the message broker. The problem is because the message broker and database are two different resources and you are going to need a distributed transaction. Here’s how you can avoid the distributed transaction but still atomically saved it to the database and publish a message.

You have your client and a database wherever you are storing your data that could a relational database or a document store doesn’t really matter but the point is in your code, you are going to be making that statement to save some data or change some data to the database and then a secret step altogether is where you need to publish that message to your message broker to a queue.

The problem is because these are two different operations. What happens if you were able to save to the database but then for whatever reason in code some you had a failure where you couldn’t actually publish the message. This could be because of many different reasons. Let’s say that the queue is unavailable for whatever reason or more likely you have a bug in your code. So you saved it to the database but you are unable to publish the message.

Now, this is a problem because if you have other services for whatever reason that need to know about that event that’s occurred because you made some state change they are not going to know about it.

So you are gonna have stale data or even worse if you have long-running processes where you need an event to occur to kind of kick off the next step it's never gonna occur. This is the problem, this is why you need a distributed transaction so you can atomically say these are all one thing.

To illustrate how the outbox pattern works is that you are ultimately going to be publishing your event and your state change together in the same transaction as your primary database. So, what that means is when you save something to your database is step one.

Step two is you are actually when you are publishing your message and this is how the API works gonna be dependent on the library you are using. They will all implement this in the API slightly differently but ultimately what’s happening is that the message that you publish that you are seemingly publishing to the broker is actually getting saved to the database as a part of the same transaction as your state change.

So then what happens is the library in a separate process or thread will then fetch from the database the message that hasn’t actually been published yet and then it will send that to your message broker or your queue.

Once that succeeds and it was able to do so it goes back to the databasing either deleting records or updating records that again it manages about okay these are the messages that have been published.

What this does allows you to publish a message technically and save your state change in one atomic transaction to the database. So when your publishing your message it’s not immediately to the message broker you are actually saving it to the database inside the same transaction as your normal state change.

So the way cap works are again it saves everything to the same database and you can do this by creating a transaction. It has an extension method for a beginning transaction where you can pass to the publisher.

So really what I’ve added is adding the transaction to the database and telling it to use the publisher to use that transaction instead of it publishing directly to the message broker. It’s not going to publish directly to the message broker, it actually going to save a record to the database.

I’ll show you the dashboard for the cap so you can see the messages publish as well.

On the database side, two tables have actually got created by cap so there's one called Published and Receive. I didn't have to do anything with these. These were just automatically created when I started running the app. Cap kind of deal with it obviously there's a configuration where I had to give it kind of the connection string to where this was all set up.

You can see here the published events. These are things that I've been publishing and it's marking if retries are occurring when it was added when it expires so it can delete it.

So there is one issue with the outbox pattern that we need to address and need to talk about.

When you have this separate process or thread a part of the library that you are using which is fetching those records from the database and then it's publishing those to your actual message broker and then it has to go back and update the database for that record that says it worked or didn't work. If it worked but it couldn't actually update the record in the database then that means that you are actually going to end up republishing that same event to the message queue again because it doesn't look like it actually was published.

This is an actual situation where you are kind of potential if there are failures with the publisher go going back to the database to say it worked. You could be publishing the same message more than once.

I'll handle dealing with duplicate messages in a separate blog post.

Have a nice day!