Shushu receives events in no particular order and long after the event originated in the client's system. Therefore, it is generally desired to depend on Shushu not caring about the order or time in which it receives events. The following text examines thoughts on how to address such issues.
Shushu provides a resources-group-membership API that takes a resource and a group and defines a relationship between the two objects. The relationship is activated and the de-activated when the resource is moved in and out of groups. Resources and Groups can be activated and de-activated infinitely many times but a resource can only belong to one group at a time. The client who is allowing these movements (i.e. Heroku's Core API) should be not responsible for keeping history of these movements, instead it should rely on Shushu for long term durability.
Table 1.1
Ephemeral Service (Core)
Resource | Group | Time | Update Order
_________+_______+______+_____________
R1 | G1 | 001 | 1
R1 | G2 | 002 | 4
R1 | G3 | 003 | 2
R1 | G2 | 004 | 3
This table represents resource-group relationships as they exist in the real world.
The update order column notes the order in which the ephemeral service sends the events to the durable service. There is no correspondence between the update order and the order in which the events occured in reality. If we were to correlate these events, the Ephemeral Service would have to block until the events were delivered to the Durable Service. This makes our Durable Service a single point of failure.
Table 1.2
Durable Service (Shushu)
Resource | Group |Time | State
__________+_______+_____+______
R1 | G1 |001 | active
R1 | G1 |002 | inactive
R1 | G2 |002 | active
R1 | G2 |003 | inactive
R1 | G3 |003 | active
R1 | G3 |004 | inactive
R1 | G2 |004 | active
This table represents the history that we would like to record with respect to the events described in Table 1.1
Since Shushu reports financial information, it is important for Shushu to detect when it is in an inconsistent state. When Shushu has detected an error state it can neglect reports involving the affected records so that errors are not propagated. With this in mind, consider the following series of updates.
UPDATE R1 G1 001
UPDATE R1 G3 003
These requests result in our server reporting the following state:
Resource | Group | Time | State
_________+_______+______+______
R1 | G1 | 001 | active
R1 | G3 | 003 | active
Also, since a resource can only belong to one group at a time we could represent our data to look like this:
Resource | Group | Time | State
_________+_______+______+______
R1 | G1 | 001 | active
R1 | G1 | 003 | inactive
R1 | G3 | 003 | active
Comparing this data with the data in Table 1.2 reveals that Shushu is now in an inconsistent state. Furthermore, it is unclear as to how Shushu would detect this inconsistency.
Let us replay Example 1. However, we will use the time field to order the events. Also, we will take Example 1 a couple of steps further with respect to Table 1.1.
The client makes the following requests to the server:
UPDATE R1 G1 001
UPDATE R1 G3 003
UPDATE R1 G2 004
UPDATE R1 G2 002
This result in our server reporting the following data:
Resource | Group | Time | State
_________+_______+______+______
R1 | G1 | 001 | active
R1 | G3 | 003 | active
R1 | G2 | 004 | active
R1 | G2 | 002 | active
We could then sort our table by time and get the following result:
Resource | Group | Time | State
_________+_______+______+______
R1 | G1 | 001 | active
R1 | G2 | 002 | active
R1 | G3 | 003 | active
R1 | G2 | 004 | active
Even though this approach resulted in a correct data model, we suffered from being in a state of inconsistency until we received the last event. Also, we had no way to detect this inconsistency.
In the last two examples, we suffered from having our system in an inconsistent state and without any way to detect such an inconsistency. In this example, we will decompose our update requests to mitigate such a problem.
UPDATE R1 G1 001 state=active
UPDATE R1 G2 003 state=inactive
UPDATE R1 G3 003 state=active
Resource | Group | Time | State
_________+_______+______+______
R1 | G1 | 001 | active
R1 | G2 | 003 | inactive
R1 | G3 | 003 | active
We are now in an inconsistent state since we are inactivating a record that was never activated. Note --this might seem unclear as we are basing our validation off of the data model and not meta-data. (i.e. R1 G2 as no active state) Nevertheless, this approach successfully indicates when a set of data is inconsistent.
Before we go singing the praises of the Decomposed API approach, consider this case:
UPDATE R1 G1 001 state=active
UPDATE R1 G1 002 state=inactive
UPDATE R1 G2 002 state=active *
UPDATE R1 G2 002 state=inactive
UPDATE R1 G2 002 state=active **
Note the stars next to lines 3 & 5. Lets say that on line 3 the client sent the message as normal. Lets also say that on line 5 the client sent the same message that was sent on line 3 but for a second time. Our server's data would look like this:
Resource | Group | Time | State
_________+_______+______+______
R1 | G1 | 001 | active
R1 | G2 | 002 | inactive
R1 | G2 | 002 | active
R1 | G2 | 002 | inactive
R1 | G2 | 002 | active
This set of data would let our financial reporting layer to believe that R1 G2 002 is active. Thus we have shown that the Decomposed API approach alone is not robust to idempotency.
We can iterate on the previous approach by adding yet another piece of data to our requests. Motivated by our need for an idempotent API, we will add meta-data --the entity id. An entity ID will represent an active/inactive pair.
UPDATE R1 G1 001 E1 state=active
UPDATE R1 G1 002 E1 state=inactive
UPDATE R1 G2 002 E2 state=active
UPDATE R1 G2 002 E2 state=inactive
UPDATE R1 G2 002 E2 state=active
After our server has de-duplicated requests, our data will look like this:
Resource | Group | Time | State | Entity
_________+_______+______+__________+_______
R1 | G1 | 001 | active | E1
R1 | G2 | 002 | inactive | E1
R1 | G2 | 002 | active | E2
R1 | G2 | 002 | inactive | E2
By the Decomposed API approach and the Entity ID approach, we have now shown an event ordering schema that is robust to out-of-order events, idempotent and independent of time inconsistencies.
QED.