airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shenoy, Gourav Ganesh" <>
Subject [#Spring17-Airavata-Courses] : Data Management for Microservices
Date Sun, 19 Feb 2017 22:24:08 GMT
Hello Devs,

As part of the Spring’17 Advanced Science Gateway Architecture course, we are working on
the topic of Data Management for Micro-services. The goal of this project is to explore possible
ways to manage data across micro-services. In a nutshell, the question at hand is whether
to have a “database per micro-service?” or “shared database for all micro-services?”.

After reading through few blogs and articles it was clear that “generally”, it is not
a good idea to maintain everything in a shared database have all micro-services access it.
Rather in order to achieve portability and maintainability of both the micro-services and
the database, it is a good practice to keep a micro-service data private – only takes care
of data which is needed for that service. Having said that, in a distributed environment it
is almost impossible to NOT have any data dependencies between micro-services (if not write,
then at least read). Like for e.g.: “User Profile” micro-service contains data that is
most often needed by others.

In such a scenario, the challenge would be to keep a micro-service data private and at the
same time enable sharing following the CAP theorem. We have identified 2 possible solutions
to achieve this:

·         2-phase commits

·         Event driven data replication

Each of these have their pros and cons related to consistency and availability. While 2-phase
commits guarantee consistency while compromising availability, event driven data replication
assures eventual consistency.

Ajinkya and me, we have already implemented a prototype for the event driven data replication
(see github<>).
We are using a message broker (RabbitMQ) as our event-communication channel. As of now, we
have identified all possible corner cases and tuned our code accordingly. We would appreciate
if anyone would volunteer to try out the 2-phase commit.


Thanks and Regards,
Gourav Shenoy
View raw message