airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shenoy, Gourav Ganesh" <goshe...@indiana.edu>
Subject [#Spring17-Airavata-Courses] : Data Management for Microservices
Date Sun, 19 Feb 2017 22:24:08 GMT
Hello Devs,

As part of the Spring’17 Advanced Science Gateway Architecture course, we are working on
the topic of Data Management for Micro-services. The goal of this project is to explore possible
ways to manage data across micro-services. In a nutshell, the question at hand is whether
to have a “database per micro-service?” or “shared database for all micro-services?”.

After reading through few blogs and articles it was clear that “generally”, it is not
a good idea to maintain everything in a shared database have all micro-services access it.
Rather in order to achieve portability and maintainability of both the micro-services and
the database, it is a good practice to keep a micro-service data private – only takes care
of data which is needed for that service. Having said that, in a distributed environment it
is almost impossible to NOT have any data dependencies between micro-services (if not write,
then at least read). Like for e.g.: “User Profile” micro-service contains data that is
most often needed by others.

In such a scenario, the challenge would be to keep a micro-service data private and at the
same time enable sharing following the CAP theorem. We have identified 2 possible solutions
to achieve this:

·         2-phase commits

·         Event driven data replication

Each of these have their pros and cons related to consistency and availability. While 2-phase
commits guarantee consistency while compromising availability, event driven data replication
assures eventual consistency.

Ajinkya and me, we have already implemented a prototype for the event driven data replication
(see github<https://github.com/airavata-courses/spring17-microservice-data-management>).
We are using a message broker (RabbitMQ) as our event-communication channel. As of now, we
have identified all possible corner cases and tuned our code accordingly. We would appreciate
if anyone would volunteer to try out the 2-phase commit.

References:
[1] http://microservices.io/patterns/data/database-per-service.html
[2] http://microservices.io/patterns/data/event-driven-architecture.html
[3] https://plainoldobjects.com/2015/09/02/does-each-microservice-really-need-its-own-database-2/
[4] https://en.wikipedia.org/wiki/CAP_theorem

Thanks and Regards,
Gourav Shenoy
Mime
View raw message