airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mayank Jaglan"<mjag...@umail.iu.edu>
Subject [#Spring17-Airavata-Courses] : Distributed Caching for Airavata Gateway
Date Fri, 17 Feb 2017 11:00:32 GMT
Hello Dev,

We are working on a way to provide single logical view (and state) for session and security
management using caching for the web applications.

For building a distributed cache system, I think, it should be able to -
-    Scale horizontally across multiple servers.
-    Scale across multiple regions (WANs).
-    Provide high availability
-    Provide Fault Tolerant, fail-over cluster
-    Perform faster/ concurrent read & writes
-    Provide data persistence, in the event of power failure
-    Install, configure and deploy with less complexity
-    Work well will popular technologies, like PhP, C++. Python, JAVA based frameworks


Following is the Github link to the example portal under consideration -
	https://github.com/airavata-courses/spring17-laravel-portal

Following is the Github issue created for this experiment -
	https://github.com/airavata-courses/spring17-laravel-portal/issues/5


	
** Proposed Solution **
Before starting this email thread, I did some experiment and I find a system like redis cluster
is very much suitable. Here are its pros and cons -

Pros:
-    Scale horizontally across multiple servers.
-    Scale across multiple regions (WANs).
-    Provide high availability
-    Provide Fault Tolerant, fail-over cluster
-    Can store data in a variety of data structures: list, array, sets and sorted sets. This
means using redis APIs, a specific attribute of an object can be modified directly in the
cache rather requesting full refresh of the object inside the redis cache. This brings over
all efficiency for larger objects.
-    Pipelining Multiple commands at once.
-    Blocking reads -- will sit and wait until another process writes data to the cache
-    Mass insertion of data to prime a cache
-    Partitions data across multiple redis master-instances
-    Can back data to disk. Provide data persistence, in the event of power failure.
-    Work well will popular technologies, like PhP, C++. Python, JAVA based frameworks

Cons:
-    Can be complex to configure -- requires consideration of data size to configure well.
-    Requires Redis server administration for monitoring, partitioning and balancing.
-    Sentinel is still a single point of failure. If the master wipes out, and Sentinel doesn't
work, the system is out!


Your feedback on this topic will be helpful.

Best,
Mayank Jaglan

Mime
View raw message