openwhisk-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [openwhisk] style95 commented on issue #4626: Allow limiting DB bloat by excluding response from Activation record in some cases
Date Tue, 17 Sep 2019 07:11:06 GMT
style95 commented on issue #4626: Allow limiting DB bloat by excluding response from Activation
record in some cases
URL: https://github.com/apache/openwhisk/issues/4626#issuecomment-532092188
 
 
   Regarding this issue, I think the main reason is the limited scalability of CouchDB.
   If CouchDB receives too many requests, it cannot handle them properly and even crashes
sometimes.
   
   Also, more and more data is stored, more and more CouchDB would be highly likely unavailable.
   It lacks the functionality to manage existing old(unused) data.
   As OW depends on "view" of CouchDB, we need to periodically trigger indexing of views to
prevent CouchDB from the crash while it indexes too much data.
   
   
   If the backend store is scaling enough to handle all data and easy to delete unnecessary(old)
data, it would solve many parts of this issue.
   
   For example, if we introduce a scalable datastore such as ElasticSearch, it would greatly
alleviate the situation. 
   
   I have observed many issues with CouchDB and we here(Naver) replaced it with ElasticSearch
and observed no issue so far.
   
   It outperforms CouchDB in many aspects such as :
   
   1. Enable full-text search of activation(including logs)
   2. Better scalability
   3. Easy to manipulate data especially in terms of deleting old(unused) data with the index
alias.
   
   AFAIK, @dubee had worked on `ElasticSearchActivaitonStore` some time back, but it was not
completed for some reasons.
   I suspect there may be some historical reason to tie up with CouchDB?
   
   If there is no other historical reason to persist CouchDB, I would like to bring up the
`ElasticSearchActivationStore` again and we are willing to contribute it as well.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message