couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Johnson <rob...@rowanshire.net>
Subject Re: Monitoring system based on CouchDB?
Date Mon, 14 Mar 2011 21:12:07 GMT
Nikolai

You would have a rapidly growing database on your nodes as well because each document update
on each node would generate a new version. 

I offer this:-

Have a document in your master database with the URLs of the nodes and have a cron job to
read that document and initiate a one time replication with each node  and compress your "master"
database (you can use a shell script and curl to do this). Now, you will need to either multi
thread the replication cron script or in some way test that the host is available before trying
the replication because if the remote host is not available your script will stall for a good
few seconds when trying to contact the remote host and if you have a lot of hosts and a short(ish)
cron cycle you can get cron job collision.
Have the status collection job on each node compress the database on each update.

Bob

On 14 Mar 2011, at 12:41, Nikolai Teofilov wrote:

> Hi all,
> 
> I need a help on a concept for a monitoring system based on CouchDB. I have master server
with a database "status" and several, around 20 nodes machines (the number of machines will
grow) performing some logging on a local instances of CouchDB. Each local machine (node) has
a database "status" as well and logs realtime data in a single document named as the computer.
The idea is to monitor all the data from all node machines in a single database on the master
server. The final result should be a database that is a collection of documents named as the
computer names of all node machines and each document reflect the current status of the node.
Then it will be easy to have long-poll handler for monitoring and rendering the changes of
all machines on the master server as a status panel...
> 
> I made a local db "status" on each node machine and started a continuous replication
to the master server "status" database. The problem is that some of the logging processes
on the node machines are working over hours and can involve logs each second. This will reflect
on the size of the master database. How to handle this correctly? I can compact the master
target database from time to time remotely. Is this a good idea or is there any other way
to make continuous replication with kind of auto compact feature? Would a compacting from
time to time will affect somehow the other continuous replications?
> 
> The second problem I have is once I delete (recreate) the source local "status" database
the old instance of the continuous replication does not work for the new created database.
I have a kind of workaround be canceling the old continuous replication and triggered it again.
This work well but I am not sure if I can expect some other problems somehow.  
> 
> Any suggestions or thoughts?
> 
> Thanks
> Nikolai


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message