couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Newson <rnew...@apache.org>
Subject Re: database dynamic hot plug (attach / detach)
Date Thu, 21 Jun 2012 10:08:28 GMT

CouchDB doesn't have a cache unless you mean that it has an open file descriptor on that file
(which, obviously, would not be affected by the removal of a hard link). All the effects you've
noticed seem to follow naturally from POSIX filesystem semantics.

CouchDB is a database server and requires all of its databases to be in the database_dir directory.
It is not currently hot-pluggable. As you have also noticed,  there are restrictions on using
a single file system too.

An enhancement to make it pluggable does sound intriguing, though I'll note that when Ubuntu
had a similar problem (distinct databases for distinct local users) they elected to run a
separate couchdb process for each user.

Bottom line: To get the feature you want, you'll need to patch CouchDB. Done well, I think
there's a good chance we'd merge it into a future release.

B.


On 21 Jun 2012, at 10:16, CDR wrote:

> I was sceptic, but I tried the link tricks (without restarting
> couchdb). 
> - with a symbolic link, (softlink, ln -s) 
>   * CouchDB does not see the database (not a file).
> - with hard link (logical, ln- L) 
>   * CouchDB effectively see the new db through the link.
>   * removing original file, couchdb still have the db file.
>   * removing the hard link, couchdb should not have it anymore, FALSE!!
>   ==>> detaching is not properly done because couchdb have a cache.
>   /!\ HUGH LIMITATION, file and link have to be on the same partition.
> It is thus not possible to use this approach for removable filesystem.
> 
> Couchdb have a cache, so we can access a db that is offline.
> Their is a way to see if the db file exist, _all_dbs list only dbs wiche
> are online (the db file is reachable).
> 
> /!\ HAVE TO CHECK CURRENT (PENDING) REQUEST before detaching. 
> 
> Alternate, but database statistic on access required, is to lock the DB.
> The easiest way is to lock de db rejecting all update with
> "validate_doc_update" and to wait at least 1 seconde before
> disconnecting the external db file (explanantion: couch db flush every
> second as in second part in "Batch Mode" paragraphe at
> http://guide.couchdb.org/draft/performance.html ).
> So even if batch mode enabled, no data losses (if small data).
> 
> REQUIERED DB ACCESS STATISTICS, because of following case :
> However I have tried a big attachement, requiering more than 1 second of
> file IO (100Mo). This does not result in a direct writting, but a long
> caching in memory (compressing?) and only a long time (more than 1s,
> roughtly 30s) after, write is performed to the disk.
> This is not acceptable to detach the database.
> 
> A way to monitor db activity would be nice, but so far, I have not found
> such in the couchDB documentation.
> 
> Does it exist at least? internally I guesss so.
> 
> Finally the db file is constantly opened (sudo lsof -r 1 dbfile). if the
> db can garantee their is no more access (all flushed) this is not really
> a trouble, but it would be proper to have a clean fcolse.
> 
> WHY : 
> - I work on a filesystem project, and I would require to mount unmount
> db like we do with filesystem. Their is a real need to cary the db fs,
> and not jsut files on standard fs (lacks of information).
> 
> Regards
> 
> Le mercredi 20 juin 2012 à 21:19 +0200, Dave Cottlehuber a écrit :
>> On 20 June 2012 17:59, CDR <cdr.devinov@free.fr> wrote:
>>> I would like to know if such behavior is possible :
>>> For the runnning instance of couchdb server, ask to attach a db (from
>>> another directory, even a usb key). Then detach it.
>> 
>> You could use a soft link to achieve this. Just don't compact the db.
>> You need to think about where the views will go also.
>> 
>> I'm assuming that you are not planning to do this on a "real"
>> server environment so the risk of having incorrect on-disk
>> formats (they change over releases) and screw-ups due to compacting
>> and view-building are less of a concern.
>> 
>> If its a server I'd avoid the monkeying around. Why is this necessary?
>> 
>>> Also can a view gather data from more than one database.
>> 
>> No, I don't know if the other forks of CouchDB support this tho.
>> 
>>> Elsewhere, the solution is to start a new server on another listening
>>> port, and then query the two servers. Then merge the view result, that
>>> is getting complicated.
>> 
>> That's called a JOIN I believe.
>> 
>> A+
>> Dave
> 
> 


Mime
View raw message