incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Enda Farrell <enda.farr...@bbc.co.uk>
Subject Re: [long] multiple couchdb invocations, single machine
Date Thu, 05 Nov 2009 15:03:56 GMT
Really sorry to have taken so long to add in a few pointers - been busy.

* We always run 4 nodes/invocations on each machine.
* We leave the "default.ini" alone.
* We create 4 separate $PORT.ini files - you MUST give each one their  
own dir for [couchdb] (you can if you're careful share the logs dir)
	[httpd]
	port = 5984
	bind_address = 0.0.0.0

	[couchdb]
	database_dir = /data/couchdb/5984 ; keep last dir same as port!
	view_index_dir = /data/couchdb/5984 ; keep last dir same as port!
	[log]
	file = /data/app-logs/couchdb/5984.log
	level = error

* Our rc.d/couchdb file is not at all like the default. Perhaps one  
day I'll get the chance to clean it up and publicise it, but at the  
core there a major change or two:
	We bring up and down all 4 at the same time.
	When you have many running instances all controlled  by a single rc.d/ 
couchdb that script needs to know about all 4
	Due to the Erlang VM not unhooking properly, we have had to setsid  
ourselves and bypass the "normal" PID files
		setsid $COUCHDB -c $CONF/default.ini -c $CONF/$PORT.ini -o $LOG/ 
start.$PORT.o.log -e $LOG/start.$PORT.e.log \
                                         0</dev/null 1>$LOG/start. 
$PORT.i.log 2>&1 & echo $! > $VR/couchdb.$PORT.pid
		$CONF = config dir = the normal one
		$PORT = local looping variable in the script
		$LOG = log dir - as above in the $PORT.ini
		$VR = "normal" var/run dir
	Note that you must give separate files for any output - including the  
pid files.

* We did find that there's code in CouchDB that stomped over the  
default "couchdb.pid" file which meant servers wouldn't stay up - but  
by redirecting the output of $! and not using -p you can avoid that.

/e


On 24 Oct 2009, at 20:03, Michael McDaniel wrote:

> On Sat, Oct 24, 2009 at 02:49:32PM -0400, Adam Kocoloski wrote:
>> On Oct 24, 2009, at 2:37 AM, Paul Davis wrote:
>>
>>> On Sat, Oct 24, 2009 at 2:26 AM, Michael McDaniel
>>> <couchdby@autosys.us> wrote:
>>>> On Sat, Oct 24, 2009 at 02:18:33AM -0400, Paul Davis wrote:
>>>>>> And, just now, I realized that it is wanting to create databases
>>>>>> in whatever directory I start from. (I do not understand why
>>>>>> it wasn't trying to create them in /usr/local/var/lib but rather
>>>>>> /usr/local/var/lib/couchdb before).
>>>>>>
>>>>>> I start PAT in /usr/local/var/lib/couchdb_pat
>>>>>> and start
>>>>>> MIKE in /usr/local/var/lib/couchdb_mike
>>>>>>
>>>>>> and I can run separate test suites and create databases with
>>>>>> each invocation that the other invocation does not see.
>>>>>>
>>>>>>
>>>>>> The trivial workaround to have it function as I want is to
>>>>>>
>>>>>> cd path/to/invocation/database/directory && couchdb -n -a
...
>>>>>>
>>>>>>
>>>>>> Of course, not a long-term strategy but will work for now.
>>>>>
>>>>> Michael,
>>>>>
>>>>> Hmm, you must've uncovered a bug here in the recent boot sequence
>>>>> updates. You've probably reduced it well enough that we should be
>>>>> able
>>>>> to get it fixed though. Thanks for the report.
>>>>>
>>>>> Paul Davis
>>>> ^ 
>>>> ^ 
>>>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>>>
>>>> glad it helped, Paul - and thanks for the laugh w/the bug report
>>>>                       (I still lurk on dev@)
>>>>
>>>>
>>>> I'll grab a new version in a day or few ...
>>>>
>>>> ~M
>>>>
>>>
>>> If you're not having fun, you're Doing It Wrong.
>>
>> Actually, I'm having trouble reproducing this bug.  I must be missing
>> something.  All I did was write the config file (this is pat.ini,
>> mike.ini is analogous)
>>
>>> macbook:~ $ cat /usr/local/couchdb/trunk/etc/couchdb/pat.ini
>>> [couchdb]
>>> database_dir = /tmp/pat
>>>
>>> [log]
>>> level = debug
>>>
>>> [httpd]
>>> port = 5986
>>
>> and start the server
>>
>>> macbook:~ $ /usr/local/couchdb/trunk/bin/couchdb -a /usr/local/
>>> couchdb/trunk/etc/couchdb/pat.ini
>>> Apache CouchDB 0.11.0b829113 (LogLevel=debug) is starting.
>>> Configuration Settings ["/usr/local/couchdb/trunk/etc/couchdb/
>>> default.ini",
>>>                        "/usr/local/couchdb/trunk/etc/couchdb/
>>> local.ini",
>>>                        "/usr/local/couchdb/trunk/etc/couchdb/
>>> pat.ini"]:
>> <snip>
>>>  [couchdb] database_dir="/tmp/pat"
>>
>>
>> and do the same for Mike, and I get the correct behavior -- Pat's  
>> DBs go
>> in /tmp/pat, Mike's in /tmp/mike.  I also tried running in the
>> background,  Nothing that I do seems to trigger this bug.
>>
>> I will note, though, that the code which looks up the database_dir  
>> falls
>> back to "." if it's not defined (see couch_server:init/1).  So that  
>> tells
>> me the config values in Michael's setup aren't accessible to
>> couch_server.
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
>  Though they show up via Futon config.html
> ~M
>
>>
>> Michael, can you check one more thing for me -- did you upgrade in  
>> place
>> to a newer version of CouchDB?  What are the contents of /usr/
>> local/lib/couchdb/erlang/lib?  We have a bug we need to fix in trunk
>> where Couch can accidentally use old versions of modules if they  
>> show up
>> in the same directory.  If that happens, well, all bets are off.   
>> Best,
>>
>> Adam
>>
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> I do not remember if I upgraded in place.  The fact that my couchdb
> build directory contains a BUG and bootstrap file from 2009/09/20
> may indicate that I did layer the newest version over top of my
> existing couchdb dir contents.  And then
>  ./bootstrap
>  make
>  sudo make install
>
>
> $ ls -rlast /usr/local/lib/couchdb/erlang/lib
> total 32
> 4 drwxr-xr-x 3 mmcdanie mmcdanie 4096 2009-02-23 15:52 ..
> 4 drwxr-xr-x 3 mmcdanie mmcdanie 4096 2009-03-07 01:27 mochiweb-r97
> 4 drwxr-xr-x 3 root     root     4096 2009-09-20 16:00 ibrowse-1.5.2
> 4 drwxr-xr-x 3 root     root     4096 2009-09-20 16:00 erlang-oauth
> 4 drwxr-xr-x 5 root     root     4096 2009-09-20 16:00  
> couch-0.11.0a817117
> 4 drwxr-xr-x 5 root     root     4096 2009-10-22 10:00  
> couch-0.11.0b828784
> 4 drwxr-xr-x 3 root     root     4096 2009-10-22 10:00 etap
> 4 drwxr-xr-x 8 mmcdanie mmcdanie 4096 2009-10-22 10:00 .
>
>
>
> When you start the respective couchdb processes, if you are already
> in the desired db directory, that is where the db files will be
> created and read from.
>
> To see the problem, I start the processes from any directory other
> than the respective db directories.
>
> My 'couchdb -V' version is the 0.11.0b828784
>
> ~M


http://www.bbc.co.uk/
This e-mail (and any attachments) is confidential and may contain personal views which are
not the views of the BBC unless specifically stated.
If you have received it in error, please delete it from your system.
Do not use, copy or disclose the information in any way nor act in reliance on it and notify
the sender immediately.
Please note that the BBC monitors e-mails sent or received.
Further communication will signify your consent to this.
					

Mime
View raw message