incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McDaniel <couch...@autosys.us>
Subject Re: [long] multiple couchdb invocations, single machine
Date Sat, 24 Oct 2009 05:45:12 GMT

 Paul, I did the -n -a -a  using custom default*.ini and custom
 local*.ini  e.g.

 $ couchdb -n -i -a /usr/local/etc/couchdb/mike/default_mike.ini -a /usr/local/etc/couchdb/mike/local_mike.ini
Erlang R13B01 (erts-5.7.2) [rq:1] [async-threads:0] [hipe] [kernel-poll:true]

Eshell V5.7.2  (abort with ^G)
1> Apache CouchDB 0.11.0b828784 (LogLevel=debug) is starting.
Configuration Settings ["/usr/local/etc/couchdb/default.ini",
                        "/usr/local/etc/couchdb/local.ini",
                        "/usr/local/etc/couchdb/mike/default_mike.ini",
                        "/usr/local/etc/couchdb/mike/local_mike.ini"]

 ...


 Checked Futon config.html and the database_dir and view_index_dir
 show as expected (i.e. the path included in the custom ini files).

 However, looking at Futon Overview from both
 http://localhost:5984/_utils/index.html
 http://localhost:5985/_utils/index.html
 shows all my original databases as

   couchdb/foo_db
   couchdb/bar_db
   ...
   couchdb/test_suite_db/with_slashes
   ...
   etc.

 Is the hard coded {default, local}.ini files in couch.app causing
 any problem ?  I will guess yes after another little test.

 I moved default.ini and local.ini out of /usr/local/etc/couchdb
 and did 
 touch default.ini
 touch local.ini

 so couchdb would start (it errored out without {default,local}.ini files).
 Then, Futon Overview showed no databases as expected.  I created a
 database from each invocation, though, and it got created in
 /usr/local/var/lib/couchdb  rather than where I would expect, i.e.
 the respective directories

  /usr/local/var/lib/couchdb_pat
  /usr/local/var/lib/couchdb_mike


 And, both invocations were able to see both databases via Futon
 Overview.  Odd, though, they each showed up as

  pat_db
  mike_db

 rather than 

  couchdb/pat_db
  couchdb/mike_db


 So how's it know about the couchdb directory at all ?

 Hmmm, I had been starting from /usr/local/var/lib/ directory.

 I then removed all of my old databases and directories out
 of /usr/local/var/lib/couchdb and restarted.

 And, just now, I realized that it is wanting to create databases
 in whatever directory I start from. (I do not understand why
 it wasn't trying to create them in /usr/local/var/lib but rather
 /usr/local/var/lib/couchdb before).  

 I start PAT in /usr/local/var/lib/couchdb_pat  
 and start
 MIKE in /usr/local/var/lib/couchdb_mike

 and I can run separate test suites and create databases with
 each invocation that the other invocation does not see.


 The trivial workaround to have it function as I want is to

 cd path/to/invocation/database/directory && couchdb -n -a ...


 Of course, not a long-term strategy but will work for now.
 

~M




On Sat, Oct 24, 2009 at 12:24:30AM -0400, Paul Davis wrote:
> Michael,
> 
> Heh, never underestimate the value of checking the basics. I've spent
> hours debuging my failure to run make dev before so sanity checks are
> always a good idea.
> 
> However you are right in that your original introspection and follow
> up indicate the db_directory.
> 
> Another simple test to try would be to copy default.ini to
> default_pat.ini and default_mike.ini setting parameters as appropriate
> there. Then your couchdb command lines would involve something along
> the lines of:
> 
> $ couchdb -n -a /path/to/default_pat.ini -a /path/to/local_path.ini ...
> 
> Let me know if that fixes bits.
> 
> People have successfully run multiple instances on a single machine so
> either we have a regression or you've uncovered a bug or something
> more random has happened.
> 
> HTH,
> Paul Davis
> 
> On Sat, Oct 24, 2009 at 12:04 AM, Michael McDaniel <couchdby@autosys.us> wrote:
> >  Though setting log level = debug, and running strace (per earlier
> >  Chris mail) might be useful for discovery, I think the empirical
> >  evidence already proves that the local.ini and local_mike.ini
> >  config files are getting read (and if they weren't, changing log
> >  level in either would have no effect).
> >
> >  Per earlier information, the proper log files are getting created which
> >  are defined in local.ini and local_mike.ini hence those ini files are
> >  getting read.  That is the only place the respective log files
> >  couchdb_pat.log and couchdb_mike.log are defined.
> >
> >  Additionally, Futon config.html indicates proper reading of the
> >  ini files based on the respectively displayed database_dir and
> >  view_index_dir variables.
> >
> >
> >  However, because some may still not be convinced from the existing
> >  evidence, and I have no immediate other ideas, and I would like some
> >  help with this, I changed from level = info to level = debug in each
> >  of local.ini and local_mike.ini (which are being read).  Looking at
> >  couchdb_pat.stdout and couchdb_mike.stdout showed the same
> >  database_dir and view_index_dir that shows per information
> >  in Futon config.html as described below.
> >
> >  I then started _mike using strace as follows:
> >
> >  sudo strace couchdb -i -p /usr/local/var/run/couchdb_prod.pid -o /home/erl/couchdb/logs/couchdb_prod.stdout
-e /home/erl/couchdb/logs/couchdb_prod.stderr -a /usr/local/etc/couchdb/prod/local_prod.ini
> foo.log 2>&1
> >
> >
> >  database_dir in foo.log shows
> >
> > writev(1, [{""..., 0}, {"  [couchdb] database_dir  =\"/usr/"..., 61}], 2  [couchdb]
database_dir  ="/usr/local/var/lib/couchdb_prod"
> >
> >
> >  view_index_dir in foo.log shows
> >
> > writev(1, [{""..., 0}, {"  [couchdb] view_index_dir=\"/usr/"..., 61}], 2  [couchdb]
view_index_dir="/usr/local/var/lib/couchdb_prod"
> >
> >
> >  It appears that the ini files are getting read.
> >
> >
> >  Has anyone run two separate invocations on one machine, using
> >  different ports with database isolation to separate directories
> >  unavailable to the other invocation ?
> >
> >  Perhaps this a use case that is not tested ?
> >
> > ~M
> >
> >
> > On Fri, Oct 23, 2009 at 10:42:46PM -0400, Paul Davis wrote:
> >> You could also set the log level to debug and start each without -b to
> >> see what config files are being read.
> >>
> >> Paul Davis
> >>
> >> On Fri, Oct 23, 2009 at 10:25 PM, Chris Stockton
> >> <chrisstocktonaz@gmail.com> wrote:
> >> > I would try starting them with strace -f and see what ini files they are
> >> > reading and look for other clues.
> >> >
> >> > On Oct 23, 2009 6:19 PM, "Michael McDaniel" <couchdby@autosys.us>
wrote:
> >> >
> >> >
> >> >  SHORT VERSION:
> >> >
> >> >  Multiple couchdb invocations on a single machine are not
> >> >  behaving as I would expect to isolate databases.
> >> >
> >> >  Does someone have a tested configuration they will share ?
> >> >
> >> >
> >> >
> >> >  LONG VERSION:
> >> >
> >> >  I have installed
> >> >
> >> >  couchdb - Apache CouchDB 0.11.0b828784
> >> >
> >> >  and am trying to do what I thought would be simple.
> >> >
> >> >  Run two independent invocations of couchdb on the same machine.
> >> >  With the expectation that by running separate invocations, I
> >> >  could keep respective databases isolated from each.
> >> >
> >> >
> >> >  Here's what I want
> >> >
> >> >  * one invocation on default port 5984   (let's call this one PAT)
> >> >  * other invocation on non-default port 5985 (let's call this one MIKE)
> >> >  * PAT should not see or have access to MIKE databases
> >> >  * MIKE should not see or have access to PAT databases
> >> >  * PAT can hang with no effect on MIKE
> >> >  * MIKE can hang and have no effect on PAT
> >> >  * long running events on either have no effect on the other
> >> >
> >> >  Here's what I tried (all pertinent directories and files are created
with
> >> >  rw permissions to invoking user).  default.ini is as created by couchdb
> >> >  build/install.
> >> > ---
> >> >
> >> >  Added the following to /usr/local/etc/couchdb/local.ini
> >> >
> >> > [couchdb]
> >> >
> >> > database_dir   = /usr/local/var/lib/couchdb_pat
> >> > view_index_dir = /usr/local/var/lib/couchdb_pat
> >> >
> >> > [log]
> >> > file = /home/erl/couchdb/logs/couch_pat.log
> >> > level = info
> >> > ---
> >> >
> >> >  Added the following to  /usr/local/etc/couchdb/mike/local_mike.ini
> >> >
> >> > [couchdb]
> >> > database_dir   = /usr/local/var/lib/couchdb_mike
> >> > view_index_dir = /usr/local/var/lib/couchdb_mike
> >> >
> >> >
> >> > [log]
> >> > file = /home/erl/couchdb/logs/couch_mike.log
> >> > level = info
> >> >
> >> > [httpd]
> >> > port = 5985
> >> > ---
> >> >
> >> >  start PAT with
> >> > couchdb  -b  -p  /usr/local/var/run/couchdb_pat.pid        \
> >> >            -o  /home/erl/couchdb/logs/couchdb_pat.stdout \
> >> >            -e  /home/erl/couchdb/logs/couchdb_pat.stderr
> >> > ---
> >> >
> >> >  start MIKE with
> >> > couchdb  -b  -p  /usr/local/var/run/couchdb_mike.pid        \
> >> >            -a  /usr/local/etc/couchdb/mike/local_mike.ini \
> >> >            -o /home/erl/couchdb/logs/couchdb_mike.stdout  \
> >> >            -e /home/erl/couchdb/logs/couchdb_mike.stderr
> >> > ---
> >> >
> >> >
> >> >  NOW, in Futon, for PAT http://localhost:5984/_utils/config.html
> >> >
> >> >
> >> >  database_dir       /usr/local/var/lib/couchdb
> >> >  database_dir       /usr/local/var/lib/couchdb_pat
> >> >  view_index_dir     /usr/local/var/lib/couchdb_pat
> >> >
> >> >  AND, for MIKE http://localhost:5985/_utils/config.html
> >> >
> >> >  database_dir       /usr/local/var/lib/couchdb
> >> >  database_dir       /usr/local/var/lib/couchdb_pat
> >> >  view_index_dir     /usr/local/var/lib/couchdb_pat
> >> >
> >> >
> >> >  BUT ... both show all the databases already existing in the default
> >> >  /usr/local/var/lib/couchdb directory.
> >> >
> >> >
> >> >  AND
> >> >
> >> >  If I create a new database from either PAT or MIKE URI
> >> >  http://localhost:5984/_utils/index.html
> >> >  http://localhost:5985/_utils/index.html
> >> >
> >> >  the new database winds up in the /usr/local/var/lib/couchdb
> >> >  directory (and, as previously mentioned, either invocation
> >> >  of couchdb can see all of the databases there).
> >> >
> >> >
> >> >  If I comment out database_dir and view_index_dir in default.ini
> >> >  then both invocations see the databases (in Futon Overview) as,
> >> >  e.g.
> >> >
> >> >   couchdb/foo_db
> >> >   couchdb/bar_db
> >> >
> >> >  Although each Futon Configuration shows only the respective
> >> >  database_dir    /usr/local/var/lib/couchdb_pat
> >> >  view_index_dir  /usr/local/var/lib/couchdb_pat
> >> >
> >> >  database_dir    /usr/local/var/lib/couchdb_mike
> >> >  view_index_dir  /usr/local/var/lib/couchdb_mike
> >> >  .
> >> >
> >> >
> >> >  Logs are getting written per respective PAT and MIKE local ini files.
> >> >
> >> >
> >> >  QUESTION:
> >> >
> >> >  How do I run more than one invocation of couchdb on a single machine
> >> >  with database and process isolation ?
> >> >
> >> >
> >> > ~Michael
> >> >

Mime
View raw message