Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of jchris@gmail.com designates
 209.85.219.168 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type
         :content-transfer-encoding;
        b=YnWyqIjj21eb+u0VBoM9HATxMFbjo5Ch1DvvzwPN+cFSNSzvzLmBqtdT9R8fSmSy8l
         FyrriksVfFxtDoFIo5CXu9zyaMciB7FGCbCyJAS8A/mWV2HbBVBqyg8Qa1/OBMb2DvID
         5RKNqJcnRfPZUgrchVfW6wa88wzkxBylV4B+M=
MIME-Version: 1.0
Sender: jchris@gmail.com
In-Reply-To: <8b1c89270905151403u67ccd1fek97fea33f5c7b0169@mail.gmail.com>
References: <8b1c89270905140533s71587e15pfa11d09f9201ba8e@mail.gmail.com>
	 <e2111bbb0905140839he17949ay9eda0540afc311e2@mail.gmail.com>
	 <8b1c89270905150055g129cfedtedbc2d07331e217a@mail.gmail.com>
	 <8b1c89270905150605u5656c883q4e0884cccb1d0799@mail.gmail.com>
	 <e2111bbb0905150713p661b9396m448893c145653c10@mail.gmail.com>
	 <8b1c89270905151403u67ccd1fek97fea33f5c7b0169@mail.gmail.com>
Date: Fri, 15 May 2009 17:30:57 -0700
Message-ID: <e282921e0905151730uc0a9a0and3f8ec140b089f7e@mail.gmail.com>
Subject: Re: View Server clarifications
From: Chris Anderson <jchris@apache.org>
To: user@couchdb.apache.org, dghosh@acm.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Fri, May 15, 2009 at 2:03 PM, Debasish Ghosh
<ghosh.debasish@gmail.com> wrote:
> Paul -
> I already did a grep and checked that neither my script nor any of my
> libraries was writing on the stdout. The one stuff which I did not check =
was
> the Scala source. Eventually I digged into the Scala source and found tha=
t
> the Scala interpreter writes some stuff on the stdout and strangely one o=
f
> them was "!!!". The writing can be configured by supplying a different
> reporter to the Scala Interpreter settings. However, what I did was to do
> away with the interpreter version and having the view server as a compile=
d
> image. Besides solving the problem, it also makes things run fast now.
> Incidentally I have been working on a Scala client for CouchDB (
> http://code.google.com/p/scouchdb), which is now available as an open
> source. Soon I plan to add the Scala view server as well.
>

Very cool. Do note that the list api will be changing for 0.10.0. I
owe dev@ a mail explaining the changes (need to write the code first
so I can explain it correctly...) Anyway, I'd just suggest writing the
list handler last, maybe I'll have the changes in by the time you get
to it.

Cheers,
Chris

> Thanks for all the help.
> - Debasish
>
> On Fri, May 15, 2009 at 7:43 PM, Paul Davis <paul.joseph.davis@gmail.com>=
wrote:
>
>> It looks like your script is printing "!!!" somehwere to standard out
>> and CouchDB is joking on it which is to be expected since !!! isn't
>> valid JSON.
>>
>> Make sure that your logging output goes to stderr and you should be fine=
.
>>
>> On Fri, May 15, 2009 at 9:05 AM, Debasish Ghosh
>> <ghosh.debasish@gmail.com> wrote:
>> > Ok .. after some more debugging sessions, it looks like the problem is
>> with
>> > launching of the view server script. I tried with some debug statement=
s
>> > (even something that writes to a file), but got nothing. The couchdb l=
og
>> > shows references to the script, but I think it cannot do a launch of i=
t.
>> The
>> > script is written in Scala and begins with the usual stuff of shell
>> > invocation ..
>> >
>> > #!/bin/sh
>> > exec ~/software/scala/scala-2.7.4.final/bin/scala -classpath
>> > ~/projects/util/lib/dispatch.jar -nocompdaemon "$0" "$@"
>> > !#
>> > //.. scala code here
>> >
>> > The script runs ok when I invoke it from shell. But looks like it gets
>> into
>> > some problem when invoked by CouchDB. Futon also comes up with the
>> following
>> > error ..
>> >
>> > Error: badmatch
>> >
>> > {{nocatch,{invalid_json,<<"!!!">>}},
>> > =A0[{couch_os_process,prompt,2},
>> > =A0 {couch_query_servers,'-start_doc_map/2-fun-0-',2},
>> > =A0 {lists,foreach,2},
>> > =A0 {couch_query_servers,start_doc_map,2},
>> > =A0 {couch_view_updater,view_compute,2},
>> > =A0 {couch_view_updater,update,1}]}
>> >
>> > And here are some more errors from couch.log ..
>> >
>> > [Fri, 15 May 2009 12:29:50 GMT] [error] [<0.3878.1>]
>> {error_report,<0.22.0>,
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 {<0.3878.1>,supervisor_report,
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0[{supervisor,{local,couch_secondary_ser=
vices}},
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 {errorContext,child_terminated},
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 {reason,{{nocatch,{invalid_json,<<"!!!=
">>}},
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0[{couch_os_process,=
prompt,2},
>> >
>> {couch_query_servers,'-start_doc_map/2-fun-0-',2},
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 {lists,foreach,2},
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 {couch_query_serve=
rs,start_doc_map,2},
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 {couch_view_update=
r,view_compute,2},
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 {couch_view_update=
r,update,1}]}},
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 {offender,[{pid,<0.3879.1>},
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{name,view_mana=
ger},
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{mfa,{couch_vie=
w,start_link,[]}},
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{restart_type,p=
ermanent},
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{shutdown,bruta=
l_kill},
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{child_type,wor=
ker}]}]}}
>> >
>> > [Fri, 15 May 2009 12:29:50 GMT] [error] [<0.4645.1>] Uncaught error in
>> HTTP
>> > request: {error,
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{ca=
se_clause,
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 {{=
nocatch,{invalid_json,<<"!!!">>}},
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0[{couch_os_process,prompt,2},
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 {couch_query_servers,
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0'-start_doc_map/2-fun-0-',2},
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 {lists,foreach,2},
>> >
>> {couch_query_servers,start_doc_map,2},
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 {couch_view_updater,view_compute,2},
>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 {couch_view_updater,update,1}]}}}
>> >
>> > Any help will be greatly appreciated.
>> >
>> > Thanks.
>> > - Debasish
>> >
>> > On Fri, May 15, 2009 at 1:25 PM, Debasish Ghosh <
>> ghosh.debasish@gmail.com>
>> > wrote:
>> >>
>> >> Thanks for the response. I have removed buffered stream calls .. the
>> >> process time out error is no more. But now I am getting an illegal JS=
ON
>> >> error as the following .. (from couch.log)
>> >>
>> >> ** Last message in was {'EXIT',<0.15838.0>,
>> >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{{nocatch,{inv=
alid_json,<<"!!!">>}},
>> >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 [{couch_os_pr=
ocess,prompt,2},
>> >>
>> >> {couch_query_servers,'-start_doc_map/2-fun-0-',2},
>> >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{lists,for=
each,2},
>> >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{couch_que=
ry_servers,start_doc_map,2},
>> >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{couch_vie=
w_updater,view_compute,2},
>> >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{couch_vie=
w_updater,update,1}]}}
>> >>
>> >> The strange part is that the script runs ok when I run from the comma=
nd
>> >> prompt and supply inputs like ["reset"]\n etc. manually. Is there any
>> way I
>> >> can poke into what commands are actually being exchanged between Couc=
hDB
>> and
>> >> the view server ?
>> >>
>> >> Thanks for the help.
>> >> - Debasish
>> >>
>> >> On Thu, May 14, 2009 at 9:09 PM, Paul Davis <
>> paul.joseph.davis@gmail.com>
>> >> wrote:
>> >>>
>> >>> The most likely cause is buffering. If you're not getting any input,
>> >>> then I'm guessing that getLines.forEach is doing some buffering to t=
ry
>> >>> and make reading from files faster. If you get one line and nothing
>> >>> else, your stdout stream is probably buffered.
>> >>>
>> >>> HTH,
>> >>> Paul Davis
>> >>>
>> >>> On Thu, May 14, 2009 at 8:33 AM, Debasish Ghosh
>> >>> <ghosh.debasish@gmail.com> wrote:
>> >>> > Hi -
>> >>> >
>> >>> > I was trying to play around with query servers using Scala as the
>> >>> > language.
>> >>> > I have registered Scala as the language in local.ini .. it shows u=
p
>> in
>> >>> > Futon
>> >>> > as well. I have a Scala script registered that reads from the
>> standard
>> >>> > input
>> >>> > and writes into the standard output. Here is a snippet from the
>> script
>> >>> > ..
>> >>> >
>> >>> > =A0scala.io.Source.fromInputStream(System.in).getLines.foreach {l =
=3D>
>> >>> > =A0 =A0 //.. expecting to get stuff like "add_fun", "map_doc", "re=
set"
>> etc.
>> >>> > here ..
>> >>> >
>> >>> > But I am not getting anything within this loop, though the process
>> gets
>> >>> > kicked off when the view server launches. After some time it comes
>> out
>> >>> > with
>> >>> > the exception ..
>> >>> >
>> >>> > =A0{"error":"case_clause","reason":"{{nocatch,{os_process_error,\"=
OS
>> >>> > process
>> >>> > timed out.\"}},\n [{couch_os_process,prompt,2},\n
>> >>> > {couch_query_servers,'-start_doc_map/2-fun-0-',2},\n
>> >>> > =A0{lists,foreach,2},\n
>> >>> > {couch_query_servers,start_doc_map,2},\n
>> >>> > {couch_view_updater,view_compute,2},\n
>> >>> > =A0{couch_view_updater,update,1}]}"}
>> >>> >
>> >>> > Am I assuming anything which is not correct ? Please help.
>> >>> >
>> >>> > Thanks.
>> >>> > - Debasish
>> >>> >
>> >>
>> >
>> >
>>
>


--=20
Chris Anderson
http://jchrisa.net
http://couch.io