couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Kowalski <...@kowalski.gd>
Subject Re: How to get back to the screen some debug info from the compiled program ?
Date Mon, 11 May 2015 00:20:22 GMT
Hi Martin,

> Whaw, I knew that dev/run was running 3 clusters at :

In fact they are three nodes that are joined into a cluster

Every node has still the "old" unclustered http interface from Couch
1.x, for dev/run we have

15984 - 16986
25984 - 26986
35984 - 36986

The code for the old CouchDB http api (nodes) is located at [1] - I
know that it is a bit confusing. The reason for it: historically
CouchDB 2.x originates from BigCouch. BigCouch was a forked version of
CouchDB with the Amazon Dynamo Paper applied _on top_ of the old plain
CouchDB.

In 2.x - on the clustered ports (15984, 25984, 35984) chttpd, rexi,
fabric & co are taking care that requests on the clustered interface
arrive at the node that is responsible for the requested part of the
db.

Changing the url is a good idea, but we will need to disable some
tests (some features are not available on the clustered interface -
e.g. compaction which just works on a per-node-level).

We would also need to change the suite (explicitly pass a write quorum
param of 3 to the requests (in case of the testsuite running against a
three nodes cluster). We would need to do that in order to get stable
tests as the cluster is just eventually consistent, meaning that per
default it will return an "OK" after a document was written to two of
our three nodes. That can lead to failing tests: Request A arrives at
Node1, the Document is written to 2 nodes, we receive an OK with "202
Accepted" instead of "201 Created".

While the third Node(Node3) is still writing the document, our
testclient got the OK and fires the next request to test if our doc
was successfully written. We hit Node3 and get a "document not found",
resulting in the testsuite to fail. We need to return every node in
the cluster under test to return our expected result to get a stable,
reliable testsuite, so we need to change the w-param (write quorum) to
n (number of nodes). An excellent presentation reagrding this topic is
[2]


[1] https://github.com/apache/couchdb-couch
[2] http://conf.couchdb.org/couchdb-conf-berlin-january-2013/slides/Introduction-to-BigCouch-CouchDB-Conf-Berlin-2013.pdf

On Mon, May 11, 2015 at 12:02 AM, Martin Lagrange
<lagrangemartin@gmail.com> wrote:
> Hey, Robert, thanks for your reply !
>
> Whaw, I knew that dev/run was running 3 clusters at :
>
>
>    - http://127.0.0.1:25984/
>    - http://127.0.0.1:35984/
>    - http://127.0.0.1:15984/
>
> but I didn't know that it was running another cluster at :
>
>    - http://localhost:15986/
>
> but I should have seen :
> https://github.com/apache/couchdb/blob/master/dev/run#L419
>
> Do you mean that the cluster 15986 is not served by the compiled app ? By
> what ? What do you mean exactly by
> "are served by apache/couchdb-couch." ?
>
> There are some integration tests for chttpd in Erlang [3]
>
>
> Yes, indeed but there are not so many and not as convenient as the
> javascript tests :D
>
>
>> - one idea
>
> for the future is to run the JS tests also against the cluster but it
>
> will need some further modifications of the  JS test suite.
>
> As a trivial approach, wouldn't it be enough to just say to couchJS command
> to run itself against the dev cluster ?
> There is this -u option to specify a uri file to run the test against.
>
> https://github.com/apache/couchdb/blob/master/test/javascript/run#L74
>
>
> Sorry in case I have misunderstood you.
>
>
> 2015-05-10 21:13 GMT+02:00 Robert Kowalski <rok@kowalski.gd>:
>
>> Hi Martin,
>>
>> see my responses inline
>>
>> On Sun, May 10, 2015 at 1:50 PM, Martin Lagrange
>> <lagrangemartin@gmail.com> wrote:
>> > Hi everyone,
>> >
>> > I am currently working on solving an issue in the rewriting module that
>> > affect the rewrite process :
>> >
>> https://github.com/apache/couchdb-chttpd/blob/master/src/chttpd_rewrite.erl
>> > and I have written some end-to-end regression tests :
>> >
>> https://github.com/apache/couchdb/blob/master/test/javascript/tests/rewrite.js
>>
>> Wow cool!
>>
>> > I modify the locally cloned couchdb-chttpd repository and compile the
>> > program again, then run the server and the tests again.
>> >
>> > make && dev/run --with-admin-party-please
>> >> test/javascript/run test/javascript/tests/rewrite.js
>> >
>> >
>> > The thing is that I would like in someway to see what happen in the
>> program
>> > during the execution, especially when this tests are running.
>> >
>> > I have written :
>> >
>> > io:format('blabla'),
>> >
>> >
>> > or
>> >
>> > couch_log:debug("blabla"),
>> >
>> >
>> >  in the erlang file but nothing seems to appear whether in the server
>> > console or in any logs.
>> >
>>
>>
>> couchjs [1] runs against the old so-called backdoorports [2], which
>> are served by apache/couchdb-couch. The cluster interface with the new
>> "public" ports for the cluster are served by "couchdb-chttpd" - the
>> part where you want t log.
>>
>> This means the JavaScript testsuite does not use code located in chttpd.
>>
>> There are some integration tests for chttpd in Erlang [3] - one idea
>> for the future is to run the JS tests also against the cluster but it
>> will need some further modifications of the  JS testsuite.
>>
>>
>> [1] https://github.com/apache/couchdb/blob/master/test/javascript/run#L27
>> [2]
>> https://github.com/apache/couchdb-couch/blob/master/priv/couch_js/http.c#L363
>> [3]
>> https://github.com/apache/couchdb-chttpd/blob/master/test/chttpd_db_test.erl
>>
>
>
>
> --
> Martin LAGRANGE

Mime
View raw message