Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of bra@fsn.hu designates
 195.228.252.137 as permitted sender)
Message-ID: <4C471D53.9040100@fsn.hu>
Date: Wed, 21 Jul 2010 18:16:19 +0200
From: Attila Nagy <bra@fsn.hu>
User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US;
 rv:1.9.1.10) Gecko/20100629 Thunderbird/3.0.5
MIME-Version: 1.0
To: "user@couchdb.apache.org" <user@couchdb.apache.org>
Subject: CouchDB with SMP HiPE? (and a benchmark with UP HiPE)
Content-Type: text/plain; charset=ISO-8859-2; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

I'm trying to speed up CouchDB (1.0.0) a little (sadly it's too slow for 
a task currently) with HiPE.
What I do is:
setenv ERLC_FLAGS '-smp +native'
and compiling seems to be OK:
/usr/local/bin/erlc -smp +native mochifmt.erl
/usr/local/bin/erlc -smp +native mochifmt_records.erl
/usr/local/bin/erlc -smp +native mochifmt_std.erl
/usr/local/bin/erlc -smp +native mochihex.erl
/usr/local/bin/erlc -smp +native mochijson.erl
/usr/local/bin/erlc -smp +native mochijson2.erl
/usr/local/bin/erlc -smp +native mochinum.erl

but when starting CouchDB, I get a lot of lines like these:
=INFO REPORT==== 21-Jul-2010::15:23:13 ===
<HiPE (v 3.7.5)> Warning: not loading native code for module 
mochiweb_http: it was compiled for an incompatible runtime system; 
please regenerate native code for this runtime system

=INFO REPORT==== 21-Jul-2010::15:23:13 ===
<HiPE (v 3.7.5)> Warning: not loading native code for module 
mochiweb_socket_server: it was compiled for an incompatible runtime 
system; please regenerate native code for this runtime system

I start erl with the options: ERL_START_OPTIONS="-smp enable -sasl 
errlog_type error +K true +A 4"

Any ideas?

BTW, I haven't got this error with SMP disabled, so I could do some 
benchmarks.
The setup consists of three nodes, I write data into A and replicate the 
DB to B and C. Sadly these two lags far behind the written node (see my 
previous e-mail)...
I use python and open a new HTTP session on every write concurrently 
(from twisted) as fast as the CPU allows.

Results:
SMP disabled: 467 writes per second (WPS), 0 connection errors
SMP enabled: 493 WPS, 109 connection errors
setenv ERLC_FLAGS '+native' (and SMP disabled): 472 WPS, 0 connection errors
setenv ERLC_FLAGS '+native \+\{hipe,\[o3\]\}' (and SMP disabled): 472 
WPS, 0 connection errors
setenv ERLC_FLAGS '+native 
\+\{hipe,\[o3,pp_beam,pp_icode,pp_rtl,pp_native\]\}' (and SMP disabled): 
477 WPS, 0 connection errors
setenv ERLC_FLAGS '+native 
\+\{hipe,\[o3,pp_beam,pp_icode,pp_rtl,pp_native\]\}'; 
ERL_START_OPTIONS="-smp enable -sasl errlog_type error +K true" (no SMP, 
no threading): 502 WPS, 0 connection errors

Conclusion (for this task on this OS/machine of course): disabling SMP 
and threads give the best performance boost (this is a dual core 
machine), HiPE gains only a very small amount of performance 
increasement (if I'm correct). Using SMP increases performance, but not 
that much like disabling threads, but it introduces connection errors 
(maybe it's FreeBSD specific).
This test has utilized only one database BTW.