couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kocoloski <kocol...@apache.org>
Subject Re: A simple comparative benchmark and look at include_docs performance.
Date Wed, 21 Apr 2010 02:36:48 GMT
Hi Chris, thanks for this report.  So, next steps:

1) What Erlang VM version are you using?  Does it have native code capabilities?  Pasting
the banner from an invocation of `erl` would suffice.

2) Two common compiler optimizations are +native and +inline.  The first one causes the compiler
to generate native machine code instead of BEAM bytecode for a particular module.  You need
to have a VM that supports it (hence my question above).  The second one inlines functions,
as you might have guessed.  json_bin_is_safe/2 is recursive, so I'm not sure +inline makes
a lot of sense in this case.  You can activate these by setting the ERLC_FLAGS environment
variable when you build.

json_bin_is_safe is an old optimization; the VM has improved dramatically in the past 2 years,
and its handling of binaries has changed in subtle ways.  It could be that the assumptions
that went into that code are no longer true.  I'd have to do some experimentation myself to
say for certain.

In the longer term, we've done some work on a fast interface to the excellent Yajl JSON library.
 JSON processing in C has a different performance profile that may be better suited to Couch's
use case.  But again, we'll need to measure.

Hmm, so that was a bit of a brain dump :/  I'd say try out the native code compilation for
mochijson2 if your Erlang build supports it (you should see a [hipe] in the erl banner). 
Cheers,

Adam

On Apr 20, 2010, at 7:48 PM, Chris Stockton wrote:

> Hello,
> 
> After some profiling we found some pretty big slowdowns as you
> suspected in the json encoding. I am very surprised by this, it seems
> that there is a lot of room for improvement in the json encoding side
> of things. I do not know the inner workings of erlang, but seeing that
> this call: mochijson2:json_bin_is_safe/ is 12% of the work of the
> response, but is a very simple string check perhaps it can be inlined
> into the callers? That would be a significant speedup in most
> languages.
> 
> -Chris
> 
> ----------------------------------
> 
> 24> eprof:total_analyse().
> FUNCTION CALLS TIME
> user_drv:io_request/3 266 19 %
> mochijson2:json_bin_is_safe/1 59095133 12 %
> erlang:port_command/2 2445617 3 %
> gen:wait_resp_mon/3 2449385 3 %
> mochijson2:json_encode_string/2 6111299 3 %
> couch_util:to_hex/1 3114803 3 %
> couch_file:read_raw_iolist/3 2272537 3 %
> mochijson2:'-json_encode_proplist/2-fun-0-'/3 3317537 2 %
> gen_server:loop/6 2281063 2 %
> prim_file:drv_command/4 2272768 2 %
> lists:foldl/3 4435469 2 %
> gen_server:handle_msg/5 2281060 2 %
> gen:do_call/4 2449386 2 %
> couch_btree:find_first_gteq/5 3914715 2 %
> prim_file:drv_get_response/1 2272768 2 %
> prim_file:pread/3 2272542 1 %
> couch_btree:lookup_kpnode/5 1380410 1 %
> couch_file:pread_iolist/2 1049994 1 %
> couch_file:handle_call/3 2272598 1 %
> gen_server:call/3 2272615 1 %
> prim_file:reverse/2 2272542 1 %
> prim_file:drv_command/2 2272767 1 %
> prim_file:translate_response/2 2272768 1 %
> couch_file:pread_term/2 1049994 1 %
> couch_btree:less/3 4088828 1 %
> couch_file:pread_binary/2 1049994 1 %
> couch_util:to_digit/1 5537536 1 %
> couch_db_updater:'-init_db/4-fun-0-'/2 4088460 1 %
> prim_file:transform_ldata/4 2272542 1 %
> mochijson2:json_encode/2 3490228 1 %
> erlang:binary_to_term/1 1049995 1 %
> erlang:monitor/2 2449861 1 %
> couch_btree:lookup/3 862756 1 %
> file:pread/3 2272542 1 %
> gen_server:reply/2 2276495 1 %
> couch_db_updater:less_docid/2 4088764 1 %
> lists:splitwith/3 1380530 1 %
> prim_file:transform_ldata/1 2272542 1 %
> couch_file:remove_block_prefixes/2 2618126 1 %
> mochijson2:json_string_is_safe/1 2934384 1 %
> gen:call/4 2449386 1 %
> erlang:demonitor/1 2449849 1 %
> lists:reverse/2 3485939 1 %
> lists:reverse/1 2249330 0 %
> couch_doc:to_json_obj/2 172539 0 %
> prim_file:drv_command/3 2272767 0 %
> erlang:iolist_to_binary/1 2273617 0 %
> couch_file:calculate_total_read_len/2 2272684 0 %
> couch_view:less_json/2 517614 0 %
> couch_db:open_doc_int/3 345101 0 %
> gen_server:decode_msg/8 2281060 0 %
> couch_btree:get_node/2 877445 0 %
> couch_db:doc_meta_info/3 172549 0 %
> erlang:md5_trap/2 0 0 %
> couch_key_tree:get_full_key_paths/4 517831 0 %
> prim_file:drv_get_response/2 2272768 0 %
> erlang:bump_reductions/1 2272768 0 %
> couch_btree:lookup_kvnode/5 345102 0 %
> couch_btree:'-lookup_kpnode/5-fun-0-'/3 690205 0 %
> erlang:'++'/2 2074206 0 %
> couch_doc:to_doc_info_path/1 172853 0 %
> mochijson2:json_encode_proplist/2 519798 0 %
> couch_httpd_view:send_json_view_row/5 172508 0 %
> dict:on_bucket/3 172906 0 %
> couch_key_tree:map_simple/3 518815 0 %
> gen_event:fetch_msg/5 173052 0 %
> gen_event:server_call/4 172890 0 %
> couch_key_tree:map/2 345706 0 %
> couch_httpd_view:view_row_with_doc/3 172485 0 %
> couch_btree:lookup/2 172552 0 %
> couch_doc:to_json_rev/2 172539 0 %
> dict:get_slot/2 346014 0 %
> prim_inet:send/2 172843 0 %
> lists:map/2 652977 0 %
> couch_httpd_view:'-make_view_fold_fun/6-fun-0 172544 0 %


Mime
View raw message