hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-576) Investigate IPC performance
Date Tue, 07 Oct 2008 01:05:44 GMT

    [ https://issues.apache.org/jira/browse/HBASE-576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12637337#action_12637337

stack commented on HBASE-576:

Profiling, looks like multithreaded client is mostly waiting.  Retrying with clean everything
so can get decent figures on throughput.

One thing I noticed profiling is that ~5% of client-side CPU is spent doing convertions of
keys into and out of bytes and booleans and longs going to HTableDescriptor (HTD isRootRegion/isMetaRegion
became hotspot when we did hacks to make binary keys work).  At a minimum, the keys should
be static finals rather than composed on each invocation and we should be putting native types
into our HTD map.   They could even be static final ImmutableBytesWritables.

Another thing to fix in client is sending the method name over as part of the Invocation.
 We've already made it so we don't send class name of parameter by subclassing hadoop RPC
and jiggering ObjectWritable.  Wouldn't take much to do same for method name.  We spend a
bunch of time sending and reading the method name; 10s of percents of CPU.

> Investigate IPC performance
> ---------------------------
>                 Key: HBASE-576
>                 URL: https://issues.apache.org/jira/browse/HBASE-576
>             Project: Hadoop HBase
>          Issue Type: Improvement
>          Components: ipc
>    Affects Versions: 0.1.0, 0.1.1, 0.1.2, 0.2.0
>            Reporter: Jim Kellerman
>            Assignee: stack
>         Attachments: pe.patch
> Turning off all file I/O, and running the PerformanceEvaluation test, of 1,048,576 sequential
writes to HBase managed to achieve only 7,285 IPCs per second.
> Running PerformanceEvaluation sequential write test modified to do an abort instead of
a commit, it was possible to do 68,337 operations per second. We are obviously spending a
lot of time doing IPCs. 
> We need to investigate to find the bottleneck. Marshalling and unmarshalling? Socket
setup and teardown?

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message