hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-576) Investigate IPC performance
Date Fri, 03 Oct 2008 05:50:44 GMT

    [ https://issues.apache.org/jira/browse/HBASE-576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12636537#action_12636537
] 

stack commented on HBASE-576:
-----------------------------

Looking at grizzly, the nio framework, and at our current RPC.

Grizzly drop-in looks straight-forward enough; add to grizzly a protocol that can send and
receive Writables.  Would be good to keep up things like the ping feature in current RPC and
not change the error types, messages, and provocations that we've come to know and love. 
 The bulk of the current RPC code which collects up method name and parameters into an Invoker
Writable would be remain though recast some.

Current RPC has single ipc Client instance per remote host (get client from cache using remote
address and socketfactory hash).  Because only one Client per remote server, then its looking
like request/response's will always run in series (though code would seem to support many
Clients contending sending requests each sleeping till its response comes back).  In-series
is fine for the usual case.  If multiple concurrent HTables in the one VM each trying to do
its lookup on .META. say, the invocations run in series too.  Not the end of the world but
could be better (Running tests to confirm).

One thing I notice is that the one Connection per remote host is propagated up into TableServers
in hbase client.  Need to undo it.

> Investigate IPC performance
> ---------------------------
>
>                 Key: HBASE-576
>                 URL: https://issues.apache.org/jira/browse/HBASE-576
>             Project: Hadoop HBase
>          Issue Type: Improvement
>          Components: ipc
>    Affects Versions: 0.1.0, 0.1.1, 0.1.2, 0.2.0
>            Reporter: Jim Kellerman
>            Assignee: stack
>         Attachments: pe.patch
>
>
> Turning off all file I/O, and running the PerformanceEvaluation test, of 1,048,576 sequential
writes to HBase managed to achieve only 7,285 IPCs per second.
> Running PerformanceEvaluation sequential write test modified to do an abort instead of
a commit, it was possible to do 68,337 operations per second. We are obviously spending a
lot of time doing IPCs. 
> We need to investigate to find the bottleneck. Marshalling and unmarshalling? Socket
setup and teardown?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message