hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicolas Liochon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-9869) Optimize HConnectionManager#getCachedLocation
Date Thu, 14 Nov 2013 12:23:22 GMT

    [ https://issues.apache.org/jira/browse/HBASE-9869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13822368#comment-13822368
] 

Nicolas Liochon commented on HBASE-9869:
----------------------------------------

bq. See any difference? Whats the math like? How large is the Map if 1M regions in it? We
remove when region is bad but ones we don't access could stick around for ever. I suppose
that serves us right if we don't access them. Folks like Lars are sensitive about clients
being well-behaved especially when embedded in an apps server. I looked around for a simple
LRU – the guava one – but we need SortedMap.


My feeling is that if you have a table with 1 million regions  you need to pay the price for
this: i.e. have enough memory on the client.
Let me do the math & test it however.
We can as well do something in the middle: do a LRU on the first map, the one on the tableName.

> Optimize HConnectionManager#getCachedLocation
> ---------------------------------------------
>
>                 Key: HBASE-9869
>                 URL: https://issues.apache.org/jira/browse/HBASE-9869
>             Project: HBase
>          Issue Type: Bug
>          Components: Client
>    Affects Versions: 0.98.0, 0.96.0
>            Reporter: Nicolas Liochon
>            Assignee: Nicolas Liochon
>             Fix For: 0.98.0, 0.96.1
>
>         Attachments: 9869.v1.patch, 9869.v1.patch, 9869.v2.patch
>
>
> It javadoc says: "TODO: This method during writing consumes 15% of CPU doing lookup".
This is still true, says Yourkit. With 0.96, we also spend more time in these methods. We
retry more, and the AsyncProcess calls it in parallel.
> I don't have the patch for this yet, but I will spend some time on it.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message