hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashish Singhi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14938) Limit the number of znodes for ZK in bulk loaded hfile replication
Date Fri, 25 Dec 2015 11:01:49 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15071505#comment-15071505

Ashish Singhi commented on HBASE-14938:

Thanks for the comments.

bq. Can we avoid having 2 loop code.. When totalNoOfFiles <maxZnodesPerRequest, I initially
thought u will get issue in 1st loop.. But then totalNoOfRequests will be 0 and so we wont
go into 1st loop at all.. I would say readability wise this can be improved.
I had that in my first patch, then as per Ted suggestion for better readability I combined
them. Anyways when files are less than the max value we will have to do a if condition check
to avoid the first loop, that condition check we have now in for loop, is that bad ?

bq. totalNoOfRequests = 1 in that case and every case we have one loop doing the work?
No, for example when we have totalNoOfFiles=6000 we will execute both the loops.

bq. How we have selected the def value to be 5000? We will get these many files to be added
in one call?
Mostly not, this is only to control the max data size limit allowed in a ZK request (as mentioned
in description).


bq. When would totalNoOfRequests be 1 ?
When totalNoOfFiles is equal to maxZnodesPerRequest.

> Limit the number of znodes for ZK in bulk loaded hfile replication
> ------------------------------------------------------------------
>                 Key: HBASE-14938
>                 URL: https://issues.apache.org/jira/browse/HBASE-14938
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Ashish Singhi
>            Assignee: Ashish Singhi
>             Fix For: 2.0.0, 1.3.0
>         Attachments: HBASE-14938(1).patch, HBASE-14938-v1.patch, HBASE-14938.patch
> In ZK the maximum allowable size of the data array is 1 MB. Until we have fixed HBASE-10295
we need to handle this.
> Approach to this problem will be discussed in the comments section.
> Note: We have done internal testing with more than 3k nodes in ZK yet to be replicated.

This message was sent by Atlassian JIRA

View raw message