hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-15205) Do not find the replication scope for every WAL#append()
Date Fri, 05 Feb 2016 12:16:40 GMT

     [ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

ramkrishna.s.vasudevan updated HBASE-15205:
-------------------------------------------
    Attachment: ScopeWALEdits_afterpatch.jpg
                HBASE-15205.patch

The patch seems bloated mainly because of lot of test case changes as the WAL.append() signature
is changed. The HTD was being passed to the WAL.append() just to get the scope of the column
families from it. Ideally it can be obtained once per HRegion and pass that to the FSWALEntry
and use it from there. 
First thought of adding some getScope() directly under HTD and return a Map from that but
that is not straight forward because HTD is public and also should we populate the map on
getScope() call every time when it is called?  
Also there was already a TODO asking why HTD is getting passed to WAL.append() so I thought
it made sense to remove it. But this patch avoids lot of garbage.  One more thing to see is
that since replication deals with families we are bound to do lot of copy to store the family
byte[] in some maps. If we could avoid that too then Offheaping work would be greatly benefited.
Will try addressing that later. Patch for QA run. Will update the patch based on this QA results.

> Do not find the replication scope for every WAL#append()
> --------------------------------------------------------
>
>                 Key: HBASE-15205
>                 URL: https://issues.apache.org/jira/browse/HBASE-15205
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>            Priority: Minor
>             Fix For: 2.0.0
>
>         Attachments: HBASE-15205.patch, ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg
>
>
> After the byte[] and char[] the other top contributor for lot of GC (though it is only
2.86%) is the UTF_8.newDecoder.
> This happens because for every WAL append we try to calculate the replication scope associate
with the families associated with the TableDescriptor. I think per WAL append doing this is
very costly and creates lot of garbage. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message