hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15205) Do not find the replication scope for every WAL#append()
Date Wed, 17 Feb 2016 06:09:18 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149960#comment-15149960
] 

stack commented on HBASE-15205:
-------------------------------

 public NavigableMap<byte[], Integer> getScopes() { should be called getReplicationScopes.

Does it have to be public? Is it only used inside in this class?  It is odd having this internal
detail exported as public method on the Region Interface.


Sometimes you use getScopes. Other times you use this.replicationScopes. The idea is that
you get the scopes once on Region open? Thats great.  

That write of a compaction marker that is in Store doesn't belong there really.... (not your
fault).

Should the replication scopes be in the WALKey and passed in that way when we do an append
(I didn't check so ignore if this is rubbish) Or, looking later, this is where they end up
anyways?  In the WalKey? Do it when we create the WALkey? Or scope is in the WALEntry and
in the WALKey? Yeah, so you are adding to WALEntry the scopes Map but it is in WALKey already?

Will we need HTableDescriptor down in WALEdit going forward? To get configs or something from
it? It is unused currently? Or in the append? Could you carry the scopes in the HTD? The region
puts them into the HTD key/value? The WAL subsystem looks for them else falls back on old
way of getting scopes? Thats a bit hacky maybe.

You are breaking WALActionsListener? Need to release note that.



> Do not find the replication scope for every WAL#append()
> --------------------------------------------------------
>
>                 Key: HBASE-15205
>                 URL: https://issues.apache.org/jira/browse/HBASE-15205
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>            Priority: Minor
>             Fix For: 2.0.0
>
>         Attachments: HBASE-15205.patch, HBASE-15205_1.patch, HBASE-15205_2.patch, HBASE-15205_3.patch,
ScopeWALEdits.jpg, ScopeWALEdits_afterpatch.jpg
>
>
> After the byte[] and char[] the other top contributor for lot of GC (though it is only
2.86%) is the UTF_8.newDecoder.
> This happens because for every WAL append we try to calculate the replication scope associate
with the families associated with the TableDescriptor. I think per WAL append doing this is
very costly and creates lot of garbage. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message