hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeongdae Kim (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-15832) memory leak in FSHLog.
Date Thu, 19 May 2016 03:47:12 GMT

     [ https://issues.apache.org/jira/browse/HBASE-15832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jeongdae Kim updated HBASE-15832:
---------------------------------
    Description: 
FSHLog module uses a map to reuse SyncFuture objects, and assumes that this map will be used
by RPC Handler threads only. but, in some cases, this assumption is wrong. 
for example, if some coprocessors are registered, and these coprocessors uses CoprocessorHConnection
insteadof HConnection, and request some puts/ or deletes throgh CoprocessorHConnection, all
mutations will be handled by hconnection's batchPool, not RPC Handlers. because hconnection's
batchPool is dynamically growing or shrinking, all new threads in hconnection are put to the
map in FSHLog, and this map will grow continuously.

in attached image file, the map to reuse SyncFuture occupies about 4GB memory and has (almost
all) entries holding hconnection's thread.

  was:
FSHLog module uses a map to reuse SyncFuture objects, and assumes that this map will be used
by RPC Handler threads only. but, in some cases, this assumption is wrong. 
for example, if some coprocessors are registered, and these coprocessors uses CoprocessorHConnection
insteadof HConnection, and request some puts/ or deletes throgh CoprocessorHConnection, all
mutations will be handled by hconnection's batchPool, not RPC Handlers. because hconnection's
batchPool is dynamically growing or shrinking, all new threads in hconnection are put to the
map in FSHLog, and this map will grow continuously.

in attached image file, the map to reuse SyncFuture accupies about 4GB memory and has (almost
all) entries holding hconnection's thread.


> memory leak in FSHLog.
> ----------------------
>
>                 Key: HBASE-15832
>                 URL: https://issues.apache.org/jira/browse/HBASE-15832
>             Project: HBase
>          Issue Type: Bug
>          Components: hbase
>    Affects Versions: 1.1.2
>            Reporter: Jeongdae Kim
>            Assignee: Vladimir Rodionov
>         Attachments: HBASE-15832-v1.patch, Screenshot-Java - -home-jeongdae-work-regionserver_jmap_104p_sn5_20160509-sn5_heap.hprof
- Eclipse -1.png
>
>
> FSHLog module uses a map to reuse SyncFuture objects, and assumes that this map will
be used by RPC Handler threads only. but, in some cases, this assumption is wrong. 
> for example, if some coprocessors are registered, and these coprocessors uses CoprocessorHConnection
insteadof HConnection, and request some puts/ or deletes throgh CoprocessorHConnection, all
mutations will be handled by hconnection's batchPool, not RPC Handlers. because hconnection's
batchPool is dynamically growing or shrinking, all new threads in hconnection are put to the
map in FSHLog, and this map will grow continuously.
> in attached image file, the map to reuse SyncFuture occupies about 4GB memory and has
(almost all) entries holding hconnection's thread.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message