hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18023) Log multi-* requests for more than threshold number of rows
Date Tue, 25 Jul 2017 04:03:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099463#comment-16099463
] 

ramkrishna.s.vasudevan commented on HBASE-18023:
------------------------------------------------

Yesterday JIRA was down.
{code}
 (HBASE-18023). Requested Number of Rows: 1380 Client: root//10.66.254.41
2017-07-24 17:53:14,012 WARN  [RpcServer.default.FPBQ.Fifo.handler=97,queue=9,port=16020]
regionserver.RSRpcServices: Large batch operation detected (greater than 1000) (HBASE-18023).
Requested Number of Rows: 1380 Client: root//10.66.254.41
2017-07-24 17:53:14,019 WARN  [RpcServer.default.FPBQ.Fifo.handler=103,queue=4,port=16020]
regionserver.RSRpcServices: Large batch operation detected (greater than 1000) (HBASE-18023).
Requested Number of Rows: 1380 Client: root//10.66.254.41
2017-07-24 17:53:14,020 WARN  [RpcServer.default.FPBQ.Fifo.handler=101,queue=2,port=16020]
regionserver.RSRpcServices: Large batch operation detected (greater than 1000) (HBASE-18023).
Requested Number of Rows: 1380 Client: root//10.66.254.41
2017-07-24 17:53:14,024 WARN  [RpcServer.default.FPBQ.Fifo.handler=100,queue=1,port=16020]
regionserver.RSRpcServices: Large batch operation detected (greater than 1000) (HBASE-18023).
Requested Number of Rows: 1380 Client: root//10.66.254.41
{code}
This was with the default PE tool with 50 and above threads. 

> Log multi-* requests for more than threshold number of rows
> -----------------------------------------------------------
>
>                 Key: HBASE-18023
>                 URL: https://issues.apache.org/jira/browse/HBASE-18023
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: Clay B.
>            Assignee: David Harju
>            Priority: Minor
>             Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2
>
>         Attachments: HBASE-18023-branch-1.3.patch, HBASE-18023-branch-1.patch, HBASE-18023.master.001.patch,
HBASE-18023.master.002.patch, HBASE-18023.master.003.patch, HBASE-18023.master.004.patch
>
>
> Today, if a user happens to do something like a large multi-put, they can get through
request throttling (e.g. it is one request) but still crash a region server with a garbage
storm. We have seen regionservers hit this issue and it is silent and deadly. The RS will
report nothing more than a mysterious garbage collection and exit out.
> Ideally, we could report a large multi-* request before starting it, in case it happens
to be deadly. Knowing the client, user and how many rows are affected would be a good start
to tracking down painful users.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message