hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "NITIN VERMA (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-13783) Improve efficiency of WASB over page blobs
Date Tue, 01 Nov 2016 23:07:58 GMT
NITIN VERMA created HADOOP-13783:
------------------------------------

             Summary: Improve efficiency of WASB over page blobs
                 Key: HADOOP-13783
                 URL: https://issues.apache.org/jira/browse/HADOOP-13783
             Project: Hadoop Common
          Issue Type: Bug
          Components: azure
            Reporter: NITIN VERMA


1)	Add telemetry to WASB driver. WASB driver is lack of any log or telemetry which makes trouble
shoot very difficult. For example, we don’t know where is high e2e latency between HBase
and Azure storage came from when Azure storage server latency was very low. Also we don’t
know why WASB can only do 166 IOPs which is way below azure storage 500 IOPs. And we had several
incidents before related to storage latency, because of lacking logs, we couldn’t find the
ownership of the incident quickly.

2)	Resolving the hot spotting issue when WASB driver partition the azure page blobs by changing
the key. Current key design is causing the hot spotting on azure storage. 




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message