hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haohui Mai (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-9680) Doing a lsr against WebImageViewer is slow
Date Thu, 21 Jan 2016 23:43:39 GMT
Haohui Mai created HDFS-9680:

             Summary: Doing a lsr against WebImageViewer is slow
                 Key: HDFS-9680
                 URL: https://issues.apache.org/jira/browse/HDFS-9680
             Project: Hadoop HDFS
          Issue Type: Improvement
            Reporter: Haohui Mai

We have experienced a performance issue that doing lsr against the WebImageViewer.

For a fsimage that has around 140m files, it takes around ~35 minutes to do the lsr across
the live cluster, but ~12 hours to do the same operation against the WebImageViewer.

I believe that the root cause is that WebImageViewer decodes the protobuf messages on-demand
which creates a lot of GC pressure. It might be better to decode it at the very beginning.

This message was sent by Atlassian JIRA

View raw message