hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liang Xie (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6591) while loop is executed tens of thousands of times in Hedged Read
Date Fri, 27 Jun 2014 03:52:24 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14045491#comment-14045491

Liang Xie commented on HDFS-6591:

bq. so it appears that we never copy to buf in this code path. I only see a copy to buf inside
the else path, when a hedged read actually happens. Did I miss something?
Because in the if path, we passed this byte[] bb (ByteBuffer.wrap(buf, offset, len)), it came
from the original end-user input parameter. that's why we don't need to do a explicit copy:)

bq. Let's set isHedgedRead = false in a JUnit After method instead of at the end of individual
seems you read "HDFS-6591.txt" patch, not "HDFS-6591-v2.txt" ?:)  i had moved it into a JUnit
Before method per [~ajisakaa]'s suggestion, i assume it has a similar effect like moving into
a After method, right? Please correct me if i'm wrong.

bq. Let's guarantee that output and input get closed. We can put the cleanup in a finally
block and call IOUtils#cleanup.
yes, let me do it in patch v3.

bq. Is it actually useful to add the new metric for HedgedReadOpsLoopNum
During debugging, i feel it's very convenient to catch/verify some unnecessary loops, after
a quick thinking, i didn't find any  other easier choice to catch such case. How about keeping
it there. when we find it's a hotspot or this feature is stable enough, then remove it ? 
(I can add a comment in v3 patch, so we'll easy to recall those stuff)

> while loop is executed tens of thousands of times  in Hedged  Read
> ------------------------------------------------------------------
>                 Key: HDFS-6591
>                 URL: https://issues.apache.org/jira/browse/HDFS-6591
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs-client
>    Affects Versions: 2.4.0
>            Reporter: LiuLei
>            Assignee: Liang Xie
>         Attachments: HDFS-6591-v2.txt, HDFS-6591.txt, LoopTooManyTimesTestCase.patch
> I download hadoop-2.4.1-rc1 code from http://people.apache.org/~acmurthy/hadoop-2.4.1-rc1/,
 I test the  Hedged  Read. I find the while loop in hedgedFetchBlockByteRange method is executed
tens of thousands of times.

This message was sent by Atlassian JIRA

View raw message