hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kihwal Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11379) DFSInputStream may infinite loop requesting block locations
Date Mon, 13 Feb 2017 15:45:41 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15863850#comment-15863850
] 

Kihwal Lee commented on HDFS-11379:
-----------------------------------

The impact of this bug can vary depending on how wide the job is, but it essentially becomes
DDOS using {{getBlockLocations()}}. In one incident, the NN could deal with the rpc load,
but it ran out of space for logging.

> DFSInputStream may infinite loop requesting block locations
> -----------------------------------------------------------
>
>                 Key: HDFS-11379
>                 URL: https://issues.apache.org/jira/browse/HDFS-11379
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs-client
>    Affects Versions: 2.7.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Critical
>             Fix For: 2.8.0, 3.0.0-alpha3
>
>         Attachments: HDFS-11379.branch-2.patch, HDFS-11379.trunk.patch
>
>
> DFSInputStream creation caches file size and initial range of locations.  If the file
is truncated (or replaced) and the client attempts to read outside the initial range, the
client goes into a tight infinite looping requesting locations for the nonexistent range.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message