Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 24 Oct 2017 21:20:01 +0000 (UTC)
From: "Ting Dai (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: <JIRA.13111808.1508879937000.50458.1508880001147@Atlassian.JIRA>
In-Reply-To: <JIRA.13111808.1508879937000@Atlassian.JIRA>
References: <JIRA.13111808.1508879937000@Atlassian.JIRA> <JIRA.13111808.1508879937715@jira-lw-us.apache.org>
Subject: [jira] [Updated] (MAPREDUCE-6990) FileInputStream.skip function can
 return 0 when the file is corrupted, causing an infinite loop
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 24 Oct 2017 21:20:08 -0000


     [ https://issues.apache.org/jira/browse/MAPREDUCE-6990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ting Dai updated MAPREDUCE-6990:
--------------------------------
    Description: 
When file is corrupted, for example, bad encoding, [Yarn-2724](https://issues.apache.org/jira/browse/YARN-2724), the FileInputStream can return 0, causing the while loop in TaskLog$Reader become infinite.

{code:java}
    public Reader(TaskAttemptID taskid, LogName kind,  long start, long end, boolean isCleanup) throws IOException {
      // find the right log file
      LogFileDetail fileDetail = getLogFileDetail(taskid, kind, isCleanup);
      // calculate the start and stop
      long size = fileDetail.length;
      if (start < 0) {
        start += size + 1;
      }
      if (end < 0) {
        end += size + 1;
      }
      start = Math.max(0, Math.min(start, size));
      end = Math.max(0, Math.min(end, size));
      start += fileDetail.start;
      end += fileDetail.start;
      bytesRemaining = end - start;
      String owner = obtainLogDirOwner(taskid);
      file = SecureIOUtils.openForRead(new File(fileDetail.location, kind.toString()),  owner, null);
      // skip upto start
      long pos = 0;
      while (pos < start) {
        long result = file.skip(start - pos);
        if (result < 0) {
          bytesRemaining = 0;
          break;
        }
        pos += result;
      }
    }
{code}

Similar bugs are [Hadoop-8614](https://issues.apache.org/jira/browse/HADOOP-8614), [Yarn-2905](https://issues.apache.org/jira/browse/YARN-2905)

  was:
When file is corrupted, for example, bad encoding, [Yarn-2724](https://issues.apache.org/jira/browse/YARN-2724), the FileInputStream can return 0, causing the while loop in TaskLog$Reader become infinite.

{code:java}
// Some comments here
    public Reader(TaskAttemptID taskid, LogName kind, 
                  long start, long end, boolean isCleanup) throws IOException {
      // find the right log file
      LogFileDetail fileDetail = getLogFileDetail(taskid, kind, isCleanup);
      // calculate the start and stop
      long size = fileDetail.length;
      if (start < 0) {
        start += size + 1;
      }
      if (end < 0) {
        end += size + 1;
      }
      start = Math.max(0, Math.min(start, size));
      end = Math.max(0, Math.min(end, size));
      start += fileDetail.start;
      end += fileDetail.start;
      bytesRemaining = end - start;
      String owner = obtainLogDirOwner(taskid);
      file = SecureIOUtils.openForRead(new File(fileDetail.location, kind.toString()), 
          owner, null);
      // skip upto start
      long pos = 0;
      while (pos < start) {
        long result = file.skip(start - pos);
        if (result < 0) {
          bytesRemaining = 0;
          break;
        }
        pos += result;
      }
    }
{code}

Similar bugs are [Hadoop-8614](https://issues.apache.org/jira/browse/HADOOP-8614), [Yarn-2905](https://issues.apache.org/jira/browse/YARN-2905)


> FileInputStream.skip function can return 0 when the file is corrupted, causing an infinite loop
> -----------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6990
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6990
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 0.23.0
>            Reporter: Ting Dai
>
> When file is corrupted, for example, bad encoding, [Yarn-2724](https://issues.apache.org/jira/browse/YARN-2724), the FileInputStream can return 0, causing the while loop in TaskLog$Reader become infinite.
> {code:java}
>     public Reader(TaskAttemptID taskid, LogName kind,  long start, long end, boolean isCleanup) throws IOException {
>       // find the right log file
>       LogFileDetail fileDetail = getLogFileDetail(taskid, kind, isCleanup);
>       // calculate the start and stop
>       long size = fileDetail.length;
>       if (start < 0) {
>         start += size + 1;
>       }
>       if (end < 0) {
>         end += size + 1;
>       }
>       start = Math.max(0, Math.min(start, size));
>       end = Math.max(0, Math.min(end, size));
>       start += fileDetail.start;
>       end += fileDetail.start;
>       bytesRemaining = end - start;
>       String owner = obtainLogDirOwner(taskid);
>       file = SecureIOUtils.openForRead(new File(fileDetail.location, kind.toString()),  owner, null);
>       // skip upto start
>       long pos = 0;
>       while (pos < start) {
>         long result = file.skip(start - pos);
>         if (result < 0) {
>           bytesRemaining = 0;
>           break;
>         }
>         pos += result;
>       }
>     }
> {code}
> Similar bugs are [Hadoop-8614](https://issues.apache.org/jira/browse/HADOOP-8614), [Yarn-2905](https://issues.apache.org/jira/browse/YARN-2905)


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org