hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12681) Fold HdfsLocatedFileStatus into HdfsFileStatus
Date Wed, 15 Nov 2017 22:15:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16254316#comment-16254316
] 

Arpit Agarwal commented on HDFS-12681:
--------------------------------------

This change may have broken some MapReduce unit tests. e.g. TestDelegatingInputFormat#testSplitting
{code}
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 4.521 sec <<< FAILURE!
- in org.apache.hadoop.mapreduce.lib.input.TestDelegatingInputFormat
testSplitting(org.apache.hadoop.mapreduce.lib.input.TestDelegatingInputFormat)  Time elapsed:
4.454 sec  <<< ERROR!
java.lang.NullPointerException: null
	at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getBlockIndex(FileInputFormat.java:458)
	at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:412)
	at org.apache.hadoop.mapreduce.lib.input.DelegatingInputFormat.getSplits(DelegatingInputFormat.java:115)
	at org.apache.hadoop.mapreduce.lib.input.TestDelegatingInputFormat.testSplitting(TestDelegatingInputFormat.java:67)
{code}

I am not sure why the breakage occurs. FileInputFormat should not be getting hold of a LocatedFileStatus
object with null locations.

> Fold HdfsLocatedFileStatus into HdfsFileStatus
> ----------------------------------------------
>
>                 Key: HDFS-12681
>                 URL: https://issues.apache.org/jira/browse/HDFS-12681
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Chris Douglas
>            Assignee: Chris Douglas
>            Priority: Minor
>             Fix For: 3.1.0
>
>         Attachments: HDFS-12681.00.patch, HDFS-12681.01.patch, HDFS-12681.02.patch, HDFS-12681.03.patch,
HDFS-12681.04.patch, HDFS-12681.05.patch, HDFS-12681.06.patch, HDFS-12681.07.patch, HDFS-12681.08.patch,
HDFS-12681.09.patch, HDFS-12681.10.patch
>
>
> {{HdfsLocatedFileStatus}} is a subtype of {{HdfsFileStatus}}, but not of {{LocatedFileStatus}}.
Conversion requires copying common fields and shedding unknown data. It would be cleaner and
sufficient for {{HdfsFileStatus}} to extend {{LocatedFileStatus}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message