hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Sasaki (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6643) org.apache.hadoop.mapred.TestTextInputFormat.testSplitableCodecs failed
Date Sat, 04 Jun 2016 06:43:59 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15315356#comment-15315356
] 

Kai Sasaki commented on MAPREDUCE-6643:
---------------------------------------

It seems to be caused by non checking corner case when byte alignment of data and current
position of stream is same. In this case BZip2 stream can return same block marker same to
previous one. 
I fixed {{BZip2Codec}} and added above test cases.

> org.apache.hadoop.mapred.TestTextInputFormat.testSplitableCodecs failed
> -----------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6643
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6643
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Haibo Chen
>            Assignee: Kai Sasaki
>         Attachments: MAPREDUCE-6643.01.patch
>
>
> Unit test TestTextInputFormat.testSplitableCodecs() failed when the seed is  1313094493.
> Stacktrace
> java.lang.AssertionError: Key in multiple partitions.
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertFalse(Assert.java:64)
> at org.apache.hadoop.mapred.TestTextInputFormat.testSplitableCodecs(TestTextInputFormat.java:223)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message