hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3472) MapFile.Reader getClosest() function returns incorrect results when before is true
Date Mon, 02 Jun 2008 18:49:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12601716#action_12601716
] 

stack commented on HADOOP-3472:
-------------------------------

+1 on patch.  I'll commit this in a day or two unless objections.

First I asserted there was indeed a problem with getClosest looking for the key that comes
BEFORE the passed key when the index interval is not 1.  In TestMapFile I did the following:

{code}
Index: src/test/org/apache/hadoop/io/TestMapFile.java
===================================================================
--- src/test/org/apache/hadoop/io/TestMapFile.java      (revision 662508)
+++ src/test/org/apache/hadoop/io/TestMapFile.java      (working copy)
@@ -39,11 +39,11 @@
     FileSystem fs = FileSystem.getLocal(conf);
     Path qualifiedDirName = fs.makeQualified(dirName);
     // Make an index entry for each insertion.
-    MapFile.Writer.setIndexInterval(conf, 1);
+    MapFile.Writer.setIndexInterval(conf, 3);
     MapFile.Writer writer = new MapFile.Writer(conf, fs,
       qualifiedDirName.toString(), Text.class, Text.class);
     // Assert that the index interval is 1
-    assertEquals(1, writer.getIndexInterval());
+    assertEquals(3, writer.getIndexInterval());
     // Add entries up to 100 in intervals of ten.
     final int FIRST_KEY = 10;
     for (int i = FIRST_KEY; i < 100; i += 10) {
@@ -62,7 +62,8 @@
     assertTrue(closest.equals(new Text("60")));
     // Get closest that falls before the passed key: 50
     closest = (Text)reader.getClosest(key, value, true);
-    assertTrue(closest.equals(new Text("50")));
+    assertTrue("Closest should be 50 but is " + closest,
+      closest.equals(new Text("50")));
     // Test get closest when we pass explicit key
     final Text TWENTY = new Text("20");
     closest = (Text)reader.getClosest(TWENTY, value);
{code}

Test fails with:

{code}
Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 0.297 sec
------------- Standard Output ---------------
2008-06-02 11:35:30,889 WARN  util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(51))
- Unable to load native-hadoop library for your platform... using builtin-java classes where
applicable
------------- ---------------- ---------------

Testcase: testGetClosest took 0.295 sec
        FAILED
Closest should be 50 but is 40
junit.framework.AssertionFailedError: Closest should be 50 but is 40
        at org.apache.hadoop.io.TestMapFile.testGetClosest(TestMapFile.java:65)
{code}

I then applied the patch.  It makes more extensive changes to the test than the patch pasted
above cleaning up TestTrue assertions replacing them with clearer TestEquals and adding assertions
that focus on the bug the patch fixes.  The test passes.

> MapFile.Reader getClosest() function returns incorrect results when before is true
> ----------------------------------------------------------------------------------
>
>                 Key: HADOOP-3472
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3472
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: io
>    Affects Versions: 0.16.0, 0.16.1, 0.16.2, 0.16.3, 0.16.4, 0.17.0
>            Reporter: Todd Lipcon
>         Attachments: hadoop_mapfile.2.patch, hadoop_mapfile.patch
>
>
> The MapFile.Reader getClosest() method returns incorrect results due to an error in seekInternal().
The test case in trunk sets the index interval of the test MapFile to 1 which obscures the
issue. There are several other errors in the test case as well that assert incorrect behavior
for getClosest().
> I've got this fixed and tested, and will attach a patch.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message