hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-2492) [MAPREDUCE] The new MapReduce API should make available task's progress to the task
Date Sun, 22 May 2011 06:00:49 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Amar Kamat updated MAPREDUCE-2492:
----------------------------------

    Attachment: MAPREDUCE-2492-v1.5.patch

Attaching a new patch with custom (low) default Filesystem block size. The only difference
between the Hudson run and the run on my local box is the number of splits (hence maps). Maybe
Hudson sets the default block size to a low value (default being 32mb). I hope that Hudson
respects the custom value for default block size set by the test case. 

The diff between the two patches is as follows
{code}
318c318
< @@ -0,0 +1,208 @@
---
> @@ -0,0 +1,217 @@
359a360,361
> +  // using a default block size of 1k
> +  private static final long DEFAULT_BLOCK_SIZE = 1024;
365c367,371
< +    fs = FileSystem.getLocal(new Configuration());
---
> +    Configuration conf = new Configuration();
> +    // set the default block size
> +    conf.setLong("fs.local.block.size", DEFAULT_BLOCK_SIZE);
> +    
> +    fs = FileSystem.getLocal(conf);
367a374,376
> +    
> +    assertEquals("Invalid block size", 
> +                 DEFAULT_BLOCK_SIZE, fs.getDefaultBlockSize());
{code}

> [MAPREDUCE] The new MapReduce API should make available task's progress to the task
> -----------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2492
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2492
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: task
>    Affects Versions: 0.23.0
>            Reporter: Amar Kamat
>            Assignee: Amar Kamat
>         Attachments: MAPREDUCE-2492-v1.3.patch, MAPREDUCE-2492-v1.4.patch, MAPREDUCE-2492-v1.5.patch
>
>
> There is no way to get the task's current progress in the new MapReduce API. It would
be nice to make it available so that the task (map/reduce) can use it. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message