hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HAMA-750) Determine the path of partition files
Date Thu, 11 Apr 2013 01:40:13 GMT
Edward J. Yoon created HAMA-750:

             Summary: Determine the path of partition files
                 Key: HAMA-750
                 URL: https://issues.apache.org/jira/browse/HAMA-750
             Project: Hama
          Issue Type: Bug
          Components: bsp core
            Reporter: Edward J. Yoon

The parent directory of input file was used to determine the path of base directory for partition
files. The problem is when input is multiple files.

  protected BSPJob partition(BSPJob job, int maxTasks) throws IOException {
    String inputPath = job.getConfiguration().get(Constants.JOB_INPUT_DIR);
    Path inputDir = new Path(inputPath);
    if (fs.isFile(inputDir)) {
      inputDir = inputDir.getParent();

    Path partitionDir = new Path(inputDir + "/partitions");

    if (fs.exists(partitionDir)) {
      fs.delete(partitionDir, true);

Simply we can create partitions on temp directory. For example, /tmp/hama-partitions/{$JOB_NAME}/

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message