hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martin Illecker (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HAMA-774) CompositeInputFormat in Hama
Date Tue, 09 Jul 2013 17:21:49 GMT

    [ https://issues.apache.org/jira/browse/HAMA-774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13701922#comment-13701922
] 

Martin Illecker edited comment on HAMA-774 at 7/9/13 5:20 PM:
--------------------------------------------------------------

Thanks, currently I'm using the same functionality like it were a directory:

{code}
if ((job.get("bsp.input.dir") != null) || (job.get("bsp.join.expr") != null)) {
  // Create the splits for the job
{code}

                
      was (Author: bafu):
    Thanks, currently I'm using the same functionality like it were a directory:

{code}
if ((job.get("bsp.input.dir") != null) || (job.get("bsp.join.expr") != null)) {
  // Create the splits for the job
{code}

It works, but I get the following output for only two input files?

{code}
INFO bsp.FileInputFormat: Total input paths to process : 1
INFO bsp.FileInputFormat: Total input paths to process : 1
INFO bsp.FileInputFormat: Total input paths to process : 1
INFO bsp.FileInputFormat: Total input paths to process : 1
{code}

Also an additional job will be submitted, maybe for partitioning?
Is this additional job really necessary?
                  
> CompositeInputFormat in Hama
> ----------------------------
>
>                 Key: HAMA-774
>                 URL: https://issues.apache.org/jira/browse/HAMA-774
>             Project: Hama
>          Issue Type: New Feature
>          Components: bsp core
>    Affects Versions: 0.6.2
>            Reporter: Martin Illecker
>            Assignee: Martin Illecker
>            Priority: Minor
>             Fix For: 0.6.3
>
>         Attachments: HAMA-774.patch
>
>
> *Adapting Hadoop CompositeInputFormat to Hama.*
> Adapt the CompositeInputFormat from Hadoop and integrate it into Hama.
> e.g., Useful for matrix multiplication example by doing a inner join of two matrices.
> Therefore the Hadoop join package has been adapted within *org.apache.hama.bsp.join*.
> Finally I would need some help to integrate the CompositeInputFormat into Hama by using
> {code} 
> job.setInputFormat(CompositeInputFormat.class);
> job.set("bsp.join.expr", CompositeInputFormat.compose("inner",
>    SequenceFileInputFormat.class, aPath, bPath));
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message