accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billie Rinaldi (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (ACCUMULO-532) Add BSP input/output formats to client package
Date Mon, 23 Apr 2012 17:54:35 GMT

     [ https://issues.apache.org/jira/browse/ACCUMULO-532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Billie Rinaldi reassigned ACCUMULO-532:
---------------------------------------

    Assignee: Edward J. Yoon  (was: Billie Rinaldi)

Ed,

I've added you to the Accumulo contributers list in JIRA and checked in a modified patch to
contrib/trunk/bsp as a module of a new accumulo-contrib project.

I modified the input and output formats to have the following form because I didn't want to
have so much duplicate code.  This way, if the MR i/o code is changed, the BSP i/o formats
will benefit from it directly.

{noformat}
public class AccumuloInputFormat extends org.apache.accumulo.core.client.mapreduce.AccumuloInputFormat
implements org.apache.hama.bsp.InputFormat<Key,Value>

public class AccumuloOutputFormat extends org.apache.accumulo.core.client.mapreduce.AccumuloOutputFormat
implements org.apache.hama.bsp.OutputFormat<Text,Mutation>
{noformat}

Let me know if you see any issues with this.  It could probably use some more testing.  I
was able to get the unit tests working (even the part commented out in the patch) but I had
to set fake input and output paths.  It seems that BSP doesn't initialize the RecordReader
and RecordWriter unless the configuration options "bsp.input.dir" and "bsp.output.dir" are
set.

{noformat}
bspjob.setInputPath(new Path("test"));
bspjob.setOutputPath(new Path("test"));
{noformat}
                
> Add BSP input/output formats to client package
> ----------------------------------------------
>
>                 Key: ACCUMULO-532
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-532
>             Project: Accumulo
>          Issue Type: New Feature
>          Components: client
>    Affects Versions: 1.5.0
>            Reporter: Edward J. Yoon
>            Assignee: Edward J. Yoon
>             Fix For: 1.5.0
>
>         Attachments: bsp.patch
>
>
> I've just wrote basic BSP input formats and its unit tests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message