accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billie Rinaldi (JIRA)" <>
Subject [jira] [Assigned] (ACCUMULO-532) Add BSP input/output formats to client package
Date Mon, 23 Apr 2012 17:54:35 GMT


Billie Rinaldi reassigned ACCUMULO-532:

    Assignee: Edward J. Yoon  (was: Billie Rinaldi)


I've added you to the Accumulo contributers list in JIRA and checked in a modified patch to
contrib/trunk/bsp as a module of a new accumulo-contrib project.

I modified the input and output formats to have the following form because I didn't want to
have so much duplicate code.  This way, if the MR i/o code is changed, the BSP i/o formats
will benefit from it directly.

public class AccumuloInputFormat extends org.apache.accumulo.core.client.mapreduce.AccumuloInputFormat
implements org.apache.hama.bsp.InputFormat<Key,Value>

public class AccumuloOutputFormat extends org.apache.accumulo.core.client.mapreduce.AccumuloOutputFormat
implements org.apache.hama.bsp.OutputFormat<Text,Mutation>

Let me know if you see any issues with this.  It could probably use some more testing.  I
was able to get the unit tests working (even the part commented out in the patch) but I had
to set fake input and output paths.  It seems that BSP doesn't initialize the RecordReader
and RecordWriter unless the configuration options "bsp.input.dir" and "bsp.output.dir" are

bspjob.setInputPath(new Path("test"));
bspjob.setOutputPath(new Path("test"));
> Add BSP input/output formats to client package
> ----------------------------------------------
>                 Key: ACCUMULO-532
>                 URL:
>             Project: Accumulo
>          Issue Type: New Feature
>          Components: client
>    Affects Versions: 1.5.0
>            Reporter: Edward J. Yoon
>            Assignee: Edward J. Yoon
>             Fix For: 1.5.0
>         Attachments: bsp.patch
> I've just wrote basic BSP input formats and its unit tests.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message