incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: [jira] [Assigned] (ACCUMULO-532) Add BSP input/output formats to client package
Date Tue, 24 Apr 2012 01:50:58 GMT
Or, just get rid of this conditions.

On Tue, Apr 24, 2012 at 10:33 AM, Edward J. Yoon <edwardyoon@apache.org> wrote:
> According to CHANGE log, I've added input/output system to BSP
> framework. But, I don't know exactly why we need to check
> (conf.get("bsp.input (or output).dir") != null) conditions when
> initialize record reader/writer objects?
>
> If there's no objection or opinion, I'd like to change like this:
>
> Index: core/src/main/java/org/apache/hama/bsp/BSPPeerImpl.java
> ===================================================================
> --- core/src/main/java/org/apache/hama/bsp/BSPPeerImpl.java     (revision 1329523)
> +++ core/src/main/java/org/apache/hama/bsp/BSPPeerImpl.java     (working copy)
> @@ -185,8 +185,7 @@
>
>     initInput();
>
> -    // just output something when the user configured it
> -    if (conf.get("bsp.output.dir") != null) {
> +    if (!bspJob.getInputFormat().getClass().equals(NullOutputFormat.class)) {
>       Path outdir = new Path(conf.get("bsp.output.dir"),
>           Task.getOutputName(partition));
>       outWriter = bspJob.getOutputFormat().getRecordWriter(fs, bspJob,
> @@ -204,8 +203,7 @@
>
>   @SuppressWarnings("unchecked")
>   public final void initInput() throws IOException {
> -    // just read input if the user defined one
> -    if (conf.get("bsp.input.dir") != null) {
> +    if (!bspJob.getInputFormat().getClass().equals(NullInputFormat.class)) {
>       InputSplit inputSplit = null;
>       // reinstantiate the split
>       try {
>
>
> On Tue, Apr 24, 2012 at 6:39 AM, Edward J. Yoon <edwardyoon@apache.org> wrote:
>> FYI,
>>
>> "Let me know if you see any issues with this.  It could probably use some more testing.
 I was able to get the unit tests working (even the part commented out in the patch) but
I had to set fake input and output paths.  It seems that BSP doesn't initialize the RecordReader
and RecordWriter unless the configuration options "bsp.input.dir" and "bsp.output.dir" are
set."
>>
>> Sent from my iPad
>>
>> Begin forwarded message:
>>
>>> From: "Billie Rinaldi (JIRA)" <jira@apache.org>
>>> Date: April 24, 2012 2:54:35 AM GMT+09:00
>>> To: dev@accumulo.apache.org
>>> Subject: [jira] [Assigned] (ACCUMULO-532) Add BSP input/output formats to client
package
>>> Reply-To: dev@accumulo.apache.org
>>>
>>> Let me know if you see any issues with this.  It could probably use some more
testing.  I was able to get the unit tests working (even the part commented out in the patch)
but I had to set fake input and output paths.  It seems that BSP doesn't initialize the RecordReader
and RecordWriter unless the configuration options "bsp.input.dir" and "bsp.output.dir" are
set.
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon



-- 
Best Regards, Edward J. Yoon
@eddieyoon

Mime
View raw message