hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: Issues about Partitioning and Record converter
Date Sun, 05 May 2013 20:09:00 GMT
I think 'record converter' should be removed. It's not good idea.
Moreover, it's unnecessarily complex. To keep vertex input reader, we
can move related classes into common module.

Let's go with my original plan.

On Sat, May 4, 2013 at 9:32 AM, Edward J. Yoon <edwardyoon@apache.org> wrote:
> Hi all,
> I'm reading our old discussions about record converter, superstep
> injection, and common module:
> - http://markmail.org/message/ol32pp2ixfazcxfc
> - http://markmail.org/message/xwtmfdrag34g5xc4
> To clarify goals and objectives:
> 1. A parallel input partition is necessary for obtaining scalability
> and elasticity of a Bulk Synchronous Parallel processing (It's not a
> memory issue, or Disk/Spilling Queue, or HAMA-644. Please don't
> shake).
> 2. Input partitioning should be handled at BSP framework level, and it
> is for every Hama jobs, not only for Graph jobs.
> 3. Unnecessary I/O Overhead need to be avoided, and NoSQLs input also
> should be considered.
> The current problem is that every input of graph jobs should be
> rewritten on HDFS. If you have a good idea, Please let me know.
> --
> Best Regards, Edward J. Yoon
> @eddieyoon

Best Regards, Edward J. Yoon

View raw message