hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: Create common module
Date Sun, 16 Dec 2012 00:57:43 GMT
>> implementation. Now my question is, if the input to be read from HBase, do
>> we write every record from HBase to sequential file format? I am a little
>> confused here.

In NoSQLs cases, they have own partitions based on ranges of key. If
the key is vertex and PartitioningRunner can be skipped, user can use
default partitions of table without re-writing splits.

On Sun, Dec 16, 2012 at 8:50 AM, Edward J. Yoon <edwardyoon@apache.org> wrote:
> +1 Looks great!
>
> On Sun, Dec 16, 2012 at 7:20 AM, Suraj Menon <menonsuraj5@gmail.com> wrote:
>> +1 for hama-io, +0 for hama-commons. Can you look at the patch for HAMA-700
>> and let me know if you have any issues going forward in that direction.
>> Basically it includes a raw format to intermediate binary format
>> conversion, that could be overridden as shown in VertexInputReader
>> implementation. Now my question is, if the input to be read from HBase, do
>> we write every record from HBase to sequential file format? I am a little
>> confused here.
>>
>> Thanks,
>> Suraj
>>
>> On Fri, Dec 14, 2012 at 2:19 AM, Tommaso Teofili
>> <tommaso.teofili@gmail.com>wrote:
>>
>>> thinking about it a bit further, maybe a separate module for IO related
>>> stuff may be worth, so hama-io instead of hama-commons.
>>> Whatever it is I'd like to avoid duplicated code.
>>> Tommaso
>>>
>>>
>>> 2012/12/13 Edward J. Yoon <edwardyoon@apache.org>
>>>
>>> > Otherwise, we have to copy VertexInputReader class to core module or
>>> > add own partitioner to graph package.
>>> >
>>> > In fact, I don't want to put any graph-related code to core module.
>>> > But if we want to support VertexInputReader, I think this is best.
>>> >
>>> > On Fri, Dec 14, 2012 at 4:22 AM, Apurv Verma <dapurv5@gmail.com> wrote:
>>> > > +0  I am not against but I think that would be an overkill for now.
>>> > >
>>> > > --
>>> > > Regards,
>>> > > Apurv Verma
>>> > >
>>> > >
>>> > >
>>> > >
>>> > > On Thu, Dec 13, 2012 at 12:32 PM, Tommaso Teofili
>>> > > <tommaso.teofili@gmail.com> wrote:
>>> > >> that sounds good to me, +1 for a hama-commons module.
>>> > >>
>>> > >> Tommaso
>>> > >>
>>> > >>
>>> > >> 2012/12/13 Edward J. Yoon <edwardyoon@apache.org>
>>> > >>
>>> > >>> Hi,
>>> > >>>
>>> > >>> I propose to create common module to share common classes among
all
>>> > >>> modules.
>>> > >>>
>>> > >>> At the moment, the main reason is to support VertexInputReader.
As I
>>> > >>> mentioned here[1], there's no way to integrate partitioner
without
>>> > >>> merging graph to core or adding graph own partitioner. If we
create
>>> > >>> common module, it'll all be clear. I'm not maven and java expert.
If
>>> > >>> I'm wrong, please correct me.
>>> > >>>
>>> > >>> 1.
>>> > >>>
>>> >
>>> https://issues.apache.org/jira/browse/HAMA-531?focusedCommentId=13483009&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13483009
>>> > >>>
>>> > >>> --
>>> > >>> Best Regards, Edward J. Yoon
>>> > >>> @eddieyoon
>>> > >>>
>>> >
>>> >
>>> >
>>> > --
>>> > Best Regards, Edward J. Yoon
>>> > @eddieyoon
>>> >
>>>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon



-- 
Best Regards, Edward J. Yoon
@eddieyoon

Mime
View raw message