mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lance Norskog <goks...@gmail.com>
Subject Re: Modelling typed vectors?
Date Tue, 12 Oct 2010 21:28:18 GMT
Ok. Now, how would one save payloads with the Vector I/O tools?

On Mon, Oct 11, 2010 at 11:30 PM, Sean Owen <srowen@gmail.com> wrote:
> It is a handy trick, and there's a lot of custom Writables involved in
> a good MapReduce pipeline, it seems to me. Yes, inside your Writable,
> use VectorWritable to manage the Vector part.
>
> On Tue, Oct 12, 2010 at 4:23 AM, Lance Norskog <goksron@gmail.com> wrote:
>> I have a M/R project where vectors of two different types are in the
>> same stream. There . They are not matched 1-to-1, in fact there are
>> different quantities of each type. The mappers may assign 3 of type A
>> and 7 of type B to one key. The reducer receives both types and
>> combines them.
>>
>> I'm now using a non-Vector design which adds a "payload" to each
>> vector, giving the type.  This is a really handy trick. In fact I'm
>> using different payload sets in different stages of a multi-stage M/R
>> job.
>>
>> How would I model this in Mahout? Would I create my own Writeable
>> tuple with a Vector and a payload object?
>>
>> --
>> Lance Norskog
>> goksron@gmail.com
>>
>



-- 
Lance Norskog
goksron@gmail.com

Mime
View raw message