mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <sro...@gmail.com>
Subject Re: Modelling typed vectors?
Date Tue, 12 Oct 2010 21:30:59 GMT
How would you shim your custom Writable into some of the bits of code that
read/write Vectors? that could be hard. Well you can modify it at least but
that's specific to your local copy. You can copy-paste. Which tools are you
thinking of? maybe there's a good tool-specific answer.

On Tue, Oct 12, 2010 at 10:28 PM, Lance Norskog <goksron@gmail.com> wrote:

> Ok. Now, how would one save payloads with the Vector I/O tools?
>
> On Mon, Oct 11, 2010 at 11:30 PM, Sean Owen <srowen@gmail.com> wrote:
> > It is a handy trick, and there's a lot of custom Writables involved in
> > a good MapReduce pipeline, it seems to me. Yes, inside your Writable,
> > use VectorWritable to manage the Vector part.
> >
> > On Tue, Oct 12, 2010 at 4:23 AM, Lance Norskog <goksron@gmail.com>
> wrote:
> >> I have a M/R project where vectors of two different types are in the
> >> same stream. There . They are not matched 1-to-1, in fact there are
> >> different quantities of each type. The mappers may assign 3 of type A
> >> and 7 of type B to one key. The reducer receives both types and
> >> combines them.
> >>
> >> I'm now using a non-Vector design which adds a "payload" to each
> >> vector, giving the type.  This is a really handy trick. In fact I'm
> >> using different payload sets in different stages of a multi-stage M/R
> >> job.
> >>
> >> How would I model this in Mahout? Would I create my own Writeable
> >> tuple with a Vector and a payload object?
> >>
> >> --
> >> Lance Norskog
> >> goksron@gmail.com
> >>
> >
>
>
>
> --
> Lance Norskog
> goksron@gmail.com
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message