hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Leonardo Gamas <leoga...@jusbrasil.com.br>
Subject Re: Ivory (per term indexing) with Hadoop
Date Thu, 26 Jan 2012 03:57:26 GMT
Luiz,

You could create a class that implements the Writable interface and holds
Doc-ID and TF, let's say DocTF. Then you emit it with context.write(term,
DocTF) in your Mapper class.
Another option is to use a generic Pair<A,B>, that implements Writable, to
hold your data.

P.S.: It's not mandatory to implement Writable. You could use another
serialization framework, but Writable will work without any additional
configuration.

2012/1/25 Luiz Antonio Falaguasta Barbosa <lafbarbosa@gmail.com>

> People,
>
> Only for explain it better, this is figure 4:
>
> [image: image.png]
>
> It seems to be difficult to implement lines 9 to 11 of map method.
>
> Does anybody how to do this? I'd tried to find it in Ivory (
> http://lintool.github.com/Ivory/) and Cloud9 (https://github.com/lintool/*
> *Cloud9 <https://github.com/lintool/Cloud9>) but I didn't get.
>
> Regards,
>
> Luiz
>
>
> 2012/1/25 Luiz Antonio Falaguasta Barbosa <lafbarbosa@gmail.com>
>
>> Hi people,
>>
>> Please, does somebody know where could I find an implementation of per
>> term inverted indexing (Ivory), like that showed in figure 4 of paper
>> http://www.dcs.gla.ac.uk/~richardm/papers/IPM_MapReduce.pdf ?
>>
>> I just would like to take some source code like that
>> http://developer.yahoo.com/hadoop/tutorial/module4.html and change it
>> with the per term indexing.
>>
>> Does somebody have it?
>>
>> Thanks in advance!
>>
>> Regards,
>>
>> Luiz
>
>
>
>
> --
> []s,
>
> Luiz
>



-- 

*Leonardo Gamas*
Software Engineer
T +55 (71) 3494-3514
C +55 (75) 8134-7440
leogamas@jusbrasil.com.br
www.jusbrasil.com.br

Mime
View raw message