hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Praveen Kumar K J V S <praveenkjvs.develo...@gmail.com>
Subject Re: Map reduce example - is it possible?
Date Tue, 03 Apr 2012 00:17:25 GMT
Hey Phatak,

With your way the mapper o/p will be some thing like

<W1_W2 0>
<W1_W3 1>
<W1_W4 0>
......

Bu then this way I will miss the pair <W3_W5 0>

Correct me if I am wrong.

Thanks,
Praveen

On Mon, Apr 2, 2012 at 12:52 PM, madhu phatak <phatak.dev@gmail.com> wrote:

> Hi,
>  Yes using Map/Reduce its possible .
>
>   1. In Mapper,
>  Read the words in line and make pair with calculated distance. Output key
> of mapper will be the this word pair and value will be distance
>
>  2.In Reducer,
>    Just sum the distance for each pair.
>
> On Wed, Mar 28, 2012 at 7:42 AM, Praveen Kumar K J V S <
> praveenkjvs.developer@gmail.com> wrote:
>
>> Hi All,
>>
>> I have a file in HDFS spanning across many blocks. Say the file has many
>> words in it from W1, W2 , W3 ...Wn.
>>
>> I want to find the edit distance between all pairs of words. Is this is
>> possible in Mapreduce.
>>
>> For example I have 2 blocks for the file: And an edit distance function
>> which returns value between 0 to 1(0 means two words are very close t0 each
>> other). Say edit distance between even and odd words is 1
>>
>> Block 1 has the words W1, W2, W3, W4
>> Block 2 has the words W1, W2, W5, W6
>>
>> Now is there a way to find all pairs edit distance. If so how?
>>
>> Thanks,
>> Praveen
>>
>
>
>
> --
> https://github.com/zinnia-phatak-dev/Nectar
>
>

Mime
View raw message