hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravikant Dindokar <ravikant.i...@gmail.com>
Subject Re: how to assign unique ID (Long Value) in mapper
Date Fri, 26 Jun 2015 09:29:35 GMT
The problem can be thought as assigning line number for each line. Is there
any inbuilt functionality in hadoop which can do this?

On Fri, Jun 26, 2015 at 1:11 PM, Ravikant Dindokar <ravikant.iisc@gmail.com>
wrote:

> yes , there can be loop in the graph
>
> On Fri, Jun 26, 2015 at 9:09 AM, Harshit Mathur <mathursharp@gmail.com>
> wrote:
>
>> Are there loops in your graph?
>>
>>
>> On Thu, Jun 25, 2015 at 10:39 PM, Ravikant Dindokar <
>> ravikant.iisc@gmail.com> wrote:
>>
>>> Hi Hadoop user,
>>>
>>> I have a file containing one line for each edge in the graph with two
>>> vertex ids (source & sink).
>>> sample:
>>> 1    2 (here 1 is source and 2 is sink node for the edge)
>>> 1    5
>>> 2    3
>>> 4    2
>>> 4    3
>>> I want to assign a unique Id (Long value )to each edge i.e for each line
>>> of the file.
>>>
>>> How to ensure assignment of unique value in distributed mapper process?
>>>
>>> Note : File size is large, so using only one reducer is not feasible.
>>>
>>> Thanks
>>> Ravikant
>>>
>>
>>
>>
>> --
>> Harshit Mathur
>>
>
>

Mime
View raw message