hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravikant Dindokar <ravikant.i...@gmail.com>
Subject how to assign unique ID (Long Value) in mapper
Date Thu, 25 Jun 2015 17:09:21 GMT
Hi Hadoop user,

I have a file containing one line for each edge in the graph with two
vertex ids (source & sink).
1    2 (here 1 is source and 2 is sink node for the edge)
1    5
2    3
4    2
4    3
I want to assign a unique Id (Long value )to each edge i.e for each line of
the file.

How to ensure assignment of unique value in distributed mapper process?

Note : File size is large, so using only one reducer is not feasible.


View raw message