Hi Hadoop use= r,

I have graph data file in the form of edge list

&l= t;Source Vertex_id> <Sink Vertex_id>

I want to assign= each edge a unique ID. In the map function I emit (key,value) as (<Sour= ce Vertex_id>, <Sink Vertex_id>)

In the reducer, for = each value , I am using a combination of static count variable, and task id= (context.getTaskAttemptID().getTaskID().getId()) to generate a unique ID. =

edgeId=3D(localcount <<16)|(taskId << 55);

I am able to generate unique IDs.

My question is if a reducer= fails will this work?=C2=A0

What exactly happens when a redu= cer fails and computed again?

PFA source code for mapper = & reducer.

Thanks

Ravikant

--089e011767f9122487051991ab7c-- --089e011767f912248d051991ab7e Content-Type: text/x-java; charset=US-ASCII; name="MakeGraphUndirectedMapper.java" Content-Disposition: attachment; filename="MakeGraphUndirectedMapper.java" Content-Transfer-Encoding: base64 X-Attachment-Id: f_ibedyo0n0 cGFja2FnZSBpbi5kcmVhbV9sYWIuaGFkb29wUGlwZWxpbmUuY2M7CgppbXBvcnQgamF2YS5pby5J T0V4Y2VwdGlvbjsKCmltcG9ydCBvcmcuYXBhY2hlLmhhZG9vcC5pby5Mb25nV3JpdGFibGU7Cmlt cG9ydCBvcmcuYXBhY2hlLmhhZG9vcC5pby5UZXh0OwppbXBvcnQgb3JnLmFwYWNoZS5oYWRvb3Au bWFwcmVkdWNlLk1hcHBlcjsKaW1wb3J0IG9yZy5hcGFjaGUuaGFkb29wLm1hcHJlZHVjZS5NYXBw ZXIuQ29udGV4dDsKCgovKgogKiBKb2IgSUQgOiAxCiAqIEpvYiBOYW1lIDogRUxfdG9fVUVMCiAq IEpvYiBEZXNjcmlwdGlvbjogQ29udmVydCBkaXJlY3RlZCB0byB1bmRpcmVjdGVkIGdyYXBoCiAq IE1hcCBJbnB1dCBGaWxlOiBFTCAoU05BUCBmaWxlKQogKiBNYXAgSW5wdXQgRm9ybWF0IDpWX3Ny YywgVl9zaW5rCiAqIE1hcCBFbWl0IDpWX3NyYywgW1Zfc2lua10gICYmIFZfc2luaywgW1Zfc3Jj XQogKiBSZWR1Y2VyIEVtaXQ6IFZfc3JjLCBWX3NpbmsgJiYgVl9zaW5rLCBWX3NyYwogKiBSZWR1 Y2VyIE91dHB1dCBGaWxlIDpVRUwKICogTm90ZSA6UmVtb3ZlIGR1cGxpY2F0ZXMKICogCiAqLwoK cHVibGljIGNsYXNzIE1ha2VHcmFwaFVuZGlyZWN0ZWRNYXBwZXIgZXh0ZW5kcyBNYXBwZXI8T2Jq ZWN0ICwgVGV4dCwgTG9uZ1dyaXRhYmxlLCBMb25nV3JpdGFibGU+IHsKCglAT3ZlcnJpZGUKCXBy b3RlY3RlZCB2b2lkIG1hcChPYmplY3Qga2V5LCBUZXh0IHZhbHVlLCBDb250ZXh0IGNvbnRleHQp CgkJCXRocm93cyBJT0V4Y2VwdGlvbiwgSW50ZXJydXB0ZWRFeGNlcHRpb24gewoJCS8vIFRPRE8g QXV0by1nZW5lcmF0ZWQgbWV0aG9kIHN0dWIKCQkKCQlTdHJpbmcgbGluZT12YWx1ZS50b1N0cmlu ZygpOwoJCWlmKCFsaW5lLnN0YXJ0c1dpdGgoIiMiKSl7CgkJU3RyaW5nW10gc3RycyA9IGxpbmUu dHJpbSgpLnNwbGl0KCJcXHMrIik7CgkJbG9uZyBzaW5rSWQ9TG9uZy5wYXJzZUxvbmcoc3Ryc1sx XSk7CiAgICAJbG9uZyBzb3VyY2VJZD1Mb25nLnBhcnNlTG9uZyhzdHJzWzBdKTsKICAgIAkKICAg IAljb250ZXh0LndyaXRlKCBuZXcgTG9uZ1dyaXRhYmxlKHNvdXJjZUlkKSxuZXcgTG9uZ1dyaXRh YmxlKHNpbmtJZCkpOwogICAgCS8qCiAgICAJICogQ29tbWVudCBvdXQgdGhlIGZvbGxvd2luZyBs aW5lIHRvIG1ha2UgaXQgZGlyZWN0ZWQKICAgIAkgKi8KICAgIAljb250ZXh0LndyaXRlKG5ldyBM b25nV3JpdGFibGUoc2lua0lkKSwgbmV3IExvbmdXcml0YWJsZShzb3VyY2VJZCkpOwogICAgCQoJ CX0KCX0KCn0K --089e011767f912248d051991ab7e Content-Type: text/x-java; charset=US-ASCII; name="MakeGraphUndirectedReducer.java" Content-Disposition: attachment; filename="MakeGraphUndirectedReducer.java" Content-Transfer-Encoding: base64 X-Attachment-Id: f_ibedyo291 cGFja2FnZSBpbi5kcmVhbV9sYWIuaGFkb29wUGlwZWxpbmUuY2M7CgppbXBvcnQgamF2YS5pby5J T0V4Y2VwdGlvbjsKaW1wb3J0IGphdmEudXRpbC5IYXNoU2V0OwppbXBvcnQgamF2YS51dGlsLlNl dDsKCmltcG9ydCBvcmcuYXBhY2hlLmhhZG9vcC5pby5Mb25nV3JpdGFibGU7CmltcG9ydCBvcmcu YXBhY2hlLmhhZG9vcC5pby5UZXh0OwppbXBvcnQgb3JnLmFwYWNoZS5oYWRvb3AubWFwcmVkdWNl LlJlZHVjZXI7CmltcG9ydCBvcmcuYXBhY2hlLmhhZG9vcC5tYXByZWR1Y2UuUmVkdWNlci5Db250 ZXh0OwppbXBvcnQgb3JnLmFwYWNoZS5oYWRvb3AubWFwcmVkdWNlLkpvYklEOwovKgogKiBKb2Ig SUQgOiAxCiAqIEpvYiBOYW1lIDogRUxfdG9fVUVMCiAqIEpvYiBEZXNjcmlwdGlvbjogQ29udmVy dCBkaXJlY3RlZCB0byB1bmRpcmVjdGVkIGdyYXBoCiAqIE1hcCBJbnB1dCBGaWxlOiBFTCAoU05B UCBmaWxlKQogKiBNYXAgSW5wdXQgRm9ybWF0IDpWX3NyYywgVl9zaW5rCiAqIE1hcCBFbWl0IDpW X3NyYywgW1Zfc2lua10gICYmIFZfc2luaywgW1Zfc3JjXQogKiBSZWR1Y2VyIEVtaXQ6IFZfc3Jj LCBWX3NpbmsgJiYgVl9zaW5rLCBWX3NyYwogKiBSZWR1Y2VyIE91dHB1dCBGaWxlIDpVRUwKICog Tm90ZSA6UmVtb3ZlIGR1cGxpY2F0ZXMKICogRWRnZUlkIDogZmlyc3QgIDggYml0cyA6IHBhcnRp dGlvbklkLCBuZXh0ICA0MGJpdHMgOiBsb2NhbCBjb3VudCAsIGxhc3QgMTYgYml0cyA6IGxlZnQg Zm9yIGZ1dHVyZSB1c2UoaW5zdGFuY2UgZGF0YSkKICovCgpwdWJsaWMgY2xhc3MgTWFrZUdyYXBo VW5kaXJlY3RlZFJlZHVjZXIgZXh0ZW5kcyBSZWR1Y2VyPCBMb25nV3JpdGFibGUsIExvbmdXcml0 YWJsZSxMb25nV3JpdGFibGUsVGV4dD4gewoKCXB1YmxpYyBzdGF0aWMgbG9uZyBsb2NhbGNvdW50 PSAwTDsKCUBPdmVycmlkZQoJcHJvdGVjdGVkIHZvaWQgcmVkdWNlKExvbmdXcml0YWJsZSBrZXks IEl0ZXJhYmxlPExvbmdXcml0YWJsZT4gdmFsdWVzICwgQ29udGV4dCBjb250ZXh0KQoJCQl0aHJv d3MgSU9FeGNlcHRpb24sIEludGVycnVwdGVkRXhjZXB0aW9uIHsKCQkKCQkKCQkvKmZpbmFsIFN0 cmluZyBqb2JJZCA9IFN5c3RlbS5nZXRlbnYoIm1hcHJlZF9qb2JfaWQiKTsKCQlTdHJpbmdbXSBz dHJzPWpvYklkLnNwbGl0KCJfIik7CgkJCgkJU3RyaW5nIHJlZHVjZXJTdHIgPXN0cnNbc3Rycy5s ZW5ndGggLSAxXTsKCQlsb25nIHJlZHVjZXJJZD1Mb25nLnBhcnNlTG9uZyhyZWR1Y2VyU3RyKTsK CQkKCQkvL1N5c3RlbS5vdXQucHJpbnRsbigiVEVTVCA6IFRBU0tJZCA6Iitjb250ZXh0LmdldFRh c2tBdHRlbXB0SUQoKS5nZXRUYXNrSUQoKS5nZXRJZCgpKyAiIHJlZHVjZXJJRCA6IityZWR1Y2Vy U3RyKTsKCQkqLwoJCUxvbmdXcml0YWJsZSBzb3VyY2VJZD1rZXk7CgkJU2V0PExvbmc+IGFkamxp c3QgPSBuZXcgSGFzaFNldDxMb25nPigpOwoJCWZvcihMb25nV3JpdGFibGUgdiA6IHZhbHVlcyl7 CgkJCWFkamxpc3QuYWRkKHYuZ2V0KCkpOwoJCX0KCQkKCQkJCQoJCWxvbmcgdGFza0lkPWNvbnRl eHQuZ2V0VGFza0F0dGVtcHRJRCgpLmdldFRhc2tJRCgpLmdldElkKCk7CgkJCgkJIGxvbmcgZWRn ZUlkOwoJCQoJCVN0cmluZyBlZGdlU2lua1BhaXI9IiI7CgkJCgkJU3RyaW5nQnVpbGRlciBzYiA9 bmV3IFN0cmluZ0J1aWxkZXIoKTsKCQkKCQlTdHJpbmcgcHJlZml4ID0gIiI7CgkJCgkJZm9yKExv bmcgc2lua0lkIDogYWRqbGlzdCl7CgkJCQoJCQlsb2NhbGNvdW50Kys7CgkJCQoJCQllZGdlSWQ9 KGxvY2FsY291bnQgPDwxNil8KHRhc2tJZCA8PCA1NSk7CgkJCQoJCQkgc2IuYXBwZW5kKHByZWZp eCk7CgkJCSAKCQkJIHByZWZpeCA9ICIsIjsKCQkJCgkJCSBlZGdlU2lua1BhaXI9IGVkZ2VJZCAr IjoiK3NpbmtJZDsKCQkJCgkJCXNiLmFwcGVuZChlZGdlU2lua1BhaXIpOwoJCQkKCQl9CgkKCQlj b250ZXh0LndyaXRlKHNvdXJjZUlkLCBuZXcgVGV4dChzYi50b1N0cmluZygpKSk7Cgl9Cgp9Cg== --089e011767f912248d051991ab7e--