Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 31BB618F14 for ; Fri, 26 Jun 2015 09:29:43 +0000 (UTC) Received: (qmail 27332 invoked by uid 500); 26 Jun 2015 09:29:38 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 27223 invoked by uid 500); 26 Jun 2015 09:29:38 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 27213 invoked by uid 99); 26 Jun 2015 09:29:38 -0000 Received: from Unknown (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 26 Jun 2015 09:29:38 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id BFB2A1A6149 for ; Fri, 26 Jun 2015 09:29:37 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.879 X-Spam-Level: ** X-Spam-Status: No, score=2.879 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id obnfIChpbb1m for ; Fri, 26 Jun 2015 09:29:37 +0000 (UTC) Received: from mail-lb0-f181.google.com (mail-lb0-f181.google.com [209.85.217.181]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with ESMTPS id 87DB94C0E8 for ; Fri, 26 Jun 2015 09:29:36 +0000 (UTC) Received: by lbbwc1 with SMTP id wc1so60872691lbb.2 for ; Fri, 26 Jun 2015 02:29:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=Y52renr+/uR0ari/buVGqEBI9EvZg0zh0wvtF9J10k8=; b=oXKOiU7ORvgLjQf3Dxx8gizwHSqgDngCPOddA04BRC/vBBHDCybGb5MZ8EMlsWbrNS E+FPlp7vpVgK3jSpFBmC9GnS6sLhlb87TvWjE94jhm/tolD/yE2BBA2vqAALo5KLrh27 yOU4Octjgg1FW79eZgHSSi82EQ9JxB5fdmYopxRDgwBH9iFJkCyA7C3dn6JqHG5W6zw2 Z/Ukhj7MI41+TvwIWNMN36NYVGGGvwPFqfDi7kzkI4Gf/NRlTYSQoFAxEx1XujJDJ5Rh MMKo9geu1tNfHOACFLmEcQB+q76qWIiuBJJ8ez+FnNnfxhhwW06twi95IRQVk0lFVynP bjmQ== MIME-Version: 1.0 X-Received: by 10.112.209.106 with SMTP id ml10mr697454lbc.112.1435310975526; Fri, 26 Jun 2015 02:29:35 -0700 (PDT) Received: by 10.25.78.13 with HTTP; Fri, 26 Jun 2015 02:29:35 -0700 (PDT) In-Reply-To: References: Date: Fri, 26 Jun 2015 14:59:35 +0530 Message-ID: Subject: Re: how to assign unique ID (Long Value) in mapper From: Ravikant Dindokar To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=001a11c25ea0d6ad4d0519686098 --001a11c25ea0d6ad4d0519686098 Content-Type: text/plain; charset=UTF-8 The problem can be thought as assigning line number for each line. Is there any inbuilt functionality in hadoop which can do this? On Fri, Jun 26, 2015 at 1:11 PM, Ravikant Dindokar wrote: > yes , there can be loop in the graph > > On Fri, Jun 26, 2015 at 9:09 AM, Harshit Mathur > wrote: > >> Are there loops in your graph? >> >> >> On Thu, Jun 25, 2015 at 10:39 PM, Ravikant Dindokar < >> ravikant.iisc@gmail.com> wrote: >> >>> Hi Hadoop user, >>> >>> I have a file containing one line for each edge in the graph with two >>> vertex ids (source & sink). >>> sample: >>> 1 2 (here 1 is source and 2 is sink node for the edge) >>> 1 5 >>> 2 3 >>> 4 2 >>> 4 3 >>> I want to assign a unique Id (Long value )to each edge i.e for each line >>> of the file. >>> >>> How to ensure assignment of unique value in distributed mapper process? >>> >>> Note : File size is large, so using only one reducer is not feasible. >>> >>> Thanks >>> Ravikant >>> >> >> >> >> -- >> Harshit Mathur >> > > --001a11c25ea0d6ad4d0519686098 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
The problem can be thought as assigning line number for ea= ch line. Is there any inbuilt functionality in hadoop which can do this?

On Fri, J= un 26, 2015 at 1:11 PM, Ravikant Dindokar <ravikant.iisc@gmail.com> wrote:
yes= , there can be loop in the graph

On Fri, J= un 26, 2015 at 9:09 AM, Harshit Mathur <mathursharp@gmail.com><= /span> wrote:
Are there = loops in your graph?


=
On Thu, Jun 25, 2015 at 10:39 PM, Ravikant Dindo= kar <ravikant.iisc@gmail.com> wrote:
Hi Hadoop u= ser,

I have a file containing one line for each edge in the gr= aph with two vertex ids (source & sink).
sample:
1=C2=A0=C2= =A0=C2=A0 2 (here 1 is source and 2 is sink node for the edge)
1= =C2=A0=C2=A0=C2=A0 5
2=C2=A0=C2=A0=C2=A0 3
4=C2=A0=C2=A0=C2=A0 2
4= =C2=A0=C2=A0=C2=A0 3
I want to assign a unique Id (Long value )to = each edge i.e for each line of the file.

How to ensure assignm= ent of unique value in distributed mapper process?

Note : File= size is large, so using only one reducer is not feasible.

Tha= nks
Ravikant



--
Harshit Mathur


--001a11c25ea0d6ad4d0519686098--