Return-Path: X-Original-To: apmail-giraph-dev-archive@www.apache.org Delivered-To: apmail-giraph-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7844D1042D for ; Fri, 3 May 2013 00:42:16 +0000 (UTC) Received: (qmail 46466 invoked by uid 500); 3 May 2013 00:42:16 -0000 Delivered-To: apmail-giraph-dev-archive@giraph.apache.org Received: (qmail 46434 invoked by uid 500); 3 May 2013 00:42:16 -0000 Mailing-List: contact dev-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@giraph.apache.org Delivered-To: mailing list dev@giraph.apache.org Received: (qmail 46421 invoked by uid 500); 3 May 2013 00:42:16 -0000 Delivered-To: apmail-incubator-giraph-dev@incubator.apache.org Received: (qmail 46415 invoked by uid 99); 3 May 2013 00:42:16 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 May 2013 00:42:16 +0000 Date: Fri, 3 May 2013 00:42:15 +0000 (UTC) From: "Wang Xu (JIRA)" To: giraph-dev@incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (GIRAPH-652) Not support only with edge input data in some algorithms. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/GIRAPH-652?page=3Dcom.atlassia= n.jira.plugin.system.issuetabpanels:all-tabpanel ] Wang Xu updated GIRAPH-652: --------------------------- Description:=20 There are two types of input vertex-centric and edge-centric. Some algorith= ms need vertex centric data, such as pairs of vertex ids and initial page r= anks. Some algorithms only look at edges. For example, connected components= can be run without any vertex values. In some conditions , we have only edge-centric data , have not vertex data = . Convert edge-centric data to vertex-centric data is not bad, but is not = better. In my job , I modify EdgeInputSplitsCallable.java to support load vertex au= to when loading edge=E3=80=82 My mainly modify is=EF=BC=9A {code:java}=20 graphState.getWorkerClientRequestProcessor().sendEdgeRequest( sourceId, readerEdge); context.progress(); // do this before potential data transfer ++inputSplitEdgesLoaded; =20 //modify begin Vertex vertex =3D this.configuration.createVertex(); = =20 if (vertex.getValue() =3D=3D null) { =09 vertex.setValue(configuration.createVertexValue()); } vertex.setConf(configuration); vertex.setGraphState(graphState); =20 vertex.initialize(edgeReader.getCurrentEdge().getTargetVertexId(), ve= rtex.getValue()); =20 PartitionOwner partitionOwner =3D=20 bspServiceWorker.getVertexPartitionOwner(vertex.getId()); graphState.getWorkerClientRequestProcessor().sendVertexRequest( partitionOwner, vertex); =20 // modify end // Update status every VERTICES_UPDATE_PERIOD edges if (inputSplitEdgesLoaded % VERTICES_UPDATE_PERIOD =3D=3D 0) { totalEdgesMeter.mark(VERTICES_UPDATE_PERIOD);=20 {code}=20 After do it =EF=BC=8C my giraph job can only load edge data in any examples= =E3=80=82 I wish a similar features can be add into in future version=E3=80=82 was: There are two types of input vertex-centric and edge-centric. Some algorith= ms need vertex centric data, such as pairs of vertex ids and initial page r= anks. Some algorithms only look at edges. For example, connected components= can be run without any vertex values. In some conditions , we have only edge-centric data , have not vertex data = . Convert edge-centric data to vertex-centrice data is not bad, but is bet= ter. In my job , I modify EdgeInputSplitsCallable.java to support load vertex au= to when loading edge=E3=80=82 My mainly modify is=EF=BC=9A {code:java}=20 graphState.getWorkerClientRequestProcessor().sendEdgeRequest( sourceId, readerEdge); context.progress(); // do this before potential data transfer ++inputSplitEdgesLoaded; =20 //modify begin Vertex vertex =3D this.configuration.createVertex(); = =20 if (vertex.getValue() =3D=3D null) { =09 vertex.setValue(configuration.createVertexValue()); } vertex.setConf(configuration); vertex.setGraphState(graphState); =20 vertex.initialize(edgeReader.getCurrentEdge().getTargetVertexId(), ve= rtex.getValue()); =20 PartitionOwner partitionOwner =3D=20 bspServiceWorker.getVertexPartitionOwner(vertex.getId()); graphState.getWorkerClientRequestProcessor().sendVertexRequest( partitionOwner, vertex); =20 // modify end // Update status every VERTICES_UPDATE_PERIOD edges if (inputSplitEdgesLoaded % VERTICES_UPDATE_PERIOD =3D=3D 0) { totalEdgesMeter.mark(VERTICES_UPDATE_PERIOD);=20 {code}=20 After do it =EF=BC=8C my giraph job can only load edge data in any examples= =E3=80=82 I wish a similar features can be add into in future version=E3=80=82 =20 > Not support only with edge input data in some algorithms. > --------------------------------------------------------- > > Key: GIRAPH-652 > URL: https://issues.apache.org/jira/browse/GIRAPH-652 > Project: Giraph > Issue Type: Wish > Components: graph > Affects Versions: 1.0.0 > Reporter: Wang Xu > Priority: Minor > > There are two types of input vertex-centric and edge-centric. Some algori= thms need vertex centric data, such as pairs of vertex ids and initial page= ranks. Some algorithms only look at edges. For example, connected componen= ts can be run without any vertex values. > In some conditions , we have only edge-centric data , have not vertex dat= a . Convert edge-centric data to vertex-centric data is not bad, but is no= t better. > In my job , I modify EdgeInputSplitsCallable.java to support load vertex = auto when loading edge=E3=80=82 > My mainly modify is=EF=BC=9A > {code:java}=20 > graphState.getWorkerClientRequestProcessor().sendEdgeRequest( > sourceId, readerEdge); > context.progress(); // do this before potential data transfer > ++inputSplitEdgesLoaded; > =20 > //modify begin > Vertex vertex =3D this.configuration.createVertex(); = =20 > if (vertex.getValue() =3D=3D null) { > =09 vertex.setValue(configuration.createVertexValue()); > } > vertex.setConf(configuration); > vertex.setGraphState(graphState); =20 > vertex.initialize(edgeReader.getCurrentEdge().getTargetVertexId(), = vertex.getValue()); > =20 > PartitionOwner partitionOwner =3D=20 > bspServiceWorker.getVertexPartitionOwner(vertex.getId()); > graphState.getWorkerClientRequestProcessor().sendVertexRequest( > partitionOwner, vertex); =20 > // modify end > // Update status every VERTICES_UPDATE_PERIOD edges > if (inputSplitEdgesLoaded % VERTICES_UPDATE_PERIOD =3D=3D 0) { > totalEdgesMeter.mark(VERTICES_UPDATE_PERIOD);=20 > {code}=20 > After do it =EF=BC=8C my giraph job can only load edge data in any exampl= es=E3=80=82 > I wish a similar features can be add into in future version=E3=80=82 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrato= rs For more information on JIRA, see: http://www.atlassian.com/software/jira