Mailing-List: contact hama-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hama-user@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of thomas.jungblut@googlemail.com
 designates 209.85.212.47 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAJ-=ys=duGsjQAnan-Ahc+YO=DK9NsVaZSGwyPiLRAScDRah7Q@mail.gmail.com>
References: <4F9AB3B0.4000608@unister-gmbh.de>
	<CAJ-=ys=duGsjQAnan-Ahc+YO=DK9NsVaZSGwyPiLRAScDRah7Q@mail.gmail.com>
Date: Fri, 27 Apr 2012 17:24:13 +0200
Message-ID: 
 <CAJ-=ysm3B3hpPNzYXM7PyUp+6Aip6zmg6x_UokP5mjzy6OJ91g@mail.gmail.com>
Subject: Re: How to check whether the target node of an edge exists?
From: Thomas Jungblut <thomas.jungblut@googlemail.com>
To: hama-user@incubator.apache.org
Content-Type: multipart/alternative; boundary=20cf302d4dd46b5b0c04beaab264

--20cf302d4dd46b5b0c04beaab264
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Oh sorry I have not read the other half of your mail.
I have made a mapreduce preprocessing step (yes mapreduce is the right
answer for that here) for that which can be found here:

https://github.com/thomasjungblut/thomasjungblut-common/blob/master/src/de/=
jungblut/crawl/processing/WebGraphProcessingJob.java
>

It traverses the graph and the reducer makes the right output which can be
processed by the job or the TextToSeq utility as a preprocessing for
pagerank, just have a look into the example package.

I appears that I am not supposed to access the vertices field of
> GraphJobRunner class in some way from within the PageRank.PageRankVertex
> class?
>

Yes in this new Pregel-Like API this falls under information-hiding. If you
have a look into the 4.0 release, there is the hardcore version of pagerank
with plain BSP, there you can access and modify all the stuff you want.
(but it is more complicated)

Hope it helps, if you have additional problems, don't hesitate to ask them.


Am 27. April 2012 17:10 schrieb Thomas Jungblut <
thomas.jungblut@googlemail.com>:

> You have to "address" dangling nodes on your adjacency list.
>
> So your input must look like:
>
>
> 0    1    2
> 1    1    2
> 2    1    2    3
> 3 <-- this one was missing causing the Null Pointer Exception.
> 5
>
> See http://wiki.apache.org/hama/PageRank under "Submit your own Webgraph"=
.
>
>> This piece of text will adjacent Site1 to Site2 and Site3, Site2 to Site=
3
>> and Site3 is a dangling node. As you can see a site is always on the
>> leftmost side (we call it the key-site), and the outlinks are seperated =
by
>> tabs (\t) as the following elements.
>> Make sure that every site's outlink can somewhere be found in the file a=
s
>> a key-site. Otherwise it will result in weird NullPointerExceptions<http=
://wiki.apache.org/hama/NullPointerExceptions>.
>>
>
>
> Good luck.
>
> Am 27. April 2012 16:56 schrieb SWP <contact-semweb@unister-gmbh.de>:
>
> I am dealing with the PageRank example
>> from  hama-dist-0.5.0-incubating-**source.tar.gz RC2
>> which I downloaded from http://people.apache.org/~**edwardyoon/dist/<htt=
p://people.apache.org/%7Eedwardyoon/dist/>
>> a few days ago.
>>
>> My input graph has some "dangling edges", that is, edges pointing to
>> non-existing nodes.
>> Here are the adjacencies of a small example. The format is:
>> source target1 target2 target3 ...
>>
>> 0    1    2
>> 1    1    2
>> 2    1    2    3
>> 5
>>
>> Your see that 2 has an edge directed to 3 but there is no adjacency list
>> given for 3.
>>
>> Now, when I run this example through pagerank-text2seq and then the
>> pagerank examle, I get a NullPointerException:
>>
>> 12/04/27 16:15:17 ERROR bsp.LocalBSPRunner: Exception during BSP
>> execution!
>> java.lang.NullPointerException
>>    at org.apache.hama.graph.**GraphJobRunner.bsp(**
>> GraphJobRunner.java:96)
>>    at org.apache.hama.bsp.**LocalBSPRunner$BSPRunner.run(**
>> LocalBSPRunner.java:256)
>>    at org.apache.hama.bsp.**LocalBSPRunner$BSPRunner.call(**
>> LocalBSPRunner.java:286)
>>    at org.apache.hama.bsp.**LocalBSPRunner$BSPRunner.call(**
>> LocalBSPRunner.java:1)
>>    at java.util.concurrent.**FutureTask$Sync.innerRun(**
>> FutureTask.java:303)
>>    at java.util.concurrent.**FutureTask.run(FutureTask.**java:138)
>>    at java.util.concurrent.**ThreadPoolExecutor$Worker.**
>> runTask(ThreadPoolExecutor.**java:886)
>>    at java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
>> ThreadPoolExecutor.java:908)
>>    at java.lang.Thread.run(Thread.**java:662)
>>
>> The problem appears to be that when GraphJobRunner's bsp() method  looks
>> up the vertex to which the message is addressed,
>> it is not found in the vertices map.
>> (By the way, if you replace 5 with 3 in the example, it works - because
>> then the target vertex can be looked up.)
>>
>> See the vertices.get(e.getKey()) statement in the code snippet below.
>> Of course one can avoid the exception by adding a check in
>> GraphJobRunner.java (at line about 95) like this:
>>
>>        if(vertices.containsKey(e.**getKey()))
>>        {
>>            vertices.get(e.getKey()).**compute(msgs.iterator());
>>        } else {
>>            System.out.println("Ignoring message(s) '" + msgs + "' sent t=
o
>> vertex '" + e.getKey() +"'");
>>        }
>>
>> However, what I really want is:
>> check within PageRank.PageRankVertex's compute() method whether the
>> target vertex exists
>> before sending out a message to it.
>>
>> That is, in PageRank.java (line 60) , instead of
>>
>>                sendMessageToNeighbors(new DoubleWritable(this.getValue()=
*
>> *.get() / numEdges));
>>
>> I would like to send messages only  to "existing" vertices, that is,
>>  those which have an adjacency list in the input.
>>
>> Any hints how this can be achieved?
>> I appears that I am not supposed to access the vertices field of
>> GraphJobRunner class in some way from within the PageRank.PageRankVertex
>> class?
>>
>> I concede that my example graph may qualify as invalid input ... but on
>> the other hand: how could I add those missing vertices after a first pas=
s
>> through the adjacency lists input?
>>
>> Clemens Gr=F6pl
>>
>> --
>>
>> Semantic Web Project, IT
>>
>> Unister GmbH
>> Barfu=DFg=E4=DFchen 11 | 04109 Leipzig
>>
>> Telefon: +49 (0)341 49288 4496
>> contact-semweb@unister-gmbh.de <mailto:%20contact-semweb@**
>> unister-gmbh.de <20contact-semweb@unister-gmbh.de>>
>> www.unister.de <http://www.unister.de>
>>
>> Vertretungsberechtigter Gesch=E4ftsf=FChrer: Thomas Wagner
>> Amtsgericht Leipzig, HRB: 19056
>>
>>
>
>
> --
> Thomas Jungblut
> Berlin <thomas.jungblut@gmail.com>
>


--=20
Thomas Jungblut
Berlin <thomas.jungblut@gmail.com>

--20cf302d4dd46b5b0c04beaab264--