giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karos Lotfifar <foad...@gmail.com>
Subject Re: Scalability issue on Giraph
Date Mon, 14 Dec 2015 21:12:53 GMT
Thanks Hasan,

My question is: is it the way others report superstep times in research
papers such as Hama, Giraph, GraphLab, etc. Are they not considering the
I/O time? (As they report good scalable reports for Pagerank calculation).

The other case is that, even taking into account the computational
supersteps excluding I/O overhead does not make any improve. Taking into
account the first superstep (superstep 0 is not considered as you say) the
timing details for amazon would be:

# Processor cores
        2           4          8         16
24
32superstep 1                5101     3642     3163    2867     3611   3435
(ms)


As you see, this is the same story! while no I/O is considered.



Regards,
Karos


On Mon, Dec 14, 2015 at 9:29 PM, Hassan Eslami <hsn.eslami@gmail.com> wrote:

> Hi,
>
> Here is my take on it:
> It may be a good idea to isolate the time spent in compute supersteps. In
> order to do that, you can look at per-superstep timing metrics and
> aggregate the times for all supersteps except input (-1) and output (last)
> superstep. This eliminates the time for IO and turns your focus only on
> time spent accessing memory and doing computation in cores. In general, if
> an application is memory-bound (for instance PageRank), increasing the
> number of threads does not necessarily decrease the time after a certain
> point, due to the fact that memory bandwidth can become a bottleneck. In
> other words, once the memory bandwidth has been saturated by a certain
> number of threads, increasing the number of threads will not decrease the
> execution time anymore.
>
> Best,
> Hassan
>
> On Mon, Dec 14, 2015 at 5:14 AM, Foad Lotfifar <foadfbf@gmail.com> wrote:
>
>> Hi,
>>
>> I have a scalability issue for Giraph and I can not find out where is the
>> problem.
>>
>> --- Cluster specs:
>> # nodes             1
>> # threads          32
>> Processor          Intel Xeon 2.0GHz
>> OS                    ubuntu 32bit
>> RAM                 64GB
>>
>> --- Giraph specs
>> Hadoop            Apache Hadoop 1.2.1
>> Giraph              1.2.0 Snapshot
>>
>> Tested Graphs:
>> amazon0302                V=262,111, E=1,234,877
>> coAuthorsCiteseer        V=227,320, E=1,628,268
>>
>>
>> I run the provided PageRank algorithm in Giraph
>> "SimplePageRankComputation" with the followng options
>>
>> (time ($HADOOP_HOME/bin/hadoop jar
>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar
>> \
>>  org.apache.giraph.GiraphRunner
>> -Dgiraph.graphPartitionerFactoryClass=org.apache.giraph.partition.HashRangePartitionerFactory
>> \
>>  org.apache.giraph.examples.PageRankComputation  \
>> -vif
>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat \
>> -vip /user/hduser/input/$file \
>> -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat \
>> -op /user/hduser/output/pagerank -w $2 \
>> -mc
>> org.apache.giraph.examples.PageRankComputation\$PageRankMasterCompute))
>> 2>&1 \
>> | tee -a ./pagerank_results/$file.GHR_$2.$iter.output.txt
>>
>> The algorithm runs without any issue. The number of supersteps is set to
>> 31 by default in the algorithm.
>>
>> *Problem:*
>> I dont get any scalability for more than 8 (or 16) processor cores that
>> is I get speedup up to 8 (or 16) cores and then the run time starts to
>> increase.
>>
>> I have run the PageRank with only one superstep as well as running other
>> algorithms such as ShortestPath algorithm. I get the same results. I can
>> not figure out where is the problem.
>>
>> 1- I have tried two options by changing the giraph.numInputThreads and
>> giraph.numOutputThreads: the performance gets a littile bit better but no
>> impact on scalability.
>> 2- Does it related to the size of the graphs? because the graphs I am
>> testing are small graphs.
>> 3- Is it a platform related issue?
>>
>> It is the timing details of amazon graph:
>>
>> # Processor cores
>> 1 2 4 8 16 24 32
>> Input
>> 3260 3447 3269 3921 4555 4766 Intialise
>> 3467 36458 45474 39091 100281 79012 Setup
>> 34 52 59 70 77 86 Shutdown
>> 9954 10226 11021 9524 13393 15930 Total
>> 135482 84483 61081 52190 58921 61898
>> HDFS READ
>> 21097485 26117723 36158199 57808783 80086015 102163071 FILE WRITE
>> 65889 109815 197667 373429 549165 724901 HDFS WRITE
>> 7330986 7331068 7331093 7330988 7330976 7331203
>>
>> Best Regards,
>> Karos
>>
>>
>>
>>
>


-- 
Regards,
Karos Lotfifar

Mime
View raw message