spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dongjoon Hyun (Jira)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-30432) reduce degree recomputation in StronglyConnectedComponents
Date Mon, 13 Jan 2020 05:28:00 GMT

     [ https://issues.apache.org/jira/browse/SPARK-30432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Dongjoon Hyun updated SPARK-30432:
----------------------------------
    Target Version/s:   (was: 2.4.5, 3.0.0)

> reduce degree recomputation in StronglyConnectedComponents
> ----------------------------------------------------------
>
>                 Key: SPARK-30432
>                 URL: https://issues.apache.org/jira/browse/SPARK-30432
>             Project: Spark
>          Issue Type: Improvement
>          Components: GraphX
>    Affects Versions: 3.0.0
>            Reporter: li xiaosen
>            Priority: Major
>
>  
> So the computation happens every time in the do-while loop, the first time the outer
while loop executes. although just once per do-while loop after, it seems, but It does reduce
a lot of recomputation;because every time it jump out of the do-while loop,there are
no vertices have only out-degree or in-degree,so it's no need to recompute degree to tag
the vertices true.
> I have done a small code proposal, because there is a problem when the pregel executions
have done,  the degree no need to be recomputed.
>  
> for example,the Email-EuAll  data set:[http://snap.stanford.edu/data/email-EuAll.html]
> do-while loop execute 10 times,and the reduce logic happend 2 times;so it would be
helpful when computing StronglyConnectedComponents to reduce degree computation.
>  
> I created a branch in my fork: [https://github.com/xs-li/spark/blob/master/graphx/src/main/scala/org/apache/spark/graphx/lib/StronglyConnectedComponents.scala]
>  
> I hope you can consider this small code proposal.
> Thank you very much,
> Best regards,
> xs-li



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message