spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hyukjin Kwon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-17877) Can not checkpoint connectedComponents resulting graph
Date Mon, 09 Oct 2017 09:37:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-17877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16196748#comment-16196748
] 

Hyukjin Kwon commented on SPARK-17877:
--------------------------------------

I tested this and the last line returned {{false}}.

> Can not checkpoint connectedComponents resulting graph
> ------------------------------------------------------
>
>                 Key: SPARK-17877
>                 URL: https://issues.apache.org/jira/browse/SPARK-17877
>             Project: Spark
>          Issue Type: Bug
>          Components: GraphX
>    Affects Versions: 1.5.2, 1.6.2, 2.0.1, 2.3.0
>            Reporter: Alexander Pivovarov
>            Priority: Minor
>
> The following code demonstrates the issue
> {code}
> import org.apache.spark.graphx._
> val users = sc.parallelize(List(3L -> "lucas", 7L -> "john", 5L -> "matt", 2L
-> "kelly"))
> val rel = sc.parallelize(List(Edge(3L, 7L, "collab"), Edge(5L, 3L, "advisor"), Edge(2L,
5L, "colleague"), Edge(5L, 7L, "pi")))
> sc.setCheckpointDir("/tmp/check")
> val g = Graph(users, rel)
> g.checkpoint   // /tmp/check/b1f46ba5-357a-4d6d-8f4d-411b64b27c2f appears
> val gg = g.connectedComponents
> gg.checkpoint
> gg.vertices.collect
> gg.edges.collect
> gg.isCheckpointed  // res5: Boolean = false,   /tmp/check still contains only 1 folder
b1f46ba5-357a-4d6d-8f4d-411b64b27c2f
> {code}
> I think the last line should return true instead of false



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message