spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-2334) Attribute Error calling PipelinedRDD.id() in pyspark
Date Sat, 06 Sep 2014 01:42:29 GMT

    [ https://issues.apache.org/jira/browse/SPARK-2334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14124239#comment-14124239
] 

Apache Spark commented on SPARK-2334:
-------------------------------------

User 'davies' has created a pull request for this issue:
https://github.com/apache/spark/pull/2296

> Attribute Error calling PipelinedRDD.id() in pyspark
> ----------------------------------------------------
>
>                 Key: SPARK-2334
>                 URL: https://issues.apache.org/jira/browse/SPARK-2334
>             Project: Spark
>          Issue Type: Bug
>          Components: PySpark
>    Affects Versions: 1.0.0, 1.1.0
>            Reporter: Diana Carroll
>
> calling the id() function of a PipelinedRDD causes an error in PySpark.  (Works fine
in Scala.)
> The second id() call here fails, the first works:
> {code}
> r1 = sc.parallelize([1,2,3])
> r1.id()
> r2=r1.map(lambda i: i+1)
> r2.id()
> {code}
> Error:
> {code}
> ---------------------------------------------------------------------------
> AttributeError                            Traceback (most recent call last)
> <ipython-input-31-a0cf66fcf645> in <module>()
> ----> 1 r2.id()
> /usr/lib/spark/python/pyspark/rdd.py in id(self)
>     180         A unique ID for this RDD (within its SparkContext).
>     181         """
> --> 182         return self._id
>     183 
>     184     def __repr__(self):
> AttributeError: 'PipelinedRDD' object has no attribute '_id'
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message