spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mateiz <...@git.apache.org>
Subject [GitHub] spark pull request: SPARK-3211 .take() is OOM-prone with empty par...
Date Wed, 27 Aug 2014 00:59:06 GMT
Github user mateiz commented on the pull request:

    https://github.com/apache/spark/pull/2117#issuecomment-53514275
  
    This looks good to me as is because it only happens in the case where the first partition
had 0 elements. @aarondav what do you think? This case will be pretty uncommon, and the other
code in take() already does a smarter increase.
    
    @ash211 please also fix this in Python, in python/pyspark/rdd.py. The code there is almost
identical.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message