spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-6993) Adding default max and min methods for JavaDoubleRDD
Date Sat, 18 Apr 2015 09:04:58 GMT

    [ https://issues.apache.org/jira/browse/SPARK-6993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14501193#comment-14501193
] 

Apache Spark commented on SPARK-6993:
-------------------------------------

User 'ogirardot' has created a pull request for this issue:
https://github.com/apache/spark/pull/5571

> Adding default max and min methods for JavaDoubleRDD
> ----------------------------------------------------
>
>                 Key: SPARK-6993
>                 URL: https://issues.apache.org/jira/browse/SPARK-6993
>             Project: Spark
>          Issue Type: Improvement
>          Components: Java API
>    Affects Versions: 1.2.3, 1.3.1, 1.4.0
>            Reporter: Olivier Girardot
>            Priority: Minor
>
> The default Scala API allows you when manipulating standard types like Double to simply
write 
> {code:scala}
> val doubles : RDD[Double] = ...
> doubles.max()
> doubles.min()
> {code}
> Using the implicit ordering, on the other hand for the Java API you have to specify a
Comparator to use : 
> {code:java}
> JavaDoubleRDD doubles = ...mapToDouble...
> doubles.max(Comparator.naturalOrder()) // for example
> doubles.min(Comparator.naturalOrder()) // for example
> {code}
> What is even less consistent is that the *stats* API doesn't ask you for any kind of
comparator and uses the default ordering by itself.
> This issue, and the associated pull request, is proposing to add default max and min
functions to the JavaDoubleRDD that will use the default Comparator.
> But as **java.util.Comparator.naturalOrder()** is only a Java 8 API, I used in my PR
the Guava - Ordering.natural() method and added the associated test.
> This PR does not add any additional dependencies, Guava is already used by core and shaded.
> Thank you for your time, 
> Olivier.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message