spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-9460) Avoid byte array allocation in StringPrefixComparator
Date Thu, 30 Jul 2015 01:43:04 GMT

    [ https://issues.apache.org/jira/browse/SPARK-9460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14647046#comment-14647046
] 

Apache Spark commented on SPARK-9460:
-------------------------------------

User 'rxin' has created a pull request for this issue:
https://github.com/apache/spark/pull/7765

> Avoid byte array allocation in StringPrefixComparator
> -----------------------------------------------------
>
>                 Key: SPARK-9460
>                 URL: https://issues.apache.org/jira/browse/SPARK-9460
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>            Reporter: Reynold Xin
>
> StringPrefixComparator converts the long values back to byte arrays in order to compare
them. We should be able to optimize this to compare the longs directly, rather than turning
the longs into byte arrays and comparing them byte by byte. 
> {code}
>     public int compare(long aPrefix, long bPrefix) {
>       // TODO: can done more efficiently
>       byte[] a = Longs.toByteArray(aPrefix);
>       byte[] b = Longs.toByteArray(bPrefix);
>       for (int i = 0; i < 8; i++) {
>         int c = UnsignedBytes.compare(a[i], b[i]);
>         if (c != 0) return c;
>       }
>       return 0;
>     }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message