hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Teddy Choi (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-12348) Byte array comparison optimization for SIMD
Date Thu, 05 Nov 2015 20:47:27 GMT

     [ https://issues.apache.org/jira/browse/HIVE-12348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Teddy Choi updated HIVE-12348:
------------------------------
    Summary: Byte array comparison optimization for SIMD  (was: String comparison optimization
for SIMD)

> Byte array comparison optimization for SIMD
> -------------------------------------------
>
>                 Key: HIVE-12348
>                 URL: https://issues.apache.org/jira/browse/HIVE-12348
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Teddy Choi
>            Assignee: Teddy Choi
>            Priority: Minor
>
> The current byte comparison implementation in Hive is basic. It handles a byte (1 byte)
at a time, so it's slow.
> There's FastByteComparisons class in Hadoop with sun.misc.Unsafe class. (https://github.com/hanborq/hadoop/blob/master/src/core/org/apache/hadoop/io/FastByteComparisons.java)
It handles a long integer (8 bytes) at a time, so it's faster.
> Java 8 has String.compare, String.equalTo intrinsics with AVX2 and SSE4.2. (http://hg.openjdk.java.net/jdk8/jdk8/hotspot/rev/038dd2875b94)
It handles 128~256 bits (16~32 bytes) at a time, so it's much faster.
> However, Unsafe.getLong and String.compare intrinsic needs additional data copies, so
the actual performance increase is smaller than "1 byte : 32 bytes" comparison.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message