Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Thu, 19 Nov 2015 08:14:11 +0000 (UTC)
From: "Gopal V (JIRA)" <jira@apache.org>
To: dev@hive.apache.org
Message-ID: <JIRA.12914383.1447920829000.120138.1447920851012@Atlassian.JIRA>
In-Reply-To: <JIRA.12914383.1447920829000@Atlassian.JIRA>
References: <JIRA.12914383.1447920829000@Atlassian.JIRA>
 <JIRA.12914383.1447920829031@arcas>
Subject: [jira] [Created] (HIVE-12463) VectorMapJoinFastKeyStore has Array
 OOB errors
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Gopal V created HIVE-12463:
------------------------------

             Summary: VectorMapJoinFastKeyStore has Array OOB errors
                 Key: HIVE-12463
                 URL: https://issues.apache.org/jira/browse/HIVE-12463
             Project: Hive
          Issue Type: Bug
          Components: Vectorization
    Affects Versions: 2.0.0
            Reporter: Gopal V
            Assignee: Gopal V


When combining different sized keys, observing an occasional error in hashtable probes.

{code}
Caused by: java.lang.ArrayIndexOutOfBoundsException: 162046429
	at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastKeyStore.equalKey(VectorMapJoinFastKeyStore.java:150)
	at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastBytesHashTable.findReadSlot(VectorMapJoinFastBytesHashTable.java:191)
	at org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.VectorMapJoinFastBytesHashMap.lookup(VectorMapJoinFastBytesHashMap.java:76)
	at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerMultiKeyOperator.process(VectorMapJoinInnerMultiKeyOperator.java:300)
	... 26 more
{code}

{code}
    // Our reading is positioned to the key.
    writeBuffers.getByteSegmentRefToCurrent(byteSegmentRef, keyLength, readPos);

    byte[] currentBytes = byteSegmentRef.getBytes();
    int currentStart = (int) byteSegmentRef.getOffset();

    for (int i = 0; i < keyLength; i++) {
      if (currentBytes[currentStart + i] != keyBytes[keyStart + i]) {
        // LOG.debug("VectorMapJoinFastKeyStore equalKey no match on bytes");
        return false;
      }
    }
{code}

This needs an identical fix to match 

{code}
    // Rare case of buffer boundary. Unfortunately we'd have to copy some bytes.

   // Rare case of buffer boundary. Unfortunately we'd have to copy some bytes.
    byte[] bytes = new byte[length];
    int destOffset = 0;
    while (destOffset < length) {
      ponderNextBufferToRead(readPos);
{code}


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)