Mailing-List: contact reviews-help@spark.apache.org; run by ezmlm
Precedence: bulk
From: hvanhovell <git@git.apache.org>
To: reviews@spark.apache.org
Reply-To: reviews@spark.apache.org
References: <git-pr-19815-spark@git.apache.org>
In-Reply-To: <git-pr-19815-spark@git.apache.org>
Subject: [GitHub] spark pull request #19815: [SPARK-22602][SQL] remove ColumnVector#loadBytes
Content-Type: text/plain
Message-Id: <20171126212449.A1069DFA09@git1-us-west.apache.org>
Date: Sun, 26 Nov 2017 21:24:49 +0000 (UTC)
archived-at: Sun, 26 Nov 2017 21:24:51 -0000

Github user hvanhovell commented on a diff in the pull request:

    https://github.com/apache/spark/pull/19815#discussion_r153082266
  
    --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/WritableColumnVector.java ---
    @@ -324,24 +315,27 @@ public void putDecimal(int rowId, Decimal value, int precision) {
       @Override
       public UTF8String getUTF8String(int rowId) {
         if (dictionary == null) {
    -      ColumnarArray a = getByteArray(rowId);
    -      return UTF8String.fromBytes(a.byteArray, a.byteArrayOffset, a.length);
    +      return childColumns[0].getUTF8String0(getArrayOffset(rowId), getArrayLength(rowId));
         } else {
           byte[] bytes = dictionary.decodeToBinary(dictionaryIds.getDictId(rowId));
           return UTF8String.fromBytes(bytes);
         }
       }
     
    +  /**
    +   * Returns a UTF8String whose data comes from [rowId, rowId + count] of this vector.
    +   * This method is similar to {@link ColumnVector#getBytes(int, int)}, but can save data copy as
    +   * UTF8String is used as a pointer.
    +   */
    +  protected abstract UTF8String getUTF8String0(int rowId, int count);
    +
       /**
        * Returns the byte array for rowId.
        */
       @Override
       public byte[] getBinary(int rowId) {
         if (dictionary == null) {
    -      ColumnarArray array = getByteArray(rowId);
    -      byte[] bytes = new byte[array.length];
    -      System.arraycopy(array.byteArray, array.byteArrayOffset, bytes, 0, bytes.length);
    -      return bytes;
    +      return childColumns[0].getBytes(getArrayOffset(rowId), getArrayLength(rowId));
    --- End diff --
    
    Same here, use `arrayData()`.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org