Mailing-List: contact dev-help@phoenix.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@phoenix.apache.org
Date: Thu, 21 Dec 2017 02:49:02 +0000 (UTC)
From: "Vincent Poon (JIRA)" <jira@apache.org>
To: dev@phoenix.apache.org
Message-ID: <JIRA.13118781.1510800624000.508169.1513824542986@Atlassian.JIRA>
In-Reply-To: <JIRA.13118781.1510800624000@Atlassian.JIRA>
References: <JIRA.13118781.1510800624000@Atlassian.JIRA> <JIRA.13118781.1510800624205@jira-lw-us.apache.org>
Subject: [jira] [Commented] (PHOENIX-4382) Immutable table
 SINGLE_CELL_ARRAY_WITH_OFFSETS values starting with separator byte return
 null in query results
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 21 Dec 2017 02:49:10 -0000


    [ https://issues.apache.org/jira/browse/PHOENIX-4382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16299450#comment-16299450 ] 

Vincent Poon commented on PHOENIX-4382:
---------------------------------------

- TINYINT or BYTE(1) should work fine since I explicitly check for length == 2, but we can add a test case to make sure
- If there are multiple null values, it actually makes things less ambiguous - that's the hack I used.  So, for example, the way we can distinguish {separatorByte, 2} (representing two nulls), and an equivalent value, is that if it is supposed to represent a null, then the previous value should have a length of 0 (we know the length by checking the offsets).  You only get the problem with a single null because you have no prior null length to check.
- yes, BYTE(2) would have the same issue.
- yea, variable length types should be fine
- good point, agree probably not worth the effort
- As for the subject, that would be accurate AFTER the patch.  Again, before my patch, any column value starting with separator byte would be broken.  That's why I found large portions of e.g. the BigInt range being returned as null, even though it's more than 2 bytes.
I think we can put that line as a known issue with V1 on the storage schemes webpage.  It's limited to just two specific two-byte fixed length values, though.

> Immutable table SINGLE_CELL_ARRAY_WITH_OFFSETS values starting with separator byte return null in query results
> ---------------------------------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-4382
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-4382
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.14.0
>            Reporter: Vincent Poon
>            Assignee: Vincent Poon
>         Attachments: PHOENIX-4382.v1.master.patch, PHOENIX-4382.v2.master.patch, UpsertBigValuesIT.java
>
>
> For immutable tables, upsert of some values like Short.MAX_VALUE results in a null value in query resultsets.  Mutable tables are not affected.  I tried with BigInt and got the same problem.
> For Short, the breaking point seems to be 32512.
> This is happening because of the way we serialize nulls.  For nulls, we write out [separatorByte, #_of_nulls].  However, some data values, like Short.MAX_VALUE, start with separatorByte, we can't distinguish between a null and these values.  Currently the code assumes it's a null when it sees a leading separatorByte, hence the incorrect query results.
> See attached test - testShort() , testBigInt()


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)