phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas D'Silva (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-4382) Immutable table SINGLE_CELL_ARRAY_WITH_OFFSETS values starting with separator byte return null in query results
Date Tue, 26 Dec 2017 21:04:03 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-4382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16304018#comment-16304018
] 

Thomas D'Silva commented on PHOENIX-4382:
-----------------------------------------

[~vincentpoon]

Thanks for the patch. Should we add a config option to determine the behavior when we see
a  {separatorByte, 1} ? By default should we assume that  {separatorByte, 1} represents a
column value that is not null? 
I assume in most cases people will not explicitly set a value to null. 

Could you also add a comment explaining how you determine the column value is null when you
see {separatorByte, 1} .

{code}
private static boolean isNullValue(int arrayIndex, byte[] bytes, int initPos,
            byte serializationVersion, boolean useShort, int indexOffset, int currOffset,
            int elementLength) {
        if (isSeparatorByte(bytes, initPos, currOffset)) {
            if (isPriorValueZeroLength(arrayIndex, bytes,
                serializationVersion, useShort, indexOffset, currOffset)) {
                return true;
            } else {
                // if there's no prior null, there can be at most 1 null
                if (elementLength == 2) {
                    byte nullByte = SortOrder.invert((byte)(0));
                    if (bytes[initPos+currOffset+1] == nullByte) {
                        return true;
                    }
                }
            }
        }
        return false;
    }
{code}

> Immutable table SINGLE_CELL_ARRAY_WITH_OFFSETS values starting with separator byte return
null in query results
> ---------------------------------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-4382
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-4382
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.14.0
>            Reporter: Vincent Poon
>            Assignee: Vincent Poon
>         Attachments: PHOENIX-4382.v1.master.patch, PHOENIX-4382.v2.master.patch, PHOENIX-4382.v3.master.patch,
UpsertBigValuesIT.java
>
>
> For immutable tables, upsert of some values like Short.MAX_VALUE results in a null value
in query resultsets.  Mutable tables are not affected.  I tried with BigInt and got the same
problem.
> For Short, the breaking point seems to be 32512.
> This is happening because of the way we serialize nulls.  For nulls, we write out [separatorByte,
#_of_nulls].  However, some data values, like Short.MAX_VALUE, start with separatorByte, we
can't distinguish between a null and these values.  Currently the code assumes it's a null
when it sees a leading separatorByte, hence the incorrect query results.
> See attached test - testShort() , testBigInt()



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message