phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Taylor (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PHOENIX-2691) Exception while unpacking resultset containing VARCHAR ARRAY of unspecified length
Date Thu, 18 Feb 2016 22:48:18 GMT

     [ https://issues.apache.org/jira/browse/PHOENIX-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

James Taylor updated PHOENIX-2691:
----------------------------------
    Attachment: PHOENIX-2691.patch

Here's a patch, [~ndimiduk] that supports group by over a single array or varbinary. If we
need to support for grouping over multiple ones of these, we could support in differently
(and still not have to change our row key code) by putting the group by values in the KeyValue
we return instead of getting them from the row key.

Would you mind reviewing, [~tdsilva] or Nick?

> Exception while unpacking resultset containing VARCHAR ARRAY of unspecified length
> ----------------------------------------------------------------------------------
>
>                 Key: PHOENIX-2691
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-2691
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.6.0, 4.7.0
>            Reporter: Nick Dimiduk
>            Assignee: James Taylor
>             Fix For: 4.7.0
>
>         Attachments: 2691.00.patch, PHOENIX-2691.patch
>
>
> I have an aggregation query that consistently throws with either an IllegalArgumentException
or an OutOfMemoryException, at the same place. Either way, the stack trace is nearly identical:
> {noformat}
> java.lang.IllegalArgumentException                                                  
                                              
>         at java.nio.Buffer.position(Buffer.java:244)                                
                                              
>         at org.apache.phoenix.schema.types.PArrayDataType.createPhoenixArray(PArrayDataType.java:1098)
                            
>         at org.apache.phoenix.schema.types.PArrayDataType.toObject(PArrayDataType.java:339)
                                       
>         at org.apache.phoenix.schema.types.PVarcharArray.toObject(PVarcharArray.java:65)
                                          
>         at org.apache.phoenix.schema.types.PDataType.toObject(PDataType.java:985)   
                                              
>         at org.apache.phoenix.compile.ExpressionProjector.getValue(ExpressionProjector.java:75)
                                   
>         at org.apache.phoenix.jdbc.PhoenixResultSet.getString(PhoenixResultSet.java:601)
                                          
>         at sqlline.Rows$Row.<init>(Rows.java:183)                             
                                                    
>         at sqlline.BufferedRows.<init>(BufferedRows.java:38)                  
                                                    
>         at sqlline.SqlLine.print(SqlLine.java:1650)                                 
                                              
>         at sqlline.Commands.execute(Commands.java:833)                              
                                              
>         at sqlline.Commands.sql(Commands.java:732)                                  
                                              
>         at sqlline.SqlLine.dispatch(SqlLine.java:808)                               
                                              
>         at sqlline.SqlLine.begin(SqlLine.java:681)                                  
                                              
>         at sqlline.SqlLine.start(SqlLine.java:398)                                  
                                              
>         at sqlline.SqlLine.main(SqlLine.java:292)
> {noformat}
> or
> {noformat}
> java.lang.OutOfMemoryError: Java heap space
>         at java.lang.reflect.Array.newArray(Native Method)
>         at java.lang.reflect.Array.newInstance(Array.java:75)
>         at org.apache.phoenix.schema.types.PArrayDataType.createPhoenixArray(PArrayDataType.java:1091)
>         at org.apache.phoenix.schema.types.PArrayDataType.toObject(PArrayDataType.java:339)
>         at org.apache.phoenix.schema.types.PVarcharArray.toObject(PVarcharArray.java:65)
>         at org.apache.phoenix.schema.types.PDataType.toObject(PDataType.java:985)
>         at org.apache.phoenix.compile.ExpressionProjector.getValue(ExpressionProjector.java:75)
>         at org.apache.phoenix.jdbc.PhoenixResultSet.getString(PhoenixResultSet.java:601)
>         at sqlline.Rows$Row.<init>(Rows.java:183)
>         at sqlline.BufferedRows.<init>(BufferedRows.java:38)
>         at sqlline.SqlLine.print(SqlLine.java:1650)
>         at sqlline.Commands.execute(Commands.java:833)
>         at sqlline.Commands.sql(Commands.java:732)
>         at sqlline.SqlLine.dispatch(SqlLine.java:808)
>         at sqlline.SqlLine.begin(SqlLine.java:681)
>         at sqlline.SqlLine.start(SqlLine.java:398)
>         at sqlline.SqlLine.main(SqlLine.java:292)
> {noformat}
> Stepping through with the debugger, it appears the {{VARCHAR ARRAY}} value is not parsed
correctly. The special case of two nulls is not accounted for in {{RowKeyValueAccessor#getLength()}}.
This results in the offsets being slightly wrong and then the value cannot be materialized
correctly. Depending on what's in the adjacent bytes either an invalid {{position}} call is
made, resulting in the {{IllegalArgumentException}}, or {{PArrayDataType.createPhoenixArray}}
attempts to allocate an array of ridiculous size, resulting in the OOM.
> It appears the types of the columns returned in the {{KeyValue}} in the {{Tuple currentRow}}
are ordered {{VARCHAR}}, {{VARCHAR ARRAY}}, {{INTEGER}}. I can share the KeyValue bytes with
you offline if that will help in debugging.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message