hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Swarnim Kulkarni (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-3253) ArrayIndexOutOfBounds exception for deeply nested structs
Date Thu, 12 Jul 2012 13:28:35 GMT

     [ https://issues.apache.org/jira/browse/HIVE-3253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Swarnim Kulkarni updated HIVE-3253:
-----------------------------------

    Description: 
It was observed that while creating table with deeply nested structs might throw this exception:

{code}
java.lang.ArrayIndexOutOfBoundsException: 9
        at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:281)
	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:263)
	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:276)
	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:263)
	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:276)
	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:263)
	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:276)
	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:263)
	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:276)
	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyStructInspector(LazyFactory.java:354)
{code}

The reason being that currently the separators array has been hardcoded to be of size 8 in
the LazySimpleSerde.

// Read the separators: We use 8 levels of separators by default, but we
// should change this when we allow users to specify more than 10 levels
// of separators through DDL.
serdeParams.separators = new byte[8];
{code}

We possible, we should increase this size or at least make it configurable to properly handle
deeply nested structs.

  was:
It was observed that while creating table with deeply nested structs might throw this exception:

{code}
java.lang.ArrayIndexOutOfBoundsException: 9
        at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:281)
	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:263)
	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:276)
	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:263)
	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:276)
	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:263)
	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:276)
	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:263)
	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:276)
	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyStructInspector(LazyFactory.java:354)


The reason being that currently the separators array has been hardcoded to be of size 8 in
the LazySimpleSerde.

{code}
// Read the separators: We use 8 levels of separators by default, but we
// should change this when we allow users to specify more than 10 levels
// of separators through DDL.
serdeParams.separators = new byte[8];
{code}

We possible, we should increase this size or at least make it configurable to properly handle
deeply nested structs.

    
> ArrayIndexOutOfBounds exception for deeply nested structs
> ---------------------------------------------------------
>
>                 Key: HIVE-3253
>                 URL: https://issues.apache.org/jira/browse/HIVE-3253
>             Project: Hive
>          Issue Type: Bug
>          Components: Serializers/Deserializers
>    Affects Versions: 0.9.0
>            Reporter: Swarnim Kulkarni
>
> It was observed that while creating table with deeply nested structs might throw this
exception:
> {code}
> java.lang.ArrayIndexOutOfBoundsException: 9
>         at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:281)
> 	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:263)
> 	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:276)
> 	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:263)
> 	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:276)
> 	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:263)
> 	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:276)
> 	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:263)
> 	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:276)
> 	at org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyStructInspector(LazyFactory.java:354)
> {code}
> The reason being that currently the separators array has been hardcoded to be of size
8 in the LazySimpleSerde.
> // Read the separators: We use 8 levels of separators by default, but we
> // should change this when we allow users to specify more than 10 levels
> // of separators through DDL.
> serdeParams.separators = new byte[8];
> {code}
> We possible, we should increase this size or at least make it configurable to properly
handle deeply nested structs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message