hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mahender Sarangam <Mahender.BigD...@outlook.com>
Subject Re: ORC does not support type conversion from INT to STRING.
Date Tue, 19 Jul 2016 21:19:31 GMT
But we are using Hive 1.2 version

On 7/19/2016 12:43 PM, Mich Talebzadeh wrote:
in Hive 2,  I don't see this issue INSERT/SELECT from INT to String column!


Dr Mich Talebzadeh



LinkedIn  https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw



http://talebzadehmich.wordpress.com


Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destruction
of data or any other property which may arise from relying on this email's technical content
is explicitly disclaimed. The author will in no case be liable for any monetary damages arising
from such loss, damage or destruction.



On 19 July 2016 at 20:39, Mahender Sarangam <Mahender.BigData@outlook.com<mailto:Mahender.BigData@outlook.com>>
wrote:


Thanks Matthew,

Currently we are in Hive 1.2 version only, Is there any setting like "hive.metastore.disallow.incompatible.col.type.changes=false;​"
in Hive 1.2 or any around apart for reloading entire table data.  For Quick workaround, we
are reloading entire data.
Can you please share with us Jira for Schema Evolution.


@Mich : Currently we have only primitive types. But I'm also interested to know "how the behavior
will be  in complex types"


/Mahender


On 7/18/2016 3:55 PM, Mich Talebzadeh wrote:
Hi Mathew,

In layman's term if I create the source ORC table column as INT and then create a target ORC
table but that column has now been defined as STRING and do an INSERT/SELECT from source table
how data is internally stored?

Is it implicitly converted into new format using CAST function or it is stored as is and just
masked?

The version of Hive I am using is 2 and it works OK for primitive data types (insert/select
from INT to String)

However, I believe Mahender is referring to Complex types?

Thanks




Dr Mich Talebzadeh



LinkedIn  https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw



http://talebzadehmich.wordpress.com


Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destruction
of data or any other property which may arise from relying on this email's technical content
is explicitly disclaimed. The author will in no case be liable for any monetary damages arising
from such loss, damage or destruction.



On 18 July 2016 at 22:31, Matthew McCline <mmccline@hortonworks.com<mailto:mmccline@hortonworks.com>>
wrote:


Hi Mahender,


Schema Evolution is available on the latest recent version of Hive.


For example, if you set hive.metastore.disallow.incompatible.col.type.changes=false;​ on
master (i.e. hive2) it will support INT to STRING conversion.


If you need to remain on an older version, then you are out of luck.


Thanks,

Matt


________________________________
From: Mahender Sarangam <Mahender.BigData@outlook.com<mailto:Mahender.BigData@outlook.com>>
Sent: Monday, July 18, 2016 1:59 PM
To: user@hive.apache.org<mailto:user@hive.apache.org>
Subject: Re: ORC does not support type conversion from INT to STRING.


Hi Mich,

Sorry for delay in responding. here is the scenario,

We have created new cluster  and we have moved all ORC File data into new cluster. We have
re-created table pointing to ORC location. We have modified data type of ORC table from INT
to String. From then onward, we were unable to fire select statement against this ORC table,
hive keep throwing exception, "Orc table select. Unable to convert Int to String". Looks like
it is bug in ORC table only. Where in we modify the datatype from int to string, is causing
problem with ORC reading/select statement, it throws exceptio. Please let me know if there
are any workaround for this scenario. Is this behavior expected previously also.


/Mahender





On 6/14/2016 11:47 AM, Mich Talebzadeh wrote:
you must excuse my ignorance

can you please elaborate on this as there seems something has gone wrong somewhere?


Dr Mich Talebzadeh



LinkedIn  https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw



http://talebzadehmich.wordpress.com<http://talebzadehmich.wordpress.com/>



On 14 June 2016 at 19:42, Mahender Sarangam <Mahender.BigData@outlook.com<mailto:Mahender.BigData@outlook.com>>
wrote:

Yes Mich. We have restored cluster from metastore.

On 6/14/2016 11:35 AM, Mich Talebzadeh wrote:
Hi Mahendar,


Did you load the meta-data DB/schema from backup and now seeing this error




Dr Mich Talebzadeh



LinkedIn  https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw



http://talebzadehmich.wordpress.com



On 14 June 2016 at 19:04, Mahender Sarangam <Mahender.BigData@outlook.com<mailto:Mahender.BigData@outlook.com>>
wrote:

ping.

On 6/13/2016 1:19 PM, Mahender Sarangam wrote:

Hi,

We are facing issue while reading data from ORC table. We have created ORC table and dumped
data into it. We have deleted cluster due to some reason. When we recreated cluster (using
Metastore) and table pointing to same location. When we perform reading from ORC table. We
see below error.

SELECT col2, Col1,
  reflect("java.util.UUID", "randomUUID") AS ID,
  Source,
 1     ,
SDate,
EDate
FROM Table ORC  JOIN Table2 _surr;

ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1465411930667_0212_1_01, diagnostics=[Task
failed, taskId=task_1465411930667_0212_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error:
Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException:
java.io.IOException: ORC does not support type conversion from INT to STRING.


I think issue is reflect("java.util.UUID", "randomUUID") AS ID

I know there is Bug raised while reading data from ORC table. Is there any workaround apart
from reloading data.

-MS











Mime
View raw message