sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Qian Xu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-2161) Incremental append on to Hive Parquet tables doesn't work
Date Sun, 01 Mar 2015 12:55:05 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-2161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14342211#comment-14342211

Qian Xu commented on SQOOP-2161:

>From the logfile, the temporary folder is "28163248000000569_31458_mgrover-haa2-4.vpc.cloudera.com_user_rating".
The segment "mgrover-haa2-4.vpc.cloudera.com" has been changed to an uuid (in rev. eeb7108).
Could you please verify the most recent trunk code?

> Incremental append on to Hive Parquet tables doesn't work
> ---------------------------------------------------------
>                 Key: SQOOP-2161
>                 URL: https://issues.apache.org/jira/browse/SQOOP-2161
>             Project: Sqoop
>          Issue Type: Bug
>          Components: hive-integration
>    Affects Versions: 1.4.5
>            Reporter: Mark Grover
>         Attachments: append_pre_created_verbose.log, new_pre_created_verbose.log
> I have some code that does incremental append to a Hive parquet table. I am able to get
the first time automatic table creation in Hive to work, but subsequent appends return errors.
Also, ideally, I would like to create the Hive table explicitly myself and only just do appends
to it.
> More concretely, the code I am using is at https://gist.github.com/markgrover/86f54663ece0943bc8ed
> I am also attaching two verbose error logs. Each of them contain, at the top, the command
> new_pre_created_verbose.log contains the error which occurs if I try to import data into
an empty Hive parquet table that have been created in Hive. I ran the create table statement
in the above gist and it matches the source schema one-to-one.
> In order to get past the above error, I don't run the hive create table command explicitly.
Sqoop then successfully creates the table and adds data to it. However, on the next run when
I want to append more data, I get another error which is detailed in append_pre_created_verbose.log

This message was sent by Atlassian JIRA

View raw message