hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Miguel Cabero <miguel.cab...@gmail.com>
Subject Re: Hive Problem
Date Tue, 29 Nov 2011 12:47:15 GMT
Hi Jurgen,

May be you can find some hints in http://www.slideshare.net/kate_ting/habits-of-effective-sqoop-users

Regards,

Miguel

On 29 Nov 2011, at 00:44, arvind@cloudera.com wrote:

> Hi Jurgen,
> 
> What version of Hive and Sqoop are you using? Also, please look under /tmp/${USER}/hive.log
file which will have more detailed information on what may be going wrong.
> 
> Thanks,
> Arvind 
> 
> On Mon, Nov 28, 2011 at 3:17 PM, Jurgen Van Gael <jurgen@rangespan.com> wrote:
> Hi,
> I am running the Cloudera CDH3 Hive distribution in pseudo-distributed
> mode on my local Mac OS Lion laptop. Hive generally works fine except
> when I use it together with Sqoop. A command like
> 
> sqoop import --connect jdbc:mysql://localhost/db --username root
> --password foobar --table sometable --warehouse-dir
> /user/hive/warehouse
> 
> completes succesfully and generates part_files, a _logs directory and
> a _SUCCESS file in the hive warehouse directory on HDFS. However, when
> I add the --import-hive part to the Sqoop command, the import still
> works but Hive seems to get into an infinite loop. Looking at the logs
> I find entries
> 
> 2011-11-28 22:54:57,279 WARN org.apache.hadoop.hdfs.StateChange: DIR*
> FSDirectory.unprotectedRenameTo: failed to rename
> /user/hive/warehouse/sometable/_SUCCESS to
> /user/hive/warehouse/sometable/_SUCCESS_copy_2 because source does not
> exist
> 2011-11-28 22:54:57,281 WARN org.apache.hadoop.hdfs.StateChange: DIR*
> FSDirectory.unprotectedRenameTo: failed to rename
> /user/hive/warehouse/sometable/_SUCCESS to
> /user/hive/warehouse/sometable/_SUCCESS_copy_3 because source does not
> exist
> 2011-11-28 22:54:57,282 WARN org.apache.hadoop.hdfs.StateChange: DIR*
> FSDirectory.unprotectedRenameTo: failed to rename
> /user/hive/warehouse/sometable/_SUCCESS to
> /user/hive/warehouse/sometable/_SUCCESS_copy_4 because source does not
> exist
> 
> I started digging into the source code and can trace it back to
> ql/metadata/Hive.java:checkPaths which tries to find a name for a
> _SUCCESS file during the actual Hive load but somehow fails because
> the Sqoop import MR job already created a _SUCCESS file. I already
> tried disabling MR creation of _SUCCESS files but Hive seems to wait
> for that file to kick off the Hive import and hence fails.
> 
> Does anyone have any suggestions on where to search next?
> 
> Thanks! Jurgen
> 


Mime
View raw message