hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jörn Franke <jornfra...@gmail.com>
Subject Re: s3a and hive
Date Tue, 15 Nov 2016 06:25:19 GMT
Is it a permission issue on the folder?

> On 15 Nov 2016, at 06:28, Stephen Sprague <spragues@gmail.com> wrote:
> 
> so i figured i try and set hive.metastore.warehouse.dir=s3a://bucket/hive and see what
would happen.   
> 
> running this query:
> 
>     insert overwrite table omniture.hit_data_aws partition (date_key=20161113) select
* from staging.hit_data_aws_ext_20161113 limit 1;
> 
> yields this error:
> 
>    Failed with exception java.io.IOException: rename for src path: s3a://trulia-dwr-cluster-dev/hive/omniture.db/hit_data_aws/date_key=20161113/.hive-
staging_hive_2016-11-15_04-57-52_085_7825126612479617470-1/-ext-10000/000000_0 to dest path:s3a://trulia-dwr-cluster-dev/hive/omniture.db/hit_data_aws/date_key=20161113/000000_0
returned false
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask.
java.io.IOException: rename for src path: s3a://trulia-dwr-cluster-dev/hive/omniture.db/hit_data_aws/date_key=20161113/.hive-staging_hive_2016-11-15_04-57-52_085_7825126612479617470-1/-ext-10000/000000_0
to dest path:s3a://trulia-dwr-cluster-dev/hive/omniture.db/hit_data_aws/date_key=20161113/000000_0
returned false
> 
> 
> is there any workaround?   i'm running hive 2.1.0 and hadoop version 2.6.0-cdh5.7.1 
.
> 
> 
> thanks,
> Stephen.

Mime
View raw message