hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Seidman <jonathan.seid...@gmail.com>
Subject Re: "Path Is Not Legal" when loading HDFS->S3
Date Mon, 26 Sep 2011 22:28:07 GMT
Hey Bradford - from my experience that error occurs when there's a conflict
between the "default.fs.name" setting and the value in the
metastore.SDS.location column in the Hive metadata. For us this has occurred
when either migrating to a new cluster or changing the NN hostname. Not sure
how all this works with AWS/EMR, but that's the first thing I'd check.

Jonathan

On Mon, Sep 26, 2011 at 5:16 PM, Bradford Stephens <
bradfordstephens@gmail.com> wrote:

> Hey amigos,
>
> I'm doing a EMR load for HDFS to S3 data. My example looks correct,
> but I'm getting an odd error. Since all the EMR data is in one
> directory, I'm copying the file to HDFS, then doing 'LOAD DATA INPATH'
> to put it back into S3.
>
> CREATE TABLE events(
> ..blahblah...
> )
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ','
> STORED AS TEXTFILE
> LOCATION 's3://outputdir/table_out/events'
> ;
>
> LOAD DATA INPATH '/user/hadoop/eos/events_20110107.csv.gz' overwrite
> INTO TABLE events;
>
> The error I get is:
> FAILED: Error in semantic analysis: line 3:17 Path is not legal
> '/user/hadoop/eos/events_20110430.csv.gz': Move from:
>
> hdfs://domU-12-31-39-14-19-F1.compute-1.internal:9000/user/hadoop/eos/events_20110430.csv.gz
> to: s3://outputdir/table_out/events is not valid. Please check that
> values for params "default.fs.name" and "hive.metastore.warehouse.dir"
> do not conflict.
>
> This is EMR, and I've checked the params and see they do not conflict.
>
>
> --
> Bradford Stephens,
> CEO and Founder, Drawn to Scale
> http://drawntoscale.com
> (530) 763-DATA
>
> http://www.drawntoscale.com -- Spire, the "Heroku for Big Data"
>

Mime
View raw message