hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <shouvanik.hal...@accenture.com>
Subject RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely
Date Tue, 10 Sep 2013 11:18:18 GMT
The issue is if I paste it in core-site.xml of Hadoop on the node in question, then will starting
datanode would do only? Actually,. I am a bit confused what all daemons to start. I have a
YARN architecture running 1 separate - Resource  Manager, 1 NameNode &  4 datanodes.

From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Tuesday, September 10, 2013 4:43 PM
To: user@hive.apache.org
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely

There is not more to start. If hive is not starting properly, it will not start at all so
you are doing ok.

Problem is when you create table, hive tries to create a directory with that name if its not
existing. Just try to put those two settings inside core-site.xml and see if the hdfs can
use it to create directory

On Tue, Sep 10, 2013 at 4:06 PM, <shouvanik.haldar@accenture.com<mailto:shouvanik.haldar@accenture.com>>
wrote:
The values are showing when I run "hive> set;" command. But it's not getting identified.
I guess I am not starting HIVE properly. Do you know what are the steps?

From: Nitin Pawar [mailto:nitinpawar432@gmail.com<mailto:nitinpawar432@gmail.com>]
Sent: Tuesday, September 10, 2013 4:04 PM

To: user@hive.apache.org<mailto:user@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely

>From the config file it looks ok. Not really sure why it is not picking up when you run
it via hive cli.

Can you try putting these in core-site.xml of hdfs.




<property>

    <name>fs.s3n.awsAccessKeyId</name>

    <value>ID</value>

</property>

<property>

    <name>fs.s3n.awsSecretAccessKey</name>

    <value>SECRET</value>

</property>

On Tue, Sep 10, 2013 at 3:11 PM, <shouvanik.haldar@accenture.com<mailto:shouvanik.haldar@accenture.com>>
wrote:
Hi Nitin,

PFA hive-site.xml. I wanted to start Hortonworks HIVE. Can you please help.

Thanks,
Shouvanik

From: Nitin Pawar [mailto:nitinpawar432@gmail.com<mailto:nitinpawar432@gmail.com>]
Sent: Tuesday, September 10, 2013 11:15 AM

To: user@hive.apache.org<mailto:user@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely

if you have set these values inside hive-site.xml and restarted hive service. It should work.

Share your hive-site.xml (please remove values for s3naccess key and secret )

On Tue, Sep 10, 2013 at 3:17 AM, <shouvanik.haldar@accenture.com<mailto:shouvanik.haldar@accenture.com>>
wrote:
Please help. I am badly stuck on this. Please help.

From: Haldar, Shouvanik
Sent: Tuesday, September 10, 2013 1:03 AM

To: 'user@hive.apache.org<mailto:user@hive.apache.org>'
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely

When I set values inside hive-site.xml & test whether values are set by typing hive>
set; I see that the values are set.

Yet, when I run "CREATE EXTERNAL TABLE..." command, I get the same error

FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access
Key ID and Secret Access Key must be specified as the username or password (respectively)
of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties
(respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

Please help.

Thanks,
Shouvanik

From: Haldar, Shouvanik
Sent: Tuesday, September 10, 2013 12:39 AM
To: 'user@hive.apache.org<mailto:user@hive.apache.org>'
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely

Hi Nitin,

Please help me with the below?

Thanks,
Shouvanik

From: Haldar, Shouvanik
Sent: Saturday, September 07, 2013 2:30 AM
To: user@hive.apache.org<mailto:user@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely

Hi ,

I tried to set the parameters inside hive-site.xml on the node from where I am running hive.
Even "hive>set;" command showed me that the parameters were successfully set. Yet, when
running hive "CREATE EXTERNAL TABLE...", I am getting the following error

FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access
Key ID and Secret Access Key must be specified as the username or password (respectively)
of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties
(respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask


Please help.

Thanks,
Shouvanik

From: Nitin Pawar [mailto:nitinpawar432@gmail.com]
Sent: Saturday, September 07, 2013 2:25 AM
To: user@hive.apache.org<mailto:user@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely

did you try setting them in hive-site.xml file ?

On Sat, Sep 7, 2013 at 2:18 AM, <shouvanik.haldar@accenture.com<mailto:shouvanik.haldar@accenture.com>>
wrote:
Hi,

Can anybody please reply to this below? I am stuck up very bad.

Regards,
Shouvanik

From: Haldar, Shouvanik
Sent: Friday, September 06, 2013 11:22 PM
To: user@hive.apache.org<mailto:user@hive.apache.org>
Subject: RE: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely

Hi,

Is it possible to permanently set the below parameters? Please help.

Thanks,
SHouvanik

From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com]
Sent: Thursday, September 05, 2013 6:02 PM

To: user@hive.apache.org<mailto:user@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely

OK got your point, As an alternative you can set these variables in hive environment before
creating the table

Hive > set fs.s3n.awsSecretAccessKey='value'
Hive > set fs.s3n.awsAccessKeyId='Value'

Hope this will help you.

Thanks
Jitendra
On Thu, Sep 5, 2013 at 5:19 PM, <shouvanik.haldar@accenture.com<mailto:shouvanik.haldar@accenture.com>>
wrote:
Hi,

Thanks for the reply.

I had seen in some posts, that you could mention these keys on the Hive CLI. If things work,
then I am later planning to include it in core-site.xml of Hadoop config. Presently, I am
not allowed to restart Hadoop services.

"create external table ....
.....
LOCATION 's3n:// <fs.s3n.awsAccessKeyId: fs.s3n>.<awsSecretAccessKey >@ap-hw-dev/service-names';

Please help me.

Thanks,
Shouvanik

From: Jitendra Yadav [mailto:jeetuyadav200890@gmail.com<mailto:jeetuyadav200890@gmail.com>]
Sent: Thursday, September 05, 2013 5:11 PM
To: user@hive.apache.org<mailto:user@hive.apache.org>
Subject: Re: error executing the tutorial -> https://cwiki.apache.org/confluence/display/Hive/HiveAws+HivingS3nRemotely

Hi,

Did you configured fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties
in your core-site.xml file?

Regards
Jitendra
On Thu, Sep 5, 2013 at 3:52 PM, <shouvanik.haldar@accenture.com<mailto:shouvanik.haldar@accenture.com>>
wrote:
Hi,

I am executing the example, but got initial roadblock while trying out "Setting up tables
(DDL Statements)" section.

When I executed the command "hive> create external table kv (key int, values string)  location
's3n://data.s3ndemo.hive/kv';"

I got the error

FAILED: Error in metadata: MetaException(message:java.lang.IllegalArgumentException: AWS Access
Key ID and Secret Access Key must be specified as the username or password (respectively)
of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties
(respectively).)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive>

Please help.


Thanks,
Shouvanik

________________________________
This message is for the designated recipient only and may contain privileged, proprietary,
or otherwise confidential information. If you have received it in error, please notify the
sender immediately and delete the original. Any other use of the e-mail by you is prohibited.

Where allowed by local law, electronic communications with Accenture and its affiliates, including
e-mail and instant messaging (including content), may be scanned by our systems for the purposes
of information security and assessment of internal compliance with Accenture policy.

______________________________________________________________________________________

www.accenture.com<http://www.accenture.com/>





--
Nitin Pawar



--
Nitin Pawar



--
Nitin Pawar



--
Nitin Pawar

Mime
View raw message