hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From amit kumar <ak3...@gmail.com>
Subject Re: Unable to move files on Hive/Hdfs
Date Mon, 04 May 2015 23:16:54 GMT
Do you have any suggestion to resolve this issue,

I am looking for a resolution.

On Tue, May 5, 2015 at 4:42 AM, Moore, Douglas <
Douglas.Moore@thinkbiganalytics.com> wrote:

>   Yep, permission problem. Weird though it seems to be moving a file
> within the same dir.
>
> Thanks for the update!
>
> - Douglas
>    From: amit kumar <ak3716@gmail.com>
> Reply-To: <user@hive.apache.org>
> Date: Tue, 5 May 2015 04:40:18 +0530
> To: <user@hive.apache.org>
> Subject: Re: Unable to move files on Hive/Hdfs
>
>  Hi Doug,
>
>  I have use CDH 5.2.1
>
>  I performed the below task, and getting the error, but after rolling
> back the below changes issue has been resolved itself.
>
>  Disable ACLs on Name Nodes
> Set Enable Access Control Lists = False
> Save Changes
> Restart Hadoop Cluster
>
>
>  Thanks,
>
>
> On Tue, May 5, 2015 at 4:36 AM, amit kumar <ak3716@gmail.com> wrote:
>
>> Hi Doug,
>>
>>  I have use CDH 5.2.1
>>
>>  Disable ACLs on Name Nodes
>>
>>
>>  Set Enable Access Control Lists = False
>>
>> Save Changes
>>
>> Restart Hadoop Cluster
>>
>>
>>
>>  Stack trace:
>>
>> 2015-05-04 10:38:18,820 INFO  [main]: exec.Task
>> (SessionState.java:printInfo(537)) - Moving data to:
>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000
>> from
>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002
>>
>> 2015-05-04 10:38:18,857 ERROR [main]: exec.Task
>> (SessionState.java:printError(546)) - Failed with exception Unable to move
>> sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002
>> to destination
>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000
>>
>> org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move
>> sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002
>> to destination
>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000
>>
>>         at
>> org.apache.hadoop.hive.ql.metadata.Hive.renameFile(Hive.java:2269)
>>
>>         at
>> org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:89)
>>
>>         at
>> org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:200)
>>
>>         at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
>>
>>         at
>> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
>>
>>         at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1516)
>>
>>         at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1283)
>>
>>         at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1101)
>>
>>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:924)
>>
>>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:914)
>>
>>         at
>> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269)
>>
>>         at
>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221)
>>
>>         at
>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431)
>>
>>         at
>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:367)
>>
>>         at
>> org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:464)
>>
>>         at
>> org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:474)
>>
>>         at
>> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756)
>>
>>         at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694)
>>
>>         at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633)
>>
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>>         at java.lang.reflect.Method.invoke(Method.java:606)
>>
>>         at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>>
>> Caused by: org.apache.hadoop.hdfs.protocol.AclException: The ACL
>> operation has been rejected.  Support for ACLs has been disabled by setting
>> dfs.namenode.acls.enabled to false.
>>
>>         at
>> org.apache.hadoop.hdfs.server.namenode.NNConf.checkAclsConfigFlag(NNConf.java:85)
>>         at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:8553)
>>
>>
>>  After rolling those same changes out, the problem resolved itself.
>>
>>
>> On Tue, May 5, 2015 at 4:28 AM, Moore, Douglas <
>> Douglas.Moore@thinkbiganalytics.com> wrote:
>>
>>>   Hi Amit,
>>>
>>>  We've seen the same error on MoveTask with Hive 0.14 / HDP 2.2
>>> release. There are lots of reasons for this though.
>>> Can you provide more details about the stack trace and version so we can
>>> compare?
>>>
>>>  For our problem we've seen some relief with SET
>>> hive.metastore.client.socket.timeout=60s
>>> but the problem still happens from time to time.
>>>
>>>  Thanks,
>>> Douglas
>>>
>>>   From: amit kumar <ak3716@gmail.com>
>>> Reply-To: <user@hive.apache.org>
>>> Date: Tue, 5 May 2015 03:12:15 +0530
>>> To: <user@hive.apache.org>
>>> Subject: Unable to move files on Hive/Hdfs
>>>
>>>   While moving the data from hive/hdfs we get below error,
>>>
>>> Please suggest on this.
>>>
>>> Moving data to:
>>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911235-1/-ext-10000
>>> Failed with exception Unable to move
>>> sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911\
>>> 235-1/-ext-10002 to destination
>>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911235-1/-ext-\
>>> 10000
>>> FAILED: Execution Error, return code 1 from
>>> org.apache.hadoop.hive.ql.exec.MoveTask
>>> MapReduce Jobs Launched:
>>> Stage-Stage-1: Map: 1 Cumulative CPU: 5.83 sec HDFS Read: 553081 HDFS
>>> Write: 489704 SUCCESS
>>> Total MapReduce CPU Time Spent: 5 seconds 830 msec
>>> Error (1). Execution Failed.
>>> 2015-05-04 10:03:13 ERROR (1) in run_hive
>>>
>>> Thanks,
>>>
>>
>>
>

Mime
View raw message