hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From amit kumar <ak3...@gmail.com>
Subject Re: Unable to move files on Hive/Hdfs
Date Mon, 04 May 2015 23:18:58 GMT
Doug,

Do i need any changes in configuration or else to resolve this issue.

Thanks

On Tue, May 5, 2015 at 4:46 AM, amit kumar <ak3716@gmail.com> wrote:

> Do you have any suggestion to resolve this issue,
>
> I am looking for a resolution.
>
> On Tue, May 5, 2015 at 4:42 AM, Moore, Douglas <
> Douglas.Moore@thinkbiganalytics.com> wrote:
>
>>   Yep, permission problem. Weird though it seems to be moving a file
>> within the same dir.
>>
>> Thanks for the update!
>>
>> - Douglas
>>    From: amit kumar <ak3716@gmail.com>
>> Reply-To: <user@hive.apache.org>
>> Date: Tue, 5 May 2015 04:40:18 +0530
>> To: <user@hive.apache.org>
>> Subject: Re: Unable to move files on Hive/Hdfs
>>
>>  Hi Doug,
>>
>>  I have use CDH 5.2.1
>>
>>  I performed the below task, and getting the error, but after rolling
>> back the below changes issue has been resolved itself.
>>
>>  Disable ACLs on Name Nodes
>> Set Enable Access Control Lists = False
>> Save Changes
>> Restart Hadoop Cluster
>>
>>
>>  Thanks,
>>
>>
>> On Tue, May 5, 2015 at 4:36 AM, amit kumar <ak3716@gmail.com> wrote:
>>
>>> Hi Doug,
>>>
>>>  I have use CDH 5.2.1
>>>
>>>  Disable ACLs on Name Nodes
>>>
>>>
>>>  Set Enable Access Control Lists = False
>>>
>>> Save Changes
>>>
>>> Restart Hadoop Cluster
>>>
>>>
>>>
>>>  Stack trace:
>>>
>>> 2015-05-04 10:38:18,820 INFO  [main]: exec.Task
>>> (SessionState.java:printInfo(537)) - Moving data to:
>>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000
>>> from
>>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002
>>>
>>> 2015-05-04 10:38:18,857 ERROR [main]: exec.Task
>>> (SessionState.java:printError(546)) - Failed with exception Unable to move
>>> sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002
>>> to destination
>>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000
>>>
>>> org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move
>>> sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10002
>>> to destination
>>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-37-43_010_529731467724830376-1/-ext-10000
>>>
>>>         at
>>> org.apache.hadoop.hive.ql.metadata.Hive.renameFile(Hive.java:2269)
>>>
>>>         at
>>> org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:89)
>>>
>>>         at
>>> org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:200)
>>>
>>>         at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
>>>
>>>         at
>>> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
>>>
>>>         at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1516)
>>>
>>>         at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1283)
>>>
>>>         at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1101)
>>>
>>>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:924)
>>>
>>>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:914)
>>>
>>>         at
>>> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269)
>>>
>>>         at
>>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221)
>>>
>>>         at
>>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431)
>>>
>>>         at
>>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:367)
>>>
>>>         at
>>> org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:464)
>>>
>>>         at
>>> org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:474)
>>>
>>>         at
>>> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756)
>>>
>>>         at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694)
>>>
>>>         at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633)
>>>
>>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>
>>>         at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>
>>>         at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>
>>>         at java.lang.reflect.Method.invoke(Method.java:606)
>>>
>>>         at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>>>
>>> Caused by: org.apache.hadoop.hdfs.protocol.AclException: The ACL
>>> operation has been rejected.  Support for ACLs has been disabled by setting
>>> dfs.namenode.acls.enabled to false.
>>>
>>>         at
>>> org.apache.hadoop.hdfs.server.namenode.NNConf.checkAclsConfigFlag(NNConf.java:85)
>>>         at
>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:8553)
>>>
>>>
>>>  After rolling those same changes out, the problem resolved itself.
>>>
>>>
>>> On Tue, May 5, 2015 at 4:28 AM, Moore, Douglas <
>>> Douglas.Moore@thinkbiganalytics.com> wrote:
>>>
>>>>   Hi Amit,
>>>>
>>>>  We've seen the same error on MoveTask with Hive 0.14 / HDP 2.2
>>>> release. There are lots of reasons for this though.
>>>> Can you provide more details about the stack trace and version so we
>>>> can compare?
>>>>
>>>>  For our problem we've seen some relief with SET
>>>> hive.metastore.client.socket.timeout=60s
>>>> but the problem still happens from time to time.
>>>>
>>>>  Thanks,
>>>> Douglas
>>>>
>>>>   From: amit kumar <ak3716@gmail.com>
>>>> Reply-To: <user@hive.apache.org>
>>>> Date: Tue, 5 May 2015 03:12:15 +0530
>>>> To: <user@hive.apache.org>
>>>> Subject: Unable to move files on Hive/Hdfs
>>>>
>>>>   While moving the data from hive/hdfs we get below error,
>>>>
>>>> Please suggest on this.
>>>>
>>>> Moving data to:
>>>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911235-1/-ext-10000
>>>> Failed with exception Unable to move
>>>> sourcehdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911\
>>>> 235-1/-ext-10002 to destination
>>>> hdfs://nameservice1/tmp/hive-srv-hdp-edh-d/hive_2015-05-04_10-02-39_841_5305383954203911235-1/-ext-\
>>>> 10000
>>>> FAILED: Execution Error, return code 1 from
>>>> org.apache.hadoop.hive.ql.exec.MoveTask
>>>> MapReduce Jobs Launched:
>>>> Stage-Stage-1: Map: 1 Cumulative CPU: 5.83 sec HDFS Read: 553081 HDFS
>>>> Write: 489704 SUCCESS
>>>> Total MapReduce CPU Time Spent: 5 seconds 830 msec
>>>> Error (1). Execution Failed.
>>>> 2015-05-04 10:03:13 ERROR (1) in run_hive
>>>>
>>>> Thanks,
>>>>
>>>
>>>
>>
>

Mime
View raw message