hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nigel Daley <nda...@yahoo-inc.com>
Subject Re: [jira] Commented: (HADOOP-2087) Errors for subsequent requests for file creation after original DFSClient goes down..
Date Tue, 23 Oct 2007 04:02:13 GMT
Is this a blocker for 0.15?

On Oct 22, 2007, at 10:28 AM, Konstantin Shvachko (JIRA) wrote:

>
>     [ https://issues.apache.org/jira/browse/HADOOP-2087? 
> page=com.atlassian.jira.plugin.system.issuetabpanels:comment- 
> tabpanel#action_12536753 ]
>
> Konstantin Shvachko commented on HADOOP-2087:
> ---------------------------------------------
>
> I think the problem here is that Task  
> task_200710200555_0005_m_000725_0 failed and was killed while it  
> was creating a file.
> Task task_200710200555_0005_m_000725_1 was started to create the  
> same file, but the lease issued for task _0 had not expired yet.
> If task _1 waited for not more than 60 seconds, the lease would  
> expire and it would be able to claim a new lease for this file.
> I think the client should retry file creation if  
> AlreadyBeingCreatedException is caught.
>
>> Errors for subsequent requests for file creation after original  
>> DFSClient goes down..
>> --------------------------------------------------------------------- 
>> ----------------
>>
>>                 Key: HADOOP-2087
>>                 URL: https://issues.apache.org/jira/browse/ 
>> HADOOP-2087
>>             Project: Hadoop
>>          Issue Type: Bug
>>          Components: dfs
>>            Reporter: Gautam Kowshik
>>             Fix For: 0.15.0
>>
>>
>> task task_200710200555_0005_m_000725_0 started writing a file and  
>> the Node went down.. so all following file creation attempts were  
>> returned with AlreadyBeingCreatedException
>> I think the dfs should handle cases wherein, if a dfsclient goes  
>> down between file creation, subsequent creates to the same file  
>> could be allowed.
>> 2007-10-20 06:23:51,189 INFO  
>> org.apache.hadoop.mapred.TaskInProgress: Error from  
>> task_200710200555_0005_m_000725_0: Task  
>> task_200710200555_0005_m_000725_0 failed to report status for 606  
>> seconds. Killing!
>> 2007-10-20 06:23:51,189 INFO org.apache.hadoop.mapred.JobTracker:  
>> Removed completed task 'task_200710200555_0005_m_000725_0' from  
>> '[tracker_address]:/127.0.0.1:44198'
>> 2007-10-20 06:23:51,209 INFO  
>> org.apache.hadoop.mapred.JobInProgress: Choosing normal task  
>> tip_200710200555_0005_m_000725
>> 2007-10-20 06:23:51,209 INFO org.apache.hadoop.mapred.JobTracker:  
>> Adding task 'task_200710200555_0005_m_000725_1' to tip  
>> tip_200710200555_0005_m_000725, for tracker '[tracker_address]:/ 
>> 127.0.0.1:50914'
>> 2007-10-20 06:28:54,991 INFO  
>> org.apache.hadoop.mapred.TaskInProgress: Error from  
>> task_200710200555_0005_m_000725_1:  
>> org.apache.hadoop.ipc.RemoteException:  
>> org.apache.hadoop.dfs.AlreadyBeingCreatedException: failed to  
>> create file /benchmarks/TestDFSIO/io_data/test_io_825 for  
>> DFSClient_task_200710200555_0005_m_000725_1 on client  
>> 72.30.50.198, because this file is already being created by  
>> DFSClient_task_200710200555_0005_m_000725_0 on 72.30.53.224
>>         at org.apache.hadoop.dfs.FSNamesystem.startFileInternal 
>> (FSNamesystem.java:881)
>>         at org.apache.hadoop.dfs.FSNamesystem.startFile 
>> (FSNamesystem.java:806)
>>         at org.apache.hadoop.dfs.NameNode.create(NameNode.java:276)
>>         at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown  
>> Source)
>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke 
>> (DelegatingMethodAccessorImpl.java:25)
>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:379)
>>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:596)
>
> -- 
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>


Mime
View raw message