hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Saar <s...@adacor.com>
Subject NotReplicatedYetException
Date Wed, 09 Jul 2008 13:47:17 GMT
Hi all,

When we are copying big files in our hadoop cluster, we become between 0
and 4 times the "NotReplicatedYetException".

Is there any Solution to prevent this Exception?
I thing it slows down?

Bigger Buffers or anything else?

Each Node has 3x 1Gb/s bonded interface.

Each Datanode has an 3Ware 9650 24 Port with 24 1TB Western Digital in
RAID 6 with one Hotspare.

all Systems are CentOS 5.1 64bit with 64bit Java JDK 1.6.0_06

> 08/07/09 15:28:17 WARN dfs.DFSClient: NotReplicatedYetException sleeping /rbc/testadacor/5.dat
retries left 1
> 08/07/09 15:28:21 INFO dfs.DFSClient: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.dfs.NotReplicatedYetException:
Not replicated yet:/rbc/testadacor/5.dat
>         at org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1131)
>         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>         at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> 
>         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>         at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>         at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>         at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>         at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>         at org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
> 
> 08/07/09 15:28:21 INFO dfs.DFSClient: Waiting for replication for 6 seconds
> 08/07/09 15:28:21 WARN dfs.DFSClient: NotReplicatedYetException sleeping /rbc/testadacor/5.dat
retries left 0
> 08/07/09 15:28:27 INFO dfs.DFSClient: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.dfs.NotReplicatedYetException:
Not replicated yet:/rbc/testadacor/5.dat
>         at org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1131)
>         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>         at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> 
>         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>         at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>         at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>         at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>         at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>         at org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
> 
> 08/07/09 15:28:27 INFO dfs.DFSClient: Waiting for replication for 12 seconds
> 08/07/09 15:28:27 WARN dfs.DFSClient: NotReplicatedYetException sleeping /rbc/testadacor/5.dat
retries left -1


-- 


mit freundlichen Grüßen

Christian Saar

................................................
Adacor Hosting GmbH
Kaiserleistrasse 51
D-63067 Offenbach am Main

Telefon  +49 (0)69 905089 2110
Telefax  +49 (0)69 905089 29
Email    saar@adacor.com

Zentrale:
Telefon +49 (0)69 905089 0
Telefax +49 (0)69 905089 29
Web     http://www.adacor.com

Amtsgericht Frankfurt am Main HRB 56690
Geschäftsführer: Thomas Wittbecker, Andreas Bachmann, Patrick Fend

-------------------------------------------------------------------
This e-mail may contain confidential and/or privileged information.
If you are not the intended recipient (or have received this e-mail
in error) please notify the sender immediately and destroy this
e-mail. Any unauthorised copying, disclosure or distribution of the
contents of this e-mail is strictly prohibited.
-------------------------------------------------------------------

Mime
View raw message