hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From montag <mkra...@about.com>
Subject Re: EC2 public AMI
Date Fri, 27 Jun 2008 21:36:49 GMT

  Yes, that's the correct format.  I have to ask, you meant
hadoop-ec2-env.sh, right?

    In any case, I just successfully tested running the scripts from Mac OS
10.5.  It don't think it could be an issue with your key pair or
hadoop-ec2-env.sh .  Otherwise, I don't believe you would be able to boot a
master successfully.  To be honest, I'm kind of at a loss.  I would suggest
breaking the scripts down into their key launch components (i.e. run
launch-master and launch-slaves manually instead of just typing in
launch-cluster).  

   Were you able to build your own image using the hadoop scripts?  

Cheers,
Mike
  

TimRobertson100 wrote:
> 
> A daft question:
> 
> Is this the correct format for the AWS account id in the
> hadoop-ec2-env.txt
>  ?
> AWS_ACCOUNT_ID=XXXX-XXXX-0848
> 
> It doesn't seem to matter if I remove the "-" as always slaves and master
> start...
> 
> 
> 
> Thanks,
> 
> Tim
> 
> 
> 
> On Fri, Jun 27, 2008 at 9:44 PM, tim robertson <timrobertson100@gmail.com>
> wrote:
> 
>> Also in my .bashrc:
>> EC2_HOME=~/dev/ibiodiversity/aws/ec2-api-tools-1.3-19403
>> export EC2_HOME
>> EC2_PRIVATE_KEY=~/.ec2/pk-MNKG53RUHACIXW4M5SRVIVNGXO5LIBV6.pem
>> export EC2_PRIVATE_KEY
>> EC2_CERT=~/.ec2/cert-MNKG53RUHACIXW4M5SRVIVNGXO5LIBV6.pem
>> export EC2_CERT
>> EC2_KEYDIR=~/.ec2
>> export EC2_KEYDIR
>>
>>
>>
>>
>> On Fri, Jun 27, 2008 at 9:33 PM, tim robertson
>> <timrobertson100@gmail.com>
>> wrote:
>>
>>> Hi Mike,
>>> Thanks for taking the time to try that!  I appreciate it.
>>>
>>> I am running
>>>
>>> - Mac OSX 10.4
>>>
>>> PATH=$PATH:/usr/local/bin:$OPT_HOME/bin:/sw/bin:$MYSQL_HOME/bin:$POSTGRES_HOME/bin:$ANT_HOME/bin:$JBOSS_HOME/bin:$JAVA_HOME/bin:$ECLIPSE_HOME:$TOMCAT_HOME/bin:$MAVEN_HOME/bin:$EC2_HOME/bin
>>>
>>> To my knowledge there is nothing quirky about my network... I tried
>>> several times to start it up and always got the same.  I am not sure why
>>> my
>>> OS should make a difference - am I not just issuing a request for an
>>> Amazon
>>> instance to be created using an image from S3?  Or do some of my key
>>> files
>>> get copied across?
>>>
>>> I did get an error on the key file the very first time, saying it's
>>> permission were too open - so I ran  "chmod 600" on the keypair, after a
>>> quick google search... could it be that some files are not getting
>>> copied
>>> correctly to the master?
>>>
>>> Cheers again,
>>>
>>> Tim
>>>
>>>
>>> On Fri, Jun 27, 2008 at 9:25 PM, montag <mkramer@about.com> wrote:
>>>
>>>>
>>>> Hi Tim,
>>>>
>>>>  That's really strange.  I launched a cluster of two and ran the pi
>>>> example
>>>> just now with no issue.  This is most likely a dead end, but what OS
>>>> are
>>>> you
>>>> running the scripts on, what's your $PATH, and is there anything quirky
>>>> about your network?  It's very weird that you are able to boot up the
>>>> master, but not the slaves.
>>>>  If you have an S3 account, I would suggest trying to create your own
>>>> image
>>>> using the hadoop bundling scripts and launching a cluster based off of
>>>> that
>>>> image.  You'll need to download the AMI tools from here:
>>>>
>>>> http://developer.amazonwebservices.com/connect/entry.jspa?externalID=368&categoryID=88
>>>>
>>>> Cheers,
>>>> Mike
>>>>
>>>>
>>>>
>>>> TimRobertson100 wrote:
>>>> >
>>>> > Hi Mike,
>>>> > Thanks for the reply!
>>>> >
>>>> > I am using ec2-api-tools-1.3-19403 which I got from
>>>> >
>>>> http://developer.amazonwebservices.com/connect/entry.jspa?externalID=351today
>>>> >
>>>> > The master seems to start, and there is an instance started for a
>>>> slave
>>>> I
>>>> > can see from the EC2 command line tools, but not through the hadoop
>>>> > console...  but it doesn't seem to function (error log is attached)
>>>> >
>>>> > Thanks for taking time to reply,
>>>> >
>>>> > Tim
>>>> > (I am new to hadoop and ec2 so please don't hesitate to point the
>>>> obvious
>>>> > out to me.  I have my own mapreduce code running on single server
>>>> mode
>>>> > fine...)
>>>> >
>>>> >
>>>> >
>>>> > On Fri, Jun 27, 2008 at 8:43 PM, montag <mkramer@about.com> wrote:
>>>> >
>>>> >>
>>>> >> Hi Tim,
>>>> >>
>>>> >>  I've had no problems launching clusters using the small hadoop
>>>> 0.17.0
>>>> >> images.  Just curious, what version of the Amazon EC2 command line
>>>> tools
>>>> >> are
>>>> >> you using?
>>>> >>
>>>> >> Thanks,
>>>> >> Mike
>>>> >>
>>>> >>
>>>> >> TimRobertson100 wrote:
>>>> >> >
>>>> >> > Hi all,
>>>> >> > I have been battling EC2 all day and getting nowhere (see other
>>>> >> message)
>>>> >> >
>>>> >> > Does anyone use the hadoop-ec2-images/hadoop-0.17.0 AMI for small
>>>> >> > instances
>>>> >> > successfully?
>>>> >> >
>>>> >> > Following http://wiki.apache.org/hadoop/AmazonEC2 unfortunately
>>>> doesn't
>>>> >> > work
>>>> >> > as the slaves don't come up (details in my earlier message).
>>>> >> >
>>>> >> > Thanks for any help or pointers to getting it running on EC2.
>>>> >> >
>>>> >> > Tim
>>>> >> >
>>>> >> >
>>>> >>
>>>> >> --
>>>> >> View this message in context:
>>>> >> http://www.nabble.com/EC2-public-AMI-tp18161182p18161469.html
>>>> >> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>>>> >>
>>>> >>
>>>> >
>>>> > Start the cluster
>>>> >
>>>> > $ pwd
>>>> > /Users/timrobertson/dev/ibiodiversity/hadoop-0.17.1/src/contrib/ec2
>>>> > $ bin/hadoop-ec2 launch-cluster test-cluster 2
>>>> > Testing for existing master in group: test-cluster
>>>> > Starting master with AMI ami-ee53b687
>>>> > Waiting for instance i-62ea3c0b to start
>>>> > .............Started as domU-12-31-39-00-8A-03.compute-1.internal
>>>> > Warning: Permanently added
>>>> > 'ec2-67-202-25-117.compute-1.amazonaws.com,67.202.25.117' (RSA) to
>>>> the
>>>> > list of known hosts.
>>>> > Copying private key to master
>>>> > id_rsa-gsg-keypair
>>>> > 100% 1670     1.6KB/s   00:00
>>>> > Master is ec2-67-202-25-117.compute-1.amazonaws.com, ip is
>>>> 67.202.25.117,
>>>> > zone is us-east-1b.
>>>> > Adding test-cluster node(s) to cluster group test-cluster with AMI
>>>> > ami-ee53b687
>>>> > i-78ea3c11
>>>> > i-7bea3c12
>>>> >
>>>> >
>>>> > Wait for the nodes to start
>>>> > RESERVATION     r-cc8449a5      544208310848    test-cluster-master
>>>> > INSTANCE        i-62ea3c0b      ami-ee53b687
>>>> > ec2-67-202-25-117.compute-1.amazonaws.com
>>>> > domU-12-31-39-00-8A-03.compute-1.internal       running gsg-keypair
>>>> 0
>>>> > m1.small        2008-06-27T09:48:00+0000       
>>>> us-east-1baki-a71cf9ce
>>>> > ari-a51cf9cc
>>>> > RESERVATION     r-c28449ab      544208310848    test-cluster
>>>> > INSTANCE        i-78ea3c11      ami-ee53b687
>>>> > ec2-75-101-232-80.compute-1.amazonaws.com
>>>> > domU-12-31-39-00-84-23.compute-1.internal       running gsg-keypair
>>>> 0
>>>> > m1.small        2008-06-27T09:49:19+0000       
>>>> us-east-1baki-a71cf9ce
>>>> > ari-a51cf9cc
>>>> > INSTANCE        i-7bea3c12      ami-ee53b687
>>>> > ec2-67-202-0-48.compute-1.amazonaws.com
>>>> > domU-12-31-39-00-85-C4.compute-1.internal       running gsg-keypair
>>>> 1
>>>> > m1.small        2008-06-27T09:49:19+0000        us-east-1b
>>>> > aki-a71cf9ce       ari-a51cf9cc
>>>> >
>>>> >
>>>> > Go to the console:
>>>> > http://ec2-67-202-25-117.compute-1.amazonaws.com:50030/jobtracker.jsp
>>>> > It says here Cluster Summary - Nodes = 0
>>>> > ???
>>>> >
>>>> >
>>>> >
>>>> > Login and run the test:
>>>> > $ bin/hadoop-ec2 login test-cluster
>>>> > Logging in to host ec2-67-202-25-117.compute-1.amazonaws.com.
>>>> >
>>>> >          __|  __|_  )  Fedora 8
>>>> >          _|  (     /    32-bit
>>>> >         ___|\___|___|
>>>> >
>>>> >  Welcome to an EC2 Public Image
>>>> >                        :-)
>>>> >     Base
>>>> >
>>>> >  --[ see /etc/ec2/release-notes ]--
>>>> >
>>>> > [root@domU-12-31-39-00-8A-03 ~]# cd /usr/local/hadoop-*
>>>> > [root@domU-12-31-39-00-8A-03 hadoop-0.17.0]# bin/hadoop jar
>>>> > hadoop-*-examples.jar pi 10 10000000
>>>> >
>>>> > Output is as follows:
>>>> >
>>>> >
>>>> > 08/06/27 05:53:47 WARN fs.FileSystem:
>>>> > "domU-12-31-39-00-8A-03.compute-1.internal:50001" is a deprecated
>>>> > filesystem name. Use
>>>> > "hdfs://domU-12-31-39-00-8A-03.compute-1.internal:50001/" instead.
>>>> > Number of Maps = 10 Samples per Map = 10000000
>>>> > 08/06/27 05:53:47 WARN fs.FileSystem:
>>>> > "domU-12-31-39-00-8A-03.compute-1.internal:50001" is a deprecated
>>>> > filesystem name. Use
>>>> > "hdfs://domU-12-31-39-00-8A-03.compute-1.internal:50001/" instead.
>>>> > 08/06/27 05:53:48 WARN fs.FileSystem:
>>>> > "domU-12-31-39-00-8A-03.compute-1.internal:50001" is a deprecated
>>>> > filesystem name. Use
>>>> > "hdfs://domU-12-31-39-00-8A-03.compute-1.internal:50001/" instead.
>>>> > 08/06/27 05:53:48 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part0 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:53:48 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part0 retries left 4
>>>> > 08/06/27 05:53:48 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part0 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:53:48 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part0 retries left 3
>>>> > 08/06/27 05:53:49 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part0 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:53:49 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part0 retries left 2
>>>> > 08/06/27 05:53:51 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part0 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:53:51 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part0 retries left 1
>>>> > 08/06/27 05:53:54 WARN dfs.DFSClient: DataStreamer Exception:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part0 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> > 08/06/27 05:53:54 WARN dfs.DFSClient: Error Recovery for block null
>>>> bad
>>>> > datanode[0]
>>>> > Wrote input for Map #0
>>>> > 08/06/27 05:53:54 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part1 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:53:54 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part1 retries left 4
>>>> > 08/06/27 05:53:54 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part1 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:53:54 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part1 retries left 3
>>>> > 08/06/27 05:53:55 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part1 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:53:55 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part1 retries left 2
>>>> > 08/06/27 05:53:57 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part1 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:53:57 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part1 retries left 1
>>>> > 08/06/27 05:54:00 WARN dfs.DFSClient: DataStreamer Exception:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part1 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> > 08/06/27 05:54:00 WARN dfs.DFSClient: Error Recovery for block null
>>>> bad
>>>> > datanode[0]
>>>> > Wrote input for Map #1
>>>> > 08/06/27 05:54:00 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part2 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:54:00 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part2 retries left 4
>>>> > 08/06/27 05:54:00 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part2 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:54:00 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part2 retries left 3
>>>> > 08/06/27 05:54:01 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part2 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:54:01 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part2 retries left 2
>>>> > 08/06/27 05:54:03 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part2 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
> 
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:54:03 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part2 retries left 1
>>>> > 08/06/27 05:54:06 WARN dfs.DFSClient: DataStreamer Exception:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part2 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> > 08/06/27 05:54:06 WARN dfs.DFSClient: Error Recovery for block null
>>>> bad
>>>> > datanode[0]
>>>> > Wrote input for Map #2
>>>> > 08/06/27 05:54:06 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part3 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> Method)
>>>> >         at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:54:06 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part3 retries left 4
>>>> > 08/06/27 05:54:06 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part3 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:54:06 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part3 retries left 3
>>>> > 08/06/27 05:54:07 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part3 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:54:07 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part3 retries left 2
>>>> > 08/06/27 05:54:09 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part3 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:54:09 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part3 retries left 1
>>>> > 08/06/27 05:54:12 WARN dfs.DFSClient: DataStreamer Exception:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part3 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> > 08/06/27 05:54:12 WARN dfs.DFSClient: Error Recovery for block null
>>>> bad
>>>> > datanode[0]
>>>> > Wrote input for Map #3
>>>> > 08/06/27 05:54:12 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part4 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:54:12 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part4 retries left 4
>>>> > 08/06/27 05:54:13 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part4 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:54:13 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part4 retries left 3
>>>> > 08/06/27 05:54:13 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part4 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:54:13 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part4 retries left 2
>>>> > 08/06/27 05:54:15 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part4 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:54:15 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part4 retries left 1
>>>> > 08/06/27 05:54:18 WARN dfs.DFSClient: DataStreamer Exception:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part4 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> > 08/06/27 05:54:18 WARN dfs.DFSClient: Error Recovery for block null
>>>> bad
>>>> > datanode[0]
>>>> > Wrote input for Map #4
>>>> > 08/06/27 05:54:18 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part5 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:54:18 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part5 retries left 4
>>>> > 08/06/27 05:54:19 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part5 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:54:19 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part5 retries left 3
>>>> > 08/06/27 05:54:19 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part5 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:54:19 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part5 retries left 2
>>>> > 08/06/27 05:54:21 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part5 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:54:21 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part5 retries left 1
>>>> > 08/06/27 05:54:24 WARN dfs.DFSClient: DataStreamer Exception:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part5 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> > 08/06/27 05:54:24 WARN dfs.DFSClient: Error Recovery for block null
>>>> bad
>>>> > datanode[0]
>>>> > Wrote input for Map #5
>>>> > 08/06/27 05:54:24 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part6 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:54:24 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part6 retries left 4
>>>> > 08/06/27 05:54:25 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part6 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:54:25 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part6 retries left 3
>>>> > 08/06/27 05:54:25 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part6 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:54:25 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part6 retries left 2
>>>> > 08/06/27 05:54:27 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part6 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:54:27 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part6 retries left 1
>>>> > 08/06/27 05:54:30 WARN dfs.DFSClient: DataStreamer Exception:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part6 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> > 08/06/27 05:54:30 WARN dfs.DFSClient: Error Recovery for block null
>>>> bad
>>>> > datanode[0]
>>>> > Wrote input for Map #6
>>>> > 08/06/27 05:54:30 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part7 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:54:30 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part7 retries left 4
>>>> > 08/06/27 05:54:31 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part7 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> >
>>>> >         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> >         at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
>>>> >
>>>> > 08/06/27 05:54:31 WARN dfs.DFSClient: NotReplicatedYetException
>>>> sleeping
>>>> > /user/root/test-mini-mr/in/part7 retries left 3
>>>> > 08/06/27 05:54:31 INFO dfs.DFSClient:
>>>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>>>> > /user/root/test-mini-mr/in/part7 could only be replicated to 0 nodes,
>>>> > instead of 1
>>>> >         at
>>>> >
>>>> org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
>>>> >         at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
>>>> >         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown
>>>> Source)
>>>> >         at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:597)
>>>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> >
>>>
>>>
> 
> 

-- 
View this message in context: http://www.nabble.com/EC2-public-AMI-tp18161182p18164284.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.


Mime
View raw message