hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Kimball <aa...@cloudera.com>
Subject Re: datanode auto down
Date Tue, 21 Jul 2009 00:14:50 GMT
A VersionMismatch occurs because you're using different builds of Hadoop on
your different nodes. All DataNodes and the NameNode must be running the
exact same compilation of Hadoop (It's very strict).

One thing I noticed in your config is that you set dfs.block.size to
2560000. That's certainly not the default value. Note that this value is in
bytes. That's 2.5 MB/block. If you've got a million or more files in there,
of any reasonable size, then you might have several million blocks.
Depending on how powerful your NameNode's hardware is, that could take up a
lot of RAM. Each block is 192 bytes, and each file is 124 bytes + the
filename length.

When you run 'bin/hadoop dfsadmin -report' or look at the status JSP at
http://named:50070/, how much RAM free does it report? You might be using up
a lot of the NameNode's available RAM, which could cause various problems
involving spending all its time in GC.

In the future I'd recommend setting your block size 5 or 10x higher. Note
that this won't affect existing blocks/files though. You may need to somehow
read and rewrite those files to take advantage of a larger block size to
alleviate block pressure.

- Aaron


On Sun, Jul 19, 2009 at 3:59 PM, mingyang <bjmingyang@gmail.com> wrote:

> time find / b / c / d / e / f / g / h / i / j / k / l-type f-print> / dev /
> null
>
> real 0m4.608s
> user 0m1.489s
> sys 0m3.068s
>
> I run this, the result is far less than eight minutes, is not where I set
> up
> right? I just changed the fs.default.name, other settings are default
> values
>
>   <name> fs.default.name </ name>
>   <value> hdfs: / / named: 9000 </ value>
>
>
>
>   <name> mapred.job.tracker </ name>
>   <value> hdfs: / / named: 9001 </ value>
>
>
>
>   <name> hadoop.tmp.dir </ name>
>   <value> / hadoop / tmp </ value>
>
>   <name> dfs.name.dir </ name>
>   <value> / hadoop / name </ value>
>
>
>   <name> dfs.data.dir </ name>
>   <value> / b, / c, / d, / e, / f, / g, / h, / i, / j, / k, / l </ value>
>
>
>
>   <name> dfs.replication </ name>
>   <value> 1 </ value>
>
>
>   <name> dfs.block.size </ name>
>   <value> 2560000 </ value>
>
>
>
> 2009/7/20 Jason Venner <jason.hadoop@gmail.com>
>
> > Did you run this command on the datanode that not responding?
> >
> > On Sun, Jul 19, 2009 at 3:59 AM, mingyang <bjmingyang@gmail.com> wrote:
> >
> > > in datanode logs , I found a new error message.
> > > Would like to help solve the problem
> > >
> > > 2009-07-19 18:40:43,464 ERROR
> > > org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(
> > > 192.168.1.97:50010,
> > > storageID=DS-148204255-192.168.1.97-50010-1247630449653,
> > > infoPort=50075, ipcPort=50020):DataXceiver
> > > java.io.IOException: Version Mismatch
> > >        at
> > >
> >
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:81)
> > >        at java.lang.Thread.run(Thread.java:619)
> > > 2009-07-19 18:40:43,464 INFO
> > > org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block
> > > blk_9198148243909908835_1418899 src: /192.168.1.94:21486 dest: /
> > > 192.168.1.97:50010
> > > 2009-07-19 18:41:43,363 INFO
> > > org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block
> > > blk_4464496078754951058_1418901 src: /192.168.1.105:17434 dest: /
> > > 192.168.1.97:50010
> > > 2009-07-19 18:41:55,318 INFO
> > > org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block
> > > blk_-998769390764365324_1418903 src: /192.168.1.105:17436 dest: /
> > > 192.168.1.97:50010
> > > 2009-07-19 18:42:51,983 INFO
> > > org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block
> > > blk_7357805261059813870_1418903 src: /192.168.1.94:16148 dest: /
> > > 192.168.1.97:50010
> > > 2009-07-19 18:43:00,936 INFO
> > > org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block
> > > blk_6766332640664853809_1418903 src: /192.168.1.105:60392 dest: /
> > > 192.168.1.97:50010
> > > 2009-07-19 18:44:03,112 INFO
> > > org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block
> > > blk_8833189579369772797_1418903 src: /192.168.1.94:16152 dest: /
> > > 192.168.1.97:50010
> > > 2009-07-19 18:44:07,105 INFO
> > > org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block
> > > blk_-92272670489114760_1418903 src: /192.168.1.94:16153 dest: /
> > > 192.168.1.97:50010
> > > 2009-07-19 18:45:15,301 INFO
> > > org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block
> > > blk_-3189060218922169017_1418903 src: /192.168.1.94:16157 dest: /
> > > 192.168.1.97:50010
> > > 2009-07-19 18:45:19,299 INFO
> > > org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block
> > > blk_2093621896162975184_1418903 src: /192.168.1.105:60400 dest: /
> > > 192.168.1.97:50010
> > > 2009-07-19 18:46:27,602 INFO
> > > org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block
> > > blk_-2879021713701012781_1418905 src: /192.168.1.105:60404 dest: /
> > > 192.168.1.97:50010
> > > 2009-07-19 18:46:27,602 INFO
> > > org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block
> > > blk_356680792598474797_1418907 src: /192.168.1.105:60406 dest: /
> > > 192.168.1.97:50010
> > > 2009-07-19 18:47:21,366 INFO
> > > org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in
> > receiveBlock
> > > for block blk_-8134427742713358791_1418899 java.io.EOFException: while
> > > trying to read 65557 bytes
> > > 2009-07-19 18:47:21,367 INFO
> > > org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder 0 for
> > > block
> > > blk_-8134427742713358791_1418899 Interrupted.
> > > 2009-07-19 18:47:21,367 INFO
> > > org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder 0 for
> > > block
> > > blk_-8134427742713358791_1418899 terminating
> > > 2009-07-19 18:47:21,367 INFO
> > > org.apache.hadoop.hdfs.server.datanode.DataNode: writeBlock
> > > blk_-8134427742713358791_1418899 received exception
> java.io.EOFException:
> > > while trying to read 65557 bytes
> > > 2009-07-19 18:47:21,367 ERROR
> > > org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(
> > > 192.168.1.97:50010,
> > > storageID=DS-148204255-192.168.1.97-50010-1247630449653,
> > > infoPort=50075, ipcPort=50020):DataXceiver
> > > java.io.EOFException: while trying to read 65557 bytes
> > >        at
> > >
> > >
> >
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.readToBuf(BlockReceiver.java:264)
> > >        at
> > >
> > >
> >
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.readNextPacket(BlockReceiver.java:308)
> > >        at
> > >
> > >
> >
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:372)
> > >        at
> > >
> > >
> >
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:524)
> > >        at
> > >
> > >
> >
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:357)
> > >        at
> > >
> > >
> >
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:103)
> > >        at java.lang.Thread.run(Thread.java:619)
> > > 2009-07-19 18:47:21,368 INFO
> > > org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in
> > receiveBlock
> > > for block blk_-346503319966521195_1418897 java.io.EOFException: while
> > > trying
> > > to read 65557 bytes
> > > 2009-07-19 18:47:21,371 INFO
> > > org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder 0 for
> > > block
> > > blk_-346503319966521195_1418897 Interrupted.
> > > 2009-07-19 18:47:21,373 INFO
> > > org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder 0 for
> > > block
> > > blk_-346503319966521195_1418897 terminating
> > >
> > > 2009/7/19 mingyang <bjmingyang@gmail.com>
> > >
> > > > But I added datanode still appear in this phenomenon, and I added a
> new
> > > > datanode, the above data only 10g
> > > > Strangely enough, wait about 10 minutes, hadoop will resume, I do not
> > > know
> > > > what the problem is this, ask for help
> > > >
> > > >
> > > > it‘s log
> > > > 09/07/19 18:46:13 INFO hdfs.DFSClient: Waiting to find target node:
> > > > 192.168.1.97:50010
> > > > 09/07/19 18:46:13 INFO hdfs.DFSClient: Exception in
> > > createBlockOutputStream
> > > > java.net.SocketTimeoutException: 66000 millis timeout wh
> > > > ile waiting for channel to be ready for read. ch :
> > > > java.nio.channels.SocketChannel[connected local=/192.168.1.94:16153
> > > remote=/192.1
> > > > 68.1.97:50010]
> > > > 09/07/19 18:46:08 INFO hdfs.DFSClient: Waiting to find target node:
> > > > 192.168.1.97:50010
> > > > 09/07/19 18:46:08 INFO hdfs.DFSClient: Exception in
> > > createBlockOutputStream
> > > > java.net.SocketTimeoutException: 66000 millis timeout wh
> > > > ile waiting for channel to be ready for read. ch :
> > > > java.nio.channels.SocketChannel[connected local=/192.168.1.94:16152
> > > remote=/192.1
> > > > 68.1.97:50010]
> > > > 09/07/19 18:45:01 INFO hdfs.DFSClient: Waiting to find target node:
> > > > 192.168.1.105:50010
> > > > 09/07/19 18:45:01 INFO hdfs.DFSClient: Exception in
> > > createBlockOutputStream
> > > > java.io.IOException: Bad connect ack with firstBadLink 1
> > > > 92.168.1.97:50010
> > > > 09/07/19 18:44:56 INFO hdfs.DFSClient: Waiting to find target node:
> > > > 192.168.1.97:50010
> > > > 09/07/19 18:44:56 INFO hdfs.DFSClient: Exception in
> > > createBlockOutputStream
> > > > java.net.SocketTimeoutException: 66000 millis timeout wh
> > > > ile waiting for channel to be ready for read. ch :
> > > > java.nio.channels.SocketChannel[connected local=/192.168.1.94:16148
> > > remote=/192.1
> > > > 68.1.97:50010]
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > 2009/7/17 mingyang <bjmingyang@gmail.com>
> > > >
> > > > I run this, the result is far less than eight minutes, is not where I
> > set
> > > >> up right? I just changed the fs.default.name, other settings are
> > > default
> > > >> values
> > > >>    <name> fs.default.name </ name>
> > > >>    <value> hdfs: / / named: 9000 </ value>
> > > >>
> > > >>
> > > >>
> > > >>    <name> mapred.job.tracker </ name>
> > > >>    <value> hdfs: / / named: 9001 </ value>
> > > >>
> > > >>
> > > >>
> > > >>    <name> hadoop.tmp.dir </ name>
> > > >>    <value> / hadoop / tmp </ value>
> > > >>
> > > >>    <name> dfs.name.dir </ name>
> > > >>    <value> / hadoop / name </ value>
> > > >>
> > > >>
> > > >>    <name> dfs.data.dir </ name>
> > > >>    <value> / b, / c, / d, / e, / f, / g, / h, / i, / j, / k,
/ l </
> > > value>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>    <name> dfs.replication </ name>
> > > >>    <value> 1 </ value>
> > > >>
> > > >>
> > > >>    <name> dfs.block.size </ name>
> > > >>    <value> 2560000 </ value>
> > > >>
> > > >>
> > > >>
> > > >> time find / b / c / d / e / f / g / h / i / j / k / l-type f-print>
> /
> > > dev
> > > >> / null
> > > >>
> > > >> real 0m4.608s
> > > >> user 0m1.489s
> > > >> sys 0m3.068s
> > > >>
> > > >>
> > > >> 2009/7/17 jason hadoop <jason.hadoop@gmail.com>
> > > >>
> > > >>> From the shell,
> > > >>> time find /b /c /d /e /f /g /h /i /j /k /l -type f -print >
> /dev/null
> > > >>>
> > > >>> Also, unless your distribution is magic, white space is not ignored
> > in
> > > >>> value
> > > >>> statements. Hopefully your actual value is
> > > >>> <value>/b,/c,/d,/e,/f,/g,/h,/i,/j,/k,/l</value>
> > > >>>
> > > >>> On Fri, Jul 17, 2009 at 6:29 AM, mingyang <bjmingyang@gmail.com>
> > > wrote:
> > > >>>
> > > >>> > Thank you for your reply, I am novice, I do not quite understand
> > how
> > > to
> > > >>> > check dfs.data.dir more than eight minutes?
> > > >>> > My settings are dfs.data.dir
> > > >>> > <property>
> > > >>> >   <name> dfs.data.dir </ name>
> > > >>> >   <value> / b, / c, / d, / e, / f, / g, / h, / i, /
j, / k, / l
> </
> > > >>> value>
> > > >>> >   </ property>
> > > >>> >
> > > >>> > Each directory has a disk mount
> > > >>> >
> > > >>> > 2009/7/17 jason hadoop <jason.hadoop@gmail.com>
> > > >>> >
> > > >>> > > Most likely problem is that the block report is taking
more
> than
> > 10
> > > >>> > > minutes.
> > > >>> > > Due to the placement of the sync blocks in the core
Datanode
> > code,
> > > >>> the
> > > >>> > > block
> > > >>> > > report locks out the heartbeat.
> > > >>> > > This can cause the namenode to think the datanode has
vanished.
> > > >>> > >
> > > >>> > > A simple way to check, is to run a find on the directory
set
> > > >>> specified
> > > >>> > for
> > > >>> > > dfs.data.dir. If this find takes more than 8 minutes
or so, you
> > are
> > > >>> in
> > > >>> > > trouble.
> > > >>> > > The only solutions are to add more datanodes, reducing
the
> block
> > > >>> count,
> > > >>> > or
> > > >>> > > increase your system io speed so that the block report
may
> > complete
> > > >>> in
> > > >>> > > time.
> > > >>> > >
> > > >>> > > On Fri, Jul 17, 2009 at 6:12 AM, mingyang <
> bjmingyang@gmail.com>
> > > >>> wrote:
> > > >>> > >
> > > >>> > > > i using hadoop storage my media files,, but when
the number
> of
> > > >>> > documents
> > > >>> > > > when more than one million,
> > > >>> > > > Hadoop start about 10-20 minutes, my datanode automatically
> > down,
> > > >>> > > > namenode log shows that the loss of heart, but
I see my
> normal
> > > >>> > datanode,
> > > >>> > > > port 50010 can be a normal telnet, use jps to see
can see
> > > datanode
> > > >>> > still
> > > >>> > > > running, but at this time have been unable to put
data to a
> > > hadoop,
> > > >>> I
> > > >>> > > guess
> > > >>> > > > datanode services is dead, hadoop does not support
more than
> > one
> > > >>> > million
> > > >>> > > > documents? How do I adjust those parameters? I
have already
> set
> > > up
> > > >>> at
> > > >>> > the
> > > >>> > > > same time the number of open file 65535
> > > >>> > > >
> > > >>> > > >
> > > >>> > > > namenode log
> > > >>> > > >
> > > >>> > > > 2009-07-17 18:14:29,330 INFO
> > > >>> > > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit:
> > > >>> > > >
> > > >>> > > > ugi=root,root,bin,daemon,sys,adm,disk,wheelip=/192.168.1.96
> > > >>> > > > cmd=setPermission
> > > >>> > > >
> > > >>> > > > src=/hadoop/tmp/mapred/system/jobtracker.info 
 dst=null
> > > >>> > > > perm=root:supergroup
> > > >>> > > > :rw-------
> > > >>> > > > 2009-07-17 18:14:29,336 INFO
> > org.apache.hadoop.hdfs.StateChange:
> > > >>> BLOCK*
> > > >>> > > > NameSystem.allocateBlock:
> > > >>> > > >
> > > >>> > > > /hadoop/tmp/mapred/system/jobtrack
> > > >>> > > > er.info. blk_-2148480138731090754_1403179
> > > >>> > > > 2009-07-17 18:14:32,958 INFO
> > org.apache.hadoop.hdfs.StateChange:
> > > >>> BLOCK*
> > > >>> > > > NameSystem.addStoredBlock: blockMap updated:
> > > >>> > > >
> > > >>> > > > 192.168.1.97:50
> > > >>> > > > 010 is added to blk_-2148480138731090754_1403179
size 4
> > > >>> > > > 2009-07-17 18:14:33,340 INFO
> > org.apache.hadoop.hdfs.StateChange:
> > > >>> DIR*
> > > >>> > > > NameSystem.completeFile: file
> > > >>> > > >
> > > >>> > > > /hadoop/tmp/mapred/system/jobtra
> > > >>> > > > cker.info is closed by DFSClient_1037557306
> > > >>> > > > 2009-07-17 18:16:21,349 INFO
> > > >>> > > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
Roll
> Edit
> > > Log
> > > >>> from
> > > >>> > > > 192.168.1.96
> > > >>> > > > 2009-07-17 18:16:21,349 INFO
> > > >>> > > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
Number
> of
> > > >>> > > > transactions:
> > > >>> > > > 7 Total time for
> > > >>> > > >
> > > >>> > > > transacti
> > > >>> > > > ons(ms): 1Number of transactions batched in Syncs:
1 Number
> of
> > > >>> syncs: 6
> > > >>> > > > SyncTimes(ms): 9
> > > >>> > > > 2009-07-17 18:17:12,171 INFO
> > > >>> > > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
Roll
> > FSImage
> > > >>> from
> > > >>> > > > 192.168.1.96
> > > >>> > > > 2009-07-17 18:17:12,171 INFO
> > > >>> > > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
Number
> of
> > > >>> > > > transactions:
> > > >>> > > > 0 Total time for
> > > >>> > > >
> > > >>> > > > transacti
> > > >>> > > > ons(ms): 0Number of transactions batched in Syncs:
0 Number
> of
> > > >>> syncs: 1
> > > >>> > > > SyncTimes(ms): 0
> > > >>> > > > 2009-07-17 18:51:00,566 INFO
> > org.apache.hadoop.hdfs.StateChange:
> > > >>> BLOCK*
> > > >>> > > > NameSystem.heartbeatCheck: lost heartbeat from
> > > >>> > > >
> > > >>> > > > 192.168.1.97:
> > > >>> > > > 50010
> > > >>> > > > 2009-07-17 18:51:25,383 INFO
> > > org.apache.hadoop.net.NetworkTopology:
> > > >>> > > > Removing
> > > >>> > > > a node: /default-rack/192.168.1.97:50010
> > > >>> > > > 2009-07-17 19:10:48,564 INFO
> > > >>> > > > org.apache.hadoop.hdfs.server.namenode.LeaseManager:
Lease
> > > [Lease.
> > > >>> > >  Holder:
> > > >>> > > > DFSClient_-
> > > >>> > > >
> > > >>> > > > 1624377199, pend
> > > >>> > > > ingcreates: 69] has expired hard limit
> > > >>> > > > 2009-07-17 19:10:48,564 INFO
> > > >>> > > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
> Recovering
> > > >>> > > > lease=[Lease.  Holder:
> > > >>> > > >
> > > >>> > > > DFSClient_-16243
> > > >>> > > > 77199, pendingcreates: 69],
> > > >>> > > >
> > > >>>
> > src=/unp/01/video/B3/94/{B394EDB2-0302-34B9-5357-4904FFFEFF36}_100.unp
> > > >>> > > >
> > > >>> > > > datanode log
> > > >>> > > >
> > > >>> > > > 2009-07-17 18:52:40,719 INFO
> > > >>> > > > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner:
> > > >>> Verification
> > > >>> > > > succeeded for blk_664105388033641
> > > >>> > > > 1514_601647
> > > >>> > > > 2009-07-17 18:52:12,421 INFO
> > > >>> > > > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner:
> > > >>> Verification
> > > >>> > > > succeeded for blk_-10747966535898
> > > >>> > > > 19594_1392025
> > > >>> > > > 2009-07-17 18:51:44,074 INFO
> > > >>> > > > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner:
> > > >>> Verification
> > > >>> > > > succeeded for blk_-63504301593802
> > > >>> > > > 31402_155334
> > > >>> > > > 2009-07-17 18:51:12,760 INFO
> > > >>> > > > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner:
> > > >>> Verification
> > > >>> > > > succeeded for blk_460729998775184
> > > >>> > > > 5359_395290
> > > >>> > > > 2009-07-17 18:50:39,977 INFO
> > > >>> > > > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner:
> > > >>> Verification
> > > >>> > > > succeeded for blk_802918354954113
> > > >>> > > > 9011_474989
> > > >>> > > > 2009-07-17 18:50:11,707 INFO
> > > >>> > > > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner:
> > > >>> Verification
> > > >>> > > > succeeded for blk_846865664811904
> > > >>> > > > 9754_1065465
> > > >>> > > > 2009-07-17 18:49:39,421 INFO
> > > >>> > > > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner:
> > > >>> Verification
> > > >>> > > > succeeded for blk_473953565994615
> > > >>> > > > 8302_532204
> > > >>> > > > 2009-07-17 18:49:11,213 INFO
> > > >>> > > > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner:
> > > >>> Verification
> > > >>> > > > succeeded for blk_-14950858387931
> > > >>> > > > 09024_354553
> > > >>> > > >
> > > >>> > > > 09/07/17 18:02:09 WARN hdfs.DFSClient: DFSOutputStream
> > > >>> > ResponseProcessor
> > > >>> > > > exception  for block
> > > >>> > > >
> > blk_-2536746364442878375_1403164java.net.SocketTimeoutException:
> > > >>> 63000
> > > >>> > > > millis timeout while waiting for channel to be
ready for
> read.
> > ch
> > > :
> > > >>> > > > java.nio.channels.SocketChannel[connected local=/
> > > >>> 192.168.1.94:54783
> > > >>> > > remote=/
> > > >>> > > > 192.168.1.97:50010]
> > > >>> > > > 11096473        09/07/17 18:02:12 INFO hdfs.DFSClient:
> > Exception
> > > in
> > > >>> > > > createBlockOutputStream java.net.SocketTimeoutException:
> 63000
> > > >>> millis
> > > >>> > > > timeout while waiting for channel to be ready for
read. ch :
> > > >>> > > > java.nio.channels.SocketChannel[connected local=/
> > > >>> 192.168.1.94:54790
> > > >>> > > remote=/
> > > >>> > > > 192.168.1.97:50010]
> > > >>> > > > 11096475        09/07/17 18:02:12 INFO hdfs.DFSClient:
> > Exception
> > > in
> > > >>> > > > createBlockOutputStream java.net.SocketTimeoutException:
> 63000
> > > >>> millis
> > > >>> > > > timeout while waiting for channel to be ready for
read. ch :
> > > >>> > > > java.nio.channels.SocketChannel[connected local=/
> > > >>> 192.168.1.94:54791
> > > >>> > > remote=/
> > > >>> > > > 192.168.1.97:50010]
> > > >>> > > >
> > > >>> > >
> > > >>> > >
> > > >>> > >
> > > >>> > > --
> > > >>> > > Pro Hadoop, a book to guide you from beginner to hadoop
> mastery,
> > > >>> > > http://www.amazon.com/dp/1430219424?tag=jewlerymall
> > > >>> > > www.prohadoopbook.com a community for Hadoop Professionals
> > > >>> > >
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> > --
> > > >>> > 致
> > > >>> > 礼!
> > > >>> >
> > > >>> >
> > > >>> > 王明阳
> > > >>> >
> > > >>>
> > > >>>
> > > >>>
> > > >>> --
> > > >>> Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> > > >>> http://www.amazon.com/dp/1430219424?tag=jewlerymall
> > > >>> www.prohadoopbook.com a community for Hadoop Professionals
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> 致
> > > >> 礼!
> > > >>
> > > >>
> > > >> 王明阳
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > 致
> > > > 礼!
> > > >
> > > >
> > > > 王明阳
> > > >
> > >
> > >
> > >
> > > --
> > > 致
> > > 礼!
> > >
> > >
> > > 王明阳
> > >
> >
> >
> >
> > --
> > Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> > http://www.amazon.com/dp/1430219424?tag=jewlerymall
> > www.prohadoopbook.com a community for Hadoop Professionals
> >
>
>
>
> --
> 致
> 礼!
>
>
> 王明阳
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message