hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "S.L" <simpleliving...@gmail.com>
Subject Hadoop Spill Exception
Date Fri, 18 Apr 2014 03:13:43 GMT
Hi All,

I am running Nutch on Hadoop2.3.0 and I get Hadoop Spill Exception even
though the disk space that is being utilized is only 30% of the available.

I am looking at the syslog file on the userlogs directory however , I do
not see much information there  except the following line. How do I know
whats causing this spill exception as disk space does not seem to be the

I see the following entry in the syslog file.

2014-04-17 12:17:28,517 INFO [FetcherThread]
org.apache.hadoop.mapred.MapTask: kvstart = 6159120(24636480); kvend =
6152164(24608656); length = 6957/6553600
2014-04-17 12:17:28,528 INFO [FetcherThread]
org.apache.hadoop.mapred.MapTask: (EQUATOR) 3661729 kvi 915428(3661712)
2014-04-17 12:17:28,529 ERROR [FetcherThread]
org.apache.nutch.fetcher.Fetcher: fetcher caught:java.io.IOException: Spill

Thanks for your help.

View raw message