hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcus Herou <marcus.he...@tailsweep.com>
Subject Re: Lingering TaskTracker$Child
Date Sun, 25 Jan 2009 17:05:53 GMT
Oh, fuck you mean modding the hadoop TaskTracker child code ?

On Sun, Jan 25, 2009 at 6:04 PM, Marcus Herou <marcus.herou@tailsweep.com>wrote:

> Thanks!
>
> So by you experience would this be good enough ? (Notice the
> System.exit(0))
>
> I implement the MapRunnable interface.
>
>  public void run(RecordReader<LongWritable, Text> recordReader,
> OutputCollector<WritableComparable, WritableComparable> outputCollector,
> Reporter reporter) throws IOException
>     {
>         this.recordReader = recordReader;
>         this.outputCollector = outputCollector;
>         this.reporter = reporter;
>         int threads =
> Integer.valueOf(this.getConf().get(getClass().getName()+".threads", "10"));
>         log.info("Starting with "+threads + " threads");
>         long timeout =
> Long.valueOf(this.getConf().get(getClass().getName()+".timeout", "600000"));
>
>         for (int i = 0; i < threads; i++)
>         {
>             // spawn threads
>             new FetcherThread().start();
>         }
>         do
>         {                                          // wait for threads to
> exit
>             try
>             {
>                 Thread.sleep(1000);
>             } catch (InterruptedException e) {}
>
>             reportStatus();
>
>             // some requests seem to hang, despite all intentions
>             synchronized (this)
>             {
>                 if ((System.currentTimeMillis() - lastRequestStart) >
> timeout)
>                 {
>                     if (log.isWarnEnabled())
>                     {
>                         log.warn("Aborting with "+activeThreads+" hung
> threads.");
>                     }
>                     return;
>                 }
>             }
>
>         } while (activeThreads > 0);
>         log.info("All threads seem to be done, exiting");
>         System.exit(0);
>
>     }
>
> On Sun, Jan 25, 2009 at 5:57 PM, jason hadoop <jason.hadoop@gmail.com>wrote:
>
>> We had trouble like that with some jobs, when the child ran additional
>> threads that were not set at daemon priority. These hold the Child JVM
>> from
>> exiting.
>> JMX was the cause in our case, but we have seen our JNI jobs do it also.
>> In the end we made a local mod that forced a System.exit in the finally
>> block of the Child main.
>>
>>
>> On Sun, Jan 25, 2009 at 8:53 AM, Marcus Herou <marcus.herou@tailsweep.com
>> >wrote:
>>
>> > Some extra info, apparently the child exits with a status of 143.
>> >
>> > 2009-01-25 17:13:11.110 INFO IPC Server handler 0 on 32274
>> > org.apache.hadoop.mapred.TaskTracker -
>> attempt_200901251629_0011_m_000005_0
>> > 1.0% 5364 docs, 0 errors, 124.7 docs/s
>> > 2009-01-25 17:13:11.114 INFO IPC Server handler 1 on 32274
>> > org.apache.hadoop.mapred.TaskTracker - Task
>> > attempt_200901251629_0011_m_000005_0 is done.
>> > 2009-01-25 17:13:11.116 INFO main org.apache.hadoop.mapred.TaskRunner -
>> > Task
>> > 'attempt_200901251629_0011_m_000005_0' done.
>> > 2009-01-25 17:13:12.644 INFO IPC Server handler 0 on 32274
>> > org.apache.hadoop.mapred.TaskTracker -
>> attempt_200901251629_0011_m_000005_0
>> > 1.0% 5364 docs, 0 errors, 124.7 docs/s
>> > 2009-01-25 17:13:12.644 INFO IPC Server handler 0 on 32274
>> > org.apache.hadoop.mapred.TaskTracker -
>> attempt_200901251629_0011_m_000005_0
>> > Ignoring status-update since task is 'done'
>> > 2009-01-25 17:13:24.996 INFO taskCleanup
>> > org.apache.hadoop.mapred.TaskTracker - Received 'KillJobAction' for job:
>> > job_200901251629_0011
>> > 2009-01-25 17:13:24.996 INFO taskCleanup
>> > org.apache.hadoop.mapred.TaskRunner
>> > - attempt_200901251629_0011_m_000005_0 done; removing files.
>> > 2009-01-25 17:47:22.668 WARN Thread-23
>> org.apache.hadoop.mapred.TaskRunner
>> > -
>> > attempt_200901251629_0001_m_000006_0 Child Error
>> > java.io.IOException: Task process exit with nonzero status of 143.
>> >        at
>> org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:462)
>> >        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:403)
>> > 2009-01-25 17:47:22.669 WARN Thread-23
>> org.apache.hadoop.mapred.TaskTracker
>> > - Error from unknown child task: attempt_200901251629_0001_m_000006_0.
>> > Ignored.
>> > 2009-01-25 17:47:22.671 WARN Thread-23
>> org.apache.hadoop.mapred.TaskTracker
>> > - Unknown child task finshed: attempt_200901251629_0001_m_000006_0.
>> > Ignored.
>> > 2009-01-25 17:47:22.713 WARN Thread-79
>> org.apache.hadoop.mapred.TaskRunner
>> > -
>> > attempt_200901251629_0002_m_000007_0 Child Error
>> > java.io.IOException: Task process exit with nonzero status of 143.
>> >        at
>> org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:462)
>> >        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:403)
>> > 2009-01-25 17:47:22.713 WARN Thread-159
>> org.apache.hadoop.mapred.TaskRunner
>> > - attempt_200901251629_0011_m_000005_0 Child Error
>> > java.io.IOException: Task process exit with nonzero status of 143.
>> >        at
>> org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:462)
>> >        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:403)
>> > 2009-01-25 17:47:22.713 WARN Thread-79
>> org.apache.hadoop.mapred.TaskTracker
>> > - Error from unknown child task: attempt_200901251629_0002_m_000007_0.
>> > Ignored.
>> > 2009-01-25 17:47:22.714 WARN Thread-159
>> > org.apache.hadoop.mapred.TaskTracker
>> > - Error from unknown child task: attempt_200901251629_0011_m_000005_0.
>> > Ignored.
>> > 2009-01-25 17:47:22.714 WARN Thread-79
>> org.apache.hadoop.mapred.TaskTracker
>> > - Unknown child task finshed: attempt_200901251629_0002_m_000007_0.
>> > Ignored.
>> > 2009-01-25 17:47:22.715 WARN Thread-159
>> > org.apache.hadoop.mapred.TaskTracker
>> > - Unknown child task finshed: attempt_200901251629_0011_m_000005_0.
>> > Ignored.
>> > 2009-01-25 17:47:22.811 WARN Thread-119
>> org.apache.hadoop.mapred.TaskRunner
>> > - attempt_200901251629_0006_m_000007_0 Child Error
>> > java.io.IOException: Task process exit with nonzero status of 143.
>> >        at
>> org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:462)
>> >        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:403)
>> > 2009-01-25 17:47:22.811 WARN Thread-119
>> > org.apache.hadoop.mapred.TaskTracker
>> > - Error from unknown child task: attempt_200901251629_0006_m_000007_0.
>> > Ignored.
>> > 2009-01-25 17:47:22.812 WARN Thread-119
>> > org.apache.hadoop.mapred.TaskTracker
>> > - Unknown child task finshed: attempt_200901251629_0006_m_000007_0.
>> > Ignored.
>> > 2009-01-25 17:47:22.960 WARN Thread-89
>> org.apache.hadoop.mapred.TaskRunner
>> > -
>> > attempt_200901251629_0003_m_000007_0 Child Error
>> > java.io.IOException: Task process exit with nonzero status of 143.
>> >        at
>> org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:462)
>> >        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:403)
>> > 2009-01-25 17:47:22.961 WARN Thread-89
>> org.apache.hadoop.mapred.TaskTracker
>> > - Error from unknown child task: attempt_200901251629_0003_m_000007_0.
>> > Ignored.
>> > 2009-01-25 17:47:22.962 WARN Thread-89
>> org.apache.hadoop.mapred.TaskTracker
>> > - Unknown child task finshed: attempt_200901251629_0003_m_000007_0.
>> > Ignored.
>> > 2009-01-25 17:47:23.060 WARN Thread-129
>> org.apache.hadoop.mapred.TaskRunner
>> > - attempt_200901251629_0008_m_000003_0 Child Error
>> > java.io.IOException: Task process exit with nonzero status of 143.
>> >        at
>> org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:462)
>> >        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:403)
>> > 2009-01-25 17:47:23.061 WARN Thread-129
>> > org.apache.hadoop.mapred.TaskTracker
>> > - Error from unknown child task: attempt_200901251629_0008_m_000003_0.
>> > Ignored.
>> > 2009-01-25 17:47:23.062 WARN Thread-129
>> > org.apache.hadoop.mapred.TaskTracker
>> > - Unknown child task finshed: attempt_200901251629_0008_m_000003_0.
>> > Ignored.
>> > 2009-01-25 17:47:23.279 WARN Thread-99
>> org.apache.hadoop.mapred.TaskRunner
>> > -
>> > attempt_200901251629_0004_m_000007_0 Child Error
>> > java.io.IOException: Task process exit with nonzero status of 143.
>> >        at
>> org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:462)
>> >        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:403)
>> > 2009-01-25 17:47:23.279 WARN Thread-99
>> org.apache.hadoop.mapred.TaskTracker
>> > - Error from unknown child task: attempt_200901251629_0004_m_000007_0.
>> > Ignored.
>> > 2009-01-25 17:47:23.280 WARN Thread-99
>> org.apache.hadoop.mapred.TaskTracker
>> > - Unknown child task finshed: attempt_200901251629_0004_m_000007_0.
>> > Ignored.
>> > 2009-01-25 17:47:23.360 WARN Thread-139
>> org.apache.hadoop.mapred.TaskRunner
>> > - attempt_200901251629_0009_m_000004_0 Child Error
>> > java.io.IOException: Task process exit with nonzero status of 143.
>> >        at
>> org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:462)
>> >        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:403)
>> > 2009-01-25 17:47:23.360 WARN Thread-139
>> > org.apache.hadoop.mapred.TaskTracker
>> > - Error from unknown child task: attempt_200901251629_0009_m_000004_0.
>> > Ignored.
>> > 2009-01-25 17:47:23.361 WARN Thread-139
>> > org.apache.hadoop.mapred.TaskTracker
>> > - Unknown child task finshed: attempt_200901251629_0009_m_000004_0.
>> > Ignored.
>> > 2009-01-25 17:47:23.445 WARN Thread-149
>> org.apache.hadoop.mapred.TaskRunner
>> > - attempt_200901251629_0010_m_000004_0 Child Error
>> > java.io.IOException: Task process exit with nonzero status of 143.
>> >        at
>> org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:462)
>> >        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:403)
>> > 2009-01-25 17:47:23.445 WARN Thread-149
>> > org.apache.hadoop.mapred.TaskTracker
>> > - Error from unknown child task: attempt_200901251629_0010_m_000004_0.
>> > Ignored.
>> > 2009-01-25 17:47:23.446 WARN Thread-149
>> > org.apache.hadoop.mapred.TaskTracker
>> > - Unknown child task finshed: attempt_200901251629_0010_m_000004_0.
>> > Ignored.
>> > 2009-01-25 17:47:23.463 WARN Thread-109
>> org.apache.hadoop.mapred.TaskRunner
>> > - attempt_200901251629_0005_m_000007_0 Child Error
>> > java.io.IOException: Task process exit with nonzero status of 143.
>> >        at
>> org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:462)
>> >        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:403)
>> > 2009-01-25 17:47:23.464 WARN Thread-109
>> > org.apache.hadoop.mapred.TaskTracker
>> > - Error from unknown child task: attempt_200901251629_0005_m_000007_0.
>> > Ignored.
>> > 2009-01-25 17:47:23.465 WARN Thread-109
>> > org.apache.hadoop.mapred.TaskTracker
>> > - Unknown child task finshed: attempt_200901251629_0005_m_000007_0.
>> > Ignored.
>> > 2009-01-25 17:48:15.315 INFO main org.apache.hadoop.mapred.TaskTracker -
>> > LaunchTaskAction: attempt_200901251629_0012_m_000004_0
>> > 2009-01-25 17:48:28.088 INFO main org.apache.hadoop.mapred.MapTask -
>> > numReduceTasks: 0
>> > 2009-01-25 17:48:34.097 INFO IPC Server handler 0 on 32274
>> > org.apache.hadoop.mapred.TaskTracker -
>> attempt_200901251629_0012_m_000004_0
>> > 0.07784176% 347 docs, 0 errors, 69.4 docs/s
>> > 2009-01-25 17:48:37.099 INFO IPC Server handler 1 on 32274
>> > org.apache.hadoop.mapred.TaskTracker -
>> attempt_200901251629_0012_m_000004_0
>> > 0.13955688% 673 docs, 0 errors, 84.1 docs/s
>> > 2009-01-25 17:48:40.100 INFO IPC Server handler 0 on 32274
>> > org.apache.hadoop.mapred.TaskTracker -
>> attempt_200901251629_0012_m_000004_0
>> > 0.20145722% 1003 docs, 0 errors, 91.2 docs/s
>> > 2009-01-25 17:48:43.102 INFO IPC Server handler 1 on 32274
>> > org.apache.hadoop.mapred.TaskTracker -
>> attempt_200901251629_0012_m_000004_0
>> > 0.26416454% 1343 docs, 0 errors, 95.9 docs/s
>> > 2009-01-25 17:48:46.104 INFO IPC Server handler 0 on 32274
>> > org.apache.hadoop.mapred.TaskTracker -
>> attempt_200901251629_0012_m_000004_0
>> > 0.32754657% 1673 docs, 0 errors, 98.4 docs/s
>> > 2009-01-25 17:48:49.105 INFO IPC Server handler 1 on 32274
>> > org.apache.hadoop.mapred.TaskTracker -
>> attempt_200901251629_0012_m_000004_0
>> > 0.39229122% 2025 docs, 0 errors, 101.3 docs/s
>> > 2009-01-25 17:48:52.106 INFO IPC Server handler 0 on 32274
>> > org.apache.hadoop.mapred.TaskTracker -
>> attempt_200901251629_0012_m_000004_0
>> > 0.44708738% 2315 docs, 0 errors, 100.7 docs/s
>> > 2009-01-25 17:48:55.108 INFO IPC Server handler 1 on 32274
>> > org.apache.hadoop.mapred.TaskTracker -
>> attempt_200901251629_0012_m_000004_0
>> > 0.5117659% 2658 docs, 0 errors, 102.2 docs/s
>> > 2009-01-25 17:48:58.109 INFO IPC Server handler 0 on 32274
>> > org.apache.hadoop.mapred.TaskTracker -
>> attempt_200901251629_0012_m_000004_0
>> > 0.5795401% 3019 docs, 0 errors, 104.1 docs/s
>> > 2009-01-25 17:49:01.111 INFO IPC Server handler 1 on 32274
>> > org.apache.hadoop.mapred.TaskTracker -
>> attempt_200901251629_0012_m_000004_0
>> > 0.6468777% 3377 docs, 0 errors, 105.5 docs/s
>> > 2009-01-25 17:49:04.112 INFO IPC Server handler 0 on 32274
>> > org.apache.hadoop.mapred.TaskTracker -
>> attempt_200901251629_0012_m_000004_0
>> > 0.71514136% 3741 docs, 0 errors, 106.9 docs/s
>> > 2009-01-25 17:49:07.114 INFO IPC Server handler 1 on 32274
>> > org.apache.hadoop.mapred.TaskTracker -
>> attempt_200901251629_0012_m_000004_0
>> > 0.78037554% 4094 docs, 0 errors, 107.7 docs/s
>> > 2009-01-25 17:49:10.115 INFO IPC Server handler 0 on 32274
>> > org.apache.hadoop.mapred.TaskTracker -
>> attempt_200901251629_0012_m_000004_0
>> > 0.84660184% 4448 docs, 0 errors, 108.5 docs/s
>> > 2009-01-25 17:49:13.117 INFO IPC Server handler 1 on 32274
>> > org.apache.hadoop.mapred.TaskTracker -
>> attempt_200901251629_0012_m_000004_0
>> > 0.9146803% 4810 docs, 0 errors, 109.3 docs/s
>> > 2009-01-25 17:49:16.118 INFO IPC Server handler 0 on 32274
>> > org.apache.hadoop.mapred.TaskTracker -
>> attempt_200901251629_0012_m_000004_0
>> > 0.9521195% 5044 docs, 0 errors, 107.3 docs/s
>> > 2009-01-25 17:49:18.626 INFO IPC Server handler 1 on 32274
>> > org.apache.hadoop.mapred.TaskTracker -
>> attempt_200901251629_0012_m_000004_0
>> > 1.0% 5364 docs, 0 errors, 107.3 docs/s
>> > 2009-01-25 17:49:18.631 INFO IPC Server handler 0 on 32274
>> > org.apache.hadoop.mapred.TaskTracker - Task
>> > attempt_200901251629_0012_m_000004_0 is done.
>> > 2009-01-25 17:49:18.631 INFO main org.apache.hadoop.mapred.TaskRunner -
>> > Task
>> > 'attempt_200901251629_0012_m_000004_0' done.
>> > 2009-01-25 17:49:19.120 INFO IPC Server handler 1 on 32274
>> > org.apache.hadoop.mapred.TaskTracker -
>> attempt_200901251629_0012_m_000004_0
>> > 1.0% 5364 docs, 0 errors, 107.3 docs/s
>> > 2009-01-25 17:49:19.120 INFO IPC Server handler 1 on 32274
>> > org.apache.hadoop.mapred.TaskTracker -
>> attempt_200901251629_0012_m_000004_0
>> > Ignoring status-update since task is 'done'
>> > 2009-01-25 17:49:35.582 INFO taskCleanup
>> > org.apache.hadoop.mapred.TaskTracker - Received 'KillJobAction' for job:
>> > job_200901251629_0012
>> > 2009-01-25 17:49:35.582 INFO taskCleanup
>> > org.apache.hadoop.mapred.TaskRunner
>> > - attempt_200901251629_0012_m_000004_0 done; removing files.
>> >
>> > # Still processes left even though the TaskTracker said: Received
>> > 'KillJobAction' for job: job_200901251629_0012
>> > root@mapreduce2:~# jps
>> > 10732 Jps
>> > 10634 TaskTracker$Child
>> > 8660 DataNode
>> > 8824 TaskTracker
>> > 8730 SecondaryNameNode
>> > 25060 Bootstrap
>> > root@mapreduce2:~# date
>> > Sun Jan 25 17:51:48 CET 2009
>> > root@mapreduce2:~#
>> >
>> > On Sun, Jan 25, 2009 at 5:42 PM, Marcus Herou <
>> marcus.herou@tailsweep.com
>> > >wrote:
>> >
>> > > Hi.
>> > >
>> > > Today I noticed when I ran a Solr Indexing job through our Hadoop
>> cluster
>> > > that the master MySQL database where screaming about "Too Many
>> > Connections".
>> > >
>> > > I wondered how that could happen so I logged into my Hadoop machines
>> and
>> > > searched through the logs. Nothing strange there. Then I just did a
>> jps:
>> > >
>> > > root@mapreduce1:~# jps
>> > > 10701 TaskTracker$Child
>> > > 9567 NameNode
>> > > 5435 TaskTracker$Child
>> > > 31801 Bootstrap
>> > > 7349 TaskTracker$Child
>> > > 6197 TaskTracker$Child
>> > > 7761 TaskTracker$Child
>> > > 10453 TaskTracker$Child
>> > > 11232 TaskTracker$Child
>> > > 11113 TaskTracker$Child
>> > > 9688 DataNode
>> > > 10877 TaskTracker$Child
>> > > 6504 TaskTracker$Child
>> > > 10236 TaskTracker$Child
>> > > 9852 TaskTracker
>> > > 6515 TaskTracker$Child
>> > > 11396 TaskTracker$Child
>> > > 11741 Jps
>> > > 6191 TaskTracker$Child
>> > > 10981 TaskTracker$Child
>> > > 7742 TaskTracker$Child
>> > > 5946 TaskTracker$Child
>> > > 11315 TaskTracker$Child
>> > > 8112 TaskTracker$Child
>> > > 11580 TaskTracker$Child
>> > > 11490 TaskTracker$Child
>> > > 5687 TaskTracker$Child
>> > > 5927 TaskTracker$Child
>> > > 27144 WrapperSimpleApp
>> > > 7368 TaskTracker$Child
>> > >
>> > > Damn! Each Child have it's own DataSource (dbcp pool) tweaked down so
>> it
>> > > only can have one active connection to any shard at any time.
>> > > Background: I ran out of connections during the Christmas holidays
>> since
>> > I
>> > > have 60 shards (10 per MySQL machine) and each required a DB-Pool
>> which
>> > > allowed too many active+idle connections.
>> > >
>> > > Anyway I have no active jobs at the moment so the children should have
>> > died
>> > > by themselves.
>> > > Fortunately I have a little nice script which kills the bastards: jps
>> > > |egrep "TaskTracker.+" | awk '{print $1}'|xargs kill
>> > > I will probably put that in a cronjob which kills long running
>> > children...
>> > >
>> > > Anyway, how can this happen ? Am I doing something really stupid along
>> > the
>> > > way ?
>> > > Hard facts:
>> > > Ubuntu Hardy-Heron, 2.6.24-19-server
>> > > java version "1.6.0_06"
>> > > Hadoop-0.18.2
>> > > It's my own classes which fires the jobs through JobClient
>> > > (JobClient.runJob(job))
>> > > I feed the jar to hadoop by issuing: job.setJar(jarFile); (comes from
>> a
>> > > bash script)
>> > > I feed deps into hadoop by issuing: job.set("tmpjars", jarFiles);
>> (comes
>> > by
>> > > parsing external CLASSPATH ENV in bash)
>> > >
>> > > The client do not complain, se example output below (I write no data
>> to
>> > > HDFS ((HDFS bytes written=774)), since I mostly use it for crawling
>> and
>> > all
>> > > crawlers/indexers access my sharding db structure directly without
>> > > intermediate storage):
>> > > 2009-01-25 17:12:11.175 INFO main
>> > org.apache.hadoop.mapred.FileInputFormat
>> > > - Total input paths to process : 1
>> > > 2009-01-25 17:12:11.176 INFO main
>> > org.apache.hadoop.mapred.FileInputFormat
>> > > - Total input paths to process : 1
>> > > 2009-01-25 17:12:11.437 INFO main org.apache.hadoop.mapred.JobClient -
>> > > Running job: job_200901251629_0011
>> > > 2009-01-25 17:12:12.439 INFO main org.apache.hadoop.mapred.JobClient -
>> >  map
>> > > 0% reduce 0%
>> > > 2009-01-25 17:12:35.481 INFO main org.apache.hadoop.mapred.JobClient -
>> >  map
>> > > 6% reduce 0%
>> > > 2009-01-25 17:12:40.493 INFO main org.apache.hadoop.mapred.JobClient -
>> >  map
>> > > 21% reduce 0%
>> > > 2009-01-25 17:12:45.502 INFO main org.apache.hadoop.mapred.JobClient -
>> >  map
>> > > 31% reduce 0%
>> > > 2009-01-25 17:12:50.511 INFO main org.apache.hadoop.mapred.JobClient -
>> >  map
>> > > 51% reduce 0%
>> > > 2009-01-25 17:12:55.520 INFO main org.apache.hadoop.mapred.JobClient -
>> >  map
>> > > 67% reduce 0%
>> > > 2009-01-25 17:13:00.533 INFO main org.apache.hadoop.mapred.JobClient -
>> >  map
>> > > 72% reduce 0%
>> > > 2009-01-25 17:13:05.543 INFO main org.apache.hadoop.mapred.JobClient -
>> >  map
>> > > 84% reduce 0%
>> > > 2009-01-25 17:13:10.552 INFO main org.apache.hadoop.mapred.JobClient -
>> >  map
>> > > 95% reduce 0%
>> > > 2009-01-25 17:13:15.560 INFO main org.apache.hadoop.mapred.JobClient -
>> >  map
>> > > 98% reduce 0%
>> > > 2009-01-25 17:13:20.568 INFO main org.apache.hadoop.mapred.JobClient -
>> > Job
>> > > complete: job_200901251629_0011
>> > > 2009-01-25 17:13:20.570 INFO main org.apache.hadoop.mapred.JobClient -
>> > > Counters: 7
>> > > 2009-01-25 17:13:20.570 INFO main org.apache.hadoop.mapred.JobClient -
>> > > File Systems
>> > > 2009-01-25 17:13:20.570 INFO main org.apache.hadoop.mapred.JobClient -
>> > > HDFS bytes read=2741143
>> > > 2009-01-25 17:13:20.570 INFO main org.apache.hadoop.mapred.JobClient -
>> > > HDFS bytes written=774
>> > > 2009-01-25 17:13:20.570 INFO main org.apache.hadoop.mapred.JobClient -
>> > > Job Counters
>> > > 2009-01-25 17:13:20.570 INFO main org.apache.hadoop.mapred.JobClient -
>> > > Rack-local map tasks=9
>> > > 2009-01-25 17:13:20.571 INFO main org.apache.hadoop.mapred.JobClient -
>> > > Launched map tasks=9
>> > > 2009-01-25 17:13:20.571 INFO main org.apache.hadoop.mapred.JobClient -
>> > > Map-Reduce Framework
>> > > 2009-01-25 17:13:20.571 INFO main org.apache.hadoop.mapred.JobClient -
>> > > Map input records=48314
>> > > 2009-01-25 17:13:20.571 INFO main org.apache.hadoop.mapred.JobClient -
>> > > Map input bytes=2732424
>> > > 2009-01-25 17:13:20.571 INFO main org.apache.hadoop.mapred.JobClient -
>> > > Map output records=0
>> > >
>> > > Any suggestions or pointers would be greatly appreciated. Hmm Coming
>> to
>> > > think about something. I start X threads from inside Hadoop almost
>> cut'n
>> > > pasted from Nutch.
>> > > If a thread somehow would linger, would Hadoop not be able to shutdown
>> > even
>> > > though there is nothing more to read from the RecordReader ?
>> > >
>> > > Kindly
>> > >
>> > > //Marcus
>> > >
>> > > --
>> > > Marcus Herou CTO and co-founder Tailsweep AB
>> > > +46702561312
>> > > marcus.herou@tailsweep.com
>> > > http://www.tailsweep.com/
>> > > http://blogg.tailsweep.com/
>> > >
>> >
>> >
>> >
>> > --
>> > Marcus Herou CTO and co-founder Tailsweep AB
>> > +46702561312
>> > marcus.herou@tailsweep.com
>> > http://www.tailsweep.com/
>> > http://blogg.tailsweep.com/
>> >
>>
>
>
>
> --
> Marcus Herou CTO and co-founder Tailsweep AB
> +46702561312
> marcus.herou@tailsweep.com
> http://www.tailsweep.com/
> http://blogg.tailsweep.com/
>



-- 
Marcus Herou CTO and co-founder Tailsweep AB
+46702561312
marcus.herou@tailsweep.com
http://www.tailsweep.com/
http://blogg.tailsweep.com/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message