hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Many child processes dont exit
Date Sat, 20 Feb 2010 14:49:54 GMT
Do you have System.exit() as the last line in your main() ?
    Job job = createSubmittableJob(conf, otherArgs);
    System.exit(job.waitForCompletion(true)? 0 : 1);


On Sat, Feb 20, 2010 at 12:32 AM, Zheng Lv <lvzheng19800619@gmail.com>wrote:

> Hello Ted,
>  Yes. Every hour a job will be created and started, and when it finished,
> it will maintain. The logs looks like normal, do you know what can lead to
> this happen?Thank you.
>    LvZheng
> 2010/2/20 Ted Yu <yuzhihong@gmail.com>
>
> > Did the number of child processes increase over time ?
> >
> > On Friday, February 19, 2010, Zheng Lv <lvzheng19800619@gmail.com>
> wrote:
> > > Hello Edson,
> > >   Thank you for your reply. I don't want to kill them, I want to know
> why
> > > these child processes don't exit, and to know how to make them exit
> > > successfully when they finish. Any ideas? Thank you.
> > >     LvZheng
> > >
> > > 2010/2/18 Edson Ramiro <erlfilho@gmail.com>
> > >
> > >> Do you want to kill them ?
> > >>
> > >> if yes, you can use
> > >>
> > >>  ./bin/slaves.sh pkill java
> > >>
> > >> but it will kill the datanode and tasktracker processes
> > >> in all slaves and you'll need to start these processes again.
> > >>
> > >> Edson Ramiro
> > >>
> > >>
> > >> On 14 February 2010 22:09, Zheng Lv <lvzheng19800619@gmail.com>
> wrote:
> > >>
> > >> > any idea?
> > >> >
> > >> > 2010/2/11 Zheng Lv <lvzheng19800619@gmail.com>
> > >> >
> > >> > > Hello Everyone,
> > >> > >   We often find many child processes in datanodes, which have
> > already
> > >> > > finished for long time. And following are the jstack log:
> > >> > >   Full thread dump Java HotSpot(TM) 64-Bit Server VM (14.3-b01
> mixed
> > >> > mode):
> > >> > > "DestroyJavaVM" prio=10 tid=0x00002aaac8019800 nid=0x2422 waiting
> on
> > >> > > condition [0x0000000000000000]
> > >> > >    java.lang.Thread.State: RUNNABLE
> > >> > > "NioProcessor-31" prio=10 tid=0x00000000439fa000 nid=0x2826
> runnable
> > >> > > [0x000000004100a000]
> > >> > >    java.lang.Thread.State: RUNNABLE
> > >> > >         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> > >> > >         at
> > >> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
> > >> > >         at
> > >> > sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
> > >> > >         at
> > >> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
> > >> > >         - locked <0x00002aaab9b5f6f8> (a sun.nio.ch.Util$1)
> > >> > >         - locked <0x00002aaab9b5f710> (a
> > >> > > java.util.Collections$UnmodifiableSet)
> > >> > >         - locked <0x00002aaab9b5f680> (a
> > sun.nio.ch.EPollSelectorImpl)
> > >> > >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
> > >> > >         at
> > >> > >
> > >> >
> > >>
> >
> org.apache.mina.transport.socket.nio.NioProcessor.select(NioProcessor.java:65)
> > >> > >         at
> > >> > >
> > >> >
> > >>
> >
> org.apache.mina.common.AbstractPollingIoProcessor$Worker.run(AbstractPollingIoProcessor.java:672)
> > >> > >         at
> > >> > >
> > >> >
> > >>
> >
> org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51)
> > >> > >         at
> > >> > >
> > >> >
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >> > >         at
> > >> > >
> > >> >
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > >> > >         at java.lang.Thread.run(Thread.java:619)
> > >> > > "pool-15-thread-1" prio=10 tid=0x00002aaac802d000 nid=0x2825
> waiting
> > on
> > >> > > condition [0x0000000041604000]
> > >> > >    java.lang.Thread.State: WAITING (parking)
> > >> > >         at sun.misc.Unsafe.park(Native Method)
> > >> > >         - parking to wait for  <0x00002aaab9b61620> (a
> > >> > >
> > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> > >> > >         at
> > >> > > java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> > >> > >         at
> > >> > >
> > >> >
> > >>
> >
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> > >> > >         at
> > >> > >
> > >> >
> > >>
> >
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
> > >> > >         at
> > >> > >
> > >> >
> > >>
> >
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
> > >> > >         at
> > >> > >
> > >> >
> > >> java.util.concurrent.ThreadPoolExecutor$Worker.run(Threa
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message