mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 王瑜 <wan...@nfs.iscas.ac.cn>
Subject Re: Re: org.apache.hadoop.mapred.MesosScheduler: Unknown/exited TaskTracker: http://slave5:50060
Date Tue, 23 Apr 2013 06:24:37 GMT
Yes! It works! Thanks very much, guodong...




Wang Yu

From: 王国栋
Date: 2013-04-23 13:32
To: wangyu
CC: mesos-dev
Subject: Re: Re: org.apache.hadoop.mapred.MesosScheduler: Unknown/exited TaskTracker: http://slave5:50060
Hmm. it seems that the mapred.mesos.master is set correctly.

if you run hadoop in local mode, use the following setting is ok
  <property>
    <name>mapred.mesos.master</name>
    <value>local</value>
  </property>

if you want to start the cluster. set mapred.mesos.master as the
mesos-master-hostname:mesos-master-port.

Make sure the dns parser result for mesos-master-hostname is the right ip.

BTW: when you starting the jobtracker, you can check mesos webUI and check
if there is hadoop framework registered.

Thanks.

Guodong


On Tue, Apr 23, 2013 at 1:24 PM, 王瑜 <wangyu@nfs.iscas.ac.cn> wrote:

> **
> Hi, Guodong,
>
> I start hadoop as you said, then I saw this error:
> 13/04/23 13:03:43 ERROR mapred.MesosScheduler: Error from scheduler driver: Cannot parse
> '@0.0.0.0:0'
>
> What's this mean? where should I change MesosScheduler code to fix this?
> Thanks very much! I am so sorry for interrupt you once again...
>
> The whole log is as follows:
>
>  [root@master hadoop-0.20.205.0]# hadoop jobtracker
> 13/04/23 13:21:04 INFO mapred.JobTracker: STARTUP_MSG:
> /************************************************************
> STARTUP_MSG: Starting JobTracker
> STARTUP_MSG:   host = master/192.168.0.2
> STARTUP_MSG:   args = []
> STARTUP_MSG:   version = 0.20.205.0
>
> STARTUP_MSG:   build =  -r ; compiled by 'root' on Sat Apr 13 11:19:33 CST 2013
> ************************************************************/
>
> 13/04/23 13:21:04 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
>
> 13/04/23 13:21:04 INFO impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats
registered.
>
> 13/04/23 13:21:04 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
>
> 13/04/23 13:21:04 INFO impl.MetricsSystemImpl: JobTracker metrics system started
>
> 13/04/23 13:21:04 INFO impl.MetricsSourceAdapter: MBean for source QueueMetrics,q=default
registered.
>
> 13/04/23 13:21:04 INFO impl.MetricsSourceAdapter: MBean for source ugi registered.
>
> 13/04/23 13:21:04 INFO delegation.AbstractDelegationTokenSecretManager: Updating the
current master key for generating delegation tokens
>
> 13/04/23 13:21:04 INFO delegation.AbstractDelegationTokenSecretManager: Starting expired
delegation token remover thread, tokenRemoverScanInterval=60 min(s)
>
> 13/04/23 13:21:04 INFO mapred.JobTracker: Scheduler configured with (memSizeForMapSlotOnJT,
memSizeForReduceSlotOnJT, limitMaxMemForMapTasks, limitMaxMemForReduceTasks) (-1, -1, -1,
-1)
>
> 13/04/23 13:21:04 INFO delegation.AbstractDelegationTokenSecretManager: Updating the
current master key for generating delegation tokens
>
> 13/04/23 13:21:04 INFO util.HostsFileReader: Refreshing hosts (include/exclude) list
>
> 13/04/23 13:21:04 INFO mapred.JobTracker: Starting jobtracker with owner as root
> 13/04/23 13:21:04 INFO ipc.Server: Starting SocketReader
>
> 13/04/23 13:21:04 INFO impl.MetricsSourceAdapter: MBean for source RpcDetailedActivityForPort9001
registered.
>
> 13/04/23 13:21:04 INFO impl.MetricsSourceAdapter: MBean for source RpcActivityForPort9001
registered.
>
> 13/04/23 13:21:04 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log)
via org.mortbay.log.Slf4jLog
>
> 13/04/23 13:21:05 INFO http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
>
> 13/04/23 13:21:05 INFO http.HttpServer: Port returned by webServer.getConnectors()[0].getLocalPort()
before open() is -1. Opening the listener on 50030
>
> 13/04/23 13:21:05 INFO http.HttpServer: listener.getLocalPort() returned 50030 webServer.getConnectors()[0].getLocalPort()
returned 50030
> 13/04/23 13:21:05 INFO http.HttpServer: Jetty bound to port 50030
> 13/04/23 13:21:05 INFO mortbay.log: jetty-6.1.26
> 13/04/23 13:21:05 INFO mortbay.log: Started
> SelectChannelConnector@0.0.0.0:50030
>
> 13/04/23 13:21:05 INFO impl.MetricsSourceAdapter: MBean for source jvm registered.
>
> 13/04/23 13:21:05 INFO impl.MetricsSourceAdapter: MBean for source JobTrackerMetrics
registered.
> 13/04/23 13:21:05 INFO mapred.JobTracker: JobTracker up at: 9001
> 13/04/23 13:21:05 INFO mapred.JobTracker: JobTracker webserver: 50030
> 13/04/23 13:21:05 INFO mapred.JobTracker: Cleaning up the system directory
>
> 13/04/23 13:21:05 INFO mapred.JobTracker: History server being initialized in embedded
mode
>
> 13/04/23 13:21:05 INFO mapred.JobHistoryServer: Started job history server at: localhost:50030
>
> 13/04/23 13:21:05 INFO mapred.JobTracker: Job History Server web address: localhost:50030
>
> 13/04/23 13:21:05 INFO mapred.CompletedJobStatusStore: Completed job store is inactive
> 13/04/23 13:21:05 INFO mapred.MesosScheduler: Starting MesosScheduler
> 13/04/23 13:21:05 INFO mapred.JobTracker: Refreshing hosts information
>
> 13/04/23 13:21:05 ERROR mapred.MesosScheduler: Error from scheduler driver: Cannot parse
'@
> 0.0.0.0:0'
> 13/04/23 13:21:05 INFO util.HostsFileReader: Setting the includes file to
> 13/04/23 13:21:05 INFO util.HostsFileReader: Setting the excludes file to
>
> 13/04/23 13:21:05 INFO util.HostsFileReader: Refreshing hosts (include/exclude) list
> 13/04/23 13:21:05 INFO mapred.JobTracker: Decommissioning 0 nodes
> 13/04/23 13:21:05 INFO ipc.Server: IPC Server Responder: starting
> 13/04/23 13:21:05 INFO ipc.Server: IPC Server listener on 9001: starting
> 13/04/23 13:21:05 INFO ipc.Server: IPC Server handler 0 on 9001: starting
> 13/04/23 13:21:05 INFO ipc.Server: IPC Server handler 1 on 9001: starting
> 13/04/23 13:21:05 INFO ipc.Server: IPC Server handler 3 on 9001: starting
> 13/04/23 13:21:05 INFO ipc.Server: IPC Server handler 2 on 9001: starting
> 13/04/23 13:21:05 INFO ipc.Server: IPC Server handler 5 on 9001: starting
> 13/04/23 13:21:05 INFO ipc.Server: IPC Server handler 4 on 9001: starting
> 13/04/23 13:21:05 INFO ipc.Server: IPC Server handler 6 on 9001: starting
> 13/04/23 13:21:05 INFO ipc.Server: IPC Server handler 7 on 9001: starting
> 13/04/23 13:21:05 INFO mapred.JobTracker: Starting RUNNING
> 13/04/23 13:21:05 INFO ipc.Server: IPC Server handler 8 on 9001: starting
> 13/04/23 13:21:05 INFO ipc.Server: IPC Server handler 9 on 9001: starting
>
> 13/04/23 13:21:32 WARN util.NativeCodeLoader: Unable to load native-hadoop library for
your platform... using builtin-java classes where applicable
>
> 13/04/23 13:21:32 INFO mapred.JobInProgress: job_201304231321_0001: nMaps=0 nReduces=0
max=-1
>
> 13/04/23 13:21:32 INFO mapred.MesosScheduler: Added job job_201304231321_0001
>
> 13/04/23 13:21:32 INFO mapred.JobTracker: Job job_201304231321_0001 added successfully
for user 'root' to queue 'default'
>
> 13/04/23 13:21:32 INFO mapred.AuditLogger: USER=root    IP=192.168.0.2  OPERATION=SUBMIT_JOB
   TARGET=job_201304231321_0001    RESULT=SUCCESS
>
> 13/04/23 13:21:32 INFO mapred.JobTracker: Initializing job_201304231321_0001
>
> 13/04/23 13:21:32 INFO mapred.JobInProgress: Initializing job_201304231321_0001
>
> 13/04/23 13:21:32 INFO mapred.JobInProgress: jobToken generated and stored with users
keys in /home/HadoopRun/tmp/mapred/system/job_201304231321_0001/jobToken
>
> 13/04/23 13:21:32 INFO mapred.JobInProgress: Input size for job job_201304231321_0001
= 0. Number of splits = 0
>
> 13/04/23 13:21:32 INFO mapred.JobInProgress: Job job_201304231321_0001 initialized successfully
with 0 map tasks and 0 reduce tasks.
>
> ------------------------------
> Wang Yu
>
>  *From:* 王国栋 <wanggd04@gmail.com>
> *Date:* 2013-04-23 11:34
> *To:* mesos-dev <mesos-dev@incubator.apache.org>; wangyu<wangyu@nfs.iscas.ac.cn>
> *Subject:* Re: Re: org.apache.hadoop.mapred.MesosScheduler:
> Unknown/exited TaskTracker: http://slave5:50060
>  Hi Yu,
>
> Mesos will just launch tasktracker on each slave node as long as the
> required resource is enough for the tasktracker. So you have to run
> NameNode, Jobtracker and DataNode by your own.
>
> Basicly, starting the hadoop on mesos is like this.
> 1. start the dfs. use hadoop/bin/start-dfs.sh. (you should configure
> core-sites.xml and hdfs-site.xml). dfs is no different from the normal one.
> 2. start jobtracker, use hadoop/bin/hadoop jobtracker (you should
> configure mapred-site.xml, this jobtracker should contains the patch for
> mesos)
>
> Then, you can use mesos web UI and jobtracker web UI to check the status
> of Jobtracker.
>
>  Guodong
>
>
> On Tue, Apr 23, 2013 at 11:06 AM, 王瑜 <wangyu@nfs.iscas.ac.cn> wrote:
>
>> Oh, yes, I start my hadoop using "start-all.sh". I know what's my
>> problem. Thanks very much!
>>
>> ps: Besides TaskTracker, is there any other roles(like JobTracker,
>> DataNode) I should stop it first?
>>
>>
>>
>>
>> Wang Yu
>>
>> 发件人: Benjamin Mahler
>> 发送时间: 2013-04-23 10:56
>> 收件人: mesos-dev@incubator.apache.org; wangyu
>> 主题: Re: Re: org.apache.hadoop.mapred.MesosScheduler: Unknown/exited
>> TaskTracker: http://slave5:50060
>>  The scheduler we wrote for Hadoop will start its own TaskTrackers,
>> meaning
>> you do not have to start any TaskTrackers yourself
>>
>> Are you starting your own TaskTrackers? Are there any TaskTrackers running
>> in your cluster?
>>
>> Looking at your jps output, is there already a TaskTracker running?
>> [root@master logs]# jps
>> 13896 RunJar
>> 14123 Jps
>> 12718 NameNode
>> 12900 DataNode
>> 13374 TaskTracker  <--- How was this started?
>> 13218 JobTracker
>>
>>
>> On Mon, Apr 22, 2013 at 7:47 PM, 王瑜 <wangyu@nfs.iscas.ac.cn> wrote:
>>
>> > Hi, Ben and Guodong,
>> >
>> > What do you mean "managing your own TaskTrackers"? How should I know
>> > whether I have manager my own TaskTrackers? Sorry, I do not familiar
>> with
>> > mesos very much.
>> > Dies it mean I do not need configure hdfs-site.xml and core-site.xml in
>> > hadoop? I do not want to run my own TaskTracker, I just want to set up
>> > hadoop on mesos, and run my MR tasks.
>> >
>> > Thanks very much for your patient reply...Maybe I have a long way to
>> go...
>> >
>> >
>> >
>> > The log messages you see:
>> > 2013-04-18 16:47:19,645 INFO org.apache.hadoop.mapred.MesosScheduler:
>> > Unknown/exited TaskTracker: http://master:50060.
>> >
>> > Are printed when mesos does not know about the TaskTracker. We currently
>> > don't support running your own TaskTrackers, as the MesosScheduler will
>> > launch them on your behalf when needed.
>> >
>> > Are you managing your own TaskTrackers? The purpose of using Hadoop with
>> > mesos is that you no longer have to do that. We will detect that jobs
>> have
>> > pending map / reduce tasks and launch TaskTrackers accordingly.
>> >
>> > Guodong may be able to help further getting set up!
>> >
>> >
>> >
>> >
>> > Wang Yu
>> >
>> > From: 王国栋
>> > Date: 2013-04-18 17:10
>> > To: mesos-dev; wangyu
>> > Subject: Re: org.apache.hadoop.mapred.MesosScheduler: Unknown/exited
>> > TaskTracker: http://slave5:50060
>> > You can check the slave log and the mesos-executor log, which is
>> normally
>> > located in the dir like
>> >
>> >
>> "/tmp/mesos/slaves/201304181115-16842879-5050-4680-13/frameworks/201304181115-16842879-5050-4680-0003/executors/executor_Task_Tracker_16/runs/latest/stderr".
>> > The log is tasktracker log.
>> >
>> > I hope it will help.
>> >
>> > Guodong
>> >
>> >
>> > On Thu, Apr 18, 2013 at 5:03 PM, 王瑜 <wangyu@nfs.iscas.ac.cn> wrote:
>> >
>> > > **
>> > > Hi All,
>> > >
>> > > I have deployed mesos on three node: master, slave1, slave5. and it
>> works
>> > > well.
>> > >  Then I set hadoop over it, using master as namenode, and master,
>> slave1,
>> > > slave5 as datanode. When I using 'jps', it looks works well.
>> > >  [root@master logs]# jps
>> > > 13896 RunJar
>> > > 14123 Jps
>> > > 12718 NameNode
>> > > 12900 DataNode
>> > > 13374 TaskTracker
>> > > 13218 JobTracker
>> > >
>> > > Then I run test benchmark, it can not go on working...
>> > >  [root@master
>> > >  hadoop-0.20.205.0]# bin/hadoop jar hadoop-examples-0.20.205.0.jar
>> > randomwriter -Dtest.randomwrite.bytes_per_map=6710886
>> > -Dtest.randomwriter.maps_per_host=10 rand
>> > > Running 30 maps.
>> > > Job started: Thu Apr 18 16:49:36 CST 2013
>> > > 13/04/18 16:49:36 INFO mapred.JobClient: Running job:
>> > job_201304181646_0001
>> > > 13/04/18 16:49:37 INFO mapred.JobClient:  map 0% reduce 0%
>> > > It stopped here.
>> > >
>> > > Then I read the log file: hadoop-root-jobtracker-master.log, it shows:
>> > >  2013-04-18 16
>> > > :46:51,724 INFO org.apache.hadoop.mapred.JobTracker: Starting RUNNING
>> > > 2013-04-18 16
>> > > :46:51,726 INFO org.apache.hadoop.ipc.Server: IPC Server handler 5 on
>> > 9001: starting
>> > > 2013-04-18 16
>> > > :46:51,727 INFO org.apache.hadoop.ipc.Server: IPC Server handler 6 on
>> > 9001: starting
>> > > 2013-04-18 16
>> > > :46:51,727 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on
>> > 9001: starting
>> > > 2013-04-18 16
>> > > :46:51,727 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on
>> > 9001: starting
>> > > 2013-04-18 16
>> > > :46:51,727 INFO org.apache.hadoop.ipc.Server: IPC Server handler 8 on
>> > 9001: starting
>> > > 2013-04-18 16
>> > > :46:52,557 INFO org.apache.hadoop.net.NetworkTopology: Adding a new
>> > node: /default-rack/master
>> > > 2013-04-18 16
>> > > :46:52,560 INFO org.apache.hadoop.mapred.JobTracker: Adding tracker
>> > tracker_master:localhost/
>> > > 127.0.0.1:44997 to host master
>> > > 2013-04-18 16
>> > > :46:52,568 INFO org.apache.hadoop.mapred.MesosScheduler:
>> Unknown/exited
>> > TaskTracker:
>> > > http://master:50060.
>> > > 2013-04-18 16
>> > > :46:55,581 INFO org.apache.hadoop.mapred.MesosScheduler:
>> Unknown/exited
>> > TaskTracker:
>> > > http://master:50060.
>> > > 2013-04-18 16
>> > > :46:58,590 INFO org.apache.hadoop.mapred.MesosScheduler:
>> Unknown/exited
>> > TaskTracker:
>> > > http://master:50060.
>> > > 2013-04-18 16
>> > > :47:01,600 INFO org.apache.hadoop.mapred.MesosScheduler:
>> Unknown/exited
>> > TaskTracker:
>> > > http://master:50060.
>> > >
>> > > 2013-04-18 16:47:04,609 INFO org.apache.hadoop.mapred.MesosScheduler:
>> > Unknown/exited TaskTracker:
>> > > http://master:50060.
>> > >
>> > > 2013-04-18 16:47:07,618 INFO org.apache.hadoop.mapred.MesosScheduler:
>> > Unknown/exited TaskTracker:
>> > > http://master:50060.
>> > >
>> > > 2013-04-18 16:47:10,625 INFO org.apache.hadoop.mapred.MesosScheduler:
>> > Unknown/exited TaskTracker:
>> > > http://master:50060.
>> > >
>> > > 2013-04-18 16:47:13,632 INFO org.apache.hadoop.mapred.MesosScheduler:
>> > Unknown/exited TaskTracker:
>> > > http://master:50060.
>> > >
>> > > 2013-04-18 16:47:13,686 INFO org.apache.hadoop.net.NetworkTopology:
>> > Adding a new node: /default-rack/slave5
>> > >
>> > > 2013-04-18 16:47:13,686 INFO org.apache.hadoop.mapred.JobTracker:
>> Adding
>> > tracker tracker_slave5:
>> > > 127.0.0.1/127.0.0.1:60621 to host slave5
>> > >
>> > > 2013-04-18 16:47:13,687 INFO org.apache.hadoop.mapred.MesosScheduler:
>> > Unknown/exited TaskTracker:
>> > > http://slave5:50060.
>> > >
>> > > 2013-04-18 16:47:16,638 INFO org.apache.hadoop.mapred.MesosScheduler:
>> > Unknown/exited TaskTracker:
>> > > http://master:50060.
>> > >
>> > > 2013-04-18 16:47:16,697 INFO org.apache.hadoop.mapred.MesosScheduler:
>> > Unknown/exited TaskTracker:
>> > > http://slave5:50060.
>> > >
>> > > 2013-04-18 16:47:19,645 INFO org.apache.hadoop.mapred.MesosScheduler:
>> > Unknown/exited TaskTracker:
>> > > http://master:50060.
>> > >
>> > > 2013-04-18 16:47:19,707 INFO org.apache.hadoop.mapred.MesosScheduler:
>> > Unknown/exited TaskTracker:
>> > > http://slave5:50060.
>> > >
>> > > 2013-04-18 16:47:22,651 INFO org.apache.hadoop.mapred.MesosScheduler:
>> > Unknown/exited TaskTracker:
>> > > http://master:50060.
>> > >
>> > > 2013-04-18 16:47:22,715 INFO org.apache.hadoop.mapred.MesosScheduler:
>> > Unknown/exited TaskTracker:
>> > > http://slave5:50060.
>> > >
>> > > 2013-04-18 16:47:25,658 INFO org.apache.hadoop.mapred.MesosScheduler:
>> > Unknown/exited TaskTracker:
>> > > http://master:50060.
>> > >
>> > > 2013-04-18 16:47:25,725 INFO org.apache.hadoop.mapred.MesosScheduler:
>> > Unknown/exited TaskTracker:
>> > > http://slave5:50060.
>> > >
>> > > 2013-04-18 16:47:28,665 INFO org.apache.hadoop.mapred.MesosScheduler:
>> > Unknown/exited TaskTracker:
>> > > http://master:50060.
>> > >
>> > > Does anybody can help me? Thanks very much!
>> > >
>> >
>>
>
>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message