incubator-mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Xie <zhiy...@fb.com>
Subject Re: About Running Spark on a cluster with mesos Re: WELCOME to mesos-dev@incubator.apache.org
Date Mon, 22 Aug 2011 22:06:47 GMT
Sorry this should have been sent to user mail list.

On 8/22/11 2:19 PM, "Joe Xie" <zhiyong@fb.com> wrote:

>Hi there:
>
>I has build and installed mesos (pre-protobuf branch for spark) and spark,
>and deployed them to a cluster. However, I have following error when I run
>spark task on the mesos URL of my master, the spark task runs fine in
>local mode. Any suggestion or help? Thanks a lot!
>Joe
>
>My os: CentOS 5.2
>Python: 2.6
>Gcc: 4.2.1
>Swig:1.3.40
>./run spark.examples.SparkPi mesos://1@xx.xx.xx.xx:5050  1
>11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registered actor on port
>50501
>11/08/22 13:31:11 INFO spark.MapOutputTrackerActor: Registered actor on
>port 50501
>11/08/22 13:31:11 INFO spark.SparkContext: Starting job...
>11/08/22 13:31:11 INFO spark.CacheTracker: Registering RDD ID 0 with cache
>11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registering RDD 0 with 1
>partitions
>11/08/22 13:31:11 INFO spark.CacheTrackerActor: Asked for current cache
>locations
>11/08/22 13:31:12 INFO spark.MesosScheduler: Final stage: Stage 0
>11/08/22 13:31:12 INFO spark.MesosScheduler: Parents of final stage:
>List()
>11/08/22 13:31:12 INFO spark.MesosScheduler: Missing parents: List()
>11/08/22 13:31:12 INFO spark.MesosScheduler: Submitting Stage 0, which has
>no missing parents
>11/08/22 13:31:12 INFO spark.MesosScheduler: Got a job with 1 tasks
>Exception in thread "Spark scheduler" java.lang.UnsatisfiedLinkError:
>org.apache.mesos.MesosSchedulerDriver.initialize()V
>	at org.apache.mesos.MesosSchedulerDriver.initialize(Native Method)
>	at
>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.java:42)
>	at
>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.java:46)
>	at spark.MesosScheduler$$anon$1.run(MesosScheduler.scala:94)
>
>Sent to wrong address previously
>
>
>>
>>
>>On 8/21/11 4:30 PM, "mesos-dev-help@incubator.apache.org"
>><mesos-dev-help@incubator.apache.org> wrote:
>>
>>>Hi! This is the ezmlm program. I'm managing the
>>>mesos-dev@incubator.apache.org mailing list.
>>>
>>>I'm working for my owner, who can be reached
>>>at mesos-dev-owner@incubator.apache.org.
>>>
>>>Acknowledgment: I have added the address
>>>
>>>   zhiyong@fb.com
>>>
>>>to the mesos-dev mailing list.
>>>
>>>Welcome to mesos-dev@incubator.apache.org!
>>>
>>>Please save this message so that you know the address you are
>>>subscribed under, in case you later want to unsubscribe or change your
>>>subscription address.
>>>
>>>
>>>--- Administrative commands for the mesos-dev list ---
>>>
>>>I can handle administrative requests automatically. Please
>>>do not send them to the list address! Instead, send
>>>your message to the correct command address:
>>>
>>>To subscribe to the list, send a message to:
>>>   <mesos-dev-subscribe@incubator.apache.org>
>>>
>>>To remove your address from the list, send a message to:
>>>   <mesos-dev-unsubscribe@incubator.apache.org>
>>>
>>>Send mail to the following for info and FAQ for this list:
>>>   <mesos-dev-info@incubator.apache.org>
>>>   <mesos-dev-faq@incubator.apache.org>
>>>
>>>Similar addresses exist for the digest list:
>>>   <mesos-dev-digest-subscribe@incubator.apache.org>
>>>   <mesos-dev-digest-unsubscribe@incubator.apache.org>
>>>
>>>To get messages 123 through 145 (a maximum of 100 per request), mail:
>>>   <mesos-dev-get.123_145@incubator.apache.org>
>>>
>>>To get an index with subject and author for messages 123-456 , mail:
>>>   <mesos-dev-index.123_456@incubator.apache.org>
>>>
>>>They are always returned as sets of 100, max 2000 per request,
>>>so you'll actually get 100-499.
>>>
>>>To receive all messages with the same subject as message 12345,
>>>send a short message to:
>>>   <mesos-dev-thread.12345@incubator.apache.org>
>>>
>>>The messages should contain one line or word of text to avoid being
>>>treated as sp@m, but I will ignore their content.
>>>Only the ADDRESS you send to is important.
>>>
>>>You can start a subscription for an alternate address,
>>>for example "john@host.domain", just add a hyphen and your
>>>address (with '=' instead of '@') after the command word:
>>><mesos-dev-subscribe-john=host.domain@incubator.apache.org>
>>>
>>>To stop subscription for this address, mail:
>>><mesos-dev-unsubscribe-john=host.domain@incubator.apache.org>
>>>
>>>In both cases, I'll send a confirmation message to that address. When
>>>you receive it, simply reply to it to complete your subscription.
>>>
>>>If despite following these instructions, you do not get the
>>>desired results, please contact my owner at
>>>mesos-dev-owner@incubator.apache.org. Please be patient, my owner is a
>>>lot slower than I am ;-)
>>>
>>>--- Enclosed is a copy of the request I received.
>>>
>>>Return-Path: <zhiyong@fb.com>
>>>Received: (qmail 39962 invoked by uid 99); 21 Aug 2011 23:30:07 -0000
>>>Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
>>>    by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 23:30:07
>>>+0000
>>>X-ASF-Spam-Status: No, hits=-0.0 required=5.0
>>>	tests=SPF_PASS
>>>X-Spam-Check-By: apache.org
>>>Received-SPF: pass (nike.apache.org: domain of zhiyong@fb.com designates
>>>67.231.145.42 as permitted sender)
>>>Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com)
>>>(67.231.145.42)
>>>    by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 23:30:00
>>>+0000
>>>Received: from pps.filterd (m0004348 [127.0.0.1])
>>>	by m0004348.ppops.net (8.14.4/8.14.4) with SMTP id p7LNTI6g015638
>>>	for 
>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator.a
>>>p
>>>a
>>>che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>[66.220.144.38])
>>>	by m0004348.ppops.net with ESMTP id yb5cerc88-2
>>>	(version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT)
>>>	for 
>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator.a
>>>p
>>>a
>>>che.org>; Sun, 21 Aug 2011 16:29:18 -0700
>>>Received: from SC-MBX01-5.TheFacebook.com ([fe80::f05e:c7f1:c7dd:e640])
>>>by
>>> sc-hub04.TheFacebook.com ([192.168.18.212]) with mapi id
>>>14.01.0289.001;
>>>Sun,
>>> 21 Aug 2011 16:29:14 -0700
>>>From: Joe Xie <zhiyong@fb.com>
>>>To:   
>>>"mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator.a
>>>p
>>>a
>>>che.org"
>>>	<mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator.
>>>a
>>>p
>>>ache.org>
>>>Subject: Re: confirm subscribe to mesos-dev@incubator.apache.org
>>>Thread-Topic: confirm subscribe to mesos-dev@incubator.apache.org
>>>Thread-Index: AQHMYFO/ABVA64TXDkGEVkIetlsT6JUn85UA
>>>Date: Sun, 21 Aug 2011 23:29:13 +0000
>>>Message-ID: <CA76E2A4.4259%zhiyong@fb.com>
>>>In-Reply-To: <1313966607.19992.ezmlm@incubator.apache.org>
>>>Accept-Language: en-US
>>>Content-Language: en-US
>>>X-MS-Has-Attach:
>>>X-MS-TNEF-Correlator:
>>>x-originating-ip: [192.168.18.252]
>>>Content-Type: text/plain; charset="us-ascii"
>>>Content-ID: <B4FEB00A5AF0F24BB8B12796F82D0E01@fb.com>
>>>Content-Transfer-Encoding: quoted-printable
>>>MIME-Version: 1.0
>>>X-Virus-Checked: Checked by ClamAV on apache.org
>>>
>>>Thanks!
>>>Joe
>>>On 8/21/11 3:43 PM, "mesos-dev-help@incubator.apache.org"
>>><mesos-dev-help@incubator.apache.org> wrote:
>>>
>>>>Hi! This is the ezmlm program. I'm managing the
>>>>mesos-dev@incubator.apache.org mailing list.
>>>>
>>>>I'm working for my owner, who can be reached
>>>>at mesos-dev-owner@incubator.apache.org.
>>>>
>>>>To confirm that you would like
>>>>
>>>>   zhiyong@fb.com
>>>>
>>>>added to the mesos-dev mailing list, please send
>>>>a short reply to this address:
>>>>
>>>>  =20
>>>>mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com@incubator
>>>>.
>>>>a
>>>>p=
>>>ac
>>>>he.org
>>>>
>>>>Usually, this happens when you just hit the "reply" button.
>>>>If this does not work, simply copy the address and paste it into
>>>>the "To:" field of a new message.
>>>>
>>>>or click here:
>>>>	mailto:mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com@i
>>>>n
>>>>c
>>>>u=
>>>ba
>>>>tor.apache.org
>>>>
>>>>This confirmation serves two purposes. First, it verifies that I am
>>>>able
>>>>to get mail through to you. Second, it protects you in case someone
>>>>forges a subscription request in your name.
>>>>
>>>>Please note that ALL Apache dev- and user- mailing lists are publicly
>>>>archived.  Do familiarize yourself with Apache's public archive policy
>>>>at
>>>>
>>>>    http://www.apache.org/foundation/public-archives.html
>>>>
>>>>prior to subscribing and posting messages to
>>>>mesos-dev@incubator.apache.org.
>>>>If you're not sure whether or not the policy applies to this mailing
>>>>list,
>>>>assume it does unless the list name contains the word "private" in it.
>>>>
>>>>Some mail programs are broken and cannot handle long addresses. If you
>>>>cannot reply to this request, instead send a message to
>>>><mesos-dev-request@incubator.apache.org> and put the
>>>>entire address listed above into the "Subject:" line.
>>>>
>>>>
>>>>--- Administrative commands for the mesos-dev list ---
>>>>
>>>>I can handle administrative requests automatically. Please
>>>>do not send them to the list address! Instead, send
>>>>your message to the correct command address:
>>>>
>>>>To subscribe to the list, send a message to:
>>>>   <mesos-dev-subscribe@incubator.apache.org>
>>>>
>>>>To remove your address from the list, send a message to:
>>>>   <mesos-dev-unsubscribe@incubator.apache.org>
>>>>
>>>>Send mail to the following for info and FAQ for this list:
>>>>   <mesos-dev-info@incubator.apache.org>
>>>>   <mesos-dev-faq@incubator.apache.org>
>>>>
>>>>Similar addresses exist for the digest list:
>>>>   <mesos-dev-digest-subscribe@incubator.apache.org>
>>>>   <mesos-dev-digest-unsubscribe@incubator.apache.org>
>>>>
>>>>To get messages 123 through 145 (a maximum of 100 per request), mail:
>>>>   <mesos-dev-get.123_145@incubator.apache.org>
>>>>
>>>>To get an index with subject and author for messages 123-456 , mail:
>>>>   <mesos-dev-index.123_456@incubator.apache.org>
>>>>
>>>>They are always returned as sets of 100, max 2000 per request,
>>>>so you'll actually get 100-499.
>>>>
>>>>To receive all messages with the same subject as message 12345,
>>>>send a short message to:
>>>>   <mesos-dev-thread.12345@incubator.apache.org>
>>>>
>>>>The messages should contain one line or word of text to avoid being
>>>>treated as sp@m, but I will ignore their content.
>>>>Only the ADDRESS you send to is important.
>>>>
>>>>You can start a subscription for an alternate address,
>>>>for example "john@host.domain", just add a hyphen and your
>>>>address (with '=3D' instead of '@') after the command word:
>>>><mesos-dev-subscribe-john=3Dhost.domain@incubator.apache.org>
>>>>
>>>>To stop subscription for this address, mail:
>>>><mesos-dev-unsubscribe-john=3Dhost.domain@incubator.apache.org>
>>>>
>>>>In both cases, I'll send a confirmation message to that address. When
>>>>you receive it, simply reply to it to complete your subscription.
>>>>
>>>>If despite following these instructions, you do not get the
>>>>desired results, please contact my owner at
>>>>mesos-dev-owner@incubator.apache.org. Please be patient, my owner is a
>>>>lot slower than I am ;-)
>>>>
>>>>--- Enclosed is a copy of the request I received.
>>>>
>>>>Return-Path: <zhiyong@fb.com>
>>>>Received: (qmail 19986 invoked by uid 99); 21 Aug 2011 22:43:26 -0000
>>>>Received: from athena.apache.org (HELO athena.apache.org)
>>>>(140.211.11.136)
>>>>    by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 22:43:26
>>>>+0000
>>>>X-ASF-Spam-Status: No, hits=3D0.2 required=3D10.0
>>>>	tests=3DASF_LIST_OPS,HTML_MESSAGE,SPF_PASS
>>>>X-Spam-Check-By: apache.org
>>>>Received-SPF: pass (athena.apache.org: domain of zhiyong@fb.com
>>>>designates 67.231.145.42 as permitted sender)
>>>>Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com)
>>>>(67.231.145.42)
>>>>    by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 22:43:19
>>>>+0000
>>>>Received: from pps.filterd (m0004347 [127.0.0.1])
>>>>	by m0004347.ppops.net (8.14.4/8.14.4) with SMTP id p7LMcbSq017336
>>>>	for <mesos-dev-subscribe@incubator.apache.org>; Sun, 21 Aug 2011
>>>>15:42:58 -0700
>>>>Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net
>>>>[66.220.144.38])
>>>>	by m0004347.ppops.net with ESMTP id yb0jhrkt9-1
>>>>	(version=3DTLSv1/SSLv3 cipher=3DAES128-SHA bits=3D128 verify=3DNOT)
>>>>	for <mesos-dev-subscribe@incubator.apache.org>; Sun, 21 Aug 2011
>>>>15:42:58 -0700
>>>>Received: from SC-MBX01-5.TheFacebook.com ([fe80::f05e:c7f1:c7dd:e640])
>>>>by
>>>> sc-hub03.TheFacebook.com ([192.168.18.198]) with mapi id
>>>>14.01.0289.001;
>>>>Sun,
>>>> 21 Aug 2011 15:42:53 -0700
>>>>From: Joe Xie <zhiyong@fb.com>
>>>>To: "mesos-dev-subscribe@incubator.apache.org"
>>>>	<mesos-dev-subscribe@incubator.apache.org>
>>>>Subject: can i join this
>>>>Thread-Topic: can i join this
>>>>Thread-Index: AQHMYFOp36obbNljKE2n3mexQg9sQQ=3D=3D
>>>>Date: Sun, 21 Aug 2011 22:42:53 +0000
>>>>Message-ID: <CA76D7F7.4256%zhiyong@fb.com>
>>>>Accept-Language: en-US
>>>>Content-Language: en-US
>>>>X-MS-Has-Attach:=20
>>>>X-MS-TNEF-Correlator:
>>>>x-originating-ip: [192.168.18.252]
>>>>Content-Type: multipart/alternative;
>>>>	boundary=3D"_000_CA76D7F74256zhiyongfbcom_"
>>>>MIME-Version: 1.0
>>>>
>>>>--_000_CA76D7F74256zhiyongfbcom_
>>>>Content-Type: text/plain; charset=3D"us-ascii"
>>>>Content-Transfer-Encoding: quoted-printable
>>>>
>>>>Best,
>>>>Joe
>>>>
>>>>--_000_CA76D7F74256zhiyongfbcom_
>>>>Content-Type: text/html; charset=3D"us-ascii"
>>>>Content-ID: <F6BAF69B18270946ABCC53E8D258B722@fb.com>
>>>>Content-Transfer-Encoding: quoted-printable
>>>>
>>>><html>
>>>><head>
>>>><meta http-equiv=3D3D"Content-Type" content=3D3D"text/html;
>>>>charset=3D3Dus-ascii"=3D
>>>>>
>>>></head>
>>>><body style=3D3D"word-wrap: break-word; -webkit-nbsp-mode: space;
>>>>-webkit-lin=3D
>>>>e-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px;
>>>>font-fami=3D
>>>>ly: Calibri, sans-serif; ">
>>>><div>Best,</div>
>>>><div>Joe</div>
>>>></body>
>>>></html>
>>>>
>>>>--_000_CA76D7F74256zhiyongfbcom_--
>>>
>>
>


Mime
View raw message