airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suresh Marru <sma...@apache.org>
Subject Re: Spinup Mesos-Marathon Cluster for Hybrid Scheduling
Date Fri, 16 Sep 2016 13:30:31 GMT
Hi Shameera,

All of these are great directions for Airavata, thank you for pushing the Ansible and Mesos
deployments on the clouds. I think it will be better if we get your scripts into Airavata
repo and all of us collectively work on it. Looks like atleast Pankaj and Gourav will also
be able to contribution in addition to you. 

Suresh

> On Sep 15, 2016, at 8:59 PM, Shenoy, Gourav Ganesh <goshenoy@indiana.edu> wrote:
> 
> Sure, thanks Shameera. I will try this.
>  
> Thanks and Regards,
> Gourav Shenoy
>  
> From: Shameera Rathnayaka <shameerainfo@gmail.com>
> Reply-To: "dev@airavata.apache.org" <dev@airavata.apache.org>
> Date: Thursday, September 15, 2016 at 8:55 PM
> To: "dev@airavata.apache.org" <dev@airavata.apache.org>
> Subject: Re: Spinup Mesos-Marathon Cluster for Hybrid Scheduling
>  
> Interesting, I am also getting the same issue. The same script worked perfectly yesterday.
I doubt some issue with marathon rpm. By removing marathon installation Mesos get installed
without any issue. 
>  
> to remove marathon installation do following to /roles/mesos-master/tasks/main.yml file.
>  
> 1. comment marathon in "install mesos and marathon" task
> 2. comment the last task which start marathon
>  
> Meanwhile, i will try to find exact reason.
>  
> ~ Shameera.
>  
> On Thu, Sep 15, 2016 at 8:32 PM Shenoy, Gourav Ganesh <goshenoy@indiana.edu <mailto:goshenoy@indiana.edu>>
wrote:
> Hi Shameera,
>  
> I am using the same image which you used (centos_ami_7_2: ami-6d1c2007).
>  
> Thanks and Regards,
> Gourav Shenoy
>  
> From: Shameera Rathnayaka <shameerainfo@gmail.com <mailto:shameerainfo@gmail.com>>
> Reply-To: "dev@airavata.apache.org <mailto:dev@airavata.apache.org>" <dev@airavata.apache.org
<mailto:dev@airavata.apache.org>>
> Date: Thursday, September 15, 2016 at 8:26 PM
> To: "dev@airavata.apache.org <mailto:dev@airavata.apache.org>" <dev@airavata.apache.org
<mailto:dev@airavata.apache.org>>
> Subject: Re: Spinup Mesos-Marathon Cluster for Hybrid Scheduling
>  
> Hi Gourav, 
>  
> According to the error, something have happened while unpacking marathon bundle, see:
 Installing : marathon-1.3.0-1.0.506.el7.x86_64                            1/1 \nerror: unpacking
of archive failed on file /usr/bin/marathon;57daffff: cpio: read\n  Verifying  : marathon-1.3.0-1.0.506.el7.x86_64
                           1/1 \n\nFailed:\n  marathon.x86_64 0:1.3.0-1.0.506.el7
>  
> What OS image and version you used to create instances? I tested with centos 7.2 and
it works fine. 
>  
> ~ Shameera.
>  
>  
> On Thu, Sep 15, 2016 at 8:14 PM Shenoy, Gourav Ganesh <goshenoy@indiana.edu <mailto:goshenoy@indiana.edu>>
wrote:
> Hi Shameera,
>  
> I am trying to build a mesos cluster on EC2 using your playbooks. But I am facing some
issues. Please find the details below:
>  
> Details:
> -          I created 4 instances on EC2 (us-east-1 region) using the cloud-provisioning
module (CloudBridge python). Out of the 4, 3 were meant to be mesos masters & 1 slave.

> Note: The instance inbound & outbount traffic is wideopen.
> 
> -          I skipped step-1 & step-2 in your README, since I manually provisioned
the instances. Next, I updated “hosts” file with public IPs for all 4 instances. And also
updated the “roles/zookeeper/vars/main.yml” file with the private IPs of 3 master instances.
> 
> -          I executed the “ansible-playbook -i hosts site.yml -t "mesos-master"”
command, and I get the following error:
> 
>  
> TASK [mesos-master : install firewalld] ****************************************
> ok: [52.91.152.1]
> ok: [52.87.235.79]
> ok: [54.167.94.186]
>  
> TASK [mesos-master : start firewalld] ******************************************
> ok: [52.91.152.1]
> ok: [52.87.235.79]
> ok: [54.167.94.186]
>  
> TASK [mesos-master : open ports] ***********************************************
> ok: [52.91.152.1] => (item=5050/tcp)
> ok: [52.87.235.79] => (item=5050/tcp)
> ok: [54.167.94.186] => (item=5050/tcp)
> ok: [52.87.235.79] => (item=8080/tcp)
> ok: [54.167.94.186] => (item=8080/tcp)
> ok: [52.91.152.1] => (item=8080/tcp)
>  
> TASK [mesos-master : install utility - TODO delete this] ***********************
> ok: [52.91.152.1] => (item=[u'vim'])
> ok: [52.87.235.79] => (item=[u'vim'])
> ok: [54.167.94.186] => (item=[u'vim'])
>  
> TASK [mesos-master : add mesosphere rpm] ***************************************
> ok: [52.91.152.1]
> ok: [52.87.235.79]
> ok: [54.167.94.186]
>  
> TASK [mesos-master : install mesos and marathon] *******************************
> failed: [52.91.152.1] (item=[u'mesos', u'marathon']) => {"changed": true, "failed":
true, "item": ["mesos", "marathon"], "msg": "Error unpacking rpm package marathon-1.3.0-1.0.506.el7.x86_64\n",
"rc": 1, "results": ["All packages providing mesos are up to date", "Loaded plugins: fastestmirror\nLoading
mirror speeds from cached hostfile\n * base: mirrors.tripadvisor.com <http://mirrors.tripadvisor.com/>\n
* extras: centos.hostingxtreme.com <http://centos.hostingxtreme.com/>\n * updates: mirrors.greenmountainaccess.net
<http://mirrors.greenmountainaccess.net/>\nResolving Dependencies\n--> Running transaction
check\n---> Package marathon.x86_64 0:1.3.0-1.0.506.el7 will be installed\n--> Finished
Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n
Package         Arch          Version                  Repository         Size\n================================================================================\nInstalling:\n
marathon        x86_64        1.3.0-1.0.506.el7        mesosphere         17 M\n\nTransaction
Summary\n================================================================================\nInstall
 1 Package\n\nTotal download size: 17 M\nInstalled size: 89 M\nDownloading packages:\nRunning
transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n
 Installing : marathon-1.3.0-1.0.506.el7.x86_64                            1/1 \nerror: unpacking
of archive failed on file /usr/bin/marathon;57daffff: cpio: read\n  Verifying  : marathon-1.3.0-1.0.506.el7.x86_64
                           1/1 \n\nFailed:\n  marathon.x86_64 0:1.3.0-1.0.506.el7        
                                  \n\nComplete!\n"]}
> failed: [52.87.235.79] (item=[u'mesos', u'marathon']) => {"changed": true, "failed":
true, "item": ["mesos", "marathon"], "msg": "Error unpacking rpm package marathon-1.3.0-1.0.506.el7.x86_64\n",
"rc": 1, "results": ["All packages providing mesos are up to date", "Loaded plugins: fastestmirror\nLoading
mirror speeds from cached hostfile\n * base: mirrors.tripadvisor.com <http://mirrors.tripadvisor.com/>\n
* extras: mirrors.evowise.com <http://mirrors.evowise.com/>\n * updates: mirrors.greenmountainaccess.net
<http://mirrors.greenmountainaccess.net/>\nResolving Dependencies\n--> Running transaction
check\n---> Package marathon.x86_64 0:1.3.0-1.0.506.el7 will be installed\n--> Finished
Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n
Package         Arch          Version                  Repository         Size\n================================================================================\nInstalling:\n
marathon        x86_64        1.3.0-1.0.506.el7        mesosphere         17 M\n\nTransaction
Summary\n================================================================================\nInstall
 1 Package\n\nTotal download size: 17 M\nInstalled size: 89 M\nDownloading packages:\nRunning
transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n
 Installing : marathon-1.3.0-1.0.506.el7.x86_64                            1/1 \nerror: unpacking
of archive failed on file /usr/bin/marathon;57daffff: cpio: read\n  Verifying  : marathon-1.3.0-1.0.506.el7.x86_64
                           1/1 \n\nFailed:\n  marathon.x86_64 0:1.3.0-1.0.506.el7        
                                  \n\nComplete!\n"]}
> failed: [54.167.94.186] (item=[u'mesos', u'marathon']) => {"changed": true, "failed":
true, "item": ["mesos", "marathon"], "msg": "Error unpacking rpm package marathon-1.3.0-1.0.506.el7.x86_64\n",
"rc": 1, "results": ["All packages providing mesos are up to date", "Loaded plugins: fastestmirror\nLoading
mirror speeds from cached hostfile\n * base: mirrors.tripadvisor.com <http://mirrors.tripadvisor.com/>\n
* extras: mirrors.evowise.com <http://mirrors.evowise.com/>\n * updates: mirrors.greenmountainaccess.net
<http://mirrors.greenmountainaccess.net/>\nResolving Dependencies\n--> Running transaction
check\n---> Package marathon.x86_64 0:1.3.0-1.0.506.el7 will be installed\n--> Finished
Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n
Package         Arch          Version                  Repository         Size\n================================================================================\nInstalling:\n
marathon        x86_64        1.3.0-1.0.506.el7        mesosphere         17 M\n\nTransaction
Summary\n================================================================================\nInstall
 1 Package\n\nTotal download size: 17 M\nInstalled size: 89 M\nDownloading packages:\nRunning
transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n
 Installing : marathon-1.3.0-1.0.506.el7.x86_64                            1/1 \nerror: unpacking
of archive failed on file /usr/bin/marathon;57daffff: cpio: read\n  Verifying  : marathon-1.3.0-1.0.506.el7.x86_64
                           1/1 \n\nFailed:\n  marathon.x86_64 0:1.3.0-1.0.506.el7        
                                  \n\nComplete!\n"]}
>  
> NO MORE HOSTS LEFT *************************************************************
>  
> RUNNING HANDLER [zookeeper : restart zookeeper] ********************************
> [WARNING]: Could not create retry file 'site.retry'.         [Errno 2] No such file or
directory: ''
>  
>  
> PLAY RECAP *********************************************************************
> 52.87.235.79               : ok=17   changed=2    unreachable=0    failed=1   
> 52.91.152.1                : ok=17   changed=2    unreachable=0    failed=1   
> 54.167.94.186              : ok=17   changed=2    unreachable=0    failed=1  
> localhost                  : ok=1    changed=0    unreachable=0    failed=0
>  
> Is there some step that I am missing? It looks like the instances are not able to communicate
because of the firewall? This is just a wild guess. Any help here is appreciated.
> 
> Thanks and Regards,
> Gourav Shenoy
>  
> From: Shameera Rathnayaka <shameerainfo@gmail.com <mailto:shameerainfo@gmail.com>>
> Reply-To: "dev@airavata.apache.org <mailto:dev@airavata.apache.org>" <dev@airavata.apache.org
<mailto:dev@airavata.apache.org>>
> Date: Monday, September 12, 2016 at 11:19 AM
> To: dev <dev@airavata.apache.org <mailto:dev@airavata.apache.org>>
> Subject: Spinup Mesos-Marathon Cluster for Hybrid Scheduling
>  
> Hi Dev, 
>  
> The effort of getting use Cloud infrastructure to run MPI and BigData jobs using Airavata,
we use Apache Mesos as  resource allocation framework to manage different type of clusters
(i.e HPC node cluster to run MPI jobs and spark, Hadoop big data clusters to run bigdata applications).
I came up with Ansible script to spin up Mesos cluster on the target set of nodes. You can
find the script herehttps://github.com/shamrath/mesos-deployment <https://github.com/shamrath/mesos-deployment>
I am thinking of  move this code to Airavata if all agreed. I would happy to answer any question
related to this. 
>  
> Thanks, 
> Shameera.
> --
> Shameera Rathnayaka
> --
> Shameera Rathnayaka
> -- 
> Shameera Rathnayaka


Mime
View raw message