mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grzegorz Graczyk <gregor...@gmail.com>
Subject Re: Mesos fetcher in dockerized slave
Date Fri, 18 Dec 2015 11:11:46 GMT
I've tried to use this flag, but cannot really run any container when this
flag is set.
I've raised this issue here:
https://www.mail-archive.com/user@mesos.apache.org/msg04975.html and here:
https://github.com/mesosphere/docker-containers/issues/6#issuecomment-155364351
but
sadly no one was able to help me...

pt., 18.12.2015 o 11:33 użytkownik Marica Antonacci <
marica.antonacci@ba.infn.it> napisał:

> OK, the problem I spotted is related to the usage of the
> flag —docker_mesos_image that allows the executor to
>
>
> --docker_mesos_image=VALUEThe docker image used to launch this mesos
> slave instance. If an image is specified, the docker containerizer assumes
> the slave is running in a docker container, and launches executors with
> docker containers in order to recover them when the slave restarts and
> recovers.
> Has anyone used this flag and tested the behavior of the fetcher?
>
> Thank you
> Marica
>
>
> Il giorno 18/dic/2015, alle ore 10:38, tommy xiao <xiaods@gmail.com> ha
> scritto:
>
> no docker_mesos_image flag in my docker run,  and the docker image is
> build by myself.
>
>
>
> 2015-12-18 17:20 GMT+08:00 Marica Antonacci <marica.antonacci@ba.infn.it>:
>
> Yes, I did check inside the container and the csv file was not downloaded
>> as shown also by the app details (see the screenshot below).
>>
>> Are you running your slave with the --docker_mesos_image flag? Can you
>> please provide me the docker run command you are using to run your
>> dockerized slave?
>>
>> Thank you very much
>>
> Marica
>>
>>
>> <Schermata 2015-12-18 alle 10.15.58.png>
>>
>
>>
>> Il giorno 18/dic/2015, alle ore 10:00, tommy xiao <xiaods@gmail.com> ha
>> scritto:
>>
>> Hi Marica,
>>
>> use your test-app json, i can run it correctly, the csv is truely
>> download by mesos slave. please check mesos-master:5050 to check the task
>> detail download files.
>>
>> you describe the app container why not found the csv, because the csv is
>> download in slave container's folder, not in app container. so if you run
>>
>> cd $MESOS_SANDBOX;
>>
>> the folder in app container is default value:
>>
>> MESOS_SANDBOX=/mnt/mesos/sandbox
>> but in real world, the sandbox is in slave container, not in app
>> container.
>>
>>
>>
>> 2015-12-18 16:11 GMT+08:00 Marica Antonacci <marica.antonacci@ba.infn.it>
>> :
>>
>>> Thank you very much,
>>>
>>> I’m using a sample application definition file, just for testing purpose:
>>>
>>> {
>>>  "id": "test-app",
>>>  "container": {
>>>    "type": "DOCKER",
>>>    "docker": {
>>>      "image": "libmesos/ubuntu"
>>>    }
>>>  },
>>>  "cpus": 1,
>>>  "mem": 512,
>>>  *"uris": [
>>> "http://www.stat.cmu.edu/~cshalizi/402/lectures/16-glm-practicals/snoqualmie.csv
>>> <http://www.stat.cmu.edu/~cshalizi/402/lectures/16-glm-practicals/snoqualmie.csv>"
>>> ],*
>>>  "cmd": "cd $MESOS_SANDBOX; ls -latr; while sleep 10; do date -u +%T;
>>> done"
>>> }
>>>
>>> Here is the docker run command line:
>>>
>>> # docker run -d -e MESOS_HOSTNAME=<SLAVE_IP> -e MESOS_IP=<SLAVE_IP>
-e
>>> MESOS_MASTER=zk://<node-1>:2181,<node-2>:2181,<node-3>:2181/mesos
>>> -e MESOS_CONTAINERIZERS=docker,mesos \
>>>                       -e MESOS_EXECUTOR_REGISTRATION_TIMEOUT=5mins -e
>>> MESOS_LOG_DIR=/var/log -e MESOS_docker_mesos_image=mesos-slave
>>>                       -v /sys/fs/cgroup:/sys/fs/cgroup -v
>>> /var/run/docker.sock:/var/run/docker.sock --name slave --net host
>>> --privileged --pid host mesos-slave
>>>
>>>
>>> As already mentioned, if I remove the environment variable
>>> MESOS_docker_mesos_image the fetcher works fine and I can see the file
>>> snoqualmie.csv inside the sandbox.
>>>
>>> Thank you again! I’m looking forward to hearing about your outcomes.
>>> Best regards,
>>> Marica
>>>
>>>
>>> Il giorno 18/dic/2015, alle ore 04:51, tommy xiao <xiaods@gmail.com> ha
>>> scritto:
>>>
>>> Hi Marica,
>>>
>>> Could you please give a sample marathon json, i can test it asap.
>>>
>>> 2015-12-18 5:11 GMT+08:00 Marica Antonacci <marica.antonacci@ba.infn.it>
>>> :
>>>
>>>> No, using the socket:
>>>>
>>>> -v /var/run/docker.sock:/var/run/docker.sock
>>>>
>>>>
>>>> Il giorno 17/dic/2015, alle ore 18:07, tommy xiao <xiaods@gmail.com>
>>>> ha scritto:
>>>>
>>>> docker in docker mode?
>>>>
>>>> 2015-12-17 19:08 GMT+08:00 Marica Antonacci <
>>>> marica.antonacci@ba.infn.it>:
>>>>
>>>>> Dear all,
>>>>>
>>>>> I'm testing the URIs fetching mechanism for both Marathon applications
>>>>> and Chronos jobs and I have found that if the slave is running inside
a
>>>>> docker container (using *docker_mesos_image* startup flag) and you
>>>>> submit the deployment of a dockerized application or job the fetcher
step
>>>>> is not performed. On the other hand, if I request the deployment of a
>>>>> non-dockerized application, the URIs are correctly fetched. Moreover,
if I
>>>>> don’t provide the docker_mesos_image flag, the fetcher works fine again
for
>>>>> both dockerized and non-dockerized applications.
>>>>>
>>>>> Therefore, it seems that the information about the URIs gets lost when
>>>>> the dockerized mesos slave spawns the executor docker container that
in
>>>>> turn launches the application docker container…Has anyone seen this
problem
>>>>> before? I would like to know if there is a workaround or a fixing.
>>>>>
>>>>> Thanks a lot in advance for you help
>>>>> Best Regards,
>>>>> Marica
>>>>>
>>>>>
>>>>> ----------------------------------------------------------
>>>>>
>>>>> Marica ANTONACCI
>>>>> INFN - National Institute of Nuclear Physics
>>>>> Via Orabona 4
>>>>> 70126 Bari - ITALY
>>>>> Phone +39 080 5443244
>>>>> Skype: marica.antonacci
>>>>> e-mail marica.antonacci@ba.infn.it
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Deshi Xiao
>>>> Twitter: xds2000
>>>> E-mail: xiaods(AT)gmail.com
>>>>
>>>>
>>>> ----------------------------------------------------------
>>>>
>>>> Marica ANTONACCI
>>>> INFN - National Institute of Nuclear Physics
>>>> Via Orabona 4
>>>> 70126 Bari - ITALY
>>>> Phone +39 080 5443244
>>>> Skype: marica.antonacci
>>>> e-mail marica.antonacci@ba.infn.it
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Deshi Xiao
>>> Twitter: xds2000
>>> E-mail: xiaods(AT)gmail.com
>>>
>>>
>>> ----------------------------------------------------------
>>>
>>> Marica ANTONACCI
>>> INFN - National Institute of Nuclear Physics
>>> Via Orabona 4
>>> 70126 Bari - ITALY
>>> Phone +39 080 5443244
>>> Skype: marica.antonacci
>>> e-mail marica.antonacci@ba.infn.it
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>> --
>> Deshi Xiao
>> Twitter: xds2000
>> E-mail: xiaods(AT)gmail.com
>> <屏幕快照 2015-12-18 下午4.55.16.png><屏幕快照 2015-12-18 下午4.55.30.png><屏幕快照
>> 2015-12-18 下午4.55.38.png>
>>
>>
>> ----------------------------------------------------------
>>
>> Marica ANTONACCI
>> INFN - National Institute of Nuclear Physics
>> Via Orabona 4
>> 70126 Bari - ITALY
>> Phone +39 080 5443244
>> Skype: marica.antonacci
>> e-mail marica.antonacci@ba.infn.it
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> --
> Deshi Xiao
> Twitter: xds2000
> E-mail: xiaods(AT)gmail.com
>
>
> ----------------------------------------------------------
>
> Marica ANTONACCI
> INFN - National Institute of Nuclear Physics
> Via Orabona 4
> 70126 Bari - ITALY
> Phone +39 080 5443244
> Skype: marica.antonacci
> e-mail marica.antonacci@ba.infn.it
>
>
>
>
>
>
>
>
>
>
>

Mime
View raw message