mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kone <vi...@twitter.com>
Subject Re: Ability to pass params to Task without TASK_LOST
Date Mon, 07 Oct 2013 08:03:12 GMT
Cool. LGTM. 

@vinodkone
Sent from my mobile 

> On Oct 7, 2013, at 12:53 AM, Vladimir Vivien <vladimir.vivien@gmail.com> wrote:
> 
> @vinodkone
> I was getting task rejection for the reasons you mentioned i think because I was same
ExecutorID chaning ExecutorInfo (Environment Vars were changing).  I changed the code to the
snippet below (hope formatting holds) .  I am using setData() as you suggested.  This works
much better.
> 
> Protos.TaskID taskId = Protos.TaskID.newBuilder().setValue("task-" + UUID.randomUUID().toString()).build();
>     Protos.TaskInfo task = Protos.TaskInfo.newBuilder()
>         .setName(taskId.getValue())
>         .setTaskId(taskId)
>         .setSlaveId(offer.getSlaveId())
>         .addResources(
>             Protos.Resource.newBuilder()
>                 .setName("cpus")
>                 .setType(Protos.Value.Type.SCALAR)
>                 .setScalar(Protos.Value.Scalar.newBuilder().setValue(1).build())
>                 .build()
>         )
>         .addResources(
>             Protos.Resource.newBuilder()
>                 .setName("mem")
>                 .setType(Protos.Value.Type.SCALAR)
>                 .setScalar(Protos.Value.Scalar.newBuilder().setValue(256).build())
>                 .build()
>         )
>         .setExecutor(
>             Protos.ExecutorInfo.newBuilder()
>                  .setExecutorId(Protos.ExecutorID.newBuilder().setValue("my-executor"))
>                  .setCommand(Protos.CommandInfo.newBuilder()
>                      .setValue(new File(TASK_CLI_COMMAND).getAbsolutePath())
>                      .build()
>                  )
>                  .setName("My Task Executor")
>             .build() // ExecutorInfo
>         )
>         .setData(ByteString.copyFromUtf8(Integer.toString(taskSize)))
>         .build();
> 
> 
>> On Sun, Oct 6, 2013 at 9:49 PM, Vinod Kone <vinod@twitter.com> wrote:
>> I see. So were you trying to set ExecutorInfo.Command() to pass environment variables
to the tasks? If yes, I would not recommend it. For one, ExecutorInfo.Command() should be
used for setting up properties specific to Executor but not tasks. This is especially true
if the Executor can run multiple tasks. Second, Mesos would reject a task if it reuses Task.ExecutorInfo.ExecutorID
but uses a different ExecutorInfo. In your case I imagine ExecutorInfo changed between tasks
because ExecutorInfo.Command changed?
>> 
>>  
>> 
>> 
>> @vinodkone
>> 
>> 
>>> On Sun, Oct 6, 2013 at 1:45 PM, Vladimir Vivien <vladimir.vivien@gmail.com>
wrote:
>>> @Vinodkone
>>> Thanks for the reply.  It clarifies things.
>>> To be clear I am using an ExecutorInfo.
>>> I am also generating new taskId for each launch request.  
>>> 
>>> But, I will look into TaskInfo.data to pass data into my task.
>>> 
>>> Thanks
>>> 
>>> 
>>>> On Sun, Oct 6, 2013 at 2:55 PM, Vinod Kone <vinod@twitter.com> wrote:
>>>> Hi Vladimir,
>>>> 
>>>> You cannot launch multiple tasks with same taskID for a given framework.
TaskInfo.Command should only be used if you want to use Mesos's Command Executor. If you have
your own executor, TaskInfo.ExecutorInfo should be set instead.
>>>> 
>>>> Also, I would recommend using TaskInfo.data to send any data that your task
might need.
>>>> 
>>>> 
>>>> 
>>>> @vinodkone
>>>> 
>>>> 
>>>>> On Sun, Oct 6, 2013 at 11:35 AM, Vladimir Vivien <vladimir.vivien@gmail.com>
wrote:
>>>>> All,
>>>>> I ran into a situation where I need to pass params/values to tasks as
they are launched from the scheduler. The only mechanism that I see is Command.Environment
variables.  
>>>>> 
>>>>> However, when I attempt to launch tasks with same environment variable
names, the master rejects subsequent tasks with the taskinfo as invalid due task already exists
(paraphrase) and the task is marked as TASK_LOST.
>>>>> 
>>>>> Is there another way to reliably pass params/values to a task?  In my
situation, I want to launch a task (upon resource offer) and pass a new value to the task
upon each Executor.launch callback.  
>>>>> 
>>>>> Thanks.
>>>>> 
>>>>> -- 
>>>>> Vladimir Vivien
>>> 
>>> 
>>> 
>>> -- 
>>> Vladimir Vivien
> 
> 
> 
> -- 
> Vladimir Vivien

Mime
View raw message