mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Till Toenshoff <toensh...@me.com>
Subject Re: Review Request 57762: Fixed environment duplication in default executor.
Date Mon, 20 Mar 2017 14:15:31 GMT


> On March 20, 2017, 11:43 a.m., Benjamin Bannier wrote:
> > src/launcher/executor.cpp
> > Lines 476-477 (patched)
> > <https://reviews.apache.org/r/57762/diff/2/?file=1668103#file1668103line476>
> >
> >     `ValueSecret` seems to do the job of `Environment::Variable` which causes a
lot of manual duplicate validation and conversions in this patch. I believe if you'd use a
map of `(string, Environment::Variable)` the patch could be simplified, e.g., something along
the lines of
> >     
> >         hashmap<string, Environment::Variable> environment;
> >     
> >         foreachpair (const string& name, const string& value, os::environment())
{
> >           Environment::Variable variable;
> >           variable.set_name(name);
> >           variable.set_type(Environment::Variable::VALUE);
> >           variable.set_value(value);
> >           environment.put(name, variable);
> >         }
> >     
> >         if (taskEnvironment.isSome()) {
> >           foreach (
> >               const Environment::Variable& variable,
> >               taskEnvironment->variables()) {
> >             environment[variable.name()] = variable;
> >           }
> >         }
> >     
> >         if (command.has_environment()) {
> >           foreach (
> >               const Environment::Variable& variable,
> >               command.environment().variables()) {
> >             environment[variable.name()] = variable;
> >           }
> >         }
> >     
> >         Environment launchEnvironment;
> >         foreachvalue (const Environment::Variable& value, environment) {
> >           launchEnvironment.add_variables()->CopyFrom(value);
> >         }

Awesome Benjamin - this is a perfect simplification - thanks so much.


- Till


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57762/#review169417
-----------------------------------------------------------


On March 20, 2017, 4:03 a.m., Till Toenshoff wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57762/
> -----------------------------------------------------------
> 
> (Updated March 20, 2017, 4:03 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Bannier, Greg Mann, Jie Yu, and Joseph Wu.
> 
> 
> Bugs: MESOS-7263
>     https://issues.apache.org/jira/browse/MESOS-7263
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> see summary.
> 
> 
> Diffs
> -----
> 
>   src/launcher/executor.cpp bd3c0cf 
> 
> 
> Diff: https://reviews.apache.org/r/57762/diff/2/
> 
> 
> Testing
> -------
> 
> make check + functional testing..
> 
> Before applying the patch;
> 
> Running 
> ```
> $ ./src/mesos-execute --name="test" --env='{"key1":"value1"}' --command='sleep 1000'
--master=127.0.0.1:5050
> ```
> will result in the following `stdout` sandbox file:
> ```
> Received SUBSCRIBED event
> Subscribed executor on lobomacpro2.fritz.box
> Received LAUNCH event
> Starting task test
> /Users/till/Development/mesos-private/build/src/mesos-containerizer launch --help="false"
--launch_info="{"command":{"environment":{"variables":[{"name":"key1","type":"VALUE","value":"value1"}]},"shell":true,"value":"sleep
1000"},"environment":{"variables":[{"name":"BIN_SH","type":"VALUE","value":"xpg4"},{"name":"DUALCASE","type":"VALUE","value":"1"},{"name":"DYLD_LIBRARY_PATH","type":"VALUE","value":"\/Users\/till\/Development\/mesos-private\/build\/src\/.libs"},{"name":"LIBPROCESS_PORT","type":"VALUE","value":"0"},{"name":"MESOS_AGENT_ENDPOINT","type":"VALUE","value":"192.168.178.20:5051"},{"name":"MESOS_CHECKPOINT","type":"VALUE","value":"0"},{"name":"MESOS_DIRECTORY","type":"VALUE","value":"\/tmp\/mesos\/slaves\/816619b6-f5ce-42d6-ad6b-2ef2001adc0a-S0\/frameworks\/c84f9ee5-8a6b-4f6c-9fc0-154bceb3c615-0000\/executors\/test\/runs\/025c1d1a-6886-4b42-9313-0b1a411d5e39"},{"name":"MESOS_EXECUTOR_ID","type":"VALUE","value":"test"},{"name":"MESOS_EXECUTOR_SHUTDOWN_GRACE_PERIOD","type"
 :"VALUE","value":"5secs"},{"name":"MESOS_FRAMEWORK_ID","type":"VALUE","value":"c84f9ee5-8a6b-4f6c-9fc0-154bceb3c615-0000"},{"name":"MESOS_HTTP_COMMAND_EXECUTOR","type":"VALUE","value":"0"},{"name":"MESOS_SANDBOX","type":"VALUE","value":"\/tmp\/mesos\/slaves\/816619b6-f5ce-42d6-ad6b-2ef2001adc0a-S0\/frameworks\/c84f9ee5-8a6b-4f6c-9fc0-154bceb3c615-0000\/executors\/test\/runs\/025c1d1a-6886-4b42-9313-0b1a411d5e39"},{"name":"MESOS_SLAVE_ID","type":"VALUE","value":"816619b6-f5ce-42d6-ad6b-2ef2001adc0a-S0"},{"name":"MESOS_SLAVE_PID","type":"VALUE","value":"slave(1)@192.168.178.20:5051"},{"name":"PATH","type":"VALUE","value":"\/usr\/local\/sbin:\/usr\/local\/bin:\/usr\/sbin:\/usr\/bin:\/sbin:\/bin"},{"name":"PWD","type":"VALUE","value":"\/private\/tmp\/mesos\/slaves\/816619b6-f5ce-42d6-ad6b-2ef2001adc0a-S0\/frameworks\/c84f9ee5-8a6b-4f6c-9fc0-154bceb3c615-0000\/executors\/test\/runs\/025c1d1a-6886-4b42-9313-0b1a411d5e39"},{"name":"SHLVL","type":"VALUE","value":"0"},{"name":"__CF_USER_TEXT
 _ENCODING","type":"VALUE","value":"0x1F5:0x0:0x0"},{"name":"key1","type":"VALUE","value":"value1"},{"name":"key1","type":"VALUE","value":"value1"}]}}"
> Forked command at 2376
> Overwriting environment variable 'key1', original: 'value1', new: 'value1'
> ```
> 
> After applying the patch;
> 
> The resulting `stdout` sandbox file is as clean as it should be:
> ```
> Received SUBSCRIBED event
> Subscribed executor on lobomacpro2.fritz.box
> Received LAUNCH event
> Starting task test
> /Users/till/Development/mesos-private/build/src/mesos-containerizer launch --help="false"
--launch_info="{"command":{"environment":{"variables":[{"name":"key1","type":"VALUE","value":"value1"}]},"shell":true,"value":"sleep
1000"},"environment":{"variables":[{"name":"PATH","type":"VALUE","value":"\/usr\/local\/sbin:\/usr\/local\/bin:\/usr\/sbin:\/usr\/bin:\/sbin:\/bin"},{"name":"PWD","type":"VALUE","value":"\/private\/tmp\/mesos\/slaves\/816619b6-f5ce-42d6-ad6b-2ef2001adc0a-S0\/frameworks\/816619b6-f5ce-42d6-ad6b-2ef2001adc0a-0000\/executors\/test\/runs\/46f219a3-2772-4d21-89b8-85f1c1ba362e"},{"name":"MESOS_SLAVE_PID","type":"VALUE","value":"slave(1)@192.168.178.20:5051"},{"name":"MESOS_FRAMEWORK_ID","type":"VALUE","value":"816619b6-f5ce-42d6-ad6b-2ef2001adc0a-0000"},{"name":"MESOS_EXECUTOR_ID","type":"VALUE","value":"test"},{"name":"MESOS_CHECKPOINT","type":"VALUE","value":"0"},{"name":"MESOS_SLAVE_ID","type":"VALUE","value":"816619b6-f5ce-42d6-ad6b-2ef2001adc0a-S0"},{"name":"MESOS
 _SANDBOX","type":"VALUE","value":"\/tmp\/mesos\/slaves\/816619b6-f5ce-42d6-ad6b-2ef2001adc0a-S0\/frameworks\/816619b6-f5ce-42d6-ad6b-2ef2001adc0a-0000\/executors\/test\/runs\/46f219a3-2772-4d21-89b8-85f1c1ba362e"},{"name":"MESOS_EXECUTOR_SHUTDOWN_GRACE_PERIOD","type":"VALUE","value":"5secs"},{"name":"SHLVL","type":"VALUE","value":"0"},{"name":"DUALCASE","type":"VALUE","value":"1"},{"name":"key1","type":"VALUE","value":"value1"},{"name":"MESOS_AGENT_ENDPOINT","type":"VALUE","value":"192.168.178.20:5051"},{"name":"DYLD_LIBRARY_PATH","type":"VALUE","value":"\/Users\/till\/Development\/mesos-private\/build\/src\/.libs"},{"name":"BIN_SH","type":"VALUE","value":"xpg4"},{"name":"MESOS_HTTP_COMMAND_EXECUTOR","type":"VALUE","value":"0"},{"name":"__CF_USER_TEXT_ENCODING","type":"VALUE","value":"0x1F5:0x0:0x0"},{"name":"MESOS_DIRECTORY","type":"VALUE","value":"\/tmp\/mesos\/slaves\/816619b6-f5ce-42d6-ad6b-2ef2001adc0a-S0\/frameworks\/816619b6-f5ce-42d6-ad6b-2ef2001adc0a-0000\/executors\/test\/
 runs\/46f219a3-2772-4d21-89b8-85f1c1ba362e"},{"name":"LIBPROCESS_PORT","type":"VALUE","value":"0"}]}}"
> Forked command at 90766
> ```
> 
> 
> File Attachments
> ----------------
> 
> update57762.patch
>   https://reviews.apache.org/media/uploaded/files/2017/03/20/f1891b61-3845-4c64-a0f1-441323965e95__update57762.patch
> 
> 
> Thanks,
> 
> Till Toenshoff
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message