airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Olivier Girardot <o.girar...@lateral-thoughts.com>
Subject Re: [DISCUSS]: Remove Mesos Executor from Airflow 2.0.0?
Date Sat, 09 Feb 2019 13:58:51 GMT
Hi, we tried to use it for a new project 1year ago, but it was too broken
to  be usable, and we didn't have the time to make it work.
Of course we are still hoping to use it one day, but that doesn't mean it
should not move to contrib, it was already clear from the documentation
that it was not a first class citizen.

Regards,

Olivier.

Le dim. 27 janv. 2019 à 12:12, Ash Berlin-Taylor <ash@apache.org> a écrit :

> That PR pre-dates 1.9.0 being released.
>
> > On 27 Jan 2019, at 06:57, airflowuser <airflowuser@protonmail.com.INVALID>
> wrote:
> >
> > The Mesos executor is being used (at least by some people) and there is
> an open PR to extend it:
> > https://github.com/apache/airflow/pull/3739
> >
> >
> > Sent with ProtonMail Secure Email.
> >
> > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > On Friday, January 25, 2019 7:30 PM, Kaxil Naik <kaxilnaik@gmail.com>
> wrote:
> >
> >> What about Dask Executor as well.
> >>
> >> On Fri, Jan 25, 2019, 16:35 Daniel Imberman <daniel.imberman@gmail.com
> >> wrote:
> >>
> >>> @sid basically my response lol
> >>> I haven't seen any emails from any users and if it's been broken for
> two
> >>> versions then there doesn't seem to be enough interest to maintain it.
> >>> On Fri, Jan 25, 2019, 8:27 AM Sid Anand r39132@gmail.com wrote:
> >>>
> >>>> LOL :_0
> >>>> -s
> >>>> On Fri, Jan 25, 2019 at 8:25 AM Bolke de Bruin bdbruin@gmail.com
> >>>> wrote:
> >>>>
> >>>>> I agree. Maybe we should make a choice on what we consider 1st class
> >>>>> executors and move others to contrib (I'm looking at you celery)
> >>>>> Sent from my iPhone
> >>>>>
> >>>>>> On 25 Jan 2019, at 12:46, Ash Berlin-Taylor ash@apache.org wrote:
> >>>>>> Is anyone using the Mesos Executor? I think we should deprecate
and
> >>>>>> remove it.
> >>>>>> The reason I say that is I have a feeling it's been broken since
> >>>>>> 1.9.0
> >>>>
> >>>>> and no-one has noticed:
> >>>>>
> >>>>>> class MesosExecutor(BaseExecutor, LoginMixin):
> >>>>>> def start(self):
> >>>>>> # ...
> >>>>>> self.log.info(
> >>>>>> 'MesosFramework master : %s, name : %s, cpu : %s, mem :
> >>>>>> %s,
> >>>>
> >>>>> checkpoint : %s',
> >>>>>
> >>>>>>           master, framework.name,
> >>>>>>           str(task_cpu), str(task_memory), str(framework.checkpoint)
> >>>>>>       )
> >>>>>>
> >>>>>>
> >>>>>> That mixin should have been LoggingMixin, not LoginMixin. But
given
> >>>>>> this
> >>>>>> bug has been in since
> >>>
> >>>
> https://github.com/apache/airflow/commit/a7a518902dcf1e7fd4bf477cf57cee691f181b29
> >>>
> >>>>> which was in the 1.9.0 release I propose we simply remove this
> executor
> >>>>> from master.
> >>>>>
> >>>>>> Thoughts?
> >>>>>> Ash
> >
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message