druid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benedict Jin <asdf2...@apache.org>
Subject Re: About maintaining the Helm's Chart of Apache Druid
Date Fri, 02 Jul 2021 08:19:07 GMT
Hi Xavier,

I'm so happy to hear that and look forward to your changes will be contributed to upstream.
In fact, Helm and Operator are not in conflict, their relationship is kind like RPM and Systemd.
You can even convert Helm into Operator, or build Operator based on Helm. And I agree with
you that it would be better if we can define user scenarios.

Regards,
Benedict Jin

On 2021/06/25 22:42:32, Xavier Léauté <xavier@confluent.io.INVALID> wrote: 
> For what it's worth, we have been using a heavily modified version of this
> helm chart at Confluent.
> 
> I would say it is good to get a Druid cluster up and running quickly, but
> we had to make some significant changes to make it easier to operate a
> Druid cluster.
> It's great for initial deployment and getting all the required dependencies
> in place, but operations are somewhat painful and require a lot of internal
> Druid knowledge to not shoot yourself in the foot.
> Our original intention was to contribute back those changes upstream, but
> we have not had the time to put it in a shape that would allow others to
> use it.
> 
> We should try to define what we want this chart to be used for, since I
> think the Druid k8s operator is probably a better choice for someone to run
> and upgrade a meaningful cluster.
> Another option would be to focus our effort on the Druid operator and maybe
> build a helm chart to get that and our external dependencies in place, I
> think we can provide a better experience that way.
> One concern with the pure helm chart is that we'll get a lot of questions
> on how to operate it that will likely take a lot of time to answer.
> Considering we'd have helm, k8s operator, and docker-compose, I think we
> should be conscious of the time it would take to maintain all those ways of
> running Druid in containers and what purpose each of them serves.
> 
> Just my 2¢,
> Xavier
> 
> On Tue, Jun 22, 2021 at 8:04 AM Benedict Jin <asdf2014@apache.org> wrote:
> 
> > Hi Jihoon Son,
> >
> > Cool, thanks a lot 👍👍👍
> >
> > Regards,
> > Benedict Jin
> >
> > On 2021/06/21 17:10:13, Jihoon Son <jihoonson@apache.org> wrote:
> > > Thanks Benedict.
> > > You can find another example of the IP clearance process here:
> > >
> > https://mail-archives.apache.org/mod_mbox/incubator-general/202106.mbox/browser
> > .
> > >
> > > On Fri, May 28, 2021 at 1:01 AM Benedict Jin <asdf2014@apache.org>
> > wrote:
> > > >
> > > > Hi Jihoon Son,
> > > >
> > > > Yes, it has only been tested on the local cluster. Next, I will add
> > the automated test part.
> > > >
> > > > Because Helm Chart usually releases a new version every time it is
> > modified. So Apache Superset has followed this approach. Of course, I agree
> > with your suggestion, we shouldn't release the version of Chart separately,
> > and follow the release rhythm of Apache Druid.
> > > >
> > > > I've been busy recently, but I also took time to study this IP
> > licensing process, and I'm filling out related forms. Thank you very much
> > for the mailing list, it's very useful to me.
> > > >
> > > > Okay, I got it. We will solve this problem as soon as possible before
> > the release of 0.22.0.
> > > >
> > > > Regards,
> > > > Benedict Jin
> > > >
> > > > On 2021/05/22 18:53:35, Jihoon Son <jihoonson@apache.org> wrote:
> > > > > Thanks for adding details.
> > > > >
> > > > > Based on your answers, I assume it is not being tested. We should
add
> > > > > tests as soon as possible. Since you have some experience in this
> > > > > area, can you add some?
> > > > > I'm also not sure why Superset releases the helm chart separately.
I
> > > > > would suggest releasing it per our regular release schedule unless
> > > > > there is a good reason for doing so. This will reduce the release
> > > > > burden of the community.
> > > > >
> > > > > So far, it seems reasonable to me to host the helm chart in the Druid
> > > > > repo. However, as I mentioned before, the migration process might
not
> > > > > be proper. I haven't had a chance to look at the IP clearance process
> > > > > closely yet and probably will not have some even in the near future.
> > > > > Benedict, you are a PMC member too. Can you please study the process
> > > > > and give us suggestions on what we should do? Since not many people
> > in
> > > > > the Druid community might not be familiar with this process, you
may
> > > > > want to ask questions in the Apache general mailing list. See
> > > > >
> > https://www.mail-archive.com/general@incubator.apache.org/msg74849.html
> > > > > as a reference.
> > > > >
> > > > > We should resolve this issue before the 0.22.0 release. Otherwise,
we
> > > > > will have to revert all changes related to the helm chart in the
> > > > > release branch because it doesn't seem necessarily a release blocker
> > > > > for 0.22.0.
> > > > >
> > > > >
> > > > > On Sun, May 2, 2021 at 8:15 PM Benedict Jin <asdf2014@apache.org>
> > wrote:
> > > > > >
> > > > > > Hi Jihoon Son,
> > > > > >
> > > > > > Thank you very much for this list of questions. The following
is
> > my personal understanding.
> > > > > >
> > > > > > 1. What is the current status of the project?
> > > > > >
> > > > > > Helm has undergone a major version upgrade. Helm3 will continue
to
> > be maintained, but Helm2 will no longer be maintained, and the Helm Chart
> > related to the Apache project is no longer maintained in Helm3. It is
> > recommended to maintain it in their respective Apache projects.
> > > > > >
> > > > > > 2. Does it reflect the most recent release of Druid?
> > > > > >
> > > > > > Will not affect the release of Druid. Druid's Helm Chart is
> > released separately. FYI, https://github.com/apache/superset/releases
> > > > > >
> > > > > > 3. How is it being tested?
> > > > > >
> > > > > > It can be verified and tested automatically through Github Action.
> > FYI,
> > https://github.com/apache/superset/blob/master/.github/workflows/superset-helm-lint.yml
> > > > > >
> > > > > > 4. Why is it best to host the helm chart in the druid repo?
> > > > > >
> > > > > > Because Helm3 no longer maintains the Helm Chart of Apache-related
> > projects, the official also recommends maintaining it in their respective
> > projects. Many projects have already done so. The maintenance in the main
> > Druid warehouse can be seen by more people and help more people quickly
> > build the Druid environment on K8S.
> > > > > >
> > > > > > Hope these answers can answer some of your doubts. Thanks again.
> > > > > >
> > > > > > Regards,
> > > > > > Benedict Jin
> > > > > >
> > > > > > On 2021/04/27 03:17:29, Julian Hyde <jhyde.apache@gmail.com>
> > wrote:
> > > > > > > This code was developed outside of the ASF, so it’s possible
> > that we need to go through the IP clearance process [1]. Can a PMC member
> > please figure out the answer to that question, and answer on this list.
> > > > > > >
> > > > > > > Has the Helm project given any indication whether they
approve
> > or disapprove of the code being copied into Druid?
> > > > > > >
> > > > > > > Does Druid intend to take ownership of the code? I.e. be
the one
> > and only copy of this code, do necessary maintenance work (especially
> > including security fixes) and accepting patches.
> > > > > > >
> > > > > > > Julian
> > > > > > >
> > > > > > > [1] https://incubator.apache.org/ip-clearance/
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > On Apr 26, 2021, at 7:33 PM, Benedict Jin <asdf2014@apache.org>
> > wrote:
> > > > > > > >
> > > > > > > > Hi Senlan,
> > > > > > > >
> > > > > > > > Thank you very much for your message and support.
I have
> > created a PR to do the migration, referring to the experience of Apache
> > Superset. FYI, https://github.com/apache/druid/pull/11163 and
> > https://github.com/apache/superset/tree/master/helm/superset .
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > > Benedict Jin
> > > > > > > >
> > > > > > > > On 2021/04/26 12:59:12, Senlan Yao <hellisorz@gmail.com>
> > wrote:
> > > > > > > >> Thanks @Benedict Jin,
> > > > > > > >> It is a good idea to maintain the Druid Helm's
Chart in the
> > Apache repository.
> > > > > > > >> Since "https://github.com/helm/charts" repo has
been
> > deprecation, we can't maintain the chart, and the k8s user can't find druid
> > chart package from "
> > https://artifacthub.io/packages/search?page=1&ts_query_web=druid" any
> > more.
> > > > > > > >> If we can maintain chart in the Apache repository,
it will
> > solve https://github.com/apache/druid/issues/5582, ans also the the k8s
> > user can install Druid from helm chart package.
> > > > > > > >>
> > > > > > > >> On 2021/04/22 01:58:46, Benedict Jin <asdf2014@apache.org>
> > wrote:
> > > > > > > >>> Hi all,
> > > > > > > >>>
> > > > > > > >>> Should we maintain the Helm's Chart in the
Apache Druid
> > repository? Now the development and maintenance of Helm's Chart has been
> > stalled. It was previously maintained by @maver1ck @AWaterColorPen and me
> > (@asdf2014). Currently,
> > https://github.com/helm/charts/tree/master/incubator/druid cannot be
> > maintained. I recommend that we just copy this directory directly to the
> > root directory of https://github.com/apache/druid and maintain it. But
> > I'm not so sure whether there is a license issue. What do you think?
> > > > > > > >>>
> > > > > > > >>> Regards,
> > > > > > > >>> Benedict Jin
> > > > > > > >>>
> > > > > > > >>>
> > ---------------------------------------------------------------------
> > > > > > > >>> To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
> > > > > > > >>> For additional commands, e-mail: dev-help@druid.apache.org
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>
> > > > > > > >>
> > ---------------------------------------------------------------------
> > > > > > > >> To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
> > > > > > > >> For additional commands, e-mail: dev-help@druid.apache.org
> > > > > > > >>
> > > > > > > >>
> > > > > > > >
> > > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
> > > > > > > > For additional commands, e-mail: dev-help@druid.apache.org
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
> > > > > > > For additional commands, e-mail: dev-help@druid.apache.org
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
> > > > > > For additional commands, e-mail: dev-help@druid.apache.org
> > > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
> > > > > For additional commands, e-mail: dev-help@druid.apache.org
> > > > >
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
> > > > For additional commands, e-mail: dev-help@druid.apache.org
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
> > > For additional commands, e-mail: dev-help@druid.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
> > For additional commands, e-mail: dev-help@druid.apache.org
> >
> >
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
For additional commands, e-mail: dev-help@druid.apache.org


Mime
View raw message