flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: [DISCUSS] Create a Flink ecosystem website
Date Thu, 21 Mar 2019 09:08:59 GMT
Okay, great.

Congxian Qiu, Daryl and I have a kick-off call later today at 2pm CET, 9pm
China time about the design of the ecosystem page (see:
https://github.com/rmetzger/flink-community-tools/issues/4)
Please let me know if others want to join as well, I can add them to the
invite.

On Wed, Mar 20, 2019 at 4:10 AM Becket Qin <becket.qin@gmail.com> wrote:

> I agree. We can start with english-only and see how it goes. The comments
> and descriptions can always be multi-lingual but that is up to the package
> owners.
>
> On Tue, Mar 19, 2019 at 6:07 PM Robert Metzger <rmetzger@apache.org>
> wrote:
>
>> Thanks.
>>
>> Do we actually want this page to be multi-language?
>>
>> I propose to make the website english-only, but maybe consider allowing
>> comments in different languages.
>> If we would make it multi-language, then we might have problems with
>> people submitting packages in non-english languages.
>>
>>
>>
>> On Tue, Mar 19, 2019 at 2:42 AM Becket Qin <becket.qin@gmail.com> wrote:
>>
>>> Done. The writeup looks great!
>>>
>>> On Mon, Mar 18, 2019 at 9:09 PM Robert Metzger <rmetzger@apache.org>
>>> wrote:
>>>
>>>> Nice, really good news on the INFRA front!
>>>> I think the hardware specs sound reasonable. And a periodic backup of
>>>> the website's database to Infra's backup solution sounds reasonable too.
>>>>
>>>> Can you accept and review my proposal for the website?
>>>>
>>>>
>>>> On Sat, Mar 16, 2019 at 3:47 PM Becket Qin <becket.qin@gmail.com>
>>>> wrote:
>>>>
>>>>> >
>>>>> > I have a very capable and motivated frontend developer who would
be
>>>>> > willing to implement what I've mocked in my proposal.
>>>>>
>>>>>
>>>>> That is awesome!
>>>>>
>>>>> I created a Jira ticket[1] to Apache Infra and got the reply. It looks
>>>>> that
>>>>> Apache infra team could provide a decent VM. The last piece is how to
>>>>> ensure the data is persisted so we won't lose the project info / user
>>>>> feedbacks when the VM is down. If Apache infra does not provide a
>>>>> persistent storage for DB backup, we can always ask for multiple VMs
>>>>> and do
>>>>> the fault tolerance by ourselves. It seems we can almost say the
>>>>> hardware
>>>>> side is also ready.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Jiangjie (Becket) Qin
>>>>>
>>>>> [1] https://issues.apache.org/jira/browse/INFRA-18010
>>>>>
>>>>> On Fri, Mar 15, 2019 at 5:39 PM Robert Metzger <rmetzger@apache.org>
>>>>> wrote:
>>>>>
>>>>> > Thank you for reaching out to Infra and the ember client.
>>>>> > When I first saw the Ember repository, I thought it is the whole
>>>>> thing
>>>>> > (frontend and backend), but while testing it, I realized it is
>>>>> "only" the
>>>>> > frontend. I'm not sure if it makes sense to adjust the Ember observer
>>>>> > client, or just write a simple UI from scratch.
>>>>> > I have a very capable and motivated frontend developer who would
be
>>>>> > willing to implement what I've mocked in my proposal.
>>>>> > In addition, I found somebody (Congxian Qiu) who seems to be eager
>>>>> to help
>>>>> > with this project for the backend:
>>>>> > https://github.com/rmetzger/flink-community-tools/issues/4
>>>>> >
>>>>> > For Infra: I made the same experience when asking for more GitHub
>>>>> > permissions for "flinkbot": They didn't respond on their mailing
>>>>> list, only
>>>>> > on Jira.
>>>>> >
>>>>> >
>>>>> >
>>>>> > On Thu, Mar 14, 2019 at 2:45 PM Becket Qin <becket.qin@gmail.com>
>>>>> wrote:
>>>>> >
>>>>> >> Thanks for writing up the specifications.
>>>>> >>
>>>>> >> Regarding the website source code, Austin found a website[1]
whose
>>>>> >> frontend code[2] is available publicly. It lacks some support
(e.g
>>>>> login),
>>>>> >> but it is still a good starting point. One thing is that I did
not
>>>>> find a
>>>>> >> License statement for that source code. I'll reach out to the
>>>>> author to see
>>>>> >> if they have any concern over our usage.
>>>>> >>
>>>>> >> Apache Infra has not replied to my email regarding some details
>>>>> about the
>>>>> >> VM. I'll open an infra Jira ticket tomorrow if there is still
no
>>>>> response.
>>>>> >>
>>>>> >> Thanks,
>>>>> >>
>>>>> >> Jiangjie (Becket) Qin
>>>>> >>
>>>>> >> [1] https://emberobserver.com/
>>>>> >> [2] https://github.com/emberobserver/client
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> On Thu, Mar 14, 2019 at 1:35 AM Robert Metzger <rmetzger@apache.org
>>>>> >
>>>>> >> wrote:
>>>>> >>
>>>>> >>> @Bowen: I agree. Confluent Hub looks nicer, but it is on
their
>>>>> company
>>>>> >>> website. I guess the likelihood that they give out code
from their
>>>>> company
>>>>> >>> website is fairly low.
>>>>> >>> @Nils: Beam's page is similar to our Ecosystem page, which
we'll
>>>>> >>> reactivate as part of this PR:
>>>>> >>> https://github.com/apache/flink-web/pull/187
>>>>> >>>
>>>>> >>> Spark-packages.org did not respond to my request.
>>>>> >>> I will propose a short specification in Becket's initial
document.
>>>>> >>>
>>>>> >>>
>>>>> >>> On Mon, Mar 11, 2019 at 11:38 AM Niels Basjes <Niels@basjes.nl>
>>>>> wrote:
>>>>> >>>
>>>>> >>>> Hi,
>>>>> >>>>
>>>>> >>>> The Beam project has something in this area that is
simply a page
>>>>> >>>> within their documentation website:
>>>>> >>>> https://beam.apache.org/documentation/sdks/java-thirdparty/
>>>>> >>>>
>>>>> >>>> Niels Basjes
>>>>> >>>>
>>>>> >>>> On Fri, Mar 8, 2019 at 11:39 PM Bowen Li <bowenli86@gmail.com>
>>>>> wrote:
>>>>> >>>> >
>>>>> >>>> > Confluent hub for Kafka is another good example
of this kind. I
>>>>> >>>> personally like it over the spark site. May worth checking
it out
>>>>> with
>>>>> >>>> Kafka folks
>>>>> >>>> >
>>>>> >>>> > On Thu, Mar 7, 2019 at 6:06 AM Becket Qin <becket.qin@gmail.com
>>>>> >
>>>>> >>>> wrote:
>>>>> >>>> >>
>>>>> >>>> >> Absolutely! Thanks for the pointer. I'll submit
a PR to update
>>>>> the
>>>>> >>>> >> ecosystem page and the navigation.
>>>>> >>>> >>
>>>>> >>>> >> Thanks,
>>>>> >>>> >>
>>>>> >>>> >> Jiangjie (Becket) Qin
>>>>> >>>> >>
>>>>> >>>> >> On Thu, Mar 7, 2019 at 8:47 PM Robert Metzger
<
>>>>> rmetzger@apache.org>
>>>>> >>>> wrote:
>>>>> >>>> >>
>>>>> >>>> >> > Okay. I will reach out to spark-packages.org
and see if
>>>>> they are
>>>>> >>>> willing
>>>>> >>>> >> > to share.
>>>>> >>>> >> >
>>>>> >>>> >> > Do you want to raise a PR to update the
ecosystem page
>>>>> (maybe sync
>>>>> >>>> with
>>>>> >>>> >> > the "Software Projects" listed here:
>>>>> >>>> >> >
>>>>> https://cwiki.apache.org/confluence/display/FLINK/Powered+by+Flink)
>>>>> >>>> and
>>>>> >>>> >> > link it in the navigation?
>>>>> >>>> >> >
>>>>> >>>> >> > Best,
>>>>> >>>> >> > Robert
>>>>> >>>> >> >
>>>>> >>>> >> >
>>>>> >>>> >> > On Thu, Mar 7, 2019 at 10:13 AM Becket
Qin <
>>>>> becket.qin@gmail.com>
>>>>> >>>> wrote:
>>>>> >>>> >> >
>>>>> >>>> >> >> Hi Robert,
>>>>> >>>> >> >>
>>>>> >>>> >> >> I think it at least worths checking
if spark-packages.org
>>>>> owners
>>>>> >>>> are
>>>>> >>>> >> >> willing to share. Thanks for volunteering
to write the
>>>>> requirement
>>>>> >>>> >> >> descriptions! In any case, that will
be very helpful.
>>>>> >>>> >> >>
>>>>> >>>> >> >> Since a static page has almost no
cost, and we will need it
>>>>> to
>>>>> >>>> redirect
>>>>> >>>> >> >> to the dynamic site anyways, how about
we first do that
>>>>> while
>>>>> >>>> working on
>>>>> >>>> >> >> the dynamic website?
>>>>> >>>> >> >>
>>>>> >>>> >> >> Thanks,
>>>>> >>>> >> >>
>>>>> >>>> >> >> Jiangjie (Becket) Qin
>>>>> >>>> >> >>
>>>>> >>>> >> >> On Thu, Mar 7, 2019 at 4:59 AM Ufuk
Celebi <uce@apache.org>
>>>>> >>>> wrote:
>>>>> >>>> >> >>
>>>>> >>>> >> >>> I like Shaoxuan's idea to keep
this a static site first. We
>>>>> >>>> could then
>>>>> >>>> >> >>> iterate on this and make it a
dynamic thing. Of course, if
>>>>> we
>>>>> >>>> have the
>>>>> >>>> >> >>> resources in the community to
quickly start with a dynamic
>>>>> site,
>>>>> >>>> I'm
>>>>> >>>> >> >>> not apposed.
>>>>> >>>> >> >>>
>>>>> >>>> >> >>> – Ufuk
>>>>> >>>> >> >>>
>>>>> >>>> >> >>> On Wed, Mar 6, 2019 at 2:31 PM
Robert Metzger <
>>>>> >>>> rmetzger@apache.org>
>>>>> >>>> >> >>> wrote:
>>>>> >>>> >> >>> >
>>>>> >>>> >> >>> > Awesome! Thanks a lot for
looking into this Becket! The
>>>>> VMs
>>>>> >>>> hosted by
>>>>> >>>> >> >>> Infra
>>>>> >>>> >> >>> > look suitable.
>>>>> >>>> >> >>> >
>>>>> >>>> >> >>> > @Shaoxuan: There is actually
already a static page. It
>>>>> used to
>>>>> >>>> be
>>>>> >>>> >> >>> linked,
>>>>> >>>> >> >>> > but has been removed from
the navigation bar for some
>>>>> reason.
>>>>> >>>> This is
>>>>> >>>> >> >>> the
>>>>> >>>> >> >>> > page: https://flink.apache.org/ecosystem.html
>>>>> >>>> >> >>> > We could update the page
and add it back to the
>>>>> navigation bar
>>>>> >>>> for the
>>>>> >>>> >> >>> > coming weeks. What do you
think?
>>>>> >>>> >> >>> >
>>>>> >>>> >> >>> > I would actually like to
push for a dynamic page right
>>>>> away.
>>>>> >>>> >> >>> >
>>>>> >>>> >> >>> > I know it's kind of a bold
move, but how do you feel
>>>>> about
>>>>> >>>> sending the
>>>>> >>>> >> >>> > owners of spark-packages.org
a short note, if they are
>>>>> >>>> interested in
>>>>> >>>> >> >>> > sharing the source? We could
maintain the code together
>>>>> in a
>>>>> >>>> public
>>>>> >>>> >> >>> repo.
>>>>> >>>> >> >>> > If they are not interested
in sharing, or we decide not
>>>>> to ask
>>>>> >>>> in the
>>>>> >>>> >> >>> first
>>>>> >>>> >> >>> > place, I'm happy to write
down a short description of the
>>>>> >>>> requirements,
>>>>> >>>> >> >>> > maybe some mockups. We could
then see if we find
>>>>> somebody here
>>>>> >>>> in the
>>>>> >>>> >> >>> > community who's willing to
implement it.
>>>>> >>>> >> >>> > Given the number of people
who are eager to contribute, I
>>>>> >>>> believe we
>>>>> >>>> >> >>> will
>>>>> >>>> >> >>> > be able to find somebody
pretty soon.
>>>>> >>>> >> >>> >
>>>>> >>>> >> >>> >
>>>>> >>>> >> >>> > On Wed, Mar 6, 2019 at 3:49
AM Becket Qin <
>>>>> >>>> becket.qin@gmail.com>
>>>>> >>>> >> >>> wrote:
>>>>> >>>> >> >>> >
>>>>> >>>> >> >>> > > Forgot to provide the
link...
>>>>> >>>> >> >>> > >
>>>>> >>>> >> >>> > > [1] https://www.apache.org/dev/services.html#blogs
>>>>> (Apache
>>>>> >>>> infra
>>>>> >>>> >> >>> services)
>>>>> >>>> >> >>> > > [2] https://www.apache.org/dev/freebsd-jails
(FreeBSD
>>>>> Jail
>>>>> >>>> provided
>>>>> >>>> >> >>> by
>>>>> >>>> >> >>> > > Apache Infra)
>>>>> >>>> >> >>> > >
>>>>> >>>> >> >>> > > On Wed, Mar 6, 2019
at 10:46 AM Becket Qin <
>>>>> >>>> becket.qin@gmail.com>
>>>>> >>>> >> >>> wrote:
>>>>> >>>> >> >>> > >
>>>>> >>>> >> >>> > >> Hi Robert,
>>>>> >>>> >> >>> > >>
>>>>> >>>> >> >>> > >> Thanks for the feedback.
These are good points. We
>>>>> should
>>>>> >>>> absolutely
>>>>> >>>> >> >>> > >> shoot for a dynamic
website to support more
>>>>> interactions in
>>>>> >>>> the
>>>>> >>>> >> >>> community.
>>>>> >>>> >> >>> > >> There might be a
few things to solve:
>>>>> >>>> >> >>> > >> 1. The website code
itself. An open source solution
>>>>> would
>>>>> >>>> be great.
>>>>> >>>> >> >>> TBH,
>>>>> >>>> >> >>> > >> I do not have much
experience on building a website.
>>>>> It'll
>>>>> >>>> be great
>>>>> >>>> >> >>> if
>>>>> >>>> >> >>> > >> someone could help
comment on the solution here.
>>>>> >>>> >> >>> > >> 2. The hardware
to host the website. Apache Infra
>>>>> provides
>>>>> >>>> a few
>>>>> >>>> >> >>> > >> services[1] that
Apache projects can leverage. I did
>>>>> not see
>>>>> >>>> >> >>> database
>>>>> >>>> >> >>> > >> service, but maybe
we can run a simple MySQL db in
>>>>> FreeBSD
>>>>> >>>> jail[2].
>>>>> >>>> >> >>> > >>
>>>>> >>>> >> >>> > >> @Bowen & vino,
thanks for the positive feedback!
>>>>> >>>> >> >>> > >>
>>>>> >>>> >> >>> > >> @Shaoxuan Wang <wshaoxuan@gmail.com>
>>>>> >>>> >> >>> > >> Thanks for the suggestion.
That sounds reasonable to
>>>>> me. We
>>>>> >>>> >> >>> probably need
>>>>> >>>> >> >>> > >> a page in the Flink
official site anyways, even just
>>>>> >>>> provide links
>>>>> >>>> >> >>> it to
>>>>> >>>> >> >>> > >> the ecosystem website.
So listing the connectors in
>>>>> that
>>>>> >>>> static
>>>>> >>>> >> >>> page seems
>>>>> >>>> >> >>> > >> something we could
start with while we are working on
>>>>> the
>>>>> >>>> dynamic
>>>>> >>>> >> >>> pages.
>>>>> >>>> >> >>> > >>
>>>>> >>>> >> >>> > >> Thanks,
>>>>> >>>> >> >>> > >>
>>>>> >>>> >> >>> > >> Jiangjie (Becket)
Qin
>>>>> >>>> >> >>> > >>
>>>>> >>>> >> >>> > >> On Wed, Mar 6, 2019
at 10:40 AM Shaoxuan Wang <
>>>>> >>>> wshaoxuan@gmail.com>
>>>>> >>>> >> >>> > >> wrote:
>>>>> >>>> >> >>> > >>
>>>>> >>>> >> >>> > >>> Hi Becket and
Robert,
>>>>> >>>> >> >>> > >>>
>>>>> >>>> >> >>> > >>> I like this
idea!  Let us roll this out with Flink
>>>>> >>>> connectors at
>>>>> >>>> >> >>> the
>>>>> >>>> >> >>> > >>> first beginning.
We can start with a static page, and
>>>>> >>>> upgrade it
>>>>> >>>> >> >>> when we
>>>>> >>>> >> >>> > >>> find a better
solution for dynamic one with rich
>>>>> functions.
>>>>> >>>> >> >>> > >>>
>>>>> >>>> >> >>> > >>> Regards,
>>>>> >>>> >> >>> > >>> Shaoxuan
>>>>> >>>> >> >>> > >>>
>>>>> >>>> >> >>> > >>>
>>>>> >>>> >> >>> > >>> On Wed, Mar
6, 2019 at 1:36 AM Robert Metzger <
>>>>> >>>> rmetzger@apache.org
>>>>> >>>> >> >>> >
>>>>> >>>> >> >>> > >>> wrote:
>>>>> >>>> >> >>> > >>>
>>>>> >>>> >> >>> > >>>> Hey Becket,
>>>>> >>>> >> >>> > >>>>
>>>>> >>>> >> >>> > >>>> This is
a great idea!
>>>>> >>>> >> >>> > >>>> For this
to be successful, we need to make sure the
>>>>> page
>>>>> >>>> is placed
>>>>> >>>> >> >>> > >>>> prominently
so that the people submitting something
>>>>> will
>>>>> >>>> get
>>>>> >>>> >> >>> attention for
>>>>> >>>> >> >>> > >>>> their contributions.
>>>>> >>>> >> >>> > >>>> I think
a dynamic site would probably be better, if
>>>>> we
>>>>> >>>> want
>>>>> >>>> >> >>> features
>>>>> >>>> >> >>> > >>>> such as
up and downvoting or comments.
>>>>> >>>> >> >>> > >>>> I would
also like this to be hosted on Apache
>>>>> infra, and
>>>>> >>>> endorsed
>>>>> >>>> >> >>> by
>>>>> >>>> >> >>> > >>>> the community.
>>>>> >>>> >> >>> > >>>>
>>>>> >>>> >> >>> > >>>> Does anybody
here know any existing software that we
>>>>> >>>> could use?
>>>>> >>>> >> >>> > >>>> The only
think I was able to find is AUR:
>>>>> >>>> >> >>> https://aur.archlinux.org/
>>>>> >>>> >> >>> > >>>> (which is
a community packages site for Arch Linux.
>>>>> The
>>>>> >>>> source
>>>>> >>>> >> >>> code of this
>>>>> >>>> >> >>> > >>>> portal is
open source, but the layout and structure
>>>>> is
>>>>> >>>> not an
>>>>> >>>> >> >>> ideal fit for
>>>>> >>>> >> >>> > >>>> our requirements)
>>>>> >>>> >> >>> > >>>>
>>>>> >>>> >> >>> > >>>> Best,
>>>>> >>>> >> >>> > >>>> Robert
>>>>> >>>> >> >>> > >>>>
>>>>> >>>> >> >>> > >>>>
>>>>> >>>> >> >>> > >>>>
>>>>> >>>> >> >>> > >>>> On Tue,
Mar 5, 2019 at 12:03 PM Becket Qin <
>>>>> >>>> becket.qin@gmail.com>
>>>>> >>>> >> >>> > >>>> wrote:
>>>>> >>>> >> >>> > >>>>
>>>>> >>>> >> >>> > >>>>> Hi folks,
>>>>> >>>> >> >>> > >>>>>
>>>>> >>>> >> >>> > >>>>> I would
like to start a discussion thread about
>>>>> creating
>>>>> >>>> a Flink
>>>>> >>>> >> >>> > >>>>> ecosystem
website. The website aims to help
>>>>> contributors
>>>>> >>>> who
>>>>> >>>> >> >>> have developed
>>>>> >>>> >> >>> > >>>>> projects
around Flink share their work with the
>>>>> >>>> community.
>>>>> >>>> >> >>> > >>>>>
>>>>> >>>> >> >>> > >>>>> Please
see the following doc for more details.
>>>>> >>>> >> >>> > >>>>>
>>>>> >>>> >> >>> > >>>>>
>>>>> >>>> >> >>>
>>>>> >>>>
>>>>> https://docs.google.com/document/d/12oCItoLbKrLGuwEUFcCfigezIR2hW3925j1hh3kGp4A/edit#
>>>>> >>>> >> >>> > >>>>>
>>>>> >>>> >> >>> > >>>>> Thanks,
>>>>> >>>> >> >>> > >>>>>
>>>>> >>>> >> >>> > >>>>> Jiangjie
(Becket) Qin
>>>>> >>>> >> >>> > >>>>>
>>>>> >>>> >> >>> > >>>>
>>>>> >>>> >> >>>
>>>>> >>>> >> >>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> --
>>>>> >>>> Best regards / Met vriendelijke groeten,
>>>>> >>>>
>>>>> >>>> Niels Basjes
>>>>> >>>>
>>>>> >>>
>>>>>
>>>>

Mime
View raw message