ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saikat Maitra <saikat.mai...@gmail.com>
Subject Re: [DISCUSS] Pub/Sub Streamer Implementation
Date Sun, 10 Nov 2019 23:00:12 GMT
Hi Emmanouil,

The master branch has been initialized. I have raised a PR to migrate Flink
module to ignite-extensions.

PR https://github.com/apache/ignite-extensions/pull/1
Jira : https://issues.apache.org/jira/browse/IGNITE-12356

Regards,
Saikat


On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <gkatzioura@gmail.com>
wrote:

> Hi all!
>
> This is an update. The integration is tested with mocks based on the
> original Ignite repo [1].
> I am currently waiting for the first commits on the new integration project
> to take place in order to create my pull request accordingly to the initial
> commits.
> In order to simulate the Pub/Sub server in a test I used a test server
> implementation [2]. Unfortunately they don't have this util uploaded in a
> maven repo, thus it needs to be hosted on a maven repo (Apache License
> 2.0). Is there a repo for utilities that Ignite uses for the test/build?
> I have also created a ticket [3]
>
> Kind regards
>
> [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> [3] https://issues.apache.org/jira/browse/IGNITE-12262
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <saikat.maitra@gmail.com>
> wrote:
>
> > Hello Ilya,
> >
> > Thank you for your email. Yes, before we remove any integration we will
> > move them to another repository and release.
> >
> > We are thinking we will start with Pub/Sub Streamer implementation and
> > continue to move other integration as mentioned in the following doc.
> >
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >
> > Regards,
> > Saikat
> >
> >
> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> > ilya.kasnacheev@gmail.com>
> > wrote:
> >
> > > Hello!
> > >
> > > My take on this: we want to offload some of integrations, but please
> > commit
> > > and release them to another project first, and only then we will decide
> > to
> > > remove them from our repository.
> > >
> > > Please also make sure they are available with same artifact name and
> API.
> > > If this has to be changed, let's wait for AI 3.0
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <saikat.maitra@gmail.com>:
> > >
> > > > Hello Emmanouil,
> > > >
> > > > As discussed earlier, I discussed with Apache Bahir community and
> they
> > > are
> > > > interested to have Apache Ignite extensions as part of Apache Bahir
> > > > project.
> > > >
> > > > Can you please share Pub/Sub streamer implementation details with
> > Apache
> > > > Bahir community and request for feedback.
> > > >
> > > > https://bahir.apache.org/
> > > >
> > > > I have also requested for contributor access in Jira for Apache Bahir
> > > > project so that I can create issues and assign to myself.
> > > >
> > > > I can help with code reviews as well.
> > > >
> > > > Here is the email thread for reference
> > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > >
> > > > Regards,
> > > > Saikat
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
> saikat.maitra@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > As discussed I have created following discussion threads on our
> > > migration
> > > > > proposals for Apache Ignite extensions:
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > > > >
> > > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > > >
> > > > > I will share update as I hear more information.
> > > > >
> > > > > Regards,
> > > > > Saikat
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
> > saikat.maitra@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > >> Hello Denis,
> > > > >>
> > > > >> Sure, sounds good. I will create a separate discussion thread
to
> get
> > > > >> community feedback on whether we
> > > > >> should join the Bahir or kick-off "Ignite Extensions" project.
> > > > >>
> > > > >> Regards,
> > > > >> Saikat
> > > > >>
> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <dmagda@apache.org>
> > > wrote:
> > > > >>
> > > > >>> Folks,
> > > > >>>
> > > > >>> The concept of Apache Bahir is precisely what we need! Saikat,
> > thanks
> > > > for
> > > > >>> doing the research. Why I believe the * idea* fits us well:
> > > > >>>
> > > > >>>    - All integrations can be stored in separate Github
> repositories
> > > and
> > > > >>>    have their dev lifecycles. We've not obliged to couple
all the
> > > > >>> integrations
> > > > >>>    in a single repo. For instance, Spark can be located in
a
> > > dedicated
> > > > >>> repo
> > > > >>>    while streaming integrations might be in a single one.
It's up
> > to
> > > > us.
> > > > >>>    - All the repositories and integrations belong to ASF.
We're
> not
> > > > >>> dumping
> > > > >>>    them on Github but rather keep supporting and developing
in
> > > > >>> accordance with
> > > > >>>    ASF vision and practices.
> > > > >>>    - There is a way to reward contributors via committership
and
> > PMC
> > > > >>>    membership. You won't get this if a project is just one
of the
> > > > >>> millions of
> > > > >>>    Github projects.
> > > > >>>
> > > > >>> Saikat, as Ignite PMC member, could you please kick-off a
> separate
> > > > >>> dev-list
> > > > >>> discussion to involve more community members and referring
to
> this
> > > > >>> thread?
> > > > >>> I think the primary question the community needs to answer
> whether
> > we
> > > > >>> should join the Bahir or kick-off "Ignite Extensions" project?
> > > > >>>
> > > > >>> -
> > > > >>> Denis
> > > > >>>
> > > > >>>
> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
> > > > zaleslaw.sin@gmail.com>
> > > > >>> wrote:
> > > > >>>
> > > > >>> > Maybe we could move all our Streaming Integrations there,
but
> > what
> > > is
> > > > >>> about
> > > > >>> > maintaining and committer permissions to the new repositories?
> > > > >>> > I see the list of the committers and PMC members there
> > > > >>> > https://bahir.apache.org/community-members/
> > > > >>> >
> > > > >>> > Could somebody from Ignite community be added to this
list as a
> > > > >>> > PMC/committer to maintain Ignite integrations?
> > > > >>> > If yes, I'd happy to join this project and collaborate
with
> these
> > > > guys
> > > > >>> > together
> > > > >>> > If no, and we should start from the zero level with
external
> PRs
> > -
> > > > >>> hmmm,
> > > > >>> > it's better to have our own external repository with
> ApacheBahirr
> > > > >>> ideology.
> > > > >>> >
> > > > >>> > Also, I agree, that all connectors could be moved there
(in
> > > > >>> ApacheBahirr
> > > > >>> > like repository), but not all modules from the page
> > > > >>> >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > >>> > because
> > > > >>> > they use different parts of core modules and could be
developed
> > > with
> > > > >>> > different velocity.
> > > > >>> >
> > > > >>> > In reality, before creation of new repositories we need
wide
> > > > discussion
> > > > >>> > based on the document mentioned above.
> > > > >>> >
> > > > >>> > P.S. Of course, we could experiment in the next release
2.8
> with
> > > one
> > > > >>> or two
> > > > >>> > integrations like twitter or ZeroMQ
> > > > >>> > Also, I'd like Denis Magda idea for the separate repositories
> in
> > > > Apache
> > > > >>> > Github to maintain them by anybody.
> > > > >>> >
> > > > >>> >
> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
> > > saikat.maitra@gmail.com
> > > > >:
> > > > >>> >
> > > > >>> > > Hi Denis, Emmanouil
> > > > >>> > >
> > > > >>> > > Thank you for your email. I think creating a separate
> > integration
> > > > >>> project
> > > > >>> > > in github and also proposing it as an apache incubator
> project
> > > will
> > > > >>> be a
> > > > >>> > > good move. The other separate integration modules
> > > > >>> > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > >>> > > can
> > > > >>> > > be moved to this new apache incubator project.
> > > > >>> > >
> > > > >>> > > There are similar integration available for Flink
and Spark
> in
> > > > Apache
> > > > >>> > Bahir
> > > > >>> > > https://bahir.apache.org/
> > > > >>> > >
> > > > >>> > > Do you think we should reach out to Apache Bahir
community
> > with a
> > > > >>> > proposal
> > > > >>> > > or we should plan to create a separate Apache Incubator
> > project?
> > > > >>> > >
> > > > >>> > > Regards,
> > > > >>> > > Saikat
> > > > >>> > >
> > > > >>> > >
> > > > >>> > >
> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil Gkatziouras
<
> > > > >>> > > gkatzioura@gmail.com>
> > > > >>> > > wrote:
> > > > >>> > >
> > > > >>> > > > Hi all!
> > > > >>> > > >
> > > > >>> > > > Based on the discussions most probably this
is going to be
> on
> > > > >>> another
> > > > >>> > > > GitHub repo. Therefore I will prepare a standalone
project
> > with
> > > > the
> > > > >>> > > Pub/Sub
> > > > >>> > > > module and once the repository is created
a pull request
> > shall
> > > be
> > > > >>> > issued.
> > > > >>> > > > If there is anything else I can do in order
to facilitate
> > this
> > > > >>> please
> > > > >>> > let
> > > > >>> > > > me know.
> > > > >>> > > >
> > > > >>> > > > If a ticket is created for this issue, I shall
be able to
> > > create
> > > > a
> > > > >>> pull
> > > > >>> > > > request and thus a build will take place on
ignite's CI.
> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
implementation
> > > may I
> > > > >>> > create
> > > > >>> > > a
> > > > >>> > > > ticket on JIRA?
> > > > >>> > > >
> > > > >>> > > > My next question has to do on how Ignite configures
the
> build
> > > > jobs
> > > > >>> in
> > > > >>> > > Team
> > > > >>> > > > City. The unit tests for my implementation
need a local
> > Pub/Sub
> > > > >>> server
> > > > >>> > up
> > > > >>> > > > and running (they provide one for testing
purposes). Will
> > this
> > > > take
> > > > >>> > > effect
> > > > >>> > > > in a form of a build script inside my implementation
or is
> it
> > > > >>> something
> > > > >>> > > > that should be configured on Team City?
> > > > >>> > > >
> > > > >>> > > > Kind regards,
> > > > >>> > > > Emmanouil
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > > *Emmanouil Gkatziouras*
> > > > >>> > > > https://egkatzioura.com/ |
> > > > >>> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >>> > > > https://github.com/gkatzioura
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda
<
> dmagda@apache.org
> > >
> > > > >>> wrote:
> > > > >>> > > >
> > > > >>> > > > > Emmanouil, Saikat,
> > > > >>> > > > >
> > > > >>> > > > > After contemplating on this, let me propose
another way
> > with
> > > > this
> > > > >>> > > > > particular integration that is aligned
with our
> > > modularization
> > > > >>> > > endeavor.
> > > > >>> > > > > The modularization [1] defines the Ignite
core, Ignite
> > > modules,
> > > > >>> and
> > > > >>> > > > > independent integrations. The first two
are to be located
> > in
> > > > >>> Ignite
> > > > >>> > > > > repositories and to be tested/updated
by the community
> for
> > > > every
> > > > >>> > > release.
> > > > >>> > > > > The modules are a subset of existing
integrations that
> are
> > > > >>> adopted
> > > > >>> > > widely
> > > > >>> > > > > and require community attention.
> > > > >>> > > > >
> > > > >>> > > > > As for the independent integrations [1],
those are
> supposed
> > > to
> > > > be
> > > > >>> > > located
> > > > >>> > > > > in a separate GitHub repo. I would advise
us to start
> > > > >>> implementing
> > > > >>> > this
> > > > >>> > > > > concept with the Pub/Sub integration.
Even though it will
> > be
> > > > >>> located
> > > > >>> > > in a
> > > > >>> > > > > separate Github repo, it will be listed
among officially
> > > > >>> available
> > > > >>> > > > > integrations on the Ignite website featuring
Emmanouil
> as a
> > > > >>> > > contributor.
> > > > >>> > > > >
> > > > >>> > > > > What do you think?
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > > > [1]
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > > > -
> > > > >>> > > > > Denis
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat
Maitra <
> > > > >>> > saikat.maitra@gmail.com
> > > > >>> > > >
> > > > >>> > > > > wrote:
> > > > >>> > > > >
> > > > >>> > > > > > Hi Emmanouil,
> > > > >>> > > > > >
> > > > >>> > > > > > The changes looks good to me. I
wanted to check if you
> > are
> > > > also
> > > > >>> > able
> > > > >>> > > to
> > > > >>> > > > > > validate that the once data is added
to Ignite cluster
> > you
> > > > are
> > > > >>> also
> > > > >>> > > > able
> > > > >>> > > > > to
> > > > >>> > > > > > access it using another Ignite client
or using rest
> > > > endpoints.
> > > > >>> > > > > >
> > > > >>> > > > > > We use Teamcity for CI process,
the details are as
> > follows
> > > > >>> > > > > >
> > > > >>> > > > > > Trigger validation of those test
suites that have been
> > > > >>> affected by
> > > > >>> > > your
> > > > >>> > > > > > changes on TeamCity <http://ci.ignite.apache.org/>:
<
> > > > >>> > > > > http://204.14.53.153/
> > > > >>> > > > > > >
> > > > >>> > > > > >
> > > > >>> > > > > >    - Locate a test suite you have
to check, press
> button
> > > > named
> > > > >>> > "..."
> > > > >>> > > > that
> > > > >>> > > > > >    is located on the left of "Run"
button. "Run custom
> > > build"
> > > > >>> > dialog
> > > > >>> > > > will
> > > > >>> > > > > >    appear;
> > > > >>> > > > > >    - Go to "Changes" tab and choose
> > > > >>> > "pull/<pull-request-number>/head"
> > > > >>> > > > in
> > > > >>> > > > > >    "Build branch" dropdown list;
> > > > >>> > > > > >    - Press "Run build" button and
monitor tests
> results.
> > > > >>> > > > > >
> > > > >>> > > > > >
> > > > >>> > > > > >
> > > > >>> > > > > > @Roman rshtykh@yahoo.com - can you
also please take a
> > > look?
> > > > >>> > > > > >
> > > > >>> > > > > > Regards,
> > > > >>> > > > > > Saikat
> > > > >>> > > > > >
> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM
Emmanouil Gkatziouras <
> > > > >>> > > > > > gkatzioura@gmail.com>
> > > > >>> > > > > > wrote:
> > > > >>> > > > > >
> > > > >>> > > > > > > Hi @Saikat Maitra
> > > > >>> > > > > > >
> > > > >>> > > > > > > I have my implementation on
an Ignite fork on my
> > personal
> > > > >>> GitHub
> > > > >>> > > > > account
> > > > >>> > > > > > > [1]
> > > > >>> > > > > > > It is based on the Kafka implementation
which is of a
> > > > polling
> > > > >>> > > nature.
> > > > >>> > > > > > > GCP does not provide a service
like Kafka. Instead it
> > > > >>> provides us
> > > > >>> > > > with
> > > > >>> > > > > > > Pub/Sub which is just like
Kafka.
> > > > >>> > > > > > > Therefore using the Kafka streamer
as a guideline
> made
> > > more
> > > > >>> sense
> > > > >>> > > to
> > > > >>> > > > > me.
> > > > >>> > > > > > >
> > > > >>> > > > > > > Apart from the implementation
I have also added a
> unit
> > > > test.
> > > > >>> > > > > > > It definitely needs some polishing
and a double check
> > on
> > > my
> > > > >>> side
> > > > >>> > to
> > > > >>> > > > > make
> > > > >>> > > > > > > the review smooth.
> > > > >>> > > > > > > For the tests a local Pub/Sub
[2] server has to be
> > setup
> > > > >>> however
> > > > >>> > I
> > > > >>> > > am
> > > > >>> > > > > not
> > > > >>> > > > > > > sure which is the best way
to do so with regards to
> > your
> > > > >>> CI/CD.
> > > > >>> > > > > > >
> > > > >>> > > > > > > [1]
> > > > >>> > >
> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > > >>> > > > > > > [2]
> > > > >>> https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> > > > >>> > > > > > >
> > > > >>> > > > > > > *Emmanouil Gkatziouras*
> > > > >>> > > > > > > https://egkatzioura.com/ |
> > > > >>> > > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >>> > > > > > > https://github.com/gkatzioura
> > > > >>> > > > > > >
> > > > >>> > > > > > >
> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11,
Saikat Maitra <
> > > > >>> > > saikat.maitra@gmail.com
> > > > >>> > > > >
> > > > >>> > > > > > > wrote:
> > > > >>> > > > > > >
> > > > >>> > > > > > > > Hello Emmanouil,
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > Can you please share more
info about the
> > > implementation?
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > We have similar implementation
for data streamers
> > with
> > > > >>> Kafka
> > > > >>> > and
> > > > >>> > > > > > RocketMQ
> > > > >>> > > > > > > >
> > > > >>> > > > > > > >
> > https://apacheignite-mix.readme.io/docs/kafka-streamer
> > > > >>> > > > > > > >
> > > > >>> > > > > > > >
> > > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > Please review and share
if you have any questions.
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > Regards,
> > > > >>> > > > > > > > Saikat
> > > > >>> > > > > > > >
> > > > >>> > > > > > > >
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at
4:56 PM Emmanouil
> > Gkatziouras <
> > > > >>> > > > > > > > gkatzioura@gmail.com>
> > > > >>> > > > > > > > wrote:
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > > Hi all,
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > By mentioning Pub/Sub
I mean Google Cloud's
> Pub/Sub
> > > [1]
> > > > >>> > > > > > > > > Yes it is indeed
cloud specific. It is the
> > messaging
> > > > >>> service
> > > > >>> > > > > provided
> > > > >>> > > > > > > by
> > > > >>> > > > > > > > > GCP.
> > > > >>> > > > > > > > > I have made the integration
with it, but I
> haven't
> > > > >>> published
> > > > >>> > it
> > > > >>> > > > yet
> > > > >>> > > > > > (I
> > > > >>> > > > > > > am
> > > > >>> > > > > > > > > waiting for the discussion/ticket
and proceed
> with
> > > the
> > > > >>> > > > > contribution).
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > Kind regards,
> > > > >>> > > > > > > > > Emmanouil
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > [1]
> https://cloud.google.com/pubsub/docs/overview
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
> > > > >>> > > > > > > > > https://egkatzioura.com/
|
> > > > >>> > > > > > > > >
> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >>> > > > > > > > > https://github.com/gkatzioura
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > On Fri, 11 Oct 2019
at 21:45, Denis Magda <
> > > > >>> dmagda@apache.org
> > > > >>> > >
> > > > >>> > > > > wrote:
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > > Hello Emmanouil,
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > Are you referring
to any specific
> implementation
> > > > >>> tailored
> > > > >>> > > for a
> > > > >>> > > > > > > cloud?
> > > > >>> > > > > > > > If
> > > > >>> > > > > > > > > > I'm not mistaken
you were mentioning AWS/GCE.
> Is
> > > > there
> > > > >>> any
> > > > >>> > > > Github
> > > > >>> > > > > > > > source
> > > > >>> > > > > > > > > > code?
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > Here is a list
of existing integrations with
> > > > streaming
> > > > >>> and
> > > > >>> > > > > pub/sub
> > > > >>> > > > > > > > > > frameworks:
> > > > >>> > > > > > > > > >
> https://apacheignite-mix.readme.io/docs/overview
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > with the source
code in Ignite master:
> > > > >>> > > > > > > > > >
> > > https://github.com/apache/ignite/tree/master/modules
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > Check out those
resources to see what are the
> > most
> > > > >>> suitable
> > > > >>> > > > > Ignite
> > > > >>> > > > > > > APIs
> > > > >>> > > > > > > > > for
> > > > >>> > > > > > > > > > such integrations.
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > @Saikat Maitra
<saikat.maitra@gmail.com> ,
> > @Roman
> > > <
> > > > >>> > > > > > rshtykh@yahoo.com
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > ,
> > > > >>> > > > > > > > > as
> > > > >>> > > > > > > > > > streaming technologies
> contributors/maintainers,
> > > > could
> > > > >>> you
> > > > >>> > > > please
> > > > >>> > > > > > > join
> > > > >>> > > > > > > > > this
> > > > >>> > > > > > > > > > thread?
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > -
> > > > >>> > > > > > > > > > Denis
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > On Thu, Oct
10, 2019 at 7:58 AM Emmanouil
> > > > Gkatziouras <
> > > > >>> > > > > > > > > > gkatzioura@gmail.com>
> > > > >>> > > > > > > > > > wrote:
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > > Hello everyone,
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > > > I started
using Ignite lately. Part of my
> work
> > > > >>> involves
> > > > >>> > > > > Pub/Sub,
> > > > >>> > > > > > > > thus I
> > > > >>> > > > > > > > > > > created
a pub/sub streamer.
> > > > >>> > > > > > > > > > > Pub/Sub
is pretty close to Kafka.
> > > > >>> > > > > > > > > > > Here's
how the streamer works. Every node
> > creates
> > > > an
> > > > >>> > > instance
> > > > >>> > > > > of
> > > > >>> > > > > > a
> > > > >>> > > > > > > > > > Pub/Sub
> > > > >>> > > > > > > > > > > client.
> > > > >>> > > > > > > > > > > The clients
subscribe to the same topic using
> > the
> > > > >>> same
> > > > >>> > > > > > subscription
> > > > >>> > > > > > > > > name.
> > > > >>> > > > > > > > > > > A message
is being sent and one of the nodes
> > will
> > > > >>> receive
> > > > >>> > > the
> > > > >>> > > > > > > > message,
> > > > >>> > > > > > > > > > > process
it and then forward it to Ignite
> using
> > > the
> > > > >>> > > extractor
> > > > >>> > > > > > > > provided.
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > > > Any thoughts?
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > > > *Emmanouil
Gkatziouras*
> > > > >>> > > > > > > > > > > https://egkatzioura.com/
|
> > > > >>> > > > > > > > > > >
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > >
> > > > >>> > > > > > >
> > > > >>> > > > > >
> > > > >>> > > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > > >>
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message