ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saikat Maitra <saikat.mai...@gmail.com>
Subject Re: [DISCUSS] Pub/Sub Streamer Implementation
Date Sun, 10 Nov 2019 23:06:51 GMT
Hi Emmanouil,

Can you please take a looks at dev utils, if this is something you are
looking for https://github.com/apache/ignite/tree/master/modules/dev-utils?

IMO, if you can release Pub/Sub server in maven and then use it as
dependency, that would be great.

Regards,
Saikat

On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <saikat.maitra@gmail.com>
wrote:

> Hi Emmanouil,
>
> The master branch has been initialized. I have raised a PR to migrate
> Flink module to ignite-extensions.
>
> PR https://github.com/apache/ignite-extensions/pull/1
> Jira : https://issues.apache.org/jira/browse/IGNITE-12356
>
> Regards,
> Saikat
>
>
> On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> gkatzioura@gmail.com> wrote:
>
>> Hi all!
>>
>> This is an update. The integration is tested with mocks based on the
>> original Ignite repo [1].
>> I am currently waiting for the first commits on the new integration
>> project
>> to take place in order to create my pull request accordingly to the
>> initial
>> commits.
>> In order to simulate the Pub/Sub server in a test I used a test server
>> implementation [2]. Unfortunately they don't have this util uploaded in a
>> maven repo, thus it needs to be hosted on a maven repo (Apache License
>> 2.0). Is there a repo for utilities that Ignite uses for the test/build?
>> I have also created a ticket [3]
>>
>> Kind regards
>>
>> [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
>> [3] https://issues.apache.org/jira/browse/IGNITE-12262
>>
>> *Emmanouil Gkatziouras*
>> https://egkatzioura.com/ |
>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> https://github.com/gkatzioura
>>
>>
>> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <saikat.maitra@gmail.com>
>> wrote:
>>
>> > Hello Ilya,
>> >
>> > Thank you for your email. Yes, before we remove any integration we will
>> > move them to another repository and release.
>> >
>> > We are thinking we will start with Pub/Sub Streamer implementation and
>> > continue to move other integration as mentioned in the following doc.
>> >
>> >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >
>> > Regards,
>> > Saikat
>> >
>> >
>> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
>> > ilya.kasnacheev@gmail.com>
>> > wrote:
>> >
>> > > Hello!
>> > >
>> > > My take on this: we want to offload some of integrations, but please
>> > commit
>> > > and release them to another project first, and only then we will
>> decide
>> > to
>> > > remove them from our repository.
>> > >
>> > > Please also make sure they are available with same artifact name and
>> API.
>> > > If this has to be changed, let's wait for AI 3.0
>> > >
>> > > Regards,
>> > > --
>> > > Ilya Kasnacheev
>> > >
>> > >
>> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <saikat.maitra@gmail.com>:
>> > >
>> > > > Hello Emmanouil,
>> > > >
>> > > > As discussed earlier, I discussed with Apache Bahir community and
>> they
>> > > are
>> > > > interested to have Apache Ignite extensions as part of Apache Bahir
>> > > > project.
>> > > >
>> > > > Can you please share Pub/Sub streamer implementation details with
>> > Apache
>> > > > Bahir community and request for feedback.
>> > > >
>> > > > https://bahir.apache.org/
>> > > >
>> > > > I have also requested for contributor access in Jira for Apache
>> Bahir
>> > > > project so that I can create issues and assign to myself.
>> > > >
>> > > > I can help with code reviews as well.
>> > > >
>> > > > Here is the email thread for reference
>> > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>> > > >
>> > > > Regards,
>> > > > Saikat
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
>> saikat.maitra@gmail.com
>> > >
>> > > > wrote:
>> > > >
>> > > > > Hello,
>> > > > >
>> > > > > As discussed I have created following discussion threads on our
>> > > migration
>> > > > > proposals for Apache Ignite extensions:
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
>> > > > >
>> > > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>> > > > >
>> > > > > I will share update as I hear more information.
>> > > > >
>> > > > > Regards,
>> > > > > Saikat
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
>> > saikat.maitra@gmail.com
>> > > >
>> > > > > wrote:
>> > > > >
>> > > > >> Hello Denis,
>> > > > >>
>> > > > >> Sure, sounds good. I will create a separate discussion thread
to
>> get
>> > > > >> community feedback on whether we
>> > > > >> should join the Bahir or kick-off "Ignite Extensions" project.
>> > > > >>
>> > > > >> Regards,
>> > > > >> Saikat
>> > > > >>
>> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <dmagda@apache.org>
>> > > wrote:
>> > > > >>
>> > > > >>> Folks,
>> > > > >>>
>> > > > >>> The concept of Apache Bahir is precisely what we need!
Saikat,
>> > thanks
>> > > > for
>> > > > >>> doing the research. Why I believe the * idea* fits us
well:
>> > > > >>>
>> > > > >>>    - All integrations can be stored in separate Github
>> repositories
>> > > and
>> > > > >>>    have their dev lifecycles. We've not obliged to couple
all
>> the
>> > > > >>> integrations
>> > > > >>>    in a single repo. For instance, Spark can be located
in a
>> > > dedicated
>> > > > >>> repo
>> > > > >>>    while streaming integrations might be in a single
one. It's
>> up
>> > to
>> > > > us.
>> > > > >>>    - All the repositories and integrations belong to
ASF. We're
>> not
>> > > > >>> dumping
>> > > > >>>    them on Github but rather keep supporting and developing
in
>> > > > >>> accordance with
>> > > > >>>    ASF vision and practices.
>> > > > >>>    - There is a way to reward contributors via committership
and
>> > PMC
>> > > > >>>    membership. You won't get this if a project is just
one of
>> the
>> > > > >>> millions of
>> > > > >>>    Github projects.
>> > > > >>>
>> > > > >>> Saikat, as Ignite PMC member, could you please kick-off
a
>> separate
>> > > > >>> dev-list
>> > > > >>> discussion to involve more community members and referring
to
>> this
>> > > > >>> thread?
>> > > > >>> I think the primary question the community needs to answer
>> whether
>> > we
>> > > > >>> should join the Bahir or kick-off "Ignite Extensions"
project?
>> > > > >>>
>> > > > >>> -
>> > > > >>> Denis
>> > > > >>>
>> > > > >>>
>> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
>> > > > zaleslaw.sin@gmail.com>
>> > > > >>> wrote:
>> > > > >>>
>> > > > >>> > Maybe we could move all our Streaming Integrations
there, but
>> > what
>> > > is
>> > > > >>> about
>> > > > >>> > maintaining and committer permissions to the new
repositories?
>> > > > >>> > I see the list of the committers and PMC members
there
>> > > > >>> > https://bahir.apache.org/community-members/
>> > > > >>> >
>> > > > >>> > Could somebody from Ignite community be added to
this list as
>> a
>> > > > >>> > PMC/committer to maintain Ignite integrations?
>> > > > >>> > If yes, I'd happy to join this project and collaborate
with
>> these
>> > > > guys
>> > > > >>> > together
>> > > > >>> > If no, and we should start from the zero level with
external
>> PRs
>> > -
>> > > > >>> hmmm,
>> > > > >>> > it's better to have our own external repository
with
>> ApacheBahirr
>> > > > >>> ideology.
>> > > > >>> >
>> > > > >>> > Also, I agree, that all connectors could be moved
there (in
>> > > > >>> ApacheBahirr
>> > > > >>> > like repository), but not all modules from the page
>> > > > >>> >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > > >>> > because
>> > > > >>> > they use different parts of core modules and could
be
>> developed
>> > > with
>> > > > >>> > different velocity.
>> > > > >>> >
>> > > > >>> > In reality, before creation of new repositories
we need wide
>> > > > discussion
>> > > > >>> > based on the document mentioned above.
>> > > > >>> >
>> > > > >>> > P.S. Of course, we could experiment in the next
release 2.8
>> with
>> > > one
>> > > > >>> or two
>> > > > >>> > integrations like twitter or ZeroMQ
>> > > > >>> > Also, I'd like Denis Magda idea for the separate
repositories
>> in
>> > > > Apache
>> > > > >>> > Github to maintain them by anybody.
>> > > > >>> >
>> > > > >>> >
>> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra
<
>> > > saikat.maitra@gmail.com
>> > > > >:
>> > > > >>> >
>> > > > >>> > > Hi Denis, Emmanouil
>> > > > >>> > >
>> > > > >>> > > Thank you for your email. I think creating
a separate
>> > integration
>> > > > >>> project
>> > > > >>> > > in github and also proposing it as an apache
incubator
>> project
>> > > will
>> > > > >>> be a
>> > > > >>> > > good move. The other separate integration modules
>> > > > >>> > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > > >>> > > can
>> > > > >>> > > be moved to this new apache incubator project.
>> > > > >>> > >
>> > > > >>> > > There are similar integration available for
Flink and Spark
>> in
>> > > > Apache
>> > > > >>> > Bahir
>> > > > >>> > > https://bahir.apache.org/
>> > > > >>> > >
>> > > > >>> > > Do you think we should reach out to Apache
Bahir community
>> > with a
>> > > > >>> > proposal
>> > > > >>> > > or we should plan to create a separate Apache
Incubator
>> > project?
>> > > > >>> > >
>> > > > >>> > > Regards,
>> > > > >>> > > Saikat
>> > > > >>> > >
>> > > > >>> > >
>> > > > >>> > >
>> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil Gkatziouras
<
>> > > > >>> > > gkatzioura@gmail.com>
>> > > > >>> > > wrote:
>> > > > >>> > >
>> > > > >>> > > > Hi all!
>> > > > >>> > > >
>> > > > >>> > > > Based on the discussions most probably
this is going to
>> be on
>> > > > >>> another
>> > > > >>> > > > GitHub repo. Therefore I will prepare
a standalone project
>> > with
>> > > > the
>> > > > >>> > > Pub/Sub
>> > > > >>> > > > module and once the repository is created
a pull request
>> > shall
>> > > be
>> > > > >>> > issued.
>> > > > >>> > > > If there is anything else I can do in
order to facilitate
>> > this
>> > > > >>> please
>> > > > >>> > let
>> > > > >>> > > > me know.
>> > > > >>> > > >
>> > > > >>> > > > If a ticket is created for this issue,
I shall be able to
>> > > create
>> > > > a
>> > > > >>> pull
>> > > > >>> > > > request and thus a build will take place
on ignite's CI.
>> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
>> implementation
>> > > may I
>> > > > >>> > create
>> > > > >>> > > a
>> > > > >>> > > > ticket on JIRA?
>> > > > >>> > > >
>> > > > >>> > > > My next question has to do on how Ignite
configures the
>> build
>> > > > jobs
>> > > > >>> in
>> > > > >>> > > Team
>> > > > >>> > > > City. The unit tests for my implementation
need a local
>> > Pub/Sub
>> > > > >>> server
>> > > > >>> > up
>> > > > >>> > > > and running (they provide one for testing
purposes). Will
>> > this
>> > > > take
>> > > > >>> > > effect
>> > > > >>> > > > in a form of a build script inside my
implementation or
>> is it
>> > > > >>> something
>> > > > >>> > > > that should be configured on Team City?
>> > > > >>> > > >
>> > > > >>> > > > Kind regards,
>> > > > >>> > > > Emmanouil
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > > > *Emmanouil Gkatziouras*
>> > > > >>> > > > https://egkatzioura.com/ |
>> > > > >>> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > >>> > > > https://github.com/gkatzioura
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda
<
>> dmagda@apache.org
>> > >
>> > > > >>> wrote:
>> > > > >>> > > >
>> > > > >>> > > > > Emmanouil, Saikat,
>> > > > >>> > > > >
>> > > > >>> > > > > After contemplating on this, let
me propose another way
>> > with
>> > > > this
>> > > > >>> > > > > particular integration that is aligned
with our
>> > > modularization
>> > > > >>> > > endeavor.
>> > > > >>> > > > > The modularization [1] defines the
Ignite core, Ignite
>> > > modules,
>> > > > >>> and
>> > > > >>> > > > > independent integrations. The first
two are to be
>> located
>> > in
>> > > > >>> Ignite
>> > > > >>> > > > > repositories and to be tested/updated
by the community
>> for
>> > > > every
>> > > > >>> > > release.
>> > > > >>> > > > > The modules are a subset of existing
integrations that
>> are
>> > > > >>> adopted
>> > > > >>> > > widely
>> > > > >>> > > > > and require community attention.
>> > > > >>> > > > >
>> > > > >>> > > > > As for the independent integrations
[1], those are
>> supposed
>> > > to
>> > > > be
>> > > > >>> > > located
>> > > > >>> > > > > in a separate GitHub repo. I would
advise us to start
>> > > > >>> implementing
>> > > > >>> > this
>> > > > >>> > > > > concept with the Pub/Sub integration.
Even though it
>> will
>> > be
>> > > > >>> located
>> > > > >>> > > in a
>> > > > >>> > > > > separate Github repo, it will be
listed among officially
>> > > > >>> available
>> > > > >>> > > > > integrations on the Ignite website
featuring Emmanouil
>> as a
>> > > > >>> > > contributor.
>> > > > >>> > > > >
>> > > > >>> > > > > What do you think?
>> > > > >>> > > > >
>> > > > >>> > > > >
>> > > > >>> > > > > [1]
>> > > > >>> > > > >
>> > > > >>> > > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > > >>> > > > >
>> > > > >>> > > > >
>> > > > >>> > > > >
>> > > > >>> > > > > -
>> > > > >>> > > > > Denis
>> > > > >>> > > > >
>> > > > >>> > > > >
>> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat
Maitra <
>> > > > >>> > saikat.maitra@gmail.com
>> > > > >>> > > >
>> > > > >>> > > > > wrote:
>> > > > >>> > > > >
>> > > > >>> > > > > > Hi Emmanouil,
>> > > > >>> > > > > >
>> > > > >>> > > > > > The changes looks good to me.
I wanted to check if you
>> > are
>> > > > also
>> > > > >>> > able
>> > > > >>> > > to
>> > > > >>> > > > > > validate that the once data
is added to Ignite cluster
>> > you
>> > > > are
>> > > > >>> also
>> > > > >>> > > > able
>> > > > >>> > > > > to
>> > > > >>> > > > > > access it using another Ignite
client or using rest
>> > > > endpoints.
>> > > > >>> > > > > >
>> > > > >>> > > > > > We use Teamcity for CI process,
the details are as
>> > follows
>> > > > >>> > > > > >
>> > > > >>> > > > > > Trigger validation of those
test suites that have been
>> > > > >>> affected by
>> > > > >>> > > your
>> > > > >>> > > > > > changes on TeamCity <http://ci.ignite.apache.org/>:
<
>> > > > >>> > > > > http://204.14.53.153/
>> > > > >>> > > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > > >    - Locate a test suite you
have to check, press
>> button
>> > > > named
>> > > > >>> > "..."
>> > > > >>> > > > that
>> > > > >>> > > > > >    is located on the left of
"Run" button. "Run custom
>> > > build"
>> > > > >>> > dialog
>> > > > >>> > > > will
>> > > > >>> > > > > >    appear;
>> > > > >>> > > > > >    - Go to "Changes" tab and
choose
>> > > > >>> > "pull/<pull-request-number>/head"
>> > > > >>> > > > in
>> > > > >>> > > > > >    "Build branch" dropdown list;
>> > > > >>> > > > > >    - Press "Run build" button
and monitor tests
>> results.
>> > > > >>> > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > > > @Roman rshtykh@yahoo.com - can
you also please take a
>> > > look?
>> > > > >>> > > > > >
>> > > > >>> > > > > > Regards,
>> > > > >>> > > > > > Saikat
>> > > > >>> > > > > >
>> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20
PM Emmanouil Gkatziouras
>> <
>> > > > >>> > > > > > gkatzioura@gmail.com>
>> > > > >>> > > > > > wrote:
>> > > > >>> > > > > >
>> > > > >>> > > > > > > Hi @Saikat Maitra
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > I have my implementation
on an Ignite fork on my
>> > personal
>> > > > >>> GitHub
>> > > > >>> > > > > account
>> > > > >>> > > > > > > [1]
>> > > > >>> > > > > > > It is based on the Kafka
implementation which is of
>> a
>> > > > polling
>> > > > >>> > > nature.
>> > > > >>> > > > > > > GCP does not provide a
service like Kafka. Instead
>> it
>> > > > >>> provides us
>> > > > >>> > > > with
>> > > > >>> > > > > > > Pub/Sub which is just like
Kafka.
>> > > > >>> > > > > > > Therefore using the Kafka
streamer as a guideline
>> made
>> > > more
>> > > > >>> sense
>> > > > >>> > > to
>> > > > >>> > > > > me.
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > Apart from the implementation
I have also added a
>> unit
>> > > > test.
>> > > > >>> > > > > > > It definitely needs some
polishing and a double
>> check
>> > on
>> > > my
>> > > > >>> side
>> > > > >>> > to
>> > > > >>> > > > > make
>> > > > >>> > > > > > > the review smooth.
>> > > > >>> > > > > > > For the tests a local Pub/Sub
[2] server has to be
>> > setup
>> > > > >>> however
>> > > > >>> > I
>> > > > >>> > > am
>> > > > >>> > > > > not
>> > > > >>> > > > > > > sure which is the best
way to do so with regards to
>> > your
>> > > > >>> CI/CD.
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > [1]
>> > > > >>> > >
>> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> > > > >>> > > > > > > [2]
>> > > > >>> https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > *Emmanouil Gkatziouras*
>> > > > >>> > > > > > > https://egkatzioura.com/
|
>> > > > >>> > > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > >>> > > > > > > https://github.com/gkatzioura
>> > > > >>> > > > > > >
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > On Sat, 12 Oct 2019 at
02:11, Saikat Maitra <
>> > > > >>> > > saikat.maitra@gmail.com
>> > > > >>> > > > >
>> > > > >>> > > > > > > wrote:
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > > Hello Emmanouil,
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > Can you please share
more info about the
>> > > implementation?
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > We have similar implementation
for data streamers
>> > with
>> > > > >>> Kafka
>> > > > >>> > and
>> > > > >>> > > > > > RocketMQ
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > >
>> > https://apacheignite-mix.readme.io/docs/kafka-streamer
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > >
>> > > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > Please review and
share if you have any questions.
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > Regards,
>> > > > >>> > > > > > > > Saikat
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > On Fri, Oct 11, 2019
at 4:56 PM Emmanouil
>> > Gkatziouras <
>> > > > >>> > > > > > > > gkatzioura@gmail.com>
>> > > > >>> > > > > > > > wrote:
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > > Hi all,
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > By mentioning
Pub/Sub I mean Google Cloud's
>> Pub/Sub
>> > > [1]
>> > > > >>> > > > > > > > > Yes it is indeed
cloud specific. It is the
>> > messaging
>> > > > >>> service
>> > > > >>> > > > > provided
>> > > > >>> > > > > > > by
>> > > > >>> > > > > > > > > GCP.
>> > > > >>> > > > > > > > > I have made the
integration with it, but I
>> haven't
>> > > > >>> published
>> > > > >>> > it
>> > > > >>> > > > yet
>> > > > >>> > > > > > (I
>> > > > >>> > > > > > > am
>> > > > >>> > > > > > > > > waiting for the
discussion/ticket and proceed
>> with
>> > > the
>> > > > >>> > > > > contribution).
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > Kind regards,
>> > > > >>> > > > > > > > > Emmanouil
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > [1]
>> https://cloud.google.com/pubsub/docs/overview
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
>> > > > >>> > > > > > > > > https://egkatzioura.com/
|
>> > > > >>> > > > > > > > >
>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > >>> > > > > > > > > https://github.com/gkatzioura
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > On Fri, 11 Oct
2019 at 21:45, Denis Magda <
>> > > > >>> dmagda@apache.org
>> > > > >>> > >
>> > > > >>> > > > > wrote:
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > > Hello Emmanouil,
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > Are you
referring to any specific
>> implementation
>> > > > >>> tailored
>> > > > >>> > > for a
>> > > > >>> > > > > > > cloud?
>> > > > >>> > > > > > > > If
>> > > > >>> > > > > > > > > > I'm not
mistaken you were mentioning AWS/GCE.
>> Is
>> > > > there
>> > > > >>> any
>> > > > >>> > > > Github
>> > > > >>> > > > > > > > source
>> > > > >>> > > > > > > > > > code?
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > Here is
a list of existing integrations with
>> > > > streaming
>> > > > >>> and
>> > > > >>> > > > > pub/sub
>> > > > >>> > > > > > > > > > frameworks:
>> > > > >>> > > > > > > > > >
>> https://apacheignite-mix.readme.io/docs/overview
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > with the
source code in Ignite master:
>> > > > >>> > > > > > > > > >
>> > > https://github.com/apache/ignite/tree/master/modules
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > Check out
those resources to see what are the
>> > most
>> > > > >>> suitable
>> > > > >>> > > > > Ignite
>> > > > >>> > > > > > > APIs
>> > > > >>> > > > > > > > > for
>> > > > >>> > > > > > > > > > such integrations.
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > @Saikat
Maitra <saikat.maitra@gmail.com> ,
>> > @Roman
>> > > <
>> > > > >>> > > > > > rshtykh@yahoo.com
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > ,
>> > > > >>> > > > > > > > > as
>> > > > >>> > > > > > > > > > streaming
technologies
>> contributors/maintainers,
>> > > > could
>> > > > >>> you
>> > > > >>> > > > please
>> > > > >>> > > > > > > join
>> > > > >>> > > > > > > > > this
>> > > > >>> > > > > > > > > > thread?
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > -
>> > > > >>> > > > > > > > > > Denis
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > On Thu,
Oct 10, 2019 at 7:58 AM Emmanouil
>> > > > Gkatziouras <
>> > > > >>> > > > > > > > > > gkatzioura@gmail.com>
>> > > > >>> > > > > > > > > > wrote:
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > > Hello
everyone,
>> > > > >>> > > > > > > > > > >
>> > > > >>> > > > > > > > > > > I started
using Ignite lately. Part of my
>> work
>> > > > >>> involves
>> > > > >>> > > > > Pub/Sub,
>> > > > >>> > > > > > > > thus I
>> > > > >>> > > > > > > > > > > created
a pub/sub streamer.
>> > > > >>> > > > > > > > > > > Pub/Sub
is pretty close to Kafka.
>> > > > >>> > > > > > > > > > > Here's
how the streamer works. Every node
>> > creates
>> > > > an
>> > > > >>> > > instance
>> > > > >>> > > > > of
>> > > > >>> > > > > > a
>> > > > >>> > > > > > > > > > Pub/Sub
>> > > > >>> > > > > > > > > > > client.
>> > > > >>> > > > > > > > > > > The
clients subscribe to the same topic
>> using
>> > the
>> > > > >>> same
>> > > > >>> > > > > > subscription
>> > > > >>> > > > > > > > > name.
>> > > > >>> > > > > > > > > > > A message
is being sent and one of the nodes
>> > will
>> > > > >>> receive
>> > > > >>> > > the
>> > > > >>> > > > > > > > message,
>> > > > >>> > > > > > > > > > > process
it and then forward it to Ignite
>> using
>> > > the
>> > > > >>> > > extractor
>> > > > >>> > > > > > > > provided.
>> > > > >>> > > > > > > > > > >
>> > > > >>> > > > > > > > > > > Any
thoughts?
>> > > > >>> > > > > > > > > > >
>> > > > >>> > > > > > > > > > > *Emmanouil
Gkatziouras*
>> > > > >>> > > > > > > > > > > https://egkatzioura.com/
|
>> > > > >>> > > > > > > > > > >
>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
>> > > > >>> > > > > > > > > > >
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > > >>
>> > > >
>> > >
>> >
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message