oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From krishna rb <krishna...@gmail.com>
Subject Re: Contributing to oodt Project
Date Sun, 12 Mar 2017 19:55:44 GMT
Thank you, that was a great explanation about the project.
I'll start working on it and get a proposal ready for review soon.
Is there any issues that I can take up to contribute to the OODT project
for now?
I feel I'll get to know the codebase properly if I contribute some more
patches soon.

Thanks,
R B Krishna

On Sun, Mar 12, 2017 at 7:44 PM, Tom Barber <tom.barber@meteorite.bi> wrote:

> Cool.
>
> We'll the Kafka idea is pretty simple, although I need to plan the
> implementation locations properly. So OODT is made up of a bunch of
> components, the most common ones being the Curator, the File Manager and
> the Workflow Manager.
>
> Currently tracking data through OODT isn't as simple as it could or perhaps
> should be, so the idea behind OODT-946 is to  implement Kafka as a pub sub
> system for OODT. For example:
>
> The curator allows users to ingest data into the OODT platform via the file
> manager, so one implementation would be to replace the current
> communication between file manager and curator with a Kafka message bus. So
> user X curates some data and when they ingest it, instead of calling the
> file manager directly the curation message is pushed onto the Kafka message
> bus and a number of services could be subscribed to this. They may be the
> file manager which would pick it up and ingest the data, but it might also
> be an Audit component that tracks the flow of data through OODT, it might
> also be the workflow manager etc.
>
> Similarly when the file manager ingests data, instead of  it simply calling
> the backend factory for Solr, Lucene or whatever to store the ingested
> data, it would be great if it again, pushed the message to Kafka, which in
> turn allowed different file manager backends to subscribe to the queue,
> perhaps to pick up specific ingestion file types or whatever. But also that
> push got optionally audited but the audit component also subscribed to the
> mesage queue, so you know when what file got ingested where, along with the
> entry into the catalog.
>
> That type of thing.
>
> On Sun, Mar 12, 2017 at 1:37 PM, krishna rb <krishnarb3@gmail.com> wrote:
>
> > Yes sir, I am currently looking for issues so that I can understand the
> > project and contribute to it.
> > Also, I am interested in participating in Gsoc 2017 and I found the
> > project: Implement Kafka Messaging System Within OODT
> > <https://issues.apache.org/jira/browse/OODT-946> very interesting.
> > I am currently learning Apache Kafka.
> > It will be immensely helpful if you could explain about the idea.
> >
> > Thanks,
> > R B Krishna
> >
> > On Mar 12, 2017 6:56 PM, "Tom Barber" <tom.barber@meteorite.bi> wrote:
> >
> > > Hi R B,
> > >
> > > You're a Java developer I assume?
> > >
> > > Thanks
> > >
> > > Tom
> > >
> > > On Sat, Mar 11, 2017 at 10:02 AM, krishna rb <krishnarb3@gmail.com>
> > wrote:
> > >
> > > > Hello everyone,
> > > > This is R B Krishna, studying 3rd year Computer Science at National
> > > > Institute of Technology, Trichy, India.
> > > > I am interested in contributing to the project.
> > > >
> > > > Could someone suggest me some issues that I could take up as a newbie
> > and
> > > > guide me through it?
> > > >
> > > > I am currently going through the codebase and trying to get
> comfortable
> > > > with it.
> > > >
> > > > Thanks,
> > > > R B Krishna.
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message