streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Franklin <m.ben.frank...@gmail.com>
Subject Re: [DISCUSS] Continuing the Momentum
Date Mon, 14 Apr 2014 14:31:44 GMT
On Fri, Apr 11, 2014 at 5:01 PM, Steve Blackmon <sblackmon@apache.org>wrote:

> On Thu, Apr 10, 2014 at 4:11 PM, Matt Franklin <m.ben.franklin@gmail.com>
> wrote:
> > tl;dr version:
> >
> > We need to discuss things on the list more and work to define streams,
> > update our public presence to support this definition and encourage
> > additional engagement.
> >
> +1, +1, +1
>
> > Long version:
> >
> > For those of you unaware, Steve Blackmon gave a nice talk on the work he
> > has been committing to Streams at ApacheCon.  As part of that talk and
> > follow on discussions, it became clear that we as a community need to do
> > some serious work to define ourselves, what we are building and why it is
> > valuable to the industry.
> >
> If anyone who missed the presentation wants to see it, I'm happy to
> host a google hangout to run through it.
>

Can you post it, or a link to it, on the website too?


>
> > Our website says we are a Lightweight server for ActivityStreams.  While
> > this is true to some degree, I think recent contributions should refine
> > this.  The new code is really about supporting flexible processing,
> > persistence and retrieval of data in multiple runtimes using strongly
> > typed, normalized data formats like ActivityStreams.  Personally, I think
> > this slightly new direction is extremely compelling, and the reaction to
> > Steve's talk seems to support that.  The question remains how does the
> > community as a whole see the project?  What value is everyone wanting to
> > get out of this effort?
> >
> The session tag-line which attracted ~20 attendees was 'Simplifying
> Real-Time data integration with Apache Streams.' From talking to
> coders and data scientists I always hear frustration with how much
> time they spend writing code and workflow to move bytes around and
> keep track of their data assets. I'd wager any survey of prominent
> open-source libraries and popular commercial APIs would have to
> conclude that schema and interface standards are completely absent
> or sparsely adopted at many layers.
>
> Standards in hardware, operating systems, networks, and relational
> databases brought about flourishing ecosystems. I believe standards in
> data interchange such as ActivityStreams can do the same for the
> social web, but not everyone will embrace standards for the sake of
> standards. If we can offer integration points to the data sources and
> repositories businesses want to work with, and demonstrate that
> Streams can handle 'fire-hose' scale data volumes with arbitrarily
> many intermediate hand-offs and processing steps on messages in
> flight, I think we will see adoption from enterprises looking to
> replace ESB-type systems that can't keep up with the volume of data
> generated (both inside and outside their networks) that they want to
> track.  Streams is pretty decent at ETL as well - a function that is
> never going away, even as the underlying tools best suited to
> performing it at scale constantly change.
>
> This future-state I'm attempting to describe will be a better one for
> researchers, hobbyists, entrepreneurs, and consumers of web products
> and services.  Configuration-driven, runtime-platform agnostic,
> software for real-time data exchange:  where community-driven
> standards such as Activity Streams can codify and evolve
> best-practices via running code.  That is a vision that I think will
> help us generate significant traction going forward.
>

Just to make sure I am understanding you correctly, you are proposing we
update the mission of the project to the following:

1)  A flexible data processing framework that can run in multiple different
runtimes.  The goal being to abstract platform complexity and allow for
business logic reuse across real-time, enterprise, web and stand-alone
executions.
2)  As a proving ground for the adoption of data format standards,
specifically ActivityStreams to start.  The community would work to drive
the adoption and evolution of such standards through real-world experience.

This sounds great, though it is slightly different than the initially
proposed functionality.  Personally, I have no objection to that, as what
you describe encompasses the original goals and expands on them; but, it
would be good for the rest of the community to weigh in.


>
> > The fact that there are not clear answers (and corresponding documented
> > statements on the website) to these questions already means we are not
> > doing a great job of following the Apache Way.  The Apache Way is about
> the
> > community and meritocratic, community-based decision making.  The ASF
> > defines it as follows:
> >
> > While there is not an official list, these six principles have been cited
> > as the core beliefs of philosophy behind the foundation, which is
> normally
> > referred to as "The Apache Way":
> >
> > collaborative software development
> >
> > commercial-friendly standard license
> >
> > consistently high quality software
> >
> > respectful, honest, technical-based interaction
> >
> > faithful implementation of standards
> >
> > security as a mandatory feature
> >
> > All of the ASF projects share these principles.
> >
> > Let's make sure we propose changes to the list, create tickets that
> support
> > wider efforts and leverage principles like lazy consensus to keep moving
> > forward in a way that supports the community.
> +1, +1, +1
>
> On Thu, Apr 10, 2014 at 4:11 PM, Matt Franklin <m.ben.franklin@gmail.com>
> wrote:
> > tl;dr version:
> >
> > We need to discuss things on the list more and work to define streams,
> > update our public presence to support this definition and encourage
> > additional engagement.
> >
> > Long version:
> >
> > For those of you unaware, Steve Blackmon gave a nice talk on the work he
> > has been committing to Streams at ApacheCon.  As part of that talk and
> > follow on discussions, it became clear that we as a community need to do
> > some serious work to define ourselves, what we are building and why it is
> > valuable to the industry.
> >
> > Our website says we are a Lightweight server for ActivityStreams.  While
> > this is true to some degree, I think recent contributions should refine
> > this.  The new code is really about supporting flexible processing,
> > persistence and retrieval of data in multiple runtimes using strongly
> > typed, normalized data formats like ActivityStreams.  Personally, I think
> > this slightly new direction is extremely compelling, and the reaction to
> > Steve's talk seems to support that.  The question remains how does the
> > community as a whole see the project?  What value is everyone wanting to
> > get out of this effort?
> >
> > The fact that there are not clear answers (and corresponding documented
> > statements on the website) to these questions already means we are not
> > doing a great job of following the Apache Way.  The Apache Way is about
> the
> > community and meritocratic, community-based decision making.  The ASF
> > defines it as follows:
> >
> > While there is not an official list, these six principles have been cited
> > as the core beliefs of philosophy behind the foundation, which is
> normally
> > referred to as "The Apache Way":
> >
> > collaborative software development
> >
> > commercial-friendly standard license
> >
> > consistently high quality software
> >
> > respectful, honest, technical-based interaction
> >
> > faithful implementation of standards
> >
> > security as a mandatory feature
> >
> > All of the ASF projects share these principles.
> >
> > Let's make sure we propose changes to the list, create tickets that
> support
> > wider efforts and leverage principles like lazy consensus to keep moving
> > forward in a way that supports the community.
>
>
>
> --
> Steve Blackmon
> sblackmon@apache.org
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message