Mailing-List: contact dev-help@streams.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@streams.incubator.apache.org
MIME-Version: 1.0
Date: Tue, 5 Jan 2016 08:35:14 -0600
Message-ID: 
 <CAJ7ACC_8Q+=CAxC3Cmj=pnOrkFrmZ5Fwc-f5dOMNR1hPhx=K8w@mail.gmail.com>
Subject: Streams Web Site, Maven Sites,
 misc Documentation - proposed next steps
From: Steve Blackmon <sblackmon@apache.org>
To: "dev@streams.incubator.apache.org" <dev@streams.incubator.apache.org>
Content-Type: multipart/alternative; boundary=e89a8ffbadeb50d7890528972531

--e89a8ffbadeb50d7890528972531
Content-Type: text/plain; charset=UTF-8

All,

Ate gave us a reality check during our board report in December and
enumerated some of the most prominent ways our documentation currently
falls short - of Apache community standards and of any reasonable
useful-ness test.  I've responded in-line below - please kick in any ideas
you have.


- the website isn't providing any practical guidance *why* or *how* to use
Streams


I agree.  This is the content that belongs on the streams website landing
page (high-level) or linked from the body (the details).  I will take a
first shot at this and circulate for feedback.

- its missing concrete examples and recognizable use-cases for which
Streams might be used, nor comparison against other solutions, and why
Streams would be a good/better solution


The examples documentation site [1] are meant to serve as both concrete
examples and recognizable use-cases, but there are a few problems with the
current setup.

   1. The examples site lacks plain-language READMEs for the root module
   and the first children of the root module (the runtime-specific
   aggregators).
   2. The examples site isn't linked from the landing page of the website.
   3. There is no permanent url for latest/current.
   4. There hasn't been a formal release of this repo.
   5. There are broken links within the markdown of some examples.
   6. The plain-language descriptions of the examples themselves could be
   improved.

All of these can be addressed (easily I think) and I'm happy to take them
on.

Additionally, a page (or several) comparing streams to frameworks such as
Storm, Spark, Samza, Flink, etc... and describing how they are
different/complementary would be valuable.  These questions come up a lot.

- the Architecture page only has some basic wording but provides no insight
whatsoever about the concrete implementation and certainly not its
architecture


I think this page needs either an multi-page outline format heavily linked
into READMEs and interfaces throughout the streams repos, or a system
diagram depicting a simple streams deployment alongside the vocabulary.
Maybe both as several pages.

- there is no public javadoc or other technical documentation linked/hosted


We do generate java-docs for releases and snapshots.  We should add a link
to the root page.  We've made progress but there are still many classes
lacking any javadoc header.  We should reiterate a commitment to fixing
that and a schedule for deprecating and deleting any class that doesn't
still serve a distinct purpose.

An index of the many resource files (json/xml schemas, conversion rules,
shell scripts, etc...) published to the project and examples release should
also placed on the main website.  Ideally comprehensive, but if not at
least enough to show a developer that they exist and can be hard-linked to
within their own project resources.


- there is a 'wiki (coming soon)' link which never has been activated


This link (actually most of streams.incubator.apache.org) predates the
regular publication of the streams-project maven site where READMEs and
other resources are hosted.  Personally I'd prefer to see documentation
improving within the code-base rather than in a separate wiki in the
near-term and suggest we just delete this link.

- the mailing list is NOT used for any form of discussion/information other
than JIRA and Git(hub) notifications
- especially the latter doesn't give a newbie any indication what is going
on, nor how to (start) interact

This is unfortunate and must change for the community to grow.  I know I'm
guilty of having ad-hoc off-list discussions with my team that lead to new
stories and pull requests.  Mea culpa, I resolve to stop doing this in 2016.

- there are no blog posts (that I am aware of) around using/trying out
Streams either


This is a critical missing piece of the puzzle.  Static HTML via apache
httpd is not the ideal platform for hosting a blog but some other projects
have made it work.  I'd like us to replace 'Latest News' with a navigable,
indexable set of blog posts going forward, where release summaries and new
examples can be published, and 'in-the-wild' mentions of the project can be
aggregated.


And I personally would prefer the dev@ list to be much more / primarily
used for actual human interaction, not just these notifications from JIRA
and Github.
Maybe change the configuration to delegate (some of) these to the commit@
list instead?


I agree.  AFAICT though commit@streams.incubator.apache.org does not
exist.  Per http://apache.org/dev/committers.html#mail we should hold a
vote to create it.  I'll open that vote today.

What I suggest is to for a while stop (or largely postpone) working on code
but instead working on any/all of the above points first...


I agree that making the website helpful is the top project priority.

Even working on a next release IMO is far less important than first
building up some explanation *why* a release of Stream is needed.


I agree that these concerns should be addressed before the next release.
The main reason for releasing soon is that per [4] there are 36 issues
resolved since the 0.2 release, including quite a few bugs and improvements
related to stability, deployment flexibility, or likely to impact the
experience of a new developer.

[1]
http://streams.incubator.apache.org/site/0.2-incubating-SNAPSHOT/streams-examples/
[2]
http://streams.incubator.apache.org/site/0.2-incubating/streams-project/apidocs/index.html
[3]
http://streams.incubator.apache.org/site/0.2-incubating/streams-project/index.html
[4]
https://issues.apache.org/jira/issues/?jql=project%20%3D%20STREAMS%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20in%20unreleasedVersions()

--e89a8ffbadeb50d7890528972531--