Mailing-List: contact dev-help@streams.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@streams.incubator.apache.org
Subject: Re: Streams Web Site, Maven Sites, misc Documentation - proposed next
 steps
To: dev@streams.incubator.apache.org
References: 
 <CAJ7ACC_8Q+=CAxC3Cmj=pnOrkFrmZ5Fwc-f5dOMNR1hPhx=K8w@mail.gmail.com>
 <568E96FC.5000200@douma.nu>
 <CAJ7ACC_V-R0j1M=TrBWz0mO54z4vaXZiXrioejp5-xUoom8Wyw@mail.gmail.com>
From: Ate Douma <ate@douma.nu>
Message-ID: <56D15E0C.4010504@douma.nu>
Date: Sat, 27 Feb 2016 09:27:56 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
 Thunderbird/38.5.1
MIME-Version: 1.0
In-Reply-To: 
 <CAJ7ACC_V-R0j1M=TrBWz0mO54z4vaXZiXrioejp5-xUoom8Wyw@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit

On 2016-02-25 23:50, Steve Blackmon wrote:
> Hello streamers,
>
> I finally consolidated some materials I've been working on into a
> nice-looking thing that is a viable candidate to replace the current
> streams web page.  Please take a look and let me know what you think.  Sometime
> soon I expect to submit a vote to take down the previous site and replace
> it with something derived from this version.
>
> The proposed new site is temporarily being hosted at:
> http://people.apache.org/~sblackmon/0.3-incubating-SNAPSHOT/streams-master/index.html
>
> I placed all of the content into markdown files which are checked into a
> branch of streams-master.  This approach allows us to author the site using
> markdown, package and deploy it with maven-site, and manage changes using
> git.

This looks cool!

I like the new overview and faq pages, definitely more concrete about Streams.

One thing which I think can 'spice up' the overview page a bit more would be a
diagram providing some very high-level interaction between data sources and
destinations and the streams role between them.

And before replacing the current site several existing pages need to be moved to
the new one. But once those are done I think this is great improvement.

>
> If you'd like to submit specific changes feel free to do so as
> pull-requests to:
> https://github.com/steveblackmon/incubator-streams-master.git
>
> Referring back to the original list of goals, some are addressed in whole
> or in part with this revision, others not just yet.
>
> ADDRESSED IN WHOLE:
> - the website isn't providing any practical guidance *why* or *how* to use
> Streams
> - there is no public javadoc or other technical documentation linked/hosted
> - there is a 'wiki (coming soon)' link which never has been activated
>
> ADDRESSED IN PART:
> - its missing concrete examples and recognizable use-cases for which
> Streams might
> be used, nor comparison against other solutions, and why Streams would be a
> good/better solution
> - the Architecture page only has some basic wording but provides no insight
> whatsoever about the concrete implementation and certainly not its
> architecture
>
> This is just a start.  Communication on the list has been dead lately, but
> I'm hoping to change that beginning with this effort to explain what is
> unique and valuable about Streams, and why.  Hopefully this take on the
> project will inspire others to get on-board and help out in the coming
> months.

+1, good work!

>
> Steve Blackmon
> sblackmon@apache.org
>
> On Thu, Jan 7, 2016 at 10:49 AM, Ate Douma <ate@douma.nu> wrote:
>
>> Hi Steve,
>>
>> Cool. I don't have much extra input or ideas right now, but looking
>> forward to the steps taken you proposed below.
>> And about getting out a next release first soon, if it is not
>> side-tracking these other steps much, sure +1
>>
>> Ate
>>
>> On 2016-01-05 15:35, Steve Blackmon wrote:
>>
>>> All,
>>>
>>> Ate gave us a reality check during our board report in December and
>>> enumerated some of the most prominent ways our documentation currently
>>> falls short - of Apache community standards and of any reasonable
>>> useful-ness test.  I've responded in-line below - please kick in any ideas
>>> you have.
>>>
>>>
>>> - the website isn't providing any practical guidance *why* or *how* to use
>>> Streams
>>>
>>>
>>> I agree.  This is the content that belongs on the streams website landing
>>> page (high-level) or linked from the body (the details).  I will take a
>>> first shot at this and circulate for feedback.
>>>
>>> - its missing concrete examples and recognizable use-cases for which
>>> Streams might be used, nor comparison against other solutions, and why
>>> Streams would be a good/better solution
>>>
>>>
>>> The examples documentation site [1] are meant to serve as both concrete
>>> examples and recognizable use-cases, but there are a few problems with the
>>> current setup.
>>>
>>>      1. The examples site lacks plain-language READMEs for the root module
>>>      and the first children of the root module (the runtime-specific
>>>      aggregators).
>>>      2. The examples site isn't linked from the landing page of the
>>> website.
>>>      3. There is no permanent url for latest/current.
>>>      4. There hasn't been a formal release of this repo.
>>>      5. There are broken links within the markdown of some examples.
>>>      6. The plain-language descriptions of the examples themselves could be
>>>
>>>      improved.
>>>
>>> All of these can be addressed (easily I think) and I'm happy to take them
>>> on.
>>>
>>> Additionally, a page (or several) comparing streams to frameworks such as
>>> Storm, Spark, Samza, Flink, etc... and describing how they are
>>> different/complementary would be valuable.  These questions come up a lot.
>>>
>>> - the Architecture page only has some basic wording but provides no
>>> insight
>>> whatsoever about the concrete implementation and certainly not its
>>> architecture
>>>
>>>
>>> I think this page needs either an multi-page outline format heavily linked
>>> into READMEs and interfaces throughout the streams repos, or a system
>>> diagram depicting a simple streams deployment alongside the vocabulary.
>>> Maybe both as several pages.
>>>
>>> - there is no public javadoc or other technical documentation
>>> linked/hosted
>>>
>>>
>>> We do generate java-docs for releases and snapshots.  We should add a link
>>> to the root page.  We've made progress but there are still many classes
>>> lacking any javadoc header.  We should reiterate a commitment to fixing
>>> that and a schedule for deprecating and deleting any class that doesn't
>>> still serve a distinct purpose.
>>>
>>> An index of the many resource files (json/xml schemas, conversion rules,
>>> shell scripts, etc...) published to the project and examples release
>>> should
>>> also placed on the main website.  Ideally comprehensive, but if not at
>>> least enough to show a developer that they exist and can be hard-linked to
>>> within their own project resources.
>>>
>>>
>>> - there is a 'wiki (coming soon)' link which never has been activated
>>>
>>>
>>> This link (actually most of streams.incubator.apache.org) predates the
>>> regular publication of the streams-project maven site where READMEs and
>>> other resources are hosted.  Personally I'd prefer to see documentation
>>> improving within the code-base rather than in a separate wiki in the
>>> near-term and suggest we just delete this link.
>>>
>>> - the mailing list is NOT used for any form of discussion/information
>>> other
>>> than JIRA and Git(hub) notifications
>>> - especially the latter doesn't give a newbie any indication what is going
>>> on, nor how to (start) interact
>>>
>>> This is unfortunate and must change for the community to grow.  I know I'm
>>> guilty of having ad-hoc off-list discussions with my team that lead to new
>>> stories and pull requests.  Mea culpa, I resolve to stop doing this in
>>> 2016.
>>>
>>> - there are no blog posts (that I am aware of) around using/trying out
>>> Streams either
>>>
>>>
>>> This is a critical missing piece of the puzzle.  Static HTML via apache
>>> httpd is not the ideal platform for hosting a blog but some other projects
>>> have made it work.  I'd like us to replace 'Latest News' with a navigable,
>>> indexable set of blog posts going forward, where release summaries and new
>>> examples can be published, and 'in-the-wild' mentions of the project can
>>> be
>>> aggregated.
>>>
>>>
>>> And I personally would prefer the dev@ list to be much more / primarily
>>> used for actual human interaction, not just these notifications from JIRA
>>> and Github.
>>> Maybe change the configuration to delegate (some of) these to the commit@
>>> list instead?
>>>
>>>
>>> I agree.  AFAICT though commit@streams.incubator.apache.org does not
>>> exist.  Per http://apache.org/dev/committers.html#mail we should hold a
>>> vote to create it.  I'll open that vote today.
>>>
>>> What I suggest is to for a while stop (or largely postpone) working on
>>> code
>>> but instead working on any/all of the above points first...
>>>
>>>
>>> I agree that making the website helpful is the top project priority.
>>>
>>> Even working on a next release IMO is far less important than first
>>> building up some explanation *why* a release of Stream is needed.
>>>
>>>
>>> I agree that these concerns should be addressed before the next release.
>>> The main reason for releasing soon is that per [4] there are 36 issues
>>> resolved since the 0.2 release, including quite a few bugs and
>>> improvements
>>> related to stability, deployment flexibility, or likely to impact the
>>> experience of a new developer.
>>>
>>> [1]
>>>
>>> http://streams.incubator.apache.org/site/0.2-incubating-SNAPSHOT/streams-examples/
>>> [2]
>>>
>>> http://streams.incubator.apache.org/site/0.2-incubating/streams-project/apidocs/index.html
>>> [3]
>>>
>>> http://streams.incubator.apache.org/site/0.2-incubating/streams-project/index.html
>>> [4]
>>>
>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20STREAMS%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20in%20unreleasedVersions()
>>>
>>>
>>
>