streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Blackmon <sblack...@apache.org>
Subject Re: Streams Web Site, Maven Sites, misc Documentation - proposed next steps
Date Tue, 01 Mar 2016 22:50:12 GMT
Following up:

I took a pass at cleaning up the release markdown files and they now look
mostly the same as they did under apache cms.

I also confirmed that site-deploy can write to a sub-directory of
streams.incubator.apache.org using scm:svn capability.

http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-master/index.html

Recall that streams-project and streams-examples each publish schema
artifacts to their maven sites, so maintaining prior versions is important.
streams-master doesn't currently, but since it's a parent of those other
two and prior experience has led to believe we're better off using the same
scheme and setting the streams-master <url/> in the pom to be a sibling of
streams-project and streams-example rather that publishing it to a
different URL entirely.  This is why the version an artifactId are being
included in the above url.

Brings up two issues -
resolving the latest release and/or snapshot version at any given time
ensuring a visitor to streams.incubator.apache.org lands on the
streams-master site for that version

While symlinks are a viable way to resolving the latest release and/or
snapshot version while keeping previous release versions around

i.e. forwarding
http://streams.incubator.apache.org/site/latest/streams-master/index.html
to
http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-master/index.html

they won't help us forward http://streams.incubator.apache.org to a
subdirectory of itself.

Good news is apache cms supports .htaccess and the following snippet can.

RewriteEngine *on*

Redirect /
http://streams.incubator.apache.org/site/latest/streams-master/index.html
Feel free to confirm by visiting streams.staging.incubator.org (you'll be
redirected to the output of the maven site-deploy)

*I ask that everyone on the list review and send me any concerns or
feedback they want incorporated before this goes live on
streams.incubator.apache.org <http://streams.incubator.apache.org>.*

Procedurally, the way I expect that to happen is:
- I'll submit a PR to streams-master/master
- That PR will merge
- Update the streams-master jenkins job to run site-deploy, and do the
needful to get it the necessary credentials
- Going forward all builds of streams-master/master will publish to
site/${version}/streams-master
- Add a few more instructions to jenkins to maintain the symlinks and
.htaccess
- If there are any manual steps required to oversee, document that on the
site itself.

Then, we'll move on to hooking streams-project and streams-examples up to
this new site (as children with an integrated breadcrumb) and to jenkins CD.

Steve Blackmon
sblackmon@apache.org

On Mon, Feb 29, 2016 at 5:19 PM, Steve Blackmon <sblackmon@apache.org>
wrote:

> One thing which I think can 'spice up' the overview page a bit more would
>> be a
>> diagram providing some very high-level interaction between data sources
>> and
>> destinations and the streams role between them.
>
>
> I agree - a diagram depicting streams mediating between sources and
> destination on the landing page is appropriate.  I'll work on this soon.
>
> And before replacing the current site several existing pages need to be
>> moved to
>> the new one. But once those are done I think this is great improvement.
>
>
> I've pushed a new version (to the source and site links from my prior
> message) that brings most of the existing content along, using the
> reporting-info plugin to generate the HTML where possible.
>
> On a few of the pages - SCM, CI, and Dependency Info - the output of the
> maven plugin simply isn't adequate for a multi-repository project such as
> this, so those are now managed within src/site/markdown
>
> Release Setup and Release Process need some cleanup but all of the content
> has been migrated.
>
> Other report-info pages - such as Dependencies, Dependency Management,
> Dependency Convergence, JavaDocs, Test JavaDocs,  - don't make sense at the
> streams-master level but do exist and are useful within streams-project,
> streams-examples, and many of their respective sub-modules.
>
> In theory those additional report items will show up when navigating into
> the site hierarchy of the sub-projects (links to each sub-project are
> present along the header), which will inherit the structure, look-and-feel,
> and breadcrumb of this site revision.  Naturally these changes must be
> deployed to streams.incubator.apache.org and sub-project pom.xml and
> site_en.xml will need some tweaking but I'm optimistic it will all fit
> together nicely.
>
> Steve Blackmon
> sblackmon@apache.org
>
> On Sat, Feb 27, 2016 at 2:27 AM, Ate Douma <ate@douma.nu> wrote:
>
>> On 2016-02-25 23:50, Steve Blackmon wrote:
>>
>>> Hello streamers,
>>>
>>> I finally consolidated some materials I've been working on into a
>>> nice-looking thing that is a viable candidate to replace the current
>>> streams web page.  Please take a look and let me know what you think.
>>> Sometime
>>> soon I expect to submit a vote to take down the previous site and replace
>>> it with something derived from this version.
>>>
>>> The proposed new site is temporarily being hosted at:
>>>
>>> http://people.apache.org/~sblackmon/0.3-incubating-SNAPSHOT/streams-master/index.html
>>>
>>> I placed all of the content into markdown files which are checked into a
>>> branch of streams-master.  This approach allows us to author the site
>>> using
>>> markdown, package and deploy it with maven-site, and manage changes using
>>> git.
>>>
>>
>> This looks cool!
>>
>> I like the new overview and faq pages, definitely more concrete about
>> Streams.
>>
>> One thing which I think can 'spice up' the overview page a bit more would
>> be a
>> diagram providing some very high-level interaction between data sources
>> and
>> destinations and the streams role between them.
>>
>> And before replacing the current site several existing pages need to be
>> moved to
>> the new one. But once those are done I think this is great improvement.
>>
>>
>>> If you'd like to submit specific changes feel free to do so as
>>> pull-requests to:
>>> https://github.com/steveblackmon/incubator-streams-master.git
>>>
>>> Referring back to the original list of goals, some are addressed in whole
>>> or in part with this revision, others not just yet.
>>>
>>> ADDRESSED IN WHOLE:
>>> - the website isn't providing any practical guidance *why* or *how* to
>>> use
>>> Streams
>>> - there is no public javadoc or other technical documentation
>>> linked/hosted
>>> - there is a 'wiki (coming soon)' link which never has been activated
>>>
>>> ADDRESSED IN PART:
>>> - its missing concrete examples and recognizable use-cases for which
>>> Streams might
>>> be used, nor comparison against other solutions, and why Streams would
>>> be a
>>> good/better solution
>>> - the Architecture page only has some basic wording but provides no
>>> insight
>>> whatsoever about the concrete implementation and certainly not its
>>> architecture
>>>
>>> This is just a start.  Communication on the list has been dead lately,
>>> but
>>> I'm hoping to change that beginning with this effort to explain what is
>>> unique and valuable about Streams, and why.  Hopefully this take on the
>>> project will inspire others to get on-board and help out in the coming
>>> months.
>>>
>>
>> +1, good work!
>>
>>
>>
>>> Steve Blackmon
>>> sblackmon@apache.org
>>>
>>> On Thu, Jan 7, 2016 at 10:49 AM, Ate Douma <ate@douma.nu> wrote:
>>>
>>> Hi Steve,
>>>>
>>>> Cool. I don't have much extra input or ideas right now, but looking
>>>> forward to the steps taken you proposed below.
>>>> And about getting out a next release first soon, if it is not
>>>> side-tracking these other steps much, sure +1
>>>>
>>>> Ate
>>>>
>>>> On 2016-01-05 15:35, Steve Blackmon wrote:
>>>>
>>>> All,
>>>>>
>>>>> Ate gave us a reality check during our board report in December and
>>>>> enumerated some of the most prominent ways our documentation currently
>>>>> falls short - of Apache community standards and of any reasonable
>>>>> useful-ness test.  I've responded in-line below - please kick in any
>>>>> ideas
>>>>> you have.
>>>>>
>>>>>
>>>>> - the website isn't providing any practical guidance *why* or *how* to
>>>>> use
>>>>> Streams
>>>>>
>>>>>
>>>>> I agree.  This is the content that belongs on the streams website
>>>>> landing
>>>>> page (high-level) or linked from the body (the details).  I will take
a
>>>>> first shot at this and circulate for feedback.
>>>>>
>>>>> - its missing concrete examples and recognizable use-cases for which
>>>>> Streams might be used, nor comparison against other solutions, and why
>>>>> Streams would be a good/better solution
>>>>>
>>>>>
>>>>> The examples documentation site [1] are meant to serve as both concrete
>>>>> examples and recognizable use-cases, but there are a few problems with
>>>>> the
>>>>> current setup.
>>>>>
>>>>>      1. The examples site lacks plain-language READMEs for the root
>>>>> module
>>>>>      and the first children of the root module (the runtime-specific
>>>>>      aggregators).
>>>>>      2. The examples site isn't linked from the landing page of the
>>>>> website.
>>>>>      3. There is no permanent url for latest/current.
>>>>>      4. There hasn't been a formal release of this repo.
>>>>>      5. There are broken links within the markdown of some examples.
>>>>>      6. The plain-language descriptions of the examples themselves
>>>>> could be
>>>>>
>>>>>      improved.
>>>>>
>>>>> All of these can be addressed (easily I think) and I'm happy to take
>>>>> them
>>>>> on.
>>>>>
>>>>> Additionally, a page (or several) comparing streams to frameworks such
>>>>> as
>>>>> Storm, Spark, Samza, Flink, etc... and describing how they are
>>>>> different/complementary would be valuable.  These questions come up a
>>>>> lot.
>>>>>
>>>>> - the Architecture page only has some basic wording but provides no
>>>>> insight
>>>>> whatsoever about the concrete implementation and certainly not its
>>>>> architecture
>>>>>
>>>>>
>>>>> I think this page needs either an multi-page outline format heavily
>>>>> linked
>>>>> into READMEs and interfaces throughout the streams repos, or a system
>>>>> diagram depicting a simple streams deployment alongside the vocabulary.
>>>>> Maybe both as several pages.
>>>>>
>>>>> - there is no public javadoc or other technical documentation
>>>>> linked/hosted
>>>>>
>>>>>
>>>>> We do generate java-docs for releases and snapshots.  We should add a
>>>>> link
>>>>> to the root page.  We've made progress but there are still many classes
>>>>> lacking any javadoc header.  We should reiterate a commitment to fixing
>>>>> that and a schedule for deprecating and deleting any class that doesn't
>>>>> still serve a distinct purpose.
>>>>>
>>>>> An index of the many resource files (json/xml schemas, conversion
>>>>> rules,
>>>>> shell scripts, etc...) published to the project and examples release
>>>>> should
>>>>> also placed on the main website.  Ideally comprehensive, but if not at
>>>>> least enough to show a developer that they exist and can be
>>>>> hard-linked to
>>>>> within their own project resources.
>>>>>
>>>>>
>>>>> - there is a 'wiki (coming soon)' link which never has been activated
>>>>>
>>>>>
>>>>> This link (actually most of streams.incubator.apache.org) predates the
>>>>> regular publication of the streams-project maven site where READMEs and
>>>>> other resources are hosted.  Personally I'd prefer to see documentation
>>>>> improving within the code-base rather than in a separate wiki in the
>>>>> near-term and suggest we just delete this link.
>>>>>
>>>>> - the mailing list is NOT used for any form of discussion/information
>>>>> other
>>>>> than JIRA and Git(hub) notifications
>>>>> - especially the latter doesn't give a newbie any indication what is
>>>>> going
>>>>> on, nor how to (start) interact
>>>>>
>>>>> This is unfortunate and must change for the community to grow.  I know
>>>>> I'm
>>>>> guilty of having ad-hoc off-list discussions with my team that lead to
>>>>> new
>>>>> stories and pull requests.  Mea culpa, I resolve to stop doing this in
>>>>> 2016.
>>>>>
>>>>> - there are no blog posts (that I am aware of) around using/trying out
>>>>> Streams either
>>>>>
>>>>>
>>>>> This is a critical missing piece of the puzzle.  Static HTML via apache
>>>>> httpd is not the ideal platform for hosting a blog but some other
>>>>> projects
>>>>> have made it work.  I'd like us to replace 'Latest News' with a
>>>>> navigable,
>>>>> indexable set of blog posts going forward, where release summaries and
>>>>> new
>>>>> examples can be published, and 'in-the-wild' mentions of the project
>>>>> can
>>>>> be
>>>>> aggregated.
>>>>>
>>>>>
>>>>> And I personally would prefer the dev@ list to be much more /
>>>>> primarily
>>>>> used for actual human interaction, not just these notifications from
>>>>> JIRA
>>>>> and Github.
>>>>> Maybe change the configuration to delegate (some of) these to the
>>>>> commit@
>>>>> list instead?
>>>>>
>>>>>
>>>>> I agree.  AFAICT though commit@streams.incubator.apache.org does not
>>>>> exist.  Per http://apache.org/dev/committers.html#mail we should hold
>>>>> a
>>>>> vote to create it.  I'll open that vote today.
>>>>>
>>>>> What I suggest is to for a while stop (or largely postpone) working on
>>>>> code
>>>>> but instead working on any/all of the above points first...
>>>>>
>>>>>
>>>>> I agree that making the website helpful is the top project priority.
>>>>>
>>>>> Even working on a next release IMO is far less important than first
>>>>> building up some explanation *why* a release of Stream is needed.
>>>>>
>>>>>
>>>>> I agree that these concerns should be addressed before the next
>>>>> release.
>>>>> The main reason for releasing soon is that per [4] there are 36 issues
>>>>> resolved since the 0.2 release, including quite a few bugs and
>>>>> improvements
>>>>> related to stability, deployment flexibility, or likely to impact the
>>>>> experience of a new developer.
>>>>>
>>>>> [1]
>>>>>
>>>>>
>>>>> http://streams.incubator.apache.org/site/0.2-incubating-SNAPSHOT/streams-examples/
>>>>> [2]
>>>>>
>>>>>
>>>>> http://streams.incubator.apache.org/site/0.2-incubating/streams-project/apidocs/index.html
>>>>> [3]
>>>>>
>>>>>
>>>>> http://streams.incubator.apache.org/site/0.2-incubating/streams-project/index.html
>>>>> [4]
>>>>>
>>>>>
>>>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20STREAMS%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20in%20unreleasedVersions()
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message