streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Franklin <m.ben.frank...@gmail.com>
Subject Re: Streams Web Site, Maven Sites, misc Documentation - proposed next steps
Date Mon, 02 May 2016 15:43:44 GMT
On Fri, Apr 29, 2016 at 6:22 PM Steve Blackmon <sblackmon@apache.org> wrote:

> The new website is now up at streams.incubator.apache.org.  It’s not
> perfect but it’s good enough and way better than it’s ever been.  It’s got
> a basic connection to Google Analytics - if you’d like to be added to the
> organization just ask.


This looks great!  I agree with Ate on a diagram or other visually engaging
component.


>
> I’m going to put together a blog post about website revamp and everything
> that there for the project blog at https://blogs.apache.org/streams/
>
> Hopefully we’ll also be able to tweet it out from @ApacheStreams if the
> PMC is able to get control of it (WIP)
>
>
> Steve Blackmon
> sblackmon@apache.org
>
> On Wed, Apr 27, 2016 at 3:28 PM Steve Blackmon <Steve Blackmon
> <Steve+Blackmon+%3Csblackmon@apache.org%3E>> wrote:
>
>> All,
>>
>> I think we are pretty close to pulling all of the work on STREAMS-401
>> (new web page) together into a pretty slick package.  Thanks for your
>> patience and feedback.
>>
>> Some new additions include diagrams on the architecture page and within
>> many modules depicting streams components and data flow among them and with
>> third-party systems, a 5-step tutorial to run a useful stream, consistent
>> look-and-feel across all three repos, integrated breadcrumbs, and
>> rudimentary github and twitter integrations.
>>
>> Feel free to browse these links to see where we’re at.
>>
>>
>> http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-master/index.html
>>
>> http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/index.html
>>
>> http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-examples/index.html
>>
>> All of the menu link now resolve, or will resolve as soon as everything
>> is promoted.  If you get any Not Available errors that you CAN’T resolve by
>> substituting ‘latest’ for ‘0.3-incubating-SNAPSHOT’ in the url, please let
>> me know.
>>
>> Additionally if you identify any major issues or have any last
>> suggestions please let me know before I kickoff the process to merge the
>> STREAMS-401 branches and promote the site later this week.
>>
>> Steve Blackmon
>> sblackmon@apache.org
>>
>> On Thu, Mar 03, 2016 at 9:48 AM Ate Douma <Ate Douma
>> <Ate+Douma+%3Cate@douma.nu%3E>> wrote:
>>
>>> +1 to everything below :)
>>>
>>> Good stuff Steve!
>>>
>>> On 2016-03-01 23:50, Steve Blackmon wrote:
>>> > Following up:
>>> >
>>> > I took a pass at cleaning up the release markdown files and they now
>>> look
>>> > mostly the same as they did under apache cms.
>>> >
>>> > I also confirmed that site-deploy can write to a sub-directory of
>>> > streams.incubator.apache.org using scm:svn capability.
>>> >
>>> > http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-master/index.html
>>>
>>> >
>>> > Recall that streams-project and streams-examples each publish schema
>>> > artifacts to their maven sites, so maintaining prior versions is
>>> important.
>>> > streams-master doesn't currently, but since it's a parent of those
>>> other
>>> > two and prior experience has led to believe we're better off using the
>>> same
>>> > scheme and setting the streams-master in the pom to be a sibling of
>>> > streams-project and streams-example rather that publishing it to a
>>> > different URL entirely. This is why the version an artifactId are
>>> being
>>> > included in the above url.
>>> >
>>> > Brings up two issues -
>>> > resolving the latest release and/or snapshot version at any given time
>>> > ensuring a visitor to streams.incubator.apache.org lands on the
>>> > streams-master site for that version
>>> >
>>> > While symlinks are a viable way to resolving the latest release and/or
>>> > snapshot version while keeping previous release versions around
>>> >
>>> > i.e. forwarding
>>> > http://streams.incubator.apache.org/site/latest/streams-master/index.html
>>>
>>> > to
>>> > http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-master/index.html
>>>
>>> >
>>> > they won't help us forward http://streams.incubator.apache.org to a
>>> > subdirectory of itself.
>>> >
>>> > Good news is apache cms supports .htaccess and the following snippet
>>> can.
>>> >
>>> > RewriteEngine *on*
>>> >
>>> > Redirect /
>>> > http://streams.incubator.apache.org/site/latest/streams-master/index.html
>>>
>>> > Feel free to confirm by visiting streams.staging.incubator.org
>>> (you'll be
>>> > redirected to the output of the maven site-deploy)
>>> >
>>> > *I ask that everyone on the list review and send me any concerns or
>>> > feedback they want incorporated before this goes live on
>>> > streams.incubator.apache.org <http://streams.incubator.apache.org>.*
>>> >
>>> > Procedurally, the way I expect that to happen is:
>>> > - I'll submit a PR to streams-master/master
>>> > - That PR will merge
>>> > - Update the streams-master jenkins job to run site-deploy, and do the
>>> > needful to get it the necessary credentials
>>> > - Going forward all builds of streams-master/master will publish to
>>> > site/${version}/streams-master
>>> > - Add a few more instructions to jenkins to maintain the symlinks and
>>> > .htaccess
>>> > - If there are any manual steps required to oversee, document that on
>>> the
>>> > site itself.
>>> >
>>> > Then, we'll move on to hooking streams-project and streams-examples up
>>> to
>>> > this new site (as children with an integrated breadcrumb) and to
>>> jenkins CD.
>>> >
>>> > Steve Blackmon
>>> > sblackmon@apache.org
>>> >
>>> > On Mon, Feb 29, 2016 at 5:19 PM, Steve Blackmon <sblackmon@apache.org>
>>>
>>> > wrote:
>>> >
>>> >> One thing which I think can 'spice up' the overview page a bit more
>>> would
>>> >>> be a
>>> >>> diagram providing some very high-level interaction between data
>>> sources
>>> >>> and
>>> >>> destinations and the streams role between them.
>>> >>
>>> >>
>>> >> I agree - a diagram depicting streams mediating between sources and
>>> >> destination on the landing page is appropriate. I'll work on this
>>> soon.
>>> >>
>>> >> And before replacing the current site several existing pages need to
>>> be
>>> >>> moved to
>>> >>> the new one. But once those are done I think this is great
>>> improvement.
>>> >>
>>> >>
>>> >> I've pushed a new version (to the source and site links from my prior
>>> >> message) that brings most of the existing content along, using the
>>> >> reporting-info plugin to generate the HTML where possible.
>>> >>
>>> >> On a few of the pages - SCM, CI, and Dependency Info - the output of
>>> the
>>> >> maven plugin simply isn't adequate for a multi-repository project
>>> such as
>>> >> this, so those are now managed within src/site/markdown
>>> >>
>>> >> Release Setup and Release Process need some cleanup but all of the
>>> content
>>> >> has been migrated.
>>> >>
>>> >> Other report-info pages - such as Dependencies, Dependency
>>> Management,
>>> >> Dependency Convergence, JavaDocs, Test JavaDocs, - don't make sense
>>> at the
>>> >> streams-master level but do exist and are useful within
>>> streams-project,
>>> >> streams-examples, and many of their respective sub-modules.
>>> >>
>>> >> In theory those additional report items will show up when navigating
>>> into
>>> >> the site hierarchy of the sub-projects (links to each sub-project are
>>> >> present along the header), which will inherit the structure,
>>> look-and-feel,
>>> >> and breadcrumb of this site revision. Naturally these changes must be
>>> >> deployed to streams.incubator.apache.org and sub-project pom.xml and
>>> >> site_en.xml will need some tweaking but I'm optimistic it will all
>>> fit
>>> >> together nicely.
>>> >>
>>> >> Steve Blackmon
>>> >> sblackmon@apache.org
>>> >>
>>> >> On Sat, Feb 27, 2016 at 2:27 AM, Ate Douma <ate@douma.nu> wrote:
>>> >>
>>> >>> On 2016-02-25 23:50, Steve Blackmon wrote:
>>> >>>
>>> >>>> Hello streamers,
>>> >>>>
>>> >>>> I finally consolidated some materials I've been working on into
a
>>> >>>> nice-looking thing that is a viable candidate to replace the
>>> current
>>> >>>> streams web page. Please take a look and let me know what you
>>> think.
>>> >>>> Sometime
>>> >>>> soon I expect to submit a vote to take down the previous site
and
>>> replace
>>> >>>> it with something derived from this version.
>>> >>>>
>>> >>>> The proposed new site is temporarily being hosted at:
>>> >>>>
>>> >>>>
>>> http://people.apache.org/~sblackmon/0.3-incubating-SNAPSHOT/streams-master/index.html
>>> >>>>
>>> >>>> I placed all of the content into markdown files which are checked
>>> into a
>>> >>>> branch of streams-master. This approach allows us to author
the
>>> site
>>> >>>> using
>>> >>>> markdown, package and deploy it with maven-site, and manage
changes
>>> using
>>> >>>> git.
>>> >>>>
>>> >>>
>>> >>> This looks cool!
>>> >>>
>>> >>> I like the new overview and faq pages, definitely more concrete
>>> about
>>> >>> Streams.
>>> >>>
>>> >>> One thing which I think can 'spice up' the overview page a bit more
>>> would
>>> >>> be a
>>> >>> diagram providing some very high-level interaction between data
>>> sources
>>> >>> and
>>> >>> destinations and the streams role between them.
>>> >>>
>>> >>> And before replacing the current site several existing pages need
to
>>> be
>>> >>> moved to
>>> >>> the new one. But once those are done I think this is great
>>> improvement.
>>> >>>
>>> >>>
>>> >>>> If you'd like to submit specific changes feel free to do so
as
>>> >>>> pull-requests to:
>>> >>>> https://github.com/steveblackmon/incubator-streams-master.git
>>> >>>>
>>> >>>> Referring back to the original list of goals, some are addressed
in
>>> whole
>>> >>>> or in part with this revision, others not just yet.
>>> >>>>
>>> >>>> ADDRESSED IN WHOLE:
>>> >>>> - the website isn't providing any practical guidance *why* or
*how*
>>> to
>>> >>>> use
>>> >>>> Streams
>>> >>>> - there is no public javadoc or other technical documentation
>>> >>>> linked/hosted
>>> >>>> - there is a 'wiki (coming soon)' link which never has been
>>> activated
>>> >>>>
>>> >>>> ADDRESSED IN PART:
>>> >>>> - its missing concrete examples and recognizable use-cases for
>>> which
>>> >>>> Streams might
>>> >>>> be used, nor comparison against other solutions, and why Streams
>>> would
>>> >>>> be a
>>> >>>> good/better solution
>>> >>>> - the Architecture page only has some basic wording but provides
no
>>> >>>> insight
>>> >>>> whatsoever about the concrete implementation and certainly not
its
>>> >>>> architecture
>>> >>>>
>>> >>>> This is just a start. Communication on the list has been dead
>>> lately,
>>> >>>> but
>>> >>>> I'm hoping to change that beginning with this effort to explain
>>> what is
>>> >>>> unique and valuable about Streams, and why. Hopefully this take
on
>>> the
>>> >>>> project will inspire others to get on-board and help out in
the
>>> coming
>>> >>>> months.
>>> >>>>
>>> >>>
>>> >>> +1, good work!
>>> >>>
>>> >>>
>>> >>>
>>> >>>> Steve Blackmon
>>> >>>> sblackmon@apache.org
>>> >>>>
>>> >>>> On Thu, Jan 7, 2016 at 10:49 AM, Ate Douma <ate@douma.nu>
wrote:
>>> >>>>
>>> >>>> Hi Steve,
>>> >>>>>
>>> >>>>> Cool. I don't have much extra input or ideas right now,
but
>>> looking
>>> >>>>> forward to the steps taken you proposed below.
>>> >>>>> And about getting out a next release first soon, if it is
not
>>> >>>>> side-tracking these other steps much, sure +1
>>> >>>>>
>>> >>>>> Ate
>>> >>>>>
>>> >>>>> On 2016-01-05 15:35, Steve Blackmon wrote:
>>> >>>>>
>>> >>>>> All,
>>> >>>>>>
>>> >>>>>> Ate gave us a reality check during our board report
in December
>>> and
>>> >>>>>> enumerated some of the most prominent ways our documentation
>>> currently
>>> >>>>>> falls short - of Apache community standards and of any
reasonable
>>> >>>>>> useful-ness test. I've responded in-line below - please
kick in
>>> any
>>> >>>>>> ideas
>>> >>>>>> you have.
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> - the website isn't providing any practical guidance
*why* or
>>> *how* to
>>> >>>>>> use
>>> >>>>>> Streams
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> I agree. This is the content that belongs on the streams
website
>>> >>>>>> landing
>>> >>>>>> page (high-level) or linked from the body (the details).
I will
>>> take a
>>> >>>>>> first shot at this and circulate for feedback.
>>> >>>>>>
>>> >>>>>> - its missing concrete examples and recognizable use-cases
for
>>> which
>>> >>>>>> Streams might be used, nor comparison against other
solutions,
>>> and why
>>> >>>>>> Streams would be a good/better solution
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> The examples documentation site [1] are meant to serve
as both
>>> concrete
>>> >>>>>> examples and recognizable use-cases, but there are a
few problems
>>> with
>>> >>>>>> the
>>> >>>>>> current setup.
>>> >>>>>>
>>> >>>>>> 1. The examples site lacks plain-language READMEs for
the root
>>> >>>>>> module
>>> >>>>>> and the first children of the root module (the runtime-specific
>>> >>>>>> aggregators).
>>> >>>>>> 2. The examples site isn't linked from the landing page
of the
>>> >>>>>> website.
>>> >>>>>> 3. There is no permanent url for latest/current.
>>> >>>>>> 4. There hasn't been a formal release of this repo.
>>> >>>>>> 5. There are broken links within the markdown of some
examples.
>>> >>>>>> 6. The plain-language descriptions of the examples themselves
>>> >>>>>> could be
>>> >>>>>>
>>> >>>>>> improved.
>>> >>>>>>
>>> >>>>>> All of these can be addressed (easily I think) and I'm
happy to
>>> take
>>> >>>>>> them
>>> >>>>>> on.
>>> >>>>>>
>>> >>>>>> Additionally, a page (or several) comparing streams
to frameworks
>>> such
>>> >>>>>> as
>>> >>>>>> Storm, Spark, Samza, Flink, etc... and describing how
they are
>>> >>>>>> different/complementary would be valuable. These questions
come
>>> up a
>>> >>>>>> lot.
>>> >>>>>>
>>> >>>>>> - the Architecture page only has some basic wording
but provides
>>> no
>>> >>>>>> insight
>>> >>>>>> whatsoever about the concrete implementation and certainly
not
>>> its
>>> >>>>>> architecture
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> I think this page needs either an multi-page outline
format
>>> heavily
>>> >>>>>> linked
>>> >>>>>> into READMEs and interfaces throughout the streams repos,
or a
>>> system
>>> >>>>>> diagram depicting a simple streams deployment alongside
the
>>> vocabulary.
>>> >>>>>> Maybe both as several pages.
>>> >>>>>>
>>> >>>>>> - there is no public javadoc or other technical documentation
>>> >>>>>> linked/hosted
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> We do generate java-docs for releases and snapshots.
We should
>>> add a
>>> >>>>>> link
>>> >>>>>> to the root page. We've made progress but there are
still many
>>> classes
>>> >>>>>> lacking any javadoc header. We should reiterate a commitment
to
>>> fixing
>>> >>>>>> that and a schedule for deprecating and deleting any
class that
>>> doesn't
>>> >>>>>> still serve a distinct purpose.
>>> >>>>>>
>>> >>>>>> An index of the many resource files (json/xml schemas,
conversion
>>> >>>>>> rules,
>>> >>>>>> shell scripts, etc...) published to the project and
examples
>>> release
>>> >>>>>> should
>>> >>>>>> also placed on the main website. Ideally comprehensive,
but if
>>> not at
>>> >>>>>> least enough to show a developer that they exist and
can be
>>> >>>>>> hard-linked to
>>> >>>>>> within their own project resources.
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> - there is a 'wiki (coming soon)' link which never has
been
>>> activated
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> This link (actually most of streams.incubator.apache.org)
>>> predates the
>>> >>>>>> regular publication of the streams-project maven site
where
>>> READMEs and
>>> >>>>>> other resources are hosted. Personally I'd prefer to
see
>>> documentation
>>> >>>>>> improving within the code-base rather than in a separate
wiki in
>>> the
>>> >>>>>> near-term and suggest we just delete this link.
>>> >>>>>>
>>> >>>>>> - the mailing list is NOT used for any form of
>>> discussion/information
>>> >>>>>> other
>>> >>>>>> than JIRA and Git(hub) notifications
>>> >>>>>> - especially the latter doesn't give a newbie any indication
what
>>> is
>>> >>>>>> going
>>> >>>>>> on, nor how to (start) interact
>>> >>>>>>
>>> >>>>>> This is unfortunate and must change for the community
to grow. I
>>> know
>>> >>>>>> I'm
>>> >>>>>> guilty of having ad-hoc off-list discussions with my
team that
>>> lead to
>>> >>>>>> new
>>> >>>>>> stories and pull requests. Mea culpa, I resolve to stop
doing
>>> this in
>>> >>>>>> 2016.
>>> >>>>>>
>>> >>>>>> - there are no blog posts (that I am aware of) around
>>> using/trying out
>>> >>>>>> Streams either
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> This is a critical missing piece of the puzzle. Static
HTML via
>>> apache
>>> >>>>>> httpd is not the ideal platform for hosting a blog but
some
>>> other
>>> >>>>>> projects
>>> >>>>>> have made it work. I'd like us to replace 'Latest News'
with a
>>> >>>>>> navigable,
>>> >>>>>> indexable set of blog posts going forward, where release
>>> summaries and
>>> >>>>>> new
>>> >>>>>> examples can be published, and 'in-the-wild' mentions
of the
>>> project
>>> >>>>>> can
>>> >>>>>> be
>>> >>>>>> aggregated.
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> And I personally would prefer the dev@ list to be much
more /
>>> >>>>>> primarily
>>> >>>>>> used for actual human interaction, not just these notifications
>>> from
>>> >>>>>> JIRA
>>> >>>>>> and Github.
>>> >>>>>> Maybe change the configuration to delegate (some of)
these to the
>>> >>>>>> commit@
>>> >>>>>> list instead?
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> I agree. AFAICT though commit@streams.incubator.apache.org
does
>>> not
>>> >>>>>> exist. Per http://apache.org/dev/committers.html#mail
we should
>>> hold
>>> >>>>>> a
>>> >>>>>> vote to create it. I'll open that vote today.
>>> >>>>>>
>>> >>>>>> What I suggest is to for a while stop (or largely postpone)
>>> working on
>>> >>>>>> code
>>> >>>>>> but instead working on any/all of the above points first...
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> I agree that making the website helpful is the top project
>>> priority.
>>> >>>>>>
>>> >>>>>> Even working on a next release IMO is far less important
than
>>> first
>>> >>>>>> building up some explanation *why* a release of Stream
is needed.
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> I agree that these concerns should be addressed before
the next
>>> >>>>>> release.
>>> >>>>>> The main reason for releasing soon is that per [4] there
are 36
>>> issues
>>> >>>>>> resolved since the 0.2 release, including quite a few
bugs and
>>> >>>>>> improvements
>>> >>>>>> related to stability, deployment flexibility, or likely
to impact
>>> the
>>> >>>>>> experience of a new developer.
>>> >>>>>>
>>> >>>>>> [1]
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> http://streams.incubator.apache.org/site/0.2-incubating-SNAPSHOT/streams-examples/
>>>
>>> >>>>>> [2]
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> http://streams.incubator.apache.org/site/0.2-incubating/streams-project/apidocs/index.html
>>>
>>> >>>>>> [3]
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> http://streams.incubator.apache.org/site/0.2-incubating/streams-project/index.html
>>>
>>> >>>>>> [4]
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20STREAMS%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20in%20unreleasedVersions()
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>
>>> >>>>
>>> >>>
>>> >>
>>> >
>>>
>>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message