Mailing-List: contact dev-help@lucy.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucy.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: <2B79A5C0-4637-4EE0-93D7-1D5C9C6FF889@justatheory.com>
References: 
 <CAAS6=7iJuWF-4M1BTnYNsEWjWmrWT00kSYRpzYNzq9FuqUgbuw@mail.gmail.com>
	<97A7DC4C-8524-490F-92B6-1251A86DC1A8@justatheory.com>
	<CAAS6=7jWN0-Wv5wBnY_whKDE5ySXX25KeiKMwpjwVx+L3QpgiQ@mail.gmail.com>
	<2B79A5C0-4637-4EE0-93D7-1D5C9C6FF889@justatheory.com>
Date: Thu, 17 May 2012 20:18:39 -0700
Message-ID: 
 <CAAS6=7i_c_n3mKRd+4o-aANPyas7zJo8DJJ=rYB0jWLfBDBnOw@mail.gmail.com>
From: Marvin Humphrey <marvin@rectangular.com>
To: dev@lucy.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Subject: Re: [lucy-dev] Markdown for documentation

Hi David,

As this discussion has progressed, I've been having second thoughts about
Markdown.  Perhaps we should consider reStructuredText as an alternative.

In my opinion, our number one priority should be to commit to an external spec
that is carved in stone.  Debating lightweight markup syntax might be even
less fun than debating query parser syntax. :P  By implementing an
authoritative spec, we limit the scope of debate and shut down a lot of
bikeshedding before it starts.

To my mind, the stability of the Markdown spec is one of its most
attractive features.  However, after further investigation, I have come to
understand that the spec is rarely implemented as-is and that the Markdown
community is deeply Balkanized and embittered on the subject of extending
it[1].  I'm uneasy about inviting Markdown's controversies into our house.

My preference is still for vanilla Markdown, more or less.  To simplify
things, let's tweak the proposal to relegate @tags to the end of the comment,
eliminating inline @tag expansion.

    A DocuComment has two sections.

        body
            Description, formatted in Markdown.  The first sentence must be a
            summary that can stand on its own.  Terminated by the first line
            which begins with an "@" character or by the end of the comment.

        tags
            The comment may end with an optional tags block.  Each tag begins
            with a supported @label at the beginning of a line, and is
            terminated by either the next tag or the end of the comment.

There's only one Markdown extension I believe we can't live without: links
without a protocol should be interpreted as API links.

    @param query A [Query](Lucy::Search::Query).

Then there's one more change I'd consider optional, since nearly all Markdown
implemenations have it: disable intra-word emphasis so that foo_bar_baz
doesn't become foo<em>bar</em>baz.  I don't really care one way or the other
because 99% of the time `foo_bar_baz` should be enclosed in backticks.

I realize that by adding those two extensions we start down the slippery
slope...  IMO, that's an argument against Markdown...

I hope that's good enough.  If it's not -- if we believe that ultimately it
will be necessary to add additional extensions -- then I think we ought to
consider reStructuredText instead.

    http://en.wikipedia.org/wiki/ReStructuredText

>From a high level, the syntactical differences between reStructuredText and
extended Markdown are IMO superficial and the feature sets are similar -- but
reStructuredText has a canonical spec which has been stable for 10 years.
reStructuredText is like extended Markdown, minus the bikeshedding and the
resentment towards the original author.

> I prefer definition lists to bulleted lists, myself.

Fortunately, the choice of whether we expand @param tags to bullet-lists or
definition-lists is an implementation detail which is separate from whether we
support lightweight markup within comments.  Let's leave this topic for
another day.

> Well, MultiMarkdown is pretty well accepted as the #2 definition. Most
> implementations follow its examples (FBOFW).

I'm skeptical.  From what I can tell, "GitHub Flavored Markdown" -- a
particularly destructive variant given how incompatible it is with the
original Markdown spec -- looks like a significant competitor.

And here's what happened when someone tried to assert the dominance of
MultiMarkdown on the markdown-discuss list last October:

    http://six.pairlist.net/pipermail/markdown-discuss/2011-October/002342.html

    > pandoc assessed 'em all, and decided on multimarkdown.

    Huh? No. Pandoc's markdown extensions aren't the same as mmd's. I have
    explained in earlier posts why I don't like mmd's table and metadata
    syntax. Pandoc goes back to 2005 and was not influenced by mmd.

Adjudicating internet popularity contests is tiresome. :P

> Note that Sundown supports quite a lot more than just vanilla Markdown. How
> would you enforce that?

It looks like you have to enable syntax extensions manually in Sundown -- so
we "enforce" vanilla Markdown simply by not enabling those extensions.

However, I'm concerned that choosing Sundown may set us up for failure because
it means that we'll have this argument again later with someone who is
*outraged* to discover that we've got their pet Markdown extension disabled.

That's why I would like to determine whether we have consensus about forgoing
additional Markdown extensions.  If we don't have consensus, we should
evaluate reStructuredText.

Marvin Humphrey

[1] http://six.pairlist.net/pipermail/markdown-discuss/2008-March/001163.html
    http://www.codinghorror.com/blog/2009/12/responsible-open-source-code-parenting.html