forrest-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ross Gardler <>
Subject Re: [PROPOSAL] A CMS for our Docs
Date Tue, 07 Jun 2005 10:30:37 GMT
David Crossley wrote:
> Ross Gardler wrote:
>>David Crossley wrote:
>>>How will the changed content get back into our SVN?
>>For development, it doesn't. Daisy is fully version controlled (and I 
>>think Lenya is). For publishing we do just as we are doing now.
> I don't yet see the distinction. ASF projects have assets:
> e.g. code, docs, etc. All assets are stored in SVN. So i would think
> that the sources for all docs are stored in ASF repositories.

This is the position that Cocoon started with. However, it is simply not 
try, not all assets are in SVN, mailing list archives and wiki documents 
being the two most relevant to this situation. This has resulted in 
Cocoon agreeing that the docs don't need to go into SVN.

However, I have to admit, I am more of a mind that the *published* docs 
should go into SVN. So here is my proposal:

The CMS is a "super wiki" - loosely controlled, low barrier to entry - 
most importantly *not* published as official docs

The published docs are managed by Forrest as normal using the 
locationmap to bring content from both SVN and the CMS
(this step may not be necessary if Lenya can use SVN as a repository)

Periodically we do a "forrest site" and commit the generated docs to SVN

The advantage of this approach is that a tight integration between Lenya 
and SVN can allow people to edit the docs offline.

>>>How is the workflow managed to get the content reviewed
>>>and then published onto the website? At the moment we
>>>do have a workflow of sorts.
>>Any registered user can edit. Only committers can publish. The Forrest 
>>generated site is only created from published changes.
>>In other words, the same workflow as at present but we replace the edit 
>>document/create patch/submit patch/review patch/apply patch process with 
>>edit document/review edit/publish edit process.
> Perhaps we are using different terminology here. I see that
> committers review/edit and then "commit" changes. The "publish"
> is a separate process whereby a group of committed edits
> is now ready and the documents are generated and moved to the
> actual website.

OK, there are two distinct processes:

One for the wiki docs:

edit in wiki -> review edit in wiki -> publish edit in wiki
     /|\                /|\                     /|\
      |                  |                       |
(almost) anyone   Docs Committers          Docs Committers

and one for the published docs:

review wiki docs -> publish in Forrest site
      /|\                    /|\
       |                      |
   committers             committers

Note I have added a new type of committer here. One with a lower barrier 
to entry. We don't have to do this we could just have the existing 
structure, but it is an option.

>>>I also agree with Ferdinand. We would need a structure to
>>>add these docs, or we end up with the un-manageable mess
>>>that they call Wiki.
> Clarification: When i said mess/Wiki then i did not specifically
> mean Daisy or whatever. I refer to the problem where any user can
> create a new document, which probably repeats content in other
> documents. Most wikis that i have seen, have an unstructured
> collection of repetitive documents.

Yes, that was my understanding of what you said. Please understand we 
have *two* documentation tools, the "in development" tools and the 
"publishing" tools. Using these tools we can manage the quality of our 
published docs whilst at the same time allowing people to just jump in 
and get their feet wet. Take my recent discussion with Tim regarding 
locationmaps and the cocoon: protocol. In those mails there is alot of 
really valuable documentation, in the case of the cocoon: protocol it is 
valuable to Forrest and Cocoon.

If Tim could just cut and paste the text into a WYSIWYM editor I believe 
we would have a decent start to a doc. Later a dev says OK, that Doc is 
pretty good, I'll add it into the published site with a few additional 
comments and fixme's.

>>The idea is that the CMS is freeform like a wiki, but the published docs 
>>only come from an Forrest generated site. In other words, we use the 
>>Wiki to allow anyone to add content/edit content. We optimise the wiki 
>>for the editors of content (as opposed to the users of content), that is 
>>committers and technical users will likely use the Wiki as the primary 
>>souce of documentation. Whilst users will use the published website.
> What will actually happen is that users will come to this preparation
> area to find the "most up-to-date docs". I suppose that we can have
> big red warning banners.

Well, right now we have 0.6 and 0.7-dev docs on the site, is it a 
problem that some users are coming to the 0.7-dev docs?

Incidentally, I still think we should continue with publishing the next 
version docs in this way.

>>The published docs are managed using Forrest. Only those documents that 
>>are of sufficient quality make it into the published docs that appear in 
>>our distributions and on our website.
> After that, how do those "finished" documents get edited again?

In the CMS just like before. Our published docs are generated from the 
CMS repository, this is no different from the current situation in which 
we publish from our SVN repository.

>>In other words, we have the chaos of constantly evolving docs under the 
>>surface, on the surface we have nice ordered, version controlled and 
>>managed publications for our users. The migration path between the two 
>>is the close attention to detali of the review process.
> Oh dear, i am feeling overwhelmed with the thought of chaos.


I mean "managed chaos". Would you call Wikipedia chaotic? I would it's 
madness in there. Spam all over the place, random edits by unknown 
people, duplication across articles. However, at the end of the day it 
is an excellent resource because it is managed well.

Then look at most of their content comes from 
wikipedia. What they do is take a dump of the wikipedia database, do 
some quality assurance work and publish it. What you end up with is the 
quality of Wikipedia without the chaos.

This is the same model as my proposal here.

>>>Will we see sensible diffs, so that the committers can
>>>oversee the content or do we get the voluminous diffs
>>>like wiki where you cannot see what has been changed?
>>>e.g. most paragraphs are one long line, so diffs are useless.
>>Yes (at least for Daisy, don't know about Lenya). You can play around by 
>>looking at the versions on Daisy home page at 
> That helps a bit. What i was more concerned about is that diffs
> need to come to a mailing list, because committers need to review
> them as they come in. We cannot go off to a website to see if there
> have been any changes.

It's better than that ;-)

(with Daisy, Gregor has already acknowledge Lenya doesn't do diffs this 
is a *big* problem)

With Daisy each registered user is able to subscribe to particular 
pages, pages within a given collection or all pages. This means that you 
could opt to have changes to a particular set of pages mailed to you 
personally. i.e. you can take "ownership" of a portion of the documentation.

We could still have the mailing list receive all change emails as well.

> Look at the diffs that come from Cocoon Wiki. Whenever there
> is one long paragraph (common on a wiki) then email diffs
> are impossible to monitor.

Yes, this is one of the advantages of the fact that the Daisy wiki 
tidies the edited content. you always get good diffs regardless of how 
the author wrote the content.

>>>Who will be able to add content? Anyone? Do they need to
>>>be a committer? Do they need to be on forrest-dev?
>>My proposal is:
>>1) any visitor can add comments (see Comments link at bottom of 
>>2) any registered visitor can make edits, but their edits are not 
>>published automaticall. People can self register but an email address 
>>and confirmation email is used so we can tackle spammers
> Not sure yet about the self-register thing. The day will come
> when there is a disruptive person (perhaps well-intentioned) that
> creates unnecessary work by continually changing stuff that they
> do not understand.

Yes, but everything is version controlled. Since those without "docs 
commit" access cannot publish (even to the wiki), they can only edit. A 
change is reviewed before it is published (to the wiki) and then 
(optionally) reviewed again before published to the official docs.

But this can be changed at any time in the future, so no need to worry now.


>>The Lenya team seem a more willing to work *with* us on integrating 
>>Forrest and Daisy. I have discussed the Daisy plugin on the Daisy dev 
>>list, I offered to donate the code to them and work on it over there 
>>since they have static only publication and books on their roadmap (both 
>>of which can be achieved with Daisy + Forrest). However, they were not 
>>interested, preferring insted to build their own system that was not 
>>dependant on Forrest. Since I need the multiple input formats of Forrest 
>>that leaves me on my own here :-(
> That does not sound very constructive. I thought that your suggestion was
> that if a content managemnent system simply exposes the source, then
> Forrest can process it. There is no "dependency".

That is exactly my proposal - no dependency. However, if Daisy were to 
adopt Forrest as the *only* publishing engine then there would be a 
dependency in the sense they could not do static publishing without 
Forrest. Since they do not need the multiple input formats they are not 
interested in using Forrest as the publishing engine.

You can see the discussion, if you are interested, at:

And the final position of the Daisy community is made cleat in another 

As you can see from this message it boils down to different use cases 
between the Daisy community and the ASF.


>>It is my thinking that since Forrest is used by many projects within 
>>Apache we should take it upon ourselves to develop and test a system 
>>intended to be used across Apache. When we agree on how to do this we 
>>should announce this on Infrastructure@a.o and invite interested parties 
>>to come and help. As long as we have enough people committed here then 
>>it will not matter if (when?) they do not come.
> Lets try then. However i don't suggest "announcing" anything to
> Infrastructure. There would surely be a requirement that has not
> been addressed that sends us back to the drawing board. It would
> be better to work with them.

My thinking precisely. I'd rather go to infra with the start of a solution.

>>I understand. I've just had these same discussions (minus the infra@a.o 
>>part) over at Cocoon so most of these issues have been discussed and 
>>answered from the Cocoon perspective. Of course, that may be different 
>>from the Forrest perspective.
> Sounds like duplication of effort to me.

Not at all, since I have offered to work on the integration of the Daisy 
repository for them, which I am doing for a client. This integration 
involves enhancements to Forrest (I want to use views for this as well, 
my clients use case involves pulling materials from different 
repositories in a single page). Because it involves work on Forrest I 
would rather be doing it here and passing the finished results to them 

The only duplication is the installation of a CMS, which is why I 
proposed Daisy over Lenya. But if Lenya is going to collaborate with us, 
as I have said, I am willing to work with them too.


View raw message