incubator-allura-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Igor Bondarenko" <>
Subject [allura:tickets] #6534 Wiki importer for github
Date Mon, 16 Sep 2013 10:44:21 GMT
Ok, that's what I found about github-supported formats, that pypeline can't handle. We can
add support for couple of formats to pypeline pretty easily (asciidoc, mediawiki), but the
rest of them require pretty much work.

### ASCIIDoc: .asciidoc

Uses GPLv2

Requires installing asciidoc package system-wide. API is distributed as a standalone python
script, so should be included directly into pypeline repo, or installed manually.

I think pypeline support can be done in 2-3 cp.

### Org Mode: .org

There are couple of org mode parsing [libraries]( for python,
but it seems that all of them just parse org mode files into tree of Orgnode objects, and
there is no support for converting that into html. I'm not familiar with this format at all.
I think adding such support might be pretty heavy.

### Pod: .pod

Perl documentation system. Seems like only perl tool exists for converting this. Should be
possible to write a python-wraaper around command-line tool and use it in pypeline, but this
may take awhile.

### RDoc: .rdoc

Ruby documentation system. Seems like only ruby tool exists for converting this. Also wrapper
for command-line tool can be created, I think.

### MediaWiki: .mediawiki, .wiki

Uses GPLv3

Can add support to pypeline using [python-mediawiki](
Allura's `mediawiki2markdown` already uses it.

Also, can convert to Allura-markdown using `mediawiki2markdown`. Both cases shouldn't be hard
to implement. 1-2 cp, I guess.


** [tickets:#6534] Wiki importer for github**

**Status:** in-progress
**Labels:** import github 42cc 
**Created:** Wed Aug 07, 2013 09:54 PM UTC by Dave Brondsema
**Last Updated:** Fri Sep 13, 2013 09:11 PM UTC
**Owner:** nobody

Wikis are git repositories and can be accessed like `git clone`
for example.  Check the main repo API first to see if the repo has wiki enabled.  You can
see for reference as an example of another
wiki importer.  It is a separate repo because it needs the "html2text" package to convert
html to markdown, and that is a GPL library.

Github supports many markup types.  Find a full list and determine what the best way to convert
them to markdown is.  My guess is that few formats will have tools available to convert them
directly to markdown, so my likely recommendation would be to render them as HTML (using [pypeline](
as a generic way to handle many of those formats) and then html2text to get it into markdown.

If html2text or any other GPL library is needed, this will have to be a separate repo from
the main Allura repo.  So please evaluate & test the conversion options first, before
putting code into place.

A second phase to all this (i.e. do it separately, after the basic import is all working)
would be to handle revision history.  This would mean going through each commit in the wiki
git repo, and converting & updating every file that changes.  This may be very time consuming,
so when we get to it, we may want it to be a checkbox option, so users only do it if they
want it.


Sent from because is subscribed to

To unsubscribe from further messages, a project admin can change settings at
 Or, if this is a mailing list, you can unsubscribe from the mailing list.
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message