incubator-allura-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cory Johns <>
Subject Re: Importer framework
Date Fri, 12 Jul 2013 18:31:08 GMT
The main use for the source, other than display purposes, is to group
importers for something like a full-project import.  In that case, the
full-project import will likely have all or most of the importers in a
single library, establishing a "canonical" value for that source.  The case
of importers growing organically with a full-project import being created
post-hoc seems less likely, though certainly possible, and anyway, the
full-project import could use the most common value if there's disagreement
and just require the differing ones to update to be included.  Or create
wrappers or something.  I assumed the values to be used for display, but
maybe we should have a separate title property and recommend that the
source property match the entry-point name for the full-project importer.

As for the status, is expecting a get_status(project) method on the
importer sufficient?  As for the return value, I don't expect that
partial-progress status information will be easy to get, but we could
define the get_status() return value to be a 0.0 to 1.0 numeric value to
support it, with 0.0 being not started, 0.5 being in-progress, and 1.0
being complete, and any further resolution in between a (rough) percentage.

On Thu, Jul 11, 2013 at 6:26 PM, Dave Brondsema <> wrote:

> Sounds pretty good to me.  It'll also enable us to more easily move the
> MediaWiki importer into a separate package.  It depends on some GPL code so
> it's already optional, but would be cleaner to have it completely separate.
>  Right now it's just a paster script, I think.
> Do we want to have anything more formal than a string for the source?  As
> long as all the importers follow the same conventions (Google? GoogleCode?)
> it'll probably be fine.
> The backend flexibility for each importer will be good.  It might be nice
> to have some lightweight integration points there too, as we build more
> features into them.  For example, we'll probably want to show the status of
> any import.  If we have a generic way to do reference that (e.g. a task id,
> or something fancier for incremental progress within a task), then we could
> show the status for any type of import.  That might actually be even more
> generally useful than imports -- I'm thinking it'd be nice for the data
> export/backup tasks too.  Anyway, it'll depend on how implementation goes
> and how the details get worked out.  Can't say yet exactly what it'd look
> like.
> On Thu, Jul 11, 2013 at 2:43 PM, Cory Johns <
> >wrote:
> > We're beginning to focus on improving and expanding our support for
> > importing and exporting project data to and from Allura, and as part of
> > that we need to develop an extensible framework for adding new import
> > mechanisms to Allura and exposing them in the UI.
> >
> > New importers will need to be able to advertise both what tool they are
> > able to import data for as well as what external source they will import
> > their data from.  The discovery mechanism will need to be able to list
> the
> > importers available by tool type, so that a given tool can provide a UI
> to
> > import from any supported source, or by source, so that we can provide an
> > integrated mechanism to import an entire project and all supported data
> > from a source.
> >
> > Entry points are an obvious choice for discovering importers, whether
> they
> > are provided by the tool itself or a third-party library or Application
> to
> > extend the tool's ability to import.  My thought is to have tools or
> > libraries add entry points to the "allura.importers" group, with each
> entry
> > point class providing import capability for a single tool from a single
> > source.  Each importer would then have attributes indicating the tool it
> > applies to, the source it imports from, and the controller for its UI.
> >
> > Here is an example importer stub with the attributes I'm thinking of:
> >
> >     class GoogleTrackerImporter(class):
> >         target_app = forgetracker.tracker_main.ForgeTrackerApp
> >         source = 'Google'
> >         controller =
> >
> > To support full-project imports, a library could expose an entry point in
> > the "allura.project_importers" group, which would point to a controller
> > that coordinates importing an entire project from an external source.
> >
> > I believe that will allow us to tie together importers in an extensible
> > way, yet give each importer flexibility in how it implements its import
> > logic.
> >
> > What do you think?
> >
> >
> > - Cory
> >
> --
> Dave Brondsema
> Principal Software Engineer -
> Dice Holdings, Inc.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message