incubator-allura-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dave Brondsema" <>
Subject [allura:tickets] #6390 Support storing & serving files from disk instead mongo gridfs
Date Tue, 25 Jun 2013 16:50:07 GMT
Webserver custom authentication via Allura would cause an allura request for every icon/screenshot
request, which is exactly what we're trying to avoid.


** [tickets:#6390] Support storing & serving files from disk instead mongo gridfs**

**Status:** open
**Labels:** performance 
**Created:** Mon Jun 24, 2013 06:02 PM UTC by Dave Brondsema
**Last Updated:** Mon Jun 24, 2013 06:02 PM UTC
**Owner:** nobody

Files like project icons and screenshots are simple files, yet go through the whole Allura
stack to be served up.  When there are a lot of requests for these, it would be much better
to serve them directly as static files via nginx or apache.  Other allura files such as attachments
are affected by this too, but they don't get as much traffic normally.

The tricky part is that permission checks are necessary, e.g. if a project or neighborhood
is not publicly accessible.

We should make Allura `File`s have an alternate backend for a regular filesystem instead of
mongo/gridfs.  This should be controlled by deployers (e.g. in the .ini file) on a per-collection
basis, to give lots of flexibility to allure site administrators.

To handle non-public permissions, we could store the files in two directory trees, one for
public files and one for private files.  Any file that is not public (`*anonymous` read permission)
should be stored in the private directory tree.  Public files can be served by the webserver
directly.  The webserver could be configured so that any requests that 404 (file not in public
tree) get proxied through to the Allura app so that permission checks are used.  An alternate
option would be to configure the webserver to use custom authentication via Allura, and then
serve all files directly (public or not).  An nginx module for that is at

Allura code for referencing file URLs (icons, screenshots, for starters) will have to be modified
to support using the different URL paths that go to the webserver directly.

When project permissions change, there needs to be a mechanism to handle to moving icons from
one location to another if their 'public' status changes.  Use event handling could work well.
 (Not needed if custom webserver auth is used).

We'll need a script to convert existing files from GridFS to the filesystem.  Potentially
a two-way script, so somebody could go the other direction too (but doesn't seem to be an
immediate need).  And would be most useful as a two-pass script: 1) move files, 2) delete
files (so #2 can be run after everything is complete).

SF has backup/delete/restore scripts, which will need to handle these files.


Sent from because is subscribed to

To unsubscribe from further messages, a project admin can change settings at
 Or, if this is a mailing list, you can unsubscribe from the mailing list.
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message