commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ralph.goers @dslextreme.com" <ralph.go...@dslextreme.com>
Subject Re: [vfs] MemcachedFilesCache for Google App Engine (object serialization)
Date Thu, 18 Jun 2009 21:39:27 GMT
And after you run mvn site they should show up in target/site/testing.html.

On Thu, Jun 18, 2009 at 2:38 PM, ralph.goers @dslextreme.com <
ralph.goers@dslextreme.com> wrote:

> No. The instructions are in
> https://svn.apache.org/repos/asf/commons/proper/vfs/trunk/xdocs/testing.xml.
>
>
> Ralph
>
>
> On Thu, Jun 18, 2009 at 9:18 AM, Vince Bonfanti <vbonfanti@gmail.com>wrote:
>
>> I'm finished with my changes to support serialization of AbstractFileName
>> and AbstractFileObject. The unit tests that are executed via "mvn test"
>> are
>> all passing. I tried to do "mvn site:stage" as you suggest, but got
>> out-of-memory errors. Is this the documentation I need to follow to run
>> the
>> external server tests?
>>
>>  http://commons.apache.org/vfs/testserver.html
>>
>> Thanks.
>> On Thu, Jun 4, 2009 at 6:40 PM, Ralph Goers <ralph.goers@dslextreme.com
>> >wrote:
>>
>> > Wow.  I'm interested to see if you can really get this to work. Just
>> make
>> > sure that after you get the existing unit tests to pass that the
>> functional
>> > tests that require external servers also pass - if you do a mvn
>> site:stage
>> > you will see the documentation on how to do that.
>> >
>> > VFS-245 is opened against AbstractFileName and probably needs to be
>> dealt
>> > with in the context of what you are doing.
>> >
>> >
>> > On Jun 4, 2009, at 2:22 PM, Vince Bonfanti wrote:
>> >
>> > I'm investigating creating a memcached-based FilesCache implementation
>> for
>> >> use with Google App Engine. The basic obstacle is that this requires
>> that
>> >> all objects that are used as keys or values must be serializable.
>> Before I
>> >> go too far down this path, I'd like to know if this is a reasonable
>> thing
>> >> to
>> >> do (or consider doing). Any thoughts or guidance would be greatly
>> >> appreciated.
>> >>
>> >> Background info: the Google App Engine (GAE) environment is inherently
>> >> distributed, where an unknown number of multiple instances of your
>> >> servlet-based web application will be running at the same time. The
>> GaeVFS
>> >> project (http://gaevfs.appspot.com) implements a VFS plugin on top of
>> the
>> >> GAE datastore; this is needed because the "real" filesystem within GAE
>> is
>> >> read-only, so GaeVFS provides a writeable filesystem for use by
>> >> applications. It seems that for proper operation of VFS, the FilesCache
>> >> implementation used by GaeVFS should (must?) also be inherently
>> >> distributed.
>> >> Fortunately, GAE supplies a memcached API that will be perfect for this
>> >> use
>> >> (if I can solve the serialization problem).
>> >>
>> >> Here's my first cut at what classes I'd need to make serializable and
>> some
>> >> notes on which fields might be marked transient:
>> >>
>> >> *AbstractFileName*
>> >> serialized fields: scheme (String), absPath (String), type (FileType)
>> >> transient fields: uri, baseName, rootUri, extension, decodedPath
>> >>
>> >> *FileType*
>> >> serialized fields: name (String), hasChildren (boolean), hasContent
>> >> (boolean), hasAttrs (boolean)
>> >> transient fields: none
>> >>
>> >> *AbstractFileObject*
>> >> serialized fields: name (AbstractFileName), content
>> (DefaultFileContent)
>> >> transient fields: fs (AbstractFileSystem),  operation (FileOperations),
>> >> attached (boolean), type (FileType), parent (FileObject), children
>> >> (FileName[]), objects (List)
>> >>
>> >> It's very important the the fs (AbstractFileSystem) field be transient
>> to
>> >> limit the number of other classes that need to be made serializable. It
>> >> looks like files are always retrieved from the files cache via the
>> >> AbstractFileSystem.getFileFromCache() method; if so then the "fs" field
>> of
>> >> AbstractFileObject can be restored within this method and doesn't need
>> to
>> >> be
>> >> serialized. We'll need to define a package-scope
>> >> AbstractFileObject.setFileSystem() method to support this. Or, this
>> could
>> >> be
>> >> done within the MemcachedFilesCache.getFile() method before returning
>> the
>> >> FileObject (but then we'd need a FileObject.setFileSystem method, or do
>> >> some
>> >> not-very-nice type casting).
>> >>
>> >> *DefaultFileContent
>> >> *serialized fields: file (AbstractFileObject), attrs (Map), roAttrs
>> (Map),
>> >> fileContentInfo (FileContentInfo), fileContentInfoFactory
>> >> (FileContentInfoFactory), openStreams (int)
>> >> transient fields: threadData (ThreadLocal)
>> >> question: what types do the "attrs" and "roAttrs" maps contain? are
>> these
>> >> serializable?
>> >>
>> > attrs and roAttrs contain attributes specific to the file system. For
>> > example, Webdav contains what you would get back from a PROPFIND method.
>> The
>> > Jar file system seems to return values found in the jar manifest., etc.
>> The
>> > values I found would all be Strings but I can't guarantee it.
>> >
>> >>
>> >> question: FileContentInfo and FileContentInforFactory are interfaces,
>> what
>> >> are the implementations of these?
>> >>
>> > DefaultFileContentInfo, FileContentInfoFileNameFactory,
>> > HttpFileContentInfoFactory, MimeFileContentInfoFactory and
>> > WebdavFileContentInfoFactory.
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> > For additional commands, e-mail: dev-help@commons.apache.org
>> >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message