jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Klimetschek (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (JCR-1509) [SUBMISSION] Amazon S3 Persistence Manager Project
Date Wed, 08 Aug 2012 08:44:10 GMT

    [ https://issues.apache.org/jira/browse/JCR-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430946#comment-13430946
] 

Alexander Klimetschek commented on JCR-1509:
--------------------------------------------

Outcome code-wise was nothing more than shown here. It turned out that S3 is too slow and
not fine-granular enough for a persistence manager. But it makes for a good DataStore, especially
for reading (someone built one proprietary).
                
> [SUBMISSION] Amazon S3 Persistence Manager Project
> --------------------------------------------------
>
>                 Key: JCR-1509
>                 URL: https://issues.apache.org/jira/browse/JCR-1509
>             Project: Jackrabbit Content Repository
>          Issue Type: New Feature
>          Components: sandbox
>            Reporter: Alexander Klimetschek
>            Assignee: Jukka Zitting
>            Priority: Minor
>         Attachments: jackrabbit-amazon.zip
>
>
> As I noted previously on the dev-list (http://markmail.org/search/?q=amazon+list%3Aorg.apache.jackrabbit.dev#query:amazon%20list%3Aorg.apache.jackrabbit.dev+page:1+mid:qw27gopsn4lnbde5+state:results)
I have written an Amazon S3 bundle persistence manager for Jackrabbit. I want to submit the
code for the sandbox, the full source is included in the zip file. Licensed under the ASF.
> The project also aims to implement a normal persistence manager (which I abandoned in
favor of the more efficient bundle pm, which is implemented, but does not work 100%), a file
system impl for S3 (only rough structure present) and an SPI impl that connects to S3 (dreaming
;-)). For more infos, I will include the README.txt of the project here:
> =================================================================
> Welcome to Jackrabbit persistence for Amazon Webservices (ie. S3)
> =================================================================
> This module contains various persistence options for using
> Amazon Webservices as backend for Jackrabbit / JCR. Amazon has
> two persistence services: S3 (public) and SimpleDB (still beta).
> The following options are available/ideas:
> - (1) persistence managers that connects to S3
>       (normal + bundle, in work, probably not very efficient)
>       
> - (2) persistence manager that connects to SimpleDB
>       (NOT feasible)
>       
> - (3) SPI implementation that connects to S3
>       (not implemented, very complicated, probably more efficient)
>       
> See details below and also TODO.txt
> Installing / Testing
> ====================
> This needs a patched Jackrabbit 1.3.x version. The patches can
> be found in the directory "patches-for-1.3". One patch will modify
> the pom of jackrabbit-core to generated the jackrabbit test jar
> for reuse in this project. To build that customized version, you need
> to do the following steps:
> 1) svn co http://svn.apache.org/repos/asf/jackrabbit/branches/1.3 jackrabbit-1.3
> 2) cd jackrabbit-1.3
> 3) apply all patches from the "patches-for-1.3" directory:
>    patch -p0 < %JR-AMAZON-PATH%/patches-for-1.3/%PATCH%.patch
> 4) mvn install
> 5) cd %JR-AMAZON-PATH%
> 6) change jackrabbit version number in pom.xml to the one you just built
>    (eg. project/parent/version = 1.3.4)
> 7) cp aws.properties.template aws.properties
> 8) enter your credentials in aws.properties
> 9) mvn test
> For debugging, you can change the logging in applications/test/log4j.properties
> and set up proxying (for monitoring the traffic with eg. tcp mon) in
> applications/test/jets3t.properties.
> Details about Implementations
> =============================
> (1) org.apache.jackrabbit.persistence.amazon.AmazonS3PersistenceManager
> http://www.amazon.com/s3
> Stores JCR Nodes and Properties inside S3 Objects. Uses UUID for Nodes and
> UUID/Name for Properties as Object names. Node references are stored
> via references/UUID.
> Configuration parameters:
> accessKey
>     Amazon AWS access key (aka account user id) [required]
> secretKey
>     Amazon AWS secret key (aka account password) [required]
>     
> bucket
>     Name of the S3 bucket to use [optional, default uses accessKey]
>     Note that bucket names are global, so using your accessKey is
>     recommended to prevent conflicts with other AWS users. 
>     
> objectPrefix
>     Prefix used for all object names [optional, default is ""]
>     Should include the workspace name ("${wsp.name}" or "version" for
>     the versioning PM) to put multiple workspaces into one bucket.
> Example XML Config:
> <PersistenceManager class="org.apache.jackrabbit.persistence.amazon.AmazonS3PersistenceManager">
>     <param name="accessKey"    value="abcde01234"/>
>     <param name="secretKey"    value="topsecret"/>
>     <param name="bucket"       value="abcde01234.jcrstore"/>
>     <param name="objectPrefix" value="${wsp.name}/"/>
> </PersistenceManager>
> -----
> (2) AmazonSimpleDBPersistenceManager
> This is *not* feasible because of the restrictions that are applied
> to SimpleDB. An item can only have up to 256 attributes, each attribute
> can only contain a string value and that one can only have 1024 chars.
> See this link for more information:
> http://docs.amazonwebservices.com/AmazonSimpleDB/2007-11-07/DeveloperGuide/SDB_API_PutAttributes.html
> -----
> (3) org.apache.jackrabbit.spi2s3
> TODO
> lots of work...
> About
> =====
> It was originally written by Alexander Klimetschek
> (alexander.klimetschek at googlemail dot com) in 2008.
> See the Apache Jackrabbit web site (http://jackrabbit.apache.org/)
> for documentation and other information. You are welcome to join the
> Jackrabbit mailing lists (http://jackrabbit.apache.org/mail-lists.html)
> to discuss this component and to use the Jackrabbit issue tracker
> (http://issues.apache.org/jira/browse/JCR) to report issues or request
> new features.
> Apache Jackrabbit is a project of the Apache Software Foundation
> (http://www.apache.org).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message