shindig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Franklin, Matthew B." <mfrank...@mitre.org>
Subject RE: Shindig neo4j backend
Date Fri, 13 Jul 2012 11:56:14 GMT


>-----Original Message-----
>From: René Peinl [mailto:rene.peinl@hof-university.de]
>Sent: Friday, July 13, 2012 1:46 AM
>To: dev@shindig.apache.org
>Cc: 'Florian Holzschuher'
>Subject: AW: Shindig neo4j backend
>
>Thanks Ryan,
>to clarify things: Rave allows to have your own backend, it just has its own
>additional database abstraction and partly has persistence classes that are
>a subset of Shindig classes on the one hand, but add a few additional things
>on the other hand.

Rave's common data abstraction is built for data that needs to be accessed from both Shindig
& Rave.  To be clear, these are convenience classes and you could absolutely use a pure
Shindig implementation, so long as you are careful to manage the properties yourself and put
encryption keys where the Rave instance can find them (for security token). 

>Dependency injection was also recommended on the Rave mailing list. I'm no
>expert of DI and have used it up to now only for things like authentication
>or logging. Therefore, I was not considering that option. I will have a look
>into how that might be an option for us. On the Rave mailing list they also
>recommended having two databases, a relational one for portal data like
>which pages exist and which users have permissions on them and the graph
>db
>for social media data.

Rave offers a Spring/Guice bridge so that the components defined as Spring beans can be bound
to Guice injectors.  There is documentation on the Rave website for how to override beans
in the context so that your own implementations could be used.

As for the database, I wouldn't restrict it completely to Relational for the portal data.
 You *could* use a Neo4j instance, but I don't really see the value in storing that data in
a graph...  I do see the value in storing portal data in MongoDB as it is pretty much document
oriented anyway.   However, Rave already has a relational DB implementation done in JPA, so
it is the shortest route.  I hope to work some on a MongoDb impl at the Apache@OSCON Social
& Widgets hackathon.

>
>Regarding the OpenSocial 2.0 missing features in Shindig it seems, that you
>have integrated patches from version 3.0 in current Shindig releases.
>I saw an open issue that already had a patch, but was scheduled for Shindig
>3.0. The roadmap also stated that OpenSocial 2.0 support is scheduled for
>Shindig 3.0. It seems that changed in the last couple of weeks. I was
>specifically missing support for events
>http://activitystrea.ms/specs/json/schema/activity-schema.html#event
>But maybe I'm not up to date anymore. Sorry for that.

I think having 3.0 code is OK so long as it is  not a breaking change for 2.5 compliance.
 Other thoughts?

>
>Regards
>René
>
>
>-----Ursprüngliche Nachricht-----
>Von: Ryan Baxter [mailto:rbaxter85@apache.org]
>Gesendet: Freitag, 13. Juli 2012 03:28
>An: dev@shindig.apache.org
>Cc: Florian Holzschuher
>Betreff: Re: Shindig neo4j backend
>
>Welcome Rene :)
>
>Let me preface this by saying I have little knowledge of how Rave is
>implemented.  I have been meaning to look at it more in depth but haven't
>had the time.
>
>I am kind of surprised Rave does not allow you to still use your own
>persistence mechanism if you so choose.  I would think they would still keep
>that option available to consumers since there are endless possibilities for
>persisting data.  If you really can't inject your own persistence model into
>Rave it sounds like option 1 is your best bet, but as you list out in your
>requirements Rave offers a ton of other functionality you want.  If you
>chose options 1 are you really forking Shindig, or just injecting your own
>implementations?  The two are very different.
>
>I am a little confused about your comments about the activity stream
>implementation.  As far as I know it is a complete implementation, do you
>have specifics on what you think is missing?
>
>Between the Rave and Shindig community I am sure we can help you figure
>out
>which project is best for your use cases.  The Shindig community will
>welcome any contributions you want to make in order to help Shindig fit your
>use cases better and we would be glad to help explain any of the current
>features you have questions about.  A lot of us have gone through some
>extensive implementations and we realize the documentation around some
>of
>the feature is lacking.
>
>As far as the OpenSocial spec your contributions will be welcomed there as
>well, especially if you provide an implementation for it :)
>
>-Ryan
>
>On Thu, Jul 12, 2012 at 11:25 AM, René Peinl <rene.peinl@hof-university.de>
>wrote:
>> Dear Shindig developers,
>>
>> I'm working together with my colleague Florian on a systems
>> integration research project that brings together a DMS, a groupware
>> system, an ERP system in an Intranet scenario and adds social media
>> capabilities under the hood of a central lightweight portal (see
>> requirements overview at the end of the email).
>>
>>
>>
>> For the portal, we are planning to use Shindig as the central
>> component and maybe Rave as a basis  for the presentation layer. Since
>> social media data is inherently graph-oriented, we are planning to use
>> neo4j as the storage engine for Shindig (and maybe Rave as well).
>> Florian has already implemented a draft of this neo4j storage layer
>> which works well for some sample data that we have generated (if
>> anyone is interested we can provide the sample data and the generator
>> for it), but due to simplicity reasons and to avoid additional
>> conversion operations in between the storage layer is currently not
>> using JPA, but directly writing the objects to the database (replacing
>e.g. both PersonImpl and PersonDB).
>>
>>
>>
>> Lately, we have discovered Rave and found out that despite its close
>> integration with Shindig it uses its own database abstraction and
>> partly competing implementations for classes that are already present in
>Shindig.
>> For example there is rave.portal.model.Person which uses JPA to store
>> a (incomplete compared to OpenSocial 2.0 specs) person object in the
>> database and there is shindig.social.opensocial.jpa.PersonDB which
>> does the same, but seems like a complete implementation of the fields
>> defined in OpenSocial 2.0.
>>
>>
>>
>> My goal is to better understand the decisions behind the current
>> implementation of both Shindig and Rave (I've posted a similar
>> question on the Rave mailing list) in order to find a solution for our
>> project that lets the Apache community benefit as much as possible
>> from our efforts, without burdening our project too much with
>non-project-specific requirements.
>>
>>
>>
>> The options we have (from my perspective) are as follows
>>
>> 1)      Kind of forking the Shindig project and using our own neo4j
>backend
>> and not using Rave
>>
>> 2)      Going one step further and integrating Rave with our own neo4j
>> Shindig backend and eliminate the duplicate classes
>>
>> 3)      Trying to use SpringData JPA abstraction layer for neo4j and
>> providing some extended functionality in own classes (e.g. a
>> friend-of-a-friend query could be implemented with relational DBs as
>> well, but doesn't make too much sense due to performance reasons) and
>> hoping that the Rave and Shindig community are helping us to eliminate
>> the duplicate implementations on JPA level.
>>
>>
>>
>> All options have their pros and cons and I'd really like to hear your
>> opinion on that. If you have the feeling that our project idea is very
>> specific and you don't see any use for the results in your scenarios
>> than it may be better for both sides if we are building a fork. On the
>> other hand, if you think that the principle idea behind our project
>> would be beneficial for your scenarios as well, we would be happy to work
>together on that.
>> We need e.g. some of the missing OpenSocial 2.0 features like full
>> activitystrea.ms support on the data layer, which is currently not
>> given in Shindig 2.5 beta2
>> (https://issues.apache.org/jira/browse/SHINDIG-1536), but we do not want
>to interfere with your roadmap and release plans.
>> We also found several common scenarios that e.g. MS SharePoint is
>> supporting, but that are not feasible with OpenSocial 2.0
>> specification (mainly due to its focus on Internet scenarios).
>> Examples are: User has checked-out a document in a DMS. For this kind
>> of check-out operation, there is no post verb in the activitystrea.ms
>> standard.  We would be happy to bring these finding into the ongoing
>> discussion about future Open Social versions.
>>
>>
>>
>> Summed up, we wanted to know how we could best cooperate with the
>> existing Shindig (and Rave) community in order to quickly achieve our
>> goals and let the Apache community benefit the most from our efforts.
>>
>> Any suggestions?
>>
>> Regards
>>
>> René
>>
>>
>>
>>
>>
>> -----------------------------------------------------------
>>
>> Prof. Dr. René Peinl
>>
>> Teaching area: architecture of Web applications
>>
>> University of applied Sciences Hof
>>
>> Alfons-Goppel-Platz 1, 95028 Hof, Bavaria, Germany
>>
>> Tel: +49-9281-409-4820
>>
>> Email: rene.peinl@hof-university.de
>>
>>
>>
>>
>>
>>
>>
>> Requirements for OpenSocial Intranet Portal
>>
>>
>>
>>
>> Presentation layer
>>
>>
>> ·         Portal entry page with admin predefined gadgets, customizable by
>> user
>> (mostly existing in Rave 0.12)
>>
>> ·         Gadget catalog (existing in Rave 0.12)
>>
>> ·         User profile page with photo, name and other attributes. Admin
>> customizable. All open social fields should be available for display.
>> Gadgets can be used to show additional data like activity stream,
>> friends network, bookmarks/tags, . (basic functionality available in
>> Rave 0.12 /portal/app/person)
>>
>> ·         User directory with overview of existing users (including paging
>> and search)
>> (not present in Rave 0.12 yet)
>>
>> ·         Ability to add friends/following people to "friends network"
>(e.g.
>> on profile page)
>> (not present in Rave 0.12 yet)
>>
>> ·         Suggestion list for new friends based on common friends,
>> interests, etc.
>> (not present in Rave 0.12 yet)
>>
>> ·         Ability to set a status message
>> (basic functionality available in Rave 0.12)
>>
>> ·         Ability to send a personal message to a friend
>> (not present in Rave 0.12 yet, maybe possible with one of the gadgets)
>>
>>
>>
>>
>> Logic layer
>>
>>
>> ·         Gadget Rendering API
>> (existing in Shindig 2.5 beta2)
>>
>> ·         Authentication against external identity and access management
>> system (IAM) using OAuth 2.0 and SAML.
>> (some references state that Apache Shiro is able to do that, but we
>> haven't seen the code implementing it)
>>
>> ·         Exposing social media data like friends, activities, . as
>RESTful
>> Web services
>> (available in Shindig 2.5 beta2)
>>
>> ·         Exposing extended social media data like friend of a friend to
>> portal gadgets as RESTful Web service
>> (draft implementation available at iisys)
>>
>>
>>
>>
>> Data layer
>>
>>
>> ·         Full support for activitystrea.ms data storage and retrieval at
>> data object layer
>> (draft implementation available at iisys, but not with JPA support)
>>
>> ·         Storing/retrieving social media data like friends, activities,
>> ratings, bookmarks, tags, . in/from a graph database like neo4j (draft
>> implementation available at iisys, it has to be discussed, whether it
>> would be better to use SpringData JPA abstraction instead of direct
>> neo4j access)
>>
>> ·         Integrating Rave and Shindig person data
>> (not clear why Rave is having its own Person class which seems like a
>> subset of Shindigs PersonDB)
>>
>> ·         Keeping Rave User and PageUser data consistent with IAM system
>>
>> ·         Storing Rave authorization data (gadget permissions, user
>> preferences, .)  in neo4j
>> (not available yet)
>>
>> ·         Storing Rave general user data in neo4j.
>> (not available yet)
>>
>>
>>
>>
>>


Mime
View raw message