polygene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Niclas Hedhman <nic...@hedhman.org>
Subject Re: Serialization issue?
Date Fri, 18 Dec 2015 01:47:53 GMT
Ahhh!!!

ValueSerialization and EntitySerialization are separate!!! I didn't realize
that before, and YES, I think you have hit the nail head-on, we need to
ditch both and create StateSerialization.

And NOW, I am happy that the number of users/apps are relatively small.
Because I would like to NOT worry about data storage compatibility, while
sorting this out.

IF data compatibility is a big issue, I'd rather build a Export/Import tool
which exports all the data in 2.x (mechanism exist, I think) and imports
the old data through the application model running on 3.0. I think it is
possible to create a generic piece for this...

Thanks for pointing out the obvious flaw, that I completely missed.

Niclas

On Fri, Dec 18, 2015 at 4:09 AM, Paul Merlin <paul@nosphere.org> wrote:

> Hey!
>
> Niclas Hedhman a écrit :
> > I think only Paul can provide insight here...
> >
> > See the stacktrace below. I am writing an entity to the FileEntityStore,
> > and to my surprise I get a deserialization exception.
> >
> > Looking closer, it seems that;
> >   a. EntityState is already keeping the state in serialized form
> >   b. propertyValueOf() is called for the serialization, only to
> immediately
> > serialize it back.
> >
> > So, ignoring my actual problem, the deeper question is; Is it really
> > expected that the EntityState is serializing an extra time when
> > serializing?? It seems wrong, but perhaps there is a deeper reason going
> on
> > here...
> >
> > Niclas
>
> In a few words, it's a bit of a mess that we need to sort out.
> I'm happy to crawl out of my busy cave to shed a bit of light on this.
>
> JSONEntityState holds state in a single org.json.JSONObject instance,
> keyed with constants and names of properties and associations.
> Similarly, JSON*AssociationStates hold state in JSONArray/JSONObject
> instances.
> The data structure in here is different than value serialization since
> the 1.x days.
>
> For a state declaration:
>
> interface User {
>   Property<String> name()
>   Association<UserPreferences> preferences()
>   ManyAssociation<Gizmo> favorites()
>   NamedAssociations<Whatever> whatever()
> }
>
> We expect a value to be serialized as something like:
>
> {
>   "name": "Paul",
>   "preferences": "B6215CF6-DF80-4C1C-90E7-3AC41A7F0339-42"
>   "favorites": [],
>   "whatever": { "name": "identity" }
> }
>
> On the other hand, JSONEntityState holds that same state as:
>
> {
>   "properties": {
>     "name": "Paul"
>   },
>   "associations": {
>     "preferences": "B6215CF6-DF80-4C1C-90E7-3AC41A7F0339-42"
>   },
>   "manyassociations": {
>     "favorites": []
>   },
>   "namedassociations": {
>     "whatever": { "name": "identity" }
>   }
> }
>
> The JSON Entity system then uses value serialization for actual
> properties values and entity references.
>
> I always wondered what is the rationale behind this additional
> distinction in "entity-style json serialization". Getting rid of it is a
> storage breaking change, but it could make things more lean both
> codewise and in actual storage subsystems.
>
>
> But I digress, back to your serialization issue Niclas.
>
> Then, RdfEntityIndexer kicks in.
> o.a.z.lib.RDF..EntityStateSerializer transforms each changed EntityState
> in org.openrdf.model.Graph. So, for each properties/*associations it
> asks the actual value to the EntityState (.propertyValueOf(..)). That
> EntityState happens to be a JSONEntityStore that needs to deserialize
> from org.json objects. This is done by first .toString() the org.json
> instance and then ValueDeserialization. And finally that value of the
> property/*association is transformed in org.openrdf.model.Graph
> statements. Phew ....
>
> Again, all this could go away if state would be serialized with the same
> structure in values/entities. We could simply talk about
> StateSerialization, not ValueSerialization and/or EntitySerialization.
>
> Or maybe there's a reason I don't know yet for not doing this ??
>
> Finally, I'm pretty sure the stacktrace below is related to ZEST-105
> which makes it useless without the corresponding assembly.
>
> Cheers
>
> /Paul
>
> > Caused by: org.apache.zest.api.value.ValueSerializationException: Don't
> > know how to deserialize {"fullName":"Niclas Hedhman","username":"niclas"}
> >     at
> >
> org.apache.zest.spi.value.ValueDeserializerAdapter.deserializeNodeGuessed(ValueDeserializerAdapter.java:788)
> >     at
> >
> org.apache.zest.spi.value.ValueDeserializerAdapter.deserializeGuessed(ValueDeserializerAdapter.java:716)
> >     at
> >
> org.apache.zest.spi.value.ValueDeserializerAdapter.doDeserialize(ValueDeserializerAdapter.java:398)
> >     at
> >
> org.apache.zest.spi.value.ValueDeserializerAdapter.deserializeRoot(ValueDeserializerAdapter.java:347)
> >     at
> >
> org.apache.zest.spi.value.ValueDeserializerAdapter.deserialize(ValueDeserializerAdapter.java:269)
> >     at
> >
> org.apache.zest.spi.entitystore.helpers.JSONEntityState.propertyValueOf(JSONEntityState.java:133)
> >     at
> >
> org.apache.zest.library.rdf.entity.EntityStateSerializer.lambda$serializeProperties$0(EntityStateSerializer.java:117)
> >     at
> >
> org.apache.zest.library.rdf.entity.EntityStateSerializer$$Lambda$177/1835571798.accept(Unknown
> > Source)
> >     at java.util.Iterator.forEachRemaining(Iterator.java:116)
> >     at
> >
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> >     at
> >
> java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
> >     at
> >
> org.apache.zest.library.rdf.entity.EntityStateSerializer.serializeProperties(EntityStateSerializer.java:116)
> >     at
> >
> org.apache.zest.library.rdf.entity.EntityStateSerializer.serialize(EntityStateSerializer.java:89)
> >     at
> >
> org.apache.zest.index.rdf.indexing.RdfIndexingService$RdfEntityIndexerMixin.indexEntityState(RdfIndexingService.java:199)
> >     at
> >
> org.apache.zest.index.rdf.indexing.RdfIndexingService$RdfEntityIndexerMixin.indexUpdates(RdfIndexingService.java:160)
> >     at
> >
> org.apache.zest.index.rdf.indexing.RdfIndexingService$RdfEntityIndexerMixin.notifyChanges(RdfIndexingService.java:114)
> >     at
> >
> org.apache.zest.spi.entitystore.StateChangeNotificationConcern$1.commit(StateChangeNotificationConcern.java:44)
> >     at
> >
> org.apache.zest.spi.entitystore.ConcurrentModificationCheckConcern$ConcurrentCheckingEntityStoreUnitOfWork$1.commit(ConcurrentModificationCheckConcern.java:118)
> >
> >
>



-- 
Niclas Hedhman, Software Developer
http://zest.apache.org - New Energy for Java

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message