polygene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Merlin <p...@nosphere.org>
Subject Re: Serialization issue?
Date Thu, 17 Dec 2015 20:09:09 GMT
Hey!

Niclas Hedhman a écrit :
> I think only Paul can provide insight here...
>
> See the stacktrace below. I am writing an entity to the FileEntityStore,
> and to my surprise I get a deserialization exception.
>
> Looking closer, it seems that;
>   a. EntityState is already keeping the state in serialized form
>   b. propertyValueOf() is called for the serialization, only to immediately
> serialize it back.
>
> So, ignoring my actual problem, the deeper question is; Is it really
> expected that the EntityState is serializing an extra time when
> serializing?? It seems wrong, but perhaps there is a deeper reason going on
> here...
>
> Niclas

In a few words, it's a bit of a mess that we need to sort out.
I'm happy to crawl out of my busy cave to shed a bit of light on this.

JSONEntityState holds state in a single org.json.JSONObject instance,
keyed with constants and names of properties and associations.
Similarly, JSON*AssociationStates hold state in JSONArray/JSONObject
instances.
The data structure in here is different than value serialization since
the 1.x days.

For a state declaration:

interface User {
  Property<String> name()
  Association<UserPreferences> preferences()
  ManyAssociation<Gizmo> favorites()
  NamedAssociations<Whatever> whatever()
}

We expect a value to be serialized as something like:

{
  "name": "Paul",
  "preferences": "B6215CF6-DF80-4C1C-90E7-3AC41A7F0339-42"
  "favorites": [],
  "whatever": { "name": "identity" }
}

On the other hand, JSONEntityState holds that same state as:

{
  "properties": {
    "name": "Paul"
  },
  "associations": {
    "preferences": "B6215CF6-DF80-4C1C-90E7-3AC41A7F0339-42"
  },
  "manyassociations": {
    "favorites": []
  },
  "namedassociations": {
    "whatever": { "name": "identity" }
  }
}

The JSON Entity system then uses value serialization for actual
properties values and entity references.

I always wondered what is the rationale behind this additional
distinction in "entity-style json serialization". Getting rid of it is a
storage breaking change, but it could make things more lean both
codewise and in actual storage subsystems.


But I digress, back to your serialization issue Niclas.

Then, RdfEntityIndexer kicks in.
o.a.z.lib.RDF..EntityStateSerializer transforms each changed EntityState
in org.openrdf.model.Graph. So, for each properties/*associations it
asks the actual value to the EntityState (.propertyValueOf(..)). That
EntityState happens to be a JSONEntityStore that needs to deserialize
from org.json objects. This is done by first .toString() the org.json
instance and then ValueDeserialization. And finally that value of the
property/*association is transformed in org.openrdf.model.Graph
statements. Phew ....

Again, all this could go away if state would be serialized with the same
structure in values/entities. We could simply talk about
StateSerialization, not ValueSerialization and/or EntitySerialization.

Or maybe there's a reason I don't know yet for not doing this ??
 
Finally, I'm pretty sure the stacktrace below is related to ZEST-105
which makes it useless without the corresponding assembly.

Cheers

/Paul

> Caused by: org.apache.zest.api.value.ValueSerializationException: Don't
> know how to deserialize {"fullName":"Niclas Hedhman","username":"niclas"}
>     at
> org.apache.zest.spi.value.ValueDeserializerAdapter.deserializeNodeGuessed(ValueDeserializerAdapter.java:788)
>     at
> org.apache.zest.spi.value.ValueDeserializerAdapter.deserializeGuessed(ValueDeserializerAdapter.java:716)
>     at
> org.apache.zest.spi.value.ValueDeserializerAdapter.doDeserialize(ValueDeserializerAdapter.java:398)
>     at
> org.apache.zest.spi.value.ValueDeserializerAdapter.deserializeRoot(ValueDeserializerAdapter.java:347)
>     at
> org.apache.zest.spi.value.ValueDeserializerAdapter.deserialize(ValueDeserializerAdapter.java:269)
>     at
> org.apache.zest.spi.entitystore.helpers.JSONEntityState.propertyValueOf(JSONEntityState.java:133)
>     at
> org.apache.zest.library.rdf.entity.EntityStateSerializer.lambda$serializeProperties$0(EntityStateSerializer.java:117)
>     at
> org.apache.zest.library.rdf.entity.EntityStateSerializer$$Lambda$177/1835571798.accept(Unknown
> Source)
>     at java.util.Iterator.forEachRemaining(Iterator.java:116)
>     at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
>     at
> java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
>     at
> org.apache.zest.library.rdf.entity.EntityStateSerializer.serializeProperties(EntityStateSerializer.java:116)
>     at
> org.apache.zest.library.rdf.entity.EntityStateSerializer.serialize(EntityStateSerializer.java:89)
>     at
> org.apache.zest.index.rdf.indexing.RdfIndexingService$RdfEntityIndexerMixin.indexEntityState(RdfIndexingService.java:199)
>     at
> org.apache.zest.index.rdf.indexing.RdfIndexingService$RdfEntityIndexerMixin.indexUpdates(RdfIndexingService.java:160)
>     at
> org.apache.zest.index.rdf.indexing.RdfIndexingService$RdfEntityIndexerMixin.notifyChanges(RdfIndexingService.java:114)
>     at
> org.apache.zest.spi.entitystore.StateChangeNotificationConcern$1.commit(StateChangeNotificationConcern.java:44)
>     at
> org.apache.zest.spi.entitystore.ConcurrentModificationCheckConcern$ConcurrentCheckingEntityStoreUnitOfWork$1.commit(ConcurrentModificationCheckConcern.java:118)
>
>

Mime
View raw message