polygene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Merlin <p...@nosphere.org>
Subject Re: Serialization issue?
Date Fri, 18 Dec 2015 12:01:03 GMT
Niclas Hedhman a écrit :
> Ahhh!!!
>
> ValueSerialization and EntitySerialization are separate!!! I didn't realize
> that before, and YES, I think you have hit the nail head-on, we need to
> ditch both and create StateSerialization.

Cool.

> And NOW, I am happy that the number of users/apps are relatively small.
> Because I would like to NOT worry about data storage compatibility, while
> sorting this out.
>
> IF data compatibility is a big issue, I'd rather build a Export/Import tool
> which exports all the data in 2.x (mechanism exist, I think) and imports
> the old data through the application model running on 3.0. I think it is
> possible to create a generic piece for this...

Sounds like a good plan to me.

> Thanks for pointing out the obvious flaw, that I completely missed.
>
> Niclas
>
> On Fri, Dec 18, 2015 at 4:09 AM, Paul Merlin <paul@nosphere.org> wrote:
>
>> Hey!
>>
>> Niclas Hedhman a écrit :
>>> I think only Paul can provide insight here...
>>>
>>> See the stacktrace below. I am writing an entity to the FileEntityStore,
>>> and to my surprise I get a deserialization exception.
>>>
>>> Looking closer, it seems that;
>>>   a. EntityState is already keeping the state in serialized form
>>>   b. propertyValueOf() is called for the serialization, only to
>> immediately
>>> serialize it back.
>>>
>>> So, ignoring my actual problem, the deeper question is; Is it really
>>> expected that the EntityState is serializing an extra time when
>>> serializing?? It seems wrong, but perhaps there is a deeper reason going
>> on
>>> here...
>>>
>>> Niclas
>> In a few words, it's a bit of a mess that we need to sort out.
>> I'm happy to crawl out of my busy cave to shed a bit of light on this.
>>
>> JSONEntityState holds state in a single org.json.JSONObject instance,
>> keyed with constants and names of properties and associations.
>> Similarly, JSON*AssociationStates hold state in JSONArray/JSONObject
>> instances.
>> The data structure in here is different than value serialization since
>> the 1.x days.
>>
>> For a state declaration:
>>
>> interface User {
>>   Property<String> name()
>>   Association<UserPreferences> preferences()
>>   ManyAssociation<Gizmo> favorites()
>>   NamedAssociations<Whatever> whatever()
>> }
>>
>> We expect a value to be serialized as something like:
>>
>> {
>>   "name": "Paul",
>>   "preferences": "B6215CF6-DF80-4C1C-90E7-3AC41A7F0339-42"
>>   "favorites": [],
>>   "whatever": { "name": "identity" }
>> }
>>
>> On the other hand, JSONEntityState holds that same state as:
>>
>> {
>>   "properties": {
>>     "name": "Paul"
>>   },
>>   "associations": {
>>     "preferences": "B6215CF6-DF80-4C1C-90E7-3AC41A7F0339-42"
>>   },
>>   "manyassociations": {
>>     "favorites": []
>>   },
>>   "namedassociations": {
>>     "whatever": { "name": "identity" }
>>   }
>> }
>>
>> The JSON Entity system then uses value serialization for actual
>> properties values and entity references.
>>
>> I always wondered what is the rationale behind this additional
>> distinction in "entity-style json serialization". Getting rid of it is a
>> storage breaking change, but it could make things more lean both
>> codewise and in actual storage subsystems.
>>
>>
>> But I digress, back to your serialization issue Niclas.
>>
>> Then, RdfEntityIndexer kicks in.
>> o.a.z.lib.RDF..EntityStateSerializer transforms each changed EntityState
>> in org.openrdf.model.Graph. So, for each properties/*associations it
>> asks the actual value to the EntityState (.propertyValueOf(..)). That
>> EntityState happens to be a JSONEntityStore that needs to deserialize
>> from org.json objects. This is done by first .toString() the org.json
>> instance and then ValueDeserialization. And finally that value of the
>> property/*association is transformed in org.openrdf.model.Graph
>> statements. Phew ....
>>
>> Again, all this could go away if state would be serialized with the same
>> structure in values/entities. We could simply talk about
>> StateSerialization, not ValueSerialization and/or EntitySerialization.
>>
>> Or maybe there's a reason I don't know yet for not doing this ??
>>
>> Finally, I'm pretty sure the stacktrace below is related to ZEST-105
>> which makes it useless without the corresponding assembly.
>>
>> Cheers
>>
>> /Paul
>>
>>> Caused by: org.apache.zest.api.value.ValueSerializationException: Don't
>>> know how to deserialize {"fullName":"Niclas Hedhman","username":"niclas"}
>>>     at
>>>
>> org.apache.zest.spi.value.ValueDeserializerAdapter.deserializeNodeGuessed(ValueDeserializerAdapter.java:788)
>>>     at
>>>
>> org.apache.zest.spi.value.ValueDeserializerAdapter.deserializeGuessed(ValueDeserializerAdapter.java:716)
>>>     at
>>>
>> org.apache.zest.spi.value.ValueDeserializerAdapter.doDeserialize(ValueDeserializerAdapter.java:398)
>>>     at
>>>
>> org.apache.zest.spi.value.ValueDeserializerAdapter.deserializeRoot(ValueDeserializerAdapter.java:347)
>>>     at
>>>
>> org.apache.zest.spi.value.ValueDeserializerAdapter.deserialize(ValueDeserializerAdapter.java:269)
>>>     at
>>>
>> org.apache.zest.spi.entitystore.helpers.JSONEntityState.propertyValueOf(JSONEntityState.java:133)
>>>     at
>>>
>> org.apache.zest.library.rdf.entity.EntityStateSerializer.lambda$serializeProperties$0(EntityStateSerializer.java:117)
>>>     at
>>>
>> org.apache.zest.library.rdf.entity.EntityStateSerializer$$Lambda$177/1835571798.accept(Unknown
>>> Source)
>>>     at java.util.Iterator.forEachRemaining(Iterator.java:116)
>>>     at
>>>
>> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
>>>     at
>>>
>> java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
>>>     at
>>>
>> org.apache.zest.library.rdf.entity.EntityStateSerializer.serializeProperties(EntityStateSerializer.java:116)
>>>     at
>>>
>> org.apache.zest.library.rdf.entity.EntityStateSerializer.serialize(EntityStateSerializer.java:89)
>>>     at
>>>
>> org.apache.zest.index.rdf.indexing.RdfIndexingService$RdfEntityIndexerMixin.indexEntityState(RdfIndexingService.java:199)
>>>     at
>>>
>> org.apache.zest.index.rdf.indexing.RdfIndexingService$RdfEntityIndexerMixin.indexUpdates(RdfIndexingService.java:160)
>>>     at
>>>
>> org.apache.zest.index.rdf.indexing.RdfIndexingService$RdfEntityIndexerMixin.notifyChanges(RdfIndexingService.java:114)
>>>     at
>>>
>> org.apache.zest.spi.entitystore.StateChangeNotificationConcern$1.commit(StateChangeNotificationConcern.java:44)
>>>     at
>>>
>> org.apache.zest.spi.entitystore.ConcurrentModificationCheckConcern$ConcurrentCheckingEntityStoreUnitOfWork$1.commit(ConcurrentModificationCheckConcern.java:118)
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message