brooklyn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aled Sage <aled.s...@gmail.com>
Subject Re: backwards compatibility for persisted brooklyn state
Date Tue, 09 Sep 2014 23:09:46 GMT
Hi all,

Agreed that a separate tool is a good idea. If written sensibly, we can 
later wire it in to give automatic upgrade support if desired.

Some other use-cases have come up requiring interesting transformation 
of persisted state:

 1. A policy was converted to an enricher.
    Requires moving it to be stored under enrichers, and references to
    the instance to be moved from an entity's policies list to the
    entity's enrichers list.
 2. Renaming fields.
    (more speculative than concrete, but it will come up at some point).

---
The upgrade tool needs a range of approaches for easy conversion. These 
could include XSLT, and Java classes that can modify the entire Brooklyn 
memento.

---
I'll look at refactoring `BrooklynMementoPersisterToObjectStore` so it 
can download and return the raw data. That is one of the first steps for 
being able to apply the transforms.

---
Andrea pointed out XMT for xstream. This involves adding migrateX() 
methods to the classes to be serialized/deserialized, to cope with 
changes in their schemas. Supporting renaming of classes is perhaps 
possibly, but involves telling the tool what class to expect (rather 
than inferring it from the classname in the XML) [2]. XMT could be worth 
more investigation.

Aled

[1] http://wiki.pmease.com/display/xmt/Getting+Started
[2] http://wiki.pmease.com/display/xmt/Change+Class+Name


On 25/08/2014 10:40, Richard Downer wrote:
> I'm also in favour of the separate tool option, for the reason that
> the operator would probably want to stay in control of this operation.
>
> I'd see a typical upgrade go as:
> - operator stops Brooklyn (leaving everything running)
> - operator runs a tool that backs up the persistence state (e.g.
> downloading the object store contents to local filesystem)
> - operator runs the upgrade tool
> - operator starts Brooklyn and sanity-checks
> - if any issues, operator takes a copy of the failed persistence state
> for postmortem diagnosis - then the original persistent state is
> re-instated and the old version of Brooklyn is started again
>
> By making some of this stuff happen "by magic" (just when running a
> new version of Brooklyn) it takes some control away from the operator.
> Since this is a very sensitive operation, and any mistakes could
> result in an orphaned infrastructure, I think we should minimise the
> magic in this area.
>
> In fact, I think I'd be in favour of Brooklyn refusing to start if
> it's pointed at a persistence store for a different version.
>
> We also need to consider that Brooklyn could also be running with
> third-party entities installed, and these would have the same xstream
> issues as the Brooklyn distribution - therefore the migration list
> also needs to be supplemented with third-party updates.
>
> Richard.
>
> On 22 August 2014 12:59, Andrea Turli <andrea.turli@cloudsoftcorp.com> wrote:
>> Aled,
>>
>> Thanks for the great summary.
>>
>> As things can be even trickier than the examples you descibed (i.e. HA with
>> different Brooklyn versions around) I'd be inclined towards option 2 where
>> an informed operator runs the tool and migrate from a versione to the other
>> with consciousness.
>>
>> Best,
>> Andrea
>> Il 22/ago/2014 13:40 "Aled Sage" <aled.sage@gmail.com> ha scritto:
>>
>>> Hi all,
>>>
>>> I've been thinking about backwards compatibility of our persisted brooklyn
>>> state, in the face of us moving classes around and/or deprecating.
>>>
>>> ---
>>> USE CASES
>>> Let's take two use-cases:
>>>
>>> 1. `MathPredicates.greaterThan` returns a (static) anonymous inner
>>>     class, so is serialized by xstream as `MathPredicates$1`.
>>>     We want to move that class to a fixed class name (e.g. `private
>>>     static class GreaterThan {...}`) suitable for xstream.
>>>     (It's a *really bad* idea to be using `MathPredicates$1`; if someone
>>>     in a future release adds some code above it then the class name
>>>     would change!)
>>> 2. I want to move PortRanges from package `brooklyn.location.basic` to
>>>     `brooklyn.util.net.PortRanges`.
>>>     However, this contains inner classes such as `SinglePort`.
>>>     If it was just code compatibility, we'd have the old `PortRanges
>>>     extends brooklyn.util.net.PortRanges`, and do some fancy stuff to
>>>     wrap the returned objects so they implement
>>>     `brooklyn.location.PortRange`.
>>>
>>> ---
>>> THE PROBLEM
>>> xstream tries to deserialize MathPredicates$1 but that class no longer
>>> exists. Even if we did leave it around (yuck!), nothing changes it to the
>>> new class name so we could never delete it!
>>>
>>> ---
>>> THE PROPOSED SOLUTION
>>> I propose we have a lookup file in brooklyn that contains all the renamed
>>> classes.
>>> We register a converter with xstream that, when deserializing, looks up
>>> this (cached) lookup file. If it contains the classname that xstream has
>>> encountered, we switch it for the new one and then let xstream continue
>>> with its deserialization.
>>>
>>> See brooklyn.entity.rebind.persister.XmlMementoSerializer for where this
>>> code would go.
>>>
>>> (Note that for the `PortRange` example, we'd require that the calling code
>>> had been updated to expect the new `brooklyn.util.net.PortRange` rather
>>> than the old `brooklyn.location.PortRange`. i.e. the returned instance
>>> would not implement all the same interfaces as the original class! We could
>>> perhaps work around that with some extra converter logic and wrapping
>>> classes.)
>>>
>>> ---
>>> ALTERNATIVE SOLUTION
>>> We could have a separate tool that is run manually once when someone is
>>> upgrading. It would walk through the persisted state, changing the old
>>> class names to the new ones.
>>>
>>> Thoughts?
>>>
>>> Aled
>>>
>>>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message