river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gerard Fulton <cgful...@gmail.com>
Subject Re: Out of curiousity...(was:Re: Migrating data in a JavaSpace)
Date Mon, 04 Feb 2013 19:45:18 GMT
Hi Greg, One of the major players providing space backed persistence is
Gigaspaces. The whole company is now modeled around concept of java spaces.
One of the big pushes for Gigaspaces is the concept of a space based
in-memory distributed data grid. Their architecture makes the space the
primary data store. The java space is the foundation for the transport,
persistence, and execution layers.

Gerard


On Mon, Feb 4, 2013 at 11:29 AM, Greg Trasuk <trasukg@stratuscom.com> wrote:

>
> Out of curiousity, how frequently do people use JavaSpaces as a
> long-term persistence mechanism?  Obviously, it's one of the
> possibilities envisioned by the JavaSpace spec, but most examples I've
> seen (and certainly the ones I've implemented) have JavaSpaces as more
> of short-term communications mechanism.  Master-worker pattern,
> scatter-gather, data flow design, or map-reduce styles tend to be
> shorter-term, for instance, even if they use a persistence mechanism to
> guard against server failure.
>
> Cheers,
>
> Greg.
>
> On Mon, 2013-02-04 at 12:49, Dan Creswell wrote:
> > On 4 February 2013 17:32, Dawid Loubser <dawid@travellinck.com> wrote:
> > > Thanks Gerard,
> > >
> > > That does sound reasonable, but wouldn't I effectively lose the unique
> > > individual codebase annotations of each entry? I have various unrelated
> > > services that interact in often-complex ways. Consider the following:
> > >
> > > * In foo-api, I have an entry called FooEvent
> > > * In my space-based timer api, I have an entry called PublishLater, and
> > > a particular instance of PublishLater contains an instance of FooEvent,
> > > and a timestamp that says when to publish the nested entry.
> > >
> > > The timer service (and the timer-api) has no knowledge of foo-api.
> There
> > > would be no generic way to write that PublishLater entry to XML, and
> > > parse it again, making sure that the nested FooEvent has the correct
> > > codebase (which will be distinct from the codebase of the higher-level
> > > Entry). I have many such occurrences of entries generically containing
> > > other entries, and the codebase has to remain intact for each.
> > >
> > > I think I will (as Dan suggested( have to write a Java-based migration
> > > tool, that (using reflection) reconstructs each Entry, taking care to,
> > > at each level, retain the proper codebase, with only the changes
> > > required for the migration. Because I'm using Rio's maven-based class
> > > loading, I know that where a codebase URL was
> "artifact:foo:bar-api:1.0"
> > > I can now reconstruct it, replacing it with "artifact:foo:bar-api:1.1".
> > >
> > > This will be very interesting indeed, and I need to do it ASAP :-( A
> > > production deployment depends on this. After reading the Entry spec, it
> > > seems that only at each top-level field of an Entry can each object
> have
> > > a different codebase, right? (and not at lower levels within those
> > > objects). If so, that'll make things a lot easier.
> >
> > I think it would be possible for something below top-level field to
> > have its own codebase but that would be extremely rare (too ugly to
> > work with).
> >
> > More importantly I don't think you need to be that generic as I
> > suspect that your codebase probably does obey the "top-level field"
> > rule you mention. You could check that somewhat by doing a JavaSpace05
> > contents and dumping out class and associated classloader plus
> > codebase if present for each entry top to bottom (in fact you could
> > store it all up in a couple of hashtables and then dump it out which'd
> > save you reading through piles of duplicates).
> >
> > >
> > > Anybody have any experience doing this the "hard" (with Java
> > > classloading) way?
> >
> > Anyone who's implemented a JavaSpace at least ;)
> >
> > Seriously, if you need some advice or whatever, punt a request up
> here....
> >
> > >
> > > Dawid
> > >
> > >
> > > On Mon, 2013-02-04 at 06:56 -0800, Gerard Fulton wrote:
> > >> One easy option may be to write a simple client using your old code to
> > >> serialize the entries in the space to XML on disk. Then launch your
> new
> > >> application and put entries into the space instance.
> > >>
> > >>
> > >> On Mon, Feb 4, 2013 at 3:34 AM, Dawid Loubser <dawid@travellinck.com>
> wrote:
> > >>
> > >> > Thanks for the quick response, Dan!
> > >> >
> > >> > I want to understand the classloading a bit better. Let me explain
> to
> > >> > you how I *think* it works. Also, for reference, I'm using the rio
> > >> > project, that has a special classloader that understands URLs in the
> > >> > form "artifact:foo:bar:1.0" and which loads classes from Maven
> > >> > artifacts, but I think it's conceptually the same as any other URL
> > >> > scheme etc.
> > >> >
> > >> > * When an  Entry it written to space, it's turned into a
> > >> > MarshalledInstance. This is annotated with the codebase (a
> collection of
> > >> > URLs). Immediate question: Is there only one codebase at the
> top-level
> > >> > of the entry, or does every object in the graph have (or can have)
> its
> > >> > own codebase?
> > >> >
> > >> > * When a worker takes/reads an entry (which might contain things
> that
> > >> > both are on the worker's classpath, and perhaps lower-level content
> that
> > >> > is not (i.e. specialisations that it does not have to understand),
> how
> > >> > does the space proxy know what to do? I imagine it uses the thread
> > >> > context class loader, but then how does it deserialise the objects
> that
> > >> > is not on that classpath (using the codebase annotation of the
> > >> > MarshalledInstance, I imagine) whilst not colliding with the classes
> > >> > already available to the worker? Using some sort of parent/child
> > >> > delegation?
> > >> >
> > >> > I've got a very tricky ClassCastException problem I'm trying to
> debug,
> > >> > where it's clearly the same class loaded by two classloaders, and
> thus
> > >> > the field cannot be assigned. I don't know how to get "in there" and
> > >> > solve the problem, it seems I can only respond to the
> > >> > UnusableEntryException, get the partial entry, and lose the rest?
> > >> >
> > >> > thanks so much,
> > >> > Dawid
> > >> >
> > >> >
> > >> > On Mon, 2013-02-04 at 11:17 +0000, Dan Creswell wrote:
> > >> > > On 4 February 2013 11:10, Dawid Loubser <dawid@travellinck.com>
> wrote:
> > >> > > > Hi all,
> > >> > > >
> > >> > > > I have a bunch of entries in a JavaSpace (representing
> long-running
> > >> > > > process state, i.e. they exist for days or weeks), and these
> contain
> > >> > > > some objects that were generated from XML (using JAXB).
That
> vocabulary
> > >> > > > has evolved (additions only) but now, of course, the computed
> > >> > > > SerialVersionUIDs will be different. When I redeploy my
workers
> that
> > >> > > > have been built against the new API, they will surely fail
when
> reading
> > >> > > > the old entries.
> > >> > > >
> > >> > > > Any strategies as to how I can migrate the data in the space?
> I'm
> > >> > > > running a persistent outrigger (snaplogstore). I was thinking
> of, in a
> > >> > > > worker with an 'old' classpath, draining the space, and
storing
> those
> > >> > > > entries in some non-java representation on disk, and then
in a
> worker
> > >> > > > with the 'new' classpath, reading those entries and
> re-populating the
> > >> > > > space.
> > >> > >
> > >> > > Slightly more complicated but it's possible to have one worker
do
> all
> > >> > > this with some classloader magic. You basically load old and
new
> > >> > > definitions into separate classloaders with the old version being
> > >> > > directly on the classpath, the other dynamically loaded from
> something
> > >> > > not on the classpath.
> > >> > >
> > >> > > Then you can take the old easily and use reflection magic to
> populate
> > >> > > a new and write it.
> > >> > >
> > >> > > One other challenge is that most JavaSpace implementations don't
> like
> > >> > > mixed schemas do probably you're better to create a second space,
> > >> > > write the migrated ones into that and then turn off the old one
> (or
> > >> > > copy back to the old once you've cleared it down/re-built it).
> > >> > >
> > >> > > >
> > >> > > > Migrating data in a space is surely something that must
have
> caused
> > >> > > > problems for somebody before, and I'd love to tackle this
> problem
> > >> > > > drawing on some experience of others.
> > >> > > >
> > >> > > > regards,
> > >> > > > Dawid
> > >> > > >
> > >> >
> > >> >
> > >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message