river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gregg Wonderly <gr...@wonderly.org>
Subject Re: MarshalledServiceItem
Date Wed, 02 Feb 2011 21:53:06 GMT
I am finding my self being slightly defensive sounding and I'm not wanting to be 
defensive about my implementation.  I want to defend the features that I need 
and thus made changes in lookup to provide.  Hopefully the inline below is not 
degenerating into to much reiteration of the issues.  There are some additional 
thoughts about other ways that lookup could be changed to provide the "string" 
information without a proxy service.

See below...

On 2/2/2011 3:26 AM, Dan Creswell wrote:
> Inline and leaving context for now.....
>
> On 1 February 2011 19:12, Gregg Wonderly<gergg@cox.net>  wrote:
>
>> On 2/1/2011 3:17 AM, Dan Creswell wrote:
>>
>>> So...
>>>
>>> On 1 February 2011 01:29, Gregg Wonderly<gregg@wonderly.org>   wrote:
>>>
>>>   One of the important things for my use of my changes to the
>>>> ServiceRegistrar interface was separating deserialization of the Entrys
>>>> from
>>>> the service object.
>>>>
>>>
>> Note "separating" used here.  I was trying to say that I need to be able to
>> "pick" which Entry objects to deserialize, and deserialize them independent
>> of deserializing the service proxy/object.
>>
>>
>>   Being able to defer downloading was also controlled by the changes to
>>>> ClassLoading to include the "neverPrefer" settings on a class name.  I
>>>> can
>>>> do that now through the recent RMIClassLoaderSPI override capabilities.
>>>>   But
>>>> I still need to be able to control which Entry is deserialized.  I need
>>>> to
>>>> be able to ask what classes, by name, are in the namespace of each Entry
>>>> object as my implementation allowed. I had made changes to Reggie
>>>> marshalling to use reflection to get the class hierarchy, and then
>>>> provided
>>>> access to the list of classnames as part of the returned, marshalled
>>>> value
>>>> in my API.
>>>>
>>>>
>>>>   Digging through classes belonging to a service implementation that is
>>> designed to be encapsulated/not your concern? Does that sound right? I
>>> gotta
>>> say, it sounds horribly invasive, very hacky and speaks of an overly
>>> complex
>>> solution as the result of treating symptoms, not problems.
>>>
>>
>> Okay, I want to go through the problems that I was dealing with, the
>> attributes of the current mechanisms, and how I inserted points of control
>> to manage these issues.
>>
>> First, let's look at the simple client mechanisms I wanted to have.  I have
>> a desktop environment, which does lookup, and then shows treeviews of
>> services based on "machine"->"group"->Service Instance structure.  The
>> deployed services contain Entry objects that provide values for these three
>> name spaces.  Since I need to look at these, they can not be preferred by
>> the service or I can't read them.  There may be a subclass that the client
>> has used, and that can be preferred no problem.  There are some other items
>> in the Entrys that I need too, such as icons.  Any of the Entry objects that
>> I need access to, and any classes which they are dependent on, and which are
>> publicly visible classes/interfaces, because of that attribute (public) can
>> not be realistically preferred.
>>
>>
> These Entry types are part of the basic infrastructure of your system aren't
> they? Why download them at all? I guess because you have to "teach" it to
> the LUS via code-download? Or you could just put them on all classpaths
> including LUS? S'far as I can see you've got an extended LUS infrastructure
> that everyone shares thus codebase seems like the wrong choice for these.
>
> Sure user's can subclass Entry types but do they? Do you even want them to?
> Why? Thus far I've seen many subclass Entry's for JavaSpaces and, I think,
> zero for LUS type annotations.
>
> See my other thread with Peter but it seems to me that you have some core
> additions to LUS infrastructure and a need to be able to filter what you
> don't want to see/is outside of a client's knowledge base. Because it's that
> inability to filter that then leads to lots of additional downloading.

The basic issue is that I need to show users what services exist.  The LUS 
maintains a very useful view of the health of the network.  Asking it what 
services exist has to happen by some 'client'.  Remember that my customers techs 
are using the client out in the field, on a slow network.  So whether they 
enumerate something from the LUS or from another "service", there will be 
network traffic.

It might be possible to enumerate services at a aggregation service, but then 
you have to unmarshall all of the services there to get the "data" that you'd 
send to a client.  I don't want to have 100s of proxies active on the network 
for this mechanism, really.  The deferred unmarshalling in discussion with Peter 
would help deal with this by allowing such an aggregation service to not have to 
unmarshall the proxy.  That was my initial intent.  It was only after I got that 
implemented that I found out how the PREFERRED.LIST really kept it from working 
at all, and so I had to do something to keep the PREFERRED.LIST from being 
accessed when not needed yet.

The service aggregation becomes a single point of failure and I'd need to 
replicate it to every server that had an LUS on it.  Then, it really becomes 
more like a secondary look up service, and so I'd still have to ask, is the LUS 
API really in the right form if there is a "mirror" of it needed?  Note that I 
made it possible to pick through a lot of the class information for the Entry 
and the service Proxy in reef's work because I was trying to get around having 
to mess with the never-preferred business.  But in the end, I just could not 
find any other way, because as soon as I wanted to unmarshall an Entry to get a 
"Name" and a "Group" and an "Icon" for my UI, PreferredClassLoading went after 
the PREFERRED.LIST, even though not a single class that I was using would be 
resolved from the codebase.

I'm not sure how I would "not activate the codebase" in some client.  It needs 
the Entry object, fully resolved.  I want the codebase to work right, so I'm not 
gonna just throw in a proxy context class loader that resolves everything 
locally.  If there is content in an Entry that is an unknown class, I will 
suffer the download to make sure that it resolves to the right codebase.

Note that I am not wanting to focus on the never-prefer issue as something that 
I need.  What I actually need is to defer download of any part of the codebase 
until the service is "activated".

What might be interesting is to figure out how to stuff the preferred list into 
the MarshalledObject annotation so that PreferredClassLoader would not need to 
retrieve the codebase elements to resolve how to perform class loading.

Currently, there is a lot of backward compatibility involved in how the 
annotation is managed.

The namespace change that we are discussing for com.sun.jini, might be the right 
moment to introduce any compatibility issues with MarshalledObject and the use 
of the RMIClassLoader annotation.

>> So my first assertion is that Entry objects will hardly ever be preferred,
>> and thus downloading of the PREFERRED.LIST is not a necessary step to
>> correctly resolve them for my use.
>>
>> The next issues is this PREFERRED.LIST file.  The implementation of that
>> technology is that the file is retrieved from the first jar file listed in
>> the codebase.  So, in order to resolve any class in the proxy or Entry
>> values, there must be a download from the associated codebase server.
>>
>> Now, one of the first things that I think about with distributed computing
>> is the business of partial failure and limiting that impact to the overall
>> performance of the system.  My deployments at my customers' sites consist of
>> copies of the same service platform with the same services on it.  There are
>> copies for both redundancy and for capacity.
>>
>> Each instance of my platform has as a starting point 20 some services each
>> with a separate codebase, but with many common jars such as jsk-dk.jar.  If
>> a lookup service responds with a list of services but the codebase server
>> doesn't respond with jars, then any attempt to download limits the
>> performance of the display of available services, and I considered an
>> "incorrect list of available services" less important than non-display or
>> slow-display of available services.
>
> As Michal says elsewhere, are you sure the solution is to try and leverage
> LUS to handle display challenges like this?

I need the data with minimal network traffic.  The LUS lookup is even something 
that generates some delays.  But, the UI, as it populates, feels largely like a 
web page load.  I've reached a point of acceptable performance that works for now.

Service lookup and selection has always been a point of contention as you well 
know.  I had always been on the side of the fence that said the LUS works fine 
the way it is.  It wasn't until I tried to figure out how to manage 100s of 
services visible and only 1 being interesting that I suddenly found myself 
setting on the fence asking, "If it almost does the right thing, is there a way 
to maintain the same interface, but add a new interface with more functionality 
that would keep me from having to completely ignore the LUS?"

So, I tried something out.  I find that given how Reggie works and is 
implemented, that the addition of the functionality as shown in Reef does 
provide some new opportunities.  But, delayed unmarshalling alone, does not 
provide for no-download until needed.  PreferredClassLoading is still in the 
mix.  My solution to that, which I think does need to be revisited.  We need to 
decide if the PREFERRED.LIST is really a problem because it causes a download. 
Service registration data will always require download, and so just to use the 
Entry values in a UI or some other client will still require download of all the 
elements of the codebase needed to resolve the classes.

In one explanation of how things are currently working with River, without my 
changes, I suggested that using Reggie and preferred class loading would be like 
opening up a Google session, running a query, and then having to download every 
matched pages content before you could see the list of search results.

Google provides the type of service that you describe by changing what the user 
has to experience to be something that is usable.  But we don't all then go and 
write a new service on top of google.  It does what is needed to find something 
to look at.  But, it still then has the struggle that you have to click on a 
link, and wait for the page to load in order to validate which content is 
appropriate to your needs.  The textual summaries don't always provide the right 
"view" of what you are looking for, even through they've tried to provide a 
textual view of the document fragment that contains what you searched for.

It is the load-before-need-is-established behavior that I am trying to raise the 
red flag about.  We need a complete different LUS if there are more than a 
handful of services that match a lookup request.

I feel really strongly against forcing clients to resolve 10s of services much 
less 100s just to see what they might choose to use.  Entry and service 
interface based filtering works to trim results dramatically.  But in the case 
of a ServiceUI discovery where you are just looking for services that have a

Service registration with a ServiceRegistrar implementation is the "I am alive" 
moment, and I think we need to maintain that single point of information because 
doing otherwise spreads knowledge out into more than one place and makes for a 
distributed database that I don't feel we don't need to add to the mix.

> What you seem to be doing is constructing a navigation structure for a GUI
> direct from the LUS per client. Why wouldn't you have some separate "service
> summary service" that builds up this navigation structure by tracking
> service arrivals and such and creating the necessary descriptive text for
> clients to pull down in one big bang?

There is some tangible benefits I can see in doing this, but I don't like the 
idea that this is another layer between the client and the LUS regarding 
availability and liveness.  As I said above, the LUS has the information, but 
it's the "unmarshalling" (ServiceRegistrar's APIs) and PreferredClassLoading 
that don't allow a client of the LUS to avoid downloading what they don't need 
to download.  That's why I went specifically at those two sources.

If I didn't have the sources to mess with, I am sure I would have had a 
different approach or done something without Jini even.

> You get caching for staters, no need to download a bunch of services until
> later and thus far fewer downloads especially as things like Entry's can be
> rendered into text reps as part of this process of building navigation
> information. (heck you could even insist all Entry's provide a toString for
> use on your platform). No reason too why the summary service couldn't build
> on demand trees for some specified set of Entry types. It could cache those
> and update them too such that if another client requests the same tree it
> gets it straight back.
>
> I guess my summary would be, I feel like you've gone for a "deep" solution
> that builds into existing infrastructure instead of building extensions atop
> that specifically tackle your problems in a more "meta" fashion. Please
> don't take offence at that, I'm not for a moment saying it's a mistake, just
> one path to solution and given what I've heard so far I think there's
> another path that hasn't been ruled out yet.

I'm good with this discussion.  That never happened before, even at the Jini 
Community meeting and later when I pushed Reef out onto java.net.

I want to really stress what I feel are the primary issues for me and my 
customers.  I also feel that given the larger set of services that might be 
visible on a broader network environment, such as the internet, that these issue 
will be important to others as well.  It sounds like Chris at least did the 
caching of jars.

Imagine if ServiceRegistrar included an XML or other string document that could 
be used by applications to get "client display" information, instead of just 
Entry object values, and that PreferredClassLoader got the PREFERRED.LIST from 
the annotation String instead of from a META-INF entry in the jar file.

Then, the ServiceRegistrar could still be a primary source of service 
registration and lookup but would include some flexibility for local needs.  We 
could even cause the XML to be constructed by the serialization process so that 
it looked something like:

<entries>
	<entry class="net.jini.lookup.entry.Name">
		<field name="name" type="java.lang.String" value="My Service"/>
	</entry>
</entries>

All the public fields would be enumerated and specified with values that would 
allow complete use of them (when they have appropriate toString() values) and 
thus we can get the text from the Entry objects directly.
`
We'd just provide an API that provided the programmatic iteration over this 
information without XML being directly visible.

So the existing lookup would be fine, and ServiceItem could have an additional 
XML string field to pass to a new API.

Just thinking out loud here...

Gregg

Mime
View raw message