river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Firmstone <j...@zeus.net.au>
Subject Re: MarshalledServiceItem
Date Wed, 02 Feb 2011 23:47:59 GMT
I like the fact that Gregg's pushed the envelope, he's pushed Jini 
beyond it's current capability.

Now I'm wondering if there's a way around the Preferred Classes problem.

In Reggie's proxy implementation, during registration, fields in Entry 
classes, if they can be resolved to standard java classes, are, however 
they are sent as a MarshalledWrapper (wrapping a MarshalledInstance), if 
they can't.

For MarshalledServiceItem, we don't have to honour preferred classes, 
since we can unmarshall everything correctly when the unmarshall method 
is called, in other words we can unmarshall the Entry's again, using 
preferred class loading and downloaded codebases.

For implementation of MarshalledServiceItem, the reggie implementation 
EntryClass can be first used to identify an entry we're interested in, 
first we check if the name matches an Entry's Class name we're 
interested in, secondly we resolve all fields of these Entry's locally 
by providing a get() method on MarshalledInstance that ignores codebase 
annotations, leaving any field that cannot be unresolved as null.

So what we get at the client is a number of Entry's that may be fully 
unmarshalled, or partly so, which should give us enough information to 
determine if the service is interesting.

When the client finds a field in an Entry in the ServiceItem, set to 
null, and it wants to know more, it must then unmarshall the 
MarshalledServiceItem, a process which then retrieves all codebases and 
correctly unmarshalls all Entry's and the Service, the original array of 
Entry's contained in the MarshalledServiceItem is discarded, so the fact 
that these were previously resolved locally doesn't matter.

Then we could leave Preferred classes alone.

Thoughts?

Peter.

Gregg Wonderly wrote:
> I am finding my self being slightly defensive sounding and I'm not 
> wanting to be defensive about my implementation.  I want to defend the 
> features that I need and thus made changes in lookup to provide.  
> Hopefully the inline below is not degenerating into to much 
> reiteration of the issues.  There are some additional thoughts about 
> other ways that lookup could be changed to provide the "string" 
> information without a proxy service.
>
> See below...
>
> On 2/2/2011 3:26 AM, Dan Creswell wrote:
>> Inline and leaving context for now.....
>>
>> On 1 February 2011 19:12, Gregg Wonderly<gergg@cox.net>  wrote:
>>
>>> On 2/1/2011 3:17 AM, Dan Creswell wrote:
>>>
>>>> So...
>>>>
>>>> On 1 February 2011 01:29, Gregg Wonderly<gregg@wonderly.org>   wrote:
>>>>
>>>>   One of the important things for my use of my changes to the
>>>>> ServiceRegistrar interface was separating deserialization of the 
>>>>> Entrys
>>>>> from
>>>>> the service object.
>>>>>
>>>>
>>> Note "separating" used here.  I was trying to say that I need to be 
>>> able to
>>> "pick" which Entry objects to deserialize, and deserialize them 
>>> independent
>>> of deserializing the service proxy/object.
>>>
>>>
>>>   Being able to defer downloading was also controlled by the changes to
>>>>> ClassLoading to include the "neverPrefer" settings on a class 
>>>>> name.  I
>>>>> can
>>>>> do that now through the recent RMIClassLoaderSPI override 
>>>>> capabilities.
>>>>>   But
>>>>> I still need to be able to control which Entry is deserialized.  I 
>>>>> need
>>>>> to
>>>>> be able to ask what classes, by name, are in the namespace of each 
>>>>> Entry
>>>>> object as my implementation allowed. I had made changes to Reggie
>>>>> marshalling to use reflection to get the class hierarchy, and then
>>>>> provided
>>>>> access to the list of classnames as part of the returned, marshalled
>>>>> value
>>>>> in my API.
>>>>>
>>>>>
>>>>>   Digging through classes belonging to a service implementation 
>>>>> that is
>>>> designed to be encapsulated/not your concern? Does that sound right? I
>>>> gotta
>>>> say, it sounds horribly invasive, very hacky and speaks of an overly
>>>> complex
>>>> solution as the result of treating symptoms, not problems.
>>>>
>>>
>>> Okay, I want to go through the problems that I was dealing with, the
>>> attributes of the current mechanisms, and how I inserted points of 
>>> control
>>> to manage these issues.
>>>
>>> First, let's look at the simple client mechanisms I wanted to have.  
>>> I have
>>> a desktop environment, which does lookup, and then shows treeviews of
>>> services based on "machine"->"group"->Service Instance structure.  The
>>> deployed services contain Entry objects that provide values for 
>>> these three
>>> name spaces.  Since I need to look at these, they can not be 
>>> preferred by
>>> the service or I can't read them.  There may be a subclass that the 
>>> client
>>> has used, and that can be preferred no problem.  There are some 
>>> other items
>>> in the Entrys that I need too, such as icons.  Any of the Entry 
>>> objects that
>>> I need access to, and any classes which they are dependent on, and 
>>> which are
>>> publicly visible classes/interfaces, because of that attribute 
>>> (public) can
>>> not be realistically preferred.
>>>
>>>
>> These Entry types are part of the basic infrastructure of your system 
>> aren't
>> they? Why download them at all? I guess because you have to "teach" 
>> it to
>> the LUS via code-download? Or you could just put them on all classpaths
>> including LUS? S'far as I can see you've got an extended LUS 
>> infrastructure
>> that everyone shares thus codebase seems like the wrong choice for 
>> these.
>>
>> Sure user's can subclass Entry types but do they? Do you even want 
>> them to?
>> Why? Thus far I've seen many subclass Entry's for JavaSpaces and, I 
>> think,
>> zero for LUS type annotations.
>>
>> See my other thread with Peter but it seems to me that you have some 
>> core
>> additions to LUS infrastructure and a need to be able to filter what you
>> don't want to see/is outside of a client's knowledge base. Because 
>> it's that
>> inability to filter that then leads to lots of additional downloading.
>
> The basic issue is that I need to show users what services exist.  The 
> LUS maintains a very useful view of the health of the network.  Asking 
> it what services exist has to happen by some 'client'.  Remember that 
> my customers techs are using the client out in the field, on a slow 
> network.  So whether they enumerate something from the LUS or from 
> another "service", there will be network traffic.
>
> It might be possible to enumerate services at a aggregation service, 
> but then you have to unmarshall all of the services there to get the 
> "data" that you'd send to a client.  I don't want to have 100s of 
> proxies active on the network for this mechanism, really.  The 
> deferred unmarshalling in discussion with Peter would help deal with 
> this by allowing such an aggregation service to not have to unmarshall 
> the proxy.  That was my initial intent.  It was only after I got that 
> implemented that I found out how the PREFERRED.LIST really kept it 
> from working at all, and so I had to do something to keep the 
> PREFERRED.LIST from being accessed when not needed yet.
>
> The service aggregation becomes a single point of failure and I'd need 
> to replicate it to every server that had an LUS on it.  Then, it 
> really becomes more like a secondary look up service, and so I'd still 
> have to ask, is the LUS API really in the right form if there is a 
> "mirror" of it needed?  Note that I made it possible to pick through a 
> lot of the class information for the Entry and the service Proxy in 
> reef's work because I was trying to get around having to mess with the 
> never-preferred business.  But in the end, I just could not find any 
> other way, because as soon as I wanted to unmarshall an Entry to get a 
> "Name" and a "Group" and an "Icon" for my UI, PreferredClassLoading 
> went after the PREFERRED.LIST, even though not a single class that I 
> was using would be resolved from the codebase.
>
> I'm not sure how I would "not activate the codebase" in some client.  
> It needs the Entry object, fully resolved.  I want the codebase to 
> work right, so I'm not gonna just throw in a proxy context class 
> loader that resolves everything locally.  If there is content in an 
> Entry that is an unknown class, I will suffer the download to make 
> sure that it resolves to the right codebase.
>
> Note that I am not wanting to focus on the never-prefer issue as 
> something that I need.  What I actually need is to defer download of 
> any part of the codebase until the service is "activated".
>
> What might be interesting is to figure out how to stuff the preferred 
> list into the MarshalledObject annotation so that PreferredClassLoader 
> would not need to retrieve the codebase elements to resolve how to 
> perform class loading.
>
> Currently, there is a lot of backward compatibility involved in how 
> the annotation is managed.
>
> The namespace change that we are discussing for com.sun.jini, might be 
> the right moment to introduce any compatibility issues with 
> MarshalledObject and the use of the RMIClassLoader annotation.
>
>>> So my first assertion is that Entry objects will hardly ever be 
>>> preferred,
>>> and thus downloading of the PREFERRED.LIST is not a necessary step to
>>> correctly resolve them for my use.
>>>
>>> The next issues is this PREFERRED.LIST file.  The implementation of 
>>> that
>>> technology is that the file is retrieved from the first jar file 
>>> listed in
>>> the codebase.  So, in order to resolve any class in the proxy or Entry
>>> values, there must be a download from the associated codebase server.
>>>
>>> Now, one of the first things that I think about with distributed 
>>> computing
>>> is the business of partial failure and limiting that impact to the 
>>> overall
>>> performance of the system.  My deployments at my customers' sites 
>>> consist of
>>> copies of the same service platform with the same services on it.  
>>> There are
>>> copies for both redundancy and for capacity.
>>>
>>> Each instance of my platform has as a starting point 20 some 
>>> services each
>>> with a separate codebase, but with many common jars such as 
>>> jsk-dk.jar.  If
>>> a lookup service responds with a list of services but the codebase 
>>> server
>>> doesn't respond with jars, then any attempt to download limits the
>>> performance of the display of available services, and I considered an
>>> "incorrect list of available services" less important than 
>>> non-display or
>>> slow-display of available services.
>>
>> As Michal says elsewhere, are you sure the solution is to try and 
>> leverage
>> LUS to handle display challenges like this?
>
> I need the data with minimal network traffic.  The LUS lookup is even 
> something that generates some delays.  But, the UI, as it populates, 
> feels largely like a web page load.  I've reached a point of 
> acceptable performance that works for now.
>
> Service lookup and selection has always been a point of contention as 
> you well know.  I had always been on the side of the fence that said 
> the LUS works fine the way it is.  It wasn't until I tried to figure 
> out how to manage 100s of services visible and only 1 being 
> interesting that I suddenly found myself setting on the fence asking, 
> "If it almost does the right thing, is there a way to maintain the 
> same interface, but add a new interface with more functionality that 
> would keep me from having to completely ignore the LUS?"
>
> So, I tried something out.  I find that given how Reggie works and is 
> implemented, that the addition of the functionality as shown in Reef 
> does provide some new opportunities.  But, delayed unmarshalling 
> alone, does not provide for no-download until needed.  
> PreferredClassLoading is still in the mix.  My solution to that, which 
> I think does need to be revisited.  We need to decide if the 
> PREFERRED.LIST is really a problem because it causes a download. 
> Service registration data will always require download, and so just to 
> use the Entry values in a UI or some other client will still require 
> download of all the elements of the codebase needed to resolve the 
> classes.
>
> In one explanation of how things are currently working with River, 
> without my changes, I suggested that using Reggie and preferred class 
> loading would be like opening up a Google session, running a query, 
> and then having to download every matched pages content before you 
> could see the list of search results.
>
> Google provides the type of service that you describe by changing what 
> the user has to experience to be something that is usable.  But we 
> don't all then go and write a new service on top of google.  It does 
> what is needed to find something to look at.  But, it still then has 
> the struggle that you have to click on a link, and wait for the page 
> to load in order to validate which content is appropriate to your 
> needs.  The textual summaries don't always provide the right "view" of 
> what you are looking for, even through they've tried to provide a 
> textual view of the document fragment that contains what you searched 
> for.
>
> It is the load-before-need-is-established behavior that I am trying to 
> raise the red flag about.  We need a complete different LUS if there 
> are more than a handful of services that match a lookup request.
>
> I feel really strongly against forcing clients to resolve 10s of 
> services much less 100s just to see what they might choose to use.  
> Entry and service interface based filtering works to trim results 
> dramatically.  But in the case of a ServiceUI discovery where you are 
> just looking for services that have a
>
> Service registration with a ServiceRegistrar implementation is the "I 
> am alive" moment, and I think we need to maintain that single point of 
> information because doing otherwise spreads knowledge out into more 
> than one place and makes for a distributed database that I don't feel 
> we don't need to add to the mix.
>
>> What you seem to be doing is constructing a navigation structure for 
>> a GUI
>> direct from the LUS per client. Why wouldn't you have some separate 
>> "service
>> summary service" that builds up this navigation structure by tracking
>> service arrivals and such and creating the necessary descriptive text 
>> for
>> clients to pull down in one big bang?
>
> There is some tangible benefits I can see in doing this, but I don't 
> like the idea that this is another layer between the client and the 
> LUS regarding availability and liveness.  As I said above, the LUS has 
> the information, but it's the "unmarshalling" (ServiceRegistrar's 
> APIs) and PreferredClassLoading that don't allow a client of the LUS 
> to avoid downloading what they don't need to download.  That's why I 
> went specifically at those two sources.
>
> If I didn't have the sources to mess with, I am sure I would have had 
> a different approach or done something without Jini even.
>
>> You get caching for staters, no need to download a bunch of services 
>> until
>> later and thus far fewer downloads especially as things like Entry's 
>> can be
>> rendered into text reps as part of this process of building navigation
>> information. (heck you could even insist all Entry's provide a 
>> toString for
>> use on your platform). No reason too why the summary service couldn't 
>> build
>> on demand trees for some specified set of Entry types. It could cache 
>> those
>> and update them too such that if another client requests the same 
>> tree it
>> gets it straight back.
>>
>> I guess my summary would be, I feel like you've gone for a "deep" 
>> solution
>> that builds into existing infrastructure instead of building 
>> extensions atop
>> that specifically tackle your problems in a more "meta" fashion. Please
>> don't take offence at that, I'm not for a moment saying it's a 
>> mistake, just
>> one path to solution and given what I've heard so far I think there's
>> another path that hasn't been ruled out yet.
>
> I'm good with this discussion.  That never happened before, even at 
> the Jini Community meeting and later when I pushed Reef out onto 
> java.net.
>
> I want to really stress what I feel are the primary issues for me and 
> my customers.  I also feel that given the larger set of services that 
> might be visible on a broader network environment, such as the 
> internet, that these issue will be important to others as well.  It 
> sounds like Chris at least did the caching of jars.
>
> Imagine if ServiceRegistrar included an XML or other string document 
> that could be used by applications to get "client display" 
> information, instead of just Entry object values, and that 
> PreferredClassLoader got the PREFERRED.LIST from the annotation String 
> instead of from a META-INF entry in the jar file.
>
> Then, the ServiceRegistrar could still be a primary source of service 
> registration and lookup but would include some flexibility for local 
> needs.  We could even cause the XML to be constructed by the 
> serialization process so that it looked something like:
>
> <entries>
>     <entry class="net.jini.lookup.entry.Name">
>         <field name="name" type="java.lang.String" value="My Service"/>
>     </entry>
> </entries>
>
> All the public fields would be enumerated and specified with values 
> that would allow complete use of them (when they have appropriate 
> toString() values) and thus we can get the text from the Entry objects 
> directly.
> `
> We'd just provide an API that provided the programmatic iteration over 
> this information without XML being directly visible.
>
> So the existing lookup would be fine, and ServiceItem could have an 
> additional XML string field to pass to a new API.
>
> Just thinking out loud here...
>
> Gregg
>


Mime
View raw message