Mailing-List: contact user-help@avro.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@avro.apache.org
Received-SPF: pass (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=X-YMail-OSG:Received:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding;
  b=TiH88cJxUWoyhZr0KQhwG7cFl9b5ijIBEtUZkqHsxqzwtHhN1ixdQBeOdbzwXXzkFxZEaBTadnLhlxVs3ZysPyTuS9ib/VVrcarH+SWxg39roszXbCaRExyU8C+yTkgzo2D2GY8YwdaEoediIHIbnHvWd2uBwaxk1yU3QPNdeS0=;
References: <602782.32870.qm@web33804.mail.mud.yahoo.com>
 <4D5F0C59.10900@apache.org>
Message-ID: <1317764980.41773.YahooMailNeo@web33804.mail.mud.yahoo.com>
Date: Tue, 4 Oct 2011 14:49:40 -0700 (PDT)
From: Wade Chandler <hwadechandler-apache@yahoo.com>
Reply-To: Wade Chandler <hwadechandler-apache@yahoo.com>
Subject: Re: Using Avro with something like OSGi or NetBeans RCP;
 modular Java systems dealing with classloader separation
To: "user@avro.apache.org" <user@avro.apache.org>
In-Reply-To: <4D5F0C59.10900@apache.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

>________________________________=0A>From: Doug Cutting <cutting@apache.org=
>=0A>To: user@avro.apache.org=0A>Sent: Friday, February 18, 2011 7:18 PM=0A=
>Subject: Re: Using Avro with something like OSGi or NetBeans RCP; modular =
Java systems dealing with classloader separation=0A=0ADoug,=0A=0AIt has bee=
n a while since you replied, and I am just now finally getting back to Avro=
 for some things I'm doing. Thanks for the information by the way. The rest=
 is inline...=0A=0A=0A>=0A>On 02/18/2011 01:16 PM, Wade Chandler wrote:=0A>=
> I have had different thoughts. I'm wanting to use as much of Avro which i=
s=0A>> already available as possible. The top two seem to be:=0A>>=0A>> 1) =
Write my own server implementation which handles multiple responders.=0A>=
=0A>This seems like a sub-optimal approach.=0A>=0A>> 2) Write a responder i=
mplementation which takes multiple providers and builds a=0A>> unified prot=
ocol to be parsed.=0A>=0A>This sounds better to me.=0A>=0A>You might be abl=
e to use SpecificResponder directly if you construct a=0A>java.lang.reflect=
.Proxy that implements all of the module interfaces and=0A>whose Invocation=
Handler dispatches to the appropriate module=0A>implementation.=A0 So this =
might look something like:=0A>=0A>public static Object createProxy(Class[] =
interfaces, Object[] impls);=0A>=0A>This proxy could then be passed to Spec=
ificResponder as the=0A>implementation.=A0 You'd then also need to construc=
t a protocol that=0A>appends the messages of all of the modules.=A0 Note ho=
wever that with this=0A>approach no two modules could contain messages of t=
he same name.=0A>=0A=0AThis definitely sounds doable on the server side. Si=
milarly I suppose I could do something similar on the client with the reque=
ster. Something I have been thinking about though the more I look at this i=
s that it would be extremely handy if RPC would allow multiple protocols. I=
 know that would mean quite a few languages would have to be changed for su=
ch a thing, but seems that wouldn't be too hard since the general mechanics=
 are already there.=0A=0A=0AI looked in Jira for such a request and didn't =
see one. Any such plans for 1.6? Would you guys be open to such support bei=
ng added? I have been thinking about forking, just to get it worked in, and=
 then seeing what folks thought about it and trying to get that worked back=
 into Avro. My fork would be private until such a thing took place as I wou=
ldn't want a competing project, but would really just want to use it for so=
me of my own things until it was an official piece of the project.=0A=0ARig=
ht now, to get something similar working without multiple protocol support =
and thus having different namespaces which I can use to herd messages I hav=
e a couple ideas:=0A=0A1) Based on your suggestion use a proxy and merge re=
sponder interfaces together. A generic responder could then direct traffic =
based on the message name (as the protocol would only have a single namespa=
ce) to the correct implementation. Do something similar on the client side =
for the requester. This assuming there would be a modular client as well. I=
n this situation I need to parse the protocol files of each module to merge=
 them into a single unified protocol. The protocols all become part of the =
applications namespace and the schemas may still retain their original name=
spaces.=0A=0A=0A2) Define a general protocol contract at the application le=
vel. All responders will implement the same protocol; which for the sake of=
 code generation could have different namespaces. Not having methods matchi=
ng the correct signature will be an error, and thus I expect them all to ha=
ve the same exact interface. There will be a generic request and a response=
 which simply takes some generic object. The different modules schemas will=
 be merged. The requests schema objects namespace will be used to direct tr=
affic. The schemas technically become the requests and replies; data in and=
 data out.=0A=0ABoth seem doable with Avro today albeit they just don't fee=
l "right". I'm very interested in multiple protocol support if there is any=
 chance such support being added would be accepted unless it is already on =
the list of things to do; in that case I would gladly help out if I can.=0A=
=0A=0A>> When modules are perhaps live updated in the=0A>> server send some=
 message telling all clients that connections must be=0A>> reconnected unle=
ss the updates require client updates which can force them to=0A>> restart =
in which case they need to reconnect anyways. After this message has=0A>> b=
een sent out, close all connections from the server side, rebuild the proto=
col=0A>> from available providers, restart the server. Clients will have a =
period of time=0A>> before they timeout after trying once receiving the mes=
sage.=0A>=0A>Could you simply restart the server, closing all client connec=
tions when=0A>they complete their currently executing request, without send=
ing any=0A>special message to the clients?=A0 Then the clients would simply=
 need to=0A>be written to retry requests when they get a connection closed =
exception.=0A>=0A=0AYes, I think that would work fine.=0A=0A=0A>> The snag =
here seems to be on the client side. It isn't straight forward exactly=0A>>=
 how the client protocols which were merged on the server would react to th=
is.=0A>> Perhaps it isn't a big deal if some how versions are the same and =
the records=0A>> and messages match up. Not sure exactly as I'm just beginn=
ing.=0A>=0A>The responder doesn't currently check that the client and serve=
r=0A>protocol names match, so this should mostly just work.=A0 The only pro=
blem=0A>I see is if two protocols have a message with the same name (as=0A>=
mentioned above).=A0 If you need to permit that, then you couldn't use the=
=0A>proxy approach above, but would need to use a responder that wraps or=
=0A>extends SpecificResponder and dispatches to the right implementation=0A=
>instance based on the protocol name, not just the message name.=A0 The=0A>=
client's protocol name is available through a ThreadLocal as=0A>Responder.g=
etRemote().=0A>=0A=0A=0ASeems in this situation I would have to have specif=
ic client connections versus all messages traveling over a single connectio=
n with merged protocols. i.e. Each individual client connection performs it=
s own hand shake and thus each one is using a specific protocol on the conn=
ection. Is that correct? Well, unless I do as we talked above on the client=
 side as well.=0A=0A=0AThis too makes me feel like a better solution would =
be to allow multiple protocols to be used for RPC. Then a simple wrapping r=
esponder or requester could handle directing the messages to the appropriat=
e specific instances. Seems a good addition to the API. Seems outside of th=
e multiple protocol support, the logic already there could be used to creat=
e specialized requesters/responders which=0A=0Aproxy to others based off me=
ssage names and their protocols; well, the C/C++ versions may need some spe=
cial things to return method pointers or something, but generally I think t=
hat is fairly accurate.=0A=0AThanks again for the advice and help, and than=
ks to you and all other devs for Avro, I like it much better than protocol =
buffers,=0A=0AWade=0A=0A=A0=0A=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=0AWade Chandler=0ASoftware Engineer and Developer=0ANetBeans Drea=
m Team Member and Contributor=0A=0A=0Ahttp://wiki.netbeans.org/wiki/view/Ne=
tBeansDreamTeam=0Ahttp://www.netbeans.org