avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marko Milicevic" <MMilice...@pssd.com>
Subject RE: reflected char array returning null package.
Date Tue, 01 Dec 2009 20:54:45 GMT
Thanks for the offer Hiral, but without doc i'm not sure what i need.
 
My original usecase was to serialize arbitrary Java objects over the
wire.  I was looking for something more efficient than Java
serialization.
 
I read of several benchmarks that showed favorable performance for Avro,
so here i am.
 
But it seems that there are significant constraints on the type of Java
object that can be serialized.  If that is the case then i tend to think
that Doug's approach would make the most sense in the long run (ie.
first define a Avro schema/protocol, then generate the Java).
 
Thanks again.
 
Marko.
.

________________________________

From: tazan007 [mailto:tazan007@gmail.com] 
Sent: Tuesday, December 01, 2009 2:57 PM
To: avro-user@hadoop.apache.org
Subject: Re: reflected char array returning null package.


Marko, I did make some changes to avro 1.0 back when I was evaluating it
for RPC and added support for it to convert String to Utf8 and Date to
long for converting Java to Avro but I never had to go from Avro back to
Java once I had the data in Avro.

If you want, I could try to apply the changes I made to 1.2 if it fits
your use-case.

-Hiral


On Tue, Dec 1, 2009 at 11:42 AM, Marko Milicevic <MMilicevic@pssd.com>
wrote:


	Thanks Doug.
	
	Is there any documentation on the Specific API and the Specific
schema
	format?
	
	Marko.
	.
	
	
	-----Original Message-----
	From: Doug Cutting [mailto:cutting@apache.org]
	Sent: Tuesday, December 01, 2009 1:22 PM
	To: avro-user@hadoop.apache.org
	Subject: Re: reflected char array returning null package.
	
	
	In Avro 1.2's reflect implementation, Strings must be
represented with
	Avro's Utf8 class.  Folks reasonably complained that this is not
very
	useful, and it's being fixed for Avro 1.3.
	
	https://issues.apache.org/jira/browse/AVRO-80
	
	I'm currently investing a lot of effort improving reflect, and
reflect
	should work considerably better in 1.3.  That said, unless you
have an
	existing RPC codebase that you need to move onto Avro with
minimal
	disruption, the specific API is much easier to use than the
reflect API.
	
	 Avro schemas are mostly an expressive subset of Java, so
mapping Avro
	to Java is much simpler than mapping Java to Avro.
	
	Doug
	
	Marko Milicevic wrote:
	> Hello.  I'm running the following code under the avro 1.2.0
release
	> and jdk 1.6.0_10.
	>
	> public interface ISink
	> {
	>     public void send( String msg );
	> }
	>
	> public class Sink implements ISink
	> {
	>       public void send( String msg )
	>       {
	>       }
	> }
	>
	>
	> public void test()
	> {
	>     SocketServer server =
	>       new SocketServer(new SpecificResponder(
	>             ISink.class, new Sink()), new
InetSocketAddress(3333));
	>
	>     SocketTransceiver client =
	>       new SocketTransceiver(new
InetSocketAddress(server.getPort()));
	>
	>     ISink proxy =
(ISink)SpecificRequestor.getClient(ISink.class,
	> client);
	>
	>     for (int i = 0; i < 1000; i++ )
	>     {
	>         proxy.send("testing");
	>     }
	>
	>     client.close();
	>     server.close();
	> }
	>
	>
	> First I was getting errors until I learned of paranamer.
	> I thought I read in the doc, that no pre-processing would be
	necessary?
	>
	> I'm fine with having to run paranamer, but want to be sure
that it is
	> required?
	>
	>
	> Now after processing with paranamer I get the following error
trying
	> to run...
	>
	> Exception in thread "main" java.lang.NullPointerException
	>       at
	>
org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:259)
	>       at
	>
org.apache.avro.reflect.ReflectData.createFieldSchema(ReflectData.java
	> :3
	> 03)
	>       at
	>
org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:291)
	>       at
	>
org.apache.avro.reflect.ReflectData.createFieldSchema(ReflectData.java
	> :3
	> 03)
	>       at
	>
org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:291)
	>       at
	>
org.apache.avro.reflect.ReflectData.getMessage(ReflectData.java:339)
	>       at
	>
org.apache.avro.reflect.ReflectData.getProtocol(ReflectData.java:318)
	>       at
	>
org.apache.avro.reflect.ReflectResponder.<init>(ReflectResponder.java:
	> 45
	> )
	>       at
	>
org.apache.avro.reflect.ReflectResponder.<init>(ReflectResponder.java:
	> 41
	> )
	>
	> //
	>
org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:259)
	>   } else if (type instanceof Class) {
	>       Class c = (Class)type;
	>       String name = c.getSimpleName();
	>>>      String space = c.getPackage().getName();
	>       if (c.getEnclosingClass() != null)          // nested
class
	>         space = c.getEnclosingClass().getName() + "$";
	>       String fullName = c.getName();
	>       Schema schema = names.get(fullName);
	>       if (schema == null) {
	>
	> Stepping though the debugger, the type appears to be "class
[C", with
	> the name "value".  I assume that is the "value" char[] in the
String
	> msg.
	>
	> Should a Class of type "class [C", return a package of null?
	>
	> If so, any idea what I'm doing incorrectly?
	>
	> Thank you.
	>
	> Marko.
	> .
	



Mime
View raw message