hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kim Chew <kchew...@gmail.com>
Subject Re: coprocessorExec got stucked with generic type
Date Tue, 25 Jun 2013 19:17:54 GMT
I think I have run into a similar situation like Pavel.

My method returns Map<Long, ArrayList<Foo>>, where Foo is

public class Foo implements Writable {
     String something;
     long    counter1;
     long    counter2;

     blah........
};

And I got the following exception when I called my coprocessor method from
a client,

java.io.NotSerializableException: Foo
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1164)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:330)
    at java.util.ArrayList.writeObject(ArrayList.java:570)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:945)
    at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1469)
    at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:330)
    at java.util.HashMap.writeObject(HashMap.java:1001)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:945)
    at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1469)
    at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:330)
    at
org.apache.hadoop.hbase.io.HbaseObjectWritable.writeObject(HbaseObjectWritable.java:540)
    at
org.apache.hadoop.hbase.client.coprocessor.ExecResult.write(ExecResult.java:76)
    at
org.apache.hadoop.hbase.io.HbaseObjectWritable.writeObject(HbaseObjectWritable.java:525)
    at
org.apache.hadoop.hbase.io.HbaseObjectWritable.write(HbaseObjectWritable.java:335)
    at
org.apache.hadoop.hbase.ipc.HBaseServer$Call.setResponse(HBaseServer.java:365)
    at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1442)

What I do not understand is, how does the predicate at line 526 in
HbaseObjectWritable is true if Foo does not implement Serializable?

 } else if (Serializable.class.isAssignableFrom(declClass))

BTW, making Foo serializable worked.

Regards,
Kim


On Tue, Jun 11, 2013 at 9:48 AM, Gary Helmling <ghelmling@gmail.com> wrote:

> Does your NameAndDistance class implement org.apache.hadoop.io.Writable?
>  If so, it _should_ be serialized correctly.  There was a past issue
> handling generic types in coprocessor endpoints, but that was fixed way
> back (long before 0.94.2).  So, as far as I know, this should all be
> working, assuming that NameAndDistance can be serialized.
>
>
> On Mon, Jun 10, 2013 at 9:36 AM, Pavel Hančar <pavel.hancar@gmail.com
> >wrote:
>
> >  It's org.apache.commons.lang.SerializationUtils
> > I have it in hbase-0.94.2-cdh4.2.1/lib/commons-lang-2.5.jar
> >  Pavel
> >
> >
> > 2013/6/10 Ted Yu <yuzhihong@gmail.com>
> >
> > > I searched for SerializationUtils class in hadoop (both branch-1 and
> > > branch-2)
> > > I also searched for SerializationUtils in hbase codebase.
> > >
> > > I didn't seem to find it.
> > >
> > > Is it an internal class of your project ?
> > >
> > > Cheers
> > >
> > > On Mon, Jun 10, 2013 at 6:11 AM, Pavel Hančar <pavel.hancar@gmail.com
> > > >wrote:
> > >
> > > >  I see, that's probably big nonsense to return ArrayList (or array)
> of
> > > > another classes from coprocessor, because it's a list of pointers.
> The
> > > > solution is to serialize it to byte[] by
> SerializationUtils.serialize(
> > > > Serializable<
> > > >
> > >
> >
> http://java.sun.com/javase/6/docs/api/java/io/Serializable.html?is-external=true
> > > > >
> > > >  obj).
> > > >   Pavel
> > > >
> > > >
> > > > 2013/6/10 Pavel Hančar <pavel.hancar@gmail.com>
> > > >
> > > > >   Hello,
> > > > > can I return from an EndPoint a generic type? I try to return
> > > > > ArrayList<NameAndDistance> from an EndPoint method (where
> > > NameAndDistance
> > > > > is a simpe class with two public variables name and distance). But
> > > when I
> > > > > return  unempty ArrayList, the coprocessorExec get stucked.
> > > > >   Thanks,
> > > > >   Pavel Hančar
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message