hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Margus Roo <mar...@roo.ee>
Subject Re: Use CodedInputStream.setSizeLimit() to increase the size limit on Read more than 64M content
Date Tue, 09 Dec 2014 13:53:53 GMT
Hi more additional information

Connecting to bigdata3/192.168.80.232:60020
IPC Client (155412783) connection to bigdata3/192.168.80.232:60020 from 
margusja: wrote request header call_id: 16 method_name: "Get" 
request_param: true
IPC Client (155412783) connection to bigdata3/192.168.80.232:60020 from 
margusja: starting, connections 2
IPC Client (155412783) connection to bigdata3/192.168.80.232:60020 from 
margusja:*got response header call_id: 16, totalSize: 68266970 bytes*
IPC Client (155412783) connection to bigdata3/192.168.80.232:60020 from 
margusja: closing ipc connection to 
bigdata3.webmedia.int/192.168.80.232:60020: Protocol message was too 
large.  May be malicious.  Use CodedInputStream.setSizeLimit() to 
increase the size limit.
com.google.protobuf.InvalidProtocolBufferException: Protocol message was 
too large.  May be malicious.  Use CodedInputStream.setSizeLimit() to 
increase the size limit.
     at 
com.google.protobuf.InvalidProtocolBufferException.sizeLimitExceeded(InvalidProtocolBufferException.java:110)
     at 
com.google.protobuf.CodedInputStream.refillBuffer(CodedInputStream.java:755)
     at 
com.google.protobuf.CodedInputStream.isAtEnd(CodedInputStream.java:701)
     at 
com.google.protobuf.CodedInputStream.readTag(CodedInputStream.java:99)
     at 
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$Result.<init>(ClientProtos.java:4086)
     at 
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$Result.<init>(ClientProtos.java:4050)
     at 
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$Result$1.parsePartialFrom(ClientProtos.java:4149)
     at 
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$Result$1.parsePartialFrom(ClientProtos.java:4144)
     at 
com.google.protobuf.CodedInputStream.readMessage(CodedInputStream.java:309)
     at 
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$GetResponse.<init>(ClientProtos.java:5951)
     at 
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$GetResponse.<init>(ClientProtos.java:5898)
     at 
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$GetResponse$1.parsePartialFrom(ClientProtos.java:5989)
     at 
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$GetResponse$1.parsePartialFrom(ClientProtos.java:5984)
     at 
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$GetResponse$Builder.mergeFrom(ClientProtos.java:6282)
     at 
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$GetResponse$Builder.mergeFrom(ClientProtos.java:6171)
     at 
com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:220)
     at 
com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:912)
     at 
com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267)
     at 
com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(AbstractMessageLite.java:290)
     at 
com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(AbstractMessage.java:926)
     at 
com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(AbstractMessageLite.java:296)
     at 
com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(AbstractMessage.java:918)
     at 
org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1099)
     at 
org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:726)
IPC Client (155412783) connection to bigdata3/192.168.80.232:60020 from 
margusja: closed
IPC Client (155412783) connection to bigdata3/192.168.80.232:60020 from 
margusja: stopped, connections 1
IPC Client (155412783) connection to bigdata2/192.168.80.70:60020 from 
margusja: closed
IPC Client (155412783) connection to bigdata2/192.168.80.70:60020 from 
margusja: stopped, connections 0

Any hints?

Margus (margusja) Roo
http://margus.roo.ee
skype: margusja
+372 51 480

On 09/12/14 08:45, Margus Roo wrote:
> Hi
>
> I am getting:
> org.apache.hadoop.hbase.client.RpcRetryingCaller@4d6a54b0, 
> java.io.IOException: Call to regionserver1/192.168.81.166:60020 failed 
> on local exception: 
> com.google.protobuf.InvalidProtocolBufferException: Protocol message 
> was too large.  May be malicious.  Use CodedInputStream.setSizeLimit() 
> to increase the size limit.
>
> I am using Java API
> ...
>         // Get image from table
>         HTablePool pool = new HTablePool(getConf(), 1);
>         HTableInterface usersTable = pool.getTable(table);
>
>         Get g = new Get(Bytes.toBytes(uid));
>         Result r = usersTable.get(g);
>
>         //g.addFamily(Bytes.toBytes("info"));
>         byte[] b = 
> r.getValue(Bytes.toBytes(cf),Bytes.toBytes("content"));
>         String image = Bytes.toString(b);
>
>         FileOutputStream imageOutFile = new 
> FileOutputStream(outFileName);
>         imageOutFile.write(Base64.decodeBase64(image));
> ...
>
> As I understand then com.google.protobuf is limiting more than 64M 
> messages.
> Is that related? https://issues.apache.org/jira/browse/HBASE-11747
>
> Is there workaround?
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message