hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "terry zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6533) [replication] replication will be block if WAL compress set differently in master and slave configuration
Date Wed, 05 Sep 2012 07:13:07 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448537#comment-13448537
] 

terry zhang commented on HBASE-6533:
------------------------------------

this is because of master sending the hlog entry in compress mode. But Slave do not know about
it. So when slave ipc hbaseserver deserilize the buffer and read the hlog entry fields error
will happen. We can let the Master send the buffer in none compress mode. then whether master
use hlog compression or not. Slave both can work fine
                
> [replication] replication will be block if WAL compress set differently in master and
slave configuration
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-6533
>                 URL: https://issues.apache.org/jira/browse/HBASE-6533
>             Project: HBase
>          Issue Type: Bug
>          Components: replication
>    Affects Versions: 0.94.0
>            Reporter: terry zhang
>            Priority: Critical
>
> as we know in hbase 0.94.0 we have a configuration below
>   <property>
>     <name>hbase.regionserver.wal.enablecompression</name>
>          <value>true</value>
>   </property>
> if we enable it in master cluster and disable it in slave cluster . Then replication
will not work. It will throw unwrapRemoteException again and again in master cluster.
> 2012-08-09 12:49:55,892 WARN org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
Can't replicate because of an error
>  on the remote cluster: 
> java.io.IOException: IPC server unable to read call parameters: Error in readFields
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>         at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
>         at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
>         at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.shipEdits(ReplicationSource.java:635)
>         at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:365)
> Caused by: org.apache.hadoop.ipc.RemoteException: IPC server unable to read call parameters:
Error in readFields
>         at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:921)
>         at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:151)
>         at $Proxy13.replicateLogEntries(Unknown Source)
>         at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.shipEdits(ReplicationSource.java:616)
>         ... 1 more 
> This is because Slave cluster can not parse the hlog entry .
> 2012-08-09 14:46:05,891 WARN org.apache.hadoop.ipc.HBaseServer: Unable to read call parameters
for client 10.232.98.89
> java.io.IOException: Error in readFields
>         at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:685)
>         at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:586)
>         at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:635)
>         at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1292)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1207)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:735)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:524)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:499)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.EOFException
>         at java.io.DataInputStream.readFully(DataInputStream.java:180)
>         at org.apache.hadoop.hbase.KeyValue.readFields(KeyValue.java:2254)
>         at org.apache.hadoop.hbase.regionserver.wal.WALEdit.readFields(WALEdit.java:146)
>         at org.apache.hadoop.hbase.regionserver.wal.HLog$Entry.readFields(HLog.java:1767)
>         at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:682)
>         ... 11 more 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message