phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fulin Sun" <su...@certusnet.com.cn>
Subject Re: Re: Phoenix 4.4 do not work with CDH 5.4
Date Thu, 04 Jun 2015 02:17:02 GMT
Hi, James

Cannot agree more for being a CDH issue here. Cause phoenix has been included in the Cloudera
labs as a beta version, 

many users are pleased to use latest 4.4 phoenix with their CDH environment. So there can
be some leverage here : )

Actually we only changed 2 related source codes. And tends out to be some hbase jar incompatibility
between CDH hbase 

and Apache HBase. The details are listed below :

          1    phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/IndexSplitTransaction.java#L289
                   MetaTableAccessor.splitRegion : The native Apache HBase source code here
takes 5 params while the CDH hbase 
version takes 6 params with a replication INT parameter additionally. We modified the above
code passing the method with 
a default value 0.
  2    phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/LocalIndexMerger.java#L84
                   RegionMergeTransactionImpl.prepareMutationsForMerge : Also the method got
wrong parameters without the replication
parameter. And we also added as a default value 0.

We happened to see that the master branch has got the above 2 parameters rightly in specific
source code. Maybe we could use that 

and recompiled phoenix again. Before that, hope the team would provide some guidance and help.


Best regards,
Sun.




CertusNet 

 
From: James Taylor
Date: 2015-06-03 22:18
To: user@phoenix.apache.org
CC: dev
Subject: Re: Phoenix 4.4 do not work with CDH 5.4
I'd look toward Cloudera to help with this. Phoenix has no control over what's in the CDH
distro, so ensuring compatibility isn't really feasible. I'd encourage any and all Phoenix
users running on CDH to let Cloudera know that you'd like to see Phoenix added to their distro.
They've started down that path by including Phoenix in their Cloudera labs, which is a good
first step.

In the meantime, what kind of source changes did you need to make, Fulin? Perhaps we can merge
those into Phoenix if they're not too invasive and don't break b/w compatibility?

Thanks,
James

On Monday, June 1, 2015, Fulin Sun <sunfl@certusnet.com.cn> wrote:
Hi, 
We had been encountered exact issue here and got finally resolved through modifying phoenix-core


source code and recompiled it specifying the CDH 5.4 version. However, this would not be a
right direction.

As the latest CDH version had promoted to integrate with hbase 1.0.0 and many hbase users
are considering 

and using CDH platform, really hope the Phoenix team would help get an appropriate solution
for this. 

Thanks,
Sun.





CertusNet 

发件人: wangkun
发送时间: 2015-06-01 16:31
收件人: user
主题: Re: Phoenix 4.4 do not work with CDH 5.4
Hi Yuhao

Thank you for your suggestion about cloudera-labs-phoenix. It will be useful for me in the
future, but now I found it still not support phoenix 4.4 which can work with spark. And I
want to do some test on phoenix spark integration.

Do you combine phoenix 4.4 with CDH 5.4 successfully? I will be appreciated if you can share
your experience on that.  


Thanks
Kevin Wang

在 2015年5月29日,下午4:05,Yuhao Bi <byh0831@gmail.com> 写道:

Hi there,

I have similar experience to you.
I tried to combine Phoenix 4.4 RC with my CDH5.4.0 cluster, you have to modify a few source
code.
But, I suggest you to use cloudera-labs-phoenix which is compatible and easy to deploy for
CDH.

http://blog.cloudera.com/blog/2015/05/apache-phoenix-joins-cloudera-labs/

source code is here: https://github.com/cloudera-labs/phoenix



2015-05-29 12:08 GMT+08:00 wangkun <kevinwang@stonesun-tech.com>:
Hi, All

I am using CDH5.4.0 (which using HBase-1.0.0) with phoenix-4.4.0-HBase-1.0. I copied the phoenix-4.4.0-HBase-1.0-server.jar
to HBase lib directory and restart HBase successfully. 

I run the sqlline.py to access it and got the following exception.

[yimr@yi07 bin]$ ./sqlline.py localhost
Setting property: [isolation, TRANSACTION_READ_COMMITTED]
issuing: !connect jdbc:phoenix:localhost none none org.apache.phoenix.jdbc.PhoenixDriver
Connecting to jdbc:phoenix:localhost
15/05/29 11:00:32 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your
platform... using builtin-java classes where applicable
15/05/29 11:00:32 INFO metrics.Metrics: Initializing metrics system: phoenix
15/05/29 11:00:32 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2-phoenix.properties
15/05/29 11:00:32 INFO trace.PhoenixMetricsSink: Writing tracing metrics to phoenix table
15/05/29 11:00:32 INFO trace.PhoenixMetricsSink: Phoenix tracing writer started
15/05/29 11:00:32 INFO impl.MetricsSinkAdapter: Sink tracing started
15/05/29 11:00:32 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
15/05/29 11:00:32 INFO impl.MetricsSystemImpl: phoenix metrics system started
15/05/29 11:00:33 INFO query.ConnectionQueryServicesImpl: Found quorum: localhost:2181
15/05/29 11:00:33 INFO client.ConnectionManager$HConnectionImplementation: Closing master
protocol: MasterService
15/05/29 11:00:33 INFO client.ConnectionManager$HConnectionImplementation: Closing zookeeper
sessionid=0x14d9a9d103c3d4f
15/05/29 11:00:33 WARN ipc.CoprocessorRpcChannel: Call failed on IOException
org.apache.hadoop.hbase.DoNotRetryIOException: org.apache.hadoop.hbase.DoNotRetryIOException:
SYSTEM.CATALOG: org.apache.hadoop.hbase.client.Scan.setRaw(Z)Lorg/apache/hadoop/hbase/client/Scan;
        at org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:84)
        at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.createTable(MetaDataEndpointImpl.java:1148)
        at org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:10515)
        at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7054)
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1741)
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1723)
        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31447)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
        at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Scan.setRaw(Z)Lorg/apache/hadoop/hbase/client/Scan;
        at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.buildDeletedTable(MetaDataEndpointImpl.java:925)
        at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.loadTable(MetaDataEndpointImpl.java:1001)
        at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.createTable(MetaDataEndpointImpl.java:1097)
        ... 10 more

        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
        at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
        at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:313)
        at org.apache.hadoop.hbase.protobuf.ProtobufUtil.execService(ProtobufUtil.java:1609)
        at org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:92)
        at org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:89)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)
        at org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:95)
        at org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel.callMethod(CoprocessorRpcChannel.java:56)
        at org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService$Stub.createTable(MetaDataProtos.java:10695)
        at org.apache.phoenix.query.ConnectionQueryServicesImpl$6.call(ConnectionQueryServicesImpl.java:1261)
        at org.apache.phoenix.query.ConnectionQueryServicesImpl$6.call(ConnectionQueryServicesImpl.java:1250)
        at org.apache.hadoop.hbase.client.HTable$16.call(HTable.java:1737)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.DoNotRetryIOException):
org.apache.hadoop.hbase.DoNotRetryIOException: SYSTEM.CATALOG: org.apache.hadoop.hbase.client.Scan.setRaw(Z)Lorg/apache/hadoop/hbase/client/Scan;
        at org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:84)
        at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.createTable(MetaDataEndpointImpl.java:1148)
        at org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:10515)
        at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7054)
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1741)
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1723)
        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31447)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
        at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Scan.setRaw(Z)Lorg/apache/hadoop/hbase/client/Scan;
        at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.buildDeletedTable(MetaDataEndpointImpl.java:925)
        at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.loadTable(MetaDataEndpointImpl.java:1001)
        at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.createTable(MetaDataEndpointImpl.java:1097)
        ... 10 more

        at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1199)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:216)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:300)
        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.execService(ClientProtos.java:31913)
        at org.apache.hadoop.hbase.protobuf.ProtobufUtil.execService(ProtobufUtil.java:1605)
        ... 13 more
15/05/29 11:00:33 WARN client.HTable: Error calling coprocessor service org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService
for row \x00SYSTEM\x00CATALOG
java.util.concurrent.ExecutionException: org.apache.hadoop.hbase.DoNotRetryIOException: org.apache.hadoop.hbase.DoNotRetryIOException:
SYSTEM.CATALOG: org.apache.hadoop.hbase.client.Scan.setRaw(Z)Lorg/apache/hadoop/hbase/client/Scan;
        at org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:84)
        at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.createTable(MetaDataEndpointImpl.java:1148)
        at org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:10515)
        at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7054)
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1741)
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1723)
        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31447)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
        at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Scan.setRaw(Z)Lorg/apache/hadoop/hbase/client/Scan;
        at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.buildDeletedTable(MetaDataEndpointImpl.java:925)
        at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.loadTable(MetaDataEndpointImpl.java:1001)
        at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.createTable(MetaDataEndpointImpl.java:1097)
        ... 10 more

        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:188)
        at org.apache.hadoop.hbase.client.HTable.coprocessorService(HTable.java:1749)
        at org.apache.hadoop.hbase.client.HTable.coprocessorService(HTable.java:1705)
        at org.apache.phoenix.query.ConnectionQueryServicesImpl.metaDataCoprocessorExec(ConnectionQueryServicesImpl.java:1024)
        at org.apache.phoenix.query.ConnectionQueryServicesImpl.metaDataCoprocessorExec(ConnectionQueryServicesImpl.java:1004)
        at org.apache.phoenix.query.ConnectionQueryServicesImpl.createTable(ConnectionQueryServicesImpl.java:1249)
        at org.apache.phoenix.query.DelegateConnectionQueryServices.createTable(DelegateConnectionQueryServices.java:112)
        at org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:1902)
        at org.apache.phoenix.schema.MetaDataClient.createTable(MetaDataClient.java:744)
        at org.apache.phoenix.compile.CreateTableCompiler$2.execute(CreateTableCompiler.java:186)
        at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:303)
        at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:295)
        at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
        at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:293)
        at org.apache.phoenix.jdbc.PhoenixStatement.executeUpdate(PhoenixStatement.java:1236)
        at org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:1891)
        at org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:1860)
        at org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:77)
        at org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:1860)
        at org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:162)
        at org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.connect(PhoenixEmbeddedDriver.java:131)
        at org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:133)
        at sqlline.DatabaseConnection.connect(DatabaseConnection.java:157)
        at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:203)
        at sqlline.Commands.connect(Commands.java:1064)
        at sqlline.Commands.connect(Commands.java:996)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
        at sqlline.SqlLine.dispatch(SqlLine.java:804)
        at sqlline.SqlLine.initArgs(SqlLine.java:588)
        at sqlline.SqlLine.begin(SqlLine.java:656)
        at sqlline.SqlLine.start(SqlLine.java:398)
        at sqlline.SqlLine.main(SqlLine.java:292)


Any suggestion in helping resolve the problem is greatly appreciated.


Thanks
Kevin Wang



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message