kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 郭鹏飞 <guopengfei19...@126.com>
Subject 【Bug Report】2.5.1 NoSuchMethodError: com.facebook.fb303.FacebookService
Date Tue, 13 Nov 2018 14:56:57 GMT

在build cube时,报
java.lang.NoSuchMethodError: com.facebook.fb303.FacebookService$Client.sendBaseOneway(Ljava/lang/String;Lorg/apache/thrift/TBase;)V
错误,看起来像是版本问题,请问有人遇到吗?应该怎样解决?

版本:2.5.1-cdh47


Exception in thread "main" java.lang.NoSuchMethodError: com.facebook.fb303.FacebookService$Client.sendBaseOneway(Ljava/lang/String;Lorg/apache/thrift/TBase;)V
	at com.facebook.fb303.FacebookService$Client.send_shutdown(FacebookService.java:436)
	at com.facebook.fb303.FacebookService$Client.shutdown(FacebookService.java:430)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:538)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:105)
	at com.sun.proxy.$Proxy19.close(Unknown Source)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2084)
	at com.sun.proxy.$Proxy19.close(Unknown Source)
	at org.apache.hadoop.hive.ql.metadata.Hive.close(Hive.java:357)
	at org.apache.hadoop.hive.ql.metadata.Hive.access$000(Hive.java:153)
	at org.apache.hadoop.hive.ql.metadata.Hive$1.remove(Hive.java:173)
	at org.apache.hadoop.hive.ql.metadata.Hive.closeCurrent(Hive.java:326)
	at org.apache.hadoop.hive.ql.session.SessionState.close(SessionState.java:1643)
	at org.apache.hadoop.hive.cli.CliSessionState.close(CliSessionState.java:66)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:701)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:634)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)


谢谢
—郭鹏飞



完整日志如下:


java.io.IOException: OS command error exit with return code: 1, error message: Java HotSpot(TM)
64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/lib/hive/lib/logging-assembly-0.1.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/lib/hive/lib/alluxio-1.7.0-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely
be removed in a future release
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/lib/hive/lib/logging-assembly-0.1.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/lib/hive/lib/alluxio-1.7.0-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/apache-kylin-2.5.1-bin-cdh57/spark/jars/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed
in 8.0
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/lib/hive/lib/logging-assembly-0.1.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/lib/hive/lib/alluxio-1.7.0-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]

Logging initialized using configuration in jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/jars/hive-common-1.1.0-cdh5.12.1.jar!/hive-log4j.properties
OK
Time taken: 2.905 seconds
OK
Time taken: 0.099 seconds
OK
Time taken: 0.584 seconds
OK
Time taken: 0.151 seconds
Query ID = olap_20181113192727_70d7dc6c-f4ad-4033-aadf-e10e3ece920e
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/jars/hive-exec-1.1.0-cdh5.12.1.jar!/shaded/parquet/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/jars/hive-jdbc-1.1.0-cdh5.12.1-standalone.jar!/shaded/parquet/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/jars/parquet-hadoop-bundle-1.5.0-cdh5.12.1.jar!/shaded/parquet/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/jars/parquet-pig-bundle-1.5.0-cdh5.12.1.jar!/shaded/parquet/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/jars/parquet-format-2.1.0-cdh5.12.1.jar!/shaded/parquet/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [shaded.parquet.org.slf4j.helpers.NOPLoggerFactory]
Starting Job = job_1541920145982_4712, Tracking URL = http://c3-d07-136-38-2.yidian.com:8088/proxy/application_1541920145982_4712/
Kill Command = /opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/bin/../lib/hadoop/bin/hadoop
job  -kill job_1541920145982_4712
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2018-11-13 19:27:25,312 Stage-1 map = 0%,  reduce = 0%
2018-11-13 19:27:31,921 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.73 sec
MapReduce Total cumulative CPU time: 2 seconds 730 msec
Ended Job = job_1541920145982_4712
Loading data to table default.kylin_intermediate_etl_odw_event_fe68595d_86f2_3f32_b068_51390782dd06
Table default.kylin_intermediate_etl_odw_event_fe68595d_86f2_3f32_b068_51390782dd06 stats:
[numFiles=1, numRows=0, totalSize=130, rawDataSize=0]
MapReduce Jobs Launched: 
Stage-Stage-1: Map: 1   Cumulative CPU: 2.73 sec   HDFS Read: 10336 HDFS Write: 235 SUCCESS
Total MapReduce CPU Time Spent: 2 seconds 730 msec
OK
Time taken: 17.454 seconds
WARN: The method class org.apache.commons.logging.impl.SLF4JLogFactory#release() was invoked.
WARN: Please see http://www.slf4j.org/codes.html#release for an explanation.
Exception in thread "main" java.lang.NoSuchMethodError: com.facebook.fb303.FacebookService$Client.sendBaseOneway(Ljava/lang/String;Lorg/apache/thrift/TBase;)V
	at com.facebook.fb303.FacebookService$Client.send_shutdown(FacebookService.java:436)
	at com.facebook.fb303.FacebookService$Client.shutdown(FacebookService.java:430)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:538)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:105)
	at com.sun.proxy.$Proxy19.close(Unknown Source)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2084)
	at com.sun.proxy.$Proxy19.close(Unknown Source)
	at org.apache.hadoop.hive.ql.metadata.Hive.close(Hive.java:357)
	at org.apache.hadoop.hive.ql.metadata.Hive.access$000(Hive.java:153)
	at org.apache.hadoop.hive.ql.metadata.Hive$1.remove(Hive.java:173)
	at org.apache.hadoop.hive.ql.metadata.Hive.closeCurrent(Hive.java:326)
	at org.apache.hadoop.hive.ql.session.SessionState.close(SessionState.java:1643)
	at org.apache.hadoop.hive.cli.CliSessionState.close(CliSessionState.java:66)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:701)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:634)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Nov 13, 2018 7:27:16 PM INFO: parquet.hadoop.codec.CodecConfig: Compression set to false
Nov 13, 2018 7:27:16 PM INFO: parquet.hadoop.codec.CodecConfig: Compression: UNCOMPRESSED
Nov 13, 2018 7:27:16 PM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block size to 134217728
Nov 13, 2018 7:27:16 PM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page size to 1048576
Nov 13, 2018 7:27:16 PM INFO: parquet.hadoop.ParquetOutputFormat: Parquet dictionary page
size to 1048576
Nov 13, 2018 7:27:16 PM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is on
Nov 13, 2018 7:27:16 PM INFO: parquet.hadoop.ParquetOutputFormat: Validation is off
Nov 13, 2018 7:27:16 PM INFO: parquet.hadoop.ParquetOutputFormat: Writer version is: PARQUET_1_0
Nov 13, 2018 7:27:16 PM INFO: parquet.hadoop.ParquetOutputFormat: Maximum row group padding
size is 8388608 bytes
Nov 13, 2018 7:27:17 PM INFO: parquet.hadoop.InternalParquetRecordWriter: Flushing mem columnStore
to file. allocated memory: 0
The command is: 
hive -e "USE default;

DROP TABLE IF EXISTS kylin_intermediate_etl_odw_event_fe68595d_86f2_3f32_b068_51390782dd06;
CREATE EXTERNAL TABLE IF NOT EXISTS kylin_intermediate_etl_odw_event_fe68595d_86f2_3f32_b068_51390782dd06
(
ETL_ODW_EVENT_PLATFORM string
,ETL_ODW_EVENT_APP_ID string
,ETL_ODW_EVENT_MASH_TYPE string
,ETL_ODW_EVENT_FACTOR string
,ETL_ODW_EVENT_BATCH string
,ETL_ODW_EVENT_POSITION string
,ETL_ODW_EVENT_PAGE_TYPE string
,ETL_ODW_EVENT_ACTION_METHOD string
,ETL_ODW_EVENT_ACTION_SRC string
,ETL_ODW_EVENT_P_DAY string
,ETL_ODW_EVENT_P_HOUR string
,ETL_ODW_EVENT_P_TYPE string
,ETL_ODW_EVENT_DWELL bigint
,ETL_ODW_EVENT_DAU_UID string
,ETL_ODW_EVENT_CLICK_DOC_UID string
)
STORED AS SEQUENCEFILE
LOCATION 'hdfs://yellow/data/kylin/kylin_metadata/kylin-b4b541bb-4abf-c44c-29ed-664b7655e67c/kylin_intermediate_etl_odw_event_fe68595d_86f2_3f32_b068_51390782dd06';
ALTER TABLE kylin_intermediate_etl_odw_event_fe68595d_86f2_3f32_b068_51390782dd06 SET TBLPROPERTIES('auto.purge'='true');
INSERT OVERWRITE TABLE kylin_intermediate_etl_odw_event_fe68595d_86f2_3f32_b068_51390782dd06
SELECT
ETL_ODW_EVENT.PLATFORM as ETL_ODW_EVENT_PLATFORM
,ETL_ODW_EVENT.APP_ID as ETL_ODW_EVENT_APP_ID
,ETL_ODW_EVENT.MASH_TYPE as ETL_ODW_EVENT_MASH_TYPE
,ETL_ODW_EVENT.FACTOR as ETL_ODW_EVENT_FACTOR
,ETL_ODW_EVENT.BATCH as ETL_ODW_EVENT_BATCH
,ETL_ODW_EVENT.POSITION as ETL_ODW_EVENT_POSITION
,ETL_ODW_EVENT.PAGE_TYPE as ETL_ODW_EVENT_PAGE_TYPE
,ETL_ODW_EVENT.ACTION_METHOD as ETL_ODW_EVENT_ACTION_METHOD
,ETL_ODW_EVENT.ACTION_SRC as ETL_ODW_EVENT_ACTION_SRC
,ETL_ODW_EVENT.P_DAY as ETL_ODW_EVENT_P_DAY
,ETL_ODW_EVENT.P_HOUR as ETL_ODW_EVENT_P_HOUR
,ETL_ODW_EVENT.P_TYPE as ETL_ODW_EVENT_P_TYPE
,ETL_ODW_EVENT.DWELL as ETL_ODW_EVENT_DWELL
,ETL_ODW_EVENT.DAU_UID as ETL_ODW_EVENT_DAU_UID
,ETL_ODW_EVENT.CLICK_DOC_UID as ETL_ODW_EVENT_CLICK_DOC_UID
FROM WORKING_DW.ETL_ODW_EVENT as ETL_ODW_EVENT 
WHERE 1=1 AND (ETL_ODW_EVENT.P_DAY >= '2018-10-25' AND ETL_ODW_EVENT.P_DAY < '2018-10-25')
;

" --hiveconf hive.merge.mapredfiles=false --hiveconf hive.auto.convert.join=true --hiveconf
dfs.replication=2 --hiveconf hive.exec.compress.output=true --hiveconf hive.auto.convert.join.noconditionaltask=true
--hiveconf mapreduce.job.split.metainfo.maxsize=-1 --hiveconf hive.merge.mapfiles=false --hiveconf
hive.auto.convert.join.noconditionaltask.size=100000000 --hiveconf hive.stats.autogather=true
	at org.apache.kylin.common.util.CliCommandExecutor.execute(CliCommandExecutor.java:95)
	at org.apache.kylin.source.hive.CreateFlatHiveTableStep.createFlatHiveTable(CreateFlatHiveTableStep.java:62)
	at org.apache.kylin.source.hive.CreateFlatHiveTableStep.doWork(CreateFlatHiveTableStep.java:99)
	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:164)
	at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:70)
	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:164)
	at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:113)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)



Mime
View raw message