kudu-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "helifu" <hzhel...@corp.netease.com>
Subject 答复: Issue in data loading in Impala + Kudu
Date Mon, 07 May 2018 06:00:54 GMT
Hi Geetika,

It would be better to ask this question in impala user mail list. Here is the impala community:
https://impala.apache.org/community.html

 

何李夫

2018-05-07 13:56:02

 

发件人: user-return-1353-hzhelifu=corp.netease.com@kudu.apache.org <user-return-1353-hzhelifu=corp.netease.com@kudu.apache.org>
代表 Geetika Gupta
发送时间: 2018年5月7日 13:42
收件人: user@kudu.apache.org
主题: Issue in data loading in Impala + Kudu

 

Hi community,

 

I was trying to load 500GB of TPCH data into kudu table using the following query:

 

insert into lineitem select * from PARQUETIMPALA500.LINEITEM


While executing the query for around 17 hrs it got cancelled as the impalad process of that
machine got aborted. Here are the logs of the impalad process.

 

impalad.ERROR

 

Log file created at: 2018/05/06 13:40:34

Running on machine: slave2

Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg

E0506 13:40:34.097759 28730 logging.cc:121] stderr will be logged to this file.

SLF4J: Class path contains multiple SLF4J bindings.

SLF4J: Found binding in [jar:file:/root/softwares/impala/fe/target/dependency/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in [jar:file:/root/softwares/impala/testdata/target/dependency/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.

SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]

18/05/06 13:40:34 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your
platform... using builtin-java classes where applicable

18/05/06 13:40:36 WARN shortcircuit.DomainSocketFactory: The short-circuit local reads feature
cannot be used because libhadoop cannot be loaded.

tcmalloc: large alloc 1073741824 bytes == 0x484434000 @  0x4135176 0x7fd9e9fc3929

tcmalloc: large alloc 2147483648 bytes == 0x7fd540f18000 @  0x4135176 0x7fd9e9fc3929

F0507 09:46:12.673912 29258 error-util.cc:148] Check failed: log_entry.count > 0 (-1831809966
vs. 0) 

*** Check failure stack trace: ***

    @          0x3fc0c0d  google::LogMessage::Fail()

    @          0x3fc24b2  google::LogMessage::SendToLog()

    @          0x3fc05e7  google::LogMessage::Flush()

    @          0x3fc3bae  google::LogMessageFatal::~LogMessageFatal()

    @          0x1bbcb31  impala::PrintErrorMap()

    @          0x1bbcd07  impala::PrintErrorMapToString()

    @          0x2decbd7  impala::Coordinator::GetErrorLog()

    @          0x1a8d634  impala::ImpalaServer::UnregisterQuery()

    @          0x1b29264  impala::ImpalaServer::CloseOperation()

    @          0x2c5ce86  apache::hive::service::cli::thrift::TCLIServiceProcessor::process_CloseOperation()

    @          0x2c56b8c  apache::hive::service::cli::thrift::TCLIServiceProcessor::dispatchCall()

    @          0x2c2fcb1  impala::ImpalaHiveServer2ServiceProcessor::dispatchCall()

    @          0x16fdb20  apache::thrift::TDispatchProcessor::process()

    @          0x18ea6b3  apache::thrift::server::TAcceptQueueServer::Task::run()

    @          0x18e2181  impala::ThriftThread::RunRunnable()

    @          0x18e3885  boost::_mfi::mf2<>::operator()()

    @          0x18e371b  boost::_bi::list3<>::operator()<>()

    @          0x18e3467  boost::_bi::bind_t<>::operator()()

    @          0x18e337a  boost::detail::function::void_function_obj_invoker0<>::invoke()

    @          0x192761c  boost::function0<>::operator()()

    @          0x1c3ebf7  impala::Thread::SuperviseThread()

    @          0x1c470cd  boost::_bi::list5<>::operator()<>()

    @          0x1c46ff1  boost::_bi::bind_t<>::operator()()

    @          0x1c46fb4  boost::detail::thread_data<>::run()

    @          0x2eedb4a  thread_proxy

    @     0x7fda1dbb16ba  start_thread

    @     0x7fda1d8e741d  clone

Wrote minidump to /tmp/minidumps/impalad/a9113d9b-bc3d-488a-1feebf9b-47b42022.dmp

 

impalad.FATAL

 

Log file created at: 2018/05/07 09:46:12

Running on machine: slave2

Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg

F0507 09:46:12.673912 29258 error-util.cc:148] Check failed: log_entry.count > 0 (-1831809966
vs. 0) 

 

Impalad.INFO

edentials={real_user=root}} blocked reactor thread for 34288.6us

I0507 09:38:14.943245 29882 outbound_call.cc:288] RPC callback for RPC call kudu.tserver.TabletServerService.Write
-> {remote=136.243.74.42:7050 <http://136.243.74.42:7050>  (slave5), user_credentials={real_user=root}}
blocked reactor thread for 35859.8us

I0507 09:38:15.942150 29882 outbound_call.cc:288] RPC callback for RPC call kudu.tserver.TabletServerService.Write
-> {remote=136.243.74.42:7050 <http://136.243.74.42:7050>  (slave5), user_credentials={real_user=root}}
blocked reactor thread for 40664.9us

I0507 09:38:17.495046 29882 outbound_call.cc:288] RPC callback for RPC call kudu.tserver.TabletServerService.Write
-> {remote=136.243.74.42:7050 <http://136.243.74.42:7050>  (slave5), user_credentials={real_user=root}}
blocked reactor thread for 49514.6us

I0507 09:46:12.664149  4507 coordinator.cc:783] Release admission control resources for query_id=3e4a4c646800e1d9:c859bb7f00000000

F0507 09:46:12.673912 29258 error-util.cc:148] Check failed: log_entry.count > 0 (-1831809966
vs. 0) 

Wrote minidump to /tmp/minidumps/impalad/a9113d9b-bc3d-488a-1feebf9b-47b42022.dmp

 

Note: 

We are executing the queries on 8 node cluster with the following configuration

Cluster : 8 Node Cluster (48 GB RAM , 8 CPU Core and 2 TB hard-disk each,
Intel(R) Core(TM) i7 CPU 950 @ 3.07GHz

 

 

-- 

Regards,

Geetika Gupta


Mime
View raw message