impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Geetika Gupta <geetika.gu...@knoldus.in>
Subject Issue in data loading in Impala + Kudu
Date Mon, 07 May 2018 06:04:24 GMT
Hi community,

I was trying to load 500GB of TPCH data into kudu table using the following
query:

*insert into lineitem select * from PARQUETIMPALA500.LINEITEM*

While executing the query for around 17 hrs it got cancelled as the impalad
process of that machine got aborted. Here are the logs of the impalad
process.

*impalad.ERROR*

Log file created at: 2018/05/06 13:40:34
Running on machine: slave2
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
E0506 13:40:34.097759 28730 logging.cc:121] stderr will be logged to this
file.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/root/softwares/
impala/fe/target/dependency/slf4j-log4j12-1.7.25.jar!/org/
slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/root/softwares/impala/testdata/target/
dependency/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/
StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
18/05/06 13:40:34 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
18/05/06 13:40:36 WARN shortcircuit.DomainSocketFactory: The short-circuit
local reads feature cannot be used because libhadoop cannot be loaded.
tcmalloc: large alloc 1073741824 bytes == 0x484434000 @  0x4135176
0x7fd9e9fc3929
tcmalloc: large alloc 2147483648 bytes == 0x7fd540f18000 @  0x4135176
0x7fd9e9fc3929
F0507 09:46:12.673912 29258 error-util.cc:148] Check failed:
log_entry.count > 0 (-1831809966 vs. 0)
*** Check failure stack trace: ***
    @          0x3fc0c0d  google::LogMessage::Fail()
    @          0x3fc24b2  google::LogMessage::SendToLog()
    @          0x3fc05e7  google::LogMessage::Flush()
    @          0x3fc3bae  google::LogMessageFatal::~LogMessageFatal()
    @          0x1bbcb31  impala::PrintErrorMap()
    @          0x1bbcd07  impala::PrintErrorMapToString()
    @          0x2decbd7  impala::Coordinator::GetErrorLog()
    @          0x1a8d634  impala::ImpalaServer::UnregisterQuery()
    @          0x1b29264  impala::ImpalaServer::CloseOperation()
    @          0x2c5ce86  apache::hive::service::cli::
thrift::TCLIServiceProcessor::process_CloseOperation()
    @          0x2c56b8c  apache::hive::service::cli::
thrift::TCLIServiceProcessor::dispatchCall()
    @          0x2c2fcb1  impala::ImpalaHiveServer2ServiceProces
sor::dispatchCall()
    @          0x16fdb20  apache::thrift::TDispatchProcessor::process()
    @          0x18ea6b3  apache::thrift::server::
TAcceptQueueServer::Task::run()
    @          0x18e2181  impala::ThriftThread::RunRunnable()
    @          0x18e3885  boost::_mfi::mf2<>::operator()()
    @          0x18e371b  boost::_bi::list3<>::operator()<>()
    @          0x18e3467  boost::_bi::bind_t<>::operator()()
    @          0x18e337a  boost::detail::function::void_
function_obj_invoker0<>::invoke()
    @          0x192761c  boost::function0<>::operator()()
    @          0x1c3ebf7  impala::Thread::SuperviseThread()
    @          0x1c470cd  boost::_bi::list5<>::operator()<>()
    @          0x1c46ff1  boost::_bi::bind_t<>::operator()()
    @          0x1c46fb4  boost::detail::thread_data<>::run()
    @          0x2eedb4a  thread_proxy
    @     0x7fda1dbb16ba  start_thread
    @     0x7fda1d8e741d  clone
Wrote minidump to /tmp/minidumps/impalad/a9113d9b-bc3d-488a-1feebf9b-
47b42022.dmp

*impalad.FATAL*

Log file created at: 2018/05/07 09:46:12
Running on machine: slave2
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
F0507 09:46:12.673912 29258 error-util.cc:148] Check failed:
log_entry.count > 0 (-1831809966 vs. 0)

*Impalad.INFO*
edentials={real_user=root}} blocked reactor thread for 34288.6us
I0507 09:38:14.943245 29882 outbound_call.cc:288] RPC callback for RPC call
kudu.tserver.TabletServerService.Write -> {remote=136.243.74.42:7050 (slave5),
user_credentials={real_user=root}} blocked reactor thread for 35859.8us
I0507 09:38:15.942150 29882 outbound_call.cc:288] RPC callback for RPC call
kudu.tserver.TabletServerService.Write -> {remote=136.243.74.42:7050 (slave5),
user_credentials={real_user=root}} blocked reactor thread for 40664.9us
I0507 09:38:17.495046 29882 outbound_call.cc:288] RPC callback for RPC call
kudu.tserver.TabletServerService.Write -> {remote=136.243.74.42:7050 (slave5),
user_credentials={real_user=root}} blocked reactor thread for 49514.6us
I0507 09:46:12.664149  4507 coordinator.cc:783] Release admission control
resources for query_id=3e4a4c646800e1d9:c859bb7f00000000
F0507 09:46:12.673912 29258 error-util.cc:148] Check failed:
log_entry.count > 0 (-1831809966 vs. 0)
Wrote minidump to /tmp/minidumps/impalad/a9113d9b-bc3d-488a-1feebf9b-
47b42022.dmp

*Note*:
We are executing the queries on 8 node cluster with the following
configuration
Cluster : 8 Node Cluster (48 GB RAM , 8 CPU Core and 2 TB hard-disk each,
Intel(R) Core(TM) i7 CPU 950 @ 3.07GHz


-- 
Regards,
Geetika Gupta

Mime
View raw message