impala-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (IMPALA-5554) crash in impala::Sorter::Run::ConvertOffsetsToPtrs
Date Tue, 27 Jun 2017 16:02:00 GMT

     [ https://issues.apache.org/jira/browse/IMPALA-5554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tim Armstrong resolved IMPALA-5554.
-----------------------------------
    Resolution: Fixed


IMPALA-5554: sorter DCHECK on null column

The bug was in the DCHECK. The DCHECK is intended to make sure that a
tuple's string data didn't get split across blocks. The logic assumed
that if the second-or-later string column was in the next-block, that
the strings were split between blocks. However, that assumption is
invalid if there are NULL strings, which do not belong in any block.

The fix for the DCHECK (which is still useful) is to count the number
of non-NULL strings and make sure that no non-NULL strings were split
between blocks.

Testing:
Added a test that reproduces the crash.

Change-Id: I7a8dee982501008efff5b5abc192cfb5e6544a90
Reviewed-on: http://gerrit.cloudera.org:8080/7295
Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com>
Tested-by: Impala Public Jenkins

> crash in impala::Sorter::Run::ConvertOffsetsToPtrs
> --------------------------------------------------
>
>                 Key: IMPALA-5554
>                 URL: https://issues.apache.org/jira/browse/IMPALA-5554
>             Project: IMPALA
>          Issue Type: Sub-task
>          Components: Distributed Exec
>    Affects Versions: Impala 2.9.0
>            Reporter: Matthew Mulder
>            Assignee: Tim Armstrong
>            Priority: Critical
>             Fix For: Impala 2.10.0
>
>
> While running concurrent TPC-DS queries at 1TB scale I caught a few of these crashes.
> https://github.com/HenryR/Impala/tree/krpc change a53d43ad7bd10587e3d19295c937125cbf244765
> {code}Program terminated with signal 6, Aborted.
> #0  0x0000003eab832625 in raise () from /lib64/libc.so.6
> Missing separate debuginfos, use: debuginfo-install cyrus-sasl-gssapi-2.1.23-15.el6_6.2.x86_64
cyrus-sasl-lib-2.1.23-15.el6_6.2.x86_64 cyrus-sasl-plain-2.1.23-15.el6_6.2.x86_64 db4-4.7.25-20.el6_7.x86_64
glibc-2.12-1.166.el6_7.7.x86_64 jdk1.8.0_121-1.8.0_121-fcs.x86_64 keyutils-libs-1.4-5.el6.x86_64
krb5-libs-1.10.3-42z1.el6_7.x86_64 libcom_err-1.41.12-22.el6.x86_64 libselinux-2.0.94-5.8.el6.x86_64
nss-softokn-freebl-3.14.3-23.el6_7.x86_64 openssl-1.0.1e-42.el6_7.4.x86_64 zlib-1.2.3-29.el6.x86_64
> (gdb) bt
> #0  0x0000003eab832625 in raise () from /lib64/libc.so.6
> #1  0x0000003eab833e05 in abort () from /lib64/libc.so.6
> #2  0x0000000002e476e4 in google::DumpStackTraceAndExit() ()
> #3  0x0000000002e3e15d in google::LogMessage::Fail() ()
> #4  0x0000000002e3fa02 in google::LogMessage::SendToLog() ()
> #5  0x0000000002e3db37 in google::LogMessage::Flush() ()
> #6  0x0000000002e410fe in google::LogMessageFatal::~LogMessageFatal() ()
> #7  0x0000000001fdc034 in impala::Sorter::Run::ConvertOffsetsToPtrs (this=0x7f9c51723290,
tuple=0x7f815a66947d) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/sorter.cc:1028
> #8  0x0000000001fe6002 in impala::Sorter::Run::GetNext<true> (this=0x7f9c51723290,
output_batch=0x7f965ed49180, eos=0x7f93391ff7bf)
>     at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/sorter.cc:868
> #9  0x0000000001fda9be in impala::Sorter::Run::GetNextBatch (this=0x7f9c51723290, output_batch=0x7f996a59ad20)
at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/sorter.cc:797
> #10 0x0000000001febfca in boost::_mfi::mf1<impala::Status, impala::Sorter::Run, impala::RowBatch**>::operator()
(this=0x7f996a59ad08, p=0x7f9c51723290, a1=0x7f996a59ad20)
>     at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/boost-1.57.0-p2/include/boost/bind/mem_fn_template.hpp:165
> #11 0x0000000001febc98 in boost::_bi::list2<boost::_bi::value<impala::Sorter::Run*>,
boost::arg<1> >::operator()<impala::Status, boost::_mfi::mf1<impala::Status,
impala::Sorter::Run, impala::RowBatch**>, boost::_bi::list1<impala::RowBatch**&>
> (this=0x7f996a59ad18, f=..., a=...) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/boost-1.57.0-p2/include/boost/bind/bind.hpp:303
> #12 0x0000000001feb62f in boost::_bi::bind_t<impala::Status, boost::_mfi::mf1<impala::Status,
impala::Sorter::Run, impala::RowBatch**>, boost::_bi::list2<boost::_bi::value<impala::Sorter::Run*>,
boost::arg<1> > >::operator()<impala::RowBatch**> (this=0x7f996a59ad08,
a1=@0x7f93391ff968) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/boost-1.57.0-p2/include/boost/bind/bind_template.hpp:32
> #13 0x0000000001feaf02 in boost::detail::function::function_obj_invoker1<boost::_bi::bind_t<impala::Status,
boost::_mfi::mf1<impala::Status, impala::Sorter::Run, impala::RowBatch**>, boost::_bi::list2<boost::_bi::value<impala::Sorter::Run*>,
boost::arg<1> > >, impala::Status, impala::RowBatch**>::invoke (function_obj_ptr=...,
a0=0x7f996a59ad20)
>     at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/boost-1.57.0-p2/include/boost/function/function_template.hpp:132
> #14 0x00000000018d7f40 in boost::function1<impala::Status, impala::RowBatch**>::operator()
(this=0x7f996a59ad00, a0=0x7f996a59ad20)
>     at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/boost-1.57.0-p2/include/boost/function/function_template.hpp:767
> #15 0x00000000018d7b0a in impala::SortedRunMerger::SortedRunWrapper::Advance (this=0x7f996a59ad00,
transfer_batch=0x0, eos=0x7f93391ffb0f)
>     at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/sorted-run-merger.cc:76
> #16 0x00000000018d7453 in impala::SortedRunMerger::AdvanceMinRow (this=0x6a3564c0, transfer_batch=0x7f92f806da80)
>     at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/sorted-run-merger.cc:187
> #17 0x00000000018d7091 in impala::SortedRunMerger::GetNext (this=0x6a3564c0, output_batch=0x7f92f806da80,
eos=0x236d09f0)
>     at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/sorted-run-merger.cc:173
> #18 0x0000000001fdfd4c in impala::Sorter::GetNext (this=0x7f9c2abe8f20, output_batch=0x7f92f806da80,
eos=0x236d09f0) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/sorter.cc:1445
> #19 0x0000000001d6e53c in impala::SortNode::GetNext (this=0x7f9bb5ac7de0, state=0x7f987f361000,
row_batch=0x7f92f806da80, eos=0x236d09f0)
>     at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/exec/sort-node.cc:103
> #20 0x0000000001d8f414 in impala::AnalyticEvalNode::ProcessChildBatches (this=0x236d0700,
state=0x7f987f361000) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/exec/analytic-eval-node.cc:610
> #21 0x0000000001d91ffb in impala::AnalyticEvalNode::GetNext (this=0x236d0700, state=0x7f987f361000,
row_batch=0x7f95a3c629a0, eos=0x2922a11c)
>     at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/exec/analytic-eval-node.cc:792
> #22 0x0000000001d6b507 in impala::SelectNode::GetNext (this=0x2922a000, state=0x7f987f361000,
row_batch=0x7f9339200370, eos=0x7f93392004af)
>     at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/exec/select-node.cc:76
> #23 0x0000000001d7421b in impala::TopNNode::Open (this=0x7f9914466000, state=0x7f987f361000)
at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/exec/topn-node.cc:153
> #24 0x00000000018f6951 in impala::FragmentInstanceState::Open (this=0x7f97963c4280) at
/data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/fragment-instance-state.cc:265
> #25 0x00000000018f4413 in impala::FragmentInstanceState::Exec (this=0x7f97963c4280) at
/data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/fragment-instance-state.cc:88
> #26 0x00000000018b8f10 in impala::QueryState::ExecFInstance (this=0x1f70c800, fis=0x7f97963c4280)
at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/query-state.cc:333
> #27 0x00000000018b7c12 in impala::QueryState::<lambda()>::operator()(void) const
(__closure=0x7f9339200bc8) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/query-state.cc:307
> #28 0x00000000018b98ed in boost::detail::function::void_function_obj_invoker0<impala::QueryState::StartFInstances()::<lambda()>,
void>::invoke(boost::detail::function::function_buffer &) (function_obj_ptr=...)
>     at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/boost-1.57.0-p2/include/boost/function/function_template.hpp:153
> #29 0x00000000016bf878 in boost::function0<void>::operator() (this=0x7f9339200bc0)
>     at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/boost-1.57.0-p2/include/boost/function/function_template.hpp:767
> #30 0x0000000001b282bf in impala::Thread::SuperviseThread (name=..., category=..., functor=...,
thread_started=0x7f9340c0bc20)
>     at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/util/thread.cc:322
> #31 0x0000000001b30c2e in boost::_bi::list4<boost::_bi::value<std::basic_string<char,
std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<std::basic_string<char,
std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<boost::function<void()>
>, boost::_bi::value<impala::Promise<long int>*> >::operator()<void (*)(const
std::basic_string<char>&, const std::basic_string<char>&, boost::function<void()>,
impala::Promise<long int>*), boost::_bi::list0>(boost::_bi::type<void>, void
(*&)(const std::basic_string<char, std::char_traits<char>, std::allocator<char>
> &, const std::basic_string<char, std::char_traits<char>, std::allocator<char>
> &, boost::function<void()>, impala::Promise<long> *), boost::_bi::list0
&, int) (this=0x7f9c5e90e3c0, f=@0x7f9c5e90e3b8, a=...)
>     at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/boost-1.57.0-p2/include/boost/bind/bind.hpp:457
> #32 0x0000000001b30b71 in boost::_bi::bind_t<void, void (*)(const std::basic_string<char,
std::char_traits<char>, std::allocator<char> >&, const std::basic_string<char,
std::char_traits<char>, std::allocator<char> >&, boost::function<void()>,
impala::Promise<long int>*), boost::_bi::list4<boost::_bi::value<std::basic_string<char,
std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<std::basic_string<char,
std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<boost::function<void()>
>, boost::_bi::value<impala::Promise<long int>*> > >::operator()(void)
(this=0x7f9c5e90e3b8)
>     at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/boost-1.57.0-p2/include/boost/bind/bind_template.hpp:20
> #33 0x0000000001b30b34 in boost::detail::thread_data<boost::_bi::bind_t<void, void
(*)(const std::basic_string<char, std::char_traits<char>, std::allocator<char>
>&, const std::basic_string<char, std::char_traits<char>, std::allocator<char>
>&, boost::function<void()>, impala::Promise<long int>*), boost::_bi::list4<boost::_bi::value<std::basic_string<char,
std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<std::basic_string<char,
std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<boost::function<void()>
>, boost::_bi::value<impala::Promise<long int>*> > > >::run(void)
(this=0x7f9c5e90e200)
>     at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/boost-1.57.0-p2/include/boost/thread/detail/thread.hpp:116
> #34 0x000000000204865a in thread_proxy ()
> #35 0x0000003eabc07aa1 in start_thread () from /lib64/libpthread.so.0
> #36 0x0000003eab8e893d in clone () from /lib64/libc.so.6{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message