impala-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joe McDonnell (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (IMPALA-5172) crash in tcmalloc::CentralFreeList::FetchFromOneSpans
Date Mon, 17 Apr 2017 21:26:41 GMT

     [ https://issues.apache.org/jira/browse/IMPALA-5172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Joe McDonnell resolved IMPALA-5172.
-----------------------------------
       Resolution: Fixed
    Fix Version/s: Impala 2.9.0

commit fcefe47d7348f01e6a6fef700b421290a3f536b3
Author: Joe McDonnell <joemcdonnell@cloudera.com>
Date:   Wed Apr 12 16:46:15 2017 -0700

    IMPALA-5172: Buffer overrun for Snappy decompression
    
    When using a preallocated buffer for decompression, a
    file corruption can lead to the expected decompressed size
    being smaller than the actual decompressed size. Since
    we use this for allocating the output buffer,
    decompression needs to be able to handle a buffer that
    is too small.
    
    Snappy does not properly handle a buffer that is too small
    and will overrun the buffer. This changes the code to
    check the decompressed length and return an error if
    the buffer is not large enough. It also adds a test to
    verify that this behavior is respected for other
    compression algorithms.
    
    Change-Id: I45b75f61e8c0ae85f9add5b13ac2b161a803d2ba
    Reviewed-on: http://gerrit.cloudera.org:8080/6625
    Reviewed-by: Dan Hecht <dhecht@cloudera.com>
    Tested-by: Dan Hecht <dhecht@cloudera.com>


> crash in tcmalloc::CentralFreeList::FetchFromOneSpans
> -----------------------------------------------------
>
>                 Key: IMPALA-5172
>                 URL: https://issues.apache.org/jira/browse/IMPALA-5172
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Backend
>    Affects Versions: Impala 2.8.0
>            Reporter: Matthew Mulder
>            Assignee: Joe McDonnell
>            Priority: Blocker
>              Labels: broken-build
>             Fix For: Impala 2.9.0
>
>         Attachments: IMPALA-5172.patch, impalad_node1.ERROR, impalad_node2.ERROR
>
>
> This was encountered during an automated test run.{code}#0  0x0000003556e328e5 in raise
() from /lib64/libc.so.6
> #1  0x0000003556e340c5 in abort () from /lib64/libc.so.6
> #2  0x00007fea49b64c55 in os::abort(bool) () from /opt/toolchain/sun-jdk-64bit-1.7.0.75/jre/lib/amd64/server/libjvm.so
> #3  0x00007fea49ce6cd7 in VMError::report_and_die() () from /opt/toolchain/sun-jdk-64bit-1.7.0.75/jre/lib/amd64/server/libjvm.so
> #4  0x00007fea49b69b6f in JVM_handle_linux_signal () from /opt/toolchain/sun-jdk-64bit-1.7.0.75/jre/lib/amd64/server/libjvm.so
> #5  <signal handler called>
> #6  0x0000000001bf2253 in tcmalloc::CentralFreeList::FetchFromOneSpans(int, void**, void**)
()
> #7  0x0000000001bf254c in tcmalloc::CentralFreeList::FetchFromOneSpansSafe(int, void**,
void**) ()
> #8  0x0000000001bf25f4 in tcmalloc::CentralFreeList::RemoveRange(void**, void**, int)
()
> #9  0x0000000001bffcc3 in tcmalloc::ThreadCache::FetchFromCentralCache(unsigned long,
unsigned long) ()
> #10 0x0000000001c0eea8 in tc_newarray ()
> #11 0x0000000000bcbd24 in allocate (this=0x7fe9ee3f6d78) at /data/jenkins/workspace/impala-umbrella-build-and-test/Impala-Toolchain/gcc-4.9.2/include/c++/4.9.2/ext/new_allocator.h:104
> #12 allocate (this=0x7fe9ee3f6d78) at /data/jenkins/workspace/impala-umbrella-build-and-test/Impala-Toolchain/gcc-4.9.2/include/c++/4.9.2/bits/alloc_traits.h:357
> #13 _M_allocate (this=0x7fe9ee3f6d78) at /data/jenkins/workspace/impala-umbrella-build-and-test/Impala-Toolchain/gcc-4.9.2/include/c++/4.9.2/bits/stl_vector.h:170
> #14 std::vector<impala::TRuntimeProfileNode, std::allocator<impala::TRuntimeProfileNode>
>::_M_emplace_back_aux<impala::TRuntimeProfileNode> (this=0x7fe9ee3f6d78) at /data/jenkins/workspace/impala-umbrella-build-and-test/Impala-Toolchain/gcc-4.9.2/include/c++/4.9.2/bits/vector.tcc:412
> #15 0x0000000000bc5747 in emplace_back<impala::TRuntimeProfileNode> (this=0x93c3400,
nodes=0x7fe9ee3f6d78) at /data/jenkins/workspace/impala-umbrella-build-and-test/Impala-Toolchain/gcc-4.9.2/include/c++/4.9.2/bits/vector.tcc:101
> #16 push_back (this=0x93c3400, nodes=0x7fe9ee3f6d78) at /data/jenkins/workspace/impala-umbrella-build-and-test/Impala-Toolchain/gcc-4.9.2/include/c++/4.9.2/bits/stl_vector.h:932
> #17 impala::RuntimeProfile::ToThrift (this=0x93c3400, nodes=0x7fe9ee3f6d78) at /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/src/util/runtime-profile.cc:758
> #18 0x0000000000bc54b2 in impala::RuntimeProfile::ToThrift (this=0x29480a00, nodes=0x7fe9ee3f6d78)
at /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/src/util/runtime-profile.cc:836
> #19 0x0000000000bc54b2 in impala::RuntimeProfile::ToThrift (this=0x1ebf6300, nodes=0x7fe9ee3f6d78)
at /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/src/util/runtime-profile.cc:836
> #20 0x0000000000bc54b2 in impala::RuntimeProfile::ToThrift (this=0x78d1700, nodes=0x7fe9ee3f6d78)
at /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/src/util/runtime-profile.cc:836
> #21 0x0000000000bc54b2 in impala::RuntimeProfile::ToThrift (this=0x598f700, nodes=0x7fe9ee3f6d78)
at /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/src/util/runtime-profile.cc:836
> #22 0x0000000000bc54b2 in impala::RuntimeProfile::ToThrift (this=0xaa04f00, nodes=0x7fe9ee3f6d78)
at /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/src/util/runtime-profile.cc:836
> #23 0x0000000000bc54b2 in impala::RuntimeProfile::ToThrift (this=0x1fdd0460, nodes=0x7fe9ee3f6d78)
at /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/src/util/runtime-profile.cc:836
> #24 0x0000000000bc5c1d in impala::RuntimeProfile::SerializeToArchiveString (this=) at
/data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/src/util/runtime-profile.cc:724
> #25 0x0000000000bc632f in impala::RuntimeProfile::SerializeToArchiveString (this=0x1fdd0460)
at /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/src/util/runtime-profile.cc:718
> #26 0x0000000000abb6c4 in impala::ImpalaServer::ArchiveQuery (this=0x7bd9c00, query=...)
at /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/src/service/impala-server.cc:685
> #27 0x0000000000abc6e6 in impala::ImpalaServer::UnregisterQuery (this=0x7bd9c00, query_id=)
at /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/src/service/impala-server.cc:986
> #28 0x0000000000af6a42 in impala::ImpalaServer::close (this=0x7bd9c00, handle=) at /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/src/service/impala-beeswax-server.cc:236
> #29 0x0000000000d523f5 in beeswax::BeeswaxServiceProcessor::process_close (this=0x7279ae0,
seqid=0, iprot=) at /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/generated-sources/gen-cpp/BeeswaxService.cpp:3543
> #30 0x0000000000d59c54 in beeswax::BeeswaxServiceProcessor::dispatchCall (this=) at /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/generated-sources/gen-cpp/BeeswaxService.cpp:2952
> #31 0x000000000080f24c in apache::thrift::TDispatchProcessor::process (this=0x7279ae0,
in=..., out=..., connectionContext=0xa817d80) at /data/jenkins/workspace/impala-umbrella-build-and-test/Impala-Toolchain/thrift-0.9.0-p8/include/thrift/TDispatchProcessor.h:121
> #32 0x0000000001b4291b in apache::thrift::server::TThreadPoolServer::Task::run() ()
> #33 0x0000000001b2a4e9 in apache::thrift::concurrency::ThreadManager::Worker::run() ()
> #34 0x00000000009fdbe9 in impala::ThriftThread::RunRunnable (this=) at /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/src/rpc/thrift-thread.cc:64
> #35 0x00000000009fe642 in operator() (function_obj_ptr=) at /data/jenkins/workspace/impala-umbrella-build-and-test/Impala-Toolchain/boost-1.57.0-p1/include/boost/bind/mem_fn_template.hpp:280
> #36 operator()<boost::_mfi::mf2<void, impala::ThriftThread, boost::shared_ptr<apache::thrift::concurrency::Runnable>,
impala::Promise<long unsigned int>*>, boost::_bi::list0> (function_obj_ptr=) at
/data/jenkins/workspace/impala-umbrella-build-and-test/Impala-Toolchain/boost-1.57.0-p1/include/boost/bind/bind.hpp:392
> #37 operator() (function_obj_ptr=) at /data/jenkins/workspace/impala-umbrella-build-and-test/Impala-Toolchain/boost-1.57.0-p1/include/boost/bind/bind_template.hpp:20
> #38 boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void,
boost::_mfi::mf2<void, impala::ThriftThread, boost::shared_ptr<apache::thrift::concurrency::Runnable>,
impala::Promise<unsigned long>*>, boost::_bi::list3<boost::_bi::value<impala::ThriftThread*>,
boost::_bi::value<boost::shared_ptr<apache::thrift::concurrency::Runnable> >,
boost::_bi::value<impala::Promise<unsigned long>*> > >, void>::invoke
(function_obj_ptr=) at /data/jenkins/workspace/impala-umbrella-build-and-test/Impala-Toolchain/boost-1.57.0-p1/include/boost/function/function_template.hpp:153
> #39 0x0000000000be2479 in operator() (name=) at /data/jenkins/workspace/impala-umbrella-build-and-test/Impala-Toolchain/boost-1.57.0-p1/include/boost/function/function_template.hpp:767
> #40 impala::Thread::SuperviseThread (name=) at /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/src/util/thread.cc:325
> #41 0x0000000000be2ec4 in operator()<void (*)(const std::basic_string<char>&,
const std::basic_string<char>&, boost::function<void()>, impala::Promise<long
int>*), boost::_bi::list0> (this=0x7de3c00) at /data/jenkins/workspace/impala-umbrella-build-and-test/Impala-Toolchain/boost-1.57.0-p1/include/boost/bind/bind.hpp:457
> #42 operator() (this=0x7de3c00) at /data/jenkins/workspace/impala-umbrella-build-and-test/Impala-Toolchain/boost-1.57.0-p1/include/boost/bind/bind_template.hpp:20
> #43 boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(const std::basic_string<char,
std::char_traits<char>, std::allocator<char> >&, const std::basic_string<char,
std::char_traits<char>, std::allocator<char> >&, boost::function<void()>,
impala::Promise<long int>*), boost::_bi::list4<boost::_bi::value<std::basic_string<char,
std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<std::basic_string<char,
std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<boost::function<void()>
>, boost::_bi::value<impala::Promise<long int>*> > > >::run(void)
(this=0x7de3c00) at /data/jenkins/workspace/impala-umbrella-build-and-test/Impala-Toolchain/boost-1.57.0-p1/include/boost/thread/detail/thread.hpp:116
> #44 0x0000000000e4e33a in thread_proxy ()
> #45 0x0000003557207851 in start_thread () from /lib64/libpthread.so.0
> #46 0x0000003556ee894d in clone () from /lib64/libc.so.6{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message