hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carl Steinbach <c...@cloudera.com>
Subject Re: Backtrace when running build/odbc/test/HiveClientTestC (from trunk)
Date Thu, 02 Dec 2010 16:12:47 GMT
Hi Satparshi,

It looks like glibc's MALLOC_CHECK feature is broken on x86_64 systems, at
least for some versions.

Some background:
* http://www.novell.com/support/viewContent.do?externalId=3113982&sliceId=1
* http://www.pubbs.net/200911/kde/23500-deactivate-malloccheck-on-x8664.html

Setting MALLOC_CHECK_ to 0 or 1 should fix the situation.

Hope this helps.

Carl

On Wed, Dec 1, 2010 at 3:43 PM, Saptarshi Guha <
saptarshi@revolution-computing.com> wrote:

> Hello,
> This very small code also fails (with similar errors)
>
> #include "stdio.h"
> #include "stdlib.h"
> #include "assert.h"
> #include "hiveclient.h"
>
> //x.cpp
> int main(char **argv, int argc){
>
>  fprintf(stderr, "Running %s...\n", __FUNCTION__);
>  char err_buf[MAX_HIVE_ERR_MSG_LEN];
>  HiveReturn retval;
>  HiveConnection* connection = DBOpenConnection(DEFAULT_DATABASE,
> DEFAULT_HOST, atoi(DEFAULT_PORT),
>                                                atoi(DEFAULT_FRAMED),
> err_buf, sizeof(err_buf));
>  if (connection == NULL) {
>    /* If this fails, make sure that Hive server is running with the connect
> parameter arguments */
>    fprintf(stderr, "Connect failed: %s\n", err_buf);
>    fprintf(stderr, "\n\n\nMAKE SURE YOU HAVE THE STANDALONE HIVESERVER
> RUNNING!\n");
>    fprintf(stderr, "Expected Connection Parameters:\n");
>    fprintf(stderr, "HOST: %s\n", DEFAULT_HOST);
>    fprintf(stderr, "PORT: %s\n", DEFAULT_PORT);
>    fprintf(stderr, "DATABASE: %s\n\n\n", DEFAULT_DATABASE);
>    assert(connection != NULL);
>  }
>  retval = DBCloseConnection(connection, err_buf, sizeof(err_buf));
>  return 1;
> }
>
> To compile, (after making suitable modifications to -L)
>  g++ -Wall -m64 -DARCH64  -I/root/hive/service/include/thrift -I/include
> -I/root/hive/service/include/thrift/fb303 -I/root/hive/service/src/gen-cpp
> -I/root/hive/metastore/src/gen-cpp -I/root/hive/odbc/src/cpp
> -L/root/hive/build/odbc/lib -L/usr/local/lib -lhiveclient -lthrift -ofoo
> x.cpp
>
>
> Cheers
> Saptarshi/Joy
>
> From: Saptarshi Guha <saptarshi@revolution-computing.com>
> Date: December 1, 2010 1:15:10 PM PST
> To: user@hive.apache.org
> Subject: Backtrace when running  build/odbc/test/HiveClientTestC (from
> trunk)
>
>
>
>
> > Hello,
> >
> > Does anyone get a backtrace when running HiveClientTestC?
> > On an EC2 instance with Hive (0.6.0), i checked out the trunk
> (RELEASE_NOTEST.txt says 0.6.0)
> > of Hive and compiled the client.
> >
> > ant compile-cpp -Dthrift.home=/usr/local
> >
> > Copied the  odbc/testdata/ to /tmp and ran
> build/odbc/test/HiveClientTestC.
> >
> > Got the following bt(see [1]) and ran the core dump through gdb (see [2])
> >
> > Any pointers would be appreciated.
> >
> > Cheers
> >
> > Saptarshi
> >
> >
> > [1] Output of build/odbc/test/HiveClientTestC
> >
> > Starting Hive Client C tests...
> >
> > Running basic_connect_disconnect_test...
> > LINE 166 0x631740
> > *** glibc detected *** ../build/odbc/test/HiveClientTestC: double free or
> corruption (!prev): 0x0000000000610ad0 ***
> > ======= Backtrace: =========
> > /lib64/libc.so.6[0x2aaaab33e832]
> > /lib64/libc.so.6(cfree+0x8c)[0x2aaaab341f2c]
> >
> /usr/local/lib/libthrift.so.0(_ZN6apache6thrift9transport7TSocketD0Ev+0x88)[0x2aaaab094ca8]
> >
> /root/hive/build/odbc/lib/libhiveclient.so(_ZN5boost14checked_deleteIN6apache6thrift9transport7TSocketEEEvPT_+0x27)[0x2aaaaadda901]
> >
> /root/hive/build/odbc/lib/libhiveclient.so(_ZN5boost6detail17sp_counted_impl_pIN6apache6thrift9transport7TSocketEE7disposeEv+0x19)[0x2aaaaaddaaab]
> >
> /usr/local/lib/libthrift.so.0(_ZN6apache6thrift9transport18TBufferedTransportD0Ev+0x8e)[0x2aaaab09fcde]
> >
> /root/hive/build/odbc/lib/libhiveclient.so(_ZN5boost14checked_deleteIN6apache6thrift9transport18TBufferedTransportEEEvPT_+0x27)[0x2aaaaadda7d3]
> >
> /root/hive/build/odbc/lib/libhiveclient.so(_ZN5boost6detail17sp_counted_impl_pIN6apache6thrift9transport18TBufferedTransportEE7disposeEv+0x19)[0x2aaaaaddab0b]
> >
> /usr/local/lib/libthrift.so.0(_ZN6apache6thrift8protocol15TBinaryProtocolD0Ev+0x89)[0x2aaaab074fd9]
> >
> /root/hive/build/odbc/lib/libhiveclient.so(_ZN5boost14checked_deleteIN6apache6thrift8protocol15TBinaryProtocolEEEvPT_+0x27)[0x2aaaaadda76f]
> >
> /root/hive/build/odbc/lib/libhiveclient.so(_ZN5boost6detail17sp_counted_impl_pIN6apache6thrift8protocol15TBinaryProtocolEE7disposeEv+0x19)[0x2aaaaaddab3b]
> >
> /root/hive/build/odbc/lib/libhiveclient.so(_ZN5boost6detail15sp_counted_base7releaseEv+0x3c)[0x2aaaaad62070]
> >
> /root/hive/build/odbc/lib/libhiveclient.so(_ZN5boost6detail12shared_countD1Ev+0x24)[0x2aaaaad620a0]
> >
> /root/hive/build/odbc/lib/libhiveclient.so(_ZN5boost10shared_ptrIN6apache6thrift8protocol9TProtocolEED1Ev+0x19)[0x2aaaaad6218f]
> >
> /root/hive/build/odbc/lib/libhiveclient.so(_ZN8facebook5fb30321FacebookServiceClientD2Ev+0x69)[0x2aaaaadad5c5]
> >
> /root/hive/build/odbc/lib/libhiveclient.so(_ZN6Apache6Hadoop4Hive25ThriftHiveMetastoreClientD2Ev+0x7c)[0x2aaaaadd5b16]
> >
> /root/hive/build/odbc/lib/libhiveclient.so(_ZN6Apache6Hadoop4Hive16ThriftHiveClientD0Ev+0x8f)[0x2aaaaadd5d9d]
> >
> /root/hive/build/odbc/lib/libhiveclient.so(_ZN5boost14checked_deleteIN6Apache6Hadoop4Hive16ThriftHiveClientEEEvPT_+0x27)[0x2aaaaadda965]
> >
> /root/hive/build/odbc/lib/libhiveclient.so(_ZN5boost6detail17sp_counted_impl_pIN6Apache6Hadoop4Hive16ThriftHiveClientEE7disposeEv+0x19)[0x2aaaaaddaa7b]
> >
> /root/hive/build/odbc/lib/libhiveclient.so(_ZN5boost6detail15sp_counted_base7releaseEv+0x3c)[0x2aaaaad62070]
> >
> /root/hive/build/odbc/lib/libhiveclient.so(_ZN5boost6detail12shared_countD1Ev+0x24)[0x2aaaaad620a0]
> >
> /root/hive/build/odbc/lib/libhiveclient.so(_ZN5boost10shared_ptrIN6Apache6Hadoop4Hive16ThriftHiveClientEED1Ev+0x19)[0x2aaaaadda43d]
> >
> /root/hive/build/odbc/lib/libhiveclient.so(_ZN14HiveConnectionD1Ev+0x2b)[0x2aaaaaddb179]
> >
> /root/hive/build/odbc/lib/libhiveclient.so(DBCloseConnection+0x151)[0x2aaaaadd9383]
> > ../build/odbc/test/HiveClientTestC[0x4012da]
> > ../build/odbc/test/HiveClientTestC[0x40de1f]
> > /lib64/libc.so.6(__libc_start_main+0xf4)[0x2aaaab2ea074]
> > ../build/odbc/test/HiveClientTestC[0x400c79]
> > ======= Memory map: ========
> > 00400000-00410000 r-xp 00000000 08:01 478303
> /root/hive/build/odbc/test/HiveClientTestC
> > 0060f000-00610000 rw-p 0000f000 08:01 478303
> /root/hive/build/odbc/test/HiveClientTestC
> > 00610000-00652000 rw-p 00610000 00:00 0
>  [heap]
> > 2aaaaaaab000-2aaaaaac6000 r-xp 00000000 08:01 278578
> /lib64/ld-2.7.so
> > 2aaaaaac6000-2aaaaaac8000 rw-p 2aaaaaac6000 00:00 0
> > 2aaaaacc5000-2aaaaacc6000 r--p 0001a000 08:01 278578
> /lib64/ld-2.7.so
> > 2aaaaacc6000-2aaaaacc7000 rw-p 0001b000 08:01 278578
> /lib64/ld-2.7.so
> > 2aaaaacc7000-2aaaaae1a000 r-xp 00000000 08:01 396665
> /root/hive/build/odbc/lib/libhiveclient.so.1.0.0
> > 2aaaaae1a000-2aaaab019000 ---p 00153000 08:01 396665
> /root/hive/build/odbc/lib/libhiveclient.so.1.0.0
> > 2aaaab019000-2aaaab027000 rw-p 00152000 08:01 396665
> /root/hive/build/odbc/lib/libhiveclient.so.1.0.0
> > 2aaaab037000-2aaaab0c7000 r-xp 00000000 08:01 213356
> Aborted (core dumped)
> >
> >
> > [2] gdb build/odbc/test/HiveClientTestC core.xxxx
> > bt
> > #0  0x00002aaaab2fcec5 in raise (sig=<value optimized out>) at
> ../nptl/sysdeps/unix/sysv/linux/raise.c:64
> > #1  0x00002aaaab2fe970 in abort () at abort.c:88
> > #2  0x00002aaaab3370db in __libc_message (do_abort=2, fmt=0x2aaaab3eef08
> "*** glibc detected *** %s: %s: 0x%s ***\n") at
> ../sysdeps/unix/sysv/linux/libc_fatal.c:170
> > #3  0x00002aaaab33e832 in _int_free (av=0x2aaaab61e980, mem=<value
> optimized out>) at malloc.c:5891
> > #4  0x00002aaaab341f2c in __libc_free (mem=0x6) at malloc.c:3626
> > #5  0x00002aaaab094ca8 in ~TSocket (this=0x610ad0) at
> src/transport/TSocket.cpp:117
> > #6  0x00002aaaaadda677 in
> boost::shared_ptr<Apache::Hadoop::Hive::ThriftHiveClient>::operator
> Apache::Hadoop::Hive::ThriftHiveClient*
> boost::shared_ptr<Apache::Hadoop::Hive::ThriftHiveClient>::* (this=0x610ad0)
> at /usr/local/include/boost/smart_ptr/detail/operator_bool.hpp:45
> > #7  0x00002aaaaadda821 in
> boost::checked_delete<apache::thrift::transport::TFramedTransport>
> (x=0x631010) at /usr/local/include/boost/checked_delete.hpp:34
> > #8  0x00002aaaab09fcde in ~TBufferedTransport (this=0x631030) at
> /usr/include/boost/detail/sp_counted_base_gcc_x86.hpp:145
> > #9  0x00002aaaaadda5af in
> apache::thrift::transport::TBufferedTransport::initPointers
> (this=0x2aaaab3e6a20) at
> /root/hive/service/include/thrift/transport/TBufferTransports.h:289
> > #10 0x00002aaaaadda881 in
> boost::shared_ptr<apache::thrift::transport::TTransport>::swap
> (this=0x6314b0, other=@0x7fffab301460)
> >    at /usr/local/include/boost/smart_ptr/shared_ptr.hpp:442
> > #11 0x00002aaaab074fd9 in
> apache::thrift::protocol::TBinaryProtocol::~TBinaryProtocol$delete ()
> >    at
> /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/bits/stl_iterator_base_funcs.h:96
> > #12 0x00002aaaaadda54b in
> apache::thrift::transport::TFramedTransport::initPointers (this=0x80) at
> /root/hive/service/include/thrift/transport/TBufferTransports.h:354
> > #13 0x00002aaaaadda8b1 in sp_counted_impl_p (this=0x631520,
> px=0x7fffab3015f0) at
> /usr/local/include/boost/smart_ptr/detail/sp_counted_impl.hpp:66
> > #14 0x00002aaaaad61dc0 in
> facebook::fb303::FacebookServiceProcessor::process_getCounters
> (this=0x2aaaaadda677, seqid=32767, iprot=0x631030, oprot=0x7fffab301380)
> >    at /root/hive/service/include/thrift/fb303/FacebookService.cpp:2684
> > #15 0x00002aaaaad61df0 in
> facebook::fb303::FacebookServiceProcessor::process_getCounters
> (this=0x2aaaaadda821, seqid=32767, iprot=0x610ad0, oprot=0x2aaaab3e6a20)
> >    at /root/hive/service/include/thrift/fb303/FacebookService.cpp:2685
> > #16 0x00002aaaaad61edf in
> facebook::fb303::FacebookServiceProcessor::process_getCounters
> (this=0x2aaaab09fcde, seqid=0, iprot=0x631010, oprot=0x2aaaab3e6a20)
> >    at /root/hive/service/include/thrift/fb303/FacebookService.cpp:2688
> > #17 0x00002aaaaadad315 in ~ThriftHiveMetastore_get_type_pargs (this=0x0)
> at /root/hive/metastore/src/gen-cpp/ThriftHiveMetastore.h:731
> > #18 0x00002aaaaadd5866 in ~ThriftHive_fetchN_pargs (this=0x631540) at
> /root/hive/service/src/gen-cpp/ThriftHive.h:272
> > #19 0x00002aaaaadd5aed in ~ThriftHiveMetastoreClient (this=0x631540,
> __vtt_parm=0x631598) at
> /root/hive/metastore/src/gen-cpp/ThriftHiveMetastore.h:4484
> > #20 0x00002aaaaadda6db in
> boost::shared_ptr<apache::thrift::transport::TTransport>::shared_ptr<apache::thrift::transport::TBufferedTransport>
> ()
> >    at /usr/local/include/boost/smart_ptr/shared_ptr.hpp:415
> > #21 0x00002aaaaadda7f1 in sp_counted_impl_p (this=0x631590, px=0x0) at
> /usr/local/include/boost/smart_ptr/detail/sp_counted_impl.hpp:66
> > #22 0x00002aaaaad61dc0 in
> facebook::fb303::FacebookServiceProcessor::process_getCounters
> (this=0x2aaaaadd5aed, seqid=32767, iprot=0x631540, oprot=0x2aaaab01e408)
> >    at /root/hive/service/include/thrift/fb303/FacebookService.cpp:2684
> > #23 0x00002aaaaad61df0 in
> facebook::fb303::FacebookServiceProcessor::process_getCounters
> (this=0x35ca98, seqid=0, iprot=0xffffffff, oprot=0x3)
> >    at /root/hive/service/include/thrift/fb303/FacebookService.cpp:2685
> > #24 0x00002aaaaadda211 in apache::thrift::transport::TTransport::borrow
> (this=0x631740) at
> /root/hive/service/include/thrift/transport/TTransport.h:178
> > #25 0x00002aaaaaddaeef in ~sp_counted_impl_p (this=0x0) at
> /usr/local/include/boost/smart_ptr/detail/sp_counted_impl.hpp:54
> > #26 0x00002aaaaadd90d3 in DBColumns (connection=0x631740,
> fpHiveToSQLType=0x7fffab301780, tbl_search_pattern=0x80 <Address 0x80 out of
> bounds>,
> >    col_search_pattern=0x2aaaaadd90d3
> "E?H\213U?H\213E?H\211\020H\213E?H\213M?H\213U?H\213u?H\213}?L\213M\020I\211??\210Z??\211E??\036H\211E?H\213]?H\213}??B4??H\211]?H\213}??U???\213E?H\203?X[??\220UH\211?SH\203?XH\211}?H\211u?H\211U?H\211M?L\211E?H\203}?",
> resultset_ptr=0x7fffab301770, err_buf=0x0, err_buf_len=46909632806913)
> >    at /root/hive/odbc/src/cpp/hiveclient.cpp:143
> > #27 0x00000000004012da in basic_connect_disconnect_test () at
> /root/hive/odbc/src/test/hiveclienttest.c:168
> > #28 0x000000000040de1f in main () at
> /root/hive/odbc/src/test/hiveclienttest.c:1357
> >
> >
> >
> > uname -a
> > Linux ip-10-100-150-245 2.6.21.7-2.fc8xen-ec2-v1.0 #1 SMP Tue Sep 1
> 10:25:30 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
> >
> >
> >
>
>

Mime
View raw message