From dev-return-6207-archive-asf-public=cust-asf.ponee.io@arrow.apache.org Tue Apr 3 07:11:50 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 70B7318064D for ; Tue, 3 Apr 2018 07:11:49 +0200 (CEST) Received: (qmail 32809 invoked by uid 500); 3 Apr 2018 05:11:48 -0000 Mailing-List: contact dev-help@arrow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@arrow.apache.org Delivered-To: mailing list dev@arrow.apache.org Received: (qmail 32790 invoked by uid 99); 3 Apr 2018 05:11:47 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Apr 2018 05:11:47 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id BC66D180513 for ; Tue, 3 Apr 2018 05:11:46 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.302 X-Spam-Level: X-Spam-Status: No, score=-0.302 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, KAM_NUMSUBJECT=0.5, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=xhochy.com header.b=Cf5xfISa; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=Ep1NlAUU Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id AAPhoIPoDiRm for ; Tue, 3 Apr 2018 05:11:44 +0000 (UTC) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 23D5B5F19C for ; Tue, 3 Apr 2018 05:11:43 +0000 (UTC) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 3AEDA20DC5 for ; Tue, 3 Apr 2018 01:11:43 -0400 (EDT) Received: from web2 ([10.202.2.212]) by compute5.internal (MEProxy); Tue, 03 Apr 2018 01:11:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xhochy.com; h= content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; bh=0B4GuhsZVr97PRWeORBaKq8dzdElK tgXZaS3rSseXCw=; b=Cf5xfISaAb2P6ZYmWACknTwT1sGTs8+pQHHu1iO/qMZgr Hpl8tfPg2Go+xhPDiekukSgwR08xpvyN65pdHRne4kC1Pyia7zz3BjJg2sKPswtJ Vgn2aOjgW4d1uoVZ3mAceUwNYXz2libPF4V8DMBKF8inTKOgFf4uwrS7PnQyIJdw ImwAcdP3+v8JCDSDOpICXP6AKrE/uhDqjSnoGtBtAkTNhA2bOMKg1fG+dxsK/THx WyDlLvhhTgKXC6e2ASwSUU7mNNb6HDjvWPxNPAu1t3o0D/L7bpLGo8rp4heJjskY ym9GGjh9mi5kTZ7uFBL+VB0hQsOBrdtwJTccXn8OA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=0B4Guh sZVr97PRWeORBaKq8dzdElKtgXZaS3rSseXCw=; b=Ep1NlAUUjhDEYBhZZrj1OC kpBqZguUxdnaQzp4D+xsYhMOVs//35NYXpm5GUe4MWtNl/qsz461jxTf/SP5dSad qdIzLPhDFqrJgXcG8wGyfr8VQxT6hRYqCumZT8y1VjRAccPn0akv+yGxaUdaBV59 7irtE4TyNehvKIm2IqmxLZdobhleLA8VVaCqw8YYz+gRLc3ms5DNjyJo6oh+WQSZ qbuUX8/RV80bX61Aw/9mTtSm9jKWBw88boZZgMVEbLOLKzAOWXiBEMnThy4/zVys ZgVwuZrkbNJjkjQguxS8MG5L6iM6QSm6lsw9obtQ+UljKY0Iw/y3x7h8EOnxC20A == X-ME-Sender: Received: by mailuser.nyi.internal (Postfix, from userid 99) id EE01E621BF; Tue, 3 Apr 2018 01:11:42 -0400 (EDT) Message-Id: <1522732302.918219.1324449296.428C4F86@webmail.messagingengine.com> From: "Uwe L. Korn" To: dev@arrow.apache.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="utf-8" X-Mailer: MessagingEngine.com Webmail Interface - ajax-bb419338 In-Reply-To: References: Subject: Re: Upstream segmentation fault when using 0.9.0, OK with 0.8.0 Date: Tue, 03 Apr 2018 07:11:42 +0200 The problem is that with ORC on, we link to a self-build protobuf version but on load in SciDB, it picks up the system one. We actually should build the packages with the system one: https://issues.apache.org/jira/browse/ARROW-2383 On Tue, Apr 3, 2018, at 7:06 AM, Rares Vernica wrote: > Hi Wes, > > That did it! Thanks so much for the pointer. > > BTW, enjoy your time off. > > Cheers, > Rares > > On Mon, Apr 2, 2018 at 12:33 PM, Wes McKinney wrote: > > > hi Rares -- does SciDB use Protocol Buffers? You may be running into a > > version conflict with the vendored protocol buffers libraries that's > > part of -DARROW_ORC=on > > > > Wes > > > > On Sun, Apr 1, 2018 at 9:48 PM, Rares Vernica wrote: > > > Hello, > > > > > > I'm using libarrow.so to build a simplified SciDB "plugin". The plugin > > gets > > > loaded dynamically into a running SciDB instance. The plugin just does > > > arrow::default_memory_pool(). With Arrow 0.8.0, the plugin gets loaded > > > successfully and can be used in SciDB. With Arrow 0.9.0, SciDB crashes > > when > > > it tires to load the .so file of the plugin. I'm not sure if it is an > > Arrow > > > issue or some non-Arrow issue that just pops up now. > > > > > > The example used here is highly simplified. I'm experiencing this > > situation > > > in the more complex accelerated_io_tools SciDB plugin > > > https://github.com/Paradigm4/accelerated_io_tools I've been using this > > > plugin for a while successfully with Arrow 0.8.0. > > > > > > I'm using a Ubuntu Trusty instance with Arrow packages from > > > red-data-tools.org. I also tried with Arrow packages compiled from > > source > > > using g++ 4.9, but the issue is still present. > > > > > > I compile the plugin like this: > > > > > > "/usr/bin/g++-4.9" -W -Wextra -Wall -Wno-unused-parameter > > > -Wno-variadic-macros -Wno-strict-aliasing -Wno-long-long -Wno-unused > > -fPIC > > > -D_STDC_FORMAT_MACROS -Wno-system-headers -g -DNDEBUG > > -D_STDC_LIMIT_MACROS > > > -fno-omit-frame-pointer -std=c++14 -DCPP11 -I. > > > -DPROJECT_ROOT="\"/opt/scidb/18.1\"" > > > -I"/opt/scidb/18.1/3rdparty/boost/include/" -I"/opt/scidb/18.1/include" > > -o > > > liblimit.so plugin.cpp LogicalLimit.cpp PhysicalLimit.cpp -shared > > > -Wl,-soname,liblimit.so -L. -L"/opt/scidb/18.1/3rdparty/boost/lib" > > > -L"/opt/scidb/18.1/lib" -Wl,-rpath,/opt/scidb/18.1/lib: -lm -larrow > > > > > > Here is the backtrace when I try to load the plugin: > > > > > > Program received signal SIGSEGV, Segmentation fault. > > > [Switching to Thread 0x7fcf89669700 (LWP 1096)] > > > 0x00007fcf96048240 in std::string::resize(unsigned long, char) () > > > from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 > > > (gdb) bt > > > #0 0x00007fcf96048240 in std::string::resize(unsigned long, char) () > > > from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 > > > #1 0x00007fcf985081e8 in > > > google::protobuf::internal::WireFormatLite::ReadString( > > google::protobuf::io::CodedInputStream*, > > > std::string*) () > > > from /usr/lib/x86_64-linux-gnu/libprotobuf.so.8 > > > #2 0x00007fcf98545539 in > > > google::protobuf::FileDescriptorProto::MergePartialFromCodedStream( > > google::protobuf::io::CodedInputStream*) > > > () > > > from /usr/lib/x86_64-linux-gnu/libprotobuf.so.8 > > > #3 0x00007fcf8859b11c in ?? () from /usr/lib/x86_64-linux-gnu/ > > libarrow.so.0 > > > #4 0x00007fcf885e1533 in ?? () from /usr/lib/x86_64-linux-gnu/ > > libarrow.so.0 > > > #5 0x00007fcf885a278a in ?? () from /usr/lib/x86_64-linux-gnu/ > > libarrow.so.0 > > > #6 0x00007fcf885808ac in ?? () from /usr/lib/x86_64-linux-gnu/ > > libarrow.so.0 > > > #7 0x00007fcf998081da in ?? () from /lib64/ld-linux-x86-64.so.2 > > > #8 0x00007fcf998082c3 in ?? () from /lib64/ld-linux-x86-64.so.2 > > > #9 0x00007fcf9980cd00 in ?? () from /lib64/ld-linux-x86-64.so.2 > > > #10 0x00007fcf99808094 in ?? () from /lib64/ld-linux-x86-64.so.2 > > > #11 0x00007fcf9980c44b in ?? () from /lib64/ld-linux-x86-64.so.2 > > > #12 0x00007fcf991cf02b in ?? () from /lib/x86_64-linux-gnu/libdl.so.2 > > > #13 0x00007fcf99808094 in ?? () from /lib64/ld-linux-x86-64.so.2 > > > #14 0x00007fcf991cf62d in ?? () from /lib/x86_64-linux-gnu/libdl.so.2 > > > #15 0x00007fcf991cf0c1 in dlopen () from /lib/x86_64-linux-gnu/libdl. > > so.2 > > > #16 0x0000000000951c8b in scidb::PluginManager::findModule(std::string > > > const&, bool*) () > > > #17 0x000000000095235f in scidb::PluginManager::loadLibrary(std::string > > > const&, bool) () > > > #18 0x0000000000ccb1a2 in > > > scidb::PhysicalLoadLibrary::execute(std::vector > shared_ptr, > > > std::allocator > >&, > > > std::shared_ptr) () > > > #19 0x0000000000a37c6f in > > > scidb::PhysicalOperator::executeWrapper(std::vector< > > std::shared_ptr, > > > std::allocator > >&, > > > std::shared_ptr) () > > > #20 0x00000000009e5628 in > > > scidb::QueryProcessorImpl::execute(std::shared_ptr > :PhysicalQueryPlanNode>, > > > std::shared_ptr, int) () > > > #21 0x00000000009e5dd9 in > > > scidb::QueryProcessorImpl::execute(std::shared_ptr) () > > > #22 0x0000000000a1c6f4 in > > > scidb::SciDBExecutor::completeExecuteQuery(scidb::QueryResult&, > > > std::shared_ptr const&) () > > > #23 0x0000000000903964 in > > > scidb::ClientMessageHandleJob::completeExecuteQuery(std:: > > shared_ptr > > > const&) () > > > #24 0x00000000009005a6 in scidb::ClientMessageHandleJob::run() () > > > #25 0x000000000093d59a in > > > scidb::Job::executeOnQueue(std::weak_ptr&, > > > std::shared_ptr&) () > > > #26 0x00000000008b34b7 in > > > scidb::WorkQueue::invokeWithContext(boost::function > > (std::weak_ptr&, > > > std::shared_ptr&)>&, > > > std::shared_ptr&, > > > std::weak_ptr&) () > > > #27 0x0000000000972ac3 in scidb::WorkQueue::WorkQueueJob::run() () > > > #28 0x000000000093d6d1 in scidb::Job::execute() () > > > #29 0x0000000000947f39 in scidb::Thread::_threadFunction() () > > > #30 0x0000000000948b39 in scidb::Thread::threadFunction(void*) () > > > #31 0x00007fcf993da184 in start_thread () > > > from /lib/x86_64-linux-gnu/libpthread.so.0 > > > #32 0x00007fcf9579a03d in clone () from /lib/x86_64-linux-gnu/libc.so.6 > > > > > > Notice libarrow in frames #3-6. Also interesting is libprotobuf in frames > > > #1-2. > > > > > > Any ideas are welcome. > > > > > > Thanks! > > > Rares > >