From dev-return-1825-archive-asf-public=cust-asf.ponee.io@orc.apache.org Mon Feb 12 23:08:06 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id 3A57418067B for ; Mon, 12 Feb 2018 23:08:06 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 2A5A5160C65; Mon, 12 Feb 2018 22:08:06 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 7D835160C30 for ; Mon, 12 Feb 2018 23:08:05 +0100 (CET) Received: (qmail 51195 invoked by uid 500); 12 Feb 2018 22:08:04 -0000 Mailing-List: contact dev-help@orc.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@orc.apache.org Delivered-To: mailing list dev@orc.apache.org Received: (qmail 50136 invoked by uid 99); 12 Feb 2018 22:08:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 12 Feb 2018 22:08:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 856931A0B72 for ; Mon, 12 Feb 2018 22:08:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.879 X-Spam-Level: * X-Spam-Status: No, score=1.879 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id jKqSYqZt2MRH for ; Mon, 12 Feb 2018 22:08:02 +0000 (UTC) Received: from mail-ot0-f172.google.com (mail-ot0-f172.google.com [74.125.82.172]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id DBB8C5F398 for ; Mon, 12 Feb 2018 22:08:01 +0000 (UTC) Received: by mail-ot0-f172.google.com with SMTP id a7so15455504otk.9 for ; Mon, 12 Feb 2018 14:08:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=jPZU4JGRcv88Uqiqfubb645yfBU0ISeh6r62WypccYM=; b=pUIHgSrhFA1tHTnEWm1uG+LcIYddWd0s7aZEThWGGCERa3HlqY6QQKpdoXF3FW0t7V D2jEYznq5DF0xc9ar3jHT2iJqGqkseH4FbX8KyTZz7ykdAY894LPNoOOgIiR3u8kMrpx GVICPiqRhg7IZZj1za5r476n6Lb09fgIjsH41l2C7pUq4kasBgx9HdCS+o0SEY9qaiG6 GZ20vGoMRvN0hlSir1zdPvBX7pcsq4qoYnpnUyCf8J71hFvHx9C9GjlRwf+gv8Y5ird4 y+37/iFutzw7a4DGdgMA8xEI6jRmoTFiV+QlBX5Vtjz4W9UgR2KTIqWxYgdrsriwxy+6 y/Rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=jPZU4JGRcv88Uqiqfubb645yfBU0ISeh6r62WypccYM=; b=bBmi8nwczLBhouj1/Zaje5CYWrJW1nDEXnE36Rp6Lc1mNHNZsSEb4FqdSXtcrim8oo ojilVbZXJGZAcB8ttxKOjRNTh5/RQrZ7yvBNWBfvnMeQFKjjWCVsUtNnTEL5OffP34FU nUoYPeeQrjkuZHj3D32Ozoug+MmVeDEgGgvSX8mkLDv6FZLwf8NOnCLX0rgHUGf9/ra1 F+sUz5oW5ppFrpfkAK7EsURSTzlA5XcqStfqDOd8NkCajfBYaoKvW5gMlQ7kJ8zPDDOE V2H81h12yBGQWjYC/NSjJ9QktHBYw0d/5Ow8s3QiVykYtmdp9boHegLqxImqYC198f3a WO7g== X-Gm-Message-State: APf1xPDiTjJf/hM0NAlmxqSWlSIaP87nh3OwTduwNLpevHa65mPJl3zV DmuaOkoZ4NYxo9kAbvpIesi0xqC+8kBVu9gqbqerGw== X-Google-Smtp-Source: AH8x225Zg9lIoHuXLcsMmDJeGtpGbh6XXNIW2YR/sIckU8aUn3JNkmUDnwaokd5rnleiXDJng2t1MHqNiRPNuNIxRYw= X-Received: by 10.157.53.5 with SMTP id o5mr9575925otc.181.1518473280812; Mon, 12 Feb 2018 14:08:00 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.29.6 with HTTP; Mon, 12 Feb 2018 14:07:59 -0800 (PST) In-Reply-To: References: From: "Owen O'Malley" Date: Mon, 12 Feb 2018 14:07:59 -0800 Message-ID: Subject: Re: Including Apache ORC as a library To: dev@orc.apache.org Content-Type: multipart/alternative; boundary="001a11c046c480faff05650b1c02" --001a11c046c480faff05650b1c02 Content-Type: text/plain; charset="UTF-8" I'm assuming that you are using the Java rather than C++ side of of the project. What you want is org.apache.orc:orc-core, which includes the protobuf class as org.apache.orc.OrcProto. That jar depends on org.apache.hive:hive-storage-api, which comes from Hive and defines the vectorized API. The ORC project also releases a variant using the "nohive" classifier. It incorporates the storage-api and protobuf libraries into orc-core and shrouds them so that they do not conflict with Hive. This allows projects that already depend on a particular version of Hive to use ORC's "nohive" variant without a conflict. ORC-core provides the vectorized API, which is very efficient and does not create any objects in the inner loop. If you want an easier API with OrcStruct, you will want to use the orc-mapreduce jar. .. Owen On Mon, Feb 12, 2018 at 1:53 PM, Matt Burgess wrote: > Hi all, sorry if this is a n00b question or has been answered, I > looked in the mailing list archives and could find anything. > > I'm trying to bring Apache ORC into Apache NiFi as basically a > third-party library, to support a processor that writes data as ORC. A > version of this processor already exists, but it uses Hive 1.2.1 which > has hive-orc. Now that Apache ORC is its own project, and we're > upgrading the Hive processors in NiFi to Hive 2.x (and 3.x), I'd like > to add a version of the processor that uses the current version of > Apache ORC (for Java). > > However, when I bring in org.apache.orc:orc:1.4.3 as a Maven > dependency, it is trying to find a JAR with those coordinates, when it > is only published as a POM (even when I set the type of artifact as > "pom"). If instead I bring in orc-core, I don't have access to > orc-proto, etc. > > What's the approach for bringing in all the necessary dependencies to > be able to use (including subclassing) ORC classes? > > Thanks in advance, > Matt > --001a11c046c480faff05650b1c02--