From users-return-8174-archive-asf-public=cust-asf.ponee.io@nifi.apache.org Mon Mar 12 15:13:34 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 0A80918064D for ; Mon, 12 Mar 2018 15:13:33 +0100 (CET) Received: (qmail 47636 invoked by uid 500); 12 Mar 2018 14:13:32 -0000 Mailing-List: contact users-help@nifi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@nifi.apache.org Delivered-To: mailing list users@nifi.apache.org Received: (qmail 47626 invoked by uid 99); 12 Mar 2018 14:13:32 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 12 Mar 2018 14:13:32 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 32B151A4F53 for ; Mon, 12 Mar 2018 14:13:32 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.121 X-Spam-Level: X-Spam-Status: No, score=-0.121 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id UVxEfEQEecA8 for ; Mon, 12 Mar 2018 14:13:31 +0000 (UTC) Received: from mail-qt0-f169.google.com (mail-qt0-f169.google.com [209.85.216.169]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 480275F2EC for ; Mon, 12 Mar 2018 14:13:30 +0000 (UTC) Received: by mail-qt0-f169.google.com with SMTP id l25so18938743qtj.1 for ; Mon, 12 Mar 2018 07:13:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=V5ynN4jKITrwCCnB8uQgzS+xxLCuhaRjgm+GsQUztgQ=; b=dZDwl7SriMfeR/0VICq7uf6L28k7xaequzoZDHEKjBWSajqV2Yq5Flj5cD17F9b31w lRq410poqiPCpkNYJgqNjCh4VHAD2TOIZbhKiVXySxl+45R04ZnmEYmqkJMB7tA8TQGf MLDZiR6X6xyQyvFBo1gPozgZxjc2TWk4X5oo5NB6g/3YcS7/VARhovNesKTCb3psziwY kjFRYQ7XMjeYFCiYeUp3rl66tBYCI1yXDOB8vwCiKh0z7mCfK5TgCxwQwoEbgXNPRuI2 4dhIm5K3ohNOhYXGnwcw/GxwLmN/2jQ1eXl9IuNOPmJDnluXsPl+swC8yQE7Z7PyR8Hc /wNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=V5ynN4jKITrwCCnB8uQgzS+xxLCuhaRjgm+GsQUztgQ=; b=lZWx0ubh1iuYUaFto34Mi8ApV69okgYtAbTTvvZapVoVjryzjgmqtdV6bcGYU+xDPW bFR5WyHSR5edPP+iPlZFaFGLa+Tm/DKWDZhHlcFiHFO/zY2oYsDwEpAQunG6qwCwNUqf 5rjbDIZyuVymrkzgaot2GpLH5EzNlpdl16RQ/5RJg6sJcf20bkyakvwyhLLnaiYgqb8/ 0VTSRjxB410S/slMrbajeHHAGpz/GNe9vJKVpwDcxloTFjIVoun8fUBSVs3OquP+UDUs LlWawjxf+dOchKmi2rY9kin8iAbJ9CxEmLMT7f8amAQIbsa3SP1fR9uVYQx2a3pWRGcw eiaw== X-Gm-Message-State: AElRT7HzpWghde0F0sydsLiBdBdwG20OHzSRjHRVl8hlVcYFrPkICrED ZQ4n+4Za119aXDPeyfLproOr6jljfHw6ukJOBJ0OZNsl X-Google-Smtp-Source: AG47ELtrNV2Fhvhc6jp1BMvYbXaDcOHH9VQWbUYD3VCBEu5h18rx1PpTd5rKWjaYFhoMZrXobpGqw3zCcz6d/pRSHpI= X-Received: by 10.200.15.58 with SMTP id e55mr11979924qtk.261.1520864009008; Mon, 12 Mar 2018 07:13:29 -0700 (PDT) MIME-Version: 1.0 Received: by 10.12.227.212 with HTTP; Mon, 12 Mar 2018 07:13:28 -0700 (PDT) In-Reply-To: <55A9554E-B4D5-41A2-92D9-52A40B4A6814@rakuten.com> References: <55A9554E-B4D5-41A2-92D9-52A40B4A6814@rakuten.com> From: Koji Kawamura Date: Mon, 12 Mar 2018 23:13:28 +0900 Message-ID: Subject: Re: PutHiveQL (1.5.0) throws un-necessary NullPointerException when parsing the query To: "users@nifi.apache.org" Cc: "Evans, JasonLee | Jay | DPD" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable (dropped dev) Hi Amit, Thanks for sharing that. The parsing query is added by NIFI-4545 in NiFi 1.5.0, to parse Hive query at Hive related processors so that it can extract table names into outgoing FlowFile attributes. Those attributes are used by ReportLineageToAtlas reporting task to report Atlas that which processor interacts with which DataSet. https://issues.apache.org/jira/browse/NIFI-4545 Parsing query at NiFi processor is not necessary for executing the query itself. So it just logs the exception while the query completes. Do you have a complete stacktrace for the NullPointerException? Also, do you mind filing a JIRA? Thanks, Koji On Fri, Mar 9, 2018 at 11:27 AM, Ranjan, Amit | Amy | DPD wrote: > Hi Team, > > While working with NiFi 1.5.0, we observed that for few queries, PutHiveQ= L > throws NullPointerException while parsing the query. However, the query d= oes > complete the execution and flow file is routes to success afterwards. > > From the observation made till now, we found that the queries containing > back-tick(s) (`) are not getting parsed. This was not the case in NiFi > 1.4.0. Will it be possible for you to confirm whether this is the desired > behavior of NiFi 1.5.0. > > I=E2=80=99ve attached a file which contains a portion of the nifi-app.log= showing > the WARN messages of parsing exception. > > Regards, > Amit