From dev-return-36684-archive-asf-public=cust-asf.ponee.io@sqoop.apache.org Mon Sep 3 13:32:41 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 8682A180647 for ; Mon, 3 Sep 2018 13:32:40 +0200 (CEST) Received: (qmail 14342 invoked by uid 500); 3 Sep 2018 11:32:39 -0000 Mailing-List: contact dev-help@sqoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@sqoop.apache.org Delivered-To: mailing list dev@sqoop.apache.org Received: (qmail 14331 invoked by uid 500); 3 Sep 2018 11:32:38 -0000 Delivered-To: apmail-incubator-sqoop-dev@incubator.apache.org Received: (qmail 14328 invoked by uid 99); 3 Sep 2018 11:32:38 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Sep 2018 11:32:38 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 93AC7180102; Mon, 3 Sep 2018 11:32:38 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.902 X-Spam-Level: * X-Spam-Status: No, score=1.902 tagged_above=-999 required=6.31 tests=[DKIM_ADSP_CUSTOM_MED=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=2, KAM_LAZY_DOMAIN_SECURITY=1, NML_ADSP_CUSTOM_MED=1.2, RCVD_IN_DNSWL_MED=-2.3] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id ahlZnHdt1C8U; Mon, 3 Sep 2018 11:32:37 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 526B45F232; Mon, 3 Sep 2018 11:32:37 +0000 (UTC) Received: from reviews.apache.org (unknown [10.41.0.12]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id F1BF2E00CB; Mon, 3 Sep 2018 11:32:36 +0000 (UTC) Received: from reviews-vm2.apache.org (localhost [IPv6:::1]) by reviews.apache.org (ASF Mail Server at reviews-vm2.apache.org) with ESMTP id BC59EC4049A; Mon, 3 Sep 2018 11:32:36 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============2244467270361684739==" MIME-Version: 1.0 Subject: Review Request 68606: Error during direct Netezza import/export can interrupt process in uncontrolled ways From: daniel voros To: daniel voros , Sqoop Date: Mon, 03 Sep 2018 11:32:36 -0000 Message-ID: <20180903113236.49310.94729@reviews-vm2.apache.org> X-ReviewBoard-URL: https://reviews.apache.org/ Auto-Submitted: auto-generated Sender: daniel voros X-ReviewGroup: Sqoop X-Auto-Response-Suppress: DR, RN, OOF, AutoReply X-ReviewRequest-URL: https://reviews.apache.org/r/68606/ X-Sender: daniel voros X-ReviewBoard-Diff-For: src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableExportMapper.java X-ReviewBoard-Diff-For: src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableImportMapper.java Reply-To: daniel voros X-ReviewRequest-Repository: sqoop-trunk --===============2244467270361684739== MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68606/ ----------------------------------------------------------- Review request for Sqoop. Bugs: SQOOP-3378 https://issues.apache.org/jira/browse/SQOOP-3378 Repository: sqoop-trunk Description ------- `SQLException` during JDBC operation in direct Netezza import/export signals parent thread to fail fast by interrupting it. We're trying to process the interrupt in the parent (main) thread, but there's no guarantee that we're not in some internal call that will process the interrupted flag and reset it before we're able to check. It is also possible that the parent thread has passed the "checking part" when it gets interrupted. In case of `NetezzaExternalTableExportMapper` this can interrupt the upload of log files. I'd recommend using some other means of communication between the threads than interrupts. Diffs ----- src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableExportMapper.java 5bf21880 src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableImportMapper.java 306062aa src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaJDBCStatementRunner.java cedfd235 src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableExportMapper.java PRE-CREATION src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableImportMapper.java PRE-CREATION Diff: https://reviews.apache.org/r/68606/diff/1/ Testing ------- added new UTs and checked manual Netezza tests (NetezzaExportManualTest, NetezzaImportManualTest) Thanks, daniel voros --===============2244467270361684739==--