Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id D7FE5200B9B for ; Wed, 12 Oct 2016 20:48:15 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id D63B0160AEE; Wed, 12 Oct 2016 18:48:15 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 29C87160ACA for ; Wed, 12 Oct 2016 20:48:15 +0200 (CEST) Received: (qmail 28813 invoked by uid 500); 12 Oct 2016 18:48:14 -0000 Mailing-List: contact reviews-help@impala.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list reviews@impala.incubator.apache.org Received: (qmail 28783 invoked by uid 99); 12 Oct 2016 18:48:13 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 12 Oct 2016 18:48:13 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id D424FC1A3D for ; Wed, 12 Oct 2016 18:48:12 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.362 X-Spam-Level: X-Spam-Status: No, score=0.362 tagged_above=-999 required=6.31 tests=[RDNS_DYNAMIC=0.363, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id 5XDMgHakVnya for ; Wed, 12 Oct 2016 18:48:10 +0000 (UTC) Received: from ip-10-146-233-104.ec2.internal (ec2-75-101-130-251.compute-1.amazonaws.com [75.101.130.251]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 7042160CE5 for ; Wed, 12 Oct 2016 18:48:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by ip-10-146-233-104.ec2.internal (8.14.4/8.14.4) with ESMTP id u9CIkRKp021837; Wed, 12 Oct 2016 18:46:27 GMT Message-Id: <201610121846.u9CIkRKp021837@ip-10-146-233-104.ec2.internal> Date: Wed, 12 Oct 2016 18:46:26 +0000 From: "Michael Brown (Code Review)" To: impala-cr@cloudera.com, reviews@impala.incubator.apache.org CC: David Knupp , Taras Bobrovytsky Reply-To: mikeb@cloudera.com X-Gerrit-MessageType: newpatchset Subject: =?UTF-8?Q?=5BImpala-ASF-CR=5D_IMPALA-4188=3A_Leopard=3A_support_external_Docker_volumes=0A?= X-Gerrit-Change-Id: Ia7d9d9253fcd7e3905e389ddeb1438cee3e24480 X-Gerrit-ChangeURL: X-Gerrit-Commit: 23eb2ecd401665a26308bd5e3408d1837fda5f77 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Content-Disposition: inline User-Agent: Gerrit/2.12.2 archived-at: Wed, 12 Oct 2016 18:48:16 -0000 Michael Brown has uploaded a new patch set (#4). Change subject: IMPALA-4188: Leopard: support external Docker volumes ...................................................................... IMPALA-4188: Leopard: support external Docker volumes To be able to run the Random Query Generator with Impala and Kudu, we need to use external Docker volumes as a workaround to KUDU-1419. This patch introduces a series of environment variables a user may tweak in order to help with that purpose. The patch assumes a viable, reasonable Docker container based on a standard Linux distribution like Ubuntu 14. To assist users, I've updated the Leopard README with instructions on the environment variables' meanings. The gist here is that the container is the source of truth, which means to create an external volume, we need the testdata off the container onto the host running Docker Engine. To do that we suggest a strategy using rsync via passwordless SSH key. Testing: I used a Cloudera Docker container that has Impala in /home/dev/Impala. Before, Kudu would fail to start due to KUDU-1419. Now, we load testdata into an external volume, build Impala, run the minicluster including Kudu, and can access the tpch_kudu data. I made flake8 fixes as well. flake8 on this file is now clean. Change-Id: Ia7d9d9253fcd7e3905e389ddeb1438cee3e24480 --- M tests/comparison/leopard/README M tests/comparison/leopard/impala_docker_env.py 2 files changed, 192 insertions(+), 56 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/78/4678/4 -- To view, visit http://gerrit.cloudera.org:8080/4678 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia7d9d9253fcd7e3905e389ddeb1438cee3e24480 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Michael Brown Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Taras Bobrovytsky