Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8FAB118CF1 for ; Wed, 20 Jan 2016 04:06:34 +0000 (UTC) Received: (qmail 14060 invoked by uid 500); 20 Jan 2016 04:06:33 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 13985 invoked by uid 500); 20 Jan 2016 04:06:32 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 13975 invoked by uid 99); 20 Jan 2016 04:06:32 -0000 Received: from Unknown (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 Jan 2016 04:06:32 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 7964E1A0564 for ; Wed, 20 Jan 2016 04:06:32 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.974 X-Spam-Level: *** X-Spam-Status: No, score=3.974 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=3, LOTS_OF_MONEY=0.001, SPF_SOFTFAIL=0.972, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=liulishuo-com.20150623.gappssmtp.com Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id H5nqlitTJifo for ; Wed, 20 Jan 2016 04:06:19 +0000 (UTC) Received: from mail-io0-f177.google.com (mail-io0-f177.google.com [209.85.223.177]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id AA0932054B for ; Wed, 20 Jan 2016 04:06:19 +0000 (UTC) Received: by mail-io0-f177.google.com with SMTP id q21so7663351iod.0 for ; Tue, 19 Jan 2016 20:06:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=liulishuo-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=0GVfg7BQtVAqZyq63/cNzSKTKQZlfB+QMWu/2om6o70=; b=Pen+C5O6dft7rRoXh6viS4E4dzUmBLlt3gz62mWVUJQfyFeJoELL1aR/I2K8V5Y9nL jreTW1dkh8/F2/bpUr2R6W0uE53lsI4BPgow60jKRw1D3Bdxs5dUw6rhyeRXvvd5kAHK lXF0HCGwoqpq6Gga/yRO6AvC+VcCIWl1GL4mA20uXwGPjgvc3W9v4m/MArlkUGTB1Job wlLSkOTCAI0zwij7/86nAYsQ2ODShtCRLh0ZIswdmxyH2SZbutDUwQB2kveTTLCtfhiL qq+i9xbf/T2nRfZdisjGqgIVP4JrXZ9FAFPZCvpTdAeNTdx5uGZMX4XLIFGBqlqqdDZ7 x21w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=0GVfg7BQtVAqZyq63/cNzSKTKQZlfB+QMWu/2om6o70=; b=gfElw2KEdBKfPhst4hYCTH+aNh2VaQ40dikmnpYLzOwOY+5BFaKk5fhHcUYJAzl2id V+/bFQ82YXTEc5AfVqfrpwlWdUYJUz0hH/iCurD2eBs69lD9BGRxrNOEgdrtX238rqIK 3wtBT7tGIxw5RqdPeTIQD9RDp90E/dA/H157JmR3/6GLScqkiJfy7xUa4i/tT6C7HA3M AqzkohaF7/Dtz2WYkVuPneHu3KsUf3tvfZjiCKosY5Uc3ckwHzMsT1GdPx1f4FPHTEgR qAInnnw/QglzDK4Eif2u62+9c1IRJq/43/6QDyZxkR20Bnp2+ryOQs4s4tPGS+VS6Sd2 a23w== X-Gm-Message-State: ALoCoQm6IGkd0lkWv4Lt/RGi0nMKzyzOKxftDQsf0kZ270rkWq/uDd2iFJglO5usc3w+SImy9my0uBXhm0mp82tv0IZQaZpdpR2eQtrLpOHzydpEfZdO2aY= MIME-Version: 1.0 X-Received: by 10.107.170.16 with SMTP id t16mr30980050ioe.95.1453262778813; Tue, 19 Jan 2016 20:06:18 -0800 (PST) Received: by 10.36.101.5 with HTTP; Tue, 19 Jan 2016 20:06:18 -0800 (PST) In-Reply-To: References: Date: Wed, 20 Jan 2016 12:06:18 +0800 Message-ID: Subject: Re: the `use database` command will change the scheme of target table? From: =?UTF-8?B?6JGj5Lqa5Yab?= To: user@hive.apache.org Content-Type: multipart/alternative; boundary=001a11427a60b2692b0529bc1bb6 --001a11427a60b2692b0529bc1bb6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable thanks Marcin, t1 created within the temp database right? which is point to HDFS. so the output directory of m/r job should be in HDFS? my problem is why the output directory was host in s3 filesystem after I *u= se prd* database. On Wed, Jan 20, 2016 at 11:52 AM, Marcin Tustin wrote: > That is the expected behaviour. Managed tables are created within the > directory of their host database. > > > On Tuesday, 19 January 2016, =E8=91=A3=E4=BA=9A=E5=86=9B wrote: > >> hi list, >> >> we use the HDFS and S3 as the Hive Filesystem at the same time. here >> has an issue: >> >> >> *scenario* 1: >> >> hive command: >> >> use default; >> >> create table temp.t1 // the database of temp which points to HDFS >> as >> select c1 from prd.t2; // the database of prd and the table t2 are >> all points to S3 >> >> it works well. >> >> >> *scenario* 2: >> >> hive command: >> >> *use prd; * >> >> create table temp.t1 // the database of temp which points to HDFS >> as >> select c1 from prd.t2; // the database of prd and the table t2 are >> all point to S3 >> >> the exception occurred with: >> >> Failed with exception Unable to move source >> s3a://warehouse-tmp/tmp/hive-ubuntu/hive_2016-01-20_xxxxxx/-ext-10001 to >> destination hdfs://hadoop-0/warehouse/temp.db/t1/ >> >> and then, I try to change the Scratch space by the configuration key: >> hive.exec.scratchdir, and set the value to hdfs://hadoop-0/*tmp-foo*/... >> , but also failed with: >> >> Unable to move source s3a://warehouse-tmp*/tmp-foo* ... to >> >> it seems to the *use database* command change the scheme of the path for >> target table? >> >> hive version: 0.13.1 >> >> >> thanks. >> > > Want to work at Handy? Check out our culture deck and open roles > > Latest news at Handy > Handy just raised $50m > led > by Fidelity > > --001a11427a60b2692b0529bc1bb6 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
thanks Marcin,=C2=A0

t1 created within the temp database right? which = is point to HDFS. =C2=A0so the output directory of m/r job should be in HDF= S?=C2=A0

my problem is why the output dir= ectory was host in s3=C2=A0filesystem after I use prd database.=C2= =A0



On Wed, Jan 20, 2016 at 11:52 A= M, Marcin Tustin <mtustin@handybook.com> wrote:
That is the expected behaviour. Managed tables ar= e created within the directory of their host database.=C2=A0


On Tuesday, 19 January 2016, =E8=91=A3=E4=BA= =9A=E5=86=9B <ric.dong@liulishuo.com> wrote:
<= div dir=3D"ltr">hi list,=C2=A0
we use the HDFS and S3 as = the Hive Filesystem at the same time. =C2=A0 here has an issue:=C2=A0

scenario 1:

hive command:
=C2=A0
use default;=C2= =A0

create table temp.t1 =C2=A0 =C2=A0 =C2=A0 // the database of tem= p which points to HDFS=C2=A0
as=C2=A0
select c1 from prd.t2; =C2=A0 = =C2=A0 // the database of prd and the table t2 are all points to S3

= it works well.=C2=A0


scenario 2:=C2=A0

hive comman= d:=C2=A0

use prd;=C2=A0

create table temp.t1 =C2=A0 = =C2=A0 =C2=A0 // the database of temp which points to HDFS=C2=A0
as=C2= =A0
select c1 from prd.t2; =C2=A0 =C2=A0 // the database of prd and the = table t2 are all point to S3

the exception occurred with:=C2=A0
<= br>Failed with exception Unable to move source s3a://warehouse-tmp/tmp/hive= -ubuntu/hive_2016-01-20_xxxxxx/-ext-10001 to destination hdfs://hadoop-0/wa= rehouse/temp.db/t1/

and then, I try to change the Scratch space by t= he configuration key: hive.exec.scratchdir, and set the value to hdfs://had= oop-0/tmp-foo/... , but also failed with:

Unable to move sour= ce s3a://warehouse-tmp/tmp-foo ... to=C2=A0

it seems to the <= i>use database command change the scheme of the path for target = table?=C2=A0

hive version: 0.13.1=C2=A0

thanks.= =C2=A0

Want to= work at Handy? Check out our=C2=A0culture deck = and open roles
=
Latest= =C2=A0news=C2=A0at Handy
Handy=C2=A0just raised $5= 0m=C2=A0led by Fidelity


--001a11427a60b2692b0529bc1bb6--