Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1A54CE7C8 for ; Fri, 15 Feb 2013 16:46:19 +0000 (UTC) Received: (qmail 11650 invoked by uid 500); 15 Feb 2013 16:46:17 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 11544 invoked by uid 500); 15 Feb 2013 16:46:17 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 11536 invoked by uid 99); 15 Feb 2013 16:46:17 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 15 Feb 2013 16:46:17 +0000 X-ASF-Spam-Status: No, hits=2.7 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_REPLYTO_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [98.139.213.129] (HELO nm4-vm0.bullet.mail.bf1.yahoo.com) (98.139.213.129) by apache.org (qpsmtpd/0.29) with SMTP; Fri, 15 Feb 2013 16:46:07 +0000 Received: from [98.139.215.140] by nm4.bullet.mail.bf1.yahoo.com with NNFMP; 15 Feb 2013 16:45:45 -0000 Received: from [98.139.212.231] by tm11.bullet.mail.bf1.yahoo.com with NNFMP; 15 Feb 2013 16:45:45 -0000 Received: from [127.0.0.1] by omp1040.mail.bf1.yahoo.com with NNFMP; 15 Feb 2013 16:45:45 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 881502.25992.bm@omp1040.mail.bf1.yahoo.com Received: (qmail 50426 invoked by uid 60001); 15 Feb 2013 16:45:45 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1360946745; bh=3V+NLJoqZaG7rGGVu/6etyHoQZpzLFSxckUqV2Pj4aE=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=wn6IeqoxLTtlDDckEC5HCqkok2VEk8iNEOCXbuatbjuFbtKMqDVlCTMnkCYrceRsQrjntv5WcW1II8qFOYY+ow0gEgYAMkVv6Q/I2rYXQw1oS58oYZ5wyNUMIIVe59/0JQmOYoKJEOp0tvMan7pN5aS0NjKIZj0p0+kgGA5uwOk= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=VeQLFrJ/0I3DgRYRQyRQsegs/KU7Bw0FgHoRXr13kUwS9fNoX8G6h1qPB6VWRtM+0i4x8nz6lCVz/5goA0TNVXA7xlq37PGZtIZ5tYE9JT42neC8j09ZxFLAo90sI4XpfDLKwkVHdmyIHj6XyvnCegm1Ad3yyNpzJyJkd7DaDIU=; X-YMail-OSG: 73PdUlEVM1ljPLlAV7kJVMLg_8r_0dtCWW7_nckp1PPg2iw SlWcMO76wwFzd47Jnwr01kTAoyHuHj6aomeJZD3Slz6DvBZhLmFuRXEJLWbV FbnbhMJQlHrglCW.trc9B4zgdsbom2YbEi0ulZ3Sypq4ZoYVMlf12.VJiqM. SOwZW.FIakCv2m3hNGOtCtGywQuq3I9jnWF2FJ7fxHwQiV9ecsltNz.MMCUE zYyMFXzXCQOwx3ObOS01fjOHs_v_QDg1EO8lvneTfqF1eBrwbIu5bt1ApsAR EZKY.ySQKeHOcIMJiNbc6S1wYra4e_15kHElEZw78G8ziQ6BYBggI6DdD15U bGgAiA3i7FB06UJcjbQB3FiL9ei9ln3NpGLs3bWfvAbzO1AQl.zeEp0u4fZS OXdT19ZnBsSlC5SvKuwMg6x0v10uKMNfB4L7Awa79kcTL0qD6ferJKyZ_fvW gqzDoHn.X6fa8LftW2YkLl.vNfzMoWlUaGqP3q90R6uHZeSS35HfwVmQ8gMk JPaN724.ACOBvN1iUao0rmJK24Ea0v5ITYU6wT31GHmpKNXRdOiagiFG0L11 YW8JHVzsevJzEgQed58JP7nBl.bqJOpmEkxU- Received: from [68.87.42.115] by web141002.mail.bf1.yahoo.com via HTTP; Fri, 15 Feb 2013 08:45:45 PST X-Rocket-MIMEInfo: 001.001,W2Nsb3VkZXJhQGxvY2FsaG9zdCBkYXRhXSQgaGRmcyBkZnMgLWxzIDcxNQpGb3VuZCAxMyBpdGVtcwotcnctci0tci0twqDCoCAxIGNsb3VkZXJhIGNsb3VkZXJhwqDCoMKgIDc4NTM5NzUgMjAxMy0wMi0xNSAwMDo0MSA3MTUvZmlsZS5jc3YgKGRpZG4ndCBsZWF2ZSByZXN0IG9mIGxzIHJlc3VsdHMpCgoKVGhhbmtzIG9uIHRoZSBkaXJlY3RvcnktLXdhc24ndCBjbGVhciBvbiB0aGF0Li4KCkpvZXkKCgoKCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwogRnJvbTogRGVhbiBXYW1wbGVyIDxkZWEBMAEBAQE- X-Mailer: YahooMailWebService/0.8.133.508 References: <1360945759.88325.YahooMailNeo@web141005.mail.bf1.yahoo.com> Message-ID: <1360946745.46060.YahooMailNeo@web141002.mail.bf1.yahoo.com> Date: Fri, 15 Feb 2013 08:45:45 -0800 (PST) From: Joseph D Antoni Reply-To: Joseph D Antoni Subject: Re: CREATE EXTERNAL TABLE Fails on Some Directories To: "user@hive.apache.org" In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="348076277-1570367376-1360946745=:46060" X-Virus-Checked: Checked by ClamAV on apache.org --348076277-1570367376-1360946745=:46060 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable [cloudera@localhost data]$ hdfs dfs -ls 715=0AFound 13 items=0A-rw-r--r--= =A0=A0 1 cloudera cloudera=A0=A0=A0 7853975 2013-02-15 00:41 715/file.csv (= didn't leave rest of ls results)=0A=0A=0AThanks on the directory--wasn't cl= ear on that..=0A=0AJoey=0A=0A=0A=0A=0A=0A________________________________= =0A From: Dean Wampler =0ATo: user@hive= .apache.org; Joseph D Antoni =0ASent: Friday, February= 15, 2013 11:37 AM=0ASubject: Re: CREATE EXTERNAL TABLE Fails on Some Direc= tories=0A =0A=0AYou confirmed that 715 is an actual directory? It didn't be= come a file by accident?=0A=0ABy the way, you don't need to include the fil= e name in the LOCATION. It will read all the files in the directory.=0A=0Ad= ean=0A=0A=0AOn Fri, Feb 15, 2013 at 10:29 AM, Joseph D Antoni wrote:=0A=0AI'm trying to create a series of external tables for a = time series of data (using the prebuilt Cloudera VM).=0A>=0A>=0A>The direct= ory structure in HDFS is as such:=0A>=0A>=0A>/711=0A>/712=0A>/713=0A>/714= =0A>/715=0A>/716=0A>/717=0A>=0A>=0A>Each directory contains the same set of= files, from a different day. They were all put into HDFS using the followi= ng script:=0A>=0A>=0A>for i in *;do hdfs dfs -put $i in $dir;done=0A>=0A>= =0A>They all show up with the same ownership/perms in HDFS.=0A>=0A>=0A>Goin= g into Hive to build the tables, I built a set of scripts to do the loads--= then did a sed (changing 711 to 712,713, etc) to a file for each day. All o= f my loads work, EXCEPT for 715 and 716.=A0=0A>=0A>=0A>Script is as follows= :=0A>=0A>=0A>create external table 715_table_name=0A>(col1 string,=0A>col2 = string)=0A>row format=0A>delimited fields terminated by ','=0A>lines termin= ated by '\n'=0A>stored as textfile=0A>location '/715/file.csv';=0A>=0A>=0A>= This is failing with:=0A>=0A>=0A>Error in Metadata MetaException(message:Go= t except: org.apache.hadoop.fs.FileAlreadyExistsException Parent Path is no= t a directory: /715 715...=0A>=0A>=0A>Like I mentioned it works for all of = the other directories, except 715 and 716. Thoughts on troubleshooting path= ?=0A>=0A>=0A>Thanks=0A>=0A>=0A>Joey D'Antoni=0A=0A=0A-- =0ADean Wampler, Ph= .D.=0Athinkbiganalytics.com=0A+1-312-339-1330 --348076277-1570367376-1360946745=:46060 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
[cloudera@localhost data]$ hdf= s dfs -ls 715
Found 13 items
-rw-r--r--   1 cloudera cloude= ra    7853975 2013-02-15 00:41 715/file.csv (didn't leave re= st of ls results)


Thanks on the directory--wasn't clear on that.= .

Joey



<= b>From: Dean Wampler <dean.= wampler@thinkbiganalytics.com>
To: user@hive.apache.org; Joseph D Antoni <jdanton1@yahoo.co= m>
Sent: Friday, F= ebruary 15, 2013 11:37 AM
Subject: Re: CREATE EXTERNAL TABLE Fails on Some Directories
=0A
You confirmed that 715 is an actual director= y? It didn't become a file by accident?

By the way, you = don't need to include the file name in the LOCATION. It will read all the f= iles in the directory.
=0A

dean

On Fri, Feb 15, 2013 at 10:29 AM, Joseph D An= toni <jdanton1@yah= oo.com> wrote:
=0A
I'm trying to create a series of external tables for a time s= eries of data (using the prebuilt Cloudera VM).
=0A

The directory structure in HDFS is as such:
=0A
/711
=0A/712
/713
=0A/714
/715
= =0A/716
/717
=0A
Each directory contains the same set of files= , from a different day. They were all put into HDFS using the following scr= ipt:
=0A

=0Afor= i in *;do hdfs dfs -put $i in $dir;done

=0AThey all show up with the same ownership/perm= s in HDFS.

=0A= Going into Hive to build the tables, I built a set of scripts to do the loa= ds--then did a sed (changing 711 to 712,713, etc) to a file for each day. A= ll of my loads work, EXCEPT for 715 and 716. 
=0A
Script is as follows:
=0A
create external table 71= 5_table_name
=0A(col1 string,
col2 string)
=0Arow format
delimited fields terminated by ','
=0Alines terminated by '\n'
stored as = textfile
=0Alocation '/715/file.csv';

=0AThis is failing with= :

=0AError in = Metadata MetaException(message:Got except: org.apache.hadoop.fs.FileAlready= ExistsException Parent Path is not a directory: /715 715...
=0A
Like I mentioned it works = for all of the other directories, except 715 and 716. Thoughts on troublesh= ooting path?
=0A


= =0AJoey D'Antoni


=

--
Dean Wampler, Ph.D.
+1-312-339-1330
=0A

<= /font>
=0A
=0A


--348076277-1570367376-1360946745=:46060--