Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 17EE695E9 for ; Tue, 28 Feb 2012 02:21:55 +0000 (UTC) Received: (qmail 68984 invoked by uid 500); 28 Feb 2012 02:21:54 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 68950 invoked by uid 500); 28 Feb 2012 02:21:54 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 68941 invoked by uid 99); 28 Feb 2012 02:21:54 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 Feb 2012 02:21:54 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of sseverance@alphaheavy.com designates 209.85.210.176 as permitted sender) Received: from [209.85.210.176] (HELO mail-iy0-f176.google.com) (209.85.210.176) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 Feb 2012 02:21:49 +0000 Received: by iagw33 with SMTP id w33so1853690iag.35 for ; Mon, 27 Feb 2012 18:21:28 -0800 (PST) Received-SPF: pass (google.com: domain of sseverance@alphaheavy.com designates 10.50.46.164 as permitted sender) client-ip=10.50.46.164; Authentication-Results: mr.google.com; spf=pass (google.com: domain of sseverance@alphaheavy.com designates 10.50.46.164 as permitted sender) smtp.mail=sseverance@alphaheavy.com; dkim=pass header.i=sseverance@alphaheavy.com Received: from mr.google.com ([10.50.46.164]) by 10.50.46.164 with SMTP id w4mr975666igm.54.1330395688511 (num_hops = 1); Mon, 27 Feb 2012 18:21:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alphaheavy.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=itR19FGI7bHOGN7ohIqpRmr32Y+qcDvxajEDnwNBVCQ=; b=BZGH5ZhOzz9vMOTwT4BBLkQGHEdShMVnJfYCQ1ez/kH1ku99lIlSdtitzeMzFE9av0 bvZssdFeQu4u9gd+O+Bj7WgyEQ4YkERSvx6x2khgeP1bqXS6ARCafXcnE7HWNhjAwHd0 eUdasNH31oEBG4aREZdAO3hw38Oy3yVp9HeKg= MIME-Version: 1.0 Received: by 10.50.46.164 with SMTP id w4mr844212igm.54.1330395688433; Mon, 27 Feb 2012 18:21:28 -0800 (PST) Received: by 10.231.3.196 with HTTP; Mon, 27 Feb 2012 18:21:28 -0800 (PST) In-Reply-To: References: Date: Mon, 27 Feb 2012 18:21:28 -0800 Message-ID: Subject: Re: INSERT INTO Overwriting Data From: Steve Severance To: user@hive.apache.org Content-Type: multipart/alternative; boundary=14dae9340e636b788a04b9fce247 X-Gm-Message-State: ALoCoQm6I4aDnRjtMMdxNATwvzqMN6E3T0xpbCs6MDP7E8/4rnxjvZXe7h2+prII74L3/uQHjTS5 X-Virus-Checked: Checked by ClamAV on apache.org --14dae9340e636b788a04b9fce247 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable system:java.class.path=3D/opt/hadoop/conf:/usr/lib/jvm/sun-jdk-1.6/lib/tool= s.jar:/opt/hadoop:/opt/hadoop/hadoop-core-0.20.2-cdh3u3.jar:/opt/hadoop/lib= /ant-contrib-1.0b3.jar:/opt/hadoop/lib/aspectjrt-1.6.5.jar:/opt/hadoop/lib/= aspectjtools-1.6.5.jar:/opt/hadoop/lib/commons-cli-1.2.jar:/opt/hadoop/lib/= commons-codec-1.4.jar:/opt/hadoop/lib/commons-daemon-1.0.1.jar:/opt/hadoop/= lib/commons-el-1.0.jar:/opt/hadoop/lib/commons-httpclient-3.1.jar:/opt/hado= op/lib/commons-lang-2.4.jar:/opt/hadoop/lib/commons-logging-1.0.4.jar:/opt/= hadoop/lib/commons-logging-api-1.0.4.jar:/opt/hadoop/lib/commons-net-1.4.1.= jar:/opt/hadoop/lib/core-3.1.1.jar:/opt/hadoop/lib/guava-r09-jarjar.jar:/op= t/hadoop/lib/hadoop-fairscheduler-0.20.2-cdh3u2.jar:/opt/hadoop/lib/hadoop-= fairscheduler-0.20.2-cdh3u3.jar:/opt/hadoop/lib/hsqldb-1.8.0.10.jar:/opt/ha= doop/lib/jackson-core-asl-1.5.2.jar:/opt/hadoop/lib/jackson-mapper-asl-1.5.= 2.jar:/opt/hadoop/lib/jasper-compiler-5.5.12.jar:/opt/hadoop/lib/jasper-run= time-5.5.12.jar:/opt/hadoop/lib/jets3t-0.6.1.jar:/opt/hadoop/lib/jetty-6.1.= 26.cloudera.1.jar:/opt/hadoop/lib/jetty-servlet-tester-6.1.26.cloudera.1.ja= r:/opt/hadoop/lib/jetty-util-6.1.26.cloudera.1.jar:/opt/hadoop/lib/jsch-0.1= .42.jar:/opt/hadoop/lib/junit-4.5.jar:/opt/hadoop/lib/kfs-0.2.2.jar:/opt/ha= doop/lib/log4j-1.2.15.jar:/opt/hadoop/lib/mockito-all-1.8.2.jar:/opt/hadoop= /lib/oro-2.0.8.jar:/opt/hadoop/lib/servlet-api-2.5-20081211.jar:/opt/hadoop= /lib/servlet-api-2.5-6.1.14.jar:/opt/hadoop/lib/slf4j-api-1.4.3.jar:/opt/ha= doop/lib/slf4j-log4j12-1.4.3.jar:/opt/hadoop/lib/xmlenc-0.52.jar:/opt/hadoo= p/lib/jsp-2.1/jsp-2.1.jar:/opt/hadoop/lib/jsp-2.1/jsp-api-2.1.jar::/opt/hiv= e/conf:/opt/hive/lib/ant-contrib-1.0b3.jar:/opt/hive/lib/antlr-2.7.7.jar:/o= pt/hive/lib/antlr-3.0.1.jar:/opt/hive/lib/antlr-runtime-3.0.1.jar:/opt/hive= /lib/asm-3.1.jar:/opt/hive/lib/commons-cli-1.2.jar:/opt/hive/lib/commons-co= dec-1.3.jar:/opt/hive/lib/commons-collections-3.2.1.jar:/opt/hive/lib/commo= ns-dbcp-1.4.jar:/opt/hive/lib/commons-lang-2.4.jar:/opt/hive/lib/commons-lo= gging-1.0.4.jar:/opt/hive/lib/commons-logging-api-1.0.4.jar:/opt/hive/lib/c= ommons-pool-1.5.4.jar:/opt/hive/lib/datanucleus-connectionpool-2.0.3.jar:/o= pt/hive/lib/datanucleus-core-2.0.3.jar:/opt/hive/lib/datanucleus-enhancer-2= .0.3.jar:/opt/hive/lib/datanucleus-rdbms-2.0.3.jar:/opt/hive/lib/derby-10.4= .2.0.jar:/opt/hive/lib/guava-r06.jar:/opt/hive/lib/hbase-0.89.0-SNAPSHOT-te= sts.jar:/opt/hive/lib/hbase-0.89.0-SNAPSHOT.jar:/opt/hive/lib/hive-anttasks= -0.9.0-SNAPSHOT.jar:/opt/hive/lib/hive-builtins-0.9.0-SNAPSHOT.jar:/opt/hiv= e/lib/hive-cli-0.9.0-SNAPSHOT.jar:/opt/hive/lib/hive-common-0.9.0-SNAPSHOT.= jar:/opt/hive/lib/hive-contrib-0.9.0-SNAPSHOT.jar:/opt/hive/lib/hive-exec-0= .9.0-SNAPSHOT.jar:/opt/hive/lib/hive-hbase-handler-0.9.0-SNAPSHOT.jar:/opt/= hive/lib/hive-hwi-0.9.0-SNAPSHOT.jar:/opt/hive/lib/hive-jdbc-0.9.0-SNAPSHOT= .jar:/opt/hive/lib/hive-metastore-0.9.0-SNAPSHOT.jar:/opt/hive/lib/hive-pdk= -0.9.0-SNAPSHOT.jar:/opt/hive/lib/hive-serde-0.9.0-SNAPSHOT.jar:/opt/hive/l= ib/hive-service-0.9.0-SNAPSHOT.jar:/opt/hive/lib/hive-shims-0.9.0-SNAPSHOT.= jar:/opt/hive/lib/hive_contrib.jar:/opt/hive/lib/javaewah-0.3.jar:/opt/hive= /lib/jdo2-api-2.3-ec.jar:/opt/hive/lib/jline-0.9.94.jar:/opt/hive/lib/json-= 20090211.jar:/opt/hive/lib/junit-4.10.jar:/opt/hive/lib/libfb303-0.7.0.jar:= /opt/hive/lib/libfb303.jar:/opt/hive/lib/libthrift-0.7.0.jar:/opt/hive/lib/= libthrift.jar:/opt/hive/lib/log4j-1.2.16.jar:/opt/hive/lib/mockito-all-1.8.= 2.jar:/opt/hive/lib/mysql-connector-java-5.1.18-bin.jar:/opt/hive/lib/slf4j= -api-1.6.1.jar:/opt/hive/lib/slf4j-log4j12-1.6.1.jar:/opt/hive/lib/stringte= mplate-3.1-b1.jar:/opt/hive/lib/velocity-1.5.jar:/opt/hive/lib/zookeeper-3.= 3.1.jar: On Mon, Feb 27, 2012 at 1:17 PM, Carl Steinbach wrote: > Hi Steve, > > It's possible that Hive is still picking up some old JARs on its > classpath. Can you please try dumping the classpath from within the HiveC= LI > and respond with the output, e.g. > > % hive -e 'set system:java.class.path' > > Thanks. > > Carl > > > On Sat, Feb 25, 2012 at 4:42 PM, Steve Severance < > sseverance@alphaheavy.com> wrote: > >> Hi, >> >> I have a job that use INSERT INTO from Hive 0.8. The query is something >> like: >> >> INSERT INTO TABLE MyTable PARTITION (dt=3D'2012-01-01) SELECT ... >> >> When I run this query both with and without dynamic partitioning it >> overwrites the data that was already in the partition. >> >> I am running Hive 0.8 on CDH3u2. I have confirmed that append is enabled >> in the job conf. >> >> Is there anything else I need to do that would enable appending to my >> hive table? >> >> Thanks. >> >> Steve >> > > --=20 --------------------------------------------- Steve Severance Alpha Heavy Industries sseverance@alphaheavy.com (240) 472 - 9645 --------------------------------------------- --14dae9340e636b788a04b9fce247 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable system:java.class.path=3D/opt/hadoop/conf:/usr/lib/jvm/sun-jdk-1.6/lib/tool= s.jar:/opt/hadoop:/opt/hadoop/hadoop-core-0.20.2-cdh3u3.jar:/opt/hadoop/lib= /ant-contrib-1.0b3.jar:/opt/hadoop/lib/aspectjrt-1.6.5.jar:/opt/hadoop/lib/= aspectjtools-1.6.5.jar:/opt/hadoop/lib/commons-cli-1.2.jar:/opt/hadoop/lib/= commons-codec-1.4.jar:/opt/hadoop/lib/commons-daemon-1.0.1.jar:/opt/hadoop/= lib/commons-el-1.0.jar:/opt/hadoop/lib/commons-httpclient-3.1.jar:/opt/hado= op/lib/commons-lang-2.4.jar:/opt/hadoop/lib/commons-logging-1.0.4.jar:/opt/= hadoop/lib/commons-logging-api-1.0.4.jar:/opt/hadoop/lib/commons-net-1.4.1.= jar:/opt/hadoop/lib/core-3.1.1.jar:/opt/hadoop/lib/guava-r09-jarjar.jar:/op= t/hadoop/lib/hadoop-fairscheduler-0.20.2-cdh3u2.jar:/opt/hadoop/lib/hadoop-= fairscheduler-0.20.2-cdh3u3.jar:/opt/hadoop/lib/hsqldb-1.8.0.10.jar:/opt/ha= doop/lib/jackson-core-asl-1.5.2.jar:/opt/hadoop/lib/jackson-mapper-asl-1.5.= 2.jar:/opt/hadoop/lib/jasper-compiler-5.5.12.jar:/opt/hadoop/lib/jasper-run= time-5.5.12.jar:/opt/hadoop/lib/jets3t-0.6.1.jar:/opt/hadoop/lib/jetty-6.1.= 26.cloudera.1.jar:/opt/hadoop/lib/jetty-servlet-tester-6.1.26.cloudera.1.ja= r:/opt/hadoop/lib/jetty-util-6.1.26.cloudera.1.jar:/opt/hadoop/lib/jsch-0.1= .42.jar:/opt/hadoop/lib/junit-4.5.jar:/opt/hadoop/lib/kfs-0.2.2.jar:/opt/ha= doop/lib/log4j-1.2.15.jar:/opt/hadoop/lib/mockito-all-1.8.2.jar:/opt/hadoop= /lib/oro-2.0.8.jar:/opt/hadoop/lib/servlet-api-2.5-20081211.jar:/opt/hadoop= /lib/servlet-api-2.5-6.1.14.jar:/opt/hadoop/lib/slf4j-api-1.4.3.jar:/opt/ha= doop/lib/slf4j-log4j12-1.4.3.jar:/opt/hadoop/lib/xmlenc-0.52.jar:/opt/hadoo= p/lib/jsp-2.1/jsp-2.1.jar:/opt/hadoop/lib/jsp-2.1/jsp-api-2.1.jar::/opt/hiv= e/conf:/opt/hive/lib/ant-contrib-1.0b3.jar:/opt/hive/lib/antlr-2.7.7.jar:/o= pt/hive/lib/antlr-3.0.1.jar:/opt/hive/lib/antlr-runtime-3.0.1.jar:/opt/hive= /lib/asm-3.1.jar:/opt/hive/lib/commons-cli-1.2.jar:/opt/hive/lib/commons-co= dec-1.3.jar:/opt/hive/lib/commons-collections-3.2.1.jar:/opt/hive/lib/commo= ns-dbcp-1.4.jar:/opt/hive/lib/commons-lang-2.4.jar:/opt/hive/lib/commons-lo= gging-1.0.4.jar:/opt/hive/lib/commons-logging-api-1.0.4.jar:/opt/hive/lib/c= ommons-pool-1.5.4.jar:/opt/hive/lib/datanucleus-connectionpool-2.0.3.jar:/o= pt/hive/lib/datanucleus-core-2.0.3.jar:/opt/hive/lib/datanucleus-enhancer-2= .0.3.jar:/opt/hive/lib/datanucleus-rdbms-2.0.3.jar:/opt/hive/lib/derby-10.4= .2.0.jar:/opt/hive/lib/guava-r06.jar:/opt/hive/lib/hbase-0.89.0-SNAPSHOT-te= sts.jar:/opt/hive/lib/hbase-0.89.0-SNAPSHOT.jar:/opt/hive/lib/hive-anttasks= -0.9.0-SNAPSHOT.jar:/opt/hive/lib/hive-builtins-0.9.0-SNAPSHOT.jar:/opt/hiv= e/lib/hive-cli-0.9.0-SNAPSHOT.jar:/opt/hive/lib/hive-common-0.9.0-SNAPSHOT.= jar:/opt/hive/lib/hive-contrib-0.9.0-SNAPSHOT.jar:/opt/hive/lib/hive-exec-0= .9.0-SNAPSHOT.jar:/opt/hive/lib/hive-hbase-handler-0.9.0-SNAPSHOT.jar:/opt/= hive/lib/hive-hwi-0.9.0-SNAPSHOT.jar:/opt/hive/lib/hive-jdbc-0.9.0-SNAPSHOT= .jar:/opt/hive/lib/hive-metastore-0.9.0-SNAPSHOT.jar:/opt/hive/lib/hive-pdk= -0.9.0-SNAPSHOT.jar:/opt/hive/lib/hive-serde-0.9.0-SNAPSHOT.jar:/opt/hive/l= ib/hive-service-0.9.0-SNAPSHOT.jar:/opt/hive/lib/hive-shims-0.9.0-SNAPSHOT.= jar:/opt/hive/lib/hive_contrib.jar:/opt/hive/lib/javaewah-0.3.jar:/opt/hive= /lib/jdo2-api-2.3-ec.jar:/opt/hive/lib/jline-0.9.94.jar:/opt/hive/lib/json-= 20090211.jar:/opt/hive/lib/junit-4.10.jar:/opt/hive/lib/libfb303-0.7.0.jar:= /opt/hive/lib/libfb303.jar:/opt/hive/lib/libthrift-0.7.0.jar:/opt/hive/lib/= libthrift.jar:/opt/hive/lib/log4j-1.2.16.jar:/opt/hive/lib/mockito-all-1.8.= 2.jar:/opt/hive/lib/mysql-connector-java-5.1.18-bin.jar:/opt/hive/lib/slf4j= -api-1.6.1.jar:/opt/hive/lib/slf4j-log4j12-1.6.1.jar:/opt/hive/lib/stringte= mplate-3.1-b1.jar:/opt/hive/lib/velocity-1.5.jar:/opt/hive/lib/zookeeper-3.= 3.1.jar:

On Mon, Feb 27, 2012 at 1:17 PM, Carl Steinb= ach <carl@clouder= a.com> wrote:
Hi Steve,

It's possible that Hive is still picking u= p some old JARs on its classpath. Can you please try dumping the classpath = from within the HiveCLI and respond with the output, e.g.

% hive -e 'set system:java.class.path'

Thanks.
C= arl


On Sat, Feb 25, 2012 a= t 4:42 PM, Steve Severance <sseverance@alphaheavy.com> wrote:
Hi,

I have a job that use= INSERT INTO from Hive 0.8. The query is something like:

INSERT INTO TABLE MyTable PARTITION (dt=3D'2012-01-01) SELECT ...<= /div>

When I run this query both with and without dynami= c partitioning it overwrites the data that was already in the partition.

I am running Hive 0.8 on CDH3u2. I have confirmed that append is e= nabled in the job conf.

Is there anything else I need to do that would en= able appending to my hive table?

Thanks.

Steve




--
------------= ---------------------------------
Steve Severance
Alpha Heavy= Industries
(240) 472 - 9645
-------------------------------------------= --

--14dae9340e636b788a04b9fce247--