From hcatalog-commits-return-1070-apmail-incubator-hcatalog-commits-archive=incubator.apache.org@incubator.apache.org Thu Sep 20 18:57:38 2012 Return-Path: X-Original-To: apmail-incubator-hcatalog-commits-archive@minotaur.apache.org Delivered-To: apmail-incubator-hcatalog-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A61C5D85E for ; Thu, 20 Sep 2012 18:57:38 +0000 (UTC) Received: (qmail 10537 invoked by uid 500); 20 Sep 2012 18:57:38 -0000 Delivered-To: apmail-incubator-hcatalog-commits-archive@incubator.apache.org Received: (qmail 10484 invoked by uid 500); 20 Sep 2012 18:57:38 -0000 Mailing-List: contact hcatalog-commits-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hcatalog-dev@incubator.apache.org Delivered-To: mailing list hcatalog-commits@incubator.apache.org Received: (qmail 10473 invoked by uid 99); 20 Sep 2012 18:57:38 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 20 Sep 2012 18:57:38 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 20 Sep 2012 18:57:36 +0000 Received: from eris.apache.org (localhost [127.0.0.1]) by eris.apache.org (Postfix) with ESMTP id C4369238899C; Thu, 20 Sep 2012 18:56:53 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1388158 - in /incubator/hcatalog/trunk: CHANGES.txt src/test/e2e/hcatalog/deployers/HCatExistingClusterDeployer.pm src/test/e2e/hcatalog/tools/generate/java/build.xml Date: Thu, 20 Sep 2012 18:56:53 -0000 To: hcatalog-commits@incubator.apache.org From: toffer@apache.org X-Mailer: svnmailer-1.0.8-patched Message-Id: <20120920185653.C4369238899C@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: toffer Date: Thu Sep 20 18:56:53 2012 New Revision: 1388158 URL: http://svn.apache.org/viewvc?rev=1388158&view=rev Log: HCAT-507 e2e harness failing on 0.23 (toffer) Modified: incubator/hcatalog/trunk/CHANGES.txt incubator/hcatalog/trunk/src/test/e2e/hcatalog/deployers/HCatExistingClusterDeployer.pm incubator/hcatalog/trunk/src/test/e2e/hcatalog/tools/generate/java/build.xml Modified: incubator/hcatalog/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/incubator/hcatalog/trunk/CHANGES.txt?rev=1388158&r1=1388157&r2=1388158&view=diff ============================================================================== --- incubator/hcatalog/trunk/CHANGES.txt (original) +++ incubator/hcatalog/trunk/CHANGES.txt Thu Sep 20 18:56:53 2012 @@ -111,6 +111,8 @@ Trunk (unreleased changes) OPTIMIZATIONS BUG FIXES + HCAT-507 e2e harness failing on 0.23 (toffer) + HCAT-502 HCatalog dependency resolution fails due to missing repo (traviscrawford) HCAT-490 HCatStorer() throws error when the same partition key is present in records in more than one tasks running as part of the same job (amalakar via traviscrawford) Modified: incubator/hcatalog/trunk/src/test/e2e/hcatalog/deployers/HCatExistingClusterDeployer.pm URL: http://svn.apache.org/viewvc/incubator/hcatalog/trunk/src/test/e2e/hcatalog/deployers/HCatExistingClusterDeployer.pm?rev=1388158&r1=1388157&r2=1388158&view=diff ============================================================================== --- incubator/hcatalog/trunk/src/test/e2e/hcatalog/deployers/HCatExistingClusterDeployer.pm (original) +++ incubator/hcatalog/trunk/src/test/e2e/hcatalog/deployers/HCatExistingClusterDeployer.pm Thu Sep 20 18:56:53 2012 @@ -184,7 +184,11 @@ sub generateData # } # Create the HDFS directories - Util::runHadoopCmd($cfg, $log, "fs -mkdir $cfg->{'hcat_data_dir'}"); + my $mkdirCmd = "fs -mkdir"; + if ($ENV{'HCAT_HADOOPVERSION'} eq "23") { + $mkdirCmd = "fs -mkdir -p" + } + Util::runHadoopCmd($cfg, $log, "$mkdirCmd $cfg->{'hcat_data_dir'}"); foreach my $table (@tables) { print "Generating data for $table->{'name'}\n"; @@ -200,12 +204,12 @@ sub generateData $self->runCmd($log, \@cmd); # Copy the data to HDFS - my $hadoop = "fs -mkdir $cfg->{'hcat_data_dir'}/$table->{'hdfs'}"; + my $hadoop = "$mkdirCmd $cfg->{'hcat_data_dir'}/$table->{'hdfs'}"; Util::runHadoopCmd($cfg, $log, $hadoop); if (defined($table->{'partitions'})) { foreach my $part (@{$table->{'partitions'}}) { - my $hadoop = "fs -mkdir + my $hadoop = "$mkdirCmd $cfg->{'hcat_data_dir'}/$table->{'hdfs'}/$table->{'name'}.$part"; Util::runHadoopCmd($cfg, $log, $hadoop); my $hadoop = "fs -copyFromLocal $table->{'name'}.$part " . Modified: incubator/hcatalog/trunk/src/test/e2e/hcatalog/tools/generate/java/build.xml URL: http://svn.apache.org/viewvc/incubator/hcatalog/trunk/src/test/e2e/hcatalog/tools/generate/java/build.xml?rev=1388158&r1=1388157&r2=1388158&view=diff ============================================================================== --- incubator/hcatalog/trunk/src/test/e2e/hcatalog/tools/generate/java/build.xml (original) +++ incubator/hcatalog/trunk/src/test/e2e/hcatalog/tools/generate/java/build.xml Thu Sep 20 18:56:53 2012 @@ -23,7 +23,7 @@ - +