Return-Path: 
 <user-return-3976-apmail-hadoop-hdfs-user-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 16655E997
	for <apmail-hadoop-hdfs-user-archive@minotaur.apache.org>;
 Sat,  5 Jan 2013 07:46:17 +0000 (UTC)
Received: (qmail 39173 invoked by uid 500); 5 Jan 2013 07:46:12 -0000
Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org
Received: (qmail 38962 invoked by uid 500); 5 Jan 2013 07:46:12 -0000
Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:user@hadoop.apache.org>
List-Id: <user.hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Delivered-To: mailing list user@hadoop.apache.org
Received: (qmail 38952 invoked by uid 99); 5 Jan 2013 07:46:11 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 05 Jan 2013 07:46:11 +0000
X-ASF-Spam-Status: No, hits=1.5 required=5.0
	tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: domain of harsh@cloudera.com designates
 209.85.210.170 as permitted sender)
Received: from [209.85.210.170] (HELO mail-ia0-f170.google.com)
 (209.85.210.170)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 05 Jan 2013 07:46:06 +0000
Received: by mail-ia0-f170.google.com with SMTP id i1so14559715iaa.1
        for <user@hadoop.apache.org>; Fri, 04 Jan 2013 23:45:46 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20120113;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type:x-gm-message-state;
        bh=HNc/FCRVauwEwGNBzm0IntqvvbP2WlcmNdrmArjNpMM=;
        b=BToREn1jySskbV7/Vs7qUDhZXc6JQRPoZB/WBKw6WfU4m58eI+ynRsAe1rOdHCs1DA
         gvhBPciQeZAGUZrOxTMKAKq9uH7nIpdyKrrbGkxmUQHiH8q1IpcimgUtyLpz6nl3p+Kb
         Ummz4Q+qC600Ah2DVJp4xPyINBNlon+0DXBlo+nytz3tSHIiK79mP0qkbl21ucFaY7eR
         O4CrygIdD8Ynr3MmoUHOtRmmPzaw3UUGUymQG3px2p0gU3mmWMKGCDLlZhGj1VzqM3wQ
         7FFH9i3eXv+dwH5dw/naa79bSU4HfssDfmVy6siZtI/objx95MCJVACcmyrHkFSA3ECz
         1o5Q==
Received: by 10.42.249.80 with SMTP id mj16mr41529875icb.53.1357371946386;
 Fri, 04 Jan 2013 23:45:46 -0800 (PST)
MIME-Version: 1.0
Received: by 10.64.32.166 with HTTP; Fri, 4 Jan 2013 23:45:26 -0800 (PST)
In-Reply-To: <447A0738-F1C6-4735-B378-744B5D6D9465@datastax.com>
References: <447A0738-F1C6-4735-B378-744B5D6D9465@datastax.com>
From: Harsh J <harsh@cloudera.com>
Date: Sat, 5 Jan 2013 13:15:26 +0530
Message-ID: 
 <CAOcnVr2sitRS6zfqg+8Yi+j8q9-mDYZuZK9jhJYHC_Wh=B5gdg@mail.gmail.com>
Subject: Re: Gridmix version 1.0.4 Error
To: "<user@hadoop.apache.org>" <user@hadoop.apache.org>
Content-Type: multipart/alternative; boundary=20cf3011d8b1b1458a04d285c825
X-Gm-Message-State: 
 ALoCoQm4gJ4hcmb8Ywu9wnqJWtQ88ghI3EPkY3FH7EyPc67mTgC/2zM6cwTsf7hAa1F8FC3hJSE0
X-Virus-Checked: Checked by ClamAV on apache.org

--20cf3011d8b1b1458a04d285c825
Content-Type: text/plain; charset=ISO-8859-1

Hi Sean,

Two questions: Why are you running this in local mode? Placing a cluster's
config directory on your java -cp will make it go distributed. And, does
that reported output directory really exist? If so, you may want to delete
it before you run GridMix.


On Sat, Jan 5, 2013 at 3:55 AM, Sean Barry <sean@datastax.com> wrote:

> Hi,
>
> I am trying to use grid mix but I keep getting the error that is shown
> below. Does anyone have some suggestions.
>
> Thanks in advance.
>
> Sean Barry
>
> hostname:gridmix seanbarry$ pwd
> /usr/local/hadoop-1.0.4/contrib/gridmix
> hostname:gridmix seanbarry$ java -cp
> /usr/local/hadoop-1.0.4/contrib/gridmix/hadoop-gridmix-1.0.4.jar:/usr/local/hadoop-1.0.4/*:/usr/local/hadoop-1.0.4/lib/*
> org.apache.hadoop.mapred.gridmix.Gridmix -generate 10m
> hdfs://hostname:54310/user/seanbarry/bench
> file:///Users/seanbarry/sfbarry/job-trace.json
> 2013-01-04 14:19:15.759 java[2416:1903] Unable to load realm info from
> SCDynamicStore
> 13/01/04 14:19:15 INFO gridmix.SubmitterUserResolver:  Current user
> resolver is SubmitterUserResolver
> 13/01/04 14:19:15 WARN gridmix.Gridmix: Resource null ignored
> 13/01/04 14:19:16 INFO gridmix.Gridmix:  Submission policy is STRESS
> 13/01/04 14:19:16 INFO gridmix.Gridmix: Generating 10.0m of test data...
> 13/01/04 14:19:16 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 13/01/04 14:19:16 INFO mapred.JobClient: Cleaning up the staging area
> file:/tmp/hadoop-seanbarry/mapred/staging/seanbarry-957366090/.staging/job_local_0001
> 13/01/04 14:19:16 ERROR security.UserGroupInformation:
> PriviledgedActionException as:seanbarry
> cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
> hdfs://hostname:54310/user/seanbarry/bench already exists
> 13/01/04 14:19:16 ERROR security.UserGroupInformation:
> PriviledgedActionException as:seanbarry
> cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
> hdfs://hostname:54310/user/seanbarry/bench already exists
> 13/01/04 14:19:16 WARN gridmix.JobSubmitter: Failed to submit
> GRIDMIX_GENDATA as seanbarry
> org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
> hdfs://hostname:54310/user/seanbarry/bench already exists
>         at
> org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:137)
>         at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:887)
>         at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>         at
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
>         at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
>         at
> org.apache.hadoop.mapred.gridmix.GenerateData$1.run(GenerateData.java:116)
>         at
> org.apache.hadoop.mapred.gridmix.GenerateData$1.run(GenerateData.java:101)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>         at
> org.apache.hadoop.mapred.gridmix.GenerateData.call(GenerateData.java:101)
>         at
> org.apache.hadoop.mapred.gridmix.GenerateData.call(GenerateData.java:57)
>         at
> org.apache.hadoop.mapred.gridmix.JobSubmitter$SubmitTask.run(JobSubmitter.java:106)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:680)
> 13/01/04 14:19:16 INFO gridmix.JobMonitor:  Job submission failed notify
> if anyone is waiting org.apache.hadoop.mapreduce.Job@79884a40
> 13/01/04 14:19:26 INFO mapred.JobClient: Cleaning up the staging area
> file:/tmp/hadoop-seanbarry/mapred/staging/seanbarry15872696/.staging/job_local_0002
> 13/01/04 14:19:26 ERROR security.UserGroupInformation:
> PriviledgedActionException as:seanbarry
> cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
> hdfs://hostname:54310/user/seanbarry/bench already exists
> 13/01/04 14:19:26 ERROR gridmix.Gridmix: Startup failed
> org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
> hdfs://hostname:54310/user/seanbarry/bench already exists
>         at
> org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:137)
>         at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:887)
>         at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>         at
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
>         at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
>         at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
>         at
> org.apache.hadoop.mapred.gridmix.Gridmix.writeInputData(Gridmix.java:118)
>         at org.apache.hadoop.mapred.gridmix.Gridmix.start(Gridmix.java:283)
>         at
> org.apache.hadoop.mapred.gridmix.Gridmix.runJob(Gridmix.java:263)
>         at
> org.apache.hadoop.mapred.gridmix.Gridmix.access$000(Gridmix.java:55)
>         at org.apache.hadoop.mapred.gridmix.Gridmix$1.run(Gridmix.java:217)
>         at org.apache.hadoop.mapred.gridmix.Gridmix$1.run(Gridmix.java:215)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>         at org.apache.hadoop.mapred.gridmix.Gridmix.run(Gridmix.java:215)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at org.apache.hadoop.mapred.gridmix.Gridmix.main(Gridmix.java:390)
> 13/01/04 14:19:26 INFO gridmix.Gridmix: Exiting...
> hostname:gridmix seanbarry$
>
>


-- 
Harsh J

--20cf3011d8b1b1458a04d285c825
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi Sean,<div><br></div><div>Two questions: Why are you running this in loca=
l mode? Placing a cluster&#39;s config directory on your java -cp will make=
 it go distributed. And, does that reported output directory really exist? =
If so, you may want to delete it before you run GridMix.</div>

<div class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On Sat, Jan 5=
, 2013 at 3:55 AM, Sean Barry <span dir=3D"ltr">&lt;<a href=3D"mailto:sean@=
datastax.com" target=3D"_blank">sean@datastax.com</a>&gt;</span> wrote:<br>=
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">

Hi,<br>
<br>
I am trying to use grid mix but I keep getting the error that is shown belo=
w. Does anyone have some suggestions.<br>
<br>
Thanks in advance.<br>
<br>
Sean Barry<br>
<br>
hostname:gridmix seanbarry$ pwd<br>
/usr/local/hadoop-1.0.4/contrib/gridmix<br>
hostname:gridmix seanbarry$ java -cp /usr/local/hadoop-1.0.4/contrib/gridmi=
x/hadoop-gridmix-1.0.4.jar:/usr/local/hadoop-1.0.4/*:/usr/local/hadoop-1.0.=
4/lib/* org.apache.hadoop.mapred.gridmix.Gridmix -generate 10m hdfs://hostn=
ame:54310/user/seanbarry/bench file:///Users/seanbarry/sfbarry/job-trace.js=
on<br>


2013-01-04 14:19:15.759 java[2416:1903] Unable to load realm info from SCDy=
namicStore<br>
13/01/04 14:19:15 INFO gridmix.SubmitterUserResolver: =A0Current user resol=
ver is SubmitterUserResolver<br>
13/01/04 14:19:15 WARN gridmix.Gridmix: Resource null ignored<br>
13/01/04 14:19:16 INFO gridmix.Gridmix: =A0Submission policy is STRESS<br>
13/01/04 14:19:16 INFO gridmix.Gridmix: Generating 10.0m of test data...<br=
>
13/01/04 14:19:16 WARN util.NativeCodeLoader: Unable to load native-hadoop =
library for your platform... using builtin-java classes where applicable<br=
>
13/01/04 14:19:16 INFO mapred.JobClient: Cleaning up the staging area file:=
/tmp/hadoop-seanbarry/mapred/staging/seanbarry-957366090/.staging/job_local=
_0001<br>
13/01/04 14:19:16 ERROR security.UserGroupInformation: PriviledgedActionExc=
eption as:seanbarry cause:org.apache.hadoop.mapred.FileAlreadyExistsExcepti=
on: Output directory hdfs://hostname:54310/user/seanbarry/bench already exi=
sts<br>


13/01/04 14:19:16 ERROR security.UserGroupInformation: PriviledgedActionExc=
eption as:seanbarry cause:org.apache.hadoop.mapred.FileAlreadyExistsExcepti=
on: Output directory hdfs://hostname:54310/user/seanbarry/bench already exi=
sts<br>


13/01/04 14:19:16 WARN gridmix.JobSubmitter: Failed to submit GRIDMIX_GENDA=
TA as seanbarry<br>
org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs:=
//hostname:54310/user/seanbarry/bench already exists<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.=
checkOutputSpecs(FileOutputFormat.java:137)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:=
887)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:=
850)<br>
=A0 =A0 =A0 =A0 at java.security.AccessController.doPrivileged(Native Metho=
d)<br>
=A0 =A0 =A0 =A0 at javax.security.auth.Subject.doAs(Subject.java:396)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.security.UserGroupInformation.doAs(Use=
rGroupInformation.java:1121)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.JobClient.submitJobInternal(Job=
Client.java:850)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.gridmix.GenerateData$1.run(Gene=
rateData.java:116)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.gridmix.GenerateData$1.run(Gene=
rateData.java:101)<br>
=A0 =A0 =A0 =A0 at java.security.AccessController.doPrivileged(Native Metho=
d)<br>
=A0 =A0 =A0 =A0 at javax.security.auth.Subject.doAs(Subject.java:396)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.security.UserGroupInformation.doAs(Use=
rGroupInformation.java:1121)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.gridmix.GenerateData.call(Gener=
ateData.java:101)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.gridmix.GenerateData.call(Gener=
ateData.java:57)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.gridmix.JobSubmitter$SubmitTask=
.run(JobSubmitter.java:106)<br>
=A0 =A0 =A0 =A0 at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(T=
hreadPoolExecutor.java:886)<br>
=A0 =A0 =A0 =A0 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Threa=
dPoolExecutor.java:908)<br>
=A0 =A0 =A0 =A0 at java.lang.Thread.run(Thread.java:680)<br>
13/01/04 14:19:16 INFO gridmix.JobMonitor: =A0Job submission failed notify =
if anyone is waiting org.apache.hadoop.mapreduce.Job@79884a40<br>
13/01/04 14:19:26 INFO mapred.JobClient: Cleaning up the staging area file:=
/tmp/hadoop-seanbarry/mapred/staging/seanbarry15872696/.staging/job_local_0=
002<br>
13/01/04 14:19:26 ERROR security.UserGroupInformation: PriviledgedActionExc=
eption as:seanbarry cause:org.apache.hadoop.mapred.FileAlreadyExistsExcepti=
on: Output directory hdfs://hostname:54310/user/seanbarry/bench already exi=
sts<br>


13/01/04 14:19:26 ERROR gridmix.Gridmix: Startup failed<br>
org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs:=
//hostname:54310/user/seanbarry/bench already exists<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.=
checkOutputSpecs(FileOutputFormat.java:137)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:=
887)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:=
850)<br>
=A0 =A0 =A0 =A0 at java.security.AccessController.doPrivileged(Native Metho=
d)<br>
=A0 =A0 =A0 =A0 at javax.security.auth.Subject.doAs(Subject.java:396)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.security.UserGroupInformation.doAs(Use=
rGroupInformation.java:1121)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.JobClient.submitJobInternal(Job=
Client.java:850)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.ja=
va:530)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.gridmix.Gridmix.writeInputData(=
Gridmix.java:118)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.gridmix.Gridmix.start(Gridmix.j=
ava:283)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.gridmix.Gridmix.runJob(Gridmix.=
java:263)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.gridmix.Gridmix.access$000(Grid=
mix.java:55)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.gridmix.Gridmix$1.run(Gridmix.j=
ava:217)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.gridmix.Gridmix$1.run(Gridmix.j=
ava:215)<br>
=A0 =A0 =A0 =A0 at java.security.AccessController.doPrivileged(Native Metho=
d)<br>
=A0 =A0 =A0 =A0 at javax.security.auth.Subject.doAs(Subject.java:396)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.security.UserGroupInformation.doAs(Use=
rGroupInformation.java:1121)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.gridmix.Gridmix.run(Gridmix.jav=
a:215)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65=
)<br>
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.gridmix.Gridmix.main(Gridmix.ja=
va:390)<br>
13/01/04 14:19:26 INFO gridmix.Gridmix: Exiting...<br>
hostname:gridmix seanbarry$<br>
<br>
</blockquote></div><br><br clear=3D"all"><div><br></div>-- <br>Harsh J
</div>

--20cf3011d8b1b1458a04d285c825--