mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sameer Tilak <ssti...@live.com>
Subject Pig local mode issue
Date Thu, 23 Jan 2014 00:24:39 GMT
Hi All,My script runs find in map reduce mode, but I get the following error when I run it
in the local mode. I have made sure that the i/p file exists. I am not sure why map reduce
is coming into picture when it is local mode.
pig -x local myscript.pig

2014-01-22 16:14:02,771 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler
- File concatenation threshold: 100 optimistic? false2014-01-22 16:14:02,805 [main] INFO 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan
size before optimization: 32014-01-22 16:14:02,806 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- Merged 2 map-only splittees.2014-01-22 16:14:02,806 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- Merged 2 out of total 3 MR operators.2014-01-22 16:14:02,806 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size after optimization: 12014-01-22 16:14:02,845 [main] INFO  org.apache.pig.tools.pigstats.ScriptState
- Pig script settings are added to the job2014-01-22 16:14:02,865 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.32014-01-22 16:14:02,876
[main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator2014-01-22
16:14:02,878 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
- BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=99408652014-01-22 16:14:02,878
[main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Setting Parallelism to 12014-01-22 16:14:02,909 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Setting up multi store job2014-01-22 16:14:02,918 [main] INFO  org.apache.pig.data.SchemaTupleFrontend
- Key [pig.schematuple] is false, will not generate code.2014-01-22 16:14:02,918 [main] INFO
 org.apache.pig.data.SchemaTupleFrontend - Starting process to move generated code to distributed
cacche2014-01-22 16:14:02,918 [main] INFO  org.apache.pig.data.SchemaTupleFrontend - Distributed
cache not supported or needed in local mode. Setting key [pig.schematuple.local.dir] with
code temp directory: /tmp/1390436042918-02014-01-22 16:14:02,978 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 1 map-reduce job(s) waiting for submission.2014-01-22 16:14:02,991 [JobControl] INFO  org.apache.hadoop.util.NativeCodeLoader
- Loaded the native-hadoop library2014-01-22 16:14:02,994 [JobControl] ERROR org.apache.hadoop.security.UserGroupInformation
- PriviledgedActionException as:userid cause:ENOENT: No such file or directory2014-01-22 16:14:03,479
[main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 0% complete2014-01-22 16:14:03,489 [main] WARN  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately
on failure.2014-01-22 16:14:03,489 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- job null has failed! Stop running all dependent jobs2014-01-22 16:14:03,490 [main] INFO
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete2014-01-22
16:14:03,492 [main] ERROR org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable
to recreate exception from backend error: ENOENT: No such file or directory	at org.apache.hadoop.io.nativeio.NativeIO.chmod(Native
Method)	at org.apache.hadoop.fs.FileUtil.execSetPermission(FileUtil.java:699)	at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:654)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)	at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:856)	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
at java.security.AccessController.doPrivileged(Native Method)	at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)	at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)	at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:824)
at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)	at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)	at
java.lang.reflect.Method.invoke(Method.java:601)	at org.apache.pig.backend.hadoop20.PigJobControl.mainLoopAction(PigJobControl.java:157)
at org.apache.pig.backend.hadoop20.PigJobControl.run(PigJobControl.java:134)	at java.lang.Thread.run(Thread.java:722)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:257)
2014-01-22 16:14:03,493 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce
job(s) failed!2014-01-22 16:14:03,493 [main] INFO  org.apache.pig.tools.pigstats.SimplePigStats
- Detected Local mode. Stats reported below may be incomplete2014-01-22 16:14:03,495 [main]
INFO  org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics: 		 	   		  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message