incubator-oozie-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From praveenesh kumar <praveen...@gmail.com>
Subject Re: Fwd: How can I configure oozie to submit different workflows from different users ?
Date Tue, 03 Apr 2012 09:23:30 GMT
Following is the oozie.log file output after starting oozie:



*******************************************************************************
  STARTUP MSG: Oozie BUILD_VERSION [3.1.3-incubating] compiled by [kamrul]
on [2012.02.29-18:52:47GMT]
  STARTUP MSG:       revision [unavailable]@[unavailable]
*******************************************************************************
2012-04-03 04:18:59,026  INFO XLogService:539 - Log4j configuration file
[oozie-log4j.properties]
2012-04-03 04:18:59,026  INFO XLogService:539 - Log4j configuration file
loaded from [/usr/local/hadoop/oozie/conf]
2012-04-03 04:18:59,027  INFO XLogService:539 - Log4j reload interval [10
sec]
2012-04-03 04:18:59,034  INFO ConfigurationService:539 - USER[-] GROUP[-]
Oozie home dir  [/usr/local/hadoop/oozie]
2012-04-03 04:18:59,034  INFO ConfigurationService:539 - USER[-] GROUP[-]
Oozie conf dir  [/usr/local/hadoop/oozie/conf]
2012-04-03 04:18:59,034  INFO ConfigurationService:539 - USER[-] GROUP[-]
Oozie conf file [oozie-site.xml]
2012-04-03 04:18:59,056  INFO ConfigurationService:539 - USER[-] GROUP[-]
Configuration change via System Property, [oozie.base.url]=[
http://lxe9700:11000/oozie]
2012-04-03 04:18:59,057  WARN Services:542 - USER[-] GROUP[-] System ID
[oozie-hado] exceeds maximum length [10], trimming
2012-04-03 04:18:59,058  INFO Services:539 - USER[-] GROUP[-] Exiting null
Entering NORMAL
2012-04-03 04:18:59,059  INFO Services:539 - USER[-] GROUP[-] Initialized
runtime directory
[/usr/local/hadoop/oozie/oozie-server/temp/oozie-hado8638076398752938513.dir]
2012-04-03 04:18:59,122  WARN AuthorizationService:542 - USER[-] GROUP[-]
Oozie running with security disabled
2012-04-03 04:18:59,122  INFO KerberosHadoopAccessorService:539 - USER[-]
GROUP[-] JOB_TRACKER_WHITELIST :[ ], Total entries :0
2012-04-03 04:18:59,122  INFO KerberosHadoopAccessorService:539 - USER[-]
GROUP[-] NAME_NODE_WHITELIST :[ ], Total entries :0
2012-04-03 04:18:59,123  INFO KerberosHadoopAccessorService:539 - USER[-]
GROUP[-] Oozie Kerberos Authentication [disabled]
2012-04-03 04:19:02,590  INFO JPAService:539 - USER[-] GROUP[-] TOKEN[-]
APP[-] JOB[-] ACTION[-] All entities initialized
2012-04-03 04:19:02,591  INFO JPAService:539 - USER[-] GROUP[-] TOKEN[-]
APP[-] JOB[-] ACTION[-] JPA configuration:
DriverClassName=org.apache.derby.jdbc.EmbeddedDriver,Url=jdbc:derby:/usr/local/hadoop/oozie/data/oozie-db;create=true,Username=sa,Password=,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false
2012-04-03 04:19:02,633  INFO HadoopAccessorService:539 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] JOB_TRACKER_WHITELIST :[ ], Total entries
:0
2012-04-03 04:19:02,634  INFO HadoopAccessorService:539 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] NAME_NODE_WHITELIST :[ ], Total entries :0
2012-04-03 04:19:02,638  INFO Services:539 - Initialized
2012-04-03 04:19:02,641  INFO Services:539 - Running with JARs for Hadoop
version [0.20.205.0]
2012-04-03 04:19:02,641  INFO Services:539 - Oozie System ID [oozie-hado]
started!
2012-04-03 04:19:12,645  INFO
StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired
lock for [org.apache.oozie.service.StatusTransitService]
2012-04-03 04:19:12,645  INFO
StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running
coordinator status service first instance
2012-04-03 04:19:12,653 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-]
APP[-] JOB[-] ACTION[-] Load state for [null]
2012-04-03 04:19:12,663 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] Load state for [null]
2012-04-03 04:19:12,664  INFO PauseTransitService:539 - USER[-] GROUP[-]
Acquired lock for [org.apache.oozie.service.PauseTransitService]
2012-04-03 04:19:12,665 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] Load state for [null]
2012-04-03 04:19:12,882 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-]
APP[-] JOB[-] ACTION[-] Precondition check for command [purge] key [null]
2012-04-03 04:19:12,882 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-]
APP[-] JOB[-] ACTION[-] Execute command [purge] key [null]
2012-04-03 04:19:12,882 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-]
APP[-] JOB[-] ACTION[-] STARTED Workflow-Purge Attempting to purge Jobs
older than [30] days.
2012-04-03 04:19:12,883 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-]
APP[-] JOB[-] ACTION[-] ENDED Workflow-Purge no workflow job to be deleted
2012-04-03 04:19:12,887 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] Precondition check for command
[bundle_purge] key [null]
2012-04-03 04:19:12,888 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [bundle_purge] key [null]
2012-04-03 04:19:12,888 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Bundle-Purge to purge Jobs older
than [7] days.
2012-04-03 04:19:12,888 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Bundle-Purge no Bundle job to be
deleted
2012-04-03 04:19:12,891  INFO
StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running
bundle status service first instance
2012-04-03 04:19:12,891 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] Precondition check for command
[coord_purge] key [null]
2012-04-03 04:19:12,892 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [coord_purge] key [null]
2012-04-03 04:19:12,892 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Coord-Purge to purge Jobs older
than [7] days.
2012-04-03 04:19:12,892 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Coord-Purge no Coord job to be
deleted
2012-04-03 04:19:12,902 DEBUG
CoordMaterializeTriggerService$CoordMaterializeTriggerRunnable:545 -
USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-]
CoordMaterializeTriggerService - Curr Date= Tue Apr 03 04:24:12 CDT 2012,
Num jobs to materialize = 0
2012-04-03 04:19:12,914 DEBUG ActionCheckerService$ActionCheckRunnable:545
- USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [] for
potential checking
2012-04-03 04:19:12,933  INFO
StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released
lock for [org.apache.oozie.service.StatusTransitService]
2012-04-03 04:19:12,949  INFO PauseTransitService:539 - USER[-] GROUP[-]
Released lock for [org.apache.oozie.service.PauseTransitService]
2012-04-03 04:19:12,990 DEBUG RecoveryService$RecoveryRunnable:545 -
USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [ WF_ACTIONS 0,
COORD_ACTIONS : 0, COORD_READY_JOBS : 0, BUNDLE_ACTIONS : 0] for potential
recovery


I am newbie in oozie. Can you please help me out in resolving this issue ?

Thanks,
Praveenesh

On Tue, Apr 3, 2012 at 2:32 PM, Mohammad Islam <mislam77@yahoo.com> wrote:

> Hi,
> Do you have kerberos on for hadoop?
>
> Can you please check what service
> (HadoopAccessorService/KerberosHadoopAccessorService) is actually active?
>
> You can find from the oozie.log when the oozie server was instantiated.
> Search for the above two service class name.
>
> Regards,
> Mohammad
>
>
>
>
> ----- Original Message -----
> From: praveenesh kumar <praveenesh@gmail.com>
> To: oozie-users@incubator.apache.org
> Cc:
> Sent: Monday, April 2, 2012 6:19 AM
> Subject: Fwd: How can I configure oozie to submit different workflows from
> different users ?
>
> ---------- Forwarded message ----------
> From: praveenesh kumar <praveenesh@gmail.com>
> Date: Mon, Apr 2, 2012 at 5:45 PM
> Subject: How can I configure oozie to submit different workflows from
> different users ?
> To: common-user@hadoop.apache.org
>
>
> Hi all,
>
> I want to use oozie to submit different workflows from different users.
> These users are able to submit hadoop jobs.
> I am using hadoop 0.20.205 and oozie 3.1.3
> I have a hadoop user as a oozie-user
>
> I have set the following things :
>
> conf/oozie-site.xml :
>
> < property >
> < name >oozie.services.ext< /name >
> < value >org.apache.oozie.service.HadoopAccessorService
> < /value >
> < description >
> To add/replace services defined in 'oozie.services' with custom
> implementations.Class names must be separated by commas.
> < /description >
> < /property >
>
> conf/core-site.xml
> < property>
> < name>hadoop.proxyuser.hadoop.hosts< /name>
> < value>*< / value>
> < /property>
> < property>
> < name>hadoop.proxyuser.hadoop.groups< /name>
> < value>*< /value>
> < /property>
>
> When I am submitting jobs as a hadoop user, I am able to run it properly.
> But when I am able to submit the same work flow  from a different user, who
> can submit the simple MR jobs to my hadoop cluster, I am getting the
> following error:
>
> JA009: java.io.IOException: java.io.IOException: The username kumar
> obtained from the conf doesn't match the username hadoop the user
> authenticated as    at
> org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3943)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>
> Caused by: java.io.IOException: The username kumar obtained from the conf
> doesn't match the username hadoop the user authenticated as
> at org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:426)
> at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3941)
> ... 11 more
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message