incubator-oozie-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohammad Islam <misla...@yahoo.com>
Subject Re: Fwd: How can I configure oozie to submit different workflows from different users ?
Date Tue, 03 Apr 2012 18:49:20 GMT
Hi Praveenesh,
Sorry for the inconvenience.
Looks like your service ext to use HadoopAccessorService is not effective. I knew there was
an issue with this setting and it was resolved.

Anyway, you could try the following a workaround:
Copy/Paste oozie.services property tag set from oozie-default.xml (could be found in webapps/oozie/docs)
to oozie-site.xml. Replace org.apache.oozie.service.KerberosHadoopAccessorService  with  HadoopAccessorService.
Restart oozie. Make sure there is no KerberosHadoopAccessorService in oozie log.
Please let us know how it goes.

Regards,
Mohammad


________________________________
From: praveenesh kumar <praveenesh@gmail.com>
To: oozie-users@incubator.apache.org; Mohammad Islam <mislam77@yahoo.com> 
Sent: Tuesday, April 3, 2012 2:23 AM
Subject: Re: Fwd: How can I configure oozie to submit different workflows from different users
?


Following is the oozie.log file output after starting oozie:


 *******************************************************************************
  STARTUP MSG: Oozie BUILD_VERSION [3.1.3-incubating] compiled by [kamrul] on [2012.02.29-18:52:47GMT]
  STARTUP MSG:       revision [unavailable]@[unavailable]
*******************************************************************************
2012-04-03 04:18:59,026  INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties]
2012-04-03 04:18:59,026  INFO XLogService:539 - Log4j configuration file loaded from [/usr/local/hadoop/oozie/conf]
2012-04-03 04:18:59,027  INFO XLogService:539 - Log4j reload interval [10 sec]
2012-04-03 04:18:59,034  INFO ConfigurationService:539 - USER[-] GROUP[-] Oozie home dir 
[/usr/local/hadoop/oozie]
2012-04-03 04:18:59,034  INFO ConfigurationService:539 - USER[-] GROUP[-] Oozie conf dir 
[/usr/local/hadoop/oozie/conf]
2012-04-03 04:18:59,034  INFO ConfigurationService:539 - USER[-] GROUP[-] Oozie conf file
[oozie-site.xml]
2012-04-03 04:18:59,056  INFO ConfigurationService:539 - USER[-] GROUP[-] Configuration change
via System Property, [oozie.base.url]=[http://lxe9700:11000/oozie]
2012-04-03 04:18:59,057  WARN Services:542 - USER[-] GROUP[-] System ID [oozie-hado] exceeds
maximum length [10], trimming
2012-04-03 04:18:59,058  INFO Services:539 - USER[-] GROUP[-] Exiting null Entering NORMAL
2012-04-03 04:18:59,059  INFO Services:539 - USER[-] GROUP[-] Initialized runtime directory
[/usr/local/hadoop/oozie/oozie-server/temp/oozie-hado8638076398752938513.dir]
2012-04-03 04:18:59,122  WARN AuthorizationService:542 - USER[-] GROUP[-] Oozie running with
security disabled
2012-04-03 04:18:59,122  INFO KerberosHadoopAccessorService:539 - USER[-] GROUP[-] JOB_TRACKER_WHITELIST
:[ ], Total entries :0
2012-04-03 04:18:59,122  INFO KerberosHadoopAccessorService:539 - USER[-] GROUP[-] NAME_NODE_WHITELIST
:[ ], Total entries :0
2012-04-03 04:18:59,123  INFO KerberosHadoopAccessorService:539 - USER[-] GROUP[-] Oozie
Kerberos Authentication [disabled]
2012-04-03 04:19:02,590  INFO JPAService:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-]
All entities initialized
2012-04-03 04:19:02,591  INFO JPAService:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-]
JPA configuration: DriverClassName=org.apache.derby.jdbc.EmbeddedDriver,Url=jdbc:derby:/usr/local/hadoop/oozie/data/oozie-db;create=true,Username=sa,Password=,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false
2012-04-03 04:19:02,633  INFO HadoopAccessorService:539 - USER[-] GROUP[-] TOKEN[-] APP[-]
JOB[-] ACTION[-] JOB_TRACKER_WHITELIST :[ ], Total entries :0
2012-04-03 04:19:02,634  INFO HadoopAccessorService:539 - USER[-] GROUP[-] TOKEN[-] APP[-]
JOB[-] ACTION[-] NAME_NODE_WHITELIST :[ ], Total entries :0
2012-04-03 04:19:02,638  INFO Services:539 - Initialized
2012-04-03 04:19:02,641  INFO Services:539 - Running with JARs for Hadoop version [0.20.205.0]
2012-04-03 04:19:02,641  INFO Services:539 - Oozie System ID [oozie-hado] started!
2012-04-03 04:19:12,645  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-]
Acquired lock for [org.apache.oozie.service.StatusTransitService]
2012-04-03 04:19:12,645  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-]
Running coordinator status service first instance
2012-04-03 04:19:12,653 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
ACTION[-] Load state for [null]
2012-04-03 04:19:12,663 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
ACTION[-] Load state for [null]
2012-04-03 04:19:12,664  INFO PauseTransitService:539 - USER[-] GROUP[-] Acquired lock for
[org.apache.oozie.service.PauseTransitService]
2012-04-03 04:19:12,665 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
ACTION[-] Load state for [null]
2012-04-03 04:19:12,882 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
ACTION[-] Precondition check for command [purge] key [null]
2012-04-03 04:19:12,882 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
ACTION[-] Execute command [purge] key [null]
2012-04-03 04:19:12,882 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
ACTION[-] STARTED Workflow-Purge Attempting to purge Jobs older than [30] days.
2012-04-03 04:19:12,883 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
ACTION[-] ENDED Workflow-Purge no workflow job to be deleted
2012-04-03 04:19:12,887 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
ACTION[-] Precondition check for command [bundle_purge] key [null]
2012-04-03 04:19:12,888 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
ACTION[-] Execute command [bundle_purge] key [null]
2012-04-03 04:19:12,888 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
ACTION[-] STARTED Bundle-Purge to purge Jobs older than [7] days.
2012-04-03 04:19:12,888 DEBUG BundlePurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
ACTION[-] ENDED Bundle-Purge no Bundle job to be deleted
2012-04-03 04:19:12,891  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-]
Running bundle status service first instance
2012-04-03 04:19:12,891 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
ACTION[-] Precondition check for command [coord_purge] key [null]
2012-04-03 04:19:12,892 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
ACTION[-] Execute command [coord_purge] key [null]
2012-04-03 04:19:12,892 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
ACTION[-] STARTED Coord-Purge to purge Jobs older than [7] days.
2012-04-03 04:19:12,892 DEBUG CoordPurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
ACTION[-] ENDED Coord-Purge no Coord job to be deleted
2012-04-03 04:19:12,902 DEBUG CoordMaterializeTriggerService$CoordMaterializeTriggerRunnable:545
- USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] CoordMaterializeTriggerService - Curr
Date= Tue Apr 03 04:24:12 CDT 2012, Num jobs to materialize = 0
2012-04-03 04:19:12,914 DEBUG ActionCheckerService$ActionCheckRunnable:545 - USER[-] GROUP[-]
TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [] for potential checking
2012-04-03 04:19:12,933  INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-]
Released lock for [org.apache.oozie.service.StatusTransitService]
2012-04-03 04:19:12,949  INFO PauseTransitService:539 - USER[-] GROUP[-] Released lock for
[org.apache.oozie.service.PauseTransitService]
2012-04-03 04:19:12,990 DEBUG RecoveryService$RecoveryRunnable:545 - USER[-] GROUP[-] TOKEN[-]
APP[-] JOB[-] ACTION[-] QUEUING [ WF_ACTIONS 0, COORD_ACTIONS : 0, COORD_READY_JOBS : 0, BUNDLE_ACTIONS
: 0] for potential recovery


I am newbie in oozie. Can you please help me out in resolving this issue ?

Thanks,
Praveenesh


On Tue, Apr 3, 2012 at 2:32 PM, Mohammad Islam <mislam77@yahoo.com> wrote:

Hi,
>Do you have kerberos on for hadoop?
>
>Can you please check what service (HadoopAccessorService/KerberosHadoopAccessorService)
is actually active?
> 
>You can find from the oozie.log when the oozie server was instantiated. Search for the
above two service class name.
>
>Regards,
>Mohammad
>
>
>
>
>
>----- Original Message -----
>From: praveenesh kumar <praveenesh@gmail.com>
>To: oozie-users@incubator.apache.org
>Cc:
>Sent: Monday, April 2, 2012 6:19 AM
>Subject: Fwd: How can I configure oozie to submit different workflows from different users
?
>
>---------- Forwarded message ----------
>From: praveenesh kumar <praveenesh@gmail.com>
>Date: Mon, Apr 2, 2012 at 5:45 PM
>Subject: How can I configure oozie to submit different workflows from
>different users ?
>To: common-user@hadoop.apache.org
>
>
>Hi all,
>
>I want to use oozie to submit different workflows from different users.
>These users are able to submit hadoop jobs.
>I am using hadoop 0.20.205 and oozie 3.1.3
>I have a hadoop user as a oozie-user
>
>I have set the following things :
>
>conf/oozie-site.xml :
>
>< property >
>< name >oozie.services.ext< /name >
>< value >org.apache.oozie.service.HadoopAccessorService
>< /value >
>< description >
>To add/replace services defined in 'oozie.services' with custom
>implementations.Class names must be separated by commas.
>< /description >
>< /property >
>
>conf/core-site.xml
>< property>
>< name>hadoop.proxyuser.hadoop.hosts< /name>
>< value>*< / value>
>< /property>
>< property>
>< name>hadoop.proxyuser.hadoop.groups< /name>
>< value>*< /value>
>< /property>
>
>When I am submitting jobs as a hadoop user, I am able to run it properly.
>But when I am able to submit the same work flow  from a different user, who
>can submit the simple MR jobs to my hadoop cluster, I am getting the
>following error:
>
>JA009: java.io.IOException: java.io.IOException: The username kumar
>obtained from the conf doesn't match the username hadoop the user
>authenticated as    at
>org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3943)
>at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>at
>sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
>at
>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
>at java.lang.reflect.Method.invoke(Method.java:597)
>at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
>at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
>at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
>at java.security.AccessController.doPrivileged(Native Method)
>at javax.security.auth.Subject.doAs(Subject.java:396)
>at
>org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>
>at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>
>Caused by: java.io.IOException: The username kumar obtained from the conf
>doesn't match the username hadoop the user authenticated as
>at org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:426)
>at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3941)
>... 11 more
>
>

Mime
View raw message