hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcos Ortiz <mlor...@uci.cu>
Subject Re: Job tracker service start issue.
Date Fri, 23 Mar 2012 15:57:51 GMT


On 03/23/2012 06:57 AM, kasi subrahmanyam wrote:
> Hi Oliver,
>
> I am not sure my suggestion might solve your problem or it might be already
> solved on your side.
> It seems the task tracker is having a problem accessing the tmp directory.
> Try going to the core and mapred site xml and change the tmp directory to a
> new one.
> If this is not yet working then manually change the permissions of theat
> directory  using :
> chmod -R 777 tmp
Please, don´t do chmod -R 777 in tmp directory. It´s not recommendable 
for production servers.
The first option is more wise:
1- change the tmp directory in the core and mapreduce files
2- chown this new directory to group hadoop, where are the mapred and 
hdfs users
>
> On Fri, Mar 23, 2012 at 3:33 PM, Olivier Sallou<olivier.sallou@irisa.fr>wrote:
>
>>
>> Le 3/23/12 8:50 AM, Manish Bhoge a écrit :
>>> I have Hadoop running on Standalone box. When I am starting deamon for
>>> namenode, secondarynamenode, job tracker, task tracker and data node, it
>> is starting gracefully. But soon after it start job tracker it doesn't
>>> show up job tracker service. when i run 'jps' it is showing me all the
>>> services including task tracker except Job Tracker.
>>>
>>> Is there any time limit that need to set up or is it going into the safe
>>> mode. Because when i saw job tracker log this what it is showing, looks
>>> like it is starting the namenode but soon after it shutdown:
>>>
>>> 2012-03-22 23:26:04,061 INFO org.apache.hadoop.mapred.JobTracker:
>> STARTUP_MSG:
>>> /************************************************************
>>> STARTUP_MSG: Starting JobTracker
>>> STARTUP_MSG:   host = manish/10.131.18.119
>>> STARTUP_MSG:   args = []
>>> STARTUP_MSG:   version = 0.20.2-cdh3u3
>>> STARTUP_MSG:   build =
>> file:///data/1/tmp/nightly_2012-02-16_09-46-24_3/hadoop-0.20-0.20.2+923.195-1~maverick
>> -r 217a3767c48ad11d4632e19a22897677268c40c4; compiled by 'root' on Thu Feb
>> 16 10:22:53 PST 2012
>>> ************************************************************/
>>> 2012-03-22 23:26:04,140 INFO
>> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager:
>> Updating the current master key for generating delegation tokens
>>> 2012-03-22 23:26:04,141 INFO
>> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager:
>> Starting expired delegation token remover thread,
>> tokenRemoverScanInterval=60 min(s)
>>> 2012-03-22 23:26:04,141 INFO
>> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager:
>> Updating the current master key for generating delegation tokens
>>> 2012-03-22 23:26:04,142 INFO org.apache.hadoop.mapred.JobTracker:
>> Scheduler configured with (memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT,
>> limitMaxMemForMapTasks, limitMaxMemForReduceTasks) (-1, -1, -1, -1)
>>> 2012-03-22 23:26:04,143 INFO org.apache.hadoop.util.HostsFileReader:
>> Refreshing hosts (include/exclude) list
>>> 2012-03-22 23:26:04,186 INFO org.apache.hadoop.mapred.JobTracker:
>> Starting jobtracker with owner as mapred
>>> 2012-03-22 23:26:04,201 INFO org.apache.hadoop.ipc.Server: Starting
>> Socket Reader #1 for port 54311
>>> 2012-03-22 23:26:04,203 INFO org.apache.hadoop.ipc.metrics.RpcMetrics:
>> Initializing RPC Metrics with hostName=JobTracker, port=54311
>>> 2012-03-22 23:26:04,206 INFO
>> org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC Metrics
>> with hostName=JobTracker, port=54311
>>> 2012-03-22 23:26:09,250 INFO org.mortbay.log: Logging to
>> org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
>> org.mortbay.log.Slf4jLog
>>> 2012-03-22 23:26:09,298 INFO org.apache.hadoop.http.HttpServer: Added
>> global filtersafety
>> (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
>>> 2012-03-22 23:26:09,318 INFO org.apache.hadoop.http.HttpServer: Port
>> returned by webServer.getConnectors()[0].getLocalPort() before open() is
>> -1. Opening the listener on 50030
>>> 2012-03-22 23:26:09,318 INFO org.apache.hadoop.http.HttpServer:
>> listener.getLocalPort() returned 50030
>> webServer.getConnectors()[0].getLocalPort() returned 50030
>>> 2012-03-22 23:26:09,318 INFO org.apache.hadoop.http.HttpServer: Jetty
>> bound to port 50030
>>> 2012-03-22 23:26:09,319 INFO org.mortbay.log: jetty-6.1.26.cloudera.1
>>> 2012-03-22 23:26:09,517 INFO org.mortbay.log: Started
>> SelectChannelConnector@0.0.0.0:50030
>>> 2012-03-22 23:26:09,519 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
>> Initializing JVM Metrics with processName=JobTracker, sessionId=
>>> 2012-03-22 23:26:09,519 INFO org.apache.hadoop.mapred.JobTracker:
>> JobTracker up at: 54311
>>> 2012-03-22 23:26:09,519 INFO org.apache.hadoop.mapred.JobTracker:
>> JobTracker webserver: 50030
>>> 2012-03-22 23:26:09,648 WARN org.apache.hadoop.mapred.JobTracker: Failed
>> to operate on mapred.system.dir
>> (hdfs://localhost:54310/app/hadoop/tmp/mapred/system) because of
>> permissions.
>>> 2012-03-22 23:26:09,648 WARN org.apache.hadoop.mapred.JobTracker: This
>> directory should be owned by the user 'mapred (auth:SIMPLE)'
>>> 2012-03-22 23:26:09,650 WARN org.apache.hadoop.mapred.JobTracker:
>> Bailing out ...
>>> org.apache.hadoop.security.AccessControlException: The systemdir
>> hdfs://localhost:54310/app/hadoop/tmp/mapred/system is not owned by mapred
>> at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2241) at
>> org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2050) at
>> org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:296) at
>> org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:288) at
>> org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4792)
>>> 2012-03-22 23:26:09,652 FATAL org.apache.hadoop.mapred.JobTracker:
>> org.apache.hadoop.security.AccessControlException: The systemdir
>> hdfs://localhost:54310/app/hadoop/tmp/mapred/system is not owned by mapred
>> at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2241) at
>> org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2050) at
>> org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:296) at
>> org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:288) at
>> org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4792) 2012-03-22
>> 23:26:09,654 INFO org.apache.hadoop.mapred.JobTracker: SHUTDOWN_MSG:
>>> /************************************************************
>> you  seem to have dfs access right issues.
>> Debian/Ubuntu packages create 2 users mapred and hdfs, you certainly
>> have the same.
>> A hdfs chmod in you hdfs dir for mapred should solve your issue : chown
>> to mapred or add write access to hadoop group as mapred and hdfs user
>> are both in hadoop group
>>
>> Olivier
>>> SHUTDOWN_MSG: Shutting down JobTracker at manish/10.131.18.119
>>> ************************************************************/
>> --
>>
>>
>> gpg key id: 4096R/326D8438  (keyring.debian.org)
>> Key fingerprint = 5FB4 6F83 D3B9 5204 6335  D26D 78DC 68DB 326D 8438
>>
>>
>>
>
> 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
> CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
>
> http://www.uci.cu
> http://www.facebook.com/universidad.uci
> http://www.flickr.com/photos/universidad_uci
>
>

-- 
Marcos Luis Ortíz Valmaseda (@marcosluis2186)
  Data Engineer at UCI
  http://marcosluis2186.posterous.com


10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION

http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci

Mime
View raw message