hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravi Shankar Nair <ravishankar.n...@gmail.com>
Subject Re: Why this problem is not solved yet ?
Date Fri, 18 May 2012 20:56:06 GMT
Hi Ravi,

Let me try and revert by couple of hours, thanks for input

Sent from my Beethoven 


On May 18, 2012, at 3:45 PM, Ravi Prakash <ravihadoop@gmail.com> wrote:

> Hi Ravishankar,
> 
> I don't see two very important processes in your jps output. Just like
> there's JobTracker and NameNode, you should also have "TaskTracker" and
> "DataNode". The JobTracker only schedules jobs. To actually run the map
> reduce tasks, it needs TaskTrackers. This is why you see the jobtracker
> accepting your jobs and then getting stuck: because it doesn't have
> TaskTrackers to run that job on.
> 
> If I were you, I'd first check why the Datanode is not coming up. All 4
> daemons are necessary for running jobs. The logs for those two should be in
> the same directory in which you find the JT's logs.
> 
> Hope this helps.
> Ravi.
> 
> On Fri, May 18, 2012 at 5:17 AM, Ravishankar Nair <
> ravishankar.nair@gmail.com> wrote:
> 
>> Additionally, attached is the output of the job that I run( I mean the
>> example program named grep)
>> 
>> 
>> On Fri, May 18, 2012 at 6:15 AM, Ravishankar Nair <
>> ravishankar.nair@gmail.com> wrote:
>> 
>>> Hi Ravi,
>>> 
>>> Yes , it Running. Here is the output:-
>>> rn13067@WSUSJXLHRN13067 /home/hadoop-1.0.3
>>> $ jps
>>> 5068 NameNode
>>> 5836 Jps
>>> 3516 JobTracker
>>> 
>>> 
>>> Here are the logs from JOBTRACKER:-
>>> 
>>> 2012-05-17 21:41:31,772 INFO org.apache.hadoop.mapred.TaskTracker:
>>> STARTUP_MSG:
>>> /************************************************************
>>> STARTUP_MSG: Starting TaskTracker
>>> 
>>> STARTUP_MSG:   host = WSUSJXLHRN13067/192.168.0.16
>>> STARTUP_MSG:   args = []
>>> STARTUP_MSG:   version = 1.0.3
>>> STARTUP_MSG:   build =
>>> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r
>>> 1335192; compiled by 'hortonfo' on Tue May  8 20:31:25 UTC 2012
>>> ************************************************************/
>>> 2012-05-17 21:41:31,944 INFO
>>> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from
>>> hadoop-metrics2.properties
>>> 2012-05-17 21:41:31,990 INFO
>>> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
>>> MetricsSystem,sub=Stats registered.
>>> 2012-05-17 21:41:31,990 INFO
>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot
>>> period at 10 second(s).
>>> 2012-05-17 21:41:31,990 INFO
>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: TaskTracker metrics
>>> system started
>>> 2012-05-17 21:41:32,256 INFO
>>> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi
>>> registered.
>>> 2012-05-17 21:41:32,256 WARN
>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already
>>> exists!
>>> 2012-05-17 21:41:32,365 INFO org.mortbay.log: Logging to
>>> org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
>>> org.mortbay.log.Slf4jLog
>>> 2012-05-17 21:41:32,412 INFO org.apache.hadoop.http.HttpServer: Added
>>> global filtersafety
>>> (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
>>> 2012-05-17 21:41:32,428 INFO org.apache.hadoop.mapred.TaskLogsTruncater:
>>> Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
>>> 2012-05-17 21:41:32,444 INFO org.apache.hadoop.mapred.TaskTracker:
>>> Starting tasktracker with owner as SYSTEM
>>> 2012-05-17 21:41:32,444 INFO org.apache.hadoop.mapred.TaskTracker: Good
>>> mapred local directories are: /tmp/hadoop-SYSTEM/mapred/local
>>> 2012-05-17 21:41:32,459 WARN org.apache.hadoop.util.NativeCodeLoader:
>>> Unable to load native-hadoop library for your platform... using
>>> builtin-java classes where applicable
>>> 2012-05-17 21:41:32,459 ERROR org.apache.hadoop.mapred.TaskTracker: Can
>>> not start task tracker because java.io.IOException: Failed to set
>>> permissions of path: \tmp\hadoop-SYSTEM\mapred\local\ttprivate to 0700
>>>    at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689)
>>>    at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:662)
>>>    at
>>> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
>>>    at
>>> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
>>>    at
>>> org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
>>>    at
>>> org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java:728)
>>>    at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1459)
>>>    at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3742)
>>> 
>>> 2012-05-17 21:41:32,459 INFO org.apache.hadoop.mapred.TaskTracker:
>>> SHUTDOWN_MSG:
>>> /************************************************************
>>> SHUTDOWN_MSG: Shutting down TaskTracker at WSUSJXLHRN13067/192.168.0.16
>>> ************************************************************/
>>> 
>>> Any clue? Thanks
>>> Regards,
>>> ravi
>>> 
>>> 
>>> 
>>> On Fri, May 18, 2012 at 12:01 AM, Ravi Prakash <ravihadoop@gmail.com>wrote:
>>> 
>>>> Ravishankar,
>>>> 
>>>> If you run $ jps, do you see a TaskTracker process running? Can you
>>>> please
>>>> post the tasktracker logs as well?
>>>> 
>>>> On Thu, May 17, 2012 at 8:49 PM, Ravishankar Nair <
>>>> ravishankar.nair@gmail.com> wrote:
>>>> 
>>>>> Dear experts,
>>>>> 
>>>>> Today is my tenth day working with Hadoop on installing on my windows
>>>>> machine. I am trying again and again because , some where someone has
>>>>> written that it works on Windows with CYGWIN.(And noone has written
>>>> that
>>>>> Hadoop wont work on Windows). I am attaching my config files.
>>>>> 
>>>>> Kindly help me, if anything can make this work.A feeble and humble
>>>> request
>>>>> to all experts out there.
>>>>> 
>>>>> Here is the error, if you search , you can see thousands have reported
>>>>> this and there is no solution I got yet, though I tried all ways
>>>> possible.
>>>>> I am using Windows XP SP3, Hadoop (tried with five versions so far
>>>>> including 1.0.3).  I am running on a single node.(machine
>>>> WSUSJXLHRN13067
>>>>> IP:- 192.168.0.16)
>>>>> When I start Hadoop, no issues in any of the versions
>>>>> 
>>>>> rn13067@WSUSJXLHRN13067 /home/hadoop-1.0.3
>>>>> $ bin/start-all.sh
>>>>> starting namenode, logging to
>>>>> 
>>>> /home/hadoop-1.0.3/libexec/../logs/hadoop-SUNDOOP-namenode-WSUSJXLHRN13067.out
>>>>> localhost: starting datanode, logging to
>>>>> 
>>>> /home/hadoop-1.0.3/libexec/../logs/hadoop-SUNDOOP-datanode-WSUSJXLHRN13067.out
>>>>> localhost: starting secondarynamenode, logging to
>>>>> 
>>>> /home/hadoop-1.0.3/libexec/../logs/hadoop-SUNDOOP-secondarynamenode-WSUSJXLHRN13067.out
>>>>> starting jobtracker, logging to
>>>>> 
>>>> /home/hadoop-1.0.3/libexec/../logs/hadoop-SUNDOOP-jobtracker-WSUSJXLHRN13067.out
>>>>> localhost: starting tasktracker, logging to
>>>>> 
>>>> /home/hadoop-1.0.3/libexec/../logs/hadoop-SUNDOOP-tasktracker-WSUSJXLHRN13067.out
>>>>> 
>>>>> 
>>>>> 
>>>>> When I run the example program, this is what is printed on my console:-
>>>>> $ bin/hadoop jar hadoop-examples-1.0.3.jar  grep input output
>>>> 'dfs[a-z.]+'
>>>>> 12/05/17 21:44:46 WARN util.NativeCodeLoader: Unable to load
>>>> native-hadoop
>>>>> library for your platform... using builtin-java classes where
>>>> applicable
>>>>> 12/05/17 21:44:46 WARN snappy.LoadSnappy: Snappy native library not
>>>> loaded
>>>>> 12/05/17 21:44:46 INFO mapred.FileInputFormat: Total input paths to
>>>>> process : 16
>>>>> 12/05/17 21:44:47 INFO mapred.JobClient: Running job:
>>>> job_201205172141_0001
>>>>> 12/05/17 21:44:48 INFO mapred.JobClient:  map 0% reduce 0%
>>>>> 
>>>>> 
>>>>> Now it is HUNG!!. IN most of the versions this is the behaviour.
>>>>> 
>>>>> Here is the log from JOBTRACKER:-
>>>>> 
>>>>> 
>>>>> 2012-05-17 21:41:28,037 INFO org.apache.hadoop.mapred.JobTracker:
>>>>> STARTUP_MSG:
>>>>> /************************************************************
>>>>> STARTUP_MSG: Starting JobTracker
>>>>> STARTUP_MSG:   host = WSUSJXLHRN13067/192.168.0.16
>>>>> STARTUP_MSG:   args = []
>>>>> STARTUP_MSG:   version = 1.0.3
>>>>> STARTUP_MSG:   build =
>>>>> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r
>>>>> 1335192; compiled by 'hortonfo' on Tue May  8 20:31:25 UTC 2012
>>>>> ************************************************************/
>>>>> 2012-05-17 21:41:28,147 INFO
>>>>> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from
>>>>> hadoop-metrics2.properties
>>>>> 2012-05-17 21:41:28,147 INFO
>>>>> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
>>>>> MetricsSystem,sub=Stats registered.
>>>>> 2012-05-17 21:41:28,162 INFO
>>>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot
>>>>> period at 10 second(s).
>>>>> 2012-05-17 21:41:28,162 INFO
>>>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: JobTracker metrics
>>>>> system started
>>>>> 2012-05-17 21:41:28,209 INFO
>>>>> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
>>>>> QueueMetrics,q=default registered.
>>>>> 2012-05-17 21:41:28,428 INFO
>>>>> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
>>>> ugi
>>>>> registered.
>>>>> 2012-05-17 21:41:28,428 WARN
>>>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi
>>>> already
>>>>> exists!
>>>>> 2012-05-17 21:41:28,428 INFO
>>>>> 
>>>> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager:
>>>>> Updating the current master key for generating delegation tokens
>>>>> 2012-05-17 21:41:28,428 INFO
>>>>> 
>>>> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager:
>>>>> Starting expired delegation token remover thread,
>>>>> tokenRemoverScanInterval=60 min(s)
>>>>> 2012-05-17 21:41:28,428 INFO
>>>>> 
>>>> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager:
>>>>> Updating the current master key for generating delegation tokens
>>>>> 2012-05-17 21:41:28,428 INFO org.apache.hadoop.mapred.JobTracker:
>>>>> Scheduler configured with (memSizeForMapSlotOnJT,
>>>> memSizeForReduceSlotOnJT,
>>>>> limitMaxMemForMapTasks, limitMaxMemForReduceTasks) (-1, -1, -1, -1)
>>>>> 2012-05-17 21:41:28,428 INFO org.apache.hadoop.util.HostsFileReader:
>>>>> Refreshing hosts (include/exclude) list
>>>>> 2012-05-17 21:41:28,444 INFO org.apache.hadoop.mapred.JobTracker:
>>>> Starting
>>>>> jobtracker with owner as rn13067
>>>>> 2012-05-17 21:41:28,475 INFO org.apache.hadoop.ipc.Server: Starting
>>>>> SocketReader
>>>>> 2012-05-17 21:41:28,475 INFO
>>>>> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
>>>>> RpcDetailedActivityForPort47111 registered.
>>>>> 2012-05-17 21:41:28,475 INFO
>>>>> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
>>>>> RpcActivityForPort47111 registered.
>>>>> 2012-05-17 21:41:28,522 INFO org.mortbay.log: Logging to
>>>>> org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
>>>>> org.mortbay.log.Slf4jLog
>>>>> 2012-05-17 21:41:28,584 INFO org.apache.hadoop.http.HttpServer: Added
>>>>> global filtersafety
>>>>> (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
>>>>> 2012-05-17 21:41:28,615 WARN org.apache.hadoop.util.NativeCodeLoader:
>>>>> Unable to load native-hadoop library for your platform... using
>>>>> builtin-java classes where applicable
>>>>> 2012-05-17 21:41:28,615 INFO org.apache.hadoop.http.HttpServer: Port
>>>>> returned by webServer.getConnectors()[0].getLocalPort() before open()
>>>> is
>>>>> -1. Opening the listener on 50030
>>>>> 2012-05-17 21:41:28,615 INFO org.apache.hadoop.http.HttpServer:
>>>>> listener.getLocalPort() returned 50030
>>>>> webServer.getConnectors()[0].getLocalPort() returned 50030
>>>>> 2012-05-17 21:41:28,615 INFO org.apache.hadoop.http.HttpServer: Jetty
>>>>> bound to port 50030
>>>>> 2012-05-17 21:41:28,615 INFO org.mortbay.log: jetty-6.1.26
>>>>> 2012-05-17 21:41:28,834 INFO org.mortbay.log: Started
>>>>> SelectChannelConnector@0.0.0.0:50030
>>>>> 2012-05-17 21:41:28,834 INFO
>>>>> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
>>>> jvm
>>>>> registered.
>>>>> 2012-05-17 21:41:28,834 INFO
>>>>> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
>>>>> JobTrackerMetrics registered.
>>>>> 2012-05-17 21:41:28,850 INFO org.apache.hadoop.mapred.JobTracker:
>>>>> JobTracker up at: 47111
>>>>> 2012-05-17 21:41:28,850 INFO org.apache.hadoop.mapred.JobTracker:
>>>>> JobTracker webserver: 50030
>>>>> 2012-05-17 21:41:29,225 INFO org.apache.hadoop.mapred.JobTracker:
>>>> Cleaning
>>>>> up the system directory
>>>>> 2012-05-17 21:41:29,772 INFO org.apache.hadoop.mapred.JobHistory:
>>>> Creating
>>>>> DONE folder at file:/C:/cygwin/home/hadoop-1.0.3/logs/history/done
>>>>> 2012-05-17 21:41:29,787 INFO org.apache.hadoop.mapred.JobTracker:
>>>> History
>>>>> server being initialized in embedded mode
>>>>> 2012-05-17 21:41:29,787 INFO org.apache.hadoop.mapred.JobHistoryServer:
>>>>> Started job history server at: localhost:50030
>>>>> 2012-05-17 21:41:29,787 INFO org.apache.hadoop.mapred.JobTracker: Job
>>>>> History Server web address: localhost:50030
>>>>> 2012-05-17 21:41:29,787 INFO
>>>>> org.apache.hadoop.mapred.CompletedJobStatusStore: Completed job store
>>>> is
>>>>> inactive
>>>>> 2012-05-17 21:41:29,990 INFO org.apache.hadoop.mapred.JobTracker:
>>>>> Refreshing hosts information
>>>>> 2012-05-17 21:41:30,006 INFO org.apache.hadoop.util.HostsFileReader:
>>>>> Setting the includes file to
>>>>> 2012-05-17 21:41:30,006 INFO org.apache.hadoop.util.HostsFileReader:
>>>>> Setting the excludes file to
>>>>> 2012-05-17 21:41:30,006 INFO org.apache.hadoop.util.HostsFileReader:
>>>>> Refreshing hosts (include/exclude) list
>>>>> 2012-05-17 21:41:30,006 INFO org.apache.hadoop.mapred.JobTracker:
>>>>> Decommissioning 0 nodes
>>>>> 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> Responder: starting
>>>>> 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> listener on 47111: starting
>>>>> 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 0 on 47111: starting
>>>>> 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 1 on 47111: starting
>>>>> 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 2 on 47111: starting
>>>>> 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 3 on 47111: starting
>>>>> 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 4 on 47111: starting
>>>>> 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 5 on 47111: starting
>>>>> 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 6 on 47111: starting
>>>>> 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 7 on 47111: starting
>>>>> 2012-05-17 21:41:30,006 INFO org.apache.hadoop.mapred.JobTracker:
>>>> Starting
>>>>> RUNNING
>>>>> 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 8 on 47111: starting
>>>>> 2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 9 on 47111: starting
>>>>> 2012-05-17 21:44:47,069 INFO org.apache.hadoop.mapred.JobInProgress:
>>>>> job_201205172141_0001: nMaps=16 nReduces=1 max=-1
>>>>> 2012-05-17 21:44:47,069 INFO org.apache.hadoop.mapred.JobTracker: Job
>>>>> job_201205172141_0001 added successfully for user 'rn13067' to queue
>>>>> 'default'
>>>>> 2012-05-17 21:44:47,069 INFO org.apache.hadoop.mapred.JobTracker:
>>>>> Initializing job_201205172141_0001
>>>>> 2012-05-17 21:44:47,069 INFO org.apache.hadoop.mapred.JobInProgress:
>>>>> Initializing job_201205172141_0001
>>>>> 2012-05-17 21:44:47,069 INFO org.apache.hadoop.mapred.AuditLogger:
>>>>> USER=rn13067    IP=192.168.0.16    OPERATION=SUBMIT_JOB
>>>>> TARGET=job_201205172141_0001    RESULT=SUCCESS
>>>>> *2012-05-17 21:44:47,084 ERROR org.apache.hadoop.mapred.JobHistory:
>>>>> Failed creating job history log file for job job_201205172141_0001
>>>>> java.io.IOException: Failed to set permissions of path:
>>>>> 
>>>> C:\cygwin\home\hadoop-1.0.3\logs\history\job_201205172141_0001_1337305487022_rn13067_grep-search
>>>>> to 0744*
>>>>>    at
>>>> org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689)
>>>>>    at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:678)
>>>>>    at
>>>>> 
>>>> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
>>>>>    at
>>>>> 
>>>> org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:286)
>>>>>    at
>>>>> 
>>>> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:385)
>>>>>    at
>>>>> 
>>>> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:364)
>>>>>    at
>>>>> 
>>>> org.apache.hadoop.mapred.JobHistory$JobInfo.logSubmitted(JobHistory.java:1696)
>>>>>    at
>>>> org.apache.hadoop.mapred.JobInProgress$3.run(JobInProgress.java:681)
>>>>>    at java.security.AccessController.doPrivileged(Native Method)
>>>>>    at javax.security.auth.Subject.doAs(Subject.java:396)
>>>>>    at
>>>>> 
>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>>>>>    at
>>>>> 
>>>> org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:678)
>>>>>    at
>>>> org.apache.hadoop.mapred.JobTracker.initJob(JobTracker.java:4207)
>>>>>    at
>>>>> 
>>>> org.apache.hadoop.mapred.EagerTaskInitializationListener$InitJob.run(EagerTaskInitializationListener.java:79)
>>>>>    at
>>>>> 
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>    at
>>>>> 
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>    at java.lang.Thread.run(Thread.java:662)
>>>>> 
>>>>> Kindly help me. CYGWIN is in the path. (As some one suggested in some
>>>>> thread).
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Warmest Regards,
>>>>> 
>>>>> Ravi
>>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> Warmest Regards,
>>> 
>>> Ravi Shankar
>>> 
>> 
>> 
>> 
>> --
>> Warmest Regards,
>> 
>> Ravi Shankar
>> 

Mime
View raw message