hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravishankar Nair <ravishankar.n...@gmail.com>
Subject Why this problem is not solved yet ?
Date Fri, 18 May 2012 01:49:43 GMT
Dear experts,

Today is my tenth day working with Hadoop on installing on my windows
machine. I am trying again and again because , some where someone has
written that it works on Windows with CYGWIN.(And noone has written that
Hadoop wont work on Windows). I am attaching my config files.

Kindly help me, if anything can make this work.A feeble and humble request
to all experts out there.

Here is the error, if you search , you can see thousands have reported this
and there is no solution I got yet, though I tried all ways possible. I am
using Windows XP SP3, Hadoop (tried with five versions so far including
1.0.3).  I am running on a single node.(machine WSUSJXLHRN13067 IP:-
192.168.0.16)
When I start Hadoop, no issues in any of the versions

rn13067@WSUSJXLHRN13067 /home/hadoop-1.0.3
$ bin/start-all.sh
starting namenode, logging to
/home/hadoop-1.0.3/libexec/../logs/hadoop-SUNDOOP-namenode-WSUSJXLHRN13067.out
localhost: starting datanode, logging to
/home/hadoop-1.0.3/libexec/../logs/hadoop-SUNDOOP-datanode-WSUSJXLHRN13067.out
localhost: starting secondarynamenode, logging to
/home/hadoop-1.0.3/libexec/../logs/hadoop-SUNDOOP-secondarynamenode-WSUSJXLHRN13067.out
starting jobtracker, logging to
/home/hadoop-1.0.3/libexec/../logs/hadoop-SUNDOOP-jobtracker-WSUSJXLHRN13067.out
localhost: starting tasktracker, logging to
/home/hadoop-1.0.3/libexec/../logs/hadoop-SUNDOOP-tasktracker-WSUSJXLHRN13067.out



When I run the example program, this is what is printed on my console:-
$ bin/hadoop jar hadoop-examples-1.0.3.jar  grep input output 'dfs[a-z.]+'
12/05/17 21:44:46 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
12/05/17 21:44:46 WARN snappy.LoadSnappy: Snappy native library not loaded
12/05/17 21:44:46 INFO mapred.FileInputFormat: Total input paths to process
: 16
12/05/17 21:44:47 INFO mapred.JobClient: Running job: job_201205172141_0001
12/05/17 21:44:48 INFO mapred.JobClient:  map 0% reduce 0%


Now it is HUNG!!. IN most of the versions this is the behaviour.

Here is the log from JOBTRACKER:-


2012-05-17 21:41:28,037 INFO org.apache.hadoop.mapred.JobTracker:
STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting JobTracker
STARTUP_MSG:   host = WSUSJXLHRN13067/192.168.0.16
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 1.0.3
STARTUP_MSG:   build =
https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r
1335192; compiled by 'hortonfo' on Tue May  8 20:31:25 UTC 2012
************************************************************/
2012-05-17 21:41:28,147 INFO org.apache.hadoop.metrics2.impl.MetricsConfig:
loaded properties from hadoop-metrics2.properties
2012-05-17 21:41:28,147 INFO
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
MetricsSystem,sub=Stats registered.
2012-05-17 21:41:28,162 INFO
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot
period at 10 second(s).
2012-05-17 21:41:28,162 INFO
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: JobTracker metrics
system started
2012-05-17 21:41:28,209 INFO
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
QueueMetrics,q=default registered.
2012-05-17 21:41:28,428 INFO
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi
registered.
2012-05-17 21:41:28,428 WARN
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already
exists!
2012-05-17 21:41:28,428 INFO
org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager:
Updating the current master key for generating delegation tokens
2012-05-17 21:41:28,428 INFO
org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager:
Starting expired delegation token remover thread,
tokenRemoverScanInterval=60 min(s)
2012-05-17 21:41:28,428 INFO
org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager:
Updating the current master key for generating delegation tokens
2012-05-17 21:41:28,428 INFO org.apache.hadoop.mapred.JobTracker: Scheduler
configured with (memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT,
limitMaxMemForMapTasks, limitMaxMemForReduceTasks) (-1, -1, -1, -1)
2012-05-17 21:41:28,428 INFO org.apache.hadoop.util.HostsFileReader:
Refreshing hosts (include/exclude) list
2012-05-17 21:41:28,444 INFO org.apache.hadoop.mapred.JobTracker: Starting
jobtracker with owner as rn13067
2012-05-17 21:41:28,475 INFO org.apache.hadoop.ipc.Server: Starting
SocketReader
2012-05-17 21:41:28,475 INFO
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
RpcDetailedActivityForPort47111 registered.
2012-05-17 21:41:28,475 INFO
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
RpcActivityForPort47111 registered.
2012-05-17 21:41:28,522 INFO org.mortbay.log: Logging to
org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
org.mortbay.log.Slf4jLog
2012-05-17 21:41:28,584 INFO org.apache.hadoop.http.HttpServer: Added
global filtersafety
(class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
2012-05-17 21:41:28,615 WARN org.apache.hadoop.util.NativeCodeLoader:
Unable to load native-hadoop library for your platform... using
builtin-java classes where applicable
2012-05-17 21:41:28,615 INFO org.apache.hadoop.http.HttpServer: Port
returned by webServer.getConnectors()[0].getLocalPort() before open() is
-1. Opening the listener on 50030
2012-05-17 21:41:28,615 INFO org.apache.hadoop.http.HttpServer:
listener.getLocalPort() returned 50030
webServer.getConnectors()[0].getLocalPort() returned 50030
2012-05-17 21:41:28,615 INFO org.apache.hadoop.http.HttpServer: Jetty bound
to port 50030
2012-05-17 21:41:28,615 INFO org.mortbay.log: jetty-6.1.26
2012-05-17 21:41:28,834 INFO org.mortbay.log: Started
SelectChannelConnector@0.0.0.0:50030
2012-05-17 21:41:28,834 INFO
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source jvm
registered.
2012-05-17 21:41:28,834 INFO
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
JobTrackerMetrics registered.
2012-05-17 21:41:28,850 INFO org.apache.hadoop.mapred.JobTracker:
JobTracker up at: 47111
2012-05-17 21:41:28,850 INFO org.apache.hadoop.mapred.JobTracker:
JobTracker webserver: 50030
2012-05-17 21:41:29,225 INFO org.apache.hadoop.mapred.JobTracker: Cleaning
up the system directory
2012-05-17 21:41:29,772 INFO org.apache.hadoop.mapred.JobHistory: Creating
DONE folder at file:/C:/cygwin/home/hadoop-1.0.3/logs/history/done
2012-05-17 21:41:29,787 INFO org.apache.hadoop.mapred.JobTracker: History
server being initialized in embedded mode
2012-05-17 21:41:29,787 INFO org.apache.hadoop.mapred.JobHistoryServer:
Started job history server at: localhost:50030
2012-05-17 21:41:29,787 INFO org.apache.hadoop.mapred.JobTracker: Job
History Server web address: localhost:50030
2012-05-17 21:41:29,787 INFO
org.apache.hadoop.mapred.CompletedJobStatusStore: Completed job store is
inactive
2012-05-17 21:41:29,990 INFO org.apache.hadoop.mapred.JobTracker:
Refreshing hosts information
2012-05-17 21:41:30,006 INFO org.apache.hadoop.util.HostsFileReader:
Setting the includes file to
2012-05-17 21:41:30,006 INFO org.apache.hadoop.util.HostsFileReader:
Setting the excludes file to
2012-05-17 21:41:30,006 INFO org.apache.hadoop.util.HostsFileReader:
Refreshing hosts (include/exclude) list
2012-05-17 21:41:30,006 INFO org.apache.hadoop.mapred.JobTracker:
Decommissioning 0 nodes
2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
Responder: starting
2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
listener on 47111: starting
2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 0 on 47111: starting
2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 1 on 47111: starting
2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 2 on 47111: starting
2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 3 on 47111: starting
2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 4 on 47111: starting
2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 5 on 47111: starting
2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 6 on 47111: starting
2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 7 on 47111: starting
2012-05-17 21:41:30,006 INFO org.apache.hadoop.mapred.JobTracker: Starting
RUNNING
2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 8 on 47111: starting
2012-05-17 21:41:30,006 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 9 on 47111: starting
2012-05-17 21:44:47,069 INFO org.apache.hadoop.mapred.JobInProgress:
job_201205172141_0001: nMaps=16 nReduces=1 max=-1
2012-05-17 21:44:47,069 INFO org.apache.hadoop.mapred.JobTracker: Job
job_201205172141_0001 added successfully for user 'rn13067' to queue
'default'
2012-05-17 21:44:47,069 INFO org.apache.hadoop.mapred.JobTracker:
Initializing job_201205172141_0001
2012-05-17 21:44:47,069 INFO org.apache.hadoop.mapred.JobInProgress:
Initializing job_201205172141_0001
2012-05-17 21:44:47,069 INFO org.apache.hadoop.mapred.AuditLogger:
USER=rn13067    IP=192.168.0.16    OPERATION=SUBMIT_JOB
TARGET=job_201205172141_0001    RESULT=SUCCESS
*2012-05-17 21:44:47,084 ERROR org.apache.hadoop.mapred.JobHistory: Failed
creating job history log file for job job_201205172141_0001
java.io.IOException: Failed to set permissions of path:
C:\cygwin\home\hadoop-1.0.3\logs\history\job_201205172141_0001_1337305487022_rn13067_grep-search
to 0744*
    at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689)
    at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:678)
    at
org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
    at
org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:286)
    at
org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:385)
    at
org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:364)
    at
org.apache.hadoop.mapred.JobHistory$JobInfo.logSubmitted(JobHistory.java:1696)
    at org.apache.hadoop.mapred.JobInProgress$3.run(JobInProgress.java:681)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
    at
org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:678)
    at org.apache.hadoop.mapred.JobTracker.initJob(JobTracker.java:4207)
    at
org.apache.hadoop.mapred.EagerTaskInitializationListener$InitJob.run(EagerTaskInitializationListener.java:79)
    at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)

Kindly help me. CYGWIN is in the path. (As some one suggested in some
thread).




-- 
Warmest Regards,

Ravi

Mime
View raw message