hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sitaraman Vilayannur <vrsitaramanietfli...@gmail.com>
Subject Re: Getting error unrecognized option -jvm on starting nodemanager
Date Tue, 24 Dec 2013 07:58:06 GMT
Found it,
 I get the following error on starting namenode in 2.2
10/contrib/capacity-scheduler/*.jar:/usr/local/Software/hadoop-0.23.10/contrib/capacity-scheduler/*.jar:/usr/local/Software/hadoop-0.23.10/contrib/capacity-scheduler/*.jar
STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common
-r 1529768; compiled by 'hortonmu' on 2013-10-07T06:28Z
STARTUP_MSG:   java = 1.7.0_45
************************************************************/
2013-12-24 13:25:48,876 INFO
org.apache.hadoop.hdfs.server.namenode.NameNode: registered UNIX
signal handlers for [TERM, HUP, INT]
2013-12-24 13:25:49,042 INFO
org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from
hadoop-metrics2.properties
2013-12-24 13:25:49,102 INFO
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot
period at 10 second(s).
2013-12-24 13:25:49,102 INFO
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics
system started
2013-12-24 13:25:49,232 WARN org.apache.hadoop.util.NativeCodeLoader:
Unable to load native-hadoop library for your platform... using
builtin-java classes where applicable
2013-12-24 13:25:49,375 INFO org.mortbay.log: Logging to
org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
org.mortbay.log.Slf4jLog
2013-12-24 13:25:49,410 INFO org.apache.hadoop.http.HttpServer: Added
global filter 'safety'
(class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
2013-12-24 13:25:49,412 INFO org.apache.hadoop.http.HttpServer: Added
filter static_user_filter
(class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter)
to context hdfs
2013-12-24 13:25:49,412 INFO org.apache.hadoop.http.HttpServer: Added
filter static_user_filter
(class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter)
to context static
2013-12-24 13:25:49,412 INFO org.apache.hadoop.http.HttpServer: Added
filter static_user_filter
(class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter)
to context logs
2013-12-24 13:25:49,422 INFO org.apache.hadoop.http.HttpServer:
dfs.webhdfs.enabled = false
2013-12-24 13:25:49,432 INFO org.apache.hadoop.http.HttpServer: Jetty
bound to port 50070
2013-12-24 13:25:49,432 INFO org.mortbay.log: jetty-6.1.26
2013-12-24 13:25:49,459 WARN org.mortbay.log: Can't reuse
/tmp/Jetty_0_0_0_0_50070_hdfs____w2cu08, using
/tmp/Jetty_0_0_0_0_50070_hdfs____w2cu08_2787234685293301311
2013-12-24 13:25:49,610 INFO org.mortbay.log: Started
SelectChannelConnector@0.0.0.0:50070
2013-12-24 13:25:49,611 INFO
org.apache.hadoop.hdfs.server.namenode.NameNode: Web-server up at:
0.0.0.0:50070
2013-12-24 13:25:49,628 WARN
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Only one image
storage directory (dfs.namenode.name.dir) configured. Beware of
dataloss due to lack of redundant storage directories!
2013-12-24 13:25:49,628 WARN
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Only one
namespace edits storage directory (dfs.namenode.edits.dir) configured.
Beware of dataloss due to lack of redundant storage directories!
2013-12-24 13:25:49,668 INFO
org.apache.hadoop.hdfs.server.namenode.HostFileManager: read includes:
HostSet(
)
2013-12-24 13:25:49,669 INFO
org.apache.hadoop.hdfs.server.namenode.HostFileManager: read excludes:
HostSet(
)
2013-12-24 13:25:49,670 INFO
org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager:
dfs.block.invalidate.limit=1000
2013-12-24 13:25:49,672 INFO org.apache.hadoop.util.GSet: Computing
capacity for map BlocksMap
2013-12-24 13:25:49,672 INFO org.apache.hadoop.util.GSet: VM type       = 64-bit
2013-12-24 13:25:49,673 INFO org.apache.hadoop.util.GSet: 2.0% max
memory = 889 MB
2013-12-24 13:25:49,673 INFO org.apache.hadoop.util.GSet: capacity
 = 2^21 = 2097152 entries
2013-12-24 13:25:49,677 INFO
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager:
dfs.block.access.token.enable=false
2013-12-24 13:25:49,677 INFO
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager:
defaultReplication         = 1
2013-12-24 13:25:49,677 INFO
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager:
maxReplication             = 512
2013-12-24 13:25:49,677 INFO
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager:
minReplication             = 1
2013-12-24 13:25:49,677 INFO
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager:
maxReplicationStreams      = 2
2013-12-24 13:25:49,677 INFO
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager:
shouldCheckForEnoughRacks  = false
2013-12-24 13:25:49,677 INFO
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager:
replicationRecheckInterval = 3000
2013-12-24 13:25:49,677 INFO
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager:
encryptDataTransfer        = false
2013-12-24 13:25:49,681 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner
  = sitaraman (auth:SIMPLE)
2013-12-24 13:25:49,681 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup
  = supergroup
2013-12-24 13:25:49,681 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
isPermissionEnabled = true
2013-12-24 13:25:49,681 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: HA Enabled: false
2013-12-24 13:25:49,682 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Append Enabled:
true
2013-12-24 13:25:49,801 INFO org.apache.hadoop.util.GSet: Computing
capacity for map INodeMap
2013-12-24 13:25:49,801 INFO org.apache.hadoop.util.GSet: VM type       = 64-bit
2013-12-24 13:25:49,801 INFO org.apache.hadoop.util.GSet: 1.0% max
memory = 889 MB
2013-12-24 13:25:49,801 INFO org.apache.hadoop.util.GSet: capacity
 = 2^20 = 1048576 entries
2013-12-24 13:25:49,802 INFO
org.apache.hadoop.hdfs.server.namenode.NameNode: Caching file names
occuring more than 10 times
2013-12-24 13:25:49,804 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
dfs.namenode.safemode.threshold-pct = 0.9990000128746033
2013-12-24 13:25:49,804 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
dfs.namenode.safemode.min.datanodes = 0
2013-12-24 13:25:49,804 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
dfs.namenode.safemode.extension     = 30000
2013-12-24 13:25:49,805 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Retry cache on
namenode is enabled
2013-12-24 13:25:49,805 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Retry cache will
use 0.03 of total heap and retry cache entry expiry time is 600000
millis
2013-12-24 13:25:49,807 INFO org.apache.hadoop.util.GSet: Computing
capacity for map Namenode Retry Cache
2013-12-24 13:25:49,807 INFO org.apache.hadoop.util.GSet: VM type       = 64-bit
2013-12-24 13:25:49,807 INFO org.apache.hadoop.util.GSet:
0.029999999329447746% max memory = 889 MB
2013-12-24 13:25:49,807 INFO org.apache.hadoop.util.GSet: capacity
 = 2^15 = 32768 entries
2013-12-24 13:25:49,816 INFO
org.apache.hadoop.hdfs.server.common.Storage: Lock on
/usr/local/Software/hadoop-0.23.10/data/hdfs/namenode/in_use.lock
acquired by nodename 19170@localhost.localdomain
2013-12-24 13:25:49,861 INFO org.mortbay.log: Stopped
SelectChannelConnector@0.0.0.0:50070
2013-12-24 13:25:49,964 INFO
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping NameNode
metrics system...
2013-12-24 13:25:49,965 INFO
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics
system stopped.
2013-12-24 13:25:49,965 INFO
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics
system shutdown complete.
2013-12-24 13:25:49,965 FATAL
org.apache.hadoop.hdfs.server.namenode.NameNode: Exception in namenode
join
java.io.IOException:
File system image contains an old layout version -39.
An upgrade to version -47 is required.
Please restart NameNode with -upgrade option.
	at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:221)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:787)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:568)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:443)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:491)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:684)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:669)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1254)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1320)
2013-12-24 13:25:49,967 INFO org.apache.hadoop.util.ExitUtil: Exiting
with status 1
2013-12-24 13:25:49,968 INFO
org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at localhost.localdomain/127.0.0.1
************************************************************/

On 12/24/13, Sitaraman Vilayannur <vrsitaramanietflists@gmail.com> wrote:
> The line beginning with ulimit that i have appended below, i thought
> was the log file?
> /usr/local/Software/hadoop-2.2.0/logs/hadoop-sitaraman-namenode-localhost.localdomain.out
> Sitaraman
> On 12/24/13, Nitin Pawar <nitinpawar432@gmail.com> wrote:
>> Without log, very hard to guess what's happening.
>>
>> Can you clean up the log directory and then start over and check for the
>> logs again.
>>
>>
>> On Tue, Dec 24, 2013 at 11:44 AM, Sitaraman Vilayannur <
>> vrsitaramanietflists@gmail.com> wrote:
>>
>>> Hi Nitin,
>>>  I moved to the release 2.2.0 on starting node manager it remains
>>> silent without errors but nodemanager dosent start....while it does in
>>> the earlier 0.23 version
>>>
>>>
>>> ./hadoop-daemon.sh start namenode
>>> starting namenode, logging to
>>>
>>> /usr/local/Software/hadoop-2.2.0/logs/hadoop-sitaraman-namenode-localhost.localdomain.out
>>> Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library
>>> /usr/local/Software/hadoop-2.2.0/lib/native/libhadoop.so.1.0.0 which
>>> might have disabled stack guard. The VM will try to fix the stack
>>> guard now.
>>> It's highly recommended that you fix the library with 'execstack -c
>>> <libfile>', or link it with '-z noexecstack'.
>>> [sitaraman@localhost sbin]$ jps
>>> 13444 Jps
>>> [sitaraman@localhost sbin]$ vi
>>>
>>> /usr/local/Software/hadoop-2.2.0/logs/hadoop-sitaraman-namenode-localhost.localdomain.out
>>>
>>>
>>> ulimit -a for user sitaraman
>>> core file size          (blocks, -c) 0
>>> data seg size           (kbytes, -d) unlimited
>>> scheduling priority             (-e) 0
>>> file size               (blocks, -f) unlimited
>>> pending signals                 (-i) 135104
>>> max locked memory       (kbytes, -l) 32
>>> max memory size         (kbytes, -m) unlimited
>>> open files                      (-n) 1024
>>> pipe size            (512 bytes, -p) 8
>>> POSIX message queues     (bytes, -q) 819200
>>> real-time priority              (-r) 0
>>> stack size              (kbytes, -s) 10240
>>> cpu time               (seconds, -t) unlimited
>>> max user processes              (-u) 135104
>>> virtual memory          (kbytes, -v) unlimited
>>> file locks                      (-x) unlimited
>>>
>>>
>>> On 12/24/13, Nitin Pawar <nitinpawar432@gmail.com> wrote:
>>> > For now you can ignore this warning,
>>> > it was your first program so you can try building other things and
>>> > slowly
>>> > run the commands mentioned the log message to fix these small
>>> > warnings.
>>> >
>>> >
>>> > On Tue, Dec 24, 2013 at 10:07 AM, Sitaraman Vilayannur <
>>> > vrsitaramanietflists@gmail.com> wrote:
>>> >
>>> >> Thanks Nitin, That worked,
>>> >> When i run the Pi example, i get the following warning at the end,
>>> >> what must i do about this warning....thanks much for your help.
>>> >> Sitaraman
>>> >> inished in 20.82 seconds
>>> >> Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library
>>> >> /usr/local/Software/hadoop-0.23.10/lib/native/libhadoop.so.1.0.0
>>> >> which
>>> >> might have disabled stack guard. The VM will try to fix the stack
>>> >> guard now.
>>> >> It's highly recommended that you fix the library with 'execstack -c
>>> >> <libfile>', or link it with '-z noexecstack'.
>>> >> 13/12/24 10:05:19 WARN util.NativeCodeLoader: Unable to load
>>> >> native-hadoop library for your platform... using builtin-java classes
>>> >> where applicable
>>> >> Estimated value of Pi is 3.14127500000000000000
>>> >> [sitaraman@localhost mapreduce]$
>>> >>
>>> >> On 12/23/13, Nitin Pawar <nitinpawar432@gmail.com> wrote:
>>> >> > Can you try starting the process as non root user.
>>> >> > Give proper permissions to the user and start it as a different
>>> >> > user.
>>> >> >
>>> >> > Thanks,
>>> >> > Nitin
>>> >> >
>>> >> >
>>> >> > On Mon, Dec 23, 2013 at 2:15 PM, Sitaraman Vilayannur <
>>> >> > vrsitaramanietflists@gmail.com> wrote:
>>> >> >
>>> >> >> Hi,
>>> >> >>  When i attempt to start nodemanager i get the following error.
>>> >> >> Any
>>> >> >> help
>>> >> >> appreciated.   I was able to start resource manager datanode,
>>> namenode
>>> >> >> and
>>> >> >> secondarynamenode,
>>> >> >>
>>> >> >>
>>> >> >>    ./yarn-daemon.sh start nodemanager
>>> >> >> starting nodemanager, logging to
>>> >> >>
>>> >>
>>> /usr/local/Software/hadoop-0.23.10/logs/yarn-root-nodemanager-localhost.localdomain.out
>>> >> >> Unrecognized option: -jvm
>>> >> >> Error: Could not create the Java Virtual Machine.
>>> >> >> Error: A fatal exception has occurred. Program will exit.
>>> >> >> [root@localhost sbin]# emacs
>>> >> >>
>>> >>
>>> /usr/local/Software/hadoop-0.23.10/logs/yarn-root-nodemanager-localhost.localdomain.out
>>> >> >> &
>>> >> >> [4] 29004
>>> >> >> [root@localhost sbin]# jps
>>> >> >> 28402 SecondaryNameNode
>>> >> >> 30280 Jps
>>> >> >> 28299 DataNode
>>> >> >> 6729 Main
>>> >> >> 26044 ResourceManager
>>> >> >> 28197 NameNode
>>> >> >>
>>> >> >
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Nitin Pawar
>>> >> >
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > Nitin Pawar
>>> >
>>>
>>
>>
>>
>> --
>> Nitin Pawar
>>
>

Mime
View raw message