ambari-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmytro Grinenko (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AMBARI-18238) HBase Master doesn't start after upgrading from HDP 2.3.6 using Ambari
Date Tue, 23 Aug 2016 16:32:21 GMT

     [ https://issues.apache.org/jira/browse/AMBARI-18238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Dmytro Grinenko updated AMBARI-18238:
-------------------------------------
    Resolution: Fixed
        Status: Resolved  (was: Patch Available)

> HBase Master doesn't start after upgrading from HDP 2.3.6 using Ambari
> ----------------------------------------------------------------------
>
>                 Key: AMBARI-18238
>                 URL: https://issues.apache.org/jira/browse/AMBARI-18238
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>    Affects Versions: trunk
>            Reporter: Dmytro Grinenko
>            Priority: Critical
>             Fix For: trunk
>
>         Attachments: AMBARI-18238.patch
>
>
> After upgrading the HBase server doesn't start throws below error.
> {code}
> 016-08-21 05:49:27,298 INFO  [main] util.ServerCommandLine: env:SERVER_GC_OPTS=-verbose:gc
-XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/var/log/hbase/gc.log-201608210549
> 2016-08-21 05:49:27,298 INFO  [main] util.ServerCommandLine: env:HADOOP_CONF=/usr/hdp/2.5.0.0-1232/hadoop/conf
> 2016-08-21 05:49:27,298 INFO  [main] util.ServerCommandLine: env:HOSTNAME=hcube1-1n03.eng.hortonworks.com
> 2016-08-21 05:49:27,299 INFO  [main] util.ServerCommandLine: env:QTDIR=/usr/lib64/qt-3.3
> 2016-08-21 05:49:27,299 INFO  [main] util.ServerCommandLine: env:NLSPATH=/usr/dt/lib/nls/msg/%L/%N.cat
> 2016-08-21 05:49:27,299 INFO  [main] util.ServerCommandLine: env:HBASE_THRIFT_OPTS=
> 2016-08-21 05:49:27,299 INFO  [main] util.ServerCommandLine: env:HBASE_HOME=/usr/hdp/current/hbase-master/bin/..
> 2016-08-21 05:49:27,299 INFO  [main] util.ServerCommandLine: env:QTLIB=/usr/lib64/qt-3.3/lib
> 2016-08-21 05:49:27,299 INFO  [main] util.ServerCommandLine: env:HOME=/home/hbase
> 2016-08-21 05:49:27,299 INFO  [main] util.ServerCommandLine: env:MALLOC_ARENA_MAX=4
> 2016-08-21 05:49:27,301 INFO  [main] util.ServerCommandLine: vmName=Java HotSpot(TM)
64-Bit Server VM, vmVendor=Oracle Corporation, vmVersion=25.60-b23
> 2016-08-21 05:49:27,302 INFO  [main] util.ServerCommandLine: vmInputArguments=[-Dproc_master,
-XX:OnOutOfMemoryError=kill -9 %p, -Dhdp.version=2.5.0.0-1232, -XX:+UseConcMarkSweepGC, -XX:ErrorFile=/var/log/hbase/hs_err_pid%p.log,
-Djava.security.auth.login.config=/usr/hdp/current/hbase-master/conf/hbase_client_jaas.conf,
-Djava.io.tmpdir=/tmp, -verbose:gc, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -Xloggc:/var/log/hbase/gc.log-201608210549,
-Xmx2048m, -Djava.security.auth.login.config=/usr/hdp/current/hbase-master/conf/hbase_master_jaas.conf,
-Dhbase.log.dir=/var/log/hbase, -Dhbase.log.file=hbase-hbase-master-hcube1-1n03.eng.hortonworks.com.log,
-Dhbase.home.dir=/usr/hdp/current/hbase-master/bin/.., -Dhbase.id.str=hbase, -Dhbase.root.logger=INFO,RFA,
-Djava.library.path=:/usr/hdp/2.5.0.0-1232/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.5.0.0-1232/hadoop/lib/native,
-Dhbase.security.logger=INFO,RFAS]
> 2016-08-21 05:49:27,715 INFO  [main] regionserver.RSRpcServices: master/hcube1-1n03.eng.hortonworks.com/172.18.129.3:16000
server-side HConnection retries=350
> 2016-08-21 05:49:27,892 INFO  [main] ipc.SimpleRpcScheduler: Using fifo as user call
queue, count=12
> 2016-08-21 05:49:27,896 INFO  [main] ipc.PhoenixRpcSchedulerFactory: Using custom Phoenix
Index RPC Handling with index rpc priority 1000 and metadata rpc priority 2000
> 2016-08-21 05:49:27,919 INFO  [main] ipc.RpcServer: master/hcube1-1n03.eng.hortonworks.com/172.18.129.3:16000:
started 10 reader(s).
> 2016-08-21 05:49:28,027 INFO  [main] impl.MetricsConfig: loaded properties from hadoop-metrics2-hbase.properties
> 2016-08-21 05:49:28,063 INFO  [main] timeline.HadoopTimelineMetricsSink: Initializing
Timeline metrics sink.
> 2016-08-21 05:49:28,063 INFO  [main] timeline.HadoopTimelineMetricsSink: Identified hostname
= hcube1-1n03.eng.hortonworks.com, serviceName = hbase
> 2016-08-21 05:49:28,174 INFO  [main] timeline.HadoopTimelineMetricsSink: Collector Uri:
http://hcube1-2n01.eng.hortonworks.com:6188/ws/v1/timeline/metrics
> 2016-08-21 05:49:28,189 INFO  [main] impl.MetricsSinkAdapter: Sink timeline started
> 2016-08-21 05:49:28,278 INFO  [main] impl.MetricsSystemImpl: Scheduled snapshot period
at 10 second(s).
> 2016-08-21 05:49:28,278 INFO  [main] impl.MetricsSystemImpl: HBase metrics system started
> 2016-08-21 05:49:28,649 INFO  [main] security.UserGroupInformation: Login successful
for user hbase/hcube1-1n03.eng.hortonworks.com@HWX.STANLEYHOTEL.COM using keytab file /etc/security/keytabs/hbase.service.keytab
> 2016-08-21 05:49:28,655 INFO  [main] hfile.CacheConfig: Allocating LruBlockCache size=791.90
MB, blockSize=64 KB
> 2016-08-21 05:49:28,663 WARN  [main] hfile.CacheConfig: Configuration 'hbase.bucketcache.percentage.in.combinedcache'
is no longer respected. See comments in http://hbase.apache.org/book.html#_changes_of_note
> 2016-08-21 05:49:28,674 INFO  [main] util.ByteBufferArray: Allocating buffers total=10
GB, sizePerBuffer=4 MB, count=2560, direct=true
> 2016-08-21 05:49:30,320 ERROR [main] master.HMasterCommandLine: Master exiting
> java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster
>         at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2515)
>         at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:235)
>         at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:139)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
>         at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
>         at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2529)
> Caused by: java.lang.OutOfMemoryError: Direct buffer memory
>         at java.nio.Bits.reserveMemory(Bits.java:658)
>         at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
>         at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
>         at org.apache.hadoop.hbase.util.ByteBufferArray.<init>(ByteBufferArray.java:65)
>         at org.apache.hadoop.hbase.io.hfile.bucket.ByteBufferIOEngine.<init>(ByteBufferIOEngine.java:47)
>         at org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.getIOEngineFromName(BucketCache.java:307)
>         at org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.<init>(BucketCache.java:217)
>         at org.apache.hadoop.hbase.io.hfile.CacheConfig.getBucketCache(CacheConfig.java:614)
>         at org.apache.hadoop.hbase.io.hfile.CacheConfig.getL2(CacheConfig.java:553)
>         at org.apache.hadoop.hbase.io.hfile.CacheConfig.instantiateBlockCache(CacheConfig.java:637)
>         at org.apache.hadoop.hbase.io.hfile.CacheConfig.<init>(CacheConfig.java:231)
>         at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:564)
>         at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:411)
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
>         at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2510)
>         ... 5 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message