hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Vimont (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-15835) HBaseTestingUtility#startMiniCluster throws "HMasterAddress already in use" RuntimeException when a local instance of HBase is running
Date Tue, 24 May 2016 10:49:12 GMT

     [ https://issues.apache.org/jira/browse/HBASE-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Daniel Vimont updated HBASE-15835:
----------------------------------
    Status: Patch Available  (was: Open)

Submitting a revised patch which includes all of the following...

Subtask 1: Remove instances of setting the ports to -1 in existing tests.
The following modules were modified to remove their (now apparently extraneous) setting of
master-info-port and region-server-port:
{code}
hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotCloneIndependence.java
hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/mob/TestMobDataBlockEncoding.java
hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/mob/TestExpiredMobFileCleaner.java
hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/mob/mapreduce/TestMobSweepJob.java
hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/mob/mapreduce/TestMobSweepReducer.java
hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/mob/mapreduce/TestMobSweepMapper.java
hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/mob/compactions/TestMobCompactor.java
hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/mob/compactions/TestPartitionedMobCompactor.java
hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/mob/TestDefaultMobStoreFlusher.java
hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMobStoreScanner.java
hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMobStoreCompaction.java
hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDeleteMobTable.java
hbase/hbase-shell/src/test/rsgroup/org/apache/hadoop/hbase/client/rsgroup/TestShellRSGroups.java
hbase/hbase-shell/src/test/java/org/apache/hadoop/hbase/client/AbstractTestShell.java
{code}

Subtask 2: Add some class-level javadoc.
The following was added to the HBaseTestingUtility class-level javadoc comment:
{code}
* In the configuration properties, default values for master-info-port and
* region-server-port are overridden such that a random port will be assigned (thus
* avoiding port contention if another local HBase instance is already running).
{code}

Subtask 3: Add a debug-level logging message for when port values are overridden to "-1".
The following code now appears at the end of the main constructor for HBaseTestingUtility.
Note the "debug" logging that has been added:
{code}
    // prevent contention for ports if other hbase thread(s) already running
    if (conf != null) {
      if (conf.getInt(HConstants.MASTER_INFO_PORT, HConstants.DEFAULT_MASTER_INFOPORT)
              == HConstants.DEFAULT_MASTER_INFOPORT) {
        conf.setInt(HConstants.MASTER_INFO_PORT, -1);
        LOG.debug("Config property " + HConstants.MASTER_INFO_PORT + " changed to -1");
      }
      if (conf.getInt(HConstants.REGIONSERVER_PORT, HConstants.DEFAULT_REGIONSERVER_PORT)
              == HConstants.DEFAULT_REGIONSERVER_PORT) {
        conf.setInt(HConstants.REGIONSERVER_PORT, -1);
        LOG.debug("Config property " + HConstants.REGIONSERVER_PORT + " changed to -1");
      }
    }
{code}

Subtask 4: Add new method to TestHBaseTestingUtility for testing port overrides.
The following new method assures that port override is taking place when it should, and is
NOT taking place when it should NOT:
{code}
  @Test
  public void testOverridingOfDefaultPorts() {

    // confirm that default port properties being overridden to "-1"
    Configuration defaultConfig = HBaseConfiguration.create();
    defaultConfig.setInt(HConstants.MASTER_INFO_PORT, HConstants.DEFAULT_MASTER_INFOPORT);
    defaultConfig.setInt(HConstants.REGIONSERVER_PORT, HConstants.DEFAULT_REGIONSERVER_PORT);
    HBaseTestingUtility htu = new HBaseTestingUtility(defaultConfig);
    assertEquals(-1, htu.getConfiguration().getInt(HConstants.MASTER_INFO_PORT, 0));
    assertEquals(-1, htu.getConfiguration().getInt(HConstants.REGIONSERVER_PORT, 0));

    // confirm that nonDefault (custom) port settings are NOT overridden
    Configuration altConfig = HBaseConfiguration.create();
    final int nonDefaultMasterInfoPort = 3333;
    final int nonDefaultRegionServerPort = 4444;
    altConfig.setInt(HConstants.MASTER_INFO_PORT, nonDefaultMasterInfoPort);
    altConfig.setInt(HConstants.REGIONSERVER_PORT, nonDefaultRegionServerPort);
    htu = new HBaseTestingUtility(altConfig);
    assertEquals(nonDefaultMasterInfoPort,
            htu.getConfiguration().getInt(HConstants.MASTER_INFO_PORT, 0));
    assertEquals(nonDefaultRegionServerPort
            , htu.getConfiguration().getInt(HConstants.REGIONSERVER_PORT, 0));
  }
{code}

> HBaseTestingUtility#startMiniCluster throws "HMasterAddress already in use" RuntimeException
when a local instance of HBase is running
> --------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-15835
>                 URL: https://issues.apache.org/jira/browse/HBASE-15835
>             Project: HBase
>          Issue Type: Bug
>          Components: API
>    Affects Versions: 2.0.0
>            Reporter: Daniel Vimont
>            Assignee: Daniel Vimont
>              Labels: easyfix
>             Fix For: 2.0.0
>
>         Attachments: HBASE-15835-v1.patch, HBASE-15835-v2.patch, HBASE-15835-v3.patch
>
>
> When a MiniCluster is being started with the {{HBaseTestUtility#startMiniCluster}} method
(most typically in the context of JUnit testing), if a local HBase instance is already running
(or for that matter, another thread with another MiniCluster is already running), the startup
will fail with a RuntimeException saying "HMasterAddress already in use", referring explicitly
to contention for the same default master info port (16010).
> This problem most recently came up in conjunction with HBASE-14876 and its sub-JIRAs
(development of new HBase-oriented Maven archetypes), but this is apparently a known issue
to veteran developers, who tend to set up the @BeforeClass sections of their test modules
with code similar to the following:
> {code}
>     UTIL = HBaseTestingUtility.createLocalHTU();
>     // disable UI's on test cluster.
>     UTIL.getConfiguration().setInt("hbase.master.info.port", -1);
>     UTIL.getConfiguration().setInt("hbase.regionserver.info.port", -1);
>     UTIL.startMiniCluster();
> {code}
> A comprehensive solution modeled on this should be put directly into HBaseTestUtility's
main constructor, using one of the following options:
> OPTION 1 (always force random port assignment):
> {code}
>     this.getConfiguration().setInt(HConstants.MASTER_INFO_PORT, -1);
>     this.getConfiguration().setInt(HConstants.REGIONSERVER_PORT, -1);
> {code}
> OPTION 2 (always force random port assignment if user has not explicitly defined alternate
port):
> {code}
>     Configuration conf = this.getConfiguration();
>     if (conf.getInt(HConstants.MASTER_INFO_PORT, HConstants.DEFAULT_MASTER_INFOPORT)
>             == HConstants.DEFAULT_MASTER_INFOPORT) {
>       conf.setInt(HConstants.MASTER_INFO_PORT, -1);
>     }
>     if (conf.getInt(HConstants.REGIONSERVER_PORT, HConstants.DEFAULT_REGIONSERVER_PORT)
>             == HConstants.DEFAULT_REGIONSERVER_PORT) {
>       conf.setInt(HConstants.REGIONSERVER_PORT, -1);
>     }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message