hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guillaume Dufrêne <dufr...@argia.fr>
Subject Re: How to initialize a test environment ?
Date Tue, 13 Oct 2009 15:59:52 GMT
stack wrote:
> A few comments interspersed in the below:
>
> 2009/10/13 Guillaume Dufrêne <dufrene@argia.fr>
>
>
>   
>> I wrote a class inspired from the Mini*Server located in Hbase Tests.
>>
>> (I use Hbase 0.20)
>>
>> Here my class :
>>
>> public class StandaloneServer {
>>
>>  private static Log log = LogFactory.getLog(StandaloneServer.class);
>>
>>  private MiniZooKeeperCluster zookeeper;
>>  private MiniHBaseCluster hbase;
>>  private MiniDFSCluster dfs;
>>
>>  private HBaseConfiguration conf;
>>
>>  public void startup() throws Exception {
>>      conf = new HBaseConfiguration();
>>
>>      log.info("Start DFS ...");
>>      dfs = new MiniDFSCluster(conf, 2, true, (String[]) null);
>>      // set file system to the mini dfs just started up
>>      FileSystem fs = dfs.getFileSystem();
>>      conf.set("fs.default.name", fs.getUri().toString());
>>
>>     
>
> You are using 0.20?  FYI, the above name is deprecated in TRUNK.  HBase
> TRUNK has been changed to use new name.
>
>
>   
Yep, I use 0.20 release, I didn't build hbase myself from the trunk.
It seems that "fs.default.name" is used in the 0.20 release as shown in 
unit tests.

>>      Path parentdir = fs.getHomeDirectory();
>>      conf.set(HConstants.HBASE_DIR, parentdir.toString());
>>      fs.mkdirs(parentdir);
>>      FSUtils.setVersion(fs, parentdir);
>>
>>      log.info("Start Zookeeper ...");
>>      zookeeper = new MiniZooKeeperCluster();
>>      int clientPort = zookeeper.startup(new File(parentdir.toString(),
>> "zookeeper"));
>>      conf.set("hbase.zookeeper.property.clientPort", Integer
>>              .toString(clientPort));
>>     
>>      // Disable regionserver info.
>>      conf.set("hbase.regionserver.info.port", "-1");
>>      conf.set("hbase.master.info.port", "-1");
>>
>>      log.info("Start HBase ...");
>>      hbase = new MiniHBaseCluster(conf, 1);
>>
>>      log.info("Check ROOT metatable connexion ...");
>>      new HTable(conf, HConstants.META_TABLE_NAME);
>>
>>  }
>>
>>  public void shutdown() {
>>      hbase.shutdown();
>>      if (dfs != null) {
>>          try {
>>              FileSystem fs = dfs.getFileSystem();
>>              if (fs != null)
>>                  fs.close();
>>          } catch (IOException e) {
>>              System.err.println("error closing file system: " + e);
>>          }
>>          try {
>>              dfs.shutdown();
>>          } catch (Exception e) { /* ignore */
>>          }
>>      }
>>  }
>>
>>     
>
> The shutdown thread stuff is different in TRUNK.
>   

Thanks for that information, I will update my code when I migrate hbase 
version.

>>  public static void main(String[] args) throws Exception {
>>      final StandaloneServer server = new StandaloneServer();
>>      server.startup();
>>
>>      // add close hook
>>      Runtime.getRuntime().addShutdownHook(new Thread() {
>>          public void run() {
>>              server.shutdown();
>>          }
>>      });
>>
>>  }
>> }
>>
>> I first wrote a main operation to easily test the behaviour ...
>> The idea is to start the server when my unit test is set up.
>>
>> I got the following error :
>> 09/10/13 14:12:13 INFO hbase.StandaloneServer: Check ROOT metatable
>> connexion ...
>> Exception in thread "main"
>> org.apache.hadoop.hbase.client.NoServerForRegionException: No server address
>> listed in -ROOT- for region .META.,,1
>>  at
>> [oahh.client]HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:660)
>>
>>  at
>> [oahh.client].HConnectionManager$TableServers.locateRegion(HConnectionManager.java:582)
>>
>>  at
>> [oahh.client].client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:549)
>>
>>  at [oahh.client].HTable.<init>(HTable.java:125)
>>  at (...) StandaloneServer.startup(StandaloneServer.java:53)
>>  at (...) StandaloneServer.main(StandaloneServer.java:76)
>>
>>     
>
>
> This happens on start up until root is deployed... you'll see one or two in
> the logs.
>   
The StackTrace is triggered from my last line of code that check access 
to META table.
The MiniHBaseCluster is supposed to wait for region server to start, 
isn't it ?
Does the "root" you pointed out be the HBase master ? (I suppose so)
What should I do to ensure this root deployment ?
do a try/catch and retry N times with a delay ?

>> And some errors about "NO JSP Support" (but I think that I can ignore them,
>> no ?)
>>
>>     
> You've turned off the info servers -- the web uis -- so should be fine.
>   
>
>
>
>   
>> So my question are :
>> - Is it the good way to start a test environment ?
>>
>>     
>
> Yeah, you seem to have figured out our ugly test setup.  You could also have
> run hbase with all defaults?  Then it runs a master and regionserver in same
> in JVM writing the local fileystem.
>   
You mean that I can run "LocalHBaseCluster" with a new 
HBaseConfiguration() without starting FS ?

If Someone has a java code sample that start a hbase, let's share ;-)
>> [...]

Thanks.
--
Guillaume.





Ce message et toutes les pièces jointes sont établis à l'attention exclusive de leurs destinataires
et sont confidentiels. Si vous recevez ce message par erreur, merci de le détruire et d'en
avertir immédiatement l'expéditeur. L'internet ne permettant pas d'assurer l'intégrité
de ce message, le contenu de ce message ne représente en aucun cas un engagement de la part
de Leroy Merlin.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message