hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Enis Soztutar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17680) Run mini cluster through JNI in tests
Date Fri, 24 Feb 2017 00:42:44 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15881646#comment-15881646

Enis Soztutar commented on HBASE-17680:

Thanks for working on this [~devaraj] and [~ted_yu]. 
This helps a lot in reducing the test execution time as well, since we do not have to wait
for java and jruby instantiation multiple times. If I interpret the above, the run times goes
from ~60 secs to <10 secs. Plus, we need the multi-regionserver capabilities as well as
killing servers, etc for the native client tests which are already available in the mini hbase

For the patch: 
 - Can we move the mini-cluster to be under {{test-util}} module? It does not belong in core.

 - I really like the way that we can call any method from HTU, Connection, Admin, Table etc,
but most of the code in mini-cluster.cc is unnecessarily in the cpp side. Can we do this instead.
Create a NativeTestingUtility.java in hbase-server, and also maybe merge mini-cluster to test-util.cc.
The java counter-part will contain almost all of the code that we need to invoke from native
(like writeConf(), create_table(), etc in the mini-cluster. This will be better, because almost
all of the code will be in the Java side which is way more maintainable. mini-cluster.cc will
just call the corresponding java method in NativeTestingUtility class. 
 - Notice that methods like tablePut, create_table, etc will be dramatically simpler. 
 - Method names in cpp usually use camel case with initial upper case, so methods like {{start_cluster()}}
should be named {{StartCluster()}}. 
 - You also need to call {{DestroyJavaVM()}} once the testing is done. Maybe add it to Shutdown()
or something. 
 - I think this {{+    compiler_flags = ['-I', '/usr/lib/jvm/java-8-openjdk-amd64/include/',
'-I', '/usr/lib/jvm/java-8-openjdk-amd64/include/linux'],
}} assumes that we are running in the docker container. Our builds are weird, where there
is the buck based build and the make based build. However, both are supposed to work without
the docker env. So maybe we need to source the JAVA_HOME. 

> Run mini cluster through JNI in tests
> -------------------------------------
>                 Key: HBASE-17680
>                 URL: https://issues.apache.org/jira/browse/HBASE-17680
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Ted Yu
>            Assignee: Ted Yu
>         Attachments: 17680.v1.txt, 17680.v3.txt, 17680.v8.txt
> Currently tests start local hbase cluster through hbase shell.
> There is less control over the configuration of the local cluster this way.
> This issue would replace hbase shell with JNI interface to mini cluster.
> We would have full control over the cluster behavior.
> Thanks to [~devaraj] who started this initiative.

This message was sent by Atlassian JIRA

View raw message