phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Taylor (JIRA)" <>
Subject [jira] [Commented] (PHOENIX-3181) Run test methods in parallel to reduce test suite run time
Date Tue, 16 Aug 2016 22:26:20 GMT


James Taylor commented on PHOENIX-3181:

In order to get reasonable parallelization from the mini cluster that is spun up, you'll need
to *not* override some of the config properties we set (as we set them on purpose to lower
values as we're not currently expecting to need any parallelization). For example
- In BaseTest.setUpConfigForMiniCluster, remove the code adjusting the config properties down:
    public static Configuration setUpConfigForMiniCluster(Configuration conf, ReadOnlyProps
overrideProps) {
        setDefaultTestConfig(conf, overrideProps);
         * The default configuration of mini cluster ends up spawning a lot of threads
         * that are not really needed by phoenix for test purposes. Limiting these threads
         * helps us in running several mini clusters at the same time without hitting 
         * the threads limit imposed by the OS. 
        conf.setInt(HConstants.REGION_SERVER_HANDLER_COUNT, 5);
        conf.setInt("hbase.regionserver.metahandler.count", 2);
        conf.setInt(HConstants.MASTER_HANDLER_COUNT, 2);
        conf.setInt("dfs.namenode.handler.count", 2);
        conf.setInt("dfs.namenode.service.handler.count", 2);
        conf.setInt("dfs.datanode.handler.count", 2);
        conf.setInt("", 2);
        conf.setInt("ipc.server.handler.threadpool.size", 2);
        conf.setInt("hbase.regionserver.hlog.syncer.count", 2);
        conf.setInt("hbase.hlog.asyncer.number", 2);
        conf.setInt("hbase.assignment.zkevent.workers", 5);
        conf.setInt("hbase.assignment.threads.max", 5);
        conf.setInt("hbase.catalogjanitor.interval", 5000);
        return conf;
- Remove the overriding/setting of these properties in QueryServicesTestImpl (so that the
regular non test defaults are used):
    private static final int DEFAULT_THREAD_POOL_SIZE = 20;
    private static final int DEFAULT_QUEUE_SIZE = 0;
    public static final int DEFAULT_INDEX_HANDLER_COUNT = 5;
    public static final int DEFAULT_METADATA_HANDLER_COUNT = 5;
    public static final int DEFAULT_HCONNECTION_POOL_CORE_SIZE = 10;
    public static final int DEFAULT_HCONNECTION_POOL_MAX_SIZE = 10;
    public static final int DEFAULT_HTABLE_MAX_THREADS = 10;

> Run test methods in parallel to reduce test suite run time
> ----------------------------------------------------------
>                 Key: PHOENIX-3181
>                 URL:
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: James Taylor
> With PHOENIX-2995, the tests within a test class can be executed in parallel since the
table names won't conflict. This should greatly reduce the time taken to run all of our tests.

This message was sent by Atlassian JIRA

View raw message