hawq-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Lynch <dly...@pivotal.io>
Subject Re: hawq 2.0 on YARN
Date Mon, 23 Nov 2015 16:48:35 GMT
here is a working config example from my lab where hawq will execute in yarn




$GPHOME/etc/hawq-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration>

    <property>
    <name>hawq_resourcemanager_query_noresource_timeout</name>
<value>30</value>
    </property>

    <property>
        <name>hawq_master_address_host</name>
        <value>node2</value>
        <description>The host name of hawq master.</description>
    </property>

    <property>
        <name>hawq_master_address_port</name>
        <value>2020</value>
        <description>The port of hawq master.</description>
    </property>

    <property>
        <name>hawq_segment_address_port</name>
        <value>40000</value>
        <description>The port of hawq segment.</description>
    </property>

    <property>
        <name>hawq_dfs_url</name>
        <value>node2:8020/hawq_default</value>
        <description>URL for accessing HDFS.</description>
    </property>

    <property>
        <name>hawq_master_directory</name>
        <value>/data/master</value>
        <description>The directory of hawq master.</description>
    </property>

    <property>
        <name>hawq_segment_directory</name>
        <value>/data/primary</value>
        <description>The directory of hawq segment.</description>
    </property>

    <property>
        <name>hawq_master_temp_directory</name>
        <value>/tmp</value>
        <description>The temporary directory reserved for hawq
master.</description>
    </property>

    <property>
        <name>hawq_segment_temp_directory</name>
        <value>/tmp</value>
        <description>The temporary directory reserved for hawq
segment.</description>
    </property>

    *<!-- HAWQ resource manager parameters -->*
*    <property>*
*        <name>hawq_resourcemanager_server_type</name>*
*        <value>yarn</value>*
*        <description>The resource manager type to start for allocating
resource.*
*                     'none' means hawq resource manager exclusively uses
whole*
*                     cluster; 'yarn' means hawq resource manager contacts
YARN*
*                     resource manager to negotiate resource.*
*        </description>*
*    </property>*

*    <property>*
*        <name>hawq_resourcemanager_segment_limit_memory_use</name>*
*        <value>64GB</value>*
*        <description>The limit of memory usage in a hawq segment when*
*                     hawq_resourcemanager_server_type is set 'none'.*
*        </description>*
*    </property>*

*    <property>*
*        <name>hawq_resourcemanager_segment_limit_core_use</name>*
*        <value>16</value>*
*        <description>The limit of virtual core usage in a hawq segment
when*
*                     hawq_resourcemanager_server_type is set 'none'.*
*        </description>*
*    </property>*

*    <property>*
*        <name>hawq_resourcemanager_yarn_resourcemanager_address</name>*
*        <value>node3:8050</value>*
*        <description>The address of YARN resource manager
server.</description>*
*    </property>*

*    <property>*
*
<name>hawq_resourcemanager_yarn_resourcemanager_scheduler_address</name>*
*        <value>node3:8030</value>*
*        <description>The address of YARN scheduler server.</description>*
*    </property>*

*    <property>*
*        <name>hawq_resourcemanager_yarn_queue</name>*
*        <value>default</value>*
*        <description>The YARN queue name to register hawq resource
manager.</description>*
*    </property>*

*    <property>*
*        <name>hawq_resourcemanager_yarn_application_name</name>*
*        <value>hawq</value>*
*        <description>The application name to register hawq resource
manager in YARN.</description>*
*    </property>*

*    <property>*
*        <name>default_segment_num</name>*
*       <value>16</value>*
*    </property>*
*    <property>*
*
<name>hawq_resourcemanager_query_vsegment_number_per_segment_limit</name>*
*       <value>8</value>*
*    </property>*
*</configuration>*





Daniel Lynch
Mon-Fri 9-5 PST
Office: 408 780 4498

On Sun, Nov 22, 2015 at 9:23 PM, Leon Zhang <leoncamel@gmail.com> wrote:

> Hi,
>
>    Is there any tutorial about how to deploy latest HAWQ 2.0-beta on YARN
> cluster?
>    I just rebuild the latest code from git, and after "hawq init cluster",
> it seems the segments does not work on YARN container. Any help will be
> appreciated.
>
>
> Thanks.
>

Mime
View raw message