hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Radar Lei <r...@pivotal.io>
Subject Re: how hawq use HA hdfs
Date Thu, 31 May 2018 02:27:21 GMT
Have you made changes to  HAWQ configuration file 'hdfs-client.xml'?

Regards,
Radar

On Thu, May 31, 2018 at 10:07 AM, <xiang.dai@sky-data.cn> wrote:

> Change the following parameter in the $GPHOME/etc/hawq-site.xml file:
>
> <property>
>     <name>hawq_dfs_url</name>
>     <value>hdpcluster/hawq_default</value>
>     <description>URL for accessing HDFS.</description></property>
>
> In the listing above:
>
>    - Replace hdpcluster with the actual service ID that is configured in
>    HDFS.
>    - Replace /hawq_default with the directory you want to use for storing
>    data on HDFS. Make sure this directory exists and is writable.
>
> In my case, i think that hdpcluster is "dx" which defined in
> core-site.xml.
>
> So i use "dx/hawq_default" but failed.
>
> Could anyone help me about this?
>
> Thanks very much.
>
>
> ------------------------------
> *From: *"Radar Lei" <rlei@pivotal.io>
> *To: *"user" <user@hawq.incubator.apache.org>, "dev" <
> dev@hawq.incubator.apache.org>
> *Sent: *Thursday, May 31, 2018 9:50:39 AM
>
> *Subject: *Re: how hawq use HA hdfs
>
> If you are installing a new HAWQ, then file space move is not required.
> I think  HAWQ will treat the host string as an url unless you configured
> HAWQ hdfs HA correctly. So please verify if you missed any other steps.
>
> Regards,
> Radar
>
> On Thu, May 31, 2018 at 9:40 AM, <xiang.dai@sky-data.cn> wrote:
>
>> I think that move filespace is the action that hawq has installed before
>> HA hdfs,
>> but in my case, i installed HA hdfs then installed hawq.
>>
>> In this way, i think there is no need to move filespace.
>>
>> Besides, my question is that why URI can not work as expected?
>> Or which URI i should use?
>>
>> ------------------------------
>> *From: *"Radar Lei" <rlei@pivotal.io>
>> *To: *"user" <user@hawq.incubator.apache.org>
>> *Sent: *Wednesday, May 30, 2018 8:30:01 PM
>> *Subject: *Re: how hawq use HA hdfs
>>
>> As the steps in the document you mentioned, you need to finish all the
>> steps to convert your HAWQ cluster to HA mode.
>> E.g. Do filespace move, make changes in hdfs-client.xml etc.
>>
>> Regards,
>> Radar
>>
>> On Wed, May 30, 2018 at 7:44 PM, <xiang.dai@sky-data.cn> wrote:
>>
>>> Hi!
>>>
>>> I has setting HA hdfs using ZKFC with below config:
>>>
>>> <configuration>
>>>     <property>
>>>         <name>fs.defaultFS</name>
>>>         <value>hdfs://dx</value>
>>>     </property>
>>>  <property>
>>>    <name>ha.zookeeper.quorum</name>
>>>    <value>192.168.60.24 <callto:192.168.60.24>:2181,192.168.60.32
>>> <callto:2181,192.168.60.32>:2181,192.168.60.37
>>> <callto:2181,192.168.60.37>:2181</value>
>>>  </property>
>>> </configuration>
>>>
>>> Then i install hawq in another node(not any node installed hdfs).
>>>
>>> I follow http://hawq.incubator.apache.org/docs/userguide/2.3.0.0-
>>> incubating/admin/HAWQFilespacesandHighAvailabilityEnabledHDFS.html
>>> to set `hawq_dfs_url` as "dx/hawq_default", but init failed when check
>>> if hdfs path is available:
>>> [WARNING]:-ERROR: Can not connect to 'hdfs://dx:0'
>>>
>>> Then i change `hawq_dfs_url` to "dx:8020/hawq_default", still failed:
>>> ERROR Failed to setup RPC connection to "dx:8020" caused by:
>>> TcpSocket.cpp: 171: HdfsNetworkConnectException: Failed to resolve
>>> address "dx:8020" Name or service not known
>>>
>>> It consider "dx" as hostname, which is not expected.
>>>
>>> I also test "hdfs://dx", "file://dx", all failed.
>>>
>>> I know that "192.168.60.24 <callto:192.168.60.24>:8020" can work, but
>>> it does not use HA hdfs.
>>>
>>> How can hawq use HA hdfs?
>>>
>>> Thanks
>>>
>>
>>
>> --
>> 戴翔
>> 南京天数信息科技有限公司
>> 电话: +86 1 <callto:+86%2015062274256>3382776490
>> 公司官网: www.sky-data.cn
>> 免费使用天数润科智能计算平台 SkyDiscovery
>>
>
>
> --
> 戴翔
> 南京天数信息科技有限公司
> 电话: +86 1 <callto:+86%2015062274256>3382776490
> 公司官网: www.sky-data.cn
> 免费使用天数润科智能计算平台 SkyDiscovery
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message