hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harald Bögeholz <harald.boegeh...@monash.edu.INVALID>
Subject Re: Support Matrix for HAWQ
Date Tue, 05 Feb 2019 10:50:08 GMT
Hi Timo,

Am 05.02.19 um 21:24 schrieb Timo Burmeister:
> Hi Lav,
> 
> the problem is, that the ambari hawq install process wants to install "/usr/bin/yum -d
0 -e 0 -y install hawq ".
> 2019-02-05 10:22:53,291 - Package['hawq'] {'retry_on_repo_unavailability': False, 'retry_count':
5}
> 2019-02-05 10:22:53,379 - Installing package hawq ('/usr/bin/yum -d 0 -e 0 -y install
hawq')
> 2019-02-05 10:22:57,199 - Execution of '/usr/bin/yum -d 0 -e 0 -y install hawq' returned
1. Error: Nothing to do
> 
> But the package now should be 'apache-hawq'!
> 
> How can I change the the ambari hawq install, to process  "/usr/bin/yum -d 0 -e 0 -y
install apache-hawq"

I just solved that using Lav's help.

You need to do two things:

Modify file
/var/lib/ambari-server/resources/common-services/HAWQ/2.0.0/metainfo.xml
and change package name in the following section

      <osSpecifics>
        <osSpecific>
          <osFamily>any</osFamily>
          <packages>
            <package>
              <name>hawq</name>
            </package>
          </packages>
        </osSpecific>
      </osSpecifics>

from hawq to apache-hawq.

Modify
/var/lib/ambari-server/resources/common-services/HAWQ/2.0.0/package/scripts/hawq_constants.py
as follows:

[ec2-user@mds-ambari scripts]$ diff hawq_constants.py.bak hawq_constants.py
42,43c42,43
< hawq_home_dir = "/usr/local/hawq/"
< hawq_config_dir = "/usr/local/hawq/etc/"
---
> hawq_home_dir = "/usr/local/apache-hawq/"
> hawq_config_dir = "/usr/local/apache-hawq/etc/"


Thanks again to Lav for helping me to figure this out.


Cheers,
Harald

> 
> Thanks
> Timo
> 
> Am 01.02.19, 19:08 schrieb "Lav Jain" <ljain@pivotal.io>:
> 
>     Glad to hear that you were able to install HAWQ using Ambari. The bug is in
>     Ambari codebase as there has been no active development on the HAWQ common
>     service in a while.
>     
>     Regards,
>     
>     
>     *Lav Jain*
>     *Pivotal Data*
>     
>     ljain@pivotal.io
>     
>     
>     On Thu, Jan 31, 2019 at 11:53 PM Harald Bögeholz
>     <harald.boegeholz@monash.edu.invalid> wrote:
>     
>     > Am 31.01.19 um 13:30 schrieb Lav Jain:
>     > > Hi Harald,
>     > >
>     > > Most HAWQ constants are defined inside
>     > >
>     > /var/lib/ambari-server/resources/common-services/HAWQ/2.0.0/package/scripts/hawq_constants.py
>     >
>     > Thank you very much again. That helped a lot, I finally got HAWQ up and
>     > running although I still had to manually fix some installation issues
>     > like distributing ssh keys for gpadmin.
>     >
>     > > The last version that we used to deploy HAWQ with Ambari (ver 2.5.1) was
>     > > over a year ago. You may also take a look at http://www.oushu.io for any
>     > > updated instructions.
>     >
>     > I am wondering if these installation issues should be filed as a bug
>     > report somewhere? But is it a bug in Ambari or a bug in the Ambari
>     > plugin which is part of HAWQ? I don't know enough about these things to
>     > determine that.
>     >
>     > Anyway, my biggest problem is solved for now.
>     >
>     > (PXF installation also failed, but I'll just ignore that until I
>     > actually need it.)
>     >
>     >
>     > Regards,
>     > Harald
>     >
>     > >
>     > >
>     > > Regards,
>     > >
>     > >
>     > > *Lav Jain*
>     > > *Pivotal Data*
>     > >
>     > > ljain@pivotal.io
>     > >
>     > >
>     > > On Tue, Jan 29, 2019 at 7:54 PM Harald Bögeholz
>     > > <harald.boegeholz@monash.edu.invalid> wrote:
>     > >
>     > >> Hi Lav,
>     > >>
>     > >>
>     > >> thanks for your quick help. I modified
>     > >> /var/lib/ambari-server/resources/common-services/HAWQ/2.0.0/metainfo.xml
>     > >> accordingly and restarted my Ambari server. Then I deleted the services
>     > >> PXF and HAWQ and tried adding them again.
>     > >>
>     > >> This time it failed a little later in the process with
>     > >>
>     > >> resource_management.core.exceptions.ExecutionFailed: Execution of 'chown
>     > >> -R gpadmin:gpadmin /usr/local/hawq/' returned 1. chown: cannot access
>     > >> '/usr/local/hawq/': No such file or directory
>     > >>
>     > >> Indeed this directory doesn't exist; instead I have
>     > /usr/local/apache-hawq
>     > >>
>     > >> Apparently something else has to be fixed but I am completely unfamiliar
>     > >> with the inner workings of Ambari.
>     > >>
>     > >> Is it possible at all to use Ambari to deploy HAWQ 2.4.0? I was
>     > >> following Ruilong Huo's recommendation earlier on this list to use
HDP
>     > >> 2.6.1 and Ambari 2.5.1.
>     > >>
>     > >>
>     > >> Thanks
>     > >> Harald
>     > >>
>     > >> Am 30.01.19 um 10:52 schrieb Lav Jain:
>     > >>> I believe you need to
>     > >>> modify
>     > >> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/metainfo.xml
>     > >>>
>     > >>>       <osSpecifics>
>     > >>>         <osSpecific>
>     > >>>           <osFamily>any</osFamily>
>     > >>>           <packages>
>     > >>>             <package>
>     > >>>               <name>hawq</name>
>     > >>>             </package>
>     > >>>           </packages>
>     > >>>         </osSpecific>
>     > >>>       </osSpecifics>
>     > >>>
>     > >>> Regards,
>     > >>>
>     > >>>
>     > >>> *Lav Jain*
>     > >>> *Pivotal Data*
>     > >>>
>     > >>> ljain@pivotal.io
>     > >>>
>     > >>>
>     > >>> On Tue, Jan 29, 2019 at 3:10 PM Harald Bögeholz
>     > >>> <harald.boegeholz@monash.edu.invalid> wrote:
>     > >>>
>     > >>>> Am 29.01.19 um 17:51 schrieb shenxl:
>     > >>>>> Hi Harald,
>     > >>>>>
>     > >>>>> the HAWQ Ambari plugin isn’t included in HAWQ binary
distribition,
>     > you
>     > >>>> can get the plugin at .
>     > >>>>>
>     > https://github.com/apache/hawq/tree/master/contrib/hawq-ambari-plugin
>     > >> <
>     > >>>> https://github.com/apache/hawq/tree/master/contrib/hawq-ambari-plugin
>     > >.
>     > >>>>
>     > >>>> Thank you for your help.
>     > >>>>
>     > >>>> I managed to install the Ambari plugin and setup a repository
for the
>     > >>>> HAWQ binaries.
>     > >>>>
>     > >>>> Now deployment fails with this error:
>     > >>>>
>     > >>>> resource_management.core.exceptions.ExecutionFailed: Execution
of
>     > >>>> '/usr/bin/yum -d 0 -e 0 -y install hawq' returned 1. Error:
Nothing to
>     > >> do
>     > >>>>
>     > >>>> I think it is because it should be yum install apache-hawq,
but I
>     > don't
>     > >>>> know where to change this.
>     > >>>>
>     > >>>> Could I get another hint?
>     > >>>>
>     > >>>> Thanks
>     > >>>> Harald
>     > >>>>
>     > >>>>>
>     > >>>>>> ---------- Forwarded message ---------
>     > >>>>>> From: Harald Bögeholz <harald.boegeholz@monash.edu.invalid>
>     > >>>>>> Date: 2019年1月29日周二 上午9:32
>     > >>>>>> Subject: Re: Support Matrix for HAWQ
>     > >>>>>> To: <dev@hawq.apache.org <mailto:dev@hawq.apache.org>>
>     > >>>>>>
>     > >>>>>>
>     > >>>>>> Am 23.01.19 um 15:01 schrieb Ruilong Huo:
>     > >>>>>>> Hi Harald,
>     > >>>>>>>
>     > >>>>>>>
>     > >>>>>>> If you prefer to use binary, you can get 2.4 release
binary at
>     > >>>> http://hawq.apache.org/#download <http://hawq.apache.org/#download>
>     > >>>>>>> For the support matrix, it works with RHEL 6.4+/7.2+
x86_64, HDP
>     > >>>> 2.5.3/2.6.1, and Ambari 2.4.2, 2.5.1.
>     > >>>>>>
>     > >>>>>> I have a Hadoop cluster using HDP 2.6.1/Ambari 2.5.1
up and running
>     > >> now.
>     > >>>>>>
>     > >>>>>> The HAWQ binary distribition doesn't seem to include
the Ambari
>     > >> plugin,
>     > >>>>>> though.
>     > >>>>>>
>     > >>>>>> So I'm stuck and don't know how to deploy HAWQ.
>     > >>>>>>
>     > >>>>>> Could somebody point me to the right instructions?
>     > >>>>>>
>     > >>>>>>
>     > >>>>>> Thanks
>     > >>>>>> Harald
>     > >>>>>>
>     > >>>>>>>
>     > >>>>>>>
>     > >>>>>>> Best regards,
>     > >>>>>>> Ruilong Huo
>     > >>>>>>> At 2019-01-23 09:35:21, "Harald Bögeholz" <
>     > >> harald.boegeholz@monash.edu
>     > >>>> <mailto:harald.boegeholz@monash.edu>.INVALID> wrote:
>     > >>>>>>>> Hello,
>     > >>>>>>>>
>     > >>>>>>>>
>     > >>>>>>>> I have seen this discussion and I'm a bit confused.
I'm about to
>     > >>>> setup a
>     > >>>>>>>> new Hadoop cluster to run HAWQ on CentOS 7.
>     > >>>>>>>>
>     > >>>>>>>> What's the best way to get this up and running?
>     > >>>>>>>>
>     > >>>>>>>> I was thinking about starting with Hortonworks
HDP. Which exact
>     > >>>> version
>     > >>>>>>>> should I use? I can't simply use the latest
one?
>     > >>>>>>>>
>     > >>>>>>>> When I have Hadoop running, can I deploy HAWQ
using Ambari? I
>     > looked
>     > >>>> in
>     > >>>>>>>> the HAWQ wiki and there are instructions for
compiling HAWQ, but I
>     > >> was
>     > >>>>>>>> hoping to be able to just drop the binaries
somewhere for Ambari
>     > to
>     > >>>>>>>> deploy them?
>     > >>>>>>>>
>     > >>>>>>>>
>     > >>>>>>>> Thanks for any help
>     > >>>>>>>> Harald
>     > >>>>>>>>
>     > >>>>>>>>
>     > >>>>>>>> Am 23.01.19 um 12:00 schrieb Yi JIN:
>     > >>>>>>>>> Hi Timo,
>     > >>>>>>>>>
>     > >>>>>>>>> Yes, this is planned in version 2.5. The
issue for tracking this
>     > is
>     > >>>> going
>     > >>>>>>>>> to be added into the release schedule.
I checked this with
>     > release
>     > >>>> manager
>     > >>>>>>>>> for this requirement. Thanks.
>     > >>>>>>>>>
>     > >>>>>>>>> Best,
>     > >>>>>>>>> Yi
>     > >>>>>>>>>
>     > >>>>>>>>> On Tue, Jan 15, 2019 at 8:20 PM Timo Burmeister
<
>     > >>>> Timo.Burmeister@freenet.ag <mailto:Timo.Burmeister@freenet.ag>>
>     > >>>>>>>>> wrote:
>     > >>>>>>>>>
>     > >>>>>>>>>> Hi Yi,
>     > >>>>>>>>>>
>     > >>>>>>>>>> do you think for HDP 3 we have to wait
for Apache HAWQ 2.5?
>     > >>>>>>>>>>
>     > >>>>>>>>>> Best,
>     > >>>>>>>>>>
>     > >>>>>>>>>> Timo
>     > >>>>>>>>>>
>     > >>>>>>>>>>
>     > >>>>>>>>>> i.V. Timo Burmeister
>     > >>>>>>>>>>
>     > >>>>>>>>>> Head of Innovationsmanagement
>     > >>>>>>>>>> IT-Strategy & New Technologies
>     > >>>>>>>>>>
>     > >>>>>>>>>>
>     > >>>>>>>>>>
>     > >>>>>>>>>> freenet AG
>     > >>>>>>>>>> Deelbögenkamp 4c
>     > >>>>>>>>>> D-22297 Hamburg
>     > >>>>>>>>>>
>     > >>>>>>>>>> Tel.: +49 (0) 4051306619
>     > >>>>>>>>>> Fax: +49 (0) 4051306659
>     > >>>>>>>>>>
>     > >>>>>>>>>> timo.burmeister@freenet.ag <mailto:timo.burmeister@freenet.ag>
>     > >>>>>>>>>>
>     > >>>>>>>>>> www.mobilcom-debitel.de <http://www.mobilcom-debitel.de/>
<
>     > >>>> http://www.mobilcom-debitel.de <http://www.mobilcom-debitel.de/>>
>     > >>>>>>>>>> www.freenet.de <http://www.freenet.de/>
<http://www.freenet.de
>     > <
>     > >>>> http://www.freenet.de/>>
>     > >>>>>>>>>> www.freenet-group.de <http://www.freenet-group.de/>
<
>     > >>>> http://www.freenet-group.de <http://www.freenet-group.de/>>
>     > >>>>>>>>>>
>     > >>>>>>>>>> Vorstand: Christoph Vilanek (Vorsitzender),
Ingo Arnold, Stephan
>     > >>>> Esch,
>     > >>>>>>>>>> Antonius Fromme, Rickmann von Platen
>     > >>>>>>>>>> Vorsitzender des Aufsichtsrats: Prof.
Dr. Helmut Thoma
>     > >>>>>>>>>> Sitz der Gesellschaft: Büdelsdorf
>     > >>>>>>>>>> HRB 7306 KI, Amtsgericht Kiel
>     > >>>>>>>>>>
>     > >>>>>>>>>>
>     > >>>>>>>>>>
>     > >>>>>>>>>>
>     > >>>>>>>>>>
>     > >>>>>>>>>> +++ freenet Group - EINE GRUPPE. ALLE
MÖGLICHKEITEN - Hier <
>     > >>>>>>>>>> https://stellenangebote.freenet-group.de/
<
>     > >>>> https://stellenangebote.freenet-group.de/>> suchen wir
noch
>     > >> Verstärkung!
>     > >>>>>>>>>> +++
>     > >>>>>>>>>>
>     > >>>>>>>>>>
>     > >>>>>>>>>>
>     > >>>>>>>>>>
>     > >>>>>>>>>> Am 15.01.19, 04:07 schrieb "Yi JIN"
<yjin@apache.org <mailto:
>     > >>>> yjin@apache.org>>:
>     > >>>>>>>>>>
>     > >>>>>>>>>>     Hi Timo,
>     > >>>>>>>>>>
>     > >>>>>>>>>>     I think HAWQ 2.4 supports HDP 2.6.1
and 2.5.3.
>     > >>>>>>>>>>
>     > >>>>>>>>>>     Best,
>     > >>>>>>>>>>
>     > >>>>>>>>>>     Yi
>     > >>>>>>>>>>
>     > >>>>>>>>>>     On Sat, Dec 22, 2018 at 2:55 AM
Timo Burmeister <
>     > >>>>>>>>>> Timo.Burmeister@freenet.ag <mailto:Timo.Burmeister@freenet.ag>>
>     > >>>>>>>>>>     wrote:
>     > >>>>>>>>>>
>     > >>>>>>>>>>     > Hi !
>     > >>>>>>>>>>     >
>     > >>>>>>>>>>     > Where can I find the supported
Hadoop Version for the new
>     > >>>> Apache
>     > >>>>>>>>>> HAWQ 2.4
>     > >>>>>>>>>>     > Release?
>     > >>>>>>>>>>     >
>     > >>>>>>>>>>     > Does Apache HAWQ 2.4 support
HDP 2.6 and HDP 3.0.1 ?
>     > >>>>>>>>>>     >
>     > >>>>>>>>>>     > Best regards
>     > >>>>>>>>>>     > Timo
>     > >>>>>>>>>>     >
>     > >>>>>>>>>>     >
>     > >>>>>>>>>>     >
>     > >>>>>>>>>>
>     > >>>>>>>>>>
>     > >>>>>>>>>>
>     > >>>>>>>>>
>     > >>>>>
>     > >>>>>
>     > >>>>
>     > >>>>
>     > >>>
>     > >>
>     > >>
>     > >
>     >
>     >
>     
> 


Mime
View raw message