hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timo Burmeister <Timo.Burmeis...@freenet.ag>
Subject Re: Support Matrix for HAWQ
Date Tue, 05 Feb 2019 12:47:51 GMT
Hi Harald,

great, now it works fine !!!

thanks for your help!

Timo

Am 05.02.19, 11:50 schrieb "Harald Bögeholz" <harald.boegeholz@monash.edu.INVALID>:

    Hi Timo,
    
    Am 05.02.19 um 21:24 schrieb Timo Burmeister:
    > Hi Lav,
    > 
    > the problem is, that the ambari hawq install process wants to install "/usr/bin/yum
-d 0 -e 0 -y install hawq ".
    > 2019-02-05 10:22:53,291 - Package['hawq'] {'retry_on_repo_unavailability': False,
'retry_count': 5}
    > 2019-02-05 10:22:53,379 - Installing package hawq ('/usr/bin/yum -d 0 -e 0 -y install
hawq')
    > 2019-02-05 10:22:57,199 - Execution of '/usr/bin/yum -d 0 -e 0 -y install hawq' returned
1. Error: Nothing to do
    > 
    > But the package now should be 'apache-hawq'!
    > 
    > How can I change the the ambari hawq install, to process  "/usr/bin/yum -d 0 -e 0
-y install apache-hawq"
    
    I just solved that using Lav's help.
    
    You need to do two things:
    
    Modify file
    /var/lib/ambari-server/resources/common-services/HAWQ/2.0.0/metainfo.xml
    and change package name in the following section
    
          <osSpecifics>
            <osSpecific>
              <osFamily>any</osFamily>
              <packages>
                <package>
                  <name>hawq</name>
                </package>
              </packages>
            </osSpecific>
          </osSpecifics>
    
    from hawq to apache-hawq.
    
    Modify
    /var/lib/ambari-server/resources/common-services/HAWQ/2.0.0/package/scripts/hawq_constants.py
    as follows:
    
    [ec2-user@mds-ambari scripts]$ diff hawq_constants.py.bak hawq_constants.py
    42,43c42,43
    < hawq_home_dir = "/usr/local/hawq/"
    < hawq_config_dir = "/usr/local/hawq/etc/"
    ---
    > hawq_home_dir = "/usr/local/apache-hawq/"
    > hawq_config_dir = "/usr/local/apache-hawq/etc/"
    
    
    Thanks again to Lav for helping me to figure this out.
    
    
    Cheers,
    Harald
    
    > 
    > Thanks
    > Timo
    > 
    > Am 01.02.19, 19:08 schrieb "Lav Jain" <ljain@pivotal.io>:
    > 
    >     Glad to hear that you were able to install HAWQ using Ambari. The bug is in
    >     Ambari codebase as there has been no active development on the HAWQ common
    >     service in a while.
    >     
    >     Regards,
    >     
    >     
    >     *Lav Jain*
    >     *Pivotal Data*
    >     
    >     ljain@pivotal.io
    >     
    >     
    >     On Thu, Jan 31, 2019 at 11:53 PM Harald Bögeholz
    >     <harald.boegeholz@monash.edu.invalid> wrote:
    >     
    >     > Am 31.01.19 um 13:30 schrieb Lav Jain:
    >     > > Hi Harald,
    >     > >
    >     > > Most HAWQ constants are defined inside
    >     > >
    >     > /var/lib/ambari-server/resources/common-services/HAWQ/2.0.0/package/scripts/hawq_constants.py
    >     >
    >     > Thank you very much again. That helped a lot, I finally got HAWQ up and
    >     > running although I still had to manually fix some installation issues
    >     > like distributing ssh keys for gpadmin.
    >     >
    >     > > The last version that we used to deploy HAWQ with Ambari (ver 2.5.1)
was
    >     > > over a year ago. You may also take a look at http://www.oushu.io for
any
    >     > > updated instructions.
    >     >
    >     > I am wondering if these installation issues should be filed as a bug
    >     > report somewhere? But is it a bug in Ambari or a bug in the Ambari
    >     > plugin which is part of HAWQ? I don't know enough about these things to
    >     > determine that.
    >     >
    >     > Anyway, my biggest problem is solved for now.
    >     >
    >     > (PXF installation also failed, but I'll just ignore that until I
    >     > actually need it.)
    >     >
    >     >
    >     > Regards,
    >     > Harald
    >     >
    >     > >
    >     > >
    >     > > Regards,
    >     > >
    >     > >
    >     > > *Lav Jain*
    >     > > *Pivotal Data*
    >     > >
    >     > > ljain@pivotal.io
    >     > >
    >     > >
    >     > > On Tue, Jan 29, 2019 at 7:54 PM Harald Bögeholz
    >     > > <harald.boegeholz@monash.edu.invalid> wrote:
    >     > >
    >     > >> Hi Lav,
    >     > >>
    >     > >>
    >     > >> thanks for your quick help. I modified
    >     > >> /var/lib/ambari-server/resources/common-services/HAWQ/2.0.0/metainfo.xml
    >     > >> accordingly and restarted my Ambari server. Then I deleted the
services
    >     > >> PXF and HAWQ and tried adding them again.
    >     > >>
    >     > >> This time it failed a little later in the process with
    >     > >>
    >     > >> resource_management.core.exceptions.ExecutionFailed: Execution
of 'chown
    >     > >> -R gpadmin:gpadmin /usr/local/hawq/' returned 1. chown: cannot
access
    >     > >> '/usr/local/hawq/': No such file or directory
    >     > >>
    >     > >> Indeed this directory doesn't exist; instead I have
    >     > /usr/local/apache-hawq
    >     > >>
    >     > >> Apparently something else has to be fixed but I am completely unfamiliar
    >     > >> with the inner workings of Ambari.
    >     > >>
    >     > >> Is it possible at all to use Ambari to deploy HAWQ 2.4.0? I was
    >     > >> following Ruilong Huo's recommendation earlier on this list to
use HDP
    >     > >> 2.6.1 and Ambari 2.5.1.
    >     > >>
    >     > >>
    >     > >> Thanks
    >     > >> Harald
    >     > >>
    >     > >> Am 30.01.19 um 10:52 schrieb Lav Jain:
    >     > >>> I believe you need to
    >     > >>> modify
    >     > >> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/metainfo.xml
    >     > >>>
    >     > >>>       <osSpecifics>
    >     > >>>         <osSpecific>
    >     > >>>           <osFamily>any</osFamily>
    >     > >>>           <packages>
    >     > >>>             <package>
    >     > >>>               <name>hawq</name>
    >     > >>>             </package>
    >     > >>>           </packages>
    >     > >>>         </osSpecific>
    >     > >>>       </osSpecifics>
    >     > >>>
    >     > >>> Regards,
    >     > >>>
    >     > >>>
    >     > >>> *Lav Jain*
    >     > >>> *Pivotal Data*
    >     > >>>
    >     > >>> ljain@pivotal.io
    >     > >>>
    >     > >>>
    >     > >>> On Tue, Jan 29, 2019 at 3:10 PM Harald Bögeholz
    >     > >>> <harald.boegeholz@monash.edu.invalid> wrote:
    >     > >>>
    >     > >>>> Am 29.01.19 um 17:51 schrieb shenxl:
    >     > >>>>> Hi Harald,
    >     > >>>>>
    >     > >>>>> the HAWQ Ambari plugin isn’t included in HAWQ binary
distribition,
    >     > you
    >     > >>>> can get the plugin at .
    >     > >>>>>
    >     > https://github.com/apache/hawq/tree/master/contrib/hawq-ambari-plugin
    >     > >> <
    >     > >>>> https://github.com/apache/hawq/tree/master/contrib/hawq-ambari-plugin
    >     > >.
    >     > >>>>
    >     > >>>> Thank you for your help.
    >     > >>>>
    >     > >>>> I managed to install the Ambari plugin and setup a repository
for the
    >     > >>>> HAWQ binaries.
    >     > >>>>
    >     > >>>> Now deployment fails with this error:
    >     > >>>>
    >     > >>>> resource_management.core.exceptions.ExecutionFailed: Execution
of
    >     > >>>> '/usr/bin/yum -d 0 -e 0 -y install hawq' returned 1. Error:
Nothing to
    >     > >> do
    >     > >>>>
    >     > >>>> I think it is because it should be yum install apache-hawq,
but I
    >     > don't
    >     > >>>> know where to change this.
    >     > >>>>
    >     > >>>> Could I get another hint?
    >     > >>>>
    >     > >>>> Thanks
    >     > >>>> Harald
    >     > >>>>
    >     > >>>>>
    >     > >>>>>> ---------- Forwarded message ---------
    >     > >>>>>> From: Harald Bögeholz <harald.boegeholz@monash.edu.invalid>
    >     > >>>>>> Date: 2019年1月29日周二 上午9:32
    >     > >>>>>> Subject: Re: Support Matrix for HAWQ
    >     > >>>>>> To: <dev@hawq.apache.org <mailto:dev@hawq.apache.org>>
    >     > >>>>>>
    >     > >>>>>>
    >     > >>>>>> Am 23.01.19 um 15:01 schrieb Ruilong Huo:
    >     > >>>>>>> Hi Harald,
    >     > >>>>>>>
    >     > >>>>>>>
    >     > >>>>>>> If you prefer to use binary, you can get 2.4
release binary at
    >     > >>>> http://hawq.apache.org/#download <http://hawq.apache.org/#download>
    >     > >>>>>>> For the support matrix, it works with RHEL
6.4+/7.2+ x86_64, HDP
    >     > >>>> 2.5.3/2.6.1, and Ambari 2.4.2, 2.5.1.
    >     > >>>>>>
    >     > >>>>>> I have a Hadoop cluster using HDP 2.6.1/Ambari
2.5.1 up and running
    >     > >> now.
    >     > >>>>>>
    >     > >>>>>> The HAWQ binary distribition doesn't seem to include
the Ambari
    >     > >> plugin,
    >     > >>>>>> though.
    >     > >>>>>>
    >     > >>>>>> So I'm stuck and don't know how to deploy HAWQ.
    >     > >>>>>>
    >     > >>>>>> Could somebody point me to the right instructions?
    >     > >>>>>>
    >     > >>>>>>
    >     > >>>>>> Thanks
    >     > >>>>>> Harald
    >     > >>>>>>
    >     > >>>>>>>
    >     > >>>>>>>
    >     > >>>>>>> Best regards,
    >     > >>>>>>> Ruilong Huo
    >     > >>>>>>> At 2019-01-23 09:35:21, "Harald Bögeholz"
<
    >     > >> harald.boegeholz@monash.edu
    >     > >>>> <mailto:harald.boegeholz@monash.edu>.INVALID>
wrote:
    >     > >>>>>>>> Hello,
    >     > >>>>>>>>
    >     > >>>>>>>>
    >     > >>>>>>>> I have seen this discussion and I'm a bit
confused. I'm about to
    >     > >>>> setup a
    >     > >>>>>>>> new Hadoop cluster to run HAWQ on CentOS
7.
    >     > >>>>>>>>
    >     > >>>>>>>> What's the best way to get this up and
running?
    >     > >>>>>>>>
    >     > >>>>>>>> I was thinking about starting with Hortonworks
HDP. Which exact
    >     > >>>> version
    >     > >>>>>>>> should I use? I can't simply use the latest
one?
    >     > >>>>>>>>
    >     > >>>>>>>> When I have Hadoop running, can I deploy
HAWQ using Ambari? I
    >     > looked
    >     > >>>> in
    >     > >>>>>>>> the HAWQ wiki and there are instructions
for compiling HAWQ, but I
    >     > >> was
    >     > >>>>>>>> hoping to be able to just drop the binaries
somewhere for Ambari
    >     > to
    >     > >>>>>>>> deploy them?
    >     > >>>>>>>>
    >     > >>>>>>>>
    >     > >>>>>>>> Thanks for any help
    >     > >>>>>>>> Harald
    >     > >>>>>>>>
    >     > >>>>>>>>
    >     > >>>>>>>> Am 23.01.19 um 12:00 schrieb Yi JIN:
    >     > >>>>>>>>> Hi Timo,
    >     > >>>>>>>>>
    >     > >>>>>>>>> Yes, this is planned in version 2.5.
The issue for tracking this
    >     > is
    >     > >>>> going
    >     > >>>>>>>>> to be added into the release schedule.
I checked this with
    >     > release
    >     > >>>> manager
    >     > >>>>>>>>> for this requirement. Thanks.
    >     > >>>>>>>>>
    >     > >>>>>>>>> Best,
    >     > >>>>>>>>> Yi
    >     > >>>>>>>>>
    >     > >>>>>>>>> On Tue, Jan 15, 2019 at 8:20 PM Timo
Burmeister <
    >     > >>>> Timo.Burmeister@freenet.ag <mailto:Timo.Burmeister@freenet.ag>>
    >     > >>>>>>>>> wrote:
    >     > >>>>>>>>>
    >     > >>>>>>>>>> Hi Yi,
    >     > >>>>>>>>>>
    >     > >>>>>>>>>> do you think for HDP 3 we have
to wait for Apache HAWQ 2.5?
    >     > >>>>>>>>>>
    >     > >>>>>>>>>> Best,
    >     > >>>>>>>>>>
    >     > >>>>>>>>>> Timo
    >     > >>>>>>>>>>
    >     > >>>>>>>>>>
    >     > >>>>>>>>>> i.V. Timo Burmeister
    >     > >>>>>>>>>>
    >     > >>>>>>>>>> Head of Innovationsmanagement
    >     > >>>>>>>>>> IT-Strategy & New Technologies
    >     > >>>>>>>>>>
    >     > >>>>>>>>>>
    >     > >>>>>>>>>>
    >     > >>>>>>>>>> freenet AG
    >     > >>>>>>>>>> Deelbögenkamp 4c
    >     > >>>>>>>>>> D-22297 Hamburg
    >     > >>>>>>>>>>
    >     > >>>>>>>>>> Tel.: +49 (0) 4051306619
    >     > >>>>>>>>>> Fax: +49 (0) 4051306659
    >     > >>>>>>>>>>
    >     > >>>>>>>>>> timo.burmeister@freenet.ag <mailto:timo.burmeister@freenet.ag>
    >     > >>>>>>>>>>
    >     > >>>>>>>>>> www.mobilcom-debitel.de <http://www.mobilcom-debitel.de/>
<
    >     > >>>> http://www.mobilcom-debitel.de <http://www.mobilcom-debitel.de/>>
    >     > >>>>>>>>>> www.freenet.de <http://www.freenet.de/>
<http://www.freenet.de
    >     > <
    >     > >>>> http://www.freenet.de/>>
    >     > >>>>>>>>>> www.freenet-group.de <http://www.freenet-group.de/>
<
    >     > >>>> http://www.freenet-group.de <http://www.freenet-group.de/>>
    >     > >>>>>>>>>>
    >     > >>>>>>>>>> Vorstand: Christoph Vilanek (Vorsitzender),
Ingo Arnold, Stephan
    >     > >>>> Esch,
    >     > >>>>>>>>>> Antonius Fromme, Rickmann von Platen
    >     > >>>>>>>>>> Vorsitzender des Aufsichtsrats:
Prof. Dr. Helmut Thoma
    >     > >>>>>>>>>> Sitz der Gesellschaft: Büdelsdorf
    >     > >>>>>>>>>> HRB 7306 KI, Amtsgericht Kiel
    >     > >>>>>>>>>>
    >     > >>>>>>>>>>
    >     > >>>>>>>>>>
    >     > >>>>>>>>>>
    >     > >>>>>>>>>>
    >     > >>>>>>>>>> +++ freenet Group - EINE GRUPPE.
ALLE MÖGLICHKEITEN - Hier <
    >     > >>>>>>>>>> https://stellenangebote.freenet-group.de/
<
    >     > >>>> https://stellenangebote.freenet-group.de/>> suchen
wir noch
    >     > >> Verstärkung!
    >     > >>>>>>>>>> +++
    >     > >>>>>>>>>>
    >     > >>>>>>>>>>
    >     > >>>>>>>>>>
    >     > >>>>>>>>>>
    >     > >>>>>>>>>> Am 15.01.19, 04:07 schrieb "Yi
JIN" <yjin@apache.org <mailto:
    >     > >>>> yjin@apache.org>>:
    >     > >>>>>>>>>>
    >     > >>>>>>>>>>     Hi Timo,
    >     > >>>>>>>>>>
    >     > >>>>>>>>>>     I think HAWQ 2.4 supports HDP
2.6.1 and 2.5.3.
    >     > >>>>>>>>>>
    >     > >>>>>>>>>>     Best,
    >     > >>>>>>>>>>
    >     > >>>>>>>>>>     Yi
    >     > >>>>>>>>>>
    >     > >>>>>>>>>>     On Sat, Dec 22, 2018 at 2:55
AM Timo Burmeister <
    >     > >>>>>>>>>> Timo.Burmeister@freenet.ag <mailto:Timo.Burmeister@freenet.ag>>
    >     > >>>>>>>>>>     wrote:
    >     > >>>>>>>>>>
    >     > >>>>>>>>>>     > Hi !
    >     > >>>>>>>>>>     >
    >     > >>>>>>>>>>     > Where can I find the supported
Hadoop Version for the new
    >     > >>>> Apache
    >     > >>>>>>>>>> HAWQ 2.4
    >     > >>>>>>>>>>     > Release?
    >     > >>>>>>>>>>     >
    >     > >>>>>>>>>>     > Does Apache HAWQ 2.4 support
HDP 2.6 and HDP 3.0.1 ?
    >     > >>>>>>>>>>     >
    >     > >>>>>>>>>>     > Best regards
    >     > >>>>>>>>>>     > Timo
    >     > >>>>>>>>>>     >
    >     > >>>>>>>>>>     >
    >     > >>>>>>>>>>     >
    >     > >>>>>>>>>>
    >     > >>>>>>>>>>
    >     > >>>>>>>>>>
    >     > >>>>>>>>>
    >     > >>>>>
    >     > >>>>>
    >     > >>>>
    >     > >>>>
    >     > >>>
    >     > >>
    >     > >>
    >     > >
    >     >
    >     >
    >     
    > 
    
    

Mime
View raw message