impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Henry Robinson <he...@apache.org>
Subject Re: Broken build from Sentry
Date Tue, 20 Jun 2017 15:56:22 GMT
I was able to run a build with EE and FE tests with Sentry reverted to 5.12
- unless there are objections I'm going to post a patch to revert the
version bump.

On 20 June 2017 at 06:53, Thomas Tauber-Marshall <tmarshall@cloudera.com>
wrote:

> So we've had a successful run of the nightlies now, and I've uploaded the
> new jars to the s3 bucket, but Sentry still fails for some reason.
>
> I filed: https://issues.apache.org/jira/browse/IMPALA-5540 to track this
>
> On Tue, Jun 20, 2017 at 1:25 AM Alexander Kolbasov <akolb@cloudera.com>
> wrote:
>
> > Note that Apache upstream story is more complicated - there was a change
> > done upstream that refactored a bunch of Sentry code that will cause
> > similar issue (I think it is SENTRY-1205). The change is present in
> Sentry
> > master but not in upstream sentry HA branch.
> >
> > On Mon, Jun 19, 2017 at 11:02 PM, Dimitris Tsirogiannis <
> > dtsirogiannis@cloudera.com> wrote:
> >
> > > +Sasha, who I believe has more up-to-date information on this.
> > >
> > > On Mon, Jun 19, 2017 at 10:56 PM, Henry Robinson <henry@apache.org>
> > wrote:
> > >
> > >> FWIW, I've been able to start Sentry by setting:
> > >>
> > >> export IMPALA_SENTRY_VERSION=1.5.1-cdh5.12.0-SNAPSHOT
> > >>
> > >> (i.e. rolling back to the previous version of Sentry). I haven't yet
> > tried
> > >> to run tests - does anyone know an ETA for a fix coming out of
> Cloudera
> > >> for
> > >> the 5.13-SNAPSHOT? If it might be a while, we should consider
> regressing
> > >> the Sentry version to unblock checkins.
> > >>
> > >> On 19 June 2017 at 15:31, Tim Armstrong <tarmstrong@cloudera.com>
> > wrote:
> > >>
> > >> > It's unfortunately not that simple. The API change has been in
> Apache
> > >> > sentry
> > >> >
> > >> > So rolling back the API change temporarily solves the problem for
> > >> Cloudera,
> > >> > but we're going to have to deal with it at some point and get Impala
> > >> > building against both versions of the API.
> > >> >
> > >> > On Mon, Jun 19, 2017 at 2:55 PM, Thomas Tauber-Marshall <
> > >> > tmarshall@cloudera.com> wrote:
> > >> >
> > >> > > Yes, the Sentry team has been contacted and they're going to
be
> > >> rolling
> > >> > it
> > >> > > back.
> > >> > >
> > >> > > On Mon, Jun 19, 2017 at 4:53 PM Todd Lipcon <todd@cloudera.com>
> > >> wrote:
> > >> > >
> > >> > > > Quick question from a bystander: it seems like Sentry committed
> an
> > >> > > > API-incompatible change. Instead of fixing on the Impala
side,
> > >> should
> > >> > the
> > >> > > > Sentry project be notified that they may want to roll back
such
> a
> > >> > change?
> > >> > > > It seems like an error on their part to do such a thing
within a
> > >> minor
> > >> > > > version.
> > >> > > >
> > >> > > > On Mon, Jun 19, 2017 at 1:56 PM, Thomas Tauber-Marshall
<
> > >> > > > tmarshall@cloudera.com> wrote:
> > >> > > >
> > >> > > > > I'm working on getting the s3 jars updated, which presumably
> > will
> > >> fix
> > >> > > > that.
> > >> > > > >
> > >> > > > > The problem (to my understanding) is that the nightlies
> haven't
> > >> > passed
> > >> > > > > since the change went into Sentry and so the Jenkins
job that
> > >> > normally
> > >> > > > > produces the new jars is still pulling in old bits.
> > >> > > > >
> > >> > > > > I've been talking with releng and they expect the new
jars to
> be
> > >> > > > available
> > >> > > > > later today.
> > >> > > > >
> > >> > > > > On Mon, Jun 19, 2017 at 3:48 PM Tim Armstrong <
> > >> > tarmstrong@cloudera.com
> > >> > > >
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > Looks like the build still breaks when starting
up sentry
> > after
> > >> my
> > >> > > fix:
> > >> > > > > >
> > >> > > > > >
> > >> > > > http://jenkins.impala.io:8080/job/ubuntu-14.04-from-scratch/
> > >> > 1547/console
> > >> > > > > >
> > >> > > > > > *20:08:54*  --> Starting the Sentry Policy
Server*20:08:59*
> > >> Error
> > >> > in
> > >> > > > > > /home/ubuntu/Impala/testdata/bin/run-all.sh at
line 58:
> > >> > > > > > $IMPALA_HOME/testdata/bin/run-sentry-service.sh
>
> \*20:08:59*
> > +
> > >> > > > > > onexit*20:08:59* + df -m*20:08:59* Filesystem
    1M-blocks
> > >> Used
> > >> > > > > > Available Use% Mounted on*20:08:59* udev     
         15070
> > >>  1
> > >> > > > > > 15070   1% /dev*20:08:59* tmpfs              
3015     1
> > >> 3015
> > >> > > > > > 1% /run*20:08:59* /dev/xvda1        161129 22275
   132204
> > 15%
> > >> > > > > > /*20:08:59* none                   1     0   
     1   0%
> > >> > > > > > /sys/fs/cgroup*20:08:59* none                
  5     0
> > >>  5
> > >> > >  0%
> > >> > > > > > /run/lock*20:08:59* none               15075 
   1     15075
> > >>  1%
> > >> > > > > > /run/shm*20:08:59* none                 100  
  0       100
> >  0%
> > >> > > > > > /run/user*20:08:59* + free -m*20:08:59*      
       total
> > >> >  used
> > >> > > > > >      free     shared    buffers     cached*20:08:59*
Mem:
> > >> > > > > > 30148      19597      10550         11       
 91
> > >> > > 14323*20:08:59*
> > >> > > > > > -/+ buffers/cache:       5182      24965*20:08:59*
Swap:
> > >> >   0
> > >> > > > > >         0          0*20:08:59* + uptime -p*20:08:59*
up 45
> > >> > > > > > minutes*20:08:59* + rm -rf /home/ubuntu/Impala/logs_stati
> > >> > c*20:08:59*
> > >> > > +
> > >> > > > > > mkdir -p /home/ubuntu/Impala/logs_static*20:08:59*
+ cp -r
> -L
> > >> > > > > > /home/ubuntu/Impala/logs /home/ubuntu/Impala/logs_stati
> > >> c*20:08:59*
> > >> > > > > > Build step 'Execute shell' marked build as failure*20:08:59*
> > Set
> > >> > > build
> > >> > > > > > name.*20:08:59* New build name is '#1547
> > >> > > > > > refs/changes/22/7222/3'*20:08:59* Variable with
name
> > >> > > > > > 'BUILD_DISPLAY_NAME' already exists, current value:
'#1547
> > >> > > > > > refs/changes/22/7222/3', new value: '#1547
> > >> > > > > > refs/changes/22/7222/3'*20:09:12* Archiving
> > artifacts*20:09:21*
> > >> > > > > > Finished: FAILURE
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > On Mon, Jun 19, 2017 at 12:23 PM, Tim Armstrong
<
> > >> > > > tarmstrong@cloudera.com
> > >> > > > > >
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > > > It's unclear if there will be incompatibility
between the
> > >> updated
> > >> > > > > client
> > >> > > > > > > and the version of sentry we use for the
minicluster. I
> > kicked
> > >> > off
> > >> > > a
> > >> > > > > test
> > >> > > > > > > run to see if it works.
> > >> > > > > > >
> > >> > > > > > > On Mon, Jun 19, 2017 at 12:06 PM, Henry Robinson
<
> > >> > henry@apache.org
> > >> > > >
> > >> > > > > > wrote:
> > >> > > > > > >
> > >> > > > > > >> Presumably this will break GVO jobs as
well - should we
> > >> commit
> > >> > > Tim's
> > >> > > > > > patch
> > >> > > > > > >> to get us moving again while Alex works
on the root
> cause?
> > >> > > > > > >>
> > >> > > > > > >> On 19 June 2017 at 09:23, Alexander Behm
<
> > >> > alex.behm@cloudera.com>
> > >> > > > > > wrote:
> > >> > > > > > >>
> > >> > > > > > >> > Meanwhile, I'll work on fixing the
root cause:
> > >> > > > > > >> > https://issues.apache.org/jira/browse/IMPALA-5530
> > >> > > > > > >> >
> > >> > > > > > >> > On Mon, Jun 19, 2017 at 9:20 AM,
Tim Armstrong <
> > >> > > > > > tarmstrong@cloudera.com
> > >> > > > > > >> >
> > >> > > > > > >> > wrote:
> > >> > > > > > >> >
> > >> > > > > > >> > > You may have noticed that Impala
doesn't build this
> > >> morning
> > >> > > > > because
> > >> > > > > > >> of a
> > >> > > > > > >> > > sentry exception class no longer
existing. I was able
> > to
> > >> > > unblock
> > >> > > > > > >> myself
> > >> > > > > > >> > > with this change, if you want
to cherry-pick it:
> > >> > > > > > >> > > https://gerrit.cloudera.org/#/c/7222/
> > >> > > > > > >> > >
> > >> > > > > > >> >
> > >> > > > > > >>
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > --
> > >> > > > Todd Lipcon
> > >> > > > Software Engineer, Cloudera
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message