impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Armstrong <tarmstr...@cloudera.com>
Subject Re: Broken build from Sentry
Date Tue, 20 Jun 2017 16:44:30 GMT
Yeah my change should build against either version by design.

On 20 Jun. 2017 9:21 am, "Henry Robinson" <henry@apache.org> wrote:

> Yes, I did. AFAICT it worked fine.
>
> On 20 June 2017 at 09:19, Alexander Behm <alex.behm@cloudera.com> wrote:
>
> > Henry, did you try the revert on top of Tim's already-checked-in change?
> >
> > On Tue, Jun 20, 2017 at 9:18 AM, Alexander Behm <alex.behm@cloudera.com>
> > wrote:
> >
> > > Let's revert the version to buy us some time. That solution is a
> ticking
> > > time bomb though since that version will disappear soon.
> > >
> > > On Tue, Jun 20, 2017 at 8:56 AM, Henry Robinson <henry@apache.org>
> > wrote:
> > >
> > >> I was able to run a build with EE and FE tests with Sentry reverted to
> > >> 5.12
> > >> - unless there are objections I'm going to post a patch to revert the
> > >> version bump.
> > >>
> > >> On 20 June 2017 at 06:53, Thomas Tauber-Marshall <
> > tmarshall@cloudera.com>
> > >> wrote:
> > >>
> > >> > So we've had a successful run of the nightlies now, and I've
> uploaded
> > >> the
> > >> > new jars to the s3 bucket, but Sentry still fails for some reason.
> > >> >
> > >> > I filed: https://issues.apache.org/jira/browse/IMPALA-5540 to track
> > >> this
> > >> >
> > >> > On Tue, Jun 20, 2017 at 1:25 AM Alexander Kolbasov <
> > akolb@cloudera.com>
> > >> > wrote:
> > >> >
> > >> > > Note that Apache upstream story is more complicated - there was
a
> > >> change
> > >> > > done upstream that refactored a bunch of Sentry code that will
> cause
> > >> > > similar issue (I think it is SENTRY-1205). The change is present
> in
> > >> > Sentry
> > >> > > master but not in upstream sentry HA branch.
> > >> > >
> > >> > > On Mon, Jun 19, 2017 at 11:02 PM, Dimitris Tsirogiannis <
> > >> > > dtsirogiannis@cloudera.com> wrote:
> > >> > >
> > >> > > > +Sasha, who I believe has more up-to-date information on
this.
> > >> > > >
> > >> > > > On Mon, Jun 19, 2017 at 10:56 PM, Henry Robinson <
> > henry@apache.org>
> > >> > > wrote:
> > >> > > >
> > >> > > >> FWIW, I've been able to start Sentry by setting:
> > >> > > >>
> > >> > > >> export IMPALA_SENTRY_VERSION=1.5.1-cdh5.12.0-SNAPSHOT
> > >> > > >>
> > >> > > >> (i.e. rolling back to the previous version of Sentry).
I
> haven't
> > >> yet
> > >> > > tried
> > >> > > >> to run tests - does anyone know an ETA for a fix coming
out of
> > >> > Cloudera
> > >> > > >> for
> > >> > > >> the 5.13-SNAPSHOT? If it might be a while, we should
consider
> > >> > regressing
> > >> > > >> the Sentry version to unblock checkins.
> > >> > > >>
> > >> > > >> On 19 June 2017 at 15:31, Tim Armstrong <
> tarmstrong@cloudera.com
> > >
> > >> > > wrote:
> > >> > > >>
> > >> > > >> > It's unfortunately not that simple. The API change
has been
> in
> > >> > Apache
> > >> > > >> > sentry
> > >> > > >> >
> > >> > > >> > So rolling back the API change temporarily solves
the problem
> > for
> > >> > > >> Cloudera,
> > >> > > >> > but we're going to have to deal with it at some
point and get
> > >> Impala
> > >> > > >> > building against both versions of the API.
> > >> > > >> >
> > >> > > >> > On Mon, Jun 19, 2017 at 2:55 PM, Thomas Tauber-Marshall
<
> > >> > > >> > tmarshall@cloudera.com> wrote:
> > >> > > >> >
> > >> > > >> > > Yes, the Sentry team has been contacted and
they're going
> to
> > be
> > >> > > >> rolling
> > >> > > >> > it
> > >> > > >> > > back.
> > >> > > >> > >
> > >> > > >> > > On Mon, Jun 19, 2017 at 4:53 PM Todd Lipcon
<
> > todd@cloudera.com
> > >> >
> > >> > > >> wrote:
> > >> > > >> > >
> > >> > > >> > > > Quick question from a bystander: it seems
like Sentry
> > >> committed
> > >> > an
> > >> > > >> > > > API-incompatible change. Instead of fixing
on the Impala
> > >> side,
> > >> > > >> should
> > >> > > >> > the
> > >> > > >> > > > Sentry project be notified that they
may want to roll
> back
> > >> such
> > >> > a
> > >> > > >> > change?
> > >> > > >> > > > It seems like an error on their part
to do such a thing
> > >> within a
> > >> > > >> minor
> > >> > > >> > > > version.
> > >> > > >> > > >
> > >> > > >> > > > On Mon, Jun 19, 2017 at 1:56 PM, Thomas
Tauber-Marshall <
> > >> > > >> > > > tmarshall@cloudera.com> wrote:
> > >> > > >> > > >
> > >> > > >> > > > > I'm working on getting the s3 jars
updated, which
> > >> presumably
> > >> > > will
> > >> > > >> fix
> > >> > > >> > > > that.
> > >> > > >> > > > >
> > >> > > >> > > > > The problem (to my understanding)
is that the nightlies
> > >> > haven't
> > >> > > >> > passed
> > >> > > >> > > > > since the change went into Sentry
and so the Jenkins
> job
> > >> that
> > >> > > >> > normally
> > >> > > >> > > > > produces the new jars is still pulling
in old bits.
> > >> > > >> > > > >
> > >> > > >> > > > > I've been talking with releng and
they expect the new
> > jars
> > >> to
> > >> > be
> > >> > > >> > > > available
> > >> > > >> > > > > later today.
> > >> > > >> > > > >
> > >> > > >> > > > > On Mon, Jun 19, 2017 at 3:48 PM
Tim Armstrong <
> > >> > > >> > tarmstrong@cloudera.com
> > >> > > >> > > >
> > >> > > >> > > > > wrote:
> > >> > > >> > > > >
> > >> > > >> > > > > > Looks like the build still
breaks when starting up
> > sentry
> > >> > > after
> > >> > > >> my
> > >> > > >> > > fix:
> > >> > > >> > > > > >
> > >> > > >> > > > > >
> > >> > > >> > > > http://jenkins.impala.io:8080/
> > job/ubuntu-14.04-from-scratch/
> > >> > > >> > 1547/console
> > >> > > >> > > > > >
> > >> > > >> > > > > > *20:08:54*  --> Starting
the Sentry Policy
> > >> Server*20:08:59*
> > >> > > >> Error
> > >> > > >> > in
> > >> > > >> > > > > > /home/ubuntu/Impala/testdata/bin/run-all.sh
at line
> > 58:
> > >> > > >> > > > > > $IMPALA_HOME/testdata/bin/run-sentry-service.sh
>
> > >> > \*20:08:59*
> > >> > > +
> > >> > > >> > > > > > onexit*20:08:59* + df -m*20:08:59*
Filesystem
> > >>  1M-blocks
> > >> > > >> Used
> > >> > > >> > > > > > Available Use% Mounted on*20:08:59*
udev
> > >>  15070
> > >> > > >>  1
> > >> > > >> > > > > > 15070   1% /dev*20:08:59* tmpfs
              3015
> >  1
> > >> > > >> 3015
> > >> > > >> > > > > > 1% /run*20:08:59* /dev/xvda1
       161129 22275
> > >> 132204
> > >> > > 15%
> > >> > > >> > > > > > /*20:08:59* none          
        1     0         1
> >  0%
> > >> > > >> > > > > > /sys/fs/cgroup*20:08:59* none
                  5
>  0
> > >> > > >>  5
> > >> > > >> > >  0%
> > >> > > >> > > > > > /run/lock*20:08:59* none  
            15075     1
> > >>  15075
> > >> > > >>  1%
> > >> > > >> > > > > > /run/shm*20:08:59* none   
             100     0
> > >>  100
> > >> > >  0%
> > >> > > >> > > > > > /run/user*20:08:59* + free
-m*20:08:59*
> > >> total
> > >> > > >> >  used
> > >> > > >> > > > > >      free     shared    buffers
    cached*20:08:59*
> > Mem:
> > >> > > >> > > > > > 30148      19597      10550
        11         91
> > >> > > >> > > 14323*20:08:59*
> > >> > > >> > > > > > -/+ buffers/cache:       5182
     24965*20:08:59*
> > Swap:
> > >> > > >> >   0
> > >> > > >> > > > > >         0          0*20:08:59*
+ uptime -p*20:08:59*
> up
> > >> 45
> > >> > > >> > > > > > minutes*20:08:59* + rm -rf
> > /home/ubuntu/Impala/logs_stati
> > >> > > >> > c*20:08:59*
> > >> > > >> > > +
> > >> > > >> > > > > > mkdir -p /home/ubuntu/Impala/logs_static*20:08:59*
+
> > cp
> > >> -r
> > >> > -L
> > >> > > >> > > > > > /home/ubuntu/Impala/logs
> /home/ubuntu/Impala/logs_stati
> > >> > > >> c*20:08:59*
> > >> > > >> > > > > > Build step 'Execute shell'
marked build as
> > >> failure*20:08:59*
> > >> > > Set
> > >> > > >> > > build
> > >> > > >> > > > > > name.*20:08:59* New build name
is '#1547
> > >> > > >> > > > > > refs/changes/22/7222/3'*20:08:59*
Variable with name
> > >> > > >> > > > > > 'BUILD_DISPLAY_NAME' already
exists, current value:
> > >> '#1547
> > >> > > >> > > > > > refs/changes/22/7222/3', new
value: '#1547
> > >> > > >> > > > > > refs/changes/22/7222/3'*20:09:12*
Archiving
> > >> > > artifacts*20:09:21*
> > >> > > >> > > > > > Finished: FAILURE
> > >> > > >> > > > > >
> > >> > > >> > > > > >
> > >> > > >> > > > > > On Mon, Jun 19, 2017 at 12:23
PM, Tim Armstrong <
> > >> > > >> > > > tarmstrong@cloudera.com
> > >> > > >> > > > > >
> > >> > > >> > > > > > wrote:
> > >> > > >> > > > > >
> > >> > > >> > > > > > > It's unclear if there
will be incompatibility
> between
> > >> the
> > >> > > >> updated
> > >> > > >> > > > > client
> > >> > > >> > > > > > > and the version of sentry
we use for the
> > minicluster. I
> > >> > > kicked
> > >> > > >> > off
> > >> > > >> > > a
> > >> > > >> > > > > test
> > >> > > >> > > > > > > run to see if it works.
> > >> > > >> > > > > > >
> > >> > > >> > > > > > > On Mon, Jun 19, 2017 at
12:06 PM, Henry Robinson <
> > >> > > >> > henry@apache.org
> > >> > > >> > > >
> > >> > > >> > > > > > wrote:
> > >> > > >> > > > > > >
> > >> > > >> > > > > > >> Presumably this will
break GVO jobs as well -
> should
> > >> we
> > >> > > >> commit
> > >> > > >> > > Tim's
> > >> > > >> > > > > > patch
> > >> > > >> > > > > > >> to get us moving again
while Alex works on the
> root
> > >> > cause?
> > >> > > >> > > > > > >>
> > >> > > >> > > > > > >> On 19 June 2017 at
09:23, Alexander Behm <
> > >> > > >> > alex.behm@cloudera.com>
> > >> > > >> > > > > > wrote:
> > >> > > >> > > > > > >>
> > >> > > >> > > > > > >> > Meanwhile, I'll
work on fixing the root cause:
> > >> > > >> > > > > > >> > https://issues.apache.org/
> jira/browse/IMPALA-5530
> > >> > > >> > > > > > >> >
> > >> > > >> > > > > > >> > On Mon, Jun 19,
2017 at 9:20 AM, Tim Armstrong <
> > >> > > >> > > > > > tarmstrong@cloudera.com
> > >> > > >> > > > > > >> >
> > >> > > >> > > > > > >> > wrote:
> > >> > > >> > > > > > >> >
> > >> > > >> > > > > > >> > > You may
have noticed that Impala doesn't build
> > >> this
> > >> > > >> morning
> > >> > > >> > > > > because
> > >> > > >> > > > > > >> of a
> > >> > > >> > > > > > >> > > sentry exception
class no longer existing. I
> was
> > >> able
> > >> > > to
> > >> > > >> > > unblock
> > >> > > >> > > > > > >> myself
> > >> > > >> > > > > > >> > > with this
change, if you want to cherry-pick
> it:
> > >> > > >> > > > > > >> > > https://gerrit.cloudera.org/#/c/7222/
> > >> > > >> > > > > > >> > >
> > >> > > >> > > > > > >> >
> > >> > > >> > > > > > >>
> > >> > > >> > > > > > >
> > >> > > >> > > > > > >
> > >> > > >> > > > > >
> > >> > > >> > > > >
> > >> > > >> > > >
> > >> > > >> > > >
> > >> > > >> > > >
> > >> > > >> > > > --
> > >> > > >> > > > Todd Lipcon
> > >> > > >> > > > Software Engineer, Cloudera
> > >> > > >> > > >
> > >> > > >> > >
> > >> > > >> >
> > >> > > >>
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message