impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim Apple <jbap...@cloudera.com>
Subject Re: Upstreaming ppc64le patches for native-toolchain
Date Mon, 17 Apr 2017 22:11:28 GMT
I'm OK with us calling this "experimental" (once the patches are in and we
think it works) and letting patches continue to land with our current
testing regime.

Later, if we add daily test reports with contributor-managed hardware or
even pre-commit testing on jenkins.impala.io in a VM, maybe we could
upgrade to other names than "experimental", like "beta" or "best-effort",
if we still want to have caveats.

How does that sound to everyone else?

On Mon, Apr 17, 2017 at 2:54 PM, Tim Armstrong <tarmstrong@cloudera.com>
wrote:

> Something like that makes sense - I think it should be an experimental or
> unsupported feature until if/when we have a critical mass of committers to
> maintain it.
>
> On Mon, Apr 17, 2017 at 2:46 PM, Jim Apple <jbapple@cloudera.com> wrote:
>
> > That makes sense to me. It would be good for us to provide support
> without
> > completely focusing all development effort on a HW platform with fewer
> > users.
> >
> > If ppc64le support lands in 2.10, and between 2.10 and 2.11 the ppc64le
> > tests start breaking for reasons nobody with HW access can debug, should
> we
> > say in 2.11 release notes that ppc64le is no longer supported?
> >
> > Or perhaps, even in the first release where we think it works, we should
> > spell it out as a platform with only "best-effort" support, that way we
> > don't have to retract support?
> >
> > On Mon, Apr 17, 2017 at 2:41 PM, Marcel Kornacker <marcel@cloudera.com>
> > wrote:
> >
> > > My main concern is that we don't unduly burden the development process.
> > As
> > > such, it makes a lot of sense to keep the PPC tests out of the regular
> > > tests and have the party that's interested in the PPC tests create the
> > > infrastructure and run those tests.
> > >
> > > On Mon, Apr 17, 2017 at 2:30 PM, Jim Apple <jbapple@cloudera.com>
> wrote:
> > >
> > > > One concern I have is sustainability. If only one Impala contributor
> > can
> > > > work with ppc64le, and that contributor is not as seasoned as some of
> > the
> > > > other committers, what happens if ppc64le breaks and the one person
> > with
> > > VM
> > > > access can't fix it?
> > > >
> > > > Part of my concern is just how flaky the current tests are, too. It
> > takes
> > > > some time to be able to distinguish broken tests that are flaky and
> > > broken
> > > > tests that are the result of a specific commit.
> > > >
> > > > My hope was that with a ppc64le VM (maybe through Qemu?) that runs on
> > > > x86-64 Linux, the other contributors could help fix things that broke
> > on
> > > > that platform.
> > > >
> > > > On Mon, Apr 17, 2017 at 12:51 PM, Marcel Kornacker <
> > marcel@cloudera.com>
> > > > wrote:
> > > >
> > > > > +1
> > > > >
> > > > > On Mon, Apr 17, 2017 at 11:56 AM, Henry Robinson <
> henry@cloudera.com
> > >
> > > > > wrote:
> > > > >
> > > > > > +1
> > > > > >
> > > > > > On Mon, Apr 17, 2017 at 9:09 AM Tim Armstrong <
> > > tarmstrong@cloudera.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > I feel like we shouldn't make PPC part of pre-commit at
least
> > > > > initially -
> > > > > > > it's an unreasonable barrier if contributors/committers
to
> debug
> > > > issues
> > > > > > on
> > > > > > > a platform they don't have easy access to. Having the testing
> > infra
> > > > is
> > > > > > > still important because we don't want to have code in there
> that
> > > will
> > > > > > > gradually bit-rot without us noticing.
> > > > > > >
> > > > > > > On Mon, Apr 17, 2017 at 8:51 AM, Silvius Rus <
> srus@cloudera.com>
> > > > > wrote:
> > > > > > >
> > > > > > > > Would it make sense to _not_ run PPC tests as part
of
> > presubmit?
> > > > > > Instead
> > > > > > > > Valencia could set up nightly tests using in-house
> > > infrastructure.
> > > > > And
> > > > > > > > share the test results, e.g., by sending them to a
new email
> > list
> > > > > > > > tests@impala.incubator.apache.org (that we'd need
to create)
> > so
> > > > > > everyone
> > > > > > > > can see when there are failures or if coverage stops
for
> > whatever
> > > > > > reason.
> > > > > > > > GCC has been doing something like this for long,
> > > > > > > > https://gcc.gnu.org/ml/gcc-testresults/2017-04/.
> > > > > > > >
> > > > > > > > On Tue, Apr 11, 2017 at 9:44 AM, Jim Apple <
> > jbapple@cloudera.com
> > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Locally, I work on native-toolchain using
a VM configured
> > > with
> > > > > > > > > > Ubuntu16.04ppc64le, 4GB RAM and 50GB of
HDD. If  we
> provide
> > > > you a
> > > > > > VM
> > > > > > > > with
> > > > > > > > > > this config, will it be sufficient ?
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > What hypervisor/emulator will it use?
> > > > > > > > >
> > > > > > > > > What are the requirements of the host OS and
host hardware?
> > > > > > > > >
> > > > > > > > > Why is the config you have it set to so important
that you
> > > > mention
> > > > > it
> > > > > > > in
> > > > > > > > > your email - will the config be locked down into
that
> config
> > or
> > > > can
> > > > > > it
> > > > > > > be
> > > > > > > > > reconfigured later?
> > > > > > > > >
> > > > > > > > > How is the VM controlled from the host OS? Keep
in mind
> that
> > a
> > > > GUI
> > > > > > > cannot
> > > > > > > > > be the only option for automated tests.
> > > > > > > > >
> > > > > > > > > FWIW, Impala's test suite probably cannot fully
complete
> > > without
> > > > at
> > > > > > > least
> > > > > > > > > 8, and I suspect 16, GB of RAM, and we might
need more disk
> > > > space,
> > > > > > too,
> > > > > > > > but
> > > > > > > > > these should be reconfigurable with most
> > hypervisors/emulators.
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > --
> > > > > > Henry Robinson
> > > > > > Software Engineer
> > > > > > Cloudera
> > > > > > 415-994-6679
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message