Mailing-List: contact dev-help@impala.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@impala.incubator.apache.org
MIME-Version: 1.0
In-Reply-To: <CAM7YqvNby0WYkqwSejdAEZ1BRhRvvsmBduhmdvdtuHr19Vy=wQ@mail.gmail.com>
References: <OFFC585649.6225DB54-ON002580FF.0018B49D-652580FF.0034C73C@notes.na.collabserv.com>
 <CAC-pSX0vxK=Lg_uGGThLtzVsEYcYT4kMy2Y4zQmUALBc5KyXJg@mail.gmail.com>
 <CAKGvSBjyXP4qkTAOGN-EXE9_k7Mo4gm22V2w88Kn49KXY=yTLQ@mail.gmail.com>
 <CAEoRBexyJ=v-Pny_s_0ih7DJVf1mFcmiHTpprX1sFevdXFQJnA@mail.gmail.com>
 <CADZba-Rpnvn0uznz7MhKtB-YiL0bf=qo+kTURYXcJBXHssVz6w@mail.gmail.com> <CAM7YqvNby0WYkqwSejdAEZ1BRhRvvsmBduhmdvdtuHr19Vy=wQ@mail.gmail.com>
From: Jim Apple <jbapple@cloudera.com>
Date: Mon, 17 Apr 2017 14:30:01 -0700
Message-ID: <CAC-pSX2SAUwUD2V2ntEoEjiRVtehtmR+GZ5Sght1wFquuNrPiQ@mail.gmail.com>
Subject: Re: Upstreaming ppc64le patches for native-toolchain
To: "dev@impala" <dev@impala.incubator.apache.org>
Cc: Manish Patil <patil2@us.ibm.com>, Nishidha Panpaliya <nishidha@us.ibm.com>,
	Sudarshan Jagadale <jagadale@us.ibm.com>, Valencia Serrao <vserrao@us.ibm.com>
Content-Type: multipart/alternative; boundary=94eb2c1b54b2d5e515054d6380a6
archived-at: Mon, 17 Apr 2017 21:30:50 -0000

--94eb2c1b54b2d5e515054d6380a6
Content-Type: text/plain; charset=UTF-8

One concern I have is sustainability. If only one Impala contributor can
work with ppc64le, and that contributor is not as seasoned as some of the
other committers, what happens if ppc64le breaks and the one person with VM
access can't fix it?

Part of my concern is just how flaky the current tests are, too. It takes
some time to be able to distinguish broken tests that are flaky and broken
tests that are the result of a specific commit.

My hope was that with a ppc64le VM (maybe through Qemu?) that runs on
x86-64 Linux, the other contributors could help fix things that broke on
that platform.

On Mon, Apr 17, 2017 at 12:51 PM, Marcel Kornacker <marcel@cloudera.com>
wrote:

> +1
>
> On Mon, Apr 17, 2017 at 11:56 AM, Henry Robinson <henry@cloudera.com>
> wrote:
>
> > +1
> >
> > On Mon, Apr 17, 2017 at 9:09 AM Tim Armstrong <tarmstrong@cloudera.com>
> > wrote:
> >
> > > I feel like we shouldn't make PPC part of pre-commit at least
> initially -
> > > it's an unreasonable barrier if contributors/committers to debug issues
> > on
> > > a platform they don't have easy access to. Having the testing infra is
> > > still important because we don't want to have code in there that will
> > > gradually bit-rot without us noticing.
> > >
> > > On Mon, Apr 17, 2017 at 8:51 AM, Silvius Rus <srus@cloudera.com>
> wrote:
> > >
> > > > Would it make sense to _not_ run PPC tests as part of presubmit?
> > Instead
> > > > Valencia could set up nightly tests using in-house infrastructure.
> And
> > > > share the test results, e.g., by sending them to a new email list
> > > > tests@impala.incubator.apache.org (that we'd need to create) so
> > everyone
> > > > can see when there are failures or if coverage stops for whatever
> > reason.
> > > > GCC has been doing something like this for long,
> > > > https://gcc.gnu.org/ml/gcc-testresults/2017-04/.
> > > >
> > > > On Tue, Apr 11, 2017 at 9:44 AM, Jim Apple <jbapple@cloudera.com>
> > wrote:
> > > >
> > > > > >
> > > > > > Locally, I work on native-toolchain using a VM configured with
> > > > > > Ubuntu16.04ppc64le, 4GB RAM and 50GB of HDD. If  we provide you a
> > VM
> > > > with
> > > > > > this config, will it be sufficient ?
> > > > > >
> > > > >
> > > > > What hypervisor/emulator will it use?
> > > > >
> > > > > What are the requirements of the host OS and host hardware?
> > > > >
> > > > > Why is the config you have it set to so important that you mention
> it
> > > in
> > > > > your email - will the config be locked down into that config or can
> > it
> > > be
> > > > > reconfigured later?
> > > > >
> > > > > How is the VM controlled from the host OS? Keep in mind that a GUI
> > > cannot
> > > > > be the only option for automated tests.
> > > > >
> > > > > FWIW, Impala's test suite probably cannot fully complete without at
> > > least
> > > > > 8, and I suspect 16, GB of RAM, and we might need more disk space,
> > too,
> > > > but
> > > > > these should be reconfigurable with most hypervisors/emulators.
> > > > >
> > > >
> > >
> > --
> > Henry Robinson
> > Software Engineer
> > Cloudera
> > 415-994-6679
> >
>

--94eb2c1b54b2d5e515054d6380a6--