drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hanifi Gunes <hgu...@maprtech.com>
Subject Re: [VOTE] Release Apache Drill 1.3.0 (rc1)
Date Tue, 10 Nov 2015 00:20:17 GMT
Using the default parallelism, I get the same set of exceptions on CentOS.

Also max # of threads per process is the system default, 319548.

On Mon, Nov 9, 2015 at 4:13 PM, Amit Hadke <amit.hadke@gmail.com> wrote:

> Unfortunately, yes I changed them at OS level.
>
> I agree that making OS level changes to run tests is not something we
> should advocate.
> It was just an observation that tests seem to pass on linux with default
> settings but on mac behavior is irregular.
>
> ~ Amit.
>
>
> On Mon, Nov 9, 2015 at 3:48 PM, Jason Altekruse <altekrusejason@gmail.com>
> wrote:
>
> > Amit,
> >
> > Did you change these at the OS level? I would hope that we wouldn't have
> to
> > make people jump through such hoops to run the build, was there a thread
> I
> > missed where we discussed this something that we should be expecting
> > everyone to do? I did see the threads about increasing the maven permgen
> > and did so accordingly, but I think adjusting something like the system
> > ulimit is something we really should try to avoid.
> >
> > On Mon, Nov 9, 2015 at 3:44 PM, Abdel Hakim Deneche <
> adeneche@maprtech.com
> > >
> > wrote:
> >
> > > I just tried with the default forkCount and got similar failures. I'm
> > > running the tests on a linux machine with the default maven options
> > >
> > > Thanks
> > >
> > > On Mon, Nov 9, 2015 at 3:21 PM, Jacques Nadeau <jacques@dremio.com>
> > wrote:
> > >
> > > > So you're changing the forkCount from the default? What happens when
> > you
> > > > run with the default?
> > > >
> > > > Also, did you guys run with extra permgen or the default? Also, can
> you
> > > > confirm what type of machine you are running on?
> > > >
> > > > Want to figure out why things are different.
> > > >
> > > > thanks,
> > > > Jacques
> > > >
> > > > --
> > > > Jacques Nadeau
> > > > CTO and Co-Founder, Dremio
> > > >
> > > > On Mon, Nov 9, 2015 at 2:23 PM, Abdel Hakim Deneche <
> > > adeneche@maprtech.com
> > > > >
> > > > wrote:
> > > >
> > > > > Saw similar failures when running unit tests with forkCount = 1:
> > > > >
> > > > > Tests in error:
> > > > > >
> >  TestImpersonationQueries.sequenceFileChainedImpersonationWithView »
> > > > > > UserRemote
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> TestImpersonationQueries.testMultiLevelImpersonationJoinEachSideReachesMaxUserHops:233->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213
> > > > > > » Rpc
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> TestImpersonationQueries.testMultiLevelImpersonationExceedsMaxUserHops:219->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213
> > > > > > » IllegalState
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> TestImpersonationQueries.avroChainedImpersonationWithView:280->BaseTestImpersonation.createView:186->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213
> > > > > > » IllegalState
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> TestImpersonationQueries.testDirectImpersonation_HasGroupReadPermissions:186->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213
> > > > > > » IllegalState
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> TestImpersonationQueries.testDirectImpersonation_NoReadPermissions:196->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213
> > > > > > » IllegalState
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> TestImpersonationQueries.testMultiLevelImpersonationEqualToMaxUserHops:210->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213
> > > > > > » IllegalState
> > > > > >   TestCountDownLatchInjection.latchInjected:146 NullPointer
> > > > > >   TestMergingReceiver.testMultipleProvidersMixedSizes:115 »
> > > > > > IllegalReferenceCount
> > > > > > Tests run: 1482, Failures: 0, Errors: 9, Skipped: 118
> > > > >
> > > > >
> > > > >  Unfortunately I'm changing my vote to -1
> > > > >
> > > > >
> > > > > On Mon, Nov 9, 2015 at 2:21 PM, Hanifi Gunes <hgunes@maprtech.com>
> > > > wrote:
> > > > >
> > > > > > -1 (binding)
> > > > > >
> > > > > > 5/5 consecutive runs I get the following consistently with
> > forkCount
> > > > set
> > > > > to
> > > > > > 1 on build.
> > > > > >
> > > > > > Tests in error:
> > > > > >
> >  TestImpersonationQueries.sequenceFileChainedImpersonationWithView »
> > > > > > UserRemote
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> TestImpersonationQueries.testMultiLevelImpersonationJoinEachSideReachesMaxUserHops:233->BaseTestQuery.updateClient:222->BaseTestQuery.
> > > > > >    updateClient:236->BaseTestQuery.updateClient:213 »
> IllegalState
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> TestImpersonationQueries.testMultiLevelImpersonationExceedsMaxUserHops:219->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:
> > > > > >   236->BaseTestQuery.updateClient:213 » IllegalState
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> TestImpersonationQueries.avroChainedImpersonationWithView:280->BaseTestImpersonation.createView:186->BaseTestQuery.updateClient:222-
> > > > > >
> > >BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213
> > > »
> > > > > > IllegalState
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> TestImpersonationQueries.testDirectImpersonation_HasGroupReadPermissions:186->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:
> > > > > > 236->BaseTestQuery.updateClient:213 » IllegalState
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> TestImpersonationQueries.testDirectImpersonation_NoReadPermissions:196->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236-
> > > > > >   >BaseTestQuery.updateClient:213 » IllegalState
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> TestImpersonationQueries.testMultiLevelImpersonationEqualToMaxUserHops:210->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:
> > > > > >   236->BaseTestQuery.updateClient:213 » IllegalState
> > > > > >
> > > > > >
> > > > > > On Mon, Nov 9, 2015 at 2:19 PM, Stefán Baxter <
> > > > stefan@activitystream.com
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I'm not a committer and not voting but I want to point
out that
> > the
> > > > > UDFs
> > > > > > > that we are using and run with 1.1 and 1.2 do not, for
some
> > reason,
> > > > run
> > > > > > > with 1.3.
> > > > > > >
> > > > > > > I'm inclined to insist that it's something we are doing
wrong
> but
> > > if
> > > > > this
> > > > > > > counts as a test then it's failing.
> > > > > > >
> > > > > > > Regards,
> > > > > > >  -Stefan
> > > > > > >
> > > > > > > On Mon, Nov 9, 2015 at 10:09 PM, Norris Lee <norrisl@simba.com
> >
> > > > wrote:
> > > > > > >
> > > > > > > > +1 Non-binding.
> > > > > > > >
> > > > > > > > Built from source on Linux, tested ODBC against various
data
> > > > sources
> > > > > > > > (Hive, DFS-csv, tsv, parquet, json)
> > > > > > > >
> > > > > > > > Norris
> > > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: Abdel Hakim Deneche [mailto:adeneche@maprtech.com]
> > > > > > > > Sent: Monday, November 09, 2015 10:16 AM
> > > > > > > > To: dev@drill.apache.org
> > > > > > > > Subject: Re: [VOTE] Release Apache Drill 1.3.0 (rc1)
> > > > > > > >
> > > > > > > > +1
> > > > > > > >
> > > > > > > > Checked checksum files and gpg signature for all 3
files.
> Built
> > > > from
> > > > > > > > source, deployed on a 4 nodes cluster then run several
window
> > > > > function
> > > > > > > > queries on TPCDS SF100. Looks great.
> > > > > > > >
> > > > > > > > On Mon, Nov 9, 2015 at 8:14 AM, Jacques Nadeau <
> > > jacques@dremio.com
> > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > I've seen another instance of DRILL-4041. Since
it is
> > > exceedingly
> > > > > > > > > rare, I don't think it is a release stopper.
I will spend
> > some
> > > > more
> > > > > > > > > time reviewing it.
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Jacques Nadeau
> > > > > > > > > CTO and Co-Founder, Dremio
> > > > > > > > >
> > > > > > > > > On Sun, Nov 8, 2015 at 7:34 PM, Amit Hadke <
> > > amit.hadke@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > +1, non-binding
> > > > > > > > > > Downloaded tarball, mvn clean install Ran
join queries,,
> > was
> > > > able
> > > > > > to
> > > > > > > > > > run queries on json, parquet and sequence
files.
> > > > > > > > > >
> > > > > > > > > > On Sun, Nov 8, 2015 at 10:01 AM, Abhijit
Pol <
> > > > abhijit@dremio.com
> > > > > >
> > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > resending the vote with details
> > > > > > > > > > >
> > > > > > > > > > > +1, non-binding
> > > > > > > > > > >
> > > > > > > > > > > ran maven tests, installed on Mac,
ran drill in
> embedded
> > > > mode,
> > > > > > > > > > > verified
> > > > > > > > > > Web
> > > > > > > > > > > UI for CSV header config changes, ran
few queries on
> CSV
> > > > files
> > > > > > > > > > > with and without header, ran few queries
against large
> > CSV
> > > > > files
> > > > > > > > > > > with header in
> > > > > > > > > > HDFS
> > > > > > > > > > >
> > > > > > > > > > > On Sun, Nov 8, 2015 at 9:07 AM, Hsuan
Yi Chu <
> > > > > > hyichu@maprtech.com>
> > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > +1
> > > > > > > > > > > > mvn clean install on mac and linux
> > > > > > > > > > > >
> > > > > > > > > > > > tried a few queries on distributed
mode.
> > > > > > > > > > > >
> > > > > > > > > > > > Things looked fine
> > > > > > > > > > > >
> > > > > > > > > > > > On Sun, Nov 8, 2015 at 8:16 AM,
Aman Sinha <
> > > > > > asinha@maprtech.com>
> > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > +1.
> > > > > > > > > > > > > Downloaded the binary tar
ball and installed on
> Mac.
> > > Ran
> > > > > > > > > > > > > Drill in
> > > > > > > > > > > > embedded
> > > > > > > > > > > > > mode.  Tested a few join,
aggregation and limit
> > queries
> > > > > > > > > > > > > against
> > > > > > > > > > parquet
> > > > > > > > > > > > > data with and without metadata
cache.  Tested
> > > > cancellation
> > > > > > for
> > > > > > > > > couple
> > > > > > > > > > > of
> > > > > > > > > > > > > queries.  Looked at query
profiles on the WebUI.
> > >  Looks
> > > > > > good.
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Sat, Nov 7, 2015 at 11:53
PM, Abhijit Pol
> > > > > > > > > > > > > <abhijit@dremio.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > +1
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Sat, Nov 7, 2015
at 9:41 PM, andrew <
> > > > > andrew@primer.org>
> > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > +1 (non-binding,
non-comitter)
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > On Nov 6,
2015, at 10:15 PM, Jacques Nadeau <
> > > > > > > > > > jacques@dremio.com>
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Hey Everybody,
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I'd like to
propose a new release candidate
> of
> > > > Apache
> > > > > > > > > > > > > > > > Drill,
> > > > > > > > > > > > version
> > > > > > > > > > > > > > > > 1.3.0.  This
is the second release candidate
> > > (rc1).
> > > > > > > > > > > > > > > > This
> > > > > > > > > > > addresses
> > > > > > > > > > > > > > some
> > > > > > > > > > > > > > > > issues identified
in the first release
> > candidate
> > > > > > > > > > > > > > > > including
> > > > > > > > > some
> > > > > > > > > > > > test
> > > > > > > > > > > > > > > > threading
issues, Parquet dictionary read
> issue
> > > > with
> > > > > > > > > > > > > > > > binary
> > > > > > > > > > > values,
> > > > > > > > > > > > > > Joda
> > > > > > > > > > > > > > > > upgrade to
avoid a concurrency bug and a
> couple
> > > > other
> > > > > > > > > > > > > > > > small
> > > > > > > > > > > things.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > The tarball
artifacts are hosted at [2] and
> the
> > > > maven
> > > > > > > > > artifacts
> > > > > > > > > > > are
> > > > > > > > > > > > > > > hosted
> > > > > > > > > > > > > > > > at [3]. This
release candidate is based on
> > commit
> > > > > > > > > > > > > > > > f17ebd2fbf2bf5ba201a635f5b6ec3615afe2305
> > located
> > > at
> > > > > > [4].
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > The vote will
be open for 72 hours ending at
> > 10PM
> > > > > > > > > > > > > > > > Pacific,
> > > > > > > > > > > November
> > > > > > > > > > > > > 9,
> > > > > > > > > > > > > > > 2015.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > [ ] +1
> > > > > > > > > > > > > > > > [ ] +0
> > > > > > > > > > > > > > > > [ ] -1
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > thanks,
> > > > > > > > > > > > > > > > Jacques
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > [1]
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > >
> > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313
> > > > > > > > > 820&version=12332946[2]
> > > > > > > > > > > > > > > >
> > > > > > http://people.apache.org/~jacques/apache-drill-1.3.0.rc1
> > > > > > > > > > > > > > > > /
> > > > > > > > > > > > > > > > [3]
> > > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > >
> > > https://repository.apache.org/content/repositories/orgapachedrill-1014
> > > > > > > > > /
> > > > > > > > > > > > > > > > [4]
> > > > > > > > > > > > > > > >
> > > > > > https://github.com/jacques-n/drill/tree/drill-1.3.0-rc1
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > Jacques Nadeau
> > > > > > > > > > > > > > > > CTO and Co-Founder,
Dremio
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > >
> > > > > > > > Abdelhakim Deneche
> > > > > > > >
> > > > > > > > Software Engineer
> > > > > > > >
> > > > > > > >   <http://www.mapr.com/>
> > > > > > > >
> > > > > > > >
> > > > > > > > Now Available - Free Hadoop On-Demand Training <
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Abdelhakim Deneche
> > > > >
> > > > > Software Engineer
> > > > >
> > > > >   <http://www.mapr.com/>
> > > > >
> > > > >
> > > > > Now Available - Free Hadoop On-Demand Training
> > > > > <
> > > > >
> > > >
> > >
> >
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > >
> > > Abdelhakim Deneche
> > >
> > > Software Engineer
> > >
> > >   <http://www.mapr.com/>
> > >
> > >
> > > Now Available - Free Hadoop On-Demand Training
> > > <
> > >
> >
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message