drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amit Hadke <amit.ha...@gmail.com>
Subject Re: [VOTE] Release Apache Drill 1.3.0 (rc1)
Date Mon, 09 Nov 2015 23:37:56 GMT
I've seen "OutOfMemoryError:unable to create new native thread". on mac,
especially for impersonation tests(minidfs).
I had to increase limits in order to fix that. (open files and max user
processes)


On Mon, Nov 9, 2015 at 3:32 PM, Jason Altekruse <altekrusejason@gmail.com>
wrote:

> On two successive attempts I saw these errors while running the tests on my
> mac. One of the commits on Friday was trying to fix an issue with leaking
> threads. It appears that there is still more that needs to be done here.
>
> It's unfortunately going to have to be -1 for me as well.
>
> Tests in error:
>   TestImpersonationQueries.testDirectImpersonation_HasGroupReadPermissions
> » UserRemote
>
>
> TestImpersonationQueries.testDirectImpersonation_NoReadPermissions:196->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213
> » Rpc
>
>
> TestImpersonationQueries.testMultiLevelImpersonationEqualToMaxUserHops:210->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213
> »
>
>
> testDirectImpersonation_HasGroupReadPermissions(org.apache.drill.exec.impersonation.TestImpersonationQueries)
> Time elapsed: 0.101 sec  <<< ERROR!
> org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
> OutOfMemoryError: unable to create new native thread
>
>
> [Error Id: 61865e6f-780a-4a76-906d-2f88191d7e5e on 10.0.1.12:31010]
>     at
>
> org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:118)
>     at
>
> org.apache.drill.exec.rpc.user.UserClient.handleReponse(UserClient.java:112)
>     at
>
> org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:47)
>     at
>
> org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:32)
>     at org.apache.drill.exec.rpc.RpcBus.handle(RpcBus.java:68)
>     at org.apache.drill.exec.rpc.RpcBus$RequestEvent.run(RpcBus.java:390)
>     at
>
> org.apache.drill.common.SerializedExecutor$RunnableProcessor.run(SerializedExecutor.java:105)
>     at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>     at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>     at java.lang.Thread.run(Thread.java:745)
>
>
> testDirectImpersonation_NoReadPermissions(org.apache.drill.exec.impersonation.TestImpersonationQueries)
> Time elapsed: 0.021 sec  <<< ERROR!
> org.apache.drill.exec.rpc.RpcException:
> org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
> OutOfMemoryError: unable to create new native thread
>
>
> [Error Id: f1cd9d09-8444-4e5f-9e89-5810d15cf5c9 on 10.0.1.12:31010]
>     at
> org.apache.drill.exec.rpc.RpcException.mapException(RpcException.java:60)
>     at
>
> org.apache.drill.exec.client.DrillClient$ListHoldingResultsListener.getResults(DrillClient.java:424)
>     at
> org.apache.drill.exec.client.DrillClient.runQuery(DrillClient.java:321)
>     at org.apache.drill.QueryTestUtil.createClient(QueryTestUtil.java:69)
>     at org.apache.drill.BaseTestQuery.updateClient(BaseTestQuery.java:213)
>     at org.apache.drill.BaseTestQuery.updateClient(BaseTestQuery.java:236)
>     at org.apache.drill.BaseTestQuery.updateClient(BaseTestQuery.java:222)
>     at
>
> org.apache.drill.exec.impersonation.TestImpersonationQueries.testDirectImpersonation_NoReadPermissions(TestImpersonationQueries.java:196)
> Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM
> ERROR: OutOfMemoryError: unable to create new native thread
>
> On Mon, Nov 9, 2015 at 2:23 PM, Abdel Hakim Deneche <adeneche@maprtech.com
> >
> wrote:
>
> > Saw similar failures when running unit tests with forkCount = 1:
> >
> > Tests in error:
> > >   TestImpersonationQueries.sequenceFileChainedImpersonationWithView »
> > > UserRemote
> > >
> > >
> >
> TestImpersonationQueries.testMultiLevelImpersonationJoinEachSideReachesMaxUserHops:233->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213
> > > » Rpc
> > >
> > >
> >
> TestImpersonationQueries.testMultiLevelImpersonationExceedsMaxUserHops:219->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213
> > > » IllegalState
> > >
> > >
> >
> TestImpersonationQueries.avroChainedImpersonationWithView:280->BaseTestImpersonation.createView:186->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213
> > > » IllegalState
> > >
> > >
> >
> TestImpersonationQueries.testDirectImpersonation_HasGroupReadPermissions:186->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213
> > > » IllegalState
> > >
> > >
> >
> TestImpersonationQueries.testDirectImpersonation_NoReadPermissions:196->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213
> > > » IllegalState
> > >
> > >
> >
> TestImpersonationQueries.testMultiLevelImpersonationEqualToMaxUserHops:210->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213
> > > » IllegalState
> > >   TestCountDownLatchInjection.latchInjected:146 NullPointer
> > >   TestMergingReceiver.testMultipleProvidersMixedSizes:115 »
> > > IllegalReferenceCount
> > > Tests run: 1482, Failures: 0, Errors: 9, Skipped: 118
> >
> >
> >  Unfortunately I'm changing my vote to -1
> >
> >
> > On Mon, Nov 9, 2015 at 2:21 PM, Hanifi Gunes <hgunes@maprtech.com>
> wrote:
> >
> > > -1 (binding)
> > >
> > > 5/5 consecutive runs I get the following consistently with forkCount
> set
> > to
> > > 1 on build.
> > >
> > > Tests in error:
> > >   TestImpersonationQueries.sequenceFileChainedImpersonationWithView »
> > > UserRemote
> > >
> > >
> >
> TestImpersonationQueries.testMultiLevelImpersonationJoinEachSideReachesMaxUserHops:233->BaseTestQuery.updateClient:222->BaseTestQuery.
> > >    updateClient:236->BaseTestQuery.updateClient:213 » IllegalState
> > >
> > >
> >
> TestImpersonationQueries.testMultiLevelImpersonationExceedsMaxUserHops:219->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:
> > >   236->BaseTestQuery.updateClient:213 » IllegalState
> > >
> > >
> >
> TestImpersonationQueries.avroChainedImpersonationWithView:280->BaseTestImpersonation.createView:186->BaseTestQuery.updateClient:222-
> > >      >BaseTestQuery.updateClient:236->BaseTestQuery.updateClient:213
»
> > > IllegalState
> > >
> > >
> >
> TestImpersonationQueries.testDirectImpersonation_HasGroupReadPermissions:186->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:
> > > 236->BaseTestQuery.updateClient:213 » IllegalState
> > >
> > >
> >
> TestImpersonationQueries.testDirectImpersonation_NoReadPermissions:196->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:236-
> > >   >BaseTestQuery.updateClient:213 » IllegalState
> > >
> > >
> >
> TestImpersonationQueries.testMultiLevelImpersonationEqualToMaxUserHops:210->BaseTestQuery.updateClient:222->BaseTestQuery.updateClient:
> > >   236->BaseTestQuery.updateClient:213 » IllegalState
> > >
> > >
> > > On Mon, Nov 9, 2015 at 2:19 PM, Stefán Baxter <
> stefan@activitystream.com
> > >
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I'm not a committer and not voting but I want to point out that the
> > UDFs
> > > > that we are using and run with 1.1 and 1.2 do not, for some reason,
> run
> > > > with 1.3.
> > > >
> > > > I'm inclined to insist that it's something we are doing wrong but if
> > this
> > > > counts as a test then it's failing.
> > > >
> > > > Regards,
> > > >  -Stefan
> > > >
> > > > On Mon, Nov 9, 2015 at 10:09 PM, Norris Lee <norrisl@simba.com>
> wrote:
> > > >
> > > > > +1 Non-binding.
> > > > >
> > > > > Built from source on Linux, tested ODBC against various data
> sources
> > > > > (Hive, DFS-csv, tsv, parquet, json)
> > > > >
> > > > > Norris
> > > > >
> > > > > -----Original Message-----
> > > > > From: Abdel Hakim Deneche [mailto:adeneche@maprtech.com]
> > > > > Sent: Monday, November 09, 2015 10:16 AM
> > > > > To: dev@drill.apache.org
> > > > > Subject: Re: [VOTE] Release Apache Drill 1.3.0 (rc1)
> > > > >
> > > > > +1
> > > > >
> > > > > Checked checksum files and gpg signature for all 3 files. Built
> from
> > > > > source, deployed on a 4 nodes cluster then run several window
> > function
> > > > > queries on TPCDS SF100. Looks great.
> > > > >
> > > > > On Mon, Nov 9, 2015 at 8:14 AM, Jacques Nadeau <jacques@dremio.com
> >
> > > > wrote:
> > > > >
> > > > > > I've seen another instance of DRILL-4041. Since it is exceedingly
> > > > > > rare, I don't think it is a release stopper. I will spend some
> more
> > > > > > time reviewing it.
> > > > > >
> > > > > > --
> > > > > > Jacques Nadeau
> > > > > > CTO and Co-Founder, Dremio
> > > > > >
> > > > > > On Sun, Nov 8, 2015 at 7:34 PM, Amit Hadke <amit.hadke@gmail.com
> >
> > > > wrote:
> > > > > >
> > > > > > > +1, non-binding
> > > > > > > Downloaded tarball, mvn clean install Ran join queries,,
was
> able
> > > to
> > > > > > > run queries on json, parquet and sequence files.
> > > > > > >
> > > > > > > On Sun, Nov 8, 2015 at 10:01 AM, Abhijit Pol <
> abhijit@dremio.com
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > resending the vote with details
> > > > > > > >
> > > > > > > > +1, non-binding
> > > > > > > >
> > > > > > > > ran maven tests, installed on Mac, ran drill in embedded
> mode,
> > > > > > > > verified
> > > > > > > Web
> > > > > > > > UI for CSV header config changes, ran few queries
on CSV
> files
> > > > > > > > with and without header, ran few queries against large
CSV
> > files
> > > > > > > > with header in
> > > > > > > HDFS
> > > > > > > >
> > > > > > > > On Sun, Nov 8, 2015 at 9:07 AM, Hsuan Yi Chu <
> > > hyichu@maprtech.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > +1
> > > > > > > > > mvn clean install on mac and linux
> > > > > > > > >
> > > > > > > > > tried a few queries on distributed mode.
> > > > > > > > >
> > > > > > > > > Things looked fine
> > > > > > > > >
> > > > > > > > > On Sun, Nov 8, 2015 at 8:16 AM, Aman Sinha <
> > > asinha@maprtech.com>
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > +1.
> > > > > > > > > > Downloaded the binary tar ball and installed
on Mac.  Ran
> > > > > > > > > > Drill in
> > > > > > > > > embedded
> > > > > > > > > > mode.  Tested a few join, aggregation and
limit queries
> > > > > > > > > > against
> > > > > > > parquet
> > > > > > > > > > data with and without metadata cache.  Tested
> cancellation
> > > for
> > > > > > couple
> > > > > > > > of
> > > > > > > > > > queries.  Looked at query profiles on the
WebUI.   Looks
> > > good.
> > > > > > > > > >
> > > > > > > > > > On Sat, Nov 7, 2015 at 11:53 PM, Abhijit
Pol
> > > > > > > > > > <abhijit@dremio.com>
> > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > +1
> > > > > > > > > > >
> > > > > > > > > > > On Sat, Nov 7, 2015 at 9:41 PM, andrew
<
> > andrew@primer.org>
> > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > +1 (non-binding, non-comitter)
> > > > > > > > > > > >
> > > > > > > > > > > > > On Nov 6, 2015, at 10:15
PM, Jacques Nadeau <
> > > > > > > jacques@dremio.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > Hey Everybody,
> > > > > > > > > > > > >
> > > > > > > > > > > > > I'd like to propose a new
release candidate of
> Apache
> > > > > > > > > > > > > Drill,
> > > > > > > > > version
> > > > > > > > > > > > > 1.3.0.  This is the second
release candidate (rc1).
> > > > > > > > > > > > > This
> > > > > > > > addresses
> > > > > > > > > > > some
> > > > > > > > > > > > > issues identified in the
first release candidate
> > > > > > > > > > > > > including
> > > > > > some
> > > > > > > > > test
> > > > > > > > > > > > > threading issues, Parquet
dictionary read issue
> with
> > > > > > > > > > > > > binary
> > > > > > > > values,
> > > > > > > > > > > Joda
> > > > > > > > > > > > > upgrade to avoid a concurrency
bug and a couple
> other
> > > > > > > > > > > > > small
> > > > > > > > things.
> > > > > > > > > > > > >
> > > > > > > > > > > > > The tarball artifacts are
hosted at [2] and the
> maven
> > > > > > artifacts
> > > > > > > > are
> > > > > > > > > > > > hosted
> > > > > > > > > > > > > at [3]. This release candidate
is based on commit
> > > > > > > > > > > > > f17ebd2fbf2bf5ba201a635f5b6ec3615afe2305
located at
> > > [4].
> > > > > > > > > > > > >
> > > > > > > > > > > > > The vote will be open for
72 hours ending at 10PM
> > > > > > > > > > > > > Pacific,
> > > > > > > > November
> > > > > > > > > > 9,
> > > > > > > > > > > > 2015.
> > > > > > > > > > > > >
> > > > > > > > > > > > > [ ] +1
> > > > > > > > > > > > > [ ] +0
> > > > > > > > > > > > > [ ] -1
> > > > > > > > > > > > >
> > > > > > > > > > > > > thanks,
> > > > > > > > > > > > > Jacques
> > > > > > > > > > > > >
> > > > > > > > > > > > > [1]
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313
> > > > > > 820&version=12332946[2]
> > > > > > > > > > > > >
> > > http://people.apache.org/~jacques/apache-drill-1.3.0.rc1
> > > > > > > > > > > > > /
> > > > > > > > > > > > > [3]
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > >
> > > https://repository.apache.org/content/repositories/orgapachedrill-1014
> > > > > > /
> > > > > > > > > > > > > [4]
> > > > > > > > > > > > >
> > > https://github.com/jacques-n/drill/tree/drill-1.3.0-rc1
> > > > > > > > > > > > >
> > > > > > > > > > > > > --
> > > > > > > > > > > > > Jacques Nadeau
> > > > > > > > > > > > > CTO and Co-Founder, Dremio
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Abdelhakim Deneche
> > > > >
> > > > > Software Engineer
> > > > >
> > > > >   <http://www.mapr.com/>
> > > > >
> > > > >
> > > > > Now Available - Free Hadoop On-Demand Training <
> > > > >
> > > >
> > >
> >
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> >
> > Abdelhakim Deneche
> >
> > Software Engineer
> >
> >   <http://www.mapr.com/>
> >
> >
> > Now Available - Free Hadoop On-Demand Training
> > <
> >
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message