drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rahul Challapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4041) Parquet library update causing random "Buffer has negative reference count"
Date Fri, 06 Nov 2015 01:15:27 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14992861#comment-14992861
] 

Rahul Challapalli commented on DRILL-4041:
------------------------------------------

[~sphillips] Do you think the below error is related to the same issue

{code}
Test : Functional/flatten_operators/100000rows/18_DRILL-2227.q
Query: select uid, flatten(lst_lst[0]) from `data.json` where 2 in (select flatten(lst_lst[1])
from `data.json`)
Failed with exception
oadd.io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
	at oadd.io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:123)
	at oadd.io.netty.buffer.WrappedByteBuf.release(WrappedByteBuf.java:825)
	at oadd.io.netty.buffer.UnsafeDirectLittleEndian.release(UnsafeDirectLittleEndian.java:238)
	at oadd.io.netty.buffer.DrillBuf.release(DrillBuf.java:252)
	at oadd.io.netty.buffer.DrillBuf.release(DrillBuf.java:259)
	at oadd.io.netty.buffer.DrillBuf.release(DrillBuf.java:239)
	at org.apache.drill.jdbc.impl.DrillResultSetImpl$ResultsListener.close(DrillResultSetImpl.java:1563)
	at org.apache.drill.jdbc.impl.DrillResultSetImpl.cleanup(DrillResultSetImpl.java:138)
	at org.apache.drill.jdbc.impl.DrillCursor.close(DrillCursor.java:333)
	at oadd.net.hydromatic.avatica.AvaticaResultSet.close(AvaticaResultSet.java:110)
	at org.apache.drill.jdbc.impl.DrillResultSetImpl.close(DrillResultSetImpl.java:169)
	at org.apache.drill.test.framework.DrillTestJdbc.executeQuery(DrillTestJdbc.java:233)
	at org.apache.drill.test.framework.DrillTestJdbc.run(DrillTestJdbc.java:89)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)
{code}

At the end of the functional run which had this failure, I have seen the below message as
well
{code}
Exception while closing client.
java.lang.RuntimeException: Exception while closing
	at oadd.org.apache.drill.common.DrillAutoCloseables.closeNoChecked(DrillAutoCloseables.java:46)
	at oadd.org.apache.drill.exec.client.DrillClient.close(DrillClient.java:284)
	at org.apache.drill.jdbc.impl.DrillConnectionImpl.closeOrWarn(DrillConnectionImpl.java:379)
	at org.apache.drill.jdbc.impl.DrillConnectionImpl.cleanup(DrillConnectionImpl.java:392)
	at org.apache.drill.jdbc.impl.DrillHandler.onConnectionClose(DrillHandler.java:36)
	at oadd.net.hydromatic.avatica.AvaticaConnection.close(AvaticaConnection.java:135)
	at org.apache.drill.test.framework.ConnectionPool.close(ConnectionPool.java:90)
	at org.apache.drill.test.framework.TestDriver.runTests(TestDriver.java:265)
	at org.apache.drill.test.framework.TestDriver.main(TestDriver.java:98)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
	at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.IllegalStateException: Failure while closing accountor.  Expected private
and shared pools to be set to initial values.  However, one or more were not.  Stats are
	zone	init	allocated	delta 
	private	0	0	0 
	shared	8589934592	8586326012	3608580.
	at oadd.org.apache.drill.exec.memory.AtomicRemainder.close(AtomicRemainder.java:199)
	at oadd.org.apache.drill.exec.memory.Accountor.close(Accountor.java:390)
	at oadd.org.apache.drill.exec.memory.TopLevelAllocator.close(TopLevelAllocator.java:175)
	at oadd.org.apache.drill.common.DrillAutoCloseables.closeNoChecked(DrillAutoCloseables.java:44)
	... 14 more
{code}

If this is a completely unrelated issue, I will raise another jira. But this started showing
up after the union vector changes have been merged.

> Parquet library update causing random "Buffer has negative reference count"
> ---------------------------------------------------------------------------
>
>                 Key: DRILL-4041
>                 URL: https://issues.apache.org/jira/browse/DRILL-4041
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Storage - Parquet
>    Affects Versions: 1.3.0
>            Reporter: Rahul Challapalli
>            Assignee: Steven Phillips
>            Priority: Critical
>
> git commit # 39582bd60c9eaaaa9b16aba4f099d434e927e7e5
> After the parquet library update commit, we started seeing the below error randomly causing
failures in the  Extended Functional Suite.
> {code}
> Failed with exception
> java.lang.IllegalArgumentException: Buffer has negative reference count.
> 	at oadd.com.google.common.base.Preconditions.checkArgument(Preconditions.java:92)
> 	at oadd.io.netty.buffer.DrillBuf.release(DrillBuf.java:250)
> 	at oadd.io.netty.buffer.DrillBuf.release(DrillBuf.java:259)
> 	at oadd.io.netty.buffer.DrillBuf.release(DrillBuf.java:259)
> 	at oadd.io.netty.buffer.DrillBuf.release(DrillBuf.java:259)
> 	at oadd.io.netty.buffer.DrillBuf.release(DrillBuf.java:239)
> 	at oadd.org.apache.drill.exec.vector.BaseDataValueVector.clear(BaseDataValueVector.java:39)
> 	at oadd.org.apache.drill.exec.vector.NullableIntVector.clear(NullableIntVector.java:150)
> 	at oadd.org.apache.drill.exec.record.SimpleVectorWrapper.clear(SimpleVectorWrapper.java:84)
> 	at oadd.org.apache.drill.exec.record.VectorContainer.zeroVectors(VectorContainer.java:312)
> 	at oadd.org.apache.drill.exec.record.VectorContainer.clear(VectorContainer.java:296)
> 	at oadd.org.apache.drill.exec.record.RecordBatchLoader.clear(RecordBatchLoader.java:183)
> 	at org.apache.drill.jdbc.impl.DrillResultSetImpl.cleanup(DrillResultSetImpl.java:139)
> 	at org.apache.drill.jdbc.impl.DrillCursor.close(DrillCursor.java:333)
> 	at oadd.net.hydromatic.avatica.AvaticaResultSet.close(AvaticaResultSet.java:110)
> 	at org.apache.drill.jdbc.impl.DrillResultSetImpl.close(DrillResultSetImpl.java:169)
> 	at org.apache.drill.test.framework.DrillTestJdbc.executeQuery(DrillTestJdbc.java:233)
> 	at org.apache.drill.test.framework.DrillTestJdbc.run(DrillTestJdbc.java:89)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:744)
> {code} 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message