drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abdel Hakim Deneche <adene...@maprtech.com>
Subject Re: anyone seen these errors on master ?
Date Wed, 02 Sep 2015 17:10:26 GMT
I am still seeing the TestValueVector unit tests fail on my linux machine
(2/3 of my builds)

On Fri, Aug 14, 2015 at 2:40 PM, Abdel Hakim Deneche <adeneche@maprtech.com>
wrote:

> You can safely ignore the TestImpersonationMetadata error, it's most
> likely caused by my change
>
> On Fri, Aug 14, 2015 at 12:36 PM, Abdel Hakim Deneche <
> adeneche@maprtech.com> wrote:
>
>> In addition to test*VectorReallocation errors I am seeing frequently on
>> my linux VM, today I saw the following error multiple times on my VM and
>> once on my Macbook:
>>
>> Tests in error:
>>>   TestImpersonationMetadata>BaseTestQuery.closeClient:239 ยป IllegalState
>>> Failure...
>>> java.lang.IllegalStateException: Failure while closing accountor.
>>> Expected private and shared pools to be set to initial values.  However,
>>> one or more were not.  Stats are
>>> zone init allocated delta
>>> private 0 0 0
>>> shared 3221225472 3214166779 7058693.
>>> at
>>> org.apache.drill.exec.memory.AtomicRemainder.close(AtomicRemainder.java:200)
>>> ~[classes/:na]
>>> at org.apache.drill.exec.memory.Accountor.close(Accountor.java:390)
>>> ~[classes/:na]
>>> at
>>> org.apache.drill.exec.memory.TopLevelAllocator.close(TopLevelAllocator.java:185)
>>> ~[classes/:na]
>>> at
>>> org.apache.drill.exec.server.BootStrapContext.close(BootStrapContext.java:75)
>>> ~[classes/:na]
>>> at com.google.common.io.Closeables.close(Closeables.java:77)
>>> ~[guava-14.0.1.jar:na]
>>> at com.google.common.io.Closeables.closeQuietly(Closeables.java:108)
>>> ~[guava-14.0.1.jar:na]
>>> at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:294)
>>> ~[classes/:na]
>>> at
>>> org.apache.drill.exec.server.Drillbit$ShutdownThread.run(Drillbit.java:332)
>>> ~[classes/:na]
>>
>>
>> On Wed, Aug 5, 2015 at 2:52 PM, Chris Westin <chriswestin42@gmail.com>
>> wrote:
>>
>>> Given that the difference is just
>>>
>>> > java.lang.Exception: Unexpected exception,
>>> > expected<org.apache.drill.exec.exception.OversizedAllocationException>
>>> but
>>> > was<org.apache.drill.exec.memory.OutOfMemoryRuntimeException>
>>>
>>> The question of "what constitutes an oversized allocation?" comes to
>>> mind.
>>> Is this test fragile relative to being run in different environments?
>>> I haven't seen the test so how is the determination that something is
>>> oversized made? It seems like that criterion sometimes fails, and we get
>>> an
>>> OOM because whatever the request is is still very large.
>>>
>>>
>>> On Wed, Aug 5, 2015 at 2:26 PM, Hanifi Gunes <hgunes@maprtech.com>
>>> wrote:
>>>
>>> > I don't seem to be able to re-prod this. Let me look at this and
>>> update you
>>> > all.
>>> >
>>> > On Thu, Aug 6, 2015 at 12:03 AM, Abdel Hakim Deneche <
>>> > adeneche@maprtech.com>
>>> > wrote:
>>> >
>>> > > I didn't make any change, I'm running 2 forks (the default). I got
>>> those
>>> > > errors 3 times now, 2 on a linux VM and 1 on a linux physical node
>>> > >
>>> > > On Wed, Aug 5, 2015 at 1:03 PM, Hanifi Gunes <hgunes@maprtech.com>
>>> > wrote:
>>> > >
>>> > > > Did you tighten your memory settings? How many forks are you
>>> running
>>> > > with?
>>> > > > I bet you are truly running out of memory while executing this
>>> > particular
>>> > > > test case.
>>> > > >
>>> > > > -H+
>>> > > >
>>> > > > On Wed, Aug 5, 2015 at 8:56 PM, Sudheesh Katkam <
>>> skatkam@maprtech.com>
>>> > > > wrote:
>>> > > >
>>> > > > > b2bbd99 committed on July 6th introduced the test.
>>> > > > >
>>> > > > > > On Aug 5, 2015, at 10:21 AM, Jinfeng Ni <jinfengni99@gmail.com
>>> >
>>> > > wrote:
>>> > > > > >
>>> > > > > > In that case,  we probably need do binary search to
figure out
>>> > which
>>> > > > > recent
>>> > > > > > patch is causing this problem.
>>> > > > > >
>>> > > > > > On Wed, Aug 5, 2015 at 10:03 AM, Abdel Hakim Deneche
<
>>> > > > > adeneche@maprtech.com>
>>> > > > > > wrote:
>>> > > > > >
>>> > > > > >> Just got those errors on master too
>>> > > > > >>
>>> > > > > >> On Wed, Aug 5, 2015 at 9:07 AM, Abdel Hakim Deneche
<
>>> > > > > adeneche@maprtech.com
>>> > > > > >>>
>>> > > > > >> wrote:
>>> > > > > >>
>>> > > > > >>> I'm seeing those errors intermittently when
building my
>>> private
>>> > > > > branch, I
>>> > > > > >>> don't believe I made any change that would have
caused them.
>>> > Anyone
>>> > > > > seen
>>> > > > > >>> them too ?
>>> > > > > >>>
>>> > > > > >>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> testBitVectorReallocation(org.apache.drill.exec.record.vector.TestValueVector)
>>> > > > > >>>> Time elapsed: 2.043 sec  <<< ERROR!
>>> > > > > >>>> java.lang.Exception: Unexpected exception,
>>> > > > > >>>>
>>> > > >
>>> expected<org.apache.drill.exec.exception.OversizedAllocationException>
>>> > > > > >> but
>>> > > > > >>>>
>>> was<org.apache.drill.exec.memory.OutOfMemoryRuntimeException>
>>> > > > > >>>> at java.nio.Bits.reserveMemory(Bits.java:658)
>>> > > > > >>>> at
>>> java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
>>> > > > > >>>> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> io.netty.buffer.UnpooledUnsafeDirectByteBuf.allocateDirect(UnpooledUnsafeDirectByteBuf.java:108)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> io.netty.buffer.UnpooledUnsafeDirectByteBuf.<init>(UnpooledUnsafeDirectByteBuf.java:69)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:50)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> io.netty.buffer.PooledByteBufAllocatorL.newDirectBuffer(PooledByteBufAllocatorL.java:130)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> io.netty.buffer.PooledByteBufAllocatorL.directBuffer(PooledByteBufAllocatorL.java:171)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.drill.exec.memory.TopLevelAllocator.buffer(TopLevelAllocator.java:100)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.drill.exec.memory.TopLevelAllocator.buffer(TopLevelAllocator.java:116)
>>> > > > > >>>> at
>>> > > > org.apache.drill.exec.vector.BitVector.reAlloc(BitVector.java:139)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.drill.exec.record.vector.TestValueVector.testBitVectorReallocation(TestValueVector.java:125)
>>> > > > > >>>
>>> > > > > >>>
>>> > > > > >>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> testFixedVectorReallocation(org.apache.drill.exec.record.vector.TestValueVector)
>>> > > > > >>>> Time elapsed: 0.436 sec  <<< ERROR!
>>> > > > > >>>> java.lang.Exception: Unexpected exception,
>>> > > > > >>>>
>>> > > >
>>> expected<org.apache.drill.exec.exception.OversizedAllocationException>
>>> > > > > >> but
>>> > > > > >>>>
>>> was<org.apache.drill.exec.memory.OutOfMemoryRuntimeException>
>>> > > > > >>>> at java.nio.Bits.reserveMemory(Bits.java:658)
>>> > > > > >>>> at
>>> java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
>>> > > > > >>>> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> io.netty.buffer.UnpooledUnsafeDirectByteBuf.allocateDirect(UnpooledUnsafeDirectByteBuf.java:108)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> io.netty.buffer.UnpooledUnsafeDirectByteBuf.<init>(UnpooledUnsafeDirectByteBuf.java:69)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:50)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> io.netty.buffer.PooledByteBufAllocatorL.newDirectBuffer(PooledByteBufAllocatorL.java:130)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> io.netty.buffer.PooledByteBufAllocatorL.directBuffer(PooledByteBufAllocatorL.java:171)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.drill.exec.memory.TopLevelAllocator.buffer(TopLevelAllocator.java:100)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.drill.exec.memory.TopLevelAllocator.buffer(TopLevelAllocator.java:116)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.drill.exec.vector.UInt4Vector.allocateBytes(UInt4Vector.java:187)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.drill.exec.vector.UInt4Vector.allocateNew(UInt4Vector.java:177)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.drill.exec.record.vector.TestValueVector.testFixedVectorReallocation(TestValueVector.java:85)
>>> > > > > >>>
>>> > > > > >>>
>>> > > > > >>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> testVariableVectorReallocation(org.apache.drill.exec.record.vector.TestValueVector)
>>> > > > > >>>> Time elapsed: 0.788 sec  <<< ERROR!
>>> > > > > >>>> java.lang.Exception: Unexpected exception,
>>> > > > > >>>>
>>> > > >
>>> expected<org.apache.drill.exec.exception.OversizedAllocationException>
>>> > > > > >> but
>>> > > > > >>>>
>>> was<org.apache.drill.exec.memory.OutOfMemoryRuntimeException>
>>> > > > > >>>> at java.nio.Bits.reserveMemory(Bits.java:658)
>>> > > > > >>>> at
>>> java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
>>> > > > > >>>> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> io.netty.buffer.UnpooledUnsafeDirectByteBuf.allocateDirect(UnpooledUnsafeDirectByteBuf.java:108)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> io.netty.buffer.UnpooledUnsafeDirectByteBuf.<init>(UnpooledUnsafeDirectByteBuf.java:69)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:50)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> io.netty.buffer.PooledByteBufAllocatorL.newDirectBuffer(PooledByteBufAllocatorL.java:130)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> io.netty.buffer.PooledByteBufAllocatorL.directBuffer(PooledByteBufAllocatorL.java:171)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.drill.exec.memory.TopLevelAllocator.buffer(TopLevelAllocator.java:100)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.drill.exec.memory.TopLevelAllocator.buffer(TopLevelAllocator.java:116)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.drill.exec.vector.VarCharVector.allocateNew(VarCharVector.java:372)
>>> > > > > >>>> at
>>> > > > > >>>>
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.drill.exec.record.vector.TestValueVector.testVariableVectorReallocation(TestValueVector.java:142)
>>> > > > > >>>
>>> > > > > >>>
>>> > > > > >>> Thanks
>>> > > > > >>> --
>>> > > > > >>>
>>> > > > > >>> Abdelhakim Deneche
>>> > > > > >>>
>>> > > > > >>> Software Engineer
>>> > > > > >>>
>>> > > > > >>>  <http://www.mapr.com/>
>>> > > > > >>>
>>> > > > > >>>
>>> > > > > >>> Now Available - Free Hadoop On-Demand Training
>>> > > > > >>> <
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
>>> > > > > >>>
>>> > > > > >>>
>>> > > > > >>
>>> > > > > >>
>>> > > > > >>
>>> > > > > >> --
>>> > > > > >>
>>> > > > > >> Abdelhakim Deneche
>>> > > > > >>
>>> > > > > >> Software Engineer
>>> > > > > >>
>>> > > > > >>  <http://www.mapr.com/>
>>> > > > > >>
>>> > > > > >>
>>> > > > > >> Now Available - Free Hadoop On-Demand Training
>>> > > > > >> <
>>> > > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
>>> > > > > >>>
>>> > > > > >>
>>> > > > >
>>> > > > >
>>> > > >
>>> > >
>>> > >
>>> > >
>>> > > --
>>> > >
>>> > > Abdelhakim Deneche
>>> > >
>>> > > Software Engineer
>>> > >
>>> > >   <http://www.mapr.com/>
>>> > >
>>> > >
>>> > > Now Available - Free Hadoop On-Demand Training
>>> > > <
>>> > >
>>> >
>>> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>>
>> --
>>
>> Abdelhakim Deneche
>>
>> Software Engineer
>>
>>   <http://www.mapr.com/>
>>
>>
>> Now Available - Free Hadoop On-Demand Training
>> <http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available>
>>
>
>
>
> --
>
> Abdelhakim Deneche
>
> Software Engineer
>
>   <http://www.mapr.com/>
>
>
> Now Available - Free Hadoop On-Demand Training
> <http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available>
>



-- 

Abdelhakim Deneche

Software Engineer

  <http://www.mapr.com/>


Now Available - Free Hadoop On-Demand Training
<http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message