Mailing-List: contact jena-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: jena-dev@incubator.apache.org
Date: Tue, 9 Aug 2011 10:30:27 +0000 (UTC)
From: "Andy Seaborne (JIRA)" <jira@apache.org>
To: jena-dev@incubator.apache.org
Message-ID: 
 <836121263.19646.1312885827213.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <216849827.17585.1312833027005.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (JENA-91) extremely large buffer is being
 created in ObjectFileStorage
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/JENA-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081554#comment-13081554 ] 

Andy Seaborne commented on JENA-91:
-----------------------------------

I believe this is caused by a deeper bug in TxTDB prototype which is causing internal corruption.  Mad buffer sizes are a symptom - they can be a symptom for quite a few things in, it just happens to show up in ObjectFileStorage

The reason it spikes and recovers is that the large allocate succeeds, but the read of the item itself leads to an inconsistency and the code throws an exception.  Then the GC quickly reclaims the byte buffer which was only ever local to the method so is a prime candidate for recycling even if direct (non-mmap) memory.

Sanity check added.

Please add any more information to this item as it turns up.

Left open for now.  (It may just disappear when the real bug gets fixed without realizing it also covers this.)


> extremely large buffer is being created in ObjectFileStorage
> ------------------------------------------------------------
>
>                 Key: JENA-91
>                 URL: https://issues.apache.org/jira/browse/JENA-91
>             Project: Jena
>          Issue Type: Bug
>          Components: TDB
>         Environment: Windows (and I presume any little endian system)
>            Reporter: Simon Helsen
>            Priority: Critical
>
> I tried to debug the OME and check why a bytebuffer is causing my native memory to explode in almost no time. It all seems to happen in this bit of code in com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage (lines 243 onwards)
>   // No - it's in the underlying file storage.
>         lengthBuffer.clear() ;
>         int x = file.read(lengthBuffer, loc) ;
>         if ( x != 4 )
>             throw new FileException("ObjectFile.read("+loc+")["+filesize+"]["+file.size()+"]: Failed to read the length : got "+x+" bytes") ;
>         int len = lengthBuffer.getInt(0) ;
>         ByteBuffer bb = ByteBuffer.allocate(len) ;
> My debugger shows that x==4. It also shows the lengthBuffer has the following content: [111, 110, 61, 95]. This amounts to the value of len=1869495647, which is rather a lot :-) Obviously, the next statement (ByteBuffer.allocate) causes the OME.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira