Mailing-List: contact jena-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: jena-dev@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of shelsen@ca.ibm.com designates
 32.97.182.144 as permitted sender)
In-Reply-To: 
 <1058071772.39175.1316294828982.JavaMail.tomcat@hel.zones.apache.org>
References: 
 <216849827.17585.1312833027005.JavaMail.tomcat@hel.zones.apache.org>
 <1058071772.39175.1316294828982.JavaMail.tomcat@hel.zones.apache.org>
To: jena-dev@incubator.apache.org
Cc: jena-dev@incubator.apache.org
MIME-Version: 1.0
Subject: Re: [jira] [Commented] (JENA-91) extremely large buffer is being
 created in
 ObjectFileStorage
Message-ID: 
 <OF19F519DF.721650D2-ON85257910.0048EE06-85257910.004906F7@ca.ibm.com>
From: Simon Helsen <shelsen@ca.ibm.com>
Date: Mon, 19 Sep 2011 09:17:31 -0400
Content-Type: multipart/alternative;
 boundary="=_alternative 004906F785257910_="

--=_alternative 004906F785257910_=
Content-Type: text/plain; charset="US-ASCII"

btw, this is why we (at IBM) have never adopted mapped mode (a.o. 
reasons). We have to support Windows and we need the ability to clear a 
directory and reuse it, so...

But see my comments in the work item, even in direct mode, we have 
problems on Windows

Simon


From:
"Andy Seaborne (JIRA)" <jira@apache.org>
To:
jena-dev@incubator.apache.org
Date:
09/17/2011 05:27 PM
Subject:
[jira] [Commented] (JENA-91) extremely large buffer is being created in 
ObjectFileStorage


    [ 
https://issues.apache.org/jira/browse/JENA-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13107264#comment-13107264 
] 

Andy Seaborne commented on JENA-91:
-----------------------------------

JENA-115 shows that reusing directories is problematic on MS windows when 
using mapped mode.  TestTransSystem works for direct mode but not for 
mapped mode for me.

The error is that existing files don't get deleted so later code see 
inconsistent junk.

TestTransSystem reuses directories via static clean().

Simon - do you, in your internal tests, reuse directories and delete index 
files?

> extremely large buffer is being created in ObjectFileStorage
> ------------------------------------------------------------
>
>                 Key: JENA-91
>                 URL: https://issues.apache.org/jira/browse/JENA-91
>             Project: Jena
>          Issue Type: Bug
>          Components: TDB
>            Reporter: Simon Helsen
>            Assignee: Andy Seaborne
>            Priority: Critical
>         Attachments: JENA-91_NodeTableTrans_r1159121.patch, 
TestTransSystem.patch, TestTransSystem2.patch, TestTransSystem3.patch, 
TestTransSystem4.patch
>
>
> I tried to debug the OME and check why a bytebuffer is causing my native 
memory to explode in almost no time. It all seems to happen in this bit of 
code in com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage (lines 243 
onwards)
>   // No - it's in the underlying file storage.
>         lengthBuffer.clear() ;
>         int x = file.read(lengthBuffer, loc) ;
>         if ( x != 4 )
>             throw new 
FileException("ObjectFile.read("+loc+")["+filesize+"]["+file.size()+"]: 
Failed to read the length : got "+x+" bytes") ;
>         int len = lengthBuffer.getInt(0) ;
>         ByteBuffer bb = ByteBuffer.allocate(len) ;
> My debugger shows that x==4. It also shows the lengthBuffer has the 
following content: [111, 110, 61, 95]. This amounts to the value of 
len=1869495647, which is rather a lot :-) Obviously, the next statement 
(ByteBuffer.allocate) causes the OME.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

 
--=_alternative 004906F785257910_=--