incubator-jena-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Simon Helsen (JIRA)" <>
Subject [jira] [Commented] (JENA-91) extremely large buffer is being created in ObjectFileStorage
Date Fri, 19 Aug 2011 21:23:27 GMT


Simon Helsen commented on JENA-91:


I think I finally have something which reflects what we are doing in your test case. I have
attached a patch which illustrates this. And when I run it, I get things like

Quad: object cannot be null


Different ids allocated: expected [000000000000000A], got [0000000000000014]

which I have seen in my own corruptions (in my own test environment I see the complete stack
trace, but the messages are the same) What I do is based on something we do all the time in
our own framework: we write and read some important meta properties as triples into the same
graph and predicate (but update values). I don't think your current tests do this kind of

Anyhow, let me know if what I do in the test case is wrong and why since we do this ourselves
right now :-) If it is not wrong, you may be able to debug this and nail the problem. Note
that we do exactly this kind of thing in TDB 0.8.7, so if it is wrong now, I'd like to know
why it wasn't wrong in TDB 0.8.7

> extremely large buffer is being created in ObjectFileStorage
> ------------------------------------------------------------
>                 Key: JENA-91
>                 URL:
>             Project: Jena
>          Issue Type: Bug
>          Components: TDB
>            Reporter: Simon Helsen
>            Assignee: Andy Seaborne
>            Priority: Critical
>         Attachments: TestTransSystem.patch, TestTransSystem2.patch
> I tried to debug the OME and check why a bytebuffer is causing my native memory to explode
in almost no time. It all seems to happen in this bit of code in com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage
(lines 243 onwards)
>   // No - it's in the underlying file storage.
>         lengthBuffer.clear() ;
>         int x =, loc) ;
>         if ( x != 4 )
>             throw new FileException(""+loc+")["+filesize+"]["+file.size()+"]:
Failed to read the length : got "+x+" bytes") ;
>         int len = lengthBuffer.getInt(0) ;
>         ByteBuffer bb = ByteBuffer.allocate(len) ;
> My debugger shows that x==4. It also shows the lengthBuffer has the following content:
[111, 110, 61, 95]. This amounts to the value of len=1869495647, which is rather a lot :-)
Obviously, the next statement (ByteBuffer.allocate) causes the OME.

This message is automatically generated by JIRA.
For more information on JIRA, see:


View raw message