jackrabbit-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Guggisberg <stefan.guggisb...@gmail.com>
Subject Re: no space left on device-exception by Property.getStream()
Date Fri, 12 Apr 2013 15:07:41 GMT
On Fri, Apr 12, 2013 at 4:53 PM, Julian Reschke <julian.reschke@gmx.de> wrote:
> On 2013-04-12 16:24, Stefan Guggisberg wrote:
>>
>> On Fri, Apr 12, 2013 at 3:29 PM, Ulrich <Forums@gombers.de> wrote:
>>>
>>> Retrieving data is completely sequential, no concurrent processing at
>>> all. I changed the code to session.logout() and session.connect() after
>>> every step, this didn't help.
>>> So the code works like this:
>>> while (String path : pathList) {
>>>       Session session = ...
>>>       Node currentNode = session.getNode(path);
>>>       Node filenode = Node.getNode("jcr:content");
>>>       Property jcrdata = filenode.getProperty("jcr:data");
>>>       InputStream is = jcrdata.getBinary().getStream();
>>>       is.close();
>>>       session.logout();
>>> }
>>>
>>> To be honest, this is not the exact code; the logic is spread over two
>>> classes - but it shows the effective data flow.
>>> Nevertheless - the problem remains.
>>> But when I retry the whole sequence later on, I get the same result -
>>> this means the buffer has been cleared in the meantime.
>>>
>>> It looks as if there is a kind of garbage collector
>>
>>
>> yes, it's your jvm's garbage collector.
>>
>>> , running asynchronously not fast enough for avoiding the error but being
>>> done after a while.
>>
>>
>> yes, that's expected behaviour. the jvm's garbage collection runs
>> async with a low priority (unless you're
>> running out of memory of course).
>>
>>> I tried to track the storage space by 'df -vk' but couldn't see a problem
>>> here.
>>
>>
>> did you check inodes as well ('df -i /')?
>>
>> as i already mentioned: reading a lot of binaries will create
>> a lot of temp files. those temp files will eventually be deleted
>> once the gc determines that they're not used anymore (see [1]).
>> but this can take some time, due to the async nature of java's gc.
>>
>> an example:
>>
>> assume you have 500mb free disk space.
>> now when you're reading 1k binaries from the repository, 1mb size each,
>> in a loop,  you're likely going to see said exception.
>> ...
>
>
> But why don't we purge the transient file once it's not needed anymore?

because determining whether it's not needed anymore is the tricky part.
and that's one thing the jvm's garbage collection is pretty good at.

> Relying on GC appears to be an anti-pattern to me...

this so called anti-pattern did work pretty well during the past 8 years... ;)

>
> Best regards, Julian

Mime
View raw message