harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Blewitt" <alex.blew...@gmail.com>
Subject Re: Re: Re: [rant] Memory options in VM -- why is the default not 'unlimited'
Date Sun, 30 Jul 2006 10:45:19 GMT
On 30/07/06, Xiao-Feng Li <xiaofeng.li@gmail.com> wrote:
> Alex, you made good point. It is definitely possible to design GC with
> dynamic heap size, we will need is careful design for both convenience
> and performance. Besides the points I mentioned previously,
> performance tradeoff with heap size is yet another a factor. On one
> hand, the heap size has a minimal to keep all the live objects during
> the execution; on the other hand, a larger heap size trigers GC less
> frequently. So sometimes the GC needs to decide if it should increase
> the heap size or just trigger a collection, assuming the heap size is
> enough to hold all the live objects.

Yes, there are obvious conditions like this which need to be decided
:-) However, I'm sure that heuristics could play a part; if the GC
remembers what it did from the previous run/runs, then perhaps it
could guide what happens next. For example, if the last GC freed up
(say) less than 5% of memory, the next decision would be to request
more memory. Or, if a GC doesn't free up the requisite amount of
memory, then a request for more memory could follow on as part of the
same GC cycle.

I'm pretty sure that hashtables have a built-in load factor (about 70%
utilisation of buckets?) and that when the hashtable goes above the
load factor, it rehashes. Could we not have some kind of load factor
in the memory, such that a post-GC compaction triggers more memory
when the load factor goes above a certain percentage (e.g. 70%)?
Obviously, the numbers would have to be fine-tuned (or be subsituted
with another -XX:GCLoadFactor non-non-standard option ;-)

Of course, it's possible that this idea wouldn't work well at all ...

> Yes, to allocate/release blocks when needed is possible. I'd give you
> more cases that requires careful design. consider a case . For
> example, if the block is bigger than an OS page in size, even the GC
> design can perfectly allocate/release its blocks, the system memory is
> still fragmented.

I would assume that a VM block size would be a multiple of OS page
sizes, for obvious performance reasons, including the one that you
mention. I can't see there being any benefit in not doing so ... can
we determine what the OS page size is, either statically or
dynamically? For example, on Mac OS X the page size is 4k, as returned
by sysconf(_SC_PAGE_SIZE) in unistd.h ...

Perhaps 4k would be a bit small, so some multiple of those is probably
needed to round up the size. But some power-of-two multiple would
probably be useful; you could take the page size and <<1 until it was
bigger than a more reasonable number (e.g. 32k?)

> Another case is, GC sometimes needs to align the
> block to certain size boundary, which also leaves fragments in system
> space. Yet another case is, if the system memory space is already
> seriously fragmented, GC may not be able to utilize the remaining
> vacant space although the sum size of the fragments can be large.

Yes, all true. The same happens with filing systems, too. But note
that if the memory is already fragmented, asking the VM to start by
attempting to claim a large space on a heap is almost certainly
destined to fail. At least but having the memory split into blocks,
it's more likely that you'll be able to obtain memory space and/or use
it.

> Many of these point to OS/JVM interactions finally. I've been looking
> at this area. :-)

Cool :-) I'm just an ideas man, mostly -- my C background is pretty
slim, or else I'd like to help out. But I've still got to get the
Pack200 stuff up and running, first.

Speaking of ideas, I've often thought that allowing a VM to 'zone' out
pages of related code would be an excellent idea. For example, if each
ClassLoader had its own zone of pages, and that new instances of
classes owned by that ClassLoader were created in its  zone, then
you'd have a way of partitioning the memory space without having to
change any Java code. So, instead of having a global memory space with
two or three generations, you'd have a set of them instead (perhaps
each with their own generations).

This would be great for J2EE servers that typically create their own
ClassLoaders, normally one per app. Furthermore, when the app is
stopped (and the ClassLoader unloaded) then all of the memory
'assigned to' that ClassLoader could be returned to the OS. It would
also mean that if a J2EE server had many apps, and not all of them
were being used, then the page(s) in the ClassLoader's zone could be
paged out, and they'd not be needed to be paged in again until the app
was requested.

Back to the file system analogy; instead of having a single filesystem
for all files (and the fragmentation that woudl go along with that),
it's like having a separate filesystem for each J2EE app so that the
app's files don't get mixed with each other.

I've often thought this would be a really cool thing to do. Does the
idea have any merit?

Alex.

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message