harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From will pugh <willp...@sourcelabs.com>
Subject Re: [rant] Memory options in VM -- why is the default not 'unlimited'
Date Sun, 30 Jul 2006 19:29:10 GMT
Isn't Full GCs a big problem?  If have a bunch of pages in virtual 
memory, and need to do a full heap walk.  Won't you basically have an 
exercise in page faults?

If you never do a full GC, aren't you going to potentially "leak" memory 
forever, and potentially get super fragmentation in the higher 
generations (some things will no longer be referenced, some will and 
when you need the ones that are you will be pulling in mostly empty 
pages), thus causing a larger working set, which can then cause many 
more page faults, which can than distroy your performance?

    --Will

Mikhail Fursov wrote:

> As we see with -Xmx unlimit we have a lot of new issues that are not 
> clear
> right now.
> IMO the best solution to make them clear is to support this option 
> (but not
> default) and to have an experience.
> I'll try to summarize the problems stated above:
>
> 1) It's unclear for GC when to collect or when to expand
>
> 2) Memory usage of simple applications will be increased
>
> 3) GC needs continues memory blocks. (Why? From mutator's side neither 
> bump
> pointer nor free-list allocators reqiure it?)
>
> 4) JVM has a different memory contract with
> OS, not like general applications: JVM allocates but never frees
> memory. Here we can
> teach GC to release memory to OS or teach
> JVMs to share memory to improve resources reuse.
>
> Anything missed?
>
> May be if we can solve all
> the problems above our JVM will become more attractive to users then 
> others
> because it behave like a 'normal application' but not like a separate
> OS inside another OS? Why not to try?
>
>
>
>
> On 7/30/06, Xiao-Feng Li <xiaofeng.li@gmail.com> wrote:
>
>>
>> Alex, you made good point. It is definitely possible to design GC with
>> dynamic heap size, we will need is careful design for both convenience
>> and performance. Besides the points I mentioned previously,
>> performance tradeoff with heap size is yet another a factor. On one
>> hand, the heap size has a minimal to keep all the live objects during
>> the execution; on the other hand, a larger heap size trigers GC less
>> frequently. So sometimes the GC needs to decide if it should increase
>> the heap size or just trigger a collection, assuming the heap size is
>> enough to hold all the live objects.
>>
>> Yes, to allocate/release blocks when needed is possible. I'd give you
>> more cases that requires careful design. consider a case . For
>> example, if the block is bigger than an OS page in size, even the GC
>> design can perfectly allocate/release its blocks, the system memory is
>> still fragmented. Another case is, GC sometimes needs to align the
>> block to certain size boundary, which also leaves fragments in system
>> space. Yet another case is, if the system memory space is already
>> seriously fragmented, GC may not be able to utilize the remaining
>> vacant space although the sum size of the fragments can be large.
>>
>> Many of these point to OS/JVM interactions finally. I've been looking
>> at this area. :-)
>>
>> Thanks,
>> xiaofeng
>>
>> On 7/30/06, Alex Blewitt <alex.blewitt@gmail.com> wrote:
>> > On 29/07/06, Salikh Zakirov <Salikh.Zakirov@intel.com> wrote:
>> > > Salikh Zakirov wrote:
>> > > > Sorry, the first version would lead to integer overflow and the to
>> assertion
>> > > > failure. The updated patch should work. It runs Hello on
>> Windows/ia32 with a warning
>> > > > about reducing maximum heap size to the virtual address range it
>> could allocate.
>> > >
>> > > This is done because GC needs to have contiguous address range 
>> for the
>> heap,
>> > > and allocates virtual address range upfront.
>> >
>> > It would seem that for an infinite amount of memory, attempting to
>> > allocate a contiguous address range for the heap would be a problem.
>> > It's also probably why other VMs have the -Xmx option, and attempt to
>> > obtain the range all in one go.
>> >
>> > However, if the GC is generational, why does it need to be contiguous?
>> > Why not allocate chunks per generation? Or, for that matter, why can't
>> > it be split into pages/blocks of memory? That way, you wouldn't need
>> > to attempt to allocate it all up front, but rather you could tack a
>> > new page/block onto the end of the memory list. Obviously, there would
>> > be special cases (like when allocating a large array that is greater
>> > than the page/block size) but all of these problems have been solved
>> > by filing systems in the past. I mean, it's not like creating a ZIP
>> > file attempts to allocate a contiguous block of 256Mb and then fills
>> > it up as files are added. (However, that is pretty much what happens
>> > when you create a DMG on Mac OS X ...)
>> >
>> > Incidentally, a very good book on file system design discusses the Be
>> > Filing System (BFS) and it's available at
>> > http://www.nobius.org/~dbg/practical-file-system-design.pdf
>> >
>> > Note that I'm mostly asking these questions in an attempt to find out
>> > why it can't be done, and/or how much would need to change to be able
>> > to do something like this instead. I can imagine that if there were
>> > some kind of page-on-demand memory obtaining, then you wouldn't need a
>> > -Xmx parameter (unless you wanted it), and that you'd be able to deal
>> > with this kind of DLL-in-the-middle problem that you saw on Windows.
>> >
>> > Alex.
>> >
>> > ---------------------------------------------------------------------
>> > Terms of use : http://incubator.apache.org/harmony/mailing.html
>> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>
>

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message