harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aleksey Shipilev" <aleksey.shipi...@gmail.com>
Subject Re: [DRLVM][GC] Let's switch back to GCv4.1 --> Re: [snapshot] Freezing the code for Milestone build?
Date Fri, 27 Apr 2007 09:49:23 GMT
Hi, Xiao-Feng!

On 4/27/07, Xiao-Feng Li <xiaofeng.li@gmail.com> wrote:
> This means GCv5 was not thoroughly tested with 1.5GB heap size yet. I
> will learn from Sergey on how to build DRLVM for 1.5GB heap size and
> go more testing then after.

I will explain the trick "How to get DRL VM acquire >1Gb of heap".

The problem is predefined base address in some of the libraries: they
are going to load at predefined location in system memory thus causing
fragmentation of possible heap space. Since there are problems on
allocating non-continuous heap, there's no possibility to allocate big
chunk of memory. So we will need to relocate some libraries to another
location.

This time we have only by-hand solution, which could be
machine-dependent. The idea is simple: try to allocate as much as we
can and see what blocks us. I've used the simple test:

	public class HeapTest {

		public static void main(String args[]) throws Exception {
			System.out.println("HeapTest started");
			System.in.read();
		}

	}

Then I run this test with max of possible heap:

	harmony-hdk-r532358/jdk/jre/bin/java -Xms900M -Xmx900M
-XX:vm.dlls=gc_gen.dll -XX:gc.use_large_page=true HeapTest

...and see the DLL distribution across the memory: you could use
ProcessExplorer from SysInternals.com to obtain that list. I have this
picture:

	Name		Base		Size
	unicode.nls	0x260000	0x16000
	locale.nls	0x280000	0x34000
	sortkey.nls	0x2C0000	0x41000
	sorttbls.nls	0x310000	0x6000
	ctype.nls	0x330000	0x3000
	zlib1.dll	0x3A0000	0x13000
	odbc32.dll	0x3C0000	0x3D000
	java.exe	0x400000	0xD000
	harmonyvm.dll	0x510000	0x424000
	dbghelp.dll	0x940000	0xA8000
	odbcint.dll	0x11E0000	0x17000
	em.dll		0x1330000	0x40000
	jitrino.dll	0x1380000	0x410000
	gc_gen.dll	0x17A0000	0x2C000
	hysig.dll	0x17E0000	0x6000
	hytext.dll	0x17F0000	0x6000
	hyzlib.dll	0x1E70000	0x13000
	vmi.dll	0x1E90000	0x6000
	hynio.dll	0x1EA0000	0x6000
	hyluni.dll	0x1EB0000	0x23000
	hyarchive.dll	0x1EE0000	0xD000
	icuinterface34.dll	0x25F0000	0x17000
	hythr.dll	0x10000000	0x407000
	hyprt.dll	0x11100000	0x18000
	[ ------------------ here goes the chunk ------------------ ]
	icuuc34.dll	0x4A800000	0xC8000
	icuin34.dll	0x4A900000	0xAA000
	icudt34.dll	0x4AD00000	0x870000
	[ --------------- and here goes the chunk -------------- ]
	mswsock.dll	0x71B20000	0x41000
	ws2help.dll	0x71BF0000	0x8000
	ws2_32.dll	0x71C00000	0x17000
	comdlg32.dll	0x762B0000	0x4A000
	userenv.dll	0x76920000	0xC4000
	psapi.dll	0x76B70000	0xB000
	secur32.dll	0x76F50000	0x13000
	user32.dll	0x77380000	0x92000
	comctl32.dll	0x77420000	0x103000
	comctl32.dll	0x77530000	0x97000
	version.dll	0x77B90000	0x8000
	msvcrt.dll	0x77BA0000	0x5A000
	gdi32.dll	0x77C00000	0x48000
	rpcrt4.dll	0x77C50000	0x9F000
	shlwapi.dll	0x77DA0000	0x52000
	kernel32.dll	0x77E40000	0x102000
	advapi32.dll	0x77F50000	0x9C000
	msvcr71.dll	0x7C340000	0x56000
	ntdll.dll	0x7C800000	0xC0000
	shell32.dll	0x7C8D0000	0x803000
	
Let's try to merge these chunks together. We will use the editbin
utility from MS Platform SDK. I checked that editbin is on my $PATH
and then run the following script:
	
	editbin /LARGEADDRESSAWARE java.exe

	editbin /LARGEADDRESSAWARE /rebase:base=0x84000000 hythr.dll
	editbin /LARGEADDRESSAWARE /rebase:base=0x84500000 hysig.dll
	editbin /LARGEADDRESSAWARE /rebase:base=0x84550000 hyprt.dll
	editbin /LARGEADDRESSAWARE /rebase:base=0x84600000 hyzlib.dll
	editbin /LARGEADDRESSAWARE /rebase:base=0x84650000 hytext.dll
	editbin /LARGEADDRESSAWARE /rebase:base=0x84700000 vmi.dll
	editbin /LARGEADDRESSAWARE /rebase:base=0x84750000 hyluni.dll
	editbin /LARGEADDRESSAWARE /rebase:base=0x84800000 hyarchive.dll
	editbin /LARGEADDRESSAWARE /rebase:base=0x84850000 hynio.dll
	editbin /LARGEADDRESSAWARE /rebase:base=0x84900000 hycharset.dll

	editbin /LARGEADDRESSAWARE /rebase:base=0x85500000 gc_cc.dll
	editbin /LARGEADDRESSAWARE /rebase:base=0x85500000 gc_gen.dll
	editbin /LARGEADDRESSAWARE /rebase:base=0x85600000 harmonyvm.dll
	editbin /LARGEADDRESSAWARE /rebase:base=0x86100000 zlib1.dll
	editbin /LARGEADDRESSAWARE /rebase:base=0x86200000 em.dll
	editbin /LARGEADDRESSAWARE /rebase:base=0x86300000 jitrino.dll
	editbin /LARGEADDRESSAWARE /rebase:base=0x87000000 hysecurity.dll
	editbin /LARGEADDRESSAWARE /REBASE:BASE=0x87030000 icuuc34.dll
	editbin /LARGEADDRESSAWARE /REBASE:BASE=0x87100000 icudt34.dll
	editbin /LARGEADDRESSAWARE /REBASE:BASE=0x87200000 icuin34.dll
	editbin /LARGEADDRESSAWARE /REBASE:BASE=0x87300000 icuin34.dll
	editbin /LARGEADDRESSAWARE /REBASE:BASE=0x87400000 icuin34.dll
	editbin /LARGEADDRESSAWARE /REBASE:BASE=0x87500000 ICUInterface34.dll

either on jre/bin and jre/bin/default directories.

The idea is simple too: we are moving the libraries at the end of
memory and base them there. Note that this script is really OS/build
dependent since the initial distribution is unknown.

After applying these transformations I was able to run the test again
with larger heap:

	$ harmony-hdk-r532358/jdk/jre/bin/java -Xms1700M -Xmx1700M
-XX:vm.dlls=gc_gen.dll -XX:gc.use_large_page=true  HeapTest
	GC use large pages.
	HeapTest started

Horray! Then I make sure that it worked out fine:

	Name	Base	Size
	unicode.nls	0x260000	0x16000
	locale.nls	0x280000	0x34000
	sortkey.nls	0x2C0000	0x41000
	sorttbls.nls	0x310000	0x6000
	ctype.nls	0x330000	0x3000
	java.exe	0x400000	0xD000
	odbcint.dll	0xCB0000	0x17000
	[ ------------------ a BI-I-I-I-G chunk here --------------- ]
	icuinterface34.dll	0x71AA0000	0x17000
	mswsock.dll	0x71B20000	0x41000
	ws2help.dll	0x71BF0000	0x8000
	ws2_32.dll	0x71C00000	0x17000
	icuin34.dll	0x72520000	0xAA000
	comdlg32.dll	0x762B0000	0x4A000
	userenv.dll	0x76920000	0xC4000
	psapi.dll	0x76B70000	0xB000
	secur32.dll	0x76F50000	0x13000
	user32.dll	0x77380000	0x92000
	comctl32.dll	0x77420000	0x103000
	comctl32.dll	0x77530000	0x97000
	version.dll	0x77B90000	0x8000
	msvcrt.dll	0x77BA0000	0x5A000
	gdi32.dll	0x77C00000	0x48000
	rpcrt4.dll	0x77C50000	0x9F000
	shlwapi.dll	0x77DA0000	0x52000
	kernel32.dll	0x77E40000	0x102000
	advapi32.dll	0x77F50000	0x9C000
	msvcr71.dll	0x7C340000	0x56000
	ntdll.dll	0x7C800000	0xC0000
	shell32.dll	0x7C8D0000	0x803000
	hythr.dll	0x84000000	0x407000
	hysig.dll	0x84500000	0x6000
	hyprt.dll	0x84550000	0x18000
	hyzlib.dll	0x84600000	0x13000
	hytext.dll	0x84650000	0x6000
	vmi.dll	0x84700000	0x6000
	hyluni.dll	0x84750000	0x23000
	hyarchive.dll	0x84800000	0xD000
	hynio.dll	0x84850000	0x6000
	gc_gen.dll	0x85500000	0x2C000
	harmonyvm.dll	0x85600000	0x424000
	zlib1.dll	0x86100000	0x13000
	em.dll	0x86200000	0x40000
	jitrino.dll	0x86300000	0x410000
	odbc32.dll	0x86800000	0x3D000
	dbghelp.dll	0x86900000	0xA8000
	icuuc34.dll	0x87030000	0xC8000
	icudt34.dll	0x87100000	0x870000

Hopefully, my script will work OOB. If there any problems, we will try
to reiterate the moving one more time on another addresses. Note that
relocating of MS Windows system libraries is the challenge (and one
could consider this unfair) - since Windows will try to fight you :)

Thanks,
Aleksey Shipilev
Intel Enterprise Solutions Software Division

Mime
View raw message