harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Regis <xu.re...@gmail.com>
Subject Re: harmony vm as a lib
Date Mon, 04 Jan 2010 06:12:52 GMT
On 2010-01-04 12:55, Tim Prepscius wrote:
> That was the plan.  I am linking hyluni as a static already.  I think
> there is some additional initialization I must do somewhere.
> But at this point, I'm more concerned with the 33 megabytes being used
> for just a partial initialization.
>
>
> If these 33 megabytes are coming because somewhere something is
> allocating 32 megabytes reserve heap, and then is going to
> sub-allocate that for java objects, then I can probably modify this
> behavior somehow to allocate as needed.  But if 33 megabytes are being
> allocated for java initializations-  then it would take a whole lot of
> work before I can use harmony.
>
> It's really hard for me to believe that 33 megabytes are coming from
> loaded classes.  I'll investigate more this week.  Hmm.. Maybe the
> garbage collector needs to be invoked or something?

IIRC, the java heap is about 4MB for simple HelloWorld, I guess the most of 
memory is consumed by vm, not classlib.

>
>
>
> Does anyone know what the minimum memory usage is for an initialized
> jvm before executing client code.. meaning after all of the necessary
> initialization classes are loaded, but before any client app dependent
> classes are loaded, etc etc etc etc?

Will some profiling tools like Valgrind or strace can help to figure out who 
allocate the largest memory blocks?

>
>
>
>
> Currently the libraries I'm linking statically are:
> apr-1.lib, ch.lib, em.lib, encoder.lib, harmonyvm.lib, hycommon.lib,
> hyprt.lib, hythr.lib, hyzip.lib, icuuc34d.lib, interpreter.lib,
> port.lib, verifier.lib, vmi.lib, zlib.lib, hyluni.lib,
> gc_gen_uncomp.lib
>
> I'm doing the static JNI initialization via a table I'm adding to by
> hand.. I was hoping to add the minimal set needed for initialization
> and things I wanted to support, and then tell the linker to remove
> dead code, and see what it removed.
>
>
> Happy New Year!!!
>
> -tim
>
> ---
>
> Code for JNI hand done registry if that explanation was confusing:
>
>
> #include<map>
> #include<string>
> #include "natives_support.h"
>
> #include "org_apache_harmony_drlvm_VMHelper.h"
> #include "java_lang_VMClassRegistry.h"
> #include "java_lang_ClassLoader.h"
> #include "java_lang_System.h"
> #include "org_apache_harmony_vm_VMStack.h"
> #include "java_lang_VMExecutionEngine.h"
>
> // wherez header?
> extern "C"
> JNIEXPORT void JNICALL Java_java_io_FileDescriptor_syncImpl (JNIEnv *
> env, jobject recv);
> extern "C"
> JNIEXPORT void JNICALL
> Java_java_io_FileDescriptor_oneTimeInitialization (JNIEnv * env,
> jclass fdClazz);
>
> typedef std::map<std::string, GenericFunctionPointer>  StaticNativeMap;
> static StaticNativeMap staticNatives;
>
> struct NameFunctionPair
> {
> 	const char *name;
> 	GenericFunctionPointer ptr;
> };
>
> #define MAKE_NAMEFUNCTIONPAIR(x) { #x, (GenericFunctionPointer)&x }
>
> NameFunctionPair staticNativesToRegister[] =
> {
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_io_FileDescriptor_oneTimeInitialization),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_io_FileDescriptor_syncImpl),
>
> 	MAKE_NAMEFUNCTIONPAIR(Java_org_apache_harmony_drlvm_VMHelper_getPointerTypeSize),
> 	MAKE_NAMEFUNCTIONPAIR(Java_org_apache_harmony_drlvm_VMHelper_isCompressedRefsMode),
> 	MAKE_NAMEFUNCTIONPAIR(Java_org_apache_harmony_drlvm_VMHelper_isCompressedVTableMode),
> 	MAKE_NAMEFUNCTIONPAIR(Java_org_apache_harmony_drlvm_VMHelper_getCompressedModeVTableBaseOffset),
> 	MAKE_NAMEFUNCTIONPAIR(Java_org_apache_harmony_drlvm_VMHelper_getCompressedModeObjectBaseOffset),
> 	MAKE_NAMEFUNCTIONPAIR(Java_org_apache_harmony_drlvm_VMHelper_getObjectVtableOffset),
> 	MAKE_NAMEFUNCTIONPAIR(Java_org_apache_harmony_drlvm_VMHelper_getClassJLCHanldeOffset),
> 	MAKE_NAMEFUNCTIONPAIR(Java_org_apache_harmony_drlvm_VMHelper_getVtableClassOffset),
>
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMExecutionEngine_exit),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMExecutionEngine_getAssertionStatus),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMExecutionEngine_getAvailableProcessors),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMExecutionEngine_getProperties),
> //	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMExecutionEngine_loadLibrary),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMExecutionEngine_traceInstructions),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMExecutionEngine_traceMethodCalls),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMExecutionEngine_currentTimeMillis),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMExecutionEngine_nanoTime),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMExecutionEngine_mapLibraryName),
>
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_ClassLoader_findLoadedClass),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_ClassLoader_registerInitiatedClass),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_ClassLoader_defineClass0),
>
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_loadBootstrapClass),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_getClassNative),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_getClassLoader0),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_getComponentType),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_getDeclaredClasses),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_getDeclaredConstructors),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_getDeclaredFields),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_getDeclaredMethods),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_getDeclaringClass),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_getInterfaces),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_getModifiers),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_getName),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_getSuperclass),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_getSystemPackages),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_initializeClass),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_isArray),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_isAssignableFrom),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_isInstance),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_isPrimitive),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_linkClass),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_loadArray),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_loadLibrary),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_getEnclosingClass),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_getEnclosingMember),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_VMClassRegistry_getSimpleName),
>
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_System_setErrUnsecure),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_System_setInUnsecure),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_System_setOutUnsecure),
> 	MAKE_NAMEFUNCTIONPAIR(Java_java_lang_System_rethrow),
>
> 	MAKE_NAMEFUNCTIONPAIR(Java_org_apache_harmony_vm_VMStack_getCallerClass),
> 	MAKE_NAMEFUNCTIONPAIR(Java_org_apache_harmony_vm_VMStack_getClasses),
> 	MAKE_NAMEFUNCTIONPAIR(Java_org_apache_harmony_vm_VMStack_getStackState),
> 	MAKE_NAMEFUNCTIONPAIR(Java_org_apache_harmony_vm_VMStack_getStackClasses),
> 	MAKE_NAMEFUNCTIONPAIR(Java_org_apache_harmony_vm_VMStack_getStackTrace),
> 	MAKE_NAMEFUNCTIONPAIR(Java_org_apache_harmony_vm_VMStack_getClassLoader0),
> 	MAKE_NAMEFUNCTIONPAIR(Java_org_apache_harmony_vm_VMStack_getThreadStackTrace),
> 	{ NULL, NULL }
> };
>
> inline void populateStaticNativesUsingSet (NameFunctionPair *nameFunctionPairs)
> {
> 	for (NameFunctionPair *i=nameFunctionPairs; i->name!=NULL; ++i)
> 		staticNatives[i->name] = i->ptr;
> }
>
> inline void populateStaticNativeMap ()
> {
> 	static bool staticNativesPopulated = false;
> 	if (staticNativesPopulated)
> 		return;
>
> 	populateStaticNativesUsingSet (staticNativesToRegister);
> 	staticNativesPopulated = true;
> }
>
> GenericFunctionPointer findStaticNative (const char *symbol)
> {
>          // lock mutex?
>
> 	populateStaticNativeMap();
>
> 	StaticNativeMap::iterator i = staticNatives.find(symbol);
> 	if (i != staticNatives.end())
> 		return i->second;
>
> 	FILE *f = fopen ("nonfoundstatics.txt", "a+t");
> 	fputs (symbol, f);
> 	fputs ("\n", f);
> 	fclose (f);
>
> 	return NULL;
> }
>
>
>
> On 1/3/10, Regis<xu.regis@gmail.com>  wrote:
>> On 2010-01-02 13:42, Tim Prepscius wrote:
>>> Greetings,
>>>
>>> I write to give a small update and ask a couple questions.
>>>
>>> The update is: I'm still getting everything static-ified.  It seems
>>> when I enabled the missing kernel + natives, those depended on other
>>> natives which depended on other natives which depended on other
>>> natives and jars etc etc.  It seems I must include pretty much every
>>> library just to get past startup.
>>>
>>> I'm now up to "Java_java_io_FileDescriptor_oneTimeInitialization"
>>> HARMONY_CACHE_SET is failing, not sure why, maybe I need to call some
>>> other initialization method somewhere.
>>
>> It requires hyluni.dll from classlib.
>>
>>>
>>> So as I'm statically linking libraries as I'm including more and more
>>> dependencies, the executable size keeps on growing.
>>>
>>> Here are my current two questions:
>>>
>>>
>>> 1.  Before java initialization, which means after "startup code" has
>>> run, but before I call JNI_CreateJavaVM, process explorer is reporting
>>> that the memory used is already ~5 megabytes.
>>>
>>> After initialization (well at the point where it crashes now) process
>>> explorer is reporting that the program is taking ~33 megabytes.
>>>
>>> So my question is:  What is the minimum memory foot print of this java
>>> executable?  Anyone know?  Meaning post initialization, pre execute
>>> code.
>>
>> I think the minimum memory foot print of executable java not only include
>> JVM
>> itself but also some classes needed by JVM when starting. Even to run a
>> "HelloWorld" program, hundreds of classes will be loaded and also native
>> dlls
>> required by them, will you also static link them together?
>>
>>>
>>> Is this JVM allocating a whole ton of memory to begin?  If so how can
>>> I change this.  Surely all of those ~28 megabytes (so far) aren't
>>> actually bi-products of little initializations happening everywhere.
>>>
>>>
>>>
>>> 2. Does any one know the minimum disk foot print of the executable.  I
>>> guess meaning the base executable plus absolutely essential dlls
>>> (which it seems is turning out to be all of them, but I can't believe
>>> this to be true).
>>>
>>>
>>> -tim
>>>
>>>
>>> On Sun, Dec 27, 2009 at 8:34 AM, Alexey Varlamov
>>> <alexey.v.varlamov@gmail.com>   wrote:
>>>>> At this point I'm trying to get the classes of
>>>>> "luni-kernal-stubs.jar" (which I renamed to kernal.jar)
>>>> Bad idea. This is the point of your failure - those are really
>>>> compilation stubs, just a blueprint of classlib-vm API.
>>>> There are real kernel classes implemented in DRLVM, unluckily buried
>>>> inside the source tree: working_vm\vm\vmcore\src\kernel_classes. You
>>>> should use them instead of luni-kernel-stubs.jar.
>>>>
>>>>> and "luni.jar" to load.
>>>>> It seems to be getting through a few, and is crashing on
>>>>> java/lang/String.
>>>>> Why? I have no idea as of yet.  I guess that is tomorrow or Sunday.
>>>>
>>>> Just because String.intern() delegates to kernel classes which are not
>>>> implemented in your setup. All you need is to switch to correct
>>>> kernel.jar, it is normally built as a part of DRLVM.
>>>>
>>>> Good luck!
>>>> --
>>>> Alexey
>>>>
>>>
>>
>>
>> --
>> Best Regards,
>> Regis.
>>
>


-- 
Best Regards,
Regis.

Mime
View raw message