Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Date: Thu, 22 Mar 2012 11:04:22 +0000 (UTC)
From: "Uwe Schindler (Commented) (JIRA)" <jira@apache.org>
To: dev@lucene.apache.org
Message-ID: 
 <1572664365.1815.1332414262328.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <465341840.11785.1331706218099.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (LUCENE-3867)
 RamUsageEstimator.NUM_BYTES_ARRAY_HEADER and other constants are incorrect
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/LUCENE-3867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13235500#comment-13235500 ] 

Uwe Schindler commented on LUCENE-3867:
---------------------------------------

Thanks for the insight.

When thinking about the reordering, I am a littel bit afraid about the "optimization" in the shallow sizeOf(Class<?>). This optimiaztion does not recurse to superclasses, as it assumes, that all field offsets are greater than those of the superclass, so finding the maximum does not need to recurse up (so it early exits).

This is generally true (also in the above printout), but not guaranteed. E.g. JRockit does it partly (it reuses space inside the superclass area to locate the byte from the subclass). In the above example still the order of fields is always Super-Sub-SubSub, but if the ordeing in the JRockit example would be like:

{noformat}
@ 8  1 Super.superByte
@ 9  7 Sub.subByte
@16  8 Super.subLong
@24  8 Sub.subLong
@32  8 SubSub.subSubLong
@40  8 SubSub.subSubByte
@48    sizeOf(SubSub.class instance)
{noformat}

The only thing the JVM cannot change is field offsets between sub classes (so the field offset of the superclass is inherited), but it could happen that *new* fields are located between super's fields (see above - it's unused space). This would also allow casting and so on (it's unused space in superclass). Unfortunately with that reordering the maximum field offset in the subclass is no longer guaranteed to be greater.

I would suggest that we remove the "optimization" in the shallow class size method. It's too risky in my opinion to underdetermine the size, because the maximum offset in the subclass is < the maximum offset in the superclass.

I hope my explanation was understandable... :-)

Dawid, what do you thing, should we remove the "optimization"? Patch is easy.
                
> RamUsageEstimator.NUM_BYTES_ARRAY_HEADER and other constants are incorrect
> --------------------------------------------------------------------------
>
>                 Key: LUCENE-3867
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3867
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: core/index
>            Reporter: Shai Erera
>            Assignee: Uwe Schindler
>            Priority: Trivial
>             Fix For: 3.6, 4.0
>
>         Attachments: LUCENE-3867-3.x.patch, LUCENE-3867-compressedOops.patch, LUCENE-3867.patch, LUCENE-3867.patch, LUCENE-3867.patch, LUCENE-3867.patch, LUCENE-3867.patch, LUCENE-3867.patch, LUCENE-3867.patch, LUCENE-3867.patch, LUCENE-3867.patch, LUCENE-3867.patch, LUCENE-3867.patch, LUCENE-3867.patch, LUCENE-3867.patch, LUCENE-3867.patch, LUCENE-3867.patch, LUCENE-3867.patch, LUCENE-3867.patch
>
>
> RamUsageEstimator.NUM_BYTES_ARRAY_HEADER is computed like that: NUM_BYTES_OBJECT_HEADER + NUM_BYTES_INT + NUM_BYTES_OBJECT_REF. The NUM_BYTES_OBJECT_REF part should not be included, at least not according to this page: http://www.javamex.com/tutorials/memory/array_memory_usage.shtml
> {quote}
> A single-dimension array is a single object. As expected, the array has the usual object header. However, this object head is 12 bytes to accommodate a four-byte array length. Then comes the actual array data which, as you might expect, consists of the number of elements multiplied by the number of bytes required for one element, depending on its type. The memory usage for one element is 4 bytes for an object reference ...
> {quote}
> While on it, I wrote a sizeOf(String) impl, and I wonder how do people feel about including such helper methods in RUE, as static, stateless, methods? It's not perfect, there's some room for improvement I'm sure, here it is:
> {code}
> 	/**
> 	 * Computes the approximate size of a String object. Note that if this object
> 	 * is also referenced by another object, you should add
> 	 * {@link RamUsageEstimator#NUM_BYTES_OBJECT_REF} to the result of this
> 	 * method.
> 	 */
> 	public static int sizeOf(String str) {
> 		return 2 * str.length() + 6 // chars + additional safeness for arrays alignment
> 				+ 3 * RamUsageEstimator.NUM_BYTES_INT // String maintains 3 integers
> 				+ RamUsageEstimator.NUM_BYTES_ARRAY_HEADER // char[] array
> 				+ RamUsageEstimator.NUM_BYTES_OBJECT_HEADER; // String object
> 	}
> {code}
> If people are not against it, I'd like to also add sizeOf(int[] / byte[] / long[] / double[] ... and String[]).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org