lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shai Erera (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-3867) RamUsageEstimator.NUM_BYTES_ARRAY_HEADER is incorrect
Date Wed, 14 Mar 2012 16:46:50 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-3867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13229332#comment-13229332
] 

Shai Erera commented on LUCENE-3867:
------------------------------------

bq. What I mean is that without looking at the code I would expect sizeOf(String[] N) to return
the actual memory taken by an array of strings.

So you mean you'd want sizeOf(String[]) be just that?

{code}
return alignObjectSize(NUM_BYTES_ARRAY_HEADER + NUM_BYTES_OBJECT_REF * arr.length);
{code}

I don't mind. I just thought that since we know how to compute sizeOf(String), we can use
that. It's an extreme case, I think, that someone will want to compute the size of String[]
which share same char[] instance ... but I don't mind if it bothers you that much, to simplify
it and document that it computes the raw size of the String[].

But I don't think that we should change sizeOf(String) to not count the char[] size. It's
part of the object, and really it's String, not like we're trying to compute the size of a
general object.

bq. Same as with other objects – traverse its fields and count them

RUE already has .estimateRamUsage(Object) which does that through reflection. I think that
sizeOf(String) can remain fast as it is now, with the comment that it my over-estimate if
the String is actually a sub-string of one original larger string. In the worse case, we'll
just be over-estimating.
                
> RamUsageEstimator.NUM_BYTES_ARRAY_HEADER is incorrect
> -----------------------------------------------------
>
>                 Key: LUCENE-3867
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3867
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: core/index
>            Reporter: Shai Erera
>            Assignee: Shai Erera
>            Priority: Trivial
>             Fix For: 3.6, 4.0
>
>         Attachments: LUCENE-3867-compressedOops.patch, LUCENE-3867.patch
>
>
> RamUsageEstimator.NUM_BYTES_ARRAY_HEADER is computed like that: NUM_BYTES_OBJECT_HEADER
+ NUM_BYTES_INT + NUM_BYTES_OBJECT_REF. The NUM_BYTES_OBJECT_REF part should not be included,
at least not according to this page: http://www.javamex.com/tutorials/memory/array_memory_usage.shtml
> {quote}
> A single-dimension array is a single object. As expected, the array has the usual object
header. However, this object head is 12 bytes to accommodate a four-byte array length. Then
comes the actual array data which, as you might expect, consists of the number of elements
multiplied by the number of bytes required for one element, depending on its type. The memory
usage for one element is 4 bytes for an object reference ...
> {quote}
> While on it, I wrote a sizeOf(String) impl, and I wonder how do people feel about including
such helper methods in RUE, as static, stateless, methods? It's not perfect, there's some
room for improvement I'm sure, here it is:
> {code}
> 	/**
> 	 * Computes the approximate size of a String object. Note that if this object
> 	 * is also referenced by another object, you should add
> 	 * {@link RamUsageEstimator#NUM_BYTES_OBJECT_REF} to the result of this
> 	 * method.
> 	 */
> 	public static int sizeOf(String str) {
> 		return 2 * str.length() + 6 // chars + additional safeness for arrays alignment
> 				+ 3 * RamUsageEstimator.NUM_BYTES_INT // String maintains 3 integers
> 				+ RamUsageEstimator.NUM_BYTES_ARRAY_HEADER // char[] array
> 				+ RamUsageEstimator.NUM_BYTES_OBJECT_HEADER; // String object
> 	}
> {code}
> If people are not against it, I'd like to also add sizeOf(int[] / byte[] / long[] / double[]
... and String[]).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message