commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hope, Matthew" <Matthew.H...@capitalone.com>
Subject RE: [primitives] Open for business
Date Tue, 02 Sep 2003 11:31:35 GMT
the implementation is VM dependant - when creating individual primitives the
sizes often are all the same for short,float, int etc.

However an _array_ of primitives is a different matter.

If you run the following code with small numbers (1,2,3,4,5,6,7,8 etc) you
will see that the toal space usage goes up in steps as it would appear 8
bytes are allocated to the arrays at a time but that it makes full use of
those bytes using 2 per short and 4 per int.

The class I use for this is below. it would be relatively simple to make it
use any Collection type but of course boxing would make it much easier to
write. Ultimately if you wish to store millions of primitives then putting
them in an array (or a whole bunch of arrays if each one is sufficiently
large to offset the wastage at the 8 byte borders) is the way to go space
wise.

I only wrote Short, Int and Long because that was all I needed the info for
at the time I wrote the class, extending it to cover the other primitives
and their wrappers isn't hard.

package quicktest.jvm;

/**
 * this tells you the size overhead for objects and primitives. it <strong>
 * depends</strong> on valid values returned from
 * Runtime.getRuntime().freeMemory() and the Runtime.getRuntime().gc() will
 * clear up appropriately.<br>
 * Make sure you use identical -Xms and -Xmx values or the result can be
bogus
 */
public class SizeTester
{
    private final int objectCount;
    private String className;
    private Object hookToPreventGC;

    /**
     * Constructor for the SizeTester object
     *
     * @param count             number of Objects to create
     * @param classOrPrimitive
     */
    public SizeTester(int count, String classOrPrimitive)
    {
        objectCount = count;
        className = classOrPrimitive;
    }

    /**
     * The main program for the SizeTester class
     *
     * @param args           The command line arguments
     * @exception Exception
     */
    public static void main(String[] args)
        throws Exception
    {
        SizeTester test = null;
        if (args.length == 1) {
            test = new SizeTester(1000000, args[0]);
        } else if (args.length == 2) {
            test = new SizeTester(Integer.parseInt(args[0]), args[1]);
        } else {
            test = new SizeTester(1000000, "java.lang.String");
        }
        test.run();
    }


    private void run()
    {
        System.out.println("attempting to create " + className + "[" +
objectCount + "]");
        Runtime.getRuntime().gc();
        long start = Runtime.getRuntime().freeMemory();
        System.out.println("starting with " + start + " bytes");
        hookToPreventGC = createObjectHook();
        Runtime.getRuntime().gc();
        long end = Runtime.getRuntime().freeMemory();
        System.out.println("ending with " + end + " bytes");
        long difference = start - end;
        if (difference < 0) {
            System.out.println("the memory avail went up as the jvm
increased the heap size. use -Xms -Xmx to force it to start at a fied
value");
        } else {
            double average = (start - end) / (double) objectCount;
            System.out.println("total memory usage for " + objectCount + "
instances is " + difference + " bytes");
            System.out.println(average + " bytes used per object on
average");
        }
    }

    private Object createObjectHook()
    {
        if (className.equals("short")) {
            return buildShortPrimitiveArray();
        } else if (className.equals("int")) {
            return buildIntPrimitiveArray();
        } else if (className.equals("long")) {
            return buildLongPrimitiveArray();
        } else if (className.equals("java.lang.Short")) {
            return buildShortObjectArray();
        } else if (className.equals("java.lang.Integer")) {
            return buildIntegerObjectArray();
        } else if (className.equals("java.lang.Long")) {
            return buildLongObjectArray();
        } else {
            return buildObjectArray();
        }
    }

    private Object[] buildObjectArray()
    {
        Class clazz = null;
        try {
            clazz = Class.forName(className);
        } catch (ClassNotFoundException e) {
            e.printStackTrace();
            System.err.println("only support classes with empty argument
constructors or short, int and long primitives and their wrappers");
            System.exit(-1);
        }
        Object[] objectArray = new Object[objectCount];
        for (int i = 0; i < objectCount; i++) {
            Object instance = null;
            try {
                /* if you want to play with Strings rather dangerous intern
feauture do it here
                 * instance = ((String)clazz.newInstance()).intern();
                 */
                instance = clazz.newInstance();
            } catch (Exception e) {
                e.printStackTrace();
                System.err.println("only support classes with empty argument
constructors or java.lang.Short, java.lang.Integer and java.lang.Long");
                System.exit(-1);
            }
            objectArray[i] = instance;
        }
        return objectArray;
    }

    private Object[] buildShortObjectArray()
    {
        Object[] objectArray = new Object[objectCount];
        for (int i = 0; i < objectCount; i++) {
            objectArray[i] = new Short((short) 1);
        }
        return objectArray;
    }

    private Object[] buildIntegerObjectArray()
    {
        Object[] objectArray = new Object[objectCount];
        for (int i = 0; i < objectCount; i++) {
            objectArray[i] = new Integer(i);
        }
        return objectArray;
    }

    private Object[] buildLongObjectArray()
    {
        Object[] objectArray = new Object[objectCount];
        for (int i = 0; i < objectCount; i++) {
            objectArray[i] = new Long(i);
        }
        return objectArray;
    }

    private Object buildShortPrimitiveArray()
    {
        short[] array = new short[objectCount];
        for (int i = 0; i < objectCount; i++) {
            array[i] = (short) (i % Short.MAX_VALUE);
        }
        return array;
    }

    private Object buildIntPrimitiveArray()
    {
        int[] array = new int[objectCount];
        for (int i = 0; i < objectCount; i++) {
            array[i] = i;
        }
        return array;
    }

    private Object buildLongPrimitiveArray()
    {
        long[] array = new long[objectCount];
        for (int i = 0; i < objectCount; i++) {
            array[i] = (long) i;
        }
        return array;
    }
}
> -----Original Message-----
> From: Juozas Baliuka [mailto:baliuka@mwm.lt] 
> Sent: 02 September 2003 12:03
> To: Jakarta Commons Developers List
> Subject: Re: [primitives] Open for business
> 
> 
> 
> ----- Original Message -----
> From: "Hope, Matthew" <Matthew.Hope@capitalone.com>
> To: "'Jakarta Commons Developers List'" 
> <commons-dev@jakarta.apache.org>
> Sent: Tuesday, September 02, 2003 10:26 AM
> Subject: RE: [primitives] Open for business
> 
> 
> > perhaps less so from 1.5 onwards since for non critical 
> performance the
> ease
> > of autoboxing is hard to beat but when you are using 
> primitives not just
> > because of other interface concerns but for performance/space
> considerations
> > a pure primitive implementation should be considerably 
> better for large
> > sizes. An array of int[1000000] for example takes up 4 
> bytes per entry*
> > verses an Object array of Integers which takes 20 bytes per entry on
> > average.
> >
> > This is 5 times the space! When you use short[] vs Short 
> the difference is
> a
> > factor of 10
> 
> All of boolean, byte, short, int, object reference and float 
> use the same
> space in JVM (4 bytes) double and long are 8 bytes.
> And all of primityve wrappers have  single primityve field, 
> objects can use
> some internal JVM data structures, but I do not think factor 
> can be 10, how
> do you tested it ?
> 
> >
> > given that the underlying implementation of IntArrayList will almost
> > certainly be an array of ints this makes a lot of sense if 
> you care about
> > the size of your memory footprint...
> >
> > Matt
> >
> > * all done on windows 1.3.1 JVM - some jvms may do things 
> differently YMMV
> >
> > > -----Original Message-----
> > > From: Yoav Shapira [mailto:yoavs@apache.org]
> > > Sent: 02 September 2003 03:49
> > > To: Jakarta Commons Developers List
> > > Subject: Re: [primitives] Open for business
> > >
> > >
> > > Howdy,
> > > I may have missed the original discussion, but is primitives
> > > going to have
> > > much of a market given the relative proximity of JDK 1.5 with its
> > > autoboxing features?  Or are there a bunch of features
> > > planned that aren't
> > > going to be in JDK 1.5 anyways?
> > >
> > > Yoav Shapira
> > >
> > > On Tue, 2 Sep 2003, Stephen Colebourne wrote:
> > >
> > > > The [primitives] project has been created and is now ready
> > > for discussion,
> > > > debate and coding.
> > > >
> > > > At present, I have NOT removed anything from [collections].
> > > I believe that
> > > > there may be people with dependencies on the primitive code
> > > there. I do not
> > > > plan to release that code however, nor will its 
> prescence block a
> > > > [collections] release. I think that this will probably be
> > > the position until
> > > > [primitives] is ready.
> > > >
> > > > So... lets try to get [primitives] sorted ;-)
> > > >
> > > > Stephen
> > > >
> > > >
> > > >
> > > 
> ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: 
> commons-dev-unsubscribe@jakarta.apache.org
> > > > For additional commands, e-mail: 
> commons-dev-help@jakarta.apache.org
> > > >
> > > >
> > >
> > > 
> ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: 
> commons-dev-help@jakarta.apache.org
> > >
> >
> > 
> **************************************************************
> ************
> > The information transmitted herewith is sensitive 
> information intended
> only
> > for use by the individual or entity to which it is addressed. If the
> reader
> > of this message is not the intended recipient, you are 
> hereby notified
> that
> > any review, retransmission, dissemination, distribution, 
> copying or other
> > use of, or taking of any action in reliance upon this information is
> > strictly prohibited. If you have received this 
> communication in error,
> > please contact the sender and delete the material from your 
> computer.
> >
> > 
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: commons-dev-help@jakarta.apache.org
> >
> >
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
> 
 
**************************************************************************
The information transmitted herewith is sensitive information intended only
for use by the individual or entity to which it is addressed. If the reader
of this message is not the intended recipient, you are hereby notified that
any review, retransmission, dissemination, distribution, copying or other
use of, or taking of any action in reliance upon this information is
strictly prohibited. If you have received this communication in error,
please contact the sender and delete the material from your computer.

Mime
View raw message