jmeter-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Philippe Mouawad <philippe.moua...@gmail.com>
Subject Re: Concurrent GB File PUTs causing Out of Memory
Date Mon, 11 Jan 2016 21:26:58 GMT
On Monday, January 11, 2016, Ahmad Alnafoosi <anafoosi@hotmail.com> wrote:

> Hi Philippe
> I just saw the patch after you mentioned it.  Thank you for pointing that
> out.
> I will add that and compile Jmeter and test again.
>
> Thanks

> The patch is HttpClient4 specific.  can we port that fix to HttpClient3.1
> as well since it has preemptive basic auth which will make my life and I am
> sure many others easier.

HttpClient 3 is very old, we plan to deprecate it il next version and drop
it in the following one.


>
> OR make HttpClient4 capable of doing the preemptive basic authentication?
> (another bug??)
>
> there is an enhancement that is il bugzilla, it is planned but I cannot
give you a date.

> thank you guys for your support.
>
> Ahmad
>
> > Date: Mon, 11 Jan 2016 22:07:17 +0100
> > Subject: Re: Concurrent GB File PUTs causing Out of Memory
> > From: philippe.mouawad@gmail.com <javascript:;>
> > To: user@jmeter.apache.org <javascript:;>
> >
> > Hi,
> > Felix provided a patch to you to test it .
> > Are you able to do so ?
> >
> > Thanks
> >
> > On Monday, January 11, 2016, Ahmad Alnafoosi <anafoosi@hotmail.com
> <javascript:;>> wrote:
> >
> > > Hi Felix, et. al.
> > > I changed the HTTP Request Implementation from HttpClient 3.1 to 4.
> > > That required significant changes since HttpClient4 implementation
> does
> > > not support preemptive basic authentication like 3.1.
> > >
> http://stackoverflow.com/questions/12560494/jmeter-basic-authentication
> > >
> > > I had add HTTP HEADER MANAGER then manually plugin base64 encoded
> > > credentials to get that running again.
> > >
> > > Even with that I am getting Out of Memory Exception with the start of
> 10
> > > threads of 100MB Object HTTP put.
> > >
> > > Any other thoughts??
> > >
> > > thanks
> > >
> > > Ahmad
> > >
> > > > From: anafoosi@hotmail.com <javascript:;> <javascript:;>
> > > > To: user@jmeter.apache.org <javascript:;> <javascript:;>
> > > > Subject: RE: Concurrent GB File PUTs causing Out of Memory
> > > > Date: Sun, 10 Jan 2016 15:07:14 -0600
> > > >
> > > > Hi Felix
> > > > Thank you for your response.
> > > >
> > > > I am using HTTP Request Sampler.
> > > >
> > > > -The PUT requests are sending large amount of data (GBs) which are
> > > failing with OOM exception.  I am sending User name and password as
> Basic
> > > auth.
> > > > the configs that I modified in the HTTP PUT request sampler are
> > > >   -Implantation: HttpClient3.1 (4.0 did not work for me with Basic
> Auth).
> > > >   -Path (Variables based on Counter)
> > > >   -Send Files with the Request (path to Object in Local Directory
> with
> > > GB size).
> > > >
> > > > In the HTTP Request Default I have added
> > > >   1- Server as a variable
> > > >   2- Port as a Variable.
> > > >
> > > > So I think what is standing out from what you mentioned is the HTTP
> > > implantation.  I will work on Changing that to 4.0 and see if I can
> see why
> > > is it failing.
> > > >
> > > > -The GET requests are will be receiving large amount of data (GBs)
> but
> > > > are not working since the prerequisite above PUTs failed.  Is there
> any
> > > memory optimization needed for GET to receive (GB) files??
> > > >
> > > >
> > > > thanks
> > > >
> > > > Ahmad
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > > Subject: Re: Concurrent GB File PUTs causing Out of Memory
> > > > > To: user@jmeter.apache.org <javascript:;> <javascript:;>
> > > > > From: felix.schumacher@internetallee.de <javascript:;>
> <javascript:;>
> > > > > Date: Sat, 9 Jan 2016 22:15:05 +0100
> > > > >
> > > > > Am 08.01.2016 um 22:23 schrieb Ahmad Alnafoosi:
> > > > > > Hi
> > > > > > I have a jmeter performance multi-stage test using HTTP REST
API.
> > > > > > The test has 10 concurrent users
> > > > > > The test fails on Object Upload (PUT) that are 100MB  (and
> larger)
> > > in size with Out of Memory Exception.
> > > > > Which sampler did you use and what was the configuration you used
> in
> > > it?
> > > > > Is your REST API sending back large amounts of data?
> > > > >
> > > > > I have tested a simple upload page with a 400 MB file and 100
> > > concurrent
> > > > > threads. The memory usage did not pass 250 MB.
> > > > >
> > > > > I used "HTTP Request" sampler with implementation "HttpClient4",
> method
> > > > > "POST" and added the file in the panel "Send File With the
> Request:".
> > > > >
> > > > > Regards,
> > > > >   Felix
> > > > > >
> > > > > >
> > > > > > I did some research on Jmeter Memory optimization.  I did not
> find
> > > anything on the issue that I am dealing with specifically.
> > > > > >
> > > > > > So I followed optimization recommendations and did the following:
> > > > > > 1- Removed all listeners from the test
> > > > > > 2- Running from non gui command line
> > > > > > 3- Saving JTL as CSV
> > > > > > 4- doing all the graph and summery as post process outside the
> test.
> > > > > > 5- Experimented and Increase heap up to 20 GB as follows
> > > > > > HEAP="-Xms20g -Xmx20g" (this allowed the 100MB to pass but still
> > > failed at 1GB and 5GB file sizes).
> > > > > > 6- I added HTTP Cache manager and enabled (Clear Cache on Each
> > > Iteration) and Limited (Max Number of elements in cache to 3).
> > > > > >
> > > > > > All of the above did not help in getting 10 concurrent users
to
> PUT
> > > 5GB.!!
> > > > > >
> > > > > > Is there a way to do 10 users uploading 5GB concurrently with
the
> > > default 512MB?
> > > > > > Does Jmeter cache in its heap all of the objects that it uploads?
> > > > > > What is the optimum heap size that is recommended for the above
> > > scenario??
> > > > > >
> > > > > > your help is appreciated.
> > > > > >
> > > > > >
> > > > > > thanks
> > > > > >
> > > > > > Ahmad
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> <javascript:;>
> > > <javascript:;>
> > > > > For additional commands, e-mail: user-help@jmeter.apache.org
> <javascript:;>
> > > <javascript:;>
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
>



-- 
Cordialement.
Philippe Mouawad.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message