jmeter-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ahmad Alnafoosi <anafo...@hotmail.com>
Subject RE: Concurrent GB File PUTs causing Out of Memory
Date Mon, 11 Jan 2016 21:17:24 GMT
Hi Philippe
I just saw the patch after you mentioned it.  Thank you for pointing that out.  
I will add that and compile Jmeter and test again. 

The patch is HttpClient4 specific.  can we port that fix to HttpClient3.1 as well since it
has preemptive basic auth which will make my life and I am sure many others easier.  

OR make HttpClient4 capable of doing the preemptive basic authentication? (another bug??)

thank you guys for your support.

Ahmad 

> Date: Mon, 11 Jan 2016 22:07:17 +0100
> Subject: Re: Concurrent GB File PUTs causing Out of Memory
> From: philippe.mouawad@gmail.com
> To: user@jmeter.apache.org
> 
> Hi,
> Felix provided a patch to you to test it .
> Are you able to do so ?
> 
> Thanks
> 
> On Monday, January 11, 2016, Ahmad Alnafoosi <anafoosi@hotmail.com> wrote:
> 
> > Hi Felix, et. al.
> > I changed the HTTP Request Implementation from HttpClient 3.1 to 4.
> > That required significant changes since HttpClient4 implementation  does
> > not support preemptive basic authentication like 3.1.
> > http://stackoverflow.com/questions/12560494/jmeter-basic-authentication
> >
> > I had add HTTP HEADER MANAGER then manually plugin base64 encoded
> > credentials to get that running again.
> >
> > Even with that I am getting Out of Memory Exception with the start of 10
> > threads of 100MB Object HTTP put.
> >
> > Any other thoughts??
> >
> > thanks
> >
> > Ahmad
> >
> > > From: anafoosi@hotmail.com <javascript:;>
> > > To: user@jmeter.apache.org <javascript:;>
> > > Subject: RE: Concurrent GB File PUTs causing Out of Memory
> > > Date: Sun, 10 Jan 2016 15:07:14 -0600
> > >
> > > Hi Felix
> > > Thank you for your response.
> > >
> > > I am using HTTP Request Sampler.
> > >
> > > -The PUT requests are sending large amount of data (GBs) which are
> > failing with OOM exception.  I am sending User name and password as Basic
> > auth.
> > > the configs that I modified in the HTTP PUT request sampler are
> > >   -Implantation: HttpClient3.1 (4.0 did not work for me with Basic Auth).
> > >   -Path (Variables based on Counter)
> > >   -Send Files with the Request (path to Object in Local Directory with
> > GB size).
> > >
> > > In the HTTP Request Default I have added
> > >   1- Server as a variable
> > >   2- Port as a Variable.
> > >
> > > So I think what is standing out from what you mentioned is the HTTP
> > implantation.  I will work on Changing that to 4.0 and see if I can see why
> > is it failing.
> > >
> > > -The GET requests are will be receiving large amount of data (GBs) but
> > > are not working since the prerequisite above PUTs failed.  Is there any
> > memory optimization needed for GET to receive (GB) files??
> > >
> > >
> > > thanks
> > >
> > > Ahmad
> > >
> > >
> > >
> > >
> > >
> > > > Subject: Re: Concurrent GB File PUTs causing Out of Memory
> > > > To: user@jmeter.apache.org <javascript:;>
> > > > From: felix.schumacher@internetallee.de <javascript:;>
> > > > Date: Sat, 9 Jan 2016 22:15:05 +0100
> > > >
> > > > Am 08.01.2016 um 22:23 schrieb Ahmad Alnafoosi:
> > > > > Hi
> > > > > I have a jmeter performance multi-stage test using HTTP REST API.
> > > > > The test has 10 concurrent users
> > > > > The test fails on Object Upload (PUT) that are 100MB  (and larger)
> > in size with Out of Memory Exception.
> > > > Which sampler did you use and what was the configuration you used in
> > it?
> > > > Is your REST API sending back large amounts of data?
> > > >
> > > > I have tested a simple upload page with a 400 MB file and 100
> > concurrent
> > > > threads. The memory usage did not pass 250 MB.
> > > >
> > > > I used "HTTP Request" sampler with implementation "HttpClient4", method
> > > > "POST" and added the file in the panel "Send File With the Request:".
> > > >
> > > > Regards,
> > > >   Felix
> > > > >
> > > > >
> > > > > I did some research on Jmeter Memory optimization.  I did not find
> > anything on the issue that I am dealing with specifically.
> > > > >
> > > > > So I followed optimization recommendations and did the following:
> > > > > 1- Removed all listeners from the test
> > > > > 2- Running from non gui command line
> > > > > 3- Saving JTL as CSV
> > > > > 4- doing all the graph and summery as post process outside the test.
> > > > > 5- Experimented and Increase heap up to 20 GB as follows
> > > > > HEAP="-Xms20g -Xmx20g" (this allowed the 100MB to pass but still
> > failed at 1GB and 5GB file sizes).
> > > > > 6- I added HTTP Cache manager and enabled (Clear Cache on Each
> > Iteration) and Limited (Max Number of elements in cache to 3).
> > > > >
> > > > > All of the above did not help in getting 10 concurrent users to PUT
> > 5GB.!!
> > > > >
> > > > > Is there a way to do 10 users uploading 5GB concurrently with the
> > default 512MB?
> > > > > Does Jmeter cache in its heap all of the objects that it uploads?
> > > > > What is the optimum heap size that is recommended for the above
> > scenario??
> > > > >
> > > > > your help is appreciated.
> > > > >
> > > > >
> > > > > thanks
> > > > >
> > > > > Ahmad
> > > > >
> > > > >
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> > <javascript:;>
> > > > For additional commands, e-mail: user-help@jmeter.apache.org
> > <javascript:;>
> > > >
> > >
> >
> 
> 
> 
> -- 
> Cordialement.
> Philippe Mouawad.
 		 	   		  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message