qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "question" <46970...@qq.com>
Subject Re: Memory keeping increasing when using Qpid, can any one help?
Date Fri, 20 Oct 2017 07:48:08 GMT
Hi Chris,


Good news. During test, we found the programs on different machines have different memory
perfomance.
Finally we found a monitor program is impacting the Qpid work, without the monitor program
the memory usage is stable.


So it is not problem of Qpid!


Thanks for your time:-)


B.R
Frank.




------------------ Original ------------------
From:  "question";<46970799@qq.com>;
Date:  Oct 20, 2017
To:  "users"<users@qpid.apache.org>; 

Subject:  Re:  Memory keeping increasing when using Qpid, can any one help?



Hi Chris,


Thanks for your mail. I see the memory usage more clearly. 
For the sender side, after I set a bigger capacity. I see the  memory usage is increase and
decrease periodically. 


But for the receiver side,  the memory usage is still keeping increasing. Here is the changed
code for both side:
Receiver:
Connection connection(broker, connectionOptions);
			connection.open();
			Session session = connection.createSession();
			Receiver receiver = session.createReceiver(address);
			receiver.setCapacity(1000);
			while (true)
			{		
				Message message;
				message.setContentObject("Hello world!");
				message.getContentObject().setEncoding("utf8");
				message = receiver.get(Duration::FOREVER);


				std::cout << message.getContent() << std::endl;
				std::cout << "To fetch: " << receiver.getAvailable() << std::endl; //changing,
most time 0
				std::cout << "UnAck: " << receiver.getUnsettled() << std::endl;  //changing,
most time 0
				std::cout << "Capacity: " << receiver.getCapacity() << std::endl;//1000


				session.acknowledge(true);
			
			}
			connection.close();



Sender:
                      Connection connection(broker, connectionOptions);
			connection.open();
			Session session = connection.createSession();
			Sender sender = session.createSender(address);
			sender.setCapacity(1000);
			for (int i = 0; i < 500000000; i++)
			{


				std::this_thread::sleep_for(std::chrono::milliseconds(10)); // to make it sending slower.
				Message message;
				message.setContentObject("Hello world!");
				message.getContentObject().setEncoding("utf8");
				sender.send(message);
				std::cout << "Available: " << sender.getAvailable() << std::endl; //changing
time to time
				std::cout << "UnAck: " << sender.getUnsettled() << std::endl; //changing
time to time
				std::cout << "Capacity: " << sender.getCapacity() << std::endl; // 1000


			}
			connection.close();
			return 0;



The receiver never used his Queque up to 1000, but why its memory usage is keep increasing?


"While you might expect that each message would
be destroyed as it goes out of scope in your loop, it so happens that this
may also not behave as expected: the Message type is a wrapper on a smart
pointer which I strongly suspect will be in use by the session until the
background session.acknowledge() operation has finished with it."


Seems it is not releasing the memory at all. Is there anyway to release them?


Thanks,
Frank




------------------ Original ------------------
From:  "Chris Richardson";<cr@fourc.eu>;
Send time: Thursday, Oct 19, 2017 7:22 PM
To: "users"<users@qpid.apache.org>; 

Subject:  Re: Memory keeping increasing when using Qpid, can any one help?



Fank,

You should bear in mind that the messaging interfaces are concurrent - in
the case of your sender, the sender.send(message) call is non-blocking and
will queue messages up to the capacity of the sender (default: 50) and will
then block until the queue has a slot available. This behaviour can be
changed (if you wish) by calling sender.send(message, true) to make each
call block. You can read more about it here: https://qpid.apache.org/
releases/qpid-cpp-1.36.0/messaging-api/book/replay.html

Similarly, the receiver will pre-fetch messages from the broker up to its
capacity setting, but the default is zero so this probably isn't relevant.
What may be relevant is that session.acknowledge() is by default
non-blocking (and can be changed to a blocking call with
session.acknowledge(true)). While you might expect that each message would
be destroyed as it goes out of scope in your loop, it so happens that this
may also not behave as expected: the Message type is a wrapper on a smart
pointer which I strongly suspect will be in use by the session until the
background session.acknowledge() operation has finished with it.

Just a small caveat on the implementation of your example sender - because
the send is a background operation, you will probably find your connection
gets closed and your process terminates before some of the last 50 messages
are sent. Calling session.sync(true) before closing the connection will
prevent this.

Would these considerations explain the memory usage you're seeing?


On 19 October 2017 at 11:13, question <46970799@qq.com> wrote:

> Dear all,
>
> Sorry to disturb you. I am new to Qpid.
> When I tried to write a simple sender and receiver by splitting the
> HelloWorld example in Qpid cpp 1.36.  I met memory problem.
>
> HelloWord1 is responsible receiving data as below:
>
>                        Connection connection(broker, connectionOptions);
> connection.open();
>
> Session session = connection.createSession();
> Receiver receiver = session.createReceiver(address);
>
> while (true)
> {
>
> Message message;
> message.setContentObject("Hello world!");
> message.getContentObject().setEncoding("utf8");
> // sender.send(message);
>
> message = receiver.fetch(Duration::FOREVER);
> std::cout << message.getContent() << std::endl;
> session.acknowledge();
>
> }
> connection.close();
>
> HelloWorld2 is responsible sending data as below:
>
> Connection connection(broker, connectionOptions);
> connection.open();
> Session session = connection.createSession();
> Sender sender = session.createSender(address);
> for (int i = 0; i < 500000000; i++)
> {
> Message message;
> message.setContentObject("Hello world!");
> message.getContentObject().setEncoding("utf8");
> sender.send(message);
>
> }
> connection.close();
>
> And I use Java qpid-broker-6.1.1 as broker.
>
> I saw the both processes HelloWord1.exe  and HelloWorld2.exe  memory usage
> increase quickly.
> Am I using the Qpid in a wrong way? Maybe I am too stupid. Can anyone
> help? :-(
>
> The two cpp files are attached...
>
> Thanks,
> Frank.
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>



-- 

*Chris Richardson*, System Architect
cr@fourc.eu


*FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu
<http://www.fourc.eu/>*

*Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook
<http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter
<http://bit.ly/fourctw>!*
Mime
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message