directory-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel Lécharny <elecha...@gmail.com>
Subject Re: apacheds @ 100% permanently
Date Tue, 09 Jul 2013 13:03:46 GMT
Le 7/9/13 2:17 PM, Slavomir Kocka a écrit :
> Thanks for response.
>
> I'll write into your answers...
>
>
> On Jul 9, 2013, at 13:33 , Emmanuel Lécharny <elecharny@gmail.com> wrote:
>
>> Le 7/9/13 12:35 PM, Slavomir Kocka a écrit :
>>> The interesting point is, that we have two servers in mirror.
>>> Not using replication from 2.0, but doing writes separately.
>>> and the second server is just fine... In identical JVM and same hardware...
>>> The difference is, that websphere uses first one as main source, and second,
as spare...
>>>
>>> Few questions anyway.
>>> 1. Where I set memory limits? I didn't change it, so currently "top" command
shows up to 2 GB RAM occupied by apacheds. Can you please point me out to doc, or just leave
default? We are not using swap.
>> There is no configuration in ApacheDS per se, it's about the JVM memory
> I understand that, I just found some commented settings in wrapper.conf. I'm running
application using command "/etc/init.d/apacheds-2.0.0_M14-default start".
> So I'm not sure, where can I set JVM params, and if defaults are not sufficient...

Sorry, I thought you were starting the server in a different way. You
can modify the wrapper.conf file to increase the default memory used by
the server (the current default is 384Mb).
>
>> you set when you start the server (or websphere).
>>> 2. I cannot give you exact numbers, only if I enable logs. 
>> Rough estimation is good enough. There is a huge difference between 2
>> additions per second and 200...
> I believe, now we have like 10 additions per second, if everything goes fine. If not,
there is like one addition per 30 seconds... :(
Damn... Are you using spinning disks or SSD ?

Also what is the underlying FS ? (and if it's a spinning disk, what is
the disk speed)
> What we do is: We run 4 servers in parallel. Each of these servers is doing write sequentially.

Are all the servers on the same physical machine ?


> So in general, there are 4 threads trying to write... As these writes are synchronous
(we do not start new write until old one finishes) amount of writes per second depends on
apacheds server response... We need at least 10 writes/s.

Ok, let me give you some insight about the performances I get on my
machine :
- I have a SSD (256Gb), a OCX Vertex 3 (so it's not top-notch, but
pretty decent).
- when I run the AddPerf test we have, I can inject up to 73 entries per
second (15 000 entries are injected)
- the number of indexes are critical : the more index you have, the
slower the server will be. The default server already uses 7 indexes.

So it should be enough, as soon as you uses a SSD.

On a spinning disk, the performances will be drastically worse (probably
by an order of magnitude).

The reason is that we have to flush on disk every time we update the
backend, in order not to lose any information. Ok, it's slightly more
complicated than that, as we use a journal and a transction manager to
guareantee that no data will be lost, even if the server crashes.

Bottom line the backend we use (JDBM) is nothing but the fastest backend
on earth. We are currently working on a new backend (mavibot, see
https://cwiki.apache.org/confluence/display/labs/Mavibot) which is at
least 2.5 times faster (I re-ran the addPerf test and got 193 add/s, a
2.6x speed up), but it's not production ready yet.

We also did some perf tests with Berkeley DB JE, but the performances we
get are pretty much similar with JDBM, for a li which licence is not
compatible.

>
>>> What I can say, that there are four parallel servers doing writes... And Whole
cell doing reads. So Writes up to 4 servers, reads up to 10 servers (4 as app reads, 10 as
infrastructure reads)
>> FYI, you have to know that while a server is writing, nothing else can
>> be done (including reads). If you have clients poudning the server with
>> writes, then it's likely that the other clients will be blocked, and in
>> this case, it might have some impact on the performances.
>>
> This is not completely clear to me. You say, that if I constantly write to LDAP, I cannot
read? 
No, in fact, between too writes, if some read requests have been
received, they will be executed. It's just that while a write *is*
processed, no other operation can run, and while a read is processed, no
write can be executed (they will just be queued)



-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com 


Mime
View raw message