[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: performance loss on real time linux



Dieter Kluenter wrote:
Due to a real time scheduler I was expecting the results to be vice
versa.
  Can a RT scheduler or other modifications to the operating system
have
a negative influence on performance of OpenLDAP?

You can't expect results to be any better than under non-RT os, since
a RT-scheduler will always add overhead to non-RT processes since it
will preempt much more than a normal OS.  I have no experience with
Monta-Vista, however my institution (Politecnico di Milano) develops
RTAI, the Real-Time Application Interface for Linux
(http://www.rtai.org/); I'll check what happens when running OpenLDAP
with that.

In the meanwhile, can you tell something more about Monta Vista's
handling of RT? Does it include the PREEMPT_RT patch, which will
likely become part of Linux in a year or so, or does it rather
implement some custom support to handle preemption?

I don't know much about Montavista yet, as they are more than restrictive about their information policy. But they claim to have introduced a preemtive kernel in 1999 and their preemtive patch is now a standard kernel option for 2.6 kernels. http://www.mvista.com/products/realtime.html

About your test: are your results an average over a realistic number
of trials? Is the database entirely in cache or does it require any
access to disk? Mallocs under a real-time scheduler might also
represent a significant bottleneck; in this case, slapd should really
handle memory internally, to minimize the amount of system calls.

My results are actually the last result of about 20 test runs, but after a warm up time of 5 runs, they didn't vary much. Both machines have 2GB ram and this are the settings for DB_CONFIG

set_lg_dir /opt/openldap/ldapDB set_cachesize 0 20000000 0 set_lg_regionmax 262144 set_lg_bsize 2097152

I didn't notice hardly any disk access during the initial tests.
Dieter,

I don't know how much this could be of any help, but today I tried to run a simple test using OpenLDAP 2.3.28 running on Linux 2.6.17 with RTAI, compared to the stock 2.3.6.15 kernel f the distribution used on that PC. I created a database with ~10000 users, fully indexed and fully cacheable (cachesize 10000). What I found (strange enough) is that the execution time of ldapsearch would differ a lot on the two systems, but sort of the reverse of what you experienced. I checked few cases:

- entry not yet in cache vs. entry already in cache
- system unloaded vs. system heavily loaded

in the case of RTAI, I also considered some hard real-time load, which gets the highest priority and accurate scheduling.

- searching for an entry not in cache, w/ RTAI, requires a little bit longer than w/o RTAI; this requires access to disk; but
- searching for the same entry, after it's in cache, requires ~0.008s w/ RTAI, and ~0.080s w/o RTAI (ten times longer! It's a slightly different kernel, but running on exactly the same machine, so, without too much investigation, I have no clue about the exact reason)


- under load (repeated disk and network access), the above entry in cache times are occasionally delayed, but not that much, on both systems. w/ RTAI the average goes to ~0.010s, with peaks to ~0.020s; w/o RTAI no significant extra delay is added to the ~0.080s. The latter case makes sense, since the system already appears significantly delayed.

- under hard RT load (30% of one CPU, and 15% of the other CPU on a dual CPU system), performances drop significantly; occasionally, the response time for not-in-cache entry, which requires access to disk, may raise to ~0.500s, with an average above ~0.100s. The response time when the entry is in cache is not significantly worse than in the case with non-RT load, ~0.020s.

This seems to indicate that with a (possibly) different hard RT scheduling there is no significant impact on basic OpenLDAP's slapd performances, so its design seems to have no impact on its execution in conjunction with RT schedulers. I have no clue, right now, about how the two RT systems compare.

I would like to acknowledge the support of Paolo Mantegazza, the main developer of RTAI, in performing those tests with a devel snapshot (called "magma") of that system; it should not differ from recent releases (3.4) in terms of scheduling policies and related issues.

Cheers, p.



Ing. Pierangelo Masarati
OpenLDAP Core Team

SysNet s.n.c.
Via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
------------------------------------------
Office:   +39.02.23998309
Mobile:   +39.333.4963172
Email:    pierangelo.masarati@sys-net.it
------------------------------------------