[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: openldap server load serving sendmail access map
I now remember sendmail does go through several iterations for various
forms of entries possible in varius maps. One being the access map...
it looks up on fqdn hostname, donain name, full ip, 3 octets, 2 octests,
and finally one octet... (as I recall)... which means it is going to make
at least 5 queries for that one map alone (too bad there wasn't a way
to make use of an optimized single LDAP query for each form of what
it is looking for).
I think we could eliminate some of the checks for forms we don't use,
that probably would have a positive impact on the load.
For optimization reference, I have the following should anyone see anything
I missed or notice something that could be done better...
I have the following "optimizations" in my slapd.conf:
index objectClass eq
index cn eq
index sendmailMTAHost eq
index sendmailMTACluster eq
index sendmailMTAMapName eq
index sendmailMTAKey eq
index sendmailMTAMapValue eq
# performance
dbcachesize 4194304
cachesize 100000
Here are the ldap database files to show their respective sizes.
There are just under 5000 entries in the access map.
-rw------- 1 root other 8192 Oct 18 15:36 cn.dbb
-rw------- 1 root other 1523712 Oct 18 16:31 dn2id.dbb
-rw------- 1 root other 3301376 Oct 18 16:31 id2entry.dbb
-rw------- 1 root other 8192 Oct 18 16:31 nextid.dbb
-rw------- 1 root other 106496 Oct 18 16:31 objectClass.dbb
-rw------- 1 root other 40960 Oct 18 15:36 sendmailMTACluster.dbb
-rw------- 1 root other 8192 Oct 18 15:39 sendmailMTAHost.dbb
-rw------- 1 root other 290816 Oct 18 15:36 sendmailMTAKey.dbb
-rw------- 1 root other 40960 Oct 18 15:36 sendmailMTAMapName.dbb
-rw------- 1 root other 270336 Oct 18 15:36 sendmailMTAMapValue.dbb
Thanks,
-- Curt
>Date: Tue, 22 Oct 2002 21:59:58 +0200 (CEST)
>Subject: Re: openldap server load serving sendmail access map
>From: "Pierangelo Masarati" <ando@sys-net.it>
>To: <openldap-software@OpenLDAP.org>
>X-Priority: 3
>Importance: Normal
>X-MSMail-Priority: Normal
>Cc: <igor@ipass.net>, <curtis@vesn.com>
>MIME-Version: 1.0
>Content-Transfer-Encoding: 8bit
>X-Spam-Filtered-By: Spamfilter version 1.1.0 {$Revision: 1.41 $}
>
>
>>
>> Curtis,
>>
>> This is proly an OT post, so you might get a better response on
>> news:comp.mail.sendmail or other sendmail related mailing lists.
>
>It is not that OT, unless all the optimization
>of the LDAP server has already been put in place.
>
>>
>> Having said that, I am also interested in this subject so hopefully Kurt
>> will not cut our discuission short. ;) In my limited research, I found
>> that the "access" query is not an issue, but rather an amount of queries
>> needed to process one email delivery. If I remember correctly, sendmail
>> will run up to 50 'access' queries for each message going thru sendmail.
>> So depending on your email volume, your LDAP server will need to be able
>> to run hunderds if not thousdands of queries per second. I will work
>> toward reducing the amount of LDAP queries needed per email transaction.
>
>It is correct that sendmail will make a very poor use of the LDAP
>map since many queries are repeated for generality without any
>caching (the same applies to samba, at least to my knowledge).
>However there's something you can do; first be sure you add the
>appropriate indices to the attributes you're using in the filter
>(the OR'ed ones). Then, use an appropriate cache. You may find
>more info in the Admin Guide, in the FAQ and in the archives
>of the mailing list.
>
>Pierangelo.
>
>>
>> -Igor
>>
>> On Tue, 22 Oct 2002, curtis wrote:
>>
>>>
>>> We are looking at migrating to LDAP as part of the architecture of our
>>> hosted e-mail product.
>>>
>>> As a test we started by making the access map served by LDAP. The
>>> sendmail gateway is running on an Ultra 2 w/300MHZ proc with sendmail
>>> 8.12.5 compiled with openldap. The LDAP server is a Ultra 2 w/dual
>>> 300MHZ procs and also runs PostgreSQL (with minimal load).
>>>
>>> Even after "optimizing" the access map query for simplicity (which
>>> lowered the load), slapd is consuming roughly 50% of the CPU. The
>>> query has been simplified to:
>>>
>>> Kaccess ldap -T<TMPF> -1 -v sendmailMTAMapValue -k
>>> (&(sendmailMTAMapName=access)(sendmailMTAKey=%0))
>>>
>>> The prior query which consumed even more CPU (but not substantially
>>> more) was:
>>>
>>> Kaccess ldap -T<TMPF> -1 -v sendmailMTAMapValue -k
>>>
(&(objectClass=sendmailMTAMapObject)(|(sendmailMTACluster=${sendmailMTACluster})
>>> (sendmailMTAHost=$j))(sendmailMTAMapName=access)(sendmailMTAKey=%0))
>>>
>>> The load on the sendmail server averages around 100 processes. The
>>> CPU load on the LDAP server is more than that of the sendmail server.
>>> While I would expect more overhead for the LDAP solution based on the
>>> flexibility and all, it seems disproportionately high.
>>>
>>> The load seemed slightly better when I bound slapd to one processor,
>>> although that could have been a slight difference of load between the
>>> unbound test and the bound test.
>>>
>>> Any input would be appreciated as an LDAP architecture would solve
>>> some headaches we have as well as lend itself to an overall better
>>> architecture.
>>>
>>> Thanks in advance,
>>>
>>> -- Curt
>>>
>>>
>>> Curtis Wilbar
>>> Virtual E-Services Network, Inc.
>>> curtis@vesn.com
>>>
>>>
>>>
>>
>> --
>> Igor
>
>
>--
>Pierangelo Masarati
>mailto:pierangelo.masarati@sys-net.it
>
>
Curtis Wilbar
President & CEO
Virtual E-Services Network, Inc.
curtis@vesn.com