[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: openldap server load serving sendmail access map
For reference, I did some debugging to see what sendmail does with the
access map...
on connect:
ldapmap_open(access, 0): opening new connection
openmap() ldap:access NULL: valid
ldapmap_lookup(access, Connect:mail.nosuchdomain.com)
ldapmap_lookup(access, mail.nosuchdomain.com)
ldapmap_lookup(access, Connect:nosuchdomain.com)
ldapmap_lookup(access, nosuchdomain.com)
ldapmap_lookup(access, Connect:com)
ldapmap_lookup(access, com)
ldapmap_lookup(access, Connect:127.0.0.1)
ldapmap_lookup(access, 127.0.0.1)
ldapmap_lookup(access, Connect:127.0.0)
ldapmap_lookup(access, 127.0.0)
ldapmap_lookup(access, Connect:127.0)
ldapmap_lookup(access, 127.0)
ldapmap_lookup(access, Connect:127)
ldapmap_lookup(access, 127)
on mail from:
ldapmap_lookup(access, TLS_Clt:)
ldapmap_lookup(access, TLS_Clt:127.0.0.1)
ldapmap_lookup(access, TLS_Clt:127.0.0)
ldapmap_lookup(access, TLS_Clt:127.0)
ldapmap_lookup(access, TLS_Clt:127)
ldapmap_lookup(access, TLS_Clt:)
ldapmap_lookup(access, From:theloser@spammer.com)
ldapmap_lookup(access, theloser@spammer.com)
ldapmap_lookup(access, From:theloser@)
ldapmap_lookup(access, theloser@)
ldapmap_lookup(access, From:spammer.com)
ldapmap_lookup(access, spammer.com)
ldapmap_lookup(access, From:com)
ldapmap_lookup(access, com)
on rcpt to:
ldapmap_lookup(access, To:losers.spammer.org)
ldapmap_lookup(access, losers.spammer.org)
ldapmap_lookup(access, To:spammer.org)
ldapmap_lookup(access, spammer.org)
ldapmap_lookup(access, To:org)
ldapmap_lookup(access, org)
ldapmap_lookup(access, Connect:127.0.0.1)
ldapmap_lookup(access, 127.0.0.1)
ldapmap_lookup(access, Connect:127.0.0)
ldapmap_lookup(access, 127.0.0)
ldapmap_lookup(access, Connect:127.0)
ldapmap_lookup(access, 127.0)
ldapmap_lookup(access, Connect:127)
ldapmap_lookup(access, 127)
This isn't pretty...
Of course I tested the worst case scenario (no matching found).
I think I'll try heavily modifying the sendmail.cf file to test only
those rules/matches we want to test, and not all the rest for features/syntax
we don't use. I'm willing to bet it will make a significant difference.
Given the way sendmail does all these possible lookups (and this is just
for the access map, we haven't gotten into the other maps), I'd say that
it is more efficient to use a local Berkeley DB for the access map, but
the LDAP solution is nicer for centralization and flexibility.
Thanks for the tips... I should have looked more to how sendmail was
using this map sooner. I have to wonder now what the overhead of all these
lookups are in just using a local Berkeley DB...... less for sure, but still
seems like the flexibility of the map is a bit overkill and could be better
served by having two or three maps take it's place using very specific
lookups for each of those two to three maps.
Thanks,
-- Curt
>Date: Tue, 22 Oct 2002 21:59:58 +0200 (CEST)
>Subject: Re: openldap server load serving sendmail access map
>From: "Pierangelo Masarati" <ando@sys-net.it>
>To: <openldap-software@OpenLDAP.org>
>Cc: <igor@ipass.net>, <curtis@vesn.com>
>
>
>>
>> Curtis,
>>
>> This is proly an OT post, so you might get a better response on
>> news:comp.mail.sendmail or other sendmail related mailing lists.
>
>It is not that OT, unless all the optimization
>of the LDAP server has already been put in place.
>
>>
>> Having said that, I am also interested in this subject so hopefully Kurt
>> will not cut our discuission short. ;) In my limited research, I found
>> that the "access" query is not an issue, but rather an amount of queries
>> needed to process one email delivery. If I remember correctly, sendmail
>> will run up to 50 'access' queries for each message going thru sendmail.
>> So depending on your email volume, your LDAP server will need to be able
>> to run hunderds if not thousdands of queries per second. I will work
>> toward reducing the amount of LDAP queries needed per email transaction.
>
>It is correct that sendmail will make a very poor use of the LDAP
>map since many queries are repeated for generality without any
>caching (the same applies to samba, at least to my knowledge).
>However there's something you can do; first be sure you add the
>appropriate indices to the attributes you're using in the filter
>(the OR'ed ones). Then, use an appropriate cache. You may find
>more info in the Admin Guide, in the FAQ and in the archives
>of the mailing list.
>
>Pierangelo.
>
>>
>> -Igor
>>
>> On Tue, 22 Oct 2002, curtis wrote:
>>
>>>
>>> We are looking at migrating to LDAP as part of the architecture of our
>>> hosted e-mail product.
>>>
>>> As a test we started by making the access map served by LDAP. The
>>> sendmail gateway is running on an Ultra 2 w/300MHZ proc with sendmail
>>> 8.12.5 compiled with openldap. The LDAP server is a Ultra 2 w/dual
>>> 300MHZ procs and also runs PostgreSQL (with minimal load).
>>>
>>> Even after "optimizing" the access map query for simplicity (which
>>> lowered the load), slapd is consuming roughly 50% of the CPU. The
>>> query has been simplified to:
>>>
>>> Kaccess ldap -T<TMPF> -1 -v sendmailMTAMapValue -k
>>> (&(sendmailMTAMapName=access)(sendmailMTAKey=%0))
>>>
>>> The prior query which consumed even more CPU (but not substantially
>>> more) was:
>>>
>>> Kaccess ldap -T<TMPF> -1 -v sendmailMTAMapValue -k
>>>
(&(objectClass=sendmailMTAMapObject)(|(sendmailMTACluster=${sendmailMTACluster})
>>> (sendmailMTAHost=$j))(sendmailMTAMapName=access)(sendmailMTAKey=%0))
>>>
>>> The load on the sendmail server averages around 100 processes. The
>>> CPU load on the LDAP server is more than that of the sendmail server.
>>> While I would expect more overhead for the LDAP solution based on the
>>> flexibility and all, it seems disproportionately high.
>>>
>>> The load seemed slightly better when I bound slapd to one processor,
>>> although that could have been a slight difference of load between the
>>> unbound test and the bound test.
>>>
>>> Any input would be appreciated as an LDAP architecture would solve
>>> some headaches we have as well as lend itself to an overall better
>>> architecture.
>>>
>>> Thanks in advance,
>>>
>>> -- Curt
>>>
>>>
>>> Curtis Wilbar
>>> Virtual E-Services Network, Inc.
>>> curtis@vesn.com
>>>
>>>
>>>
>>
>> --
>> Igor
>
>
>--
>Pierangelo Masarati
>mailto:pierangelo.masarati@sys-net.it
>
>
Curtis Wilbar
President & CEO
Virtual E-Services Network, Inc.
curtis@vesn.com