[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: Openldap2.4.16 performance issue
FWIW, our entire ldap infrastructure is on VMs - with one exception: on an overloaded VM cluster, we had tons of random auth failures - put in a pizza box machine to replace one of the VMs (and changed load balancer config to use the VM ldap box in that location only as a fail over backup).
Our mirrored masters are VMs as well.
It's all about the load - there's nothing inherently bad about OpenLDAP on VMs - many people simply overload them.
- chris
Chris Jacobs, Systems Administrator
Apollo Group | Apollo Marketing | Aptimus
2001 6th Ave Ste 3200 | Seattle, WA 98121
phone: 206.839-8245 | cell: 206.601.3256 | Fax: 208.441.9661
email: chris.jacobs@apollogrp.edu
----- Original Message -----
From: openldap-technical-bounces@OpenLDAP.org <openldap-technical-bounces@OpenLDAP.org>
To: Dieter Kluenter <dieter@dkluenter.de>
Cc: openldap-technical@openldap.org <openldap-technical@openldap.org>
Sent: Mon Aug 23 11:13:43 2010
Subject: Re: Openldap2.4.16 performance issue
----- Original Message -----
> From: "Dieter Kluenter" <dieter@dkluenter.de>
> To: openldap-technical@openldap.org
> Sent: Saturday, August 21, 2010 3:07:04 AM
> Subject: Re: Openldap2.4.16 performance issue
> "Singh, Devender (GE Capital, consultant)" <Devender.Singh2@ge.com>
> writes:
>
> > Hi Dieter,
> >
> > Please find the below details:
> >
> > 1. hardware related
> >
> > - type of storage - Simply SATA had disk attached.
> >
> > - raid level, if any- No RAID
> >
> > - file system of disk(s)- ext3 on LVm
> >
> > - type of network, 100MB, 1G, 10G
> >
> >
> >
> > 2. is this host running on a virtual machine or on bare metal.
> >
> > - if virtual machine, -Yes, OS installed on VM
> >
> > -- what type ---Don’t know
>
> There is nothing suspicious, except for the virtual machine. Your
> really should carefully check layout and configuration of this VM, do
> not use virtual disks.
>
Hi Dieter,
Could you kindly explain what this means? I've been all over the inter-webs and I'm not finding anything concrete about OpenLDAP and VM's or Databases and VM's in general. The closest I came was about some database latency studies and some VMware propaganda.
We are about to launch a master and two replicas (utilizing delta-syncrepl) running in Ubuntu 10.04 on a VMware VSphere ESXi 4.0 cluster with four IBM x3650-M2's (2 Quad Nahalem and 64GB memory each) with virtual disks carved from NFS mounts to all of the VSphere servers in the cluster to facilitate HA and FT - by the VMware book.
(BTW, I took one of Howard's old posts to heart and we are following the Ubuntu playbook and we have purchased Canonical 24x7 support/maintenance. :-) )
We have had no problems with this environment in development and get better results than on bare metal with or without RAID. I know that it is recommended that the logs live on another "disk" from the database and RAID is frowned upon, but I have difficulty with a few points:
1) Separate, unprotected disks seems illogical. The last log and the BDB files are necessary to start BDB in slapd, correct? So, if you lose either disk, you're in trouble. Backups are ok, but daily seems too long a time. Seeing as we process new user accounts every 15 minutes, this would not be ideal for us.
2) RAID-1 I can understand having an issue on writes. But what about LUNs in FC from a NetApp 3140? Virtual disks on NFS in VMware? Both of these are in the best practices of both VMware and NetApp documentation.
3) 13-disk 7200RPM SATA RAID-DP (NetApp) is far faster than a single disk, dual diak or RAID-1, so why wouldn't you use SAN/NAS storage?
I seriously want to understand the VM concern as it pertains to OpenLDAP. I think more and more people are doing this very thing and will benefit from this discussion.
Our database is 86K+ DN's averaging about 40 attributes each. We've tuned the HDB cache to 768MB in a shared memory segment and the pertinent master slapd.conf file shows:
shm_key 100
cachesize 200000
idlcachesize 600000
dncachesize 400000
checkpoint 1024 15
.
.
.
# main database
.
.
.
index objectClass eq
index cn eq,sub
index sn eq,sub
index gn eq,sub
index mail eq,sub
index uid eq,sub
index displayname sub,eq
index memberUid eq,sub
index uidNumber eq
index gidNumber eq
index sambaSID eq
index sambaSIDlist eq
index sambaDomainName eq
index sambaPrimaryGroupSID eq
index sambaGroupType eq
index entryCSN eq
index entryUUID eq
index default sub,eq
Replicas have identical indexes and shared memory usage. Basically, just running database population tests with full checking turned on, I get the following results:
Ubuntu 10.04.1 on all with OpenLDAP 2.4.21/BDB 4.7.25 (all generate 200-10MB log files):
IBM x3550 2-quad 5450 16GB, RAID-1 15K 73GB 3.0Gb/s SAS, 86K DN's - 30 minutes
IBM x3550 2-quad 5450 16GB, Single 15K 73GB 3.0Gb/s SAS, 86K DN's - 28 minutes
IBM x3550 2-quad 5450 16GB, 2-single 15K 73GB (db and logs on separate disks) 3.0Gb/s SAS, 86K DN's - 28 minutes
VMware guest 2-vCPU, 3GB memory, 100GB virtual disk on VMware NFS mount of 13-1TB 7200RPM SATA disks in NetApp 3140 - 4 minutes.
We can replicate to another VM in 9 minutes and two VMs simultaneously to the same 13-disk aggregate in 13 minutes. Aside from VM clock skew problems, I don't see the benefit of Bare Metal and I'm feeling pretty dumb at the moment.
Any insight from you, Quanah and/or Howard is humbly accepted and appreciated - I am here to learn. :-)
Thank you,
Bill
> -Dieter
>
> -- Dieter Klünter | Systemberatung
> sip: 7770535@sipgate.de
> http://www.dpunkt.de/buecher/2104.html GPG Key ID:8EF7B6C6
This message is private and confidential. If you have received it in error, please notify the sender and remove it from your system.