RE: Problems with case folding of UTF-8

["Howard Chu" <hyc@highlandsun.com>]

> -----Original Message-----
> From: owner-openldap-devel@OpenLDAP.org
> [mailto:owner-openldap-devel@OpenLDAP.org]On Behalf Of Pierangelo
> Masarati

> > On Mon, Dec 10, 2001 at 10:05:57AM +0100, Pierangelo Masarati wrote:

> > > b) breaks the current DN normalization workaround in slapd because
> > > the resulting normalized DN is longer than the input one (a six-char
> > > '\c3\89' is turned into a seven-char 'E\cc\81' when there's an
> > > equivalent six-char representation)
> >
> > Yes, there can be multiple equivalent representations, and the
> > normalized representation is often not the shortest one, so we
> > have to allow for the strong to grow, which means that you need
> > to do new allocation for the normalized string. UTF8Normalize or
> > whatever I called it, does this.
>
> Well, at present the issue is that most of the code calls dn_normalize
> expecting in-place normalization. Of course the new dnNormalize routine
> will obsolete that, but in the meanwhile we need to deal with in-place
> behavior.  I'm a bit scared about speeding up the replacement because
> it might interfere with other developers if I commit too often; on the
> other hand I need to commit very often, as you may notice, to keep the
> pace of the other developers: the last weekend I got 250 messages, most
> of which commits by Howard :)

OK, I will work less on weekends from now on.  :)

> Seriously, the "multiple equivalent representations" again scare me a bit,
> because unless our normalization routines are pretty robust, uniquely
> choosing the same representation regardless of the input, we won't end up
> with a unique string (not even structural) representation of the DNs.

> If at all possible, I'd rather prefer the short representation regardless
> of the input form.

> Pierangelo.

This makes sense to me. I wonder why we should be forced to choose a longer
representation; as long as our conversion is self-consistent (always chooses
the same representation) we should be free to choose the form we want.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support