On 07 Dec 2015, at 12:18 , EuanM <euanmee@gmail.com> wrote:

Hannes,

The Unicode standard provide compatibility codepoints for
compatibility purposes and prefers all characters to be represented
composed form - as that way they are comparable and sortable.

(Some composed characters have *more than one* compatibility codepoint.

The canonical example is the composed character #(0041 030a) which can
be represented by EITHER the compatibility codepoint #(00c5) "Latin
Capital Letter A with Ring" above OR by #(212b) "Angstrom sign" )

On 7 December 2015 at 08:17, H. Hirzel <hannes.hirzel@gmail.com> wrote:
On 12/7/15, EuanM <euanmee@gmail.com> wrote:
My current thinking for collation sequences:

All strings being collated have had all compatibility codepoints
expanded into composed sequences.

What does the Unicode manual suggest? (www.unicode.org reference?)

While nice, it is by no means required to keep/convert the strings to Normalizion form D, as long as the operation returns the same results as if they were (http://unicode.org/reports/tr10/#Main_Algorithm):

"Step 1. Produce a normalized form of each input string, applying S1.1.

S1.1 Use the Unicode canonical algorithm to decompose characters according to the canonical mappings. That is, put the string into Normalization Form D (see [UAX15]).

Conformant implementations may skip this step in certain circumstances, as long as they get the same results. For techniques that may be useful in such an approach, seeSection 6.5, Avoiding Normalization."

So it's a tradeoff between whether you complicate other string operations that create/modify strings to maintain normalized data, or take the hit when collating.

Cheers,

Henry