Handling text correctly.
WHAT A STRING IS
A sequence of characters, represented as bytes according to an encoding.
WHY ENCODING MATTERS
The same bytes mean different characters under different encodings.
WHAT TO USE
The international standard encoding, everywhere.
WHAT GOES WRONG
Text read with the wrong encoding, producing unreadable characters Assuming one character equals one byte Assuming one character equals one unit of storage
WHY THAT LAST POINT
Many characters occupy several units, and some visible characters are composed of several.
WHAT THAT AFFECTS
Counting length Truncating text Reversing text Comparing text
WHAT TO BE CAREFUL WITH
Truncating text by byte or unit count, which can split a character Case conversion, which is language-dependent Sorting, which differs by locale
WHAT TO USE FOR COMPARISON
The language's proper comparison facilities, not byte comparison, where locale matters.
WHAT TO NORMALISE
Text that may be composed differently but appear identical.
WHY
Otherwise two visually identical strings compare as different.
WHAT TO ALWAYS SPECIFY
The encoding, when reading and writing files.
