Subject: Re: Bug#208816: considers UTF-16 files to be binary

From: Philip Martin <[email-masked]>
Date: Wed, 29 Oct 2003 00:28:33 +0000

When Subversion updates (while local changes are present), merges or
diffs a text file it does so on a line by line basis, with the single
byte '\n' being used to identify line endings.

UTF-16 is a variable width encoding with 2-byte code units.  I'm not
sure what encodings are valid, is it possible for a 0x0a byte to
appear anywhere other than in the last byte of a UTF-16 character?  If
so, then were Subversion to treat a UTF-16 file as text it might split
such characters and so generate an invalid UTF-16 file.

If you want to treat UTF-16 files as text I suggest you configure
Subversion to use external diff/diff3 commands, and choose ones that
do the right thing on UTF-16 files.

-- 
Philip Martin