Debian Bug report logs - #208816
considers UTF-16 files to be binary

Reply or subscribe to this bug.

Display info messages

Message #5 received at [email protected] (full text, mbox, reply):

From: Wichert Akkerman <[email protected]>

To: Debian Bug Tracking System <[email protected]>

Subject: considers UTF-16 files to be binary

Date: Fri, 5 Sep 2003 11:47:09 +0200

Package: subversion
Version: 0.26.0-1
Severity: normal

svn treats UTF-16 textfiles as binary.

Wichert.


-- System Information:
Debian Release: testing/unstable
Architecture: i386
Kernel: Linux tornado 2.6.0-test4 #11 Sat Aug 30 21:32:00 CEST 2003 i686
Locale: LANG=C, LC_CTYPE=en_GB.UTF-8

Versions of packages subversion depends on:
ii  db4.1-util                    4.1.25-6   Berkeley v4.1 Database Utilities
ii  libapr0                       2.0.47-1   The Apache Portable Runtime
ii  libc6                         2.3.2-4    GNU C Library: Shared libraries an
ii  libdb4.1                      4.1.25-6   Berkeley v4.1 Database Libraries [
ii  libexpat1                     1.95.6-6   XML parsing C library - runtime li
ii  libldap2                      2.1.22-1   OpenLDAP libraries
ii  libneon23                     0.23.9-2   An HTTP and WebDAV client library
ii  libssl0.9.7                   0.9.7b-2   SSL shared libraries
ii  libsvn0                       0.26.0-1   Subversion shared libraries - in d
ii  libxml2                       2.5.10-1   GNOME XML library
ii  patch                         2.5.9-1    Apply a diff file to an original
ii  zlib1g                        1:1.1.4-14 compression library - runtime

-- no debconf information


-- 
Wichert Akkerman <[email protected]>    It is simple to make things.
http://www.wiggy.net/                   It is hard to make things simple.

Message #10 received at [email protected] (full text, mbox, reply):

From: Philip Martin <[email protected]>

To: Wichert Akkerman <[email protected]>, [email protected]

Subject: Re: Bug#208816: considers UTF-16 files to be binary

Date: Wed, 29 Oct 2003 00:28:33 +0000

When Subversion updates (while local changes are present), merges or
diffs a text file it does so on a line by line basis, with the single
byte '\n' being used to identify line endings.

UTF-16 is a variable width encoding with 2-byte code units.  I'm not
sure what encodings are valid, is it possible for a 0x0a byte to
appear anywhere other than in the last byte of a UTF-16 character?  If
so, then were Subversion to treat a UTF-16 file as text it might split
such characters and so generate an invalid UTF-16 file.

If you want to treat UTF-16 files as text I suggest you configure
Subversion to use external diff/diff3 commands, and choose ones that
do the right thing on UTF-16 files.

-- 
Philip Martin

Send a report that this bug log contains spam.