[Zope] automagic bome header at start of utf16 content?

Jürgen Herrmann Juergen.Herrmann at XLhost.de
Thu Jan 8 04:33:16 EST 2009


i'm debugging a problem with our licence generation application.
it spits out a ini file for our actual licence generator program
(windows exe). to maintain backwards compatibility with an older
version i try to encode the content windows-1252 if possibly,
utf16 otherwise.

on top of the method id do somthing like this:
REQUEST.RESPONSE.setHeader('Content-Type', 'x-limiall-ini')
REQUEST.RESPONSE.setHeader('Content-Disposition', 'attachment; filename=bla.blub')

lines is a list of unicode strings. then i do:
  return '\r\n'.join(lines).encode('windows-1252')
except UnicodeEncodeError:
  return '\r\n'.join(lines).encode('utf16')

if there are unicode chars in the strings, that cannot be encoded
in windows-1252, i get a file with a correct BOM header at the start
('\xFF\xFE'), which is exactly what we need. but i wonder where it
comes from. i already sent the request directly to the zope server
omitting our apache proxy and monitored traffic with wireshark. the
com header comes from zope. i did not find anything in zope's code
that heuristically finds out this is utf16 content and prepends the
BOM header. so i'm a bit confused where zope takes it's wisdom from :)

best regards,
jürgen herrmann

More information about the Zope mailing list