[Zope] Zope 2.6.1 and UTF-8

[Chris Withers]

Trying again to bring it on list ;-)

Chris Withers wrote:
> (bringing on-list in case others are interested)
> 
> Toby Dickenson wrote:
> 
>>> I've got some stuff that's in strings, so I guess not unicode, but
>>> which is UTF-8 encoded, and I'm wondering how I make sure Zope does "the
>>> right thing" here. Are there any docs about? 
> 
> 
> (and just to be clear, I'm using Zope 2.6.1 with ZODB 3.1, what 
> differences will that make?)
> 
>> Ive submitted a chapter to one of the books that Chris M maintains... 
>> last I looked it still wasnt merged :(
>> There is some info at
>> http://zope.org/Members/htrd/howto/unicode 
>> http://zope.org/Members/htrd/howto/unicode-zdg-changes
> 
> 
> Just had a read of these, very interesting...
> 
>> 1. convert your strings to either unicode objects or latin-1, so that 
>> dtml or zpt can do the right thing when combining them. (Ive *still* 
>> not used zpt for this, but I assume it works).
> 
> 
> I will be using ZPT for this, what changes did you make so that ZPT's 
> return unicode strings?
> 
>> I recommend converting all language strings to unicode at the earliest 
>> opportunity as a general principal.
> 
> 
> Hmmm, that's interesting. I'd been planning on keeping everything as 
> UTF-8 encoded strings rather than actual unicode. What leads you to 
> suggest storing everything as unicode?
> 
>> 2. set a 'Content-Type' header with the value 'text/html; 
>> charset=UTF-8' (or whatever you prefer, but anything other than utf8 
>> has other complications) so that ZPublisher knows how to transmit the 
>> unicode response over http.
> 
> 
> What are these complications?
> (luckily I'm going to be using UTF-8 ;-)
> 
>> 3. If there are http forms on those pages, you need to add extra 
>> marshalling tags so that ZPublisher knows what encoding your browser 
>> used when submitting the form.
> 
> 
> If I do, do I then end up with unicode or strings encoded with the 
> character set I specify?
> 
> Finally, is ZCTextIndex compatible with either unicode or strings that 
> contain UTF-8 encoding?
> 
> cheers,
> 
> Chris
>