[Zope-dev] Zope 2.6.0 ZMI Problem for CJK(Collector 623) patch.

Heiichiro NAKAMURA nheiich@quantumfusion.com
Sun, 08 Dec 2002 18:20:59 -0800


On Sun, 8 Dec 2002 21:58:16 +0000
Toby Dickenson <tdickenson@geminidataloggers.com> wrote:

> On Thursday 05 December 2002 9:36 pm, Toby Dickenson wrote:
> > Yes, I have an idea. I hope to find time to flesh it out early next week.
> 
> I propose:



Here is my understanding of your proposal:



1. "management_page_charset" property for non-Latin-1
------------------------------------------------------------------------
    Affected application: ZMI only
    Limitation:
       - Standard Objects in Zope (such as "title" property) must be
         Non-Unicode.
       - All objects must be Non-Unicode in order to work in ZMI.
       - If there is even one Unicode object there, all data will be
         assumed as 'Latin-1' and non-Latin-1 data cannot be used in
         any object.
       - REQUEST.set('management_page_charset','UTF-8') does not work
         when 'management_page_charset' is set as global property.
    Homework (new issues):
       - define the safe steps of implementation of Unicode Support
       - find a way of smooth migration of MBCS-to-Unicode
------------------------------------------------------------------------


2. Behaviour of handling of text in ZMI
------------------------------------------------------------------------
When UnicodeType Properties are contained:
     - encoding of input data: 'Latin-1'
     - internal representation: 'UCS-2/UTF-16' if UnicodeType(ex. ustring)
                                'Latin-1' if not UnicodeType(ex. string)
     - encoding of output data: 'Latin-1'

When UnicodeType Properties are NOT contained:
     - encoding of input data: any
     - internal representation: same as input (no conversion)
     - encoding of output data: same as internal (no conversion)

    Limitation:
     - Non-Latin-1 value cannot be used when creating the first Unicode
       Property.
------------------------------------------------------------------------

(Any correction is welcomed)

I'll ask power users of Japan-Zope-User-Group-ML about opinions
regarding to the proposal.



> but I 
> suggest that compatability with this feature should not hold back any
> future enhancements to the ZMI which rely on using unicode)

I guess this statement is somewhat ambiguous. Probably we could say
something like that? :
------------------------------------------------------------------------
  1. Any future enhancements to the ZMI which rely on using unicode
  should be carefully defined, examined, evaluated and feasibility-tested
  so that all issues/limitations can be clarified and the consequent
  impact of these issues/limitation can be evaluated in order to
  avoid hassle implementation integrated into the official version.

  2. Any experimental enhancements to the ZMI which rely on using unicode
  must be integrated into the official version in the manner that
  it doesn't affect the behaviour of 'legacy' applications, until
  the Unicode Support gets matured enough to handle Unicode object
  with all languages/encodings.
------------------------------------------------------------------------




---
Heiichiro NAKAMURA <nheiich@quantumfusion.com>





On Sun, 8 Dec 2002 21:58:16 +0000
Toby Dickenson <tdickenson@geminidataloggers.com> wrote:

> On Thursday 05 December 2002 9:36 pm, Toby Dickenson wrote:
> > On Thursday 05 December 2002 8:41 pm, Heiichiro NAKAMURA wrote:
> > > Does anyone have any other idea for the Collector 623 issue?
> > > I hope better ideas will be posted..
> >
> > Yes, I have an idea. I hope to find time to flesh it out early next week.
> 
> I propose:
> 
> 1. Currently you can set a management_page_charset property to override the 
> global assumption that legacy ZMI pages are latin1. This is a hack that 
> happens to mostly work by accident, provided your ZMI pages do not encounter 
> a unicode object. I propose promoting this to a standard documented feature.
> 
> Is there anyone who actually uses this feature who can contribute some 
> documentation?
> 
> (Note that feature will never work with pages that do encounter unicode 
> objects. This means Zope should try to avoid gratuitous unicode usage, but I 
> suggest that compatability with this feature should not hold back any future 
> enhancements to the ZMI which rely on using unicode)
> 
> 
> 2. The documented way of setting a ZMI-page-specific encoding is 
> REQUEST.set('management_page_charset','UTF-8'). Currently 
> management_page_charset as a global property will override this page specific 
> setting. This is a bug that needs to be fixed. 
> 
> 
> 3. I propose changing the encoding used by the standard 'Properties' ZMI page. 
> Currently it uses UTF8 always, and assumes that 8bit string properties are 
> latin1. I propose that this policy is used only if a unicode property has 
> already been defined on this object. If it has not, it uses the same encoding 
> policy as every other ZMI page - that is the value of the 
> management_page_charset property, or latin-1 if it has not been set.
> 
> A disadvantage of this change is that it will not be possible to set a 
> non-latin-1 initial value when creating the first unicode property. I think 
> this is just about acceptable.