[Zope-dev] Catalog and Unicode

[Florent Guillaume]

We're seeing problems in one application here due to the catalog and  
interactions with Unicode. Here's what happens:

- an object is indexed with a Unicode title, so in the catalog the  
metadata tuple has for instance (u'cafe',)
- later that title is changed to latin-1, so the new metadata tuple  
would be ('caf\xe9',)

The problem is that Catalog.py has in updateMetadata() the code:

             if data.get(index, 0) != newDataRecord:
                 data[index] = newDataRecord

The simple comparison in the first line provokes a  
UnicodeDecodeError, you can reproduce by a simple:

     python -c "u'e' == '\xe9'"

This understandable, but in the case of the catalog really not  
helpful. I propose to change the code above to:

             try:
                 changed = data.get(index, 0) != newDataRecord
             except UnicodeDecodeError:
                 changed = True
            if changed:
                 data[index] = newDataRecord

Objections ?

Florent

-- 
Florent Guillaume, Nuxeo (Paris, France)   CTO, Director of R&D
+33 1 40 33 71 59   http://nuxeo.com   fg at nuxeo.com