[Zope-Coders] Re: [Zope-Checkins] CVS: Zope27/lib/python/TAL - TALInterpreter.py:1.68.26.2

[Florent Guillaume]

[I'm moving this to zope-coders]
[About my recent checkins adding an indirection for StringIO inside PT
and TAL]

On Mon, 2002-09-09 at 15:23, Guido van Rossum wrote:
> > Ok, here's the motivation for those (tiny) changes.
>=20
> The "(tiny)" suggests you already feel guilty.  Good! :-)
>=20
> > I'm working on internationalizing several sites, and in both cases I
> > absolutely need Unicode working in PT. Currently the framework I use is
> > Localizer (but another framework would have the same problems).
> >=20
> > Before that change, if you wanted a Unicode-aware Zope, you had to take
> > 2.5.1 and patch it, or take 2.6 and patch it, or take 2.7 and patch it.
> >=20
> > With that change, it's a simple matter for Localizer to provide a bette=
r
> > StringIO to the parts of TAL and PT that need one to function correctly=
:
> > it monkey-patches the method (dynamically replaces it in the class --
> > yes it's somewhat ugly but much better that telling people to use
> > "patch", and besides everybody's doing it :-)
>=20
> More indicators of guilt. :-)

Monkey-patching really is a funny sport, anyone will tell you that :-)

> Did you consider alternative approaches?  E.g. you could provide an
> official API (e.g. in the form of a module-global function) that
> allows one to set the StringIO implementation used.

Because I didn't want to impact any other uses of StringIO, I changed
only the two modules that were posing problems for i18n.

> And have you tried to find a solution that allows one to select the
> StringIO implementation more selectively, e.g. only for templates used
> by such-and-such a product?  I'm not entirely sure of the scope of
> your needs, so I have to be vague here; but I wonder if providing a
> different StringIO couldn't aversely affect other parts of a site.
> The global approach makes this hard to debug: you install product X,
> and later you find that page Y, which makes no use of X, breaks (or
> becomes 100x slower).  It will be very hard for the poor site
> maintainer to track the failure of page Y down to the earlier install
> of product X!

I don't think it would really be useful, as a Unicode-aware StringIO
would be a superset of the standard one. There could be speed issues
though.

> > I completely agree that this is not a panacea, and that it would be muc=
h
> > better to put the correct StringIO in Zope and be done with it, but a
> > completely corrected Unicode-aware StringIO is something I don't have
> > and which is, from what I gather, not trivial. Guido probably has more
> > info on that.
>=20
> Depends on what you want.  Since Python has no standard API for
> writing Unicode to files, this is indeed nontrivial.  I think the
> Python StringIO.py might accidentally support Unicode.

It doesn't (python 2.2):
from StringIO import StringIO
s =3D StringIO()
s.write('=E9')
s.write(u'a')
s.getvalue()
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.2/StringIO.py", line 169, in getvalue
    self.buf +=3D ''.join(self.buflist)
UnicodeError: ASCII decoding error: ordinal not in range(128)

But then it's not clear what should be done in any case. In this example
the first '=E9' is in a "native" coding and shouldn't be allowed by the
application. But because TALES can get its values from python code, it's
conceivable that we can receive native strings and have to decide what
to do with them.

Localizer's choice is to convert all Unicode strings to standard strings
in the desired output charset, and leave "native" strings alone
(supposing the application has generated them in the correct way).

2.6's choice is to allow building a complete response using Unicode
strings, and do the conversion only upon publishing to the client. But
then we have to convert a mix of non-unicode strings and unicode
strings, which can cause the problems outlined above.

> Since you don't have a completely correct StringIO implementation,
> what *do* you have?  Are you sure that's not going to break other uses
> of page templates?

Well if something break at least it's going to be Localizer's fault and
not Zope's. That was the idea.

> > In the meantime, I can use a very stupid unicode-StringIO in Localizer
> > (it converts everything to the REQUEST charset at .write() time) withou=
t
> > impacting core Zope. I can point people at Localizer and tell them to
> > install Localizer and Zope 2.6 and that's it, no messy patching
> > required.
> >=20
> > And when the time is right, we can put an adequate StringIO in Zope and
> > forget this little mess (which, I don't consider that messy).
>=20
> The word monkey-patch is the very definition of messy.
>=20
> > I'm sorry I haven't discussed these changes before, I should have. This
> > is not an architectural change at all, simply a temporary hook for othe=
r
> > code, I didn't think it would pose problems.
> >=20
> > What do you think about it now. Should I revert them?
>=20
> Ask the Zope Pope.

Jim, do you want those reverted?

Again, for the record, my argument to leave those in is: they don't
harm, they'll be removed later, and in the meantime third-party products
can still function.


Florent



--=20
Florent Guillaume, Nuxeo (Paris, France)
+33 1 40 33 79 87  http://nuxeo.com  mailto:fg@nuxeo.com