[Nuxeo-localizer] Re: [Lleu-commits] CVS: Localizer __init__.py,1.46,1.46.2.1

[Juan David Ibáñez Palomar]

Florent Guillaume wrote:

> On Mon, 2002-10-14 at 18:12, Juan David Ibáñez Palomar wrote:
>  
>
> > Florent Guillaume wrote:
> >
> >    
> >
> > > Update of /cvsroot/lleu/Localizer
> > > In directory usw-pr-cvs1:/tmp/cvs-serv11851
> > >
> > > Modified Files:
> > >     Tag: branch-1-0
> > > 	__init__.py 
> > > Log Message:
> > > Updated to take into account the fact that Zope 2.6 now is completely
> > > Unicode-aware, no need to patch it.
> > >
> > > We shouldn't even need to patch the StringIO, but I left what we had for
> > > now. If the envvar LOCALIZER_USE_ZOPE_UNICODE is defined, then no
> > > patching at all is done. I think that this should be the default, but
> > > I'm not the one to decide.
> > >      
> > And I agree, if it works.
> >
> > My only concerns is, will it work with Zope 2.6b1 or is it
> > required a CVS version? I don't want to wait Zope 2.6b2 to
> > release Localizer 0.9.2
> >    
>
> It works with b1 and b2.
>
> I'd like Zope Unicode handling can be the default, but there is a case
> where it fails which I want to fix before 2.6 final. If a ZPT page
> contains non-ASCII characters in the local charset (for instance 'é')
> and it uses Unicode strings with tal:content="python:u'foo'" or using
> i18n:translate (which calls Localizer through TranslationService) then
> Zope attempts to join 'é' and u'foo' and fails. Localizer does it
> correctly because its StringIO converts everything to the final output
> charset *before* doing the join, however it's slow.
>
>  

Ah! then it has a too weak Unicode support for my taste


> It could be argued that a page using Localizer should *never* have any
> plain 8bit string, because it would mean it isn't translated, but I'm
> not sure it's ok for everyone. For instance I don't know how
> LocalContent works, maybe it outputs 8bit strings.
>
>  

Now LocalContent objects store and return the data as Unicode
strings, but previous versions did it as normal strings. If
somebody upgrades to a new Localizer version LocalContent
objects still return normal strings (until they are edited).

Anyway, it doesn't really matters. A web page can have strings
that come from different sources, being Localizer only a part
of them. Some of these strings could be in an encoding different
than ASCII. I can't make the assumption that they will be in
ASCII, that's not acceptable.

However, the current situation is also unacceptable. Now it's
assumed that normal strings are in Latin-1, but this maybe not
the case. And for the first time (as far as I know) we have a
user reporting this problem (see the thread started by Sean
Treadway).


As conclusion we apply the patch by default, until the problem is
correctly solved in Zope.


Regards,

--
J. David Ibáñez, http://www.j-david.net
Software Engineer / Ingénieur Logiciel / Ingeniero de Software