[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [GNUe] RE: gnue-common bug: mysqldbdrv.py with MySQLdb 1.2.2 always
Re: [GNUe] RE: gnue-common bug: mysqldbdrv.py with MySQLdb 1.2.2 always decodes query result using latin1
Thu, 12 Jul 2007 21:10:46 +0300
Thank You for support.
This is how actual (not patched) gnue-common works:
comment = GNUe Test Database
provider = mysql
host = 10.17.13.40
port = 3308
dbname = gnue
username = gnue
password = gnue
ownerpwd = gnue
encoding = cp1251
So mysqldbdrv.py makes "SET NAMES cp1251".
Query statements encoded fine and sent to server with correct encoding
cp1251 because i have also set cp1251 with sys.setdefaultencoding().
Problem with decoding of query result:
Actual mysqldbdrv.py not passes charset to MySQLdb.Connection constructor.
Then MySQLdb.Connection constructor calls Connection.set_character_set
function with internal-default latin1 client charset.
Please see the code:
def set_character_set(self, charset):
"""Set the connection character set to charset. The character
set can only be changed in MySQL-4.1 and newer. If you try
to change the character set from the current value in an
older version, NotSupportedError will be raised."""
if self.character_set_name() != charset:
if self._server_version < (4, 1):
raise NotSupportedError, "server is too old to set charset"
self.query('SET NAMES %s' % charset)
self.string_decoder.charset = charset
self.unicode_literal.charset = charset
So the variable "self.string_decoder.charset" actually set to latin1. Then
"self.string_decoder" called to decode strings from query result. This strings
are "cp1251" encoded, type str, (because of "SET NAMES cp1251"). The
result is unicode strings with special latin chars instead cyrillic
As you see, this function can work with mysql >= 4.1. So we can also
allow encoding with MySQL 4.1.
Patched gnue-common with MySQLdb 1.2.2 on mysql-5.0.27: OK
Patched gnue-common with MySQLdb 1.2.1_p2 on mysql-5.0.27: OK
Patched gnue-common with MySQLdb 1.2.2 on mysql-4.1.15: OK
Patched gnue-common with MySQLdb 1.2.2 on mysql-4.0.21: ERROR
gnue-schema: (1193, "Unknown system variable 'NAMES'")
It is not a gnue assertion because i have not set version in
My testing environment:
mysql-5.0.27 and mysql-4.1.15
MySQL-python 1.2.1-p2 and MySQL-python 1.2.2
gnue-common 0.6.8 + mysqldbdrv.py patch
----- Original Message -----
From: Johannes Vetter <address@hidden>
Date: Thursday, July 12, 2007, 11:22:13 AM
Subject: [GNUe] RE: gnue-common bug: mysqldbdrv.py with MySQLdb 1.2.2
always decodes query result using latin1
JV> Hi Oleg,
JV> I'm lacking a MySQLdb 1.2.2 driver as well as a MySQL 5 Server here, so
JV> I cannot reproduce your problem or verify your patch.
JV> But the value for "self._encoding" is taken from your connections.conf.
JV> If there is no such value provided, it will default to "utf-8". This
JV> value will be used for all encoding and decoding of the unicode values
JV> in use.
JV> So if you appear to have everything encoded/decoded in latin-1 you're
JV> likely having an "encoding=latin-1 (or iso8859-1)" in your
JV> connections.conf (for that connection).
JV> Aside from that, your patch suggests setting the charset via keyword
JV> instead of performing an explicit "SET NAMES" when using a MySQL 5+. As
JV> you've mentioned this works fine with the MySQLdb 1.2.2 driver. Can you
JV> check this with a MySQLdb 1.2.1 driver ? If it works there too, we can
JV> integrate your patch.
JV> Thanks for your suggestions,
JV> Have a nice day