Re: [Gnue-dev] Fw: [ mysql-python-Bugs-536260 ] Problem with fetchmany

[Harald Meyer]

Well the first guy's me. This was a problem I had when using python 2.2
with mysqldb.
I guess the first answer to this describes the problem, as we afair use
an own cursor class. He then argues that, if one uses an own cursor
class, one bust prehend parameterless calls of fetchmany.
At least one other person reported this (see
http://sourceforge.net/tracker/index.php?func=detail&aid=497492&group_i
d=22307&atid=374932 ) and they fixed it.
The main difference seems to be that they replaced
end = self.rownumber + size or self.arraysize
with
end = self.rownumber + (size or self.arraysize)

This shold work, as now nothing like self.rownumber + None can happen.

I can't verify this as I am not using python 2.2 at the moment. If
anybody has it:
The line you have to change is 234 in cursors.py of mysqldb. Or get
mysqldb from the cvs

Harald Meyer
ps: I am wondering why no crash happens under python 2.1

----- Original Message -----
From: "Derek Neighbors" <address@hidden>
To: "Harald Meyer" <address@hidden>
Cc: <address@hidden>
Sent: Thursday, May 02, 2002 8:53 AM
Subject: Re: [Gnue-dev] Fw: [ mysql-python-Bugs-536260 ] Problem with
fetchmany


> Im not sure I follow.  The first reply seems to say we are wrong we
need to use =None (which is what we had and it gave us type errors)
Then the next guy says you cant do None as it will give you type errors
(which is what we experienced and why we changed it).  He then goes to
say the bug lies with them and submitted a patch.
>
> Have you applied this patch and verified if you problem went away?
If so we know its a bug in their stuff.  Let me know if I am misreading
the below. (its kind of rough to follow)
>
> -Derek
>
> On Sat, 27 Apr 2002 08:54:49 +0200
> address@hidden (Harald Meyer) wrote:
>
> > This might be of interest. I reported that problem to the mysqldb
guys
> > and it seems that they are arguing whether this is a mysqldb fault
or
> > a gnue fault, because we call fetchmany without doing this check.
> >
> > Harald Meyer
> >
> > ----- Original Message -----
> > From: <address@hidden>
> > To: <address@hidden>
> > Sent: Saturday, April 27, 2002 1:44 AM
> > Subject: [ mysql-python-Bugs-536260 ] Problem with fetchmany
> >
> >
> > > Bugs item #536260, was opened at 2002-03-28 09:06
> > > You can respond by visiting:
> > >
> >
http://sourceforge.net/tracker/?func=detail&atid=374932&aid=536260&grou
> > p_id=22307
> > >
> > > Category: MySQLdb
> > > Group: None
> > > Status: Open
> > > Resolution: None
> > > Priority: 5
> > > Submitted By: Nobody/Anonymous (nobody)
> > > Assigned to: Andy Dustman (adustman)
> > > Summary: Problem with fetchmany
> > >
> > > Initial Comment:
> > > If fetchmany in cursors.py is called with no
> > > parameters, mysqldb crashes. The Problem seems to be
> > > the definition of fetchmany:
> > >
> > > def fetchmany(self, size=None)
> > >
> > > The size=None gives it a wrong type. If it is replaced
> > > with size=0 no problem occour.
> > >
> >
> -------------------------------------------------------------------
> > > ---
> > >
> > > >Comment By: Andy Dustman (adustman)
> > > Date: 2002-04-26 18:44
> > >
> > > Message:
> > > Logged In: YES
> > > user_id=71372
> > >
> > > No, I don't think so. _mysql_ResultObject_fetch_row() cannot
> > > be passed a None because of this:
> > >
> > > if (!PyArg_ParseTupleAndKeywords(args, kwargs,
> > > "|ii:fetch_row", kwlist,
> > > &maxrows, &how))
> > >
> > > The format code only allows for an integer. If maxrows is
> > > non-zero, the section of code you quote cannot be reached;
> > > instead:
> > >
> > > if (maxrows) {
> > > if (!(r = PyTuple_New(maxrows))) goto error;
> > > rowsadded = _mysql__fetch_row(self, &r, skiprows, maxrows,
> > > convert_row);
> > > if (rowsadded == -1) goto error;
> > > } else { ...
> > >
> > > In fact, the section of code you quote is only reachable if
> > > a non-standard cursor class is used (one with
> > > CursorUseMixIn). I am starting to think there *is* a problem
> > > there, however, but not because of the condition. Basically
> > > that condition is saying, if we didn't get all the rows we
> > > requested, we're done. Otherwise, there may be more rows to
> > > fetch. But what I think I need after that statement is
> > > something like this:
> > >
> > > #if PY_VERSION_HEX < 0x02020000
> > > if (_PyTuple_Resize(r, skiprows+maxrows, 0) == -1) goto error;
> > > #else
> > > if (_PyTuple_Resize(r, skiprows+maxrows) == -1) goto error;
> > > #endif
> > >
> > > because more space needs to be allocated prior to calling
> > > _mysql__fetch_row() again. I'd appreciate it if you tested
> > > this fix (add it immediately after the if (rowsadded <
> > > maxrows) break; line). Even so, this seems to be a pretty
> > > hard thing to trigger. You have to use a SSCursor, call
> > > cursor.fetchmany(0) or cursor.fetchall() (they both
> > > ultimately call _mysql_ResultObject_fetch_row), and then
> > > have at least 1000 rows in the result set.
> > >
> >
> -------------------------------------------------------------------
> > > ---
> > >
> > > Comment By: Danny Yoo (dyoo)
> > > Date: 2002-04-26 18:29
> > >
> > > Message:
> > > Logged In: YES
> > > user_id=49843
> > >
> > > Fixed, I think.  See patch:
> > >
> > >
> >
http://sourceforge.net/tracker/index.php?func=detail&aid=549301&group_i
> > d=22307&$
> > >
> > >
> > > Hope this helps!
> > >
> >
> -------------------------------------------------------------------
> > > ---
> > >
> > > Comment By: Danny Yoo (dyoo)
> > > Date: 2002-04-26 17:35
> > >
> > > Message:
> > > Logged In: YES
> > > user_id=49843
> > >
> > > See bug 549279: I think I figured out what's causing the
> > > segfaults.  It's a subtle boundary condition in
> > > _mysql._mysql_ResultObject_fetch_row, around line 850 or so:
> > >
> > > if (!(r = PyTuple_New(maxrows))) goto error;
> > > while (1) {
> > > rowsadded = _mysql__fetch_row(self, &r, skiprows,
> > > maxrows, convert_row);
> > > printf("rowsadded = %d\n", rowsadded); /*debug*/
> > > if (rowsadded == -1) goto error;
> > > skiprows += rowsadded;
> > > if (rowsadded < maxrows) break;
> > >             ^^^^^^^^^^^^^^^^^^^^
> > >
> > > The last statement with "rowsadded < maxrows" is buggy: it
> > > should be "rowsadded <= maxrows".  Otherwise, we add more
> > > elements in an already-full tuple, and that'll cause the
> > > segfault.
> > >
> > >
> > >
> >
> -------------------------------------------------------------------
> > > ---
> > >
> > > You can respond by visiting:
> > >
> >
http://sourceforge.net/tracker/?func=detail&atid=374932&aid=536260&grou
> > p_id=22307
> > >
> >
> >
> >
> > _______________________________________________
> > Gnue-dev mailing list
> > address@hidden
> > http://mail.gnu.org/mailman/listinfo/gnue-dev
> >