[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Octave-bug-tracker] [bug #60387] ftp class method 'dir' extremely slow
From: |
Rik |
Subject: |
[Octave-bug-tracker] [bug #60387] ftp class method 'dir' extremely slow |
Date: |
Tue, 13 Apr 2021 13:58:49 -0400 (EDT) |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36 |
Follow-up Comment #9, bug #60387 (project octave):
This code seems really, really complicated for doing what it does. Actions
start by calling a method function of an old-style classdef object in @ftp
dir. From there it jumps to C++ function in libinterp/corefcn/__ftp__.cc.
Functions there call other functions in url_manager.cc. Eventually, some
calls seem to be split between Octave's own iostream class and url_transfer
class which is in liboctave/utils. And over in liboctave the code is split
between a class that just holds a rep pointer and the base class which
actually does the work.
Anyways, at the heart of the dir loop are calls to get_fileinfo which is
located in liboctave/util/url-transfer.cc. The function is
void get_fileinfo (const std::string& filename, double& filesize,
time_t& filetime, bool& fileisdir)
{
std::string path = pwd ();
m_url = "ftp://" + m_host_or_url + '/' + path + '/' + filename;
SETOPT (CURLOPT_URL, m_url.c_str ());
SETOPT (CURLOPT_FILETIME, 1);
SETOPT (CURLOPT_HEADERFUNCTION, throw_away);
SETOPT (CURLOPT_WRITEFUNCTION, throw_away);
// FIXME
// The MDTM command fails for a directory on the servers I tested
// so this is a means of testing for directories. It also means
// I can't get the date of directories!
perform ();
if (! good ())
{
fileisdir = true;
filetime = -1;
filesize = 0;
return;
}
fileisdir = false;
time_t ft;
curl_easy_getinfo (m_curl, CURLINFO_FILETIME, &ft);
filetime = ft;
double fs;
curl_easy_getinfo (m_curl, CURLINFO_CONTENT_LENGTH_DOWNLOAD, &fs);
filesize = fs;
SETOPT (CURLOPT_WRITEFUNCTION, write_data);
SETOPT (CURLOPT_HEADERFUNCTION, 0);
SETOPT (CURLOPT_FILETIME, 0);
m_url = "ftp://" + m_host_or_url;
SETOPT (CURLOPT_URL, m_url.c_str ());
// The MDTM command seems to reset the path to the root with the
// servers I tested with, so cd again into the correct path. Make
// the path absolute so that this will work even with servers that
// don't end up in the root after an MDTM command.
cwd ('/' + path);
}
This is performing a large number of cURL operations and is likely where the
slow down lies. The 'cwd' command alone is not costless.
_______________________________________________________
Reply to this item at:
<https://savannah.gnu.org/bugs/?60387>
_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/
- [Octave-bug-tracker] [bug #60387] ftp class method 'dir' extremely slow, Muhali, 2021/04/13
- [Octave-bug-tracker] [bug #60387] ftp class method 'dir' extremely slow, Dmitri A. Sergatskov, 2021/04/13
- [Octave-bug-tracker] [bug #60387] ftp class method 'dir' extremely slow, Rik, 2021/04/13
- [Octave-bug-tracker] [bug #60387] ftp class method 'dir' extremely slow, Dmitri A. Sergatskov, 2021/04/13
- [Octave-bug-tracker] [bug #60387] ftp class method 'dir' extremely slow, Rik, 2021/04/13
- [Octave-bug-tracker] [bug #60387] ftp class method 'dir' extremely slow, John W. Eaton, 2021/04/13
- [Octave-bug-tracker] [bug #60387] ftp class method 'dir' extremely slow, John W. Eaton, 2021/04/13
- [Octave-bug-tracker] [bug #60387] ftp class method 'dir' extremely slow, Rik, 2021/04/13
- [Octave-bug-tracker] [bug #60387] ftp class method 'dir' extremely slow, Rik, 2021/04/13
- [Octave-bug-tracker] [bug #60387] ftp class method 'dir' extremely slow,
Rik <=
- [Octave-bug-tracker] [bug #60387] ftp class method 'dir' extremely slow, John W. Eaton, 2021/04/13
- [Octave-bug-tracker] [bug #60387] ftp class method 'dir' extremely slow, Rik, 2021/04/13
- [Octave-bug-tracker] [bug #60387] ftp class method 'dir' extremely slow, Rik, 2021/04/13
- [Octave-bug-tracker] [bug #60387] ftp class method 'dir' extremely slow, John W. Eaton, 2021/04/14
- [Octave-bug-tracker] [bug #60387] ftp class method 'dir' extremely slow, Rik, 2021/04/14
- [Octave-bug-tracker] [bug #60387] ftp class method 'dir' extremely slow, Rik, 2021/04/17