Re: [Bug-wget] [GSoC Proposal] Python3-based FTP Server for Test Suit and Test Suit Refactoring

[Darshit Shah]

On Fri, Mar 7, 2014 at 3:32 PM, 陈子杭 (Zihang Chen) <address@hidden>wrote:

> Thanks for your comments! :D
>
>
> 2014-03-07 22:17 GMT+08:00 Darshit Shah <address@hidden>:
>
> Hi Zihang,
>>
>> The proposal seems good.
>>
>>
>> On Fri, Mar 7, 2014 at 11:39 AM, 陈子杭 (Zihang Chen) <address@hidden>wrote:
>>
>>> * this is only a draft, so it's not written according to the GNU GSoC
>>> template. When discussion is done, I'll get a formal one done (with
>>> schedule etc.) *
>>>
>>> Briefing
>>> ====
>>>
>>> As is known, a new Python3-based test suite was implemented to replace
>>> the
>>> old one. And a configurable HTTP server utilizing `http.server` and
>>> `http.BaseHTTPRequestHandler` was also implemented. Thus the HTTP server
>>> can be configured to respond certain headers according to the
>>> configuration
>>> in separate test cases.
>>>
>>> However, a configurable FTP server was not required to be implemented,
>>> which breaks the consistency of the test suite of wget. So it's necessary
>>> that a configurable FTP server be built.
>>>
>>> Initial Design
>>> ====
>>>
>>> The test suite consists of two parts: `WgetTest.py` (module that actually
>>> runs the tests), servers (configurable HTTP and FTP server). When running
>>> test cases, `HTTPTest` instances collect configurations from test cases
>>> and
>>> start the servers with them so that the HTTP handler will handle the
>>> incoming requests according to the configurations (also known as rules).
>>>
>>> About the FTP server, I think the handler architecture can be applied
>>> here:
>>> implement multiple handlers corresponding to the FTP commands and
>>> register
>>> them in server instances automatically. When registering handlers, the
>>> server should also inform the handlers about the configurations. When new
>>> FTP command arrives, the server invoke the corresponding handler, and
>>> then
>>> the handler look up the rules itself and do the expected actions. When
>>> doing real FTP actions, I intend to use third-party FTP library like the
>>> current FTP server (I fear sockets so best not directly touch them).
>>>
>>> There's two things that are unclear:
>>> * the necessity of implementing all the FTP command. I presume that wget
>>> wouldn't utilize all of them.
>>>
>> If you're using a library, e.g. pyftpdlib, I don't see why you'd have to
>> implement
>> any commands on your own.
>>
> If the server is configurable, I think I'll have to intercept the commands
> so I can apply rules on them. When I say implement FTP command, I mean
> creating handle for the command, and invoke library in the handle. I think
> that's what mislead you probably.
>
Oh. Okay then. In that scenario, you could probably add the command
handlers
iteratively as and when they are required by a test case.

Another way would be to look at Wget's FTP client implementation and see
what is the
superset of commands that it may send. If you're not comfortable with C, I
would help
you out with that.

>    Simply use the library for it.
>>
>>> * what will the configurations be, do I need to design the tests myself?
>>> Or
>>> should I also port the Perl scripts to Python3?
>>>
>> Porting the existing Perl tests should give you a good idea of the
>> various
>> configurations that will be needed. Looking up the Wget source is also
>> often
>> a good idea. Either ways, some of us can help out with identifying and
>> writing
>> test cases when the time comes.
>>
>>
>>> Test Suite Refactoring
>>> ====
>>>
>>> While reading source, I don't find the test suite pythonic. Multiple
>>> parts
>>> of it seems unmaintainable. If the test suite can be refactored into a
>>> better coded one,  maintainability can further be achieved.
>>>
>>> Yes. I agree the current python based test suite might advantage from a
>> few refactorings. There are a few places where the test suite is quite
>> un-pythonic
>> and fixing those would be a nice idea.
>>
>> Examples of unpythonicness:
>>> * docstrings out of classes or methods
>>> * mutable objects as default argument values
>>> * multiple use of operator is (controversial)
>>> * writing `_replace_substring (WgetTest.py:119)` instead of utilizing the
>>> format specification mini-language
>>>
>>> Examples of unmaintainability:
>>> * referencing attributes which are defined in subclasses
>>> (`WgetTest.py:61`
>>> references `self.servers` which is defined in its subclass
>>> `WgetTest.py:254`)
>>> * inappropriate variable names (in `WgetTest.py:254` `self.servers`
>>> actually becomes an integer while it regarded normally as a list of
>>> servers)
>>>
>> Yes, the variable names are horrible aren't they? That should be
>> refactored. (Shouldn't
>> take much time ).
>>
>>> * no intuitive, self-declarative module hierarchy
>>>
>> The module hierarchy is slightly complicated. Making it more intuitive
>> would help.
>>
>>>
>>> About me
>>> ====
>>> My name's Zihang Chen. I'm a 3rd undergraduate student from China
>>> (timezone: UTC+8). Please forgive me for any mistaken uses of the English
>>> language. I have been using Python for 3 years. I mostly code for myself
>>> (assignment, interest etc.) Never worked for any open source
>>> organizations
>>> before, so I'm thrilled to apply for GSoC. You can visit my GitHub
>>> page here<https://github.com/mad4alcohol>
>>>
>>> .
>>>
>>> Looking forward to your comments. Thanks very much.
>>>
>>
>>
>>
>> --
>> Thanking You,
>> Darshit Shah
>>
>>
>
>
> --
> Regards,
> Chen Zihang,
> Computer School of Wuhan University
> ---
> 此致
> 陈子杭
> 武汉大学计算机学院
>
>


-- 
Thanking You,
Darshit Shah