Re: [Maposmatic-dev] [PATCH maposmatic] Improve the file cleanup mechanism

[Thomas Petazzoni]

On Thu,  4 Feb 2010 22:54:05 +0100
Maxime Petazzoni <address@hidden> wrote:

> Previously, when the rendering directory was over the defined
> threshold, files where removed progressively, oldest first, to make
> up some space. No information was kept about jobs whose files were
> removed, making it harder to keep track of valid jobs with files
> available.
> 
> This change introduces two new things:
> 
>   1. a new job status, 3, for jobs processed but now without files, or
>      "obsolete" jobs.
>   2. a new cleanup mechanism that considers jobs as the atomic unit of
>      cleaning instead of files, as this would leave with jobs without
>      all their renderings (which didn't make much sense).
> 
> The cleanup function underwent the following modifications:
> 
>   * files are now sorted by content modification time and not last
>     metadata change (a simple chmod could mess up the order);
>   * thumbnails are excluded from the list of considered files for
>     removal (this is still is discussion, but for now let's keep
> them);
>   * when a file needs to be removed, all files from its parent job are
>     removed and the job's status is set to 3 (see
>     MapRenderingJob#remove_all_files).
>   * if no parent job can be found, it's an orphaned file and can be
>     safely removed. Files starting with a '.' are of course preserved;
>   * some logging improvements during the cleanup phase.
> 
> New 'job-done-obsolete' and 'job-error-obsolete' status icons are now
> available, and the status icon filename is now inferred with a custom
> template tag (this also led to some cleanup in extratags.py).
> 
> The file size of the renderings is also displayed next to each format
> in the job information.
> 
> Signed-off-by: Maxime Petazzoni <address@hidden>

ACK.

During the deployment on dev.m.o, we must figure out the right Python
script to properly update the job status to 3 for existing jobs for
which the files have already been removed. Therefore, when the
deployment will take place on www.m.o, we'll be able to do this easily.

Cheers,

Thomas
-- 
Thomas Petazzoni                         http://thomas.enix.org
Promouvoir et défendre le Logiciel Libre http://www.april.org
Logiciels Libres à Toulouse              http://www.toulibre.org

signature.asc
Description: PGP signature