[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Pan-users] Multiple Server support for PAN: Need a spec
From: |
Frank Bruno |
Subject: |
Re: [Pan-users] Multiple Server support for PAN: Need a spec |
Date: |
Sat, 26 Oct 2002 11:37:46 -0400 |
One suggestion is to look at the database format for BNR which seems to handle
multiple servers quite well: (information from posts regarding BNR1 in the BNR
forum on Yahoo groups) Hopefully this might be useful.
Here is some description about the cache file structure used
by BNR
in version 1.1.3 and later (internally called "cache file
version 3").
The file begins with a header record:
THeaderRec = packed record
FCount: Integer; // Total count of articles in cache
FMinIDs: array[0..30] of Integer; // First available article
nums
FIDCounts: array[0..30] of Integer; // Number of articles in
each list
FHostLen: array[0..30] of Byte; // Server host name lengths
FVersion: Byte; // Version number of the cached file
end;
(This description is only valid for files where FVersion = 3).
After this header record there are a number of server names whose
length is given by the FHostLen values above.
After these the rest of the file contains an article record
followed by
variable length of text data, followed by the next article record
etc.
as follows:
TArtRecV3 = packed record
FIDs: array[0..30] of Integer; // ID numbers on various servers
FBytes: Integer; // Size of this article (as text)
FFileSize: Integer; // Size of the decoded file
FFlags: Cardinal; // Flags of this article
FTime: Integer; // Date and time of this article
FPartSize: Integer; // FileSize of a single part (if MultiPart)
FCurPart: Word; // Part number (if Multipart)
FMaxPart: Word; // Number of parts (if MultiPart)
FCurFile: Word; // File number
FMaxFile: Word; // Number of files in the posted set
FPriority: TDownloadPriorityType; // Downloading priority
FMsgIDLen: Byte; // Length of FMessageID
FSubLen: Byte; // Length of FSubject
FNameLen: Byte; // Length of FFilename
FFromLen: Byte; // Length of FFrom
end;
All Integers and Cardinals are 4 bytes. FTime is in DOSFileDateTime
format.
TDownloadPriorityType = (dptNone, dptLow, dptNormal, dptHigh,
dptHigher,
dptHighest);
(in C this would be an enum and takes 1 byte). The last 4 values
above give
the lengths
of the following variable-length strings (they can be zero, in
which case
see below).
FFlags is a 32-bit vector as follows:
TArtFlag = ( afRead, // This article has been marked read
afDownloaded, // This article has been downloaded
afMarked, // This article is marked for download
afApproxSize, // We know the approximate file size
afExactName, // We know the exact file name
afBadSize, // Article has questionable size
afExactSize, // We know the exact file size
afMultiPart, // This article is part of a thread
// Byte 1: Run-time flags, not saved in cache
afGoodParts, // All the parts are available
afBadParts, // Some parts are missing
afWanted, // This article is Wanted
afKilled, // This article has been Killed
afEncodeUUE, // This article is encoded with UUE
afEncodeMIME, // This article is encoded with MIME
afExpired, // This article has been expired
afNowDownloading, // This article is currently being downloaded
// Byte 2: Extra flags and speedup flags
afMarkedAlone, // This article has been marked alone
afOtherServerWaiting, // Another server is waiting for this article
to
get downloaded
afHideIfCombined, // Same as (afGoodParts in Flags) and (CurPart >
0)
and (CurPart < MaxPart)
afHeaderOnly, // Download header only
afBuildSubject, // Subject needs to be built before shown
afHideIfOverview, // Hide this article from overview view mode
afFree26,
afFree27,
// Byte 3: These flags give the file check existence bitmask
afFree30,
afEDBSize, // Same size file exists on RemDrvDB
afESize, // Same size file exists on disk
afEDBName, // Same name file exists on RemDrvDB
afEName, // Same name file exists on disk
afEDBBoth, // Same name&size file exists on RemDrvDB
afEBoth, // Same name&size file exists on disk
afReserved // THIS FLAG MUST NOT BE USED!
);
FSubLen, FNameLen and FFromLen can be zero, in which case this
article either has empty strings or it uses the same subject, name
or
filename as the previous article as follows:
If afEName flag is set, this article uses the same FileName as the
previous article in cache.
If afEBoth flag is set, this article uses the same Subject as the
previous article in cache.
If afESize flag is set, this article uses the same From header as
the previous article in cache.
A couple of things I forgot to mention:
1) If the afBuildSubject flag is set, the subject in the cache file
needs some formatting to get it into human-readable form. In that
case the subject may contain the following tags:
const
FILENUMCHAR = #1;
FILENOFCHAR = #2;
FILENAMECHAR = #3;
PARTNUMCHAR = #4;
FILESIZECHAR = #5;
FILESIZEKBCHAR = #6;
(Here #1 is Delphi's way of saying ASCII code 1, '\001' in C)
When you find those flags from the subject they are replaced by:
FILENUMCHAR: FCurFile/FMaxFile
FILENOFCHAR: FCurFile of FMaxFile
FILENAMECHAR: FFileName
PARTNUMCHAR: FCurPart/FMaxPart
FILESIZECHAR: FFileSize
FILESIZEKBCHAR: If FFileSize / 1024 is an integer, use it, else
use FFileSize / 1024 formatted to show 2 decimals.
2) FBytes value is not used in BNR versions 1.1.4 and later,
instead
the Bytes/PartSize conversion uses the following formula:
FBytes := Round(FPartSize * 62 / 45) + 500;
"J.B. Moreno" wrote:
> On 10/21/02 12:33 PM, Charles Kerr at <address@hidden> wrote:
>
> -snip-
> > Many servers don't handle retrieving bodies keyed by Message-ID;
> > using the group's index number for the article is more reliable.
>
> Article numbers are certainly more reliable, but do many people have a
> problem with MID? The only circumstances under which that is likely to be a
> problem is when the history database isn't large enough -- and even then it
> is most likely only a problem with older articles that are about to expire
> (otherwise the size of the history is set way too low).
>
> --
> J.B. Moreno
>
> _______________________________________________
> Pan-users mailing list
> address@hidden
> http://mail.nongnu.org/mailman/listinfo/pan-users