Re: [Koha-devel] switching from marc_words to zebra [LONG]

[Thomas D]

4 July 2005


Quoting Paul POULAIN <address@hidden> :
> ---------------- Beginning of the original message ------------------
> 

snip

> in Koha 2.2, there are a lot of tables to manage biblios :

[snip]

> 
> * marc_biblio, is a table that contains only a few
> informations :
> - biblionumber (biblio PK)
> - bibid (marc PK. It's a design mistake I made, for sure)

What was the design mistake here?

[snip]


> move to ZEBRA
> =============

[snip]

> It should really not be a pain to move to zebra with this
> structure : 
> every call with a MARC::Record (NEWxxxxyyyy subs) manages the
> storing of 
> the MARC::Record in marc_* tables. We could replace this code
> with a 
> zebra insert/update, using biblio.biblionumber as primary key.
> How to manage biblios and items ? My idea here would be to
> store biblio 
> + all items informations in zebra, using a full MARC::Record,
> that 
> contains biblio and items.

For MARC21, would you use the 852 holdings location and other fields in MARC
bibliographic?  MARC21 has a MARC holdings format as a separate record type
from MARC bibliographic, http://www.loc.gov/marc/holdings/echdhome.html . 
UNIMARC now also has a UNIMARC holdings format as a separate record type,
http://www.ifla.org/VI/8/projects/UNIMARC-HoldingsFormat04.pdf .  I would
certainly advocate moving towards full use of the available formats but
perhaps one should approach the question by degrees to preserve development
resources.

[snip]

> 
> The MARC Editor
> ===============
> Some users thinks Koha MARC Editor could be improved. The best
> solution 
> would be, imho, to provide an API to use an external MARC
> editor if the 
> library prefers.
> However, some libraries are happy with what exists. So the
> MARC editor 
> should be kept (& improved where possible). so
> marc_*_structure tables 
> are still needed. Some fields could be removed probably, as
> they are 
> related to search (like seealso), and will be handled by zebra
> config 
> file. This still has to be investigated.
> 
> For libraries that prefers an external MARC editor, we could
> create a 
> webservice, where the user does an http request, with iso2709
> data in 
> parameters, with the requested operation.
> This should be quite easy to do (the problem being to know how
> the 
> external software can handle this. If someone has an idea or
> an 
> experience on this, feel free to post here ;-) )
> 

An API for any editor seems the ideal solution here.  This may be a point at
which Koha should make overtures to cooperating with other systems editors
including proprietary systems while working on improving its own editor.

> 
> The authority problem
> =====================
> Authorities have to be linked to the biblio that uses them.
> Thus, when 
> an authority is modified, all biblios using them are
> automatically 
> modified (script in misc/merge_authority.pl in Koha cvs &
> 2.2.x)
> 
> To keep trace of the link, Koha uses a $9 local subfield. In
> UNIMARC, 
> the $3 can also be used for this. I don't know if something
> equivalent 
> to $3 exists in MARC21 (could not find information on 
> http//www.loc.gov/marc/)
> Many scripts make a heavy use of marc_subfield_table $9 data.
> For 
> example, when you find an authority in authority module, you
> get the 
> number of biblios using this authority. This number is
> calculated with a 
> SQL request on $9 subfield.
> To handle this with zebra, we have 2 solutions :
> - create a table just with the link (biblionumber / authority
> number) 
> that we could query
> - query zebra with exact $9 subfield value
> 
> I don't know zebra enough to be sure of the best way to do it.
> Any 
> suggestion/experience welcomed.

Curiously MARC21 and its predecessors seem to have left this out of the
format.  Only the thesaurus is given in the MARC21 record.  The method of
matching a thesaurus control number is left to the system implementation. 
This does seem troublesome to have a mutable key used in a MARC
bibliographic record for matching to an authority record.  Of course, this
reflects the controlled authority thesauri preceding the computerisation and
electronic exchange of the authority data.  The MARC authority formats were
developed after the MARC bibliographic format and MARC bibliographic does
not seem to have been updated to use an immutable key as in UNIMARC.  I do
not know the standard system implementation practise.  I will ask a few
places and see what comes back.

In MARC21 leader position 18 sometimes supplemented by 40$e specifies the
descriptive rules governing names,
http://www.loc.gov/marc/relators/reladesc.html .  Whether that is sufficient
to really know what names authority database is used leaves me in some doubt.

In MARC21, the 2nd indicator designates the most prevalent US and Canadian
thesauri for subjects, otherwise $2 is used,
http://www.loc.gov/marc/relators/relasour.html .  This is equivalent to $2
for subjects in UNIMARC, http://www.ifla.org/VI/3/p1996-1/appx-g.htm  

> 
> The authority problem (another one...)
> ======================================
> Authorities are MARC::Records too... (without items)
> So they also have auth_structure & auth_word & all the infos
> that are in 
> biblios (except items level, as there is no "authority"
> items).
> so we could imagine to have 2 zebra databases : one for
> biblios and one 
> for authorities.
> Everything previously in this mail can be copied here. That's
> something 
> we could investigate after moving MARC biblios to zebra, as we
> would 
> have more experience on this tool.
> 

This seems reasonable.  Taking the model change by stages should avoid
breaking everything at once for the development version when difficulties occur.

[snip]

> 
> -- 
> Paul POULAIN
> Consultant indépendant en logiciels libres
> responsable francophone de koha (SIGB libre
> http://www.koha-fr.org)
> 
> 
> -------------------------------------------------------
> SF.Net email is sponsored by: Discover Easy Linux Migration
> Strategies
> from IBM. Find simple to follow Roadmaps, straightforward
> articles,
> informative Webcasts and more! Get everything you need to get
> up to
> speed, fast.
> http://ads.osdn.com/?ad_idt77&alloc_id?492&op=click
> _______________________________________________
> Koha-devel mailing list
> address@hidden
> https://lists.sourceforge.net/lists/listinfo/koha-devel
> 
> ------------------- End of the original message ---------------------


Thomas D

---------------------------------------------
Protect your mails from viruses thanks to Alinto Premium services 
http://www.alinto.com