[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Koha-cvs] CVS: koha-doc/en/sysadmin/migration CDS_ISIS_to_Koha.html,NON
|
From: |
skemotah |
|
Subject: |
[Koha-cvs] CVS: koha-doc/en/sysadmin/migration CDS_ISIS_to_Koha.html,NONE,1.1 CDS_ISIS_to_Koha.xml,NONE,1.1 |
|
Date: |
Tue, 16 Nov 2004 04:08:17 -0800 |
Update of /cvsroot/koha/koha-doc/en/sysadmin/migration
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4271
Added Files:
CDS_ISIS_to_Koha.html CDS_ISIS_to_Koha.xml
Log Message:
Load CDS/ISIS to Koha guide
--- NEW FILE ---
<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>Migrating a CDS/ISIS based system to Koha</title><link
rel="stylesheet" href="html.css" type="text/css"><meta name="generator"
content="DocBook XSL Stylesheets V1.65.1"></head><body bgcolor="white"
text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div
class="article" lang="en"><div class="titlepage"><div><div><h2 class="title"><a
name="d0e1"></a>Migrating a CDS/ISIS based system to Koha</h2></div><div><div
class="author"><h3 class="author"><span class="firstname">Baiju</span> <span
class="surname">M</span></h3><tt class="email"><<a href="mailto:baijum81 AT
hailmail.net">baijum81 AT hailmail.net</a>></tt></div></div><div><p
class="pubdate">2004-11-16</p></div><div><div class="revhistory"><table
border="1" width="100%" summary="Revision history"><tr><th align="left"
valign="top" colspan="3"><b>Revision History</b></th></tr><tr><td
align="left">Revision 2.0.0p1</td><td align="left">2004-11-15</td><td
align="left">bm</td></tr><tr><td align="left" colspa
n="3"><p>Small additions by BM.</p></td></tr><tr><td align="left">Revision
2.0.0</td><td align="left">2004-04-01</td><td align="left">bm</td></tr><tr><td
align="left" colspan="3"><p>Initial XML markup (by Stephen Hedges) of a wiki
page posted at <a
href="http://www.saas.nsw.edu.au/koha_wiki/index.php?page=CDS%2FISIS+based+system+to+Koha";
target="_top">http://www.saas.nsw.edu.au/koha_wiki/index.php?page=CDS%2FISIS+based+system+to+Koha</a>
on 2004-04-01, with links to program listings
expanded.</p></td></tr></table></div></div></div><div></div><hr></div><div
class="toc"><dl><dt><span class="section"><a href="#d0e41">1. Copyright and
License</a></span></dt><dt><span class="section"><a href="#d0e69">2.
Disclaimer</a></span></dt><dt><span class="section"><a href="#d0e76">3.
Introduction</a></span></dt><dt><span class="section"><a href="#d0e87">4.
Migrating Book Data</a></span></dt><dt><span class="section"><a
href="#d0e130">5. Migrating Member Data</a></span></dt></dl></div><font
color="red"><authorblurb><p>By Baiju
M</p></authorblurb></font><div class="section" lang="en"><div
class="titlepage"><div><div><h2 class="title" style="clear: both"><a
name="d0e41"></a>1. Copyright and
License</h2></div></div><div></div></div><p>Copyright 2004 Baiju M <tt
class="email"><<a href="mailto:baijum81 AT hailmail.net">baijum81 AT
hailmail.net</a>></tt></p><p>This document is related to Koha and is
licensed to you under the GNU General Public License version 2 or later (<a
href="http://www.gnu.org/licenses/gpl.html";
target="_top">http://www.gnu.org/licenses/gpl.html</a>).</p><p>Koha-related
documents may be reproduced and distributed in whole or in part, in any medium
physical or electronic, as long as this copyright notice is retained on all
copies.</p><p>You may create a derivative work and distribute it provided that
you:</p><div class="orderedlist"><ol type="1"><li><p>License the derivative
work with this same license, or the Linux Docume
ntation Project License (<a href="???"
target="_top">http://www.tldp.org/COPYRIGHT.html</a>). Include a copyright
notice and at least a pointer to the license used.</p></li><li><p>Give due
credit to previous authors and major
contributors.</p></li></ol></div><p>Commercial redistribution is allowed and
encouraged; however, the author would like to be notified of any such
distributions.</p></div><div class="section" lang="en"><div
class="titlepage"><div><div><h2 class="title" style="clear: both"><a
name="d0e69"></a>2. Disclaimer</h2></div></div><div></div></div><p>No
liability for the contents of this document can be accepted. Use the concepts,
examples and information at your own risk. There may be errors and inaccuracies
that could be damaging to your system. Although this is highly unlikely,
proceed with caution; the author(s) do not take any responsibility.</p><p>All
copyrights are held by their respective owners, unless specifically noted
otherwise. Use of a term in
this document should not be regarded as affecting the validity of any
trademark or service mark. Naming of particular products or brands should not
be seen as endorsements.</p></div><div class="section" lang="en"><div
class="titlepage"><div><div><h2 class="title" style="clear: both"><a
name="d0e76"></a>3. Introduction</h2></div></div><div></div></div><p>I am
doing this migration from CDS/ISIS (version : 1.5) system to Koha for St.
Joseph's College, Devagiri, Calicut. <a href="http://devagiricollege.org/";
target="_top">http://devagiricollege.org/</a></p><p>In the current system it is
possible to do circulation and other basic functionalities of the
library.</p><p>The work is in progress, so give any pointers if you
have.</p></div><div class="section" lang="en"><div
class="titlepage"><div><div><h2 class="title" style="clear: both"><a
name="d0e87"></a>4. Migrating Book
Data</h2></div></div><div></div></div><div class="itemizedlist"><ul
type="disc"><li><p>First I expor
ted Books data into XML using CDS/ISIS export tool (available in the ISIS
Utils menu).</p><div class="note" style="margin-left: 0.5in; margin-right:
0.5in;"><h3 class="title">Note</h3><p>Users of previous CDS/ISIS versions (i.e,
1.3 or before), export documents data from CDS/ISIS and save it as Common
Communication Format (CCF), which is a standard based on ISO 2709. Import this
ISO data into CDS/ISIS version 1.4 or later. This step is necessary to export
the data as XML, the old versions of ISIS do not have support for exporting
document data as XML directly. <i><span class="remark">FIXME: ISIS 1.3.x
support exporting XML?</span></i></p></div></li><li><p>Then I wrote a python
script (shown below) to convert XML to tab delimited text format.</p><div
class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3
class="title">Note</h3><p>This step is not necessary, but you can do some
corrections in data quickly.</p></div><div class="tip" style="margin-left:
0.5in; margin-
right: 0.5in;"><h3 class="title">Tip</h3><p>Use Emacs macro or Vim recording
for quick parsing.</p></div><pre class="programlisting">## ciddtf.py : Convert
ISIS Document Database to Text File
##Before Parsing :
##Change RECORDX\xc3\x91_5\xc3\xaf\x03\x12 to RECORD
##Change & to and
##fix problems of <Tag_97> manualy.
import libxml2
record = []
def Record(i, node):
sub_node = node.children
while sub_node is not None:
if sub_node.type == "element":
record[i][sub_node.name] = sub_node.get_content()
sub_node = sub_node.next
else:
sub_node = sub_node.next
# Memory debug specific
libxml2.debugMemory(1)
isis_data = libxml2.parseFile ('../xmls/docs.xml')
root = isis_data.getRootElement()
node = root.children
i = 0
while node is not None:
if node.type != "element":
node = node.next
continue
if node is None:
break
if node.name == "RECORD":
##print node.get_content()
record.append({})
Record(i, node)
i = i + 1
node = node.next
else:
print "unhandled node in <isis_xml>: " + node.name
isis_data.freeDoc()
# Memory debug specific
libxml2.cleanupParser()
if libxml2.debugMemory(1) == 0:
print "OK"
else:
print "Memory leak %d bytes" % (libxml2.debugMemory(1))
libxml2.dumpMemory()
####################################################################
record_file = open('../outs/docs.txt', 'w')
tag = ['Tag_4', 'Tag_7', 'Tag_11', 'Tag_12', 'Tag_13', 'Tag_18', 'Tag_19',
'Tag_20', 'Tag_21', 'Tag_24', 'Tag_38', 'Tag_10', 'Tag_114', 'Tag_115',
'Tag_110', 'Tag_5', 'Tag_122', 'Tag_97', 'Tag_151', 'Tag_153', 'Tag_123']
for j in record:
if j.has_key('Tag_691'):
record_file.write(j['Tag_691'])
##Check docs without 114 tag
## if j.has_key('Tag_114'):
## pass
## else:
## print j['Tag_691']
else:
record_file.write('_B_L_A_N_K_')
for h in tag:
if j.has_key(h):
record_file.write('\t'+j[h])
else:
record_file.write('\t_B_L_A_N_K_')
record_file.write('\n')</pre><p>Here is a sample XML:</p><div
class="literallayout"><p><?xml version="1.0" encoding="ISO-8859-1"?><br>
<!DOCTYPE DATABASE_SJCLDB SYSTEM "SJCLDB.dtd"><br>
<DATABASE_SJCLDB><br>
<RECORD><br>
<Tag_691>43009</Tag_691><br>
<Tag_4>Chiriyute pithamahan: V K N Krutikalekkurichulla paddhanangal</Tag_4><br>
<Tag_11>DC Books</Tag_11><br>
<Tag_12>Kottayam</Tag_12><br>
<Tag_13>1997</Tag_13><br>
<Tag_18>Roy</Tag_18><br>
<Tag_19>P C</Tag_19><br>
<Tag_38>Ed</Tag_38><br>
<Tag_10>134p</Tag_10><br>
<Tag_114>894.37 Roy</Tag_114><br>
<Tag_115>N7</Tag_115><br>
<Tag_110>GL-M3</Tag_110><br>
<Tag_122>2000-03-28</Tag_122><br>
<Tag_151>Rs.40.00</Tag_151><br>
<Tag_123>Tom</Tag_123><br>
</RECORD><br>
<RECORD><br>
<Tag_691>43018</Tag_691><br>
<Tag_4>Edanadan pattu</Tag_4><br>
<Tag_11>Current Books</Tag_11><br>
<Tag_12>Kottayam</Tag_12><br>
<Tag_13>1997</Tag_13><br>
<Tag_18>Sajitha</Tag_18><br>
<Tag_19>K R</Tag_19><br>
<Tag_38>Ed</Tag_38><br>
<Tag_10>160p</Tag_10><br>
<Tag_114>398.87 Saj</Tag_114><br>
<Tag_115>N7</Tag_115><br>
<Tag_110>GL-3</Tag_110><br>
<Tag_122>2000-03-28</Tag_122><br>
<Tag_151>Rs.50.00</Tag_151><br>
<Tag_123>Tom</Tag_123><br>
</RECORD><br>
<RECORD><br>
<Tag_691>38980</Tag_691><br>
<Tag_4>Plays</Tag_4><br>
<Tag_11>EWP</Tag_11><br>
<Tag_12>New York</Tag_12><br>
<Tag_13>1989</Tag_13><br>
<Tag_18>O'Neill</Tag_18><br>
<Tag_19>Eugene</Tag_19><br>
<Tag_114>O7,2 One</Tag_114><br>
<Tag_115>M9.1</Tag_115><br>
<Tag_122>1996-06-27</Tag_122><br>
</RECORD><br>
<RECORD><br>
<Tag_691>44579</Tag_691><br>
<Tag_4>Arangukaanatha Nadan</Tag_4><br>
<Tag_11>Current</Tag_11><br>
<Tag_12>Thrissur</Tag_12><br>
<Tag_13>1998</Tag_13><br>
<Tag_18>Thikkodian</Tag_18><br>
<Tag_10>396p</Tag_10><br>
<Tag_114>894.8 Thi/Ara</Tag_114><br>
<Tag_115>N8</Tag_115><br>
<Tag_110>GL-M8</Tag_110><br>
<Tag_122>2001-06-19</Tag_122><br>
<Tag_151>Rs.135.00</Tag_151><br>
<Tag_123>Tom</Tag_123><br>
</RECORD><br>
<RECORD><br>
<Tag_691>16469</Tag_691><br>
<Tag_4>Sartre: A collection of critical essays</Tag_4><br>
<Tag_11>Prentice Hall</Tag_11><br>
<Tag_12>USA</Tag_12><br>
<Tag_13>1965</Tag_13><br>
<Tag_18>Kern</Tag_18><br>
<Tag_19>Edith</Tag_19><br>
<Tag_38>Ed</Tag_38><br>
<Tag_10>180p</Tag_10><br>
<Tag_114>O-,4 M79</Tag_114><br>
<Tag_115>J5</Tag_115><br>
<Tag_110>Twentieth century views</Tag_110><br>
<Tag_122>1998-09-26</Tag_122><br>
</RECORD><br>
<RECORD><br>
<Tag_691>42068</Tag_691><br>
<Tag_4>Internet for beginners</Tag_4><br>
<Tag_11>Icon Books</Tag_11><br>
<Tag_12>Australia</Tag_12><br>
<Tag_13>1997</Tag_13><br>
<Tag_18>Brunner</Tag_18><br>
<Tag_19>laurel</Tag_19><br>
<Tag_20>Jevtic</Tag_20><br>
<Tag_21>Zoran</Tag_21><br>
<Tag_10>175p</Tag_10><br>
<Tag_115>N7</Tag_115><br>
<Tag_122>1998/09/26</Tag_122><br>
<Tag_123>CPYR</Tag_123><br>
</RECORD><br>
<RECORD><br>
<Tag_691>42367</Tag_691><br>
<Tag_4>Text book of Biochemistry with clinical correlations</Tag_4><br>
<Tag_7>Ed.4</Tag_7><br>
<Tag_11>Wiley-Liss/John Wiley and Sons</Tag_11><br>
<Tag_12>New York</Tag_12><br>
<Tag_13>1997</Tag_13><br>
<Tag_18>Devlin</Tag_18><br>
<Tag_19>Thomas M</Tag_19><br>
<Tag_38>Ed</Tag_38><br>
<Tag_10>1186p</Tag_10><br>
<Tag_114>612.015 Dev/KD</Tag_114><br>
<Tag_115>N7</Tag_115><br>
<Tag_110>Zoology</Tag_110><br>
<Tag_122>1998-10-08</Tag_122><br>
<Tag_97>$.30.95</Tag_97><br>
</RECORD><br>
</DATABASE_SJCLDB> </p></div></li><li><p>Now parse the tab separated
file using a perl script (shown below) and convert it as MARC 21 or
UNIMARC.</p><div class="note" style="margin-left: 0.5in; margin-right:
0.5in;"><h3 class="title">Note</h3><p>Before doing this last step, please study
Stephen Hedges' migration guide here: <a
href="http://www.skemotah.com/Migrating_to_Koha.html";
target="_top">http://www.skemotah.com/Migrating_to_Koha.html</a></p></div><pre
class="programlisting">package cmftf;
## Create MARC From Text File
## Copyright 2004 Baiju M <address@hidden>
## This program is licensed under GNU GPL.
use strict;
use MARC::Record;
my $input_file = "../outs/docs.txt";
my $output_file = "../outs/docs.out";
my $repeated_items = " ";
open(INFILE, $input_file) or die "Can't open $input_file: $!";
open(OUTFILE, ">>", $output_file) or die "Can't open $output_file: $!";
my $c = 0;
while (<INFILE>) {
$c++;
my $biblionumber_ftf_691_090_c, my $title_ftf_4_245_a, my
$edition_ftf_7_250_a, my $publisher_ftf_11_260_b, my
$publishcountry_ftf_12_260_a, my $publishyear_ftf_13_260_c, my
$author1ee_ftf_18_100_a, my $author1se_ftf_19_100_a, my
$author2ee_ftf_20_700_a, my $author2se_ftf_21_700_a, my
$adiauthors_ftf_24_700_a, my $role_ftf_38_590_a, my $phydescr_ftf_10_300_a, my
$classno_ftf_114_852_k, my $classno_ftf_115_852_h, my $keyword_ftf_5_520_a, my
$voldetails_ftf_110_440_v, my $dateofentry_ftf_122_952_v, my
$notes_ftf_97_500_a, my $pubprice_151_952_r, my $currency_153_590_b, my
$reccreated_123_508_a, my $remainder;
($biblionumber_ftf_691_090_c, $title_ftf_4_245_a, $edition_ftf_7_250_a,
$publisher_ftf_11_260_b, $publishcountry_ftf_12_260_a,
$publishyear_ftf_13_260_c, $author1ee_ftf_18_100_a, $author1se_ftf_19_100_a,
$author2ee_ftf_20_700_a, $author2se_ftf_21_700_a, $adiauthors_ftf_24_700_a,
$role_ftf_38_590_a, $phydescr_ftf_10_300_a, $classno_ftf_114_852_k,
$classno_ftf_115_852_h, $keyword_ftf_5_520_a, $voldetails_ftf_110_440_v,
$dateofentry_ftf_122_952_v, $notes_ftf_97_500_a, $pubprice_151_952_r,
$currency_153_590_b, $reccreated_123_508_a, $remainder) = split(/\t/, $_, 23);
if (($biblionumber_ftf_691_090_c !~ /[^\d]/) and ($title_ftf_4_245_a !~
/_B_L_A_N_K_/) and ($repeated_items !~ /$biblionumber_ftf_691_090_c/) ) {
$repeated_items .= " $biblionumber_ftf_691_090_c";
my @biblionumber_array;
open(INF, $input_file) or die "Can't open $input_file: $!";
my $i = 1;
while (<INF>) {
$i++;
if ($i >= $c) {
my $biblionumber_ftf_691_090_c2, my $title_ftf_4_245_a2, my
$edition_ftf_7_250_a2, my $publisher_ftf_11_260_b2, my
$publishcountry_ftf_12_260_a2, my $publishyear_ftf_13_260_c2, my
$author1ee_ftf_18_100_a2, my $author1se_ftf_19_100_a2, my
$author2ee_ftf_20_700_a2, my $author2se_ftf_21_700_a2, my
$adiauthors_ftf_24_700_a2, my $role_ftf_38_590_a2, my $phydescr_ftf_10_300_a2,
my $classno_ftf_114_852_k2, my $classno_ftf_115_852_h2, my
$keyword_ftf_5_520_a2, my $voldetails_ftf_110_440_v2, my
$dateofentry_ftf_122_952_v2, my $notes_ftf_97_500_a2, my $pubprice_151_952_r2,
my $currency_153_590_b2, my $reccreated_123_508_a2, my $remainder2;
($biblionumber_ftf_691_090_c2, $title_ftf_4_245_a2,
$edition_ftf_7_250_a2, $publisher_ftf_11_260_b2, $publishcountry_ftf_12_260_a2,
$publishyear_ftf_13_260_c2, $author1ee_ftf_18_100_a2, $author1se_ftf_19_100_a2,
$author2ee_ftf_20_700_a2, $author2se_ftf_21_700_a2, $adiauthors_ftf_24_700_a2,
$role_ftf_38_590_a2, $phydescr_ftf_10_300_a2, $classno_ftf_114_852_k2,
$classno_ftf_115_852_h2, $keyword_ftf_5_520_a2, $voldetails_ftf_110_440_v2,
$dateofentry_ftf_122_952_v2, $notes_ftf_97_500_a2, $pubprice_151_952_r2,
$currency_153_590_b2, $reccreated_123_508_a2, $remainder2) = split(/\t/, $_,
23);
if (($biblionumber_ftf_691_090_c2 !~ /[^\d]/) and
($title_ftf_4_245_a2 !~ /_B_L_A_N_K_/) and ($title_ftf_4_245_a2 eq
$title_ftf_4_245_a) and ($author1ee_ftf_18_100_a2 eq $author1ee_ftf_18_100_a)){
print
"$biblionumber_ftf_691_090_c:$biblionumber_ftf_691_090_c2\t";
$repeated_items .= " $biblionumber_ftf_691_090_c2";
push @biblionumber_array, $biblionumber_ftf_691_090_c2;
}
}
}
print "\n";
close(INF);
my $record = MARC::Record->new();
if ($author1ee_ftf_18_100_a =~ /_B_L_A_N_K_/) {
$author1ee_ftf_18_100_a = "";
}
if ($author1se_ftf_19_100_a =~ /_B_L_A_N_K_/) {
$author1se_ftf_19_100_a = "";
}
my $author1_ftf = join(" ", $author1se_ftf_19_100_a,
$author1ee_ftf_18_100_a);
if ($author1_ftf ne "") {
my $author1 = MARC::Field->new(
'100','1','',
a => $author1_ftf
);
$record->add_fields($author1);
}
if ($title_ftf_4_245_a !~ /_B_L_A_N_K_/) {
my $title = MARC::Field->new(
'245','1','2',
a => $title_ftf_4_245_a
);
$record->add_fields($title);
}
if ($edition_ftf_7_250_a !~ /_B_L_A_N_K_/) {
my $edition = MARC::Field->new(
'250','','',
a => $edition_ftf_7_250_a
);
$record->add_fields($edition);
}
if ($publishcountry_ftf_12_260_a !~ /_B_L_A_N_K_/) {
my $publishcountry = MARC::Field->new(
'260','','',
a =>
$publishcountry_ftf_12_260_a
);
$record->add_fields($publishcountry);
}
if ($publisher_ftf_11_260_b =~ /_B_L_A_N_K_/) {
$publisher_ftf_11_260_b = "";
}
if ($publishyear_ftf_13_260_c =~ /_B_L_A_N_K_/) {
$publishyear_ftf_13_260_c = "";
}
my $publisher = MARC::Field->new(
'260','','',
b => $publisher_ftf_11_260_b,
c => $publishyear_ftf_13_260_c
);
$record->add_fields($publisher);
if ($phydescr_ftf_10_300_a =~ /_B_L_A_N_K_/) {
$phydescr_ftf_10_300_a = "";
}
my $phydescr = MARC::Field->new(
'300','','',
a => $phydescr_ftf_10_300_a,
f => 'BOOK'
);
$record->append_fields($phydescr);
if ($voldetails_ftf_110_440_v !~ /_B_L_A_N_K_/) {
my $voldetails = MARC::Field->new(
'440','','3',
v => $voldetails_ftf_110_440_v
);
$record->add_fields($voldetails);
}
if ($notes_ftf_97_500_a !~ /_B_L_A_N_K_/) {
my $notes = MARC::Field->new(
'500','','',
a => $notes_ftf_97_500_a
);
$record->add_fields($notes);
}
if ($reccreated_123_508_a !~ /_B_L_A_N_K_/) {
my $reccreated = MARC::Field->new(
'508','','',
a => $reccreated_123_508_a
);
$record->add_fields($reccreated);
}
if ($keyword_ftf_5_520_a !~ /_B_L_A_N_K_/) {
my $keyword = MARC::Field->new(
'520','','',
a => $keyword_ftf_5_520_a
);
$record->add_fields($keyword);
}
if ($role_ftf_38_590_a =~ /_B_L_A_N_K_/) {
$role_ftf_38_590_a = '';
}
if ($currency_153_590_b =~ /_B_L_A_N_K_/) {
$currency_153_590_b = '';
}
my $role = MARC::Field->new(
'590','','',
a => $role_ftf_38_590_a,
b => $currency_153_590_b
);
$record->add_fields($role);
if ($author2ee_ftf_20_700_a =~ /_B_L_A_N_K_/) {
$author2ee_ftf_20_700_a = "";
}
if ($author2se_ftf_21_700_a =~ /_B_L_A_N_K_/) {
$author2se_ftf_21_700_a = "";
}
if ($adiauthors_ftf_24_700_a =~ /_B_L_A_N_K_/) {
$adiauthors_ftf_24_700_a = "";
}
my $adiauthors_ftf = join(" ", $author2se_ftf_21_700_a,
$author2ee_ftf_20_700_a, $adiauthors_ftf_24_700_a);
if ($adiauthors_ftf ne "") {
my $adiauthors = MARC::Field->new(
'700','1','',
a => $adiauthors_ftf
);
$record->add_fields($adiauthors);
}
if ($classno_ftf_114_852_k =~ /_B_L_A_N_K_/) {
if ($classno_ftf_115_852_h =~ /_B_L_A_N_K_/) {
my $classno = MARC::Field->new(
'852','','',
k => '999.9999'
);
$record->add_fields($classno);
}
else {
if ($classno_ftf_115_852_h =~ /^\d\d\d/) {
my $classno = MARC::Field->new(
'852','','',
k =>
$classno_ftf_115_852_h
);
$record->add_fields($classno);
}
else {
my $classno = MARC::Field->new(
'852','','',
k => '999.9998',
h =>
$classno_ftf_115_852_h
);
$record->add_fields($classno);
}
}
}
else {
if ($classno_ftf_114_852_k =~ /^\d\d\d/) {
if ($classno_ftf_115_852_h =~ /_B_L_A_N_K_/) {
$classno_ftf_115_852_h = "";
}
my $classno = MARC::Field->new(
'852','','',
k => $classno_ftf_114_852_k,
h => $classno_ftf_115_852_h
);
$record->add_fields($classno);
}
else {
if ($classno_ftf_115_852_h =~ /^\d\d\d/) {
my $classno = MARC::Field->new(
'852','','',
k =>
$classno_ftf_115_852_h,
h =>
$classno_ftf_114_852_k
);
$record->add_fields($classno);
}
else {
if ($classno_ftf_115_852_h =~ /_B_L_A_N_K_/) {
$classno_ftf_115_852_h = "";
}
my $classno = MARC::Field->new(
'852','','',
k => '999.9997',
h =>
$classno_ftf_115_852_h
);
$record->add_fields($classno);
}
}
}
if ($pubprice_151_952_r =~ /_B_L_A_N_K_/) {
$pubprice_151_952_r = '';
}
if ($dateofentry_ftf_122_952_v =~ /_B_L_A_N_K_/) {
$dateofentry_ftf_122_952_v = '';
}
my @biblionumber_array_tmp = sort @biblionumber_array;
foreach (@biblionumber_array_tmp) {
my $biblionumber = $_;
my $barcode = MARC::Field->new(
'952','','',
b => "MAIN",
d => "MAIN",
p => $biblionumber,
r => $pubprice_151_952_r,
u => $biblionumber,
v => $dateofentry_ftf_122_952_v
);
$record->add_fields($barcode);
}
#print "\n";
print OUTFILE $record->as_usmarc();
}
else {
#print $_;
}
}
close(OUTFILE);</pre></li><li><p>And finally use koha's <tt
class="filename">bulkmarcimport.pl</tt> script to convert this to
SQL.</p></li></ul></div></div><div class="section" lang="en"><div
class="titlepage"><div><div><h2 class="title" style="clear: both"><a
name="d0e130"></a>5. Migrating Member
Data</h2></div></div><div></div></div><div class="itemizedlist"><ul
type="disc"><li><p>To convert member details to Koha, first I exported member
database into XML, then wrote a python script to convert it to tab separated
text file:</p><pre class="programlisting">##cimdtf.py : Convert ISIS Member
Database to Text File
##Before Parsing :
##Change & to and
##fix problems of <Tag_*> manualy.
##tag = ['Tag_201', 'Tag_202', 'Tag_203', 'Tag_216', 'Tag_217']
##Tag_201 : Name -> firstname
##Tag_202 : Membership number -> cardnumber
##Tag_203 : Division -> streetaddress
##Tag_216 : Status (UG,PG, etc.) -> categorycode
##Tag_217 : Joining date -> dateenrolled
## CREATE TABLE borrowers (
##01 borrowernumber int(11) NOT NULL auto_increment,
##02 cardnumber varchar(16) NOT NULL default '', ##Tag_202
##03 surname text NOT NULL, ##Tag_201
##04 firstname text NOT NULL, ##' '
##05 title text, ##' '
##06 othernames text, ##''
##07 initials text NOT NULL, ##''
##08 streetaddress text NOT NULL, ##Tag_203
##09 suburb text, ##NULL
##10 city text NOT NULL, ##Tag_203 (Use this?)
##11 phone text NOT NULL, ##''
##12 emailaddress text, ##''
##13 faxnumber text, ##''
##14 textmessaging text, ##''
##15 altstreetaddress text, ##NULL
##16 altsuburb text, ##NULL
##17 altcity text, ##NULL
##18 altphone text, ##''
##19 dateofbirth date default NULL, ##'0000-00-00'
##20 branchcode varchar(4) NOT NULL default '', ##'MAIN'
##21 categorycode char(2) default NULL, ##Tag_216
##22 dateenrolled date default NULL, ##Tag_217
##23 gonenoaddress tinyint(1) default NULL, ##NULL
##24 lost tinyint(1) default NULL, ##NULL
##25 debarred tinyint(1) default NULL, ##NULL
##26 studentnumber text, ##NULL
##27 school text, #NULL
##28 contactname text, ##' '
##29 borrowernotes text, ##''
##30 guarantor int(11) default NULL, ##NULL
##31 area char(2) default NULL, ##'' or NULL
##32 ethnicity varchar(50) default NULL, ##'' or NULL
##33 ethnotes varchar(255) default NULL, ##'' or NULL
##34 sex char(1) default NULL, ##NULL
##35 expiry date default NULL, ##'0000-00-00'
##36 altnotes varchar(255) default NULL, ##''
##37 altrelationship varchar(100) default NULL, ##'workplace'
##38 streetcity text, ##''
##39 phoneday varchar(50) default NULL, ##''
##40 preferredcont char(1) default NULL, ##NULL
##41 physstreet varchar(100) default NULL, ##''
##42 password varchar(30) default NULL, ##NULL
##43 flags int(11) default NULL, ##NULL
##44 userid varchar(30) default NULL, ##NULL
##45 homezipcode varchar(25) default NULL, ##''
##46 zipcode varchar(25) default NULL, ##''
##47 UNIQUE KEY cardnumber (cardnumber),
##48 KEY borrowernumber (borrowernumber)
## ) ;
import libxml2
record = []
def Record(i, node):
sub_node = node.children
while sub_node is not None:
if sub_node.type == "element":
record[i][sub_node.name] = sub_node.get_content()
sub_node = sub_node.next
else:
sub_node = sub_node.next
## Memory debug specific
libxml2.debugMemory(1)
isis_data = libxml2.parseFile ('../xmls/members.xml')
root = isis_data.getRootElement()
node = root.children
i = 0
while node is not None:
if node.type != "element":
node = node.next
continue
if node is None:
break
if node.name == "RECORD":
##print node.get_content()
record.append({})
Record(i, node)
i = i + 1
node = node.next
else:
print "unhandled node in <isis_xml>: " + node.name
isis_data.freeDoc()
# Memory debug specific
libxml2.cleanupParser()
if libxml2.debugMemory(1) == 0:
print "OK"
else:
print "Memory leak %d bytes" % (libxml2.debugMemory(1))
libxml2.dumpMemory()
####################################################################
output_file = open('../outs/members.txt', 'w')
tag = ['Tag_202', 'Tag_203', 'Tag_216', 'Tag_217']
for j in record:
if j.has_key('Tag_202') and j.has_key('Tag_201') and j.has_key('Tag_216'):
output_file.write('\t') ##01
output_file.write(j['Tag_202']+'\t') ##02
output_file.write(j['Tag_201']+'\t') ##03
output_file.write(' \t') ##04
output_file.write(' \t') ##05
output_file.write('\t') ##06
output_file.write('\t') ##07
if j.has_key('Tag_203'): ##08
output_file.write(j['Tag_203']+'\t')
else:
output_file.write('\t')
output_file.write('NULL\t') ##09
output_file.write('\t') ##10
output_file.write('\t') ##11
output_file.write('\t') ##12
output_file.write('\t') ##13
output_file.write('\t') ##14
output_file.write('NULL\t') ##15
output_file.write('NULL\t') ##16
output_file.write('NULL\t') ##17
output_file.write('\t') ##18
output_file.write('0000-00-00\t') ##19
output_file.write('MAIN\t') ##20
if j.has_key('Tag_216'): ##21
output_file.write(j['Tag_216']+'\t')
else:
output_file.write('OT\t')
pass
if j.has_key('Tag_217'): ##22
output_file.write(j['Tag_217']+'\t')
else:
output_file.write('0000-00-00\t')
pass
output_file.write('NULL\t') ##23
output_file.write('NULL\t') ##24
output_file.write('NULL\t') ##25
output_file.write('NULL\t') ##26
output_file.write('NULL\t') ##27
output_file.write(' \t') ##28
output_file.write('\t') ##29
output_file.write('NULL\t') ##30
output_file.write('NULL\t') ##31
output_file.write('NULL\t') ##32
output_file.write('NULL\t') ##33
output_file.write('M\t') ##34
output_file.write('0000-00-00\t') ##35
output_file.write('NULL\t') ##36
output_file.write('workplace\t') ##37
output_file.write('NULL\t') ##38
output_file.write('NULL\t') ##39
output_file.write('NULL\t') ##40
output_file.write('NULL\t') ##41
output_file.write('NULL\t') ##42
output_file.write('NULL\t') ##43
output_file.write('NULL\t') ##44
output_file.write('NULL\t') ##45
output_file.write('NULL\t') ##46
output_file.write('\n')</pre></li><li><p>Later to load it into
MySQL:</p><p><b class="userinput"><tt>LOAD DATA LOCAL INFILE "members.txt" INTO
TABLE borrowers;</tt></b></p></li></ul></div></div></div></body></html>
--- NEW FILE ---
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd";>
<article>
<title>Migrating a CDS/ISIS based system to Koha</title>
<articleinfo>
<pubdate>2004-11-16</pubdate>
<author>
<firstname>Baiju</firstname>
<surname>M</surname>
<email>baijum81 AT hailmail.net</email>
</author>
<revhistory>
<revision>
<revnumber>2.0.0p1</revnumber>
<date>2004-11-15</date>
<authorinitials>bm</authorinitials>
<revdescription>
<para>Small additions by BM.</para>
</revdescription>
</revision>
<revision>
<revnumber>2.0.0</revnumber>
<date>2004-04-01</date>
<authorinitials>bm</authorinitials>
<revdescription>
<para>Initial XML markup (by Stephen Hedges) of a wiki page posted
at <ulink
url="http://www.saas.nsw.edu.au/koha_wiki/index.php?page=CDS%2FISIS+based+system+to+Koha";>http://www.saas.nsw.edu.au/koha_wiki/index.php?page=CDS%2FISIS+based+system+to+Koha</ulink>
on 2004-04-01, with links to program listings expanded.</para>
</revdescription>
</revision>
</revhistory>
</articleinfo>
<authorblurb>
<para>By Baiju M</para>
</authorblurb>
<section>
<title id="copyright">Copyright and License</title>
<para>Copyright 2004 Baiju M <email>baijum81 AT
hailmail.net</email></para>
<para>This document is related to Koha and is licensed to you under the
GNU General Public License version 2 or later (<ulink
url="http://www.gnu.org/licenses/gpl.html";>http://www.gnu.org/licenses/gpl.html</ulink>).</para>
<para>Koha-related documents may be reproduced and distributed in whole or
in part, in any medium physical or electronic, as long as this copyright
notice is retained on all copies.</para>
<para>You may create a derivative work and distribute it provided that
you:</para>
<orderedlist>
<listitem>
<para>License the derivative work with this same license, or the Linux
Documentation Project License (<ulink
url="???">http://www.tldp.org/COPYRIGHT.html</ulink>). Include a
copyright notice and at least a pointer to the license used.</para>
</listitem>
<listitem>
<para>Give due credit to previous authors and major
contributors.</para>
</listitem>
</orderedlist>
<para>Commercial redistribution is allowed and encouraged; however, the
author would like to be notified of any such distributions.</para>
</section>
<section>
<title id="disclaimer">Disclaimer</title>
<para>No liability for the contents of this document can be accepted. Use
the concepts, examples and information at your own risk. There may be
errors and inaccuracies that could be damaging to your system. Although
this is highly unlikely, proceed with caution; the author(s) do not take
any responsibility.</para>
<para>All copyrights are held by their respective owners, unless
specifically noted otherwise. Use of a term in this document should not be
regarded as affecting the validity of any trademark or service mark.
Naming of particular products or brands should not be seen as
endorsements.</para>
</section>
<section>
<title>Introduction</title>
<para>I am doing this migration from CDS/ISIS (version : 1.5) system to
Koha for St. Joseph's College, Devagiri, Calicut. <ulink
url="http://devagiricollege.org/";>http://devagiricollege.org/</ulink></para>
<para>In the current system it is possible to do circulation and other
basic functionalities of the library.</para>
<para>The work is in progress, so give any pointers if you have.</para>
</section>
<section>
<title>Migrating Book Data</title>
<itemizedlist>
<listitem>
<para>First I exported Books data into XML using CDS/ISIS export tool
(available in the ISIS Utils menu).</para>
<note>
<para>Users of previous CDS/ISIS versions (i.e, 1.3 or before),
export documents data from CDS/ISIS and save it as Common
Communication Format (CCF), which is a standard based on ISO 2709.
Import this ISO data into CDS/ISIS version 1.4 or later. This step
is necessary to export the data as XML, the old versions of ISIS do
not have support for exporting document data as XML directly.
<remark>FIXME: ISIS 1.3.x support exporting XML?</remark></para>
</note>
</listitem>
<listitem>
<para>Then I wrote a python script (shown below) to convert XML to tab
delimited text format.<note>
<para>This step is not necessary, but you can do some corrections
in data quickly.</para>
</note><tip>
<para>Use Emacs macro or Vim recording for quick parsing.</para>
</tip><programlisting>## ciddtf.py : Convert ISIS Document Database
to Text File
##Before Parsing :
##Change RECORDX\xc3\x91_5\xc3\xaf\x03\x12 to RECORD
##Change & to and
##fix problems of <Tag_97> manualy.
import libxml2
record = []
def Record(i, node):
sub_node = node.children
while sub_node is not None:
if sub_node.type == "element":
record[i][sub_node.name] = sub_node.get_content()
sub_node = sub_node.next
else:
sub_node = sub_node.next
# Memory debug specific
libxml2.debugMemory(1)
isis_data = libxml2.parseFile ('../xmls/docs.xml')
root = isis_data.getRootElement()
node = root.children
i = 0
while node is not None:
if node.type != "element":
node = node.next
continue
if node is None:
break
if node.name == "RECORD":
##print node.get_content()
record.append({})
Record(i, node)
i = i + 1
node = node.next
else:
print "unhandled node in <isis_xml>: " + node.name
isis_data.freeDoc()
# Memory debug specific
libxml2.cleanupParser()
if libxml2.debugMemory(1) == 0:
print "OK"
else:
print "Memory leak %d bytes" % (libxml2.debugMemory(1))
libxml2.dumpMemory()
####################################################################
record_file = open('../outs/docs.txt', 'w')
tag = ['Tag_4', 'Tag_7', 'Tag_11', 'Tag_12', 'Tag_13', 'Tag_18', 'Tag_19',
'Tag_20', 'Tag_21', 'Tag_24', 'Tag_38', 'Tag_10', 'Tag_114', 'Tag_115',
'Tag_110', 'Tag_5', 'Tag_122', 'Tag_97', 'Tag_151', 'Tag_153', 'Tag_123']
for j in record:
if j.has_key('Tag_691'):
record_file.write(j['Tag_691'])
##Check docs without 114 tag
## if j.has_key('Tag_114'):
## pass
## else:
## print j['Tag_691']
else:
record_file.write('_B_L_A_N_K_')
for h in tag:
if j.has_key(h):
record_file.write('\t'+j[h])
else:
record_file.write('\t_B_L_A_N_K_')
record_file.write('\n')</programlisting></para>
<para>Here is a sample XML:<literallayout><?xml version="1.0"
encoding="ISO-8859-1"?>
<!DOCTYPE DATABASE_SJCLDB SYSTEM "SJCLDB.dtd">
<DATABASE_SJCLDB>
<RECORD>
<Tag_691>43009</Tag_691>
<Tag_4>Chiriyute pithamahan: V K N Krutikalekkurichulla
paddhanangal</Tag_4>
<Tag_11>DC Books</Tag_11>
<Tag_12>Kottayam</Tag_12>
<Tag_13>1997</Tag_13>
<Tag_18>Roy</Tag_18>
<Tag_19>P C</Tag_19>
<Tag_38>Ed</Tag_38>
<Tag_10>134p</Tag_10>
<Tag_114>894.37 Roy</Tag_114>
<Tag_115>N7</Tag_115>
<Tag_110>GL-M3</Tag_110>
<Tag_122>2000-03-28</Tag_122>
<Tag_151>Rs.40.00</Tag_151>
<Tag_123>Tom</Tag_123>
</RECORD>
<RECORD>
<Tag_691>43018</Tag_691>
<Tag_4>Edanadan pattu</Tag_4>
<Tag_11>Current Books</Tag_11>
<Tag_12>Kottayam</Tag_12>
<Tag_13>1997</Tag_13>
<Tag_18>Sajitha</Tag_18>
<Tag_19>K R</Tag_19>
<Tag_38>Ed</Tag_38>
<Tag_10>160p</Tag_10>
<Tag_114>398.87 Saj</Tag_114>
<Tag_115>N7</Tag_115>
<Tag_110>GL-3</Tag_110>
<Tag_122>2000-03-28</Tag_122>
<Tag_151>Rs.50.00</Tag_151>
<Tag_123>Tom</Tag_123>
</RECORD>
<RECORD>
<Tag_691>38980</Tag_691>
<Tag_4>Plays</Tag_4>
<Tag_11>EWP</Tag_11>
<Tag_12>New York</Tag_12>
<Tag_13>1989</Tag_13>
<Tag_18>O'Neill</Tag_18>
<Tag_19>Eugene</Tag_19>
<Tag_114>O7,2 One</Tag_114>
<Tag_115>M9.1</Tag_115>
<Tag_122>1996-06-27</Tag_122>
</RECORD>
<RECORD>
<Tag_691>44579</Tag_691>
<Tag_4>Arangukaanatha Nadan</Tag_4>
<Tag_11>Current</Tag_11>
<Tag_12>Thrissur</Tag_12>
<Tag_13>1998</Tag_13>
<Tag_18>Thikkodian</Tag_18>
<Tag_10>396p</Tag_10>
<Tag_114>894.8 Thi/Ara</Tag_114>
<Tag_115>N8</Tag_115>
<Tag_110>GL-M8</Tag_110>
<Tag_122>2001-06-19</Tag_122>
<Tag_151>Rs.135.00</Tag_151>
<Tag_123>Tom</Tag_123>
</RECORD>
<RECORD>
<Tag_691>16469</Tag_691>
<Tag_4>Sartre: A collection of critical essays</Tag_4>
<Tag_11>Prentice Hall</Tag_11>
<Tag_12>USA</Tag_12>
<Tag_13>1965</Tag_13>
<Tag_18>Kern</Tag_18>
<Tag_19>Edith</Tag_19>
<Tag_38>Ed</Tag_38>
<Tag_10>180p</Tag_10>
<Tag_114>O-,4 M79</Tag_114>
<Tag_115>J5</Tag_115>
<Tag_110>Twentieth century views</Tag_110>
<Tag_122>1998-09-26</Tag_122>
</RECORD>
<RECORD>
<Tag_691>42068</Tag_691>
<Tag_4>Internet for beginners</Tag_4>
<Tag_11>Icon Books</Tag_11>
<Tag_12>Australia</Tag_12>
<Tag_13>1997</Tag_13>
<Tag_18>Brunner</Tag_18>
<Tag_19>laurel</Tag_19>
<Tag_20>Jevtic</Tag_20>
<Tag_21>Zoran</Tag_21>
<Tag_10>175p</Tag_10>
<Tag_115>N7</Tag_115>
<Tag_122>1998/09/26</Tag_122>
<Tag_123>CPYR</Tag_123>
</RECORD>
<RECORD>
<Tag_691>42367</Tag_691>
<Tag_4>Text book of Biochemistry with clinical correlations</Tag_4>
<Tag_7>Ed.4</Tag_7>
<Tag_11>Wiley-Liss/John Wiley and Sons</Tag_11>
<Tag_12>New York</Tag_12>
<Tag_13>1997</Tag_13>
<Tag_18>Devlin</Tag_18>
<Tag_19>Thomas M</Tag_19>
<Tag_38>Ed</Tag_38>
<Tag_10>1186p</Tag_10>
<Tag_114>612.015 Dev/KD</Tag_114>
<Tag_115>N7</Tag_115>
<Tag_110>Zoology</Tag_110>
<Tag_122>1998-10-08</Tag_122>
<Tag_97>$.30.95</Tag_97>
</RECORD>
</DATABASE_SJCLDB> </literallayout></para>
</listitem>
<listitem>
<para>Now parse the tab separated file using a perl script (shown
below) and convert it as MARC 21 or UNIMARC.<note>
<para>Before doing this last step, please study Stephen Hedges'
migration guide here: <ulink
url="http://www.skemotah.com/Migrating_to_Koha.html";>http://www.skemotah.com/Migrating_to_Koha.html</ulink></para>
</note><programlisting>package cmftf;
## Create MARC From Text File
## Copyright 2004 Baiju M <address@hidden>
## This program is licensed under GNU GPL.
use strict;
use MARC::Record;
my $input_file = "../outs/docs.txt";
my $output_file = "../outs/docs.out";
my $repeated_items = " ";
open(INFILE, $input_file) or die "Can't open $input_file: $!";
open(OUTFILE, ">>", $output_file) or die "Can't open $output_file: $!";
my $c = 0;
while (<INFILE>) {
$c++;
my $biblionumber_ftf_691_090_c, my $title_ftf_4_245_a, my
$edition_ftf_7_250_a, my $publisher_ftf_11_260_b, my
$publishcountry_ftf_12_260_a, my $publishyear_ftf_13_260_c, my
$author1ee_ftf_18_100_a, my $author1se_ftf_19_100_a, my
$author2ee_ftf_20_700_a, my $author2se_ftf_21_700_a, my
$adiauthors_ftf_24_700_a, my $role_ftf_38_590_a, my $phydescr_ftf_10_300_a, my
$classno_ftf_114_852_k, my $classno_ftf_115_852_h, my $keyword_ftf_5_520_a, my
$voldetails_ftf_110_440_v, my $dateofentry_ftf_122_952_v, my
$notes_ftf_97_500_a, my $pubprice_151_952_r, my $currency_153_590_b, my
$reccreated_123_508_a, my $remainder;
($biblionumber_ftf_691_090_c, $title_ftf_4_245_a, $edition_ftf_7_250_a,
$publisher_ftf_11_260_b, $publishcountry_ftf_12_260_a,
$publishyear_ftf_13_260_c, $author1ee_ftf_18_100_a, $author1se_ftf_19_100_a,
$author2ee_ftf_20_700_a, $author2se_ftf_21_700_a, $adiauthors_ftf_24_700_a,
$role_ftf_38_590_a, $phydescr_ftf_10_300_a, $classno_ftf_114_852_k,
$classno_ftf_115_852_h, $keyword_ftf_5_520_a, $voldetails_ftf_110_440_v,
$dateofentry_ftf_122_952_v, $notes_ftf_97_500_a, $pubprice_151_952_r,
$currency_153_590_b, $reccreated_123_508_a, $remainder) = split(/\t/, $_, 23);
if (($biblionumber_ftf_691_090_c !~ /[^\d]/) and ($title_ftf_4_245_a !~
/_B_L_A_N_K_/) and ($repeated_items !~ /$biblionumber_ftf_691_090_c/) ) {
$repeated_items .= " $biblionumber_ftf_691_090_c";
my @biblionumber_array;
open(INF, $input_file) or die "Can't open $input_file: $!";
my $i = 1;
while (<INF>) {
$i++;
if ($i >= $c) {
my $biblionumber_ftf_691_090_c2, my $title_ftf_4_245_a2, my
$edition_ftf_7_250_a2, my $publisher_ftf_11_260_b2, my
$publishcountry_ftf_12_260_a2, my $publishyear_ftf_13_260_c2, my
$author1ee_ftf_18_100_a2, my $author1se_ftf_19_100_a2, my
$author2ee_ftf_20_700_a2, my $author2se_ftf_21_700_a2, my
$adiauthors_ftf_24_700_a2, my $role_ftf_38_590_a2, my $phydescr_ftf_10_300_a2,
my $classno_ftf_114_852_k2, my $classno_ftf_115_852_h2, my
$keyword_ftf_5_520_a2, my $voldetails_ftf_110_440_v2, my
$dateofentry_ftf_122_952_v2, my $notes_ftf_97_500_a2, my $pubprice_151_952_r2,
my $currency_153_590_b2, my $reccreated_123_508_a2, my $remainder2;
($biblionumber_ftf_691_090_c2, $title_ftf_4_245_a2,
$edition_ftf_7_250_a2, $publisher_ftf_11_260_b2, $publishcountry_ftf_12_260_a2,
$publishyear_ftf_13_260_c2, $author1ee_ftf_18_100_a2, $author1se_ftf_19_100_a2,
$author2ee_ftf_20_700_a2, $author2se_ftf_21_700_a2, $adiauthors_ftf_24_700_a2,
$role_ftf_38_590_a2, $phydescr_ftf_10_300_a2, $classno_ftf_114_852_k2,
$classno_ftf_115_852_h2, $keyword_ftf_5_520_a2, $voldetails_ftf_110_440_v2,
$dateofentry_ftf_122_952_v2, $notes_ftf_97_500_a2, $pubprice_151_952_r2,
$currency_153_590_b2, $reccreated_123_508_a2, $remainder2) = split(/\t/, $_,
23);
if (($biblionumber_ftf_691_090_c2 !~ /[^\d]/) and
($title_ftf_4_245_a2 !~ /_B_L_A_N_K_/) and ($title_ftf_4_245_a2 eq
$title_ftf_4_245_a) and ($author1ee_ftf_18_100_a2 eq $author1ee_ftf_18_100_a)){
print
"$biblionumber_ftf_691_090_c:$biblionumber_ftf_691_090_c2\t";
$repeated_items .= " $biblionumber_ftf_691_090_c2";
push @biblionumber_array, $biblionumber_ftf_691_090_c2;
}
}
}
print "\n";
close(INF);
my $record = MARC::Record->new();
if ($author1ee_ftf_18_100_a =~ /_B_L_A_N_K_/) {
$author1ee_ftf_18_100_a = "";
}
if ($author1se_ftf_19_100_a =~ /_B_L_A_N_K_/) {
$author1se_ftf_19_100_a = "";
}
my $author1_ftf = join(" ", $author1se_ftf_19_100_a,
$author1ee_ftf_18_100_a);
if ($author1_ftf ne "") {
my $author1 = MARC::Field->new(
'100','1','',
a => $author1_ftf
);
$record->add_fields($author1);
}
if ($title_ftf_4_245_a !~ /_B_L_A_N_K_/) {
my $title = MARC::Field->new(
'245','1','2',
a => $title_ftf_4_245_a
);
$record->add_fields($title);
}
if ($edition_ftf_7_250_a !~ /_B_L_A_N_K_/) {
my $edition = MARC::Field->new(
'250','','',
a => $edition_ftf_7_250_a
);
$record->add_fields($edition);
}
if ($publishcountry_ftf_12_260_a !~ /_B_L_A_N_K_/) {
my $publishcountry = MARC::Field->new(
'260','','',
a =>
$publishcountry_ftf_12_260_a
);
$record->add_fields($publishcountry);
}
if ($publisher_ftf_11_260_b =~ /_B_L_A_N_K_/) {
$publisher_ftf_11_260_b = "";
}
if ($publishyear_ftf_13_260_c =~ /_B_L_A_N_K_/) {
$publishyear_ftf_13_260_c = "";
}
my $publisher = MARC::Field->new(
'260','','',
b => $publisher_ftf_11_260_b,
c => $publishyear_ftf_13_260_c
);
$record->add_fields($publisher);
if ($phydescr_ftf_10_300_a =~ /_B_L_A_N_K_/) {
$phydescr_ftf_10_300_a = "";
}
my $phydescr = MARC::Field->new(
'300','','',
a => $phydescr_ftf_10_300_a,
f => 'BOOK'
);
$record->append_fields($phydescr);
if ($voldetails_ftf_110_440_v !~ /_B_L_A_N_K_/) {
my $voldetails = MARC::Field->new(
'440','','3',
v => $voldetails_ftf_110_440_v
);
$record->add_fields($voldetails);
}
if ($notes_ftf_97_500_a !~ /_B_L_A_N_K_/) {
my $notes = MARC::Field->new(
'500','','',
a => $notes_ftf_97_500_a
);
$record->add_fields($notes);
}
if ($reccreated_123_508_a !~ /_B_L_A_N_K_/) {
my $reccreated = MARC::Field->new(
'508','','',
a => $reccreated_123_508_a
);
$record->add_fields($reccreated);
}
if ($keyword_ftf_5_520_a !~ /_B_L_A_N_K_/) {
my $keyword = MARC::Field->new(
'520','','',
a => $keyword_ftf_5_520_a
);
$record->add_fields($keyword);
}
if ($role_ftf_38_590_a =~ /_B_L_A_N_K_/) {
$role_ftf_38_590_a = '';
}
if ($currency_153_590_b =~ /_B_L_A_N_K_/) {
$currency_153_590_b = '';
}
my $role = MARC::Field->new(
'590','','',
a => $role_ftf_38_590_a,
b => $currency_153_590_b
);
$record->add_fields($role);
if ($author2ee_ftf_20_700_a =~ /_B_L_A_N_K_/) {
$author2ee_ftf_20_700_a = "";
}
if ($author2se_ftf_21_700_a =~ /_B_L_A_N_K_/) {
$author2se_ftf_21_700_a = "";
}
if ($adiauthors_ftf_24_700_a =~ /_B_L_A_N_K_/) {
$adiauthors_ftf_24_700_a = "";
}
my $adiauthors_ftf = join(" ", $author2se_ftf_21_700_a,
$author2ee_ftf_20_700_a, $adiauthors_ftf_24_700_a);
if ($adiauthors_ftf ne "") {
my $adiauthors = MARC::Field->new(
'700','1','',
a => $adiauthors_ftf
);
$record->add_fields($adiauthors);
}
if ($classno_ftf_114_852_k =~ /_B_L_A_N_K_/) {
if ($classno_ftf_115_852_h =~ /_B_L_A_N_K_/) {
my $classno = MARC::Field->new(
'852','','',
k => '999.9999'
);
$record->add_fields($classno);
}
else {
if ($classno_ftf_115_852_h =~ /^\d\d\d/) {
my $classno = MARC::Field->new(
'852','','',
k =>
$classno_ftf_115_852_h
);
$record->add_fields($classno);
}
else {
my $classno = MARC::Field->new(
'852','','',
k => '999.9998',
h =>
$classno_ftf_115_852_h
);
$record->add_fields($classno);
}
}
}
else {
if ($classno_ftf_114_852_k =~ /^\d\d\d/) {
if ($classno_ftf_115_852_h =~ /_B_L_A_N_K_/) {
$classno_ftf_115_852_h = "";
}
my $classno = MARC::Field->new(
'852','','',
k => $classno_ftf_114_852_k,
h => $classno_ftf_115_852_h
);
$record->add_fields($classno);
}
else {
if ($classno_ftf_115_852_h =~ /^\d\d\d/) {
my $classno = MARC::Field->new(
'852','','',
k =>
$classno_ftf_115_852_h,
h =>
$classno_ftf_114_852_k
);
$record->add_fields($classno);
}
else {
if ($classno_ftf_115_852_h =~ /_B_L_A_N_K_/) {
$classno_ftf_115_852_h = "";
}
my $classno = MARC::Field->new(
'852','','',
k => '999.9997',
h =>
$classno_ftf_115_852_h
);
$record->add_fields($classno);
}
}
}
if ($pubprice_151_952_r =~ /_B_L_A_N_K_/) {
$pubprice_151_952_r = '';
}
if ($dateofentry_ftf_122_952_v =~ /_B_L_A_N_K_/) {
$dateofentry_ftf_122_952_v = '';
}
my @biblionumber_array_tmp = sort @biblionumber_array;
foreach (@biblionumber_array_tmp) {
my $biblionumber = $_;
my $barcode = MARC::Field->new(
'952','','',
b => "MAIN",
d => "MAIN",
p => $biblionumber,
r => $pubprice_151_952_r,
u => $biblionumber,
v => $dateofentry_ftf_122_952_v
);
$record->add_fields($barcode);
}
#print "\n";
print OUTFILE $record->as_usmarc();
}
else {
#print $_;
}
}
close(OUTFILE);</programlisting></para>
</listitem>
<listitem>
<para>And finally use koha's <filename>bulkmarcimport.pl</filename>
script to convert this to SQL.</para>
</listitem>
</itemizedlist>
</section>
<section>
<title>Migrating Member Data</title>
<itemizedlist>
<listitem>
<para>To convert member details to Koha, first I exported member
database into XML, then wrote a python script to convert it to tab
separated text file:<programlisting>##cimdtf.py : Convert ISIS Member
Database to Text File
##Before Parsing :
##Change & to and
##fix problems of <Tag_*> manualy.
##tag = ['Tag_201', 'Tag_202', 'Tag_203', 'Tag_216', 'Tag_217']
##Tag_201 : Name -> firstname
##Tag_202 : Membership number -> cardnumber
##Tag_203 : Division -> streetaddress
##Tag_216 : Status (UG,PG, etc.) -> categorycode
##Tag_217 : Joining date -> dateenrolled
## CREATE TABLE borrowers (
##01 borrowernumber int(11) NOT NULL auto_increment,
##02 cardnumber varchar(16) NOT NULL default '', ##Tag_202
##03 surname text NOT NULL, ##Tag_201
##04 firstname text NOT NULL, ##' '
##05 title text, ##' '
##06 othernames text, ##''
##07 initials text NOT NULL, ##''
##08 streetaddress text NOT NULL, ##Tag_203
##09 suburb text, ##NULL
##10 city text NOT NULL, ##Tag_203 (Use this?)
##11 phone text NOT NULL, ##''
##12 emailaddress text, ##''
##13 faxnumber text, ##''
##14 textmessaging text, ##''
##15 altstreetaddress text, ##NULL
##16 altsuburb text, ##NULL
##17 altcity text, ##NULL
##18 altphone text, ##''
##19 dateofbirth date default NULL, ##'0000-00-00'
##20 branchcode varchar(4) NOT NULL default '', ##'MAIN'
##21 categorycode char(2) default NULL, ##Tag_216
##22 dateenrolled date default NULL, ##Tag_217
##23 gonenoaddress tinyint(1) default NULL, ##NULL
##24 lost tinyint(1) default NULL, ##NULL
##25 debarred tinyint(1) default NULL, ##NULL
##26 studentnumber text, ##NULL
##27 school text, #NULL
##28 contactname text, ##' '
##29 borrowernotes text, ##''
##30 guarantor int(11) default NULL, ##NULL
##31 area char(2) default NULL, ##'' or NULL
##32 ethnicity varchar(50) default NULL, ##'' or NULL
##33 ethnotes varchar(255) default NULL, ##'' or NULL
##34 sex char(1) default NULL, ##NULL
##35 expiry date default NULL, ##'0000-00-00'
##36 altnotes varchar(255) default NULL, ##''
##37 altrelationship varchar(100) default NULL, ##'workplace'
##38 streetcity text, ##''
##39 phoneday varchar(50) default NULL, ##''
##40 preferredcont char(1) default NULL, ##NULL
##41 physstreet varchar(100) default NULL, ##''
##42 password varchar(30) default NULL, ##NULL
##43 flags int(11) default NULL, ##NULL
##44 userid varchar(30) default NULL, ##NULL
##45 homezipcode varchar(25) default NULL, ##''
##46 zipcode varchar(25) default NULL, ##''
##47 UNIQUE KEY cardnumber (cardnumber),
##48 KEY borrowernumber (borrowernumber)
## ) ;
import libxml2
record = []
def Record(i, node):
sub_node = node.children
while sub_node is not None:
if sub_node.type == "element":
record[i][sub_node.name] = sub_node.get_content()
sub_node = sub_node.next
else:
sub_node = sub_node.next
## Memory debug specific
libxml2.debugMemory(1)
isis_data = libxml2.parseFile ('../xmls/members.xml')
root = isis_data.getRootElement()
node = root.children
i = 0
while node is not None:
if node.type != "element":
node = node.next
continue
if node is None:
break
if node.name == "RECORD":
##print node.get_content()
record.append({})
Record(i, node)
i = i + 1
node = node.next
else:
print "unhandled node in <isis_xml>: " + node.name
isis_data.freeDoc()
# Memory debug specific
libxml2.cleanupParser()
if libxml2.debugMemory(1) == 0:
print "OK"
else:
print "Memory leak %d bytes" % (libxml2.debugMemory(1))
libxml2.dumpMemory()
####################################################################
output_file = open('../outs/members.txt', 'w')
tag = ['Tag_202', 'Tag_203', 'Tag_216', 'Tag_217']
for j in record:
if j.has_key('Tag_202') and j.has_key('Tag_201') and j.has_key('Tag_216'):
output_file.write('\t') ##01
output_file.write(j['Tag_202']+'\t') ##02
output_file.write(j['Tag_201']+'\t') ##03
output_file.write(' \t') ##04
output_file.write(' \t') ##05
output_file.write('\t') ##06
output_file.write('\t') ##07
if j.has_key('Tag_203'): ##08
output_file.write(j['Tag_203']+'\t')
else:
output_file.write('\t')
output_file.write('NULL\t') ##09
output_file.write('\t') ##10
output_file.write('\t') ##11
output_file.write('\t') ##12
output_file.write('\t') ##13
output_file.write('\t') ##14
output_file.write('NULL\t') ##15
output_file.write('NULL\t') ##16
output_file.write('NULL\t') ##17
output_file.write('\t') ##18
output_file.write('0000-00-00\t') ##19
output_file.write('MAIN\t') ##20
if j.has_key('Tag_216'): ##21
output_file.write(j['Tag_216']+'\t')
else:
output_file.write('OT\t')
pass
if j.has_key('Tag_217'): ##22
output_file.write(j['Tag_217']+'\t')
else:
output_file.write('0000-00-00\t')
pass
output_file.write('NULL\t') ##23
output_file.write('NULL\t') ##24
output_file.write('NULL\t') ##25
output_file.write('NULL\t') ##26
output_file.write('NULL\t') ##27
output_file.write(' \t') ##28
output_file.write('\t') ##29
output_file.write('NULL\t') ##30
output_file.write('NULL\t') ##31
output_file.write('NULL\t') ##32
output_file.write('NULL\t') ##33
output_file.write('M\t') ##34
output_file.write('0000-00-00\t') ##35
output_file.write('NULL\t') ##36
output_file.write('workplace\t') ##37
output_file.write('NULL\t') ##38
output_file.write('NULL\t') ##39
output_file.write('NULL\t') ##40
output_file.write('NULL\t') ##41
output_file.write('NULL\t') ##42
output_file.write('NULL\t') ##43
output_file.write('NULL\t') ##44
output_file.write('NULL\t') ##45
output_file.write('NULL\t') ##46
output_file.write('\n')</programlisting></para>
</listitem>
<listitem>
<para>Later to load it into MySQL:</para>
<para><userinput>LOAD DATA LOCAL INFILE "members.txt" INTO TABLE
borrowers;</userinput></para>
</listitem>
</itemizedlist>
</section>
</article>
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- [Koha-cvs] CVS: koha-doc/en/sysadmin/migration CDS_ISIS_to_Koha.html,NONE,1.1 CDS_ISIS_to_Koha.xml,NONE,1.1,
skemotah <=