Re: [Gnumed-bugs] GNUmed 1.5.0 - invalid [v19] schema structure

[Karsten Hilbert]

On Thu, Jan 15, 2015 at 05:42:26PM +0000, Jim Busser wrote:
> Date: Thu, 15 Jan 2015 17:42:26 +0000
> From: "Busser, Jim" <address@hidden>
> To: "address@hidden" <address@hidden>
> Subject: Re: [Gnumed-bugs] GNUmed 1.5.0 - invalid [v19] schema structure
> 
> On 2015-01-15, at 1:34 AM, Karsten Hilbert <address@hidden> wrote:
> 
> > So (in this case, because there _are_ fixups changing the hash
> > between your v19 version and the final v19):
> > 
> >     cd server/bootstrap/
> >     ./fixup-db.sh 19
> >     ./upgrade-db.sh 19 20
>       
> After doing (from server 19.8 tarball)
> 
>       cd /Users/djb/Downloads/gnumed-server.19.8/server/bootstrap
>       ./fixup-db.sh 19

That will not help any because, naturally, a 19.8 tarball
cannot contain fixes introduced by (and thereby able to
change the hash of) later server versions (say, 19.9 -
19.14)...

> I then tried from (from server 20.1 tarball)
> 
>       cd /Users/djb/Downloads/gnumed-server.20.1/server/bootstrap
>       ./upgrade-db.sh 19 20
> 
> but it still complained
> 
>       2015-01-15 08:30:07 DEBUG gm.db 
> (/Users/djb/Downloads/gnumed-server.20.1/Gnumed/pycommon/gmPG2.py::database_schema_compatible()
>  #588): 2015-01-15 07:47:01.503023-08 - 19.8 - 
> v19-find_potentially_misappropriated_soap-fixup.sql
>       2015-01-15 08:30:07 ERROR gm.bootstrapper 
> (./bootstrap_gm_db_system.py::__create_db() #771): invalid [v19] schema 
> structure in GNUmed template database [gnumed_v19]
>       2015-01-15 08:30:07 ERROR gm.bootstrapper 
> (./bootstrap_gm_db_system.py::__bootstrap() #599): Cannot create database.
>       2015-01-15 08:30:07 ERROR gm.bootstrapper 
> (./bootstrap_gm_db_system.py::bootstrap() #1309): Cannot bootstrap bundle 
> [v19_fixups-pre_v20]. 
> 
> and a fresh fingerprint of my v19 still gives the same fingerprint as before 
> I (re) ran the fixups contained with 19.8
> 
>       9765373098b03fb208332498f34cd4b5
> 
> but on a hunch, I reran the fixups from within the ***20.1*** tarball

That is possible, too, yes.

Alternatively, you could have downloaded the v19.latest
server and run the fixups contained in there which are 19.8
fixups + 19.9 fixups + 19.10 + 19.11 + ... you get the
picture. Each release adds fixups as needed and each release
can be run to get any previous-within-major database up to
the state-of-release. So, running fixups from 19.11 against
your 19.8 would have gotten you up to 19.11 (but not to
19.12, 13, or 14 and thus not out of the hash problem).

Say, assuming if you had run v19.12 fixups against your 19.8
database you'd have gotten up to 19.12 ...

> after which the fingerprint became
> 
>       57f009a159f55f77525cc0291e0c8b60

... after which you _would_ have been able to successfully
run the 19->20 upgrade (even though not being on 19.14)
because that upgrade would

a) find a valid 19.12 hash
b) re-run _all_ v19 fixups, including 19.13 and 19.14 (if any)

In short: If a database upgrade fails due to a schema hash
mismatch then the database-to-be-upgraded is either borked or
- in case a more invasive (= hash changing) fixup was needed
in the previous major revision - not "sufficiently" uptodate.

In that case it is always a good idea to manually run the
fixups for the latest-minor-of-to-be-upgraded version. That
can be done from either the latest previous-major tarball or
from the new-major tarball.

> which, on further perusal, is explained by the fact that after the release of
> 
>       ...gnumed-server.19.8/server/bootstrap/fixup_db-v19.conf
> 
> more fixups were added

Exactly.

> (presumably at 20.0,

No, at 19.12, 19.13, and/or 19.14 -- and then, of course,
forward-ported to v20 to be released with the next v20.x ...

> and therefore contained in 20.1)

as witnessed here:

>       ...gnumed-server.20.1/server/bootstrap/fixup_db-v19.conf
> 
> had these additional fixups
> 
>       v19-bill-v_bills-fixup.sql
>       v19-bill-bill_item-fixup.sql
>       v19-dem-org-idx-fixup.sql
>       v19-dem-v_orgs_v_org_units-fixup.sql
>       v19-dem-v_praxis_branches-fixup.sql

the last 2 of which arose from bug reports by yourself, BTW ;-/

> so, when running
> 
>       ./upgrade-db.sh x x+1
> 
> is it manageable for the script to "call" (run) the fixup

The fixups _are_ already applied by the upgrader from within
its conf file.

> contained within the same directory level

That is not an assumption the upgrader can make but I am
trying to think of ways to make this happen.

> or is it incumbent
> on us to each time ensure to run, from whichever tarball we
> wish to represent the new version, both
> 
>       ./fixup-db.sh x
>       ./upgrade-db.sh x x+1

No, only if the hash changed (which will be apparent from the
hash related failure documented in the log file).

Karsten
-- 
GPG key ID E4071346 @ eu.pool.sks-keyservers.net
E167 67FD A291 2BEA 73BD  4537 78B9 A9F9 E407 1346