[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: pkg.m status and question

From: Philip Nienhuis
Subject: Re: pkg.m status and question
Date: Mon, 6 Jan 2020 23:44:24 +0100
User-agent: Mozilla/5.0 (Windows NT 6.1; rv:52.0) Gecko/20100101 Firefox/52.0

Mike Miller wrote:
On Mon, Jan 06, 2020 at 04:26:21 -0600, PhilipNienhuis wrote:
My motive for asking is that this setup of package info makes for needlessly
inefficient and complicated code. For each tiny action the relevant structs
have to be unwrapped first. IMO dealing with a struct array rather than cell
array of structs would make the code much more transparent and much more
efficient as well.

I agree with this motivation, having seen this before too. I can't speak
to why it was originally done as a cell array of scalar structs.

I do have a speculation. The current design reads each pkg DESCRIPTION
file into a scalar struct, which is then added to a cell array of all
pkgs. A problem arises if you have pkg A which has a "Url" field in its
DESCRIPTION, and pkg B which has a "SystemRequirements" field in its
DESCRIPTION. How do you easily combine two structs with unlike fields
into a single struct array with a superset of all fields?

By way of example,

    a.Name = "first";
    a.Description = "A package";
    a.Url = "https://www.example.org/";;
    b.Name = "second";
    b.Description = "Another package";
    b.SystemRequirements = "fftw3";
    all = [a, b];

doesn't work. But the current cell array implementation does.

Thanks Mike, that's one thing I figured (and mentioned as first reason). But I'm not so impressed. At my work we have complete input data sets for our groundwater models in nested Matlab structs (some up to 100's of MBytes on disk in compressed v7.3 format, in perhaps 5-10 nesting levels); such heterogeneity as you mention is vital and a feature we have quite a bit of experience with.

Probably too much detail now, but here goes:
We need a basic struct with all required fields that are of the same type for all packages. All fields that are heterogeneous (not required for each package and/or not used by the package manager) go in substructs or cell arrays in a dedicated field of the main struct. That way we move heterogeneity from the outside wrapper into a much less prominent location inside, out of the way of regular package manager operations. Upgrading an old package database to a new format can be done with a dedicated function; such a function can't be too complicated and as it runs just twice (global and local) doesn't need much optimization (read: developer time). Maybe that function sniffs what the database looks like and acts if it's the old cell type. Saving and reading the package database is otherwise unaffected. When installing a new package a check could be due for new fields in the "Description" substruct/cell array. Maybe it isn't even necessary.

But read on, another step is needed as well:

Carné rightly mentions backward compatibility. I surmise that is because the global octave_packages file by default lives in the share/octave/ directory, and as a consequence it is shared between octave releases (and theoretically between OS-es if it served to e.g., remote clients). (Even the local octave-packages file is/can be shared between Octave releases, but I ignore that for now.) IMO that is a wrong place to begin with as octave_packages has "hard" references to arch-dependent subdirs as well; each package in the package database has just one reference to an arch-dependent subdir. That implies that OF packages with binary modules cannot be shared reliably between Octave releases anyway (I've been bitten very often by that). Nor can packages be shared between different OS-es. It can be argued that even the arch-independent directory cannot be shared between Octave releases as .m-files can invoke functions or constructs that are new or dropped or renamed in different releases. All in all, IMO a somewhat better place for the package database would be in share/octave/<release>/ and the packages themselves as well.
Or am I wrong? please enlighten me then.

That said, I wonder how much practical situations exist where OF packages are shared between Octave releases on networks, or even clients with different OS-es.

My impression is that here's a quite a bit of legacy in the way, hindering simplification of package management.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]