Re: More progress with the Guix Data Service

[Amirouche]

On 2019-05-17 09:56, Christopher Baines wrote:
> Hey,
>
> In summary, I think the Guix Data Service might be getting useful 
> enough
> that setting it up properly might be a good next step, and I'd be
> interested in what others think?

Yeah. And it looks good :)

> A bit over a month ago, I sent out an update about one of the things
> I've been working on, something I've been calling the "Guix Data
> Service" [1].
>
> 1: https://lists.gnu.org/archive/html/guix-devel/2019-04/msg00094.html
>
> Since then, I've made some more progress. There's a new statistics page
> [2]. I've got used to using Sqitch [3] to help manage the database
> schema, and I've added some basic tests.

statistic page is a 404. I am very interested to learn how you use 
sqitch.

> 2: https://prototype-guix-data-service.cbaines.net/statistic
> 3: http://guix.gnu.org/packages/sqitch-0.9999/
>
> The error handling for loading new revisions is also more resilient 
> now.
>
> As well as listening to the Guix Commits mailing list for emails about
> new revisions, more of the information in these emails is now stored, 
> in
> particular, the time they were sent, and the branch the email applies
> to. This can be seen on the new Branches page [4].
>
> 4: https://prototype-guix-data-service.cbaines.net/branches

Here when I click on a commit, I would expect the message of the commit
and the build status of the impacted softwares, e.g.

https://prototype-guix-data-service.cbaines.net/revision/d59c90a5bfdd6b723bea939b8538c7c9b3c1b2a6

> The content negotiation a little bit, at least in terms of the code, 
> and
> JSON output support has been added to more pages.
>
> There's now a basic search function on the packages page [5], and the
> location, and the licenses for packages is now being stored (which can
> be seen on the page for a package, for example [6]).
>
> 5:
> https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/packages?search_query=Guile&field=version&field=synopsis&after_name=&limit_results=1000
> 6:
> https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/package/0ad/0.0.23b-alpha
>
> The location and license information is something I added specifically,
> as I noticed the Repology [7] service scraped [8] these from the Guix
> website.
>
> 7: https://repology.org/
> 8:
> https://github.com/repology/repology/blob/master/repology/fetchers/fetchers/guix.py
>
> While the Guix Data Service started as something to enable better
> understanding patches in an automated way, I think there are more uses
> for it, and initially, it's probably better to focus on the simple 
> ones.

It is not clear right now that it is related to patches, there is no 
patch
anywhere to see.

> The Repology use case is pretty simple, I think ideally there would be
> some machine readable data about the current state of packages in Guix
> available over the internet, and Repology would be able to download 
> that
> on a regular basis.

Repology is nice, but I would prefer wikidata support.

> The URL is a bit long, but I think that is now close to being possible
> with the Guix data service. I haven't got something working yet to
> easily access data for the latest revision, but for a particular
> revision, you can request a JSON file containing all the information I
> think Repology currently gets about all packages. For example:
>
>
> https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/packages.json?field=version&field=synopsis&field=description&field=home-page&field=location&field=licenses&limit_results=99999

FWIW it is very slow.

> Does anyone have any thoughts on this?

It seems to me it would be useful to have that informations somewhere. 
Like you
explain having the correct build information is a must have IMO along 
the logs.

Where is the code?


Thanks for sharing!