[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[bug#45972] Julia importer?
From: |
Nicolò Balzarotti |
Subject: |
[bug#45972] Julia importer? |
Date: |
Sat, 30 Jan 2021 21:13:14 +0100 |
zimoun <zimon.toutoune@gmail.com> writes:
> Hi,
Hi Simon!
>>
>> (invoke-julia "using Pkg; Pkg.TOML.parsefile("Project.toml")["name"] |>
>> println")
>
> With a bit more glue, could this be transformed into something like
> “julia->guix-package”? And so have a Julia package importer, even if it
> fails for some cases.
Well, if you mean "Can we use Pkg.jl to generate package definitions for
us?" the answer is "probably yes, but I never investigated this". That
line uses just Julia Base TOML.jl, which for some reason is defined
inside module Pkg.jl (and it's not top-level).
If you instead meant "Can we have a Julia importer?" some time ago I
wrote one in Julia, I've not used it in the last year, but it did work
quite well back then. I attach it here for reference.
But before digging into Pkg3, I need to push a new set of packages which
contains julia-jllwrappers, which is needed for Julia packages which
require binary distributions to work. After that, it will be possible
to create an importer that work "ok" even with packages depending on
.so libraries.
using Pkg
using Pkg.TOML
using LibGit2
const base = ["Base64",
"CRC32c",
"Dates",
"DelimitedFiles",
"Distributed",
"FileWatching",
"Future",
"InteractiveUtils",
"Libdl",
"LibGit2",
"LinearAlgebra",
"Logging",
"Markdown",
"Mmap",
"Pkg",
"Printf",
"Profile",
"Random",
"REPL",
"Serialization",
"SHA",
"SharedArrays",
"Sockets",
"SparseArrays",
"Statistics",
"SuiteSparse",
"Test",
"Unicode",
"UUIDs"]
const disabled = ["WinRPM", "Homebrew", "CMake",
"Docile", "Color",
"HTTPClient", "ICU", "Calendar", "LegacyStrings",
"Nulls"]
registrypath = expanduser("~/.julia-old/registries/General/")
registry = joinpath(registrypath, "Registry.toml")
const register = TOML.parse(join(readlines(registry), "\n"))
jlpkgname(info) = "julia-$(lowercase(info.name))"
function getrev(path)
versions = TOML.parse(join(readlines(path), "\n"))
versionv = findmax(map(x -> VersionNumber(x), keys(versions) |> collect))
(rev = versions[string(versionv[1])]["git-tree-sha1"],
ver = versionv[1])
end
function getpackagebyuuid(uuid)
uuid in keys(register["packages"]) || return nothing
path = register["packages"][uuid]["path"]
getpath(x) = joinpath(registrypath, joinpath(path, x))
package = TOML.parse(join(readlines(getpath("Package.toml")), "\n"))
deppath = getpath("Deps.toml")
isfile(deppath) || return nothing
deps = TOML.parse(join(readlines(deppath), "\n"))
(name = package["name"],
uuid = package["uuid"],
repo = package["repo"],
deps = deps,
vers = getrev(getpath("Versions.toml")))
end
function getpackage(wanted)
uuid = findfirst(p -> lowercase(p["name"]) == lowercase(wanted),
register["packages"])
uuid == nothing && return nothing
return getpackagebyuuid(uuid)
end
function getdeps!(deps, vers, recursive, out)
flat(arr::Array) = mapreduce(x -> isa(x, Array) ? flat(x) : x, append!,
arr, init=[])
v = map(p -> VersionNumber.(split(p, '-')), keys(deps) |> collect)
valid = findall(x -> length(x) == 1 ||
(x[2] == v"0" && x[1] <= vers) ||
x[1] <= vers <= x[2], v)
f = flat(map(x -> values(x), values(collect(values(deps))[valid])))
push!.(Ref(out), f)
# if recursive
# push!
# end
nothing
end
function have(info)
file = "/home/nixo/git/guix/gnu/packages/julia-xyz.scm"
return "(name \"" * jlpkgname(info) * "\")" in strip.(readlines(file))
end
function gethash(info)
wd = mktempdir()
if info.name in base ||
info.name in disabled ||
have(info)
return ""
end
println(stderr, "Cloning $(info.name) in $wd")
repo = LibGit2.clone(info.repo, wd)
hash = cd(wd) do
out = Pipe()
try
LibGit2.checkout!(repo,
string(LibGit2.GitHash(LibGit2.peel(LibGit2.GitCommit,
LibGit2.GitTag(repo, "v" * string(info.vers.ver))))))
catch e
try
LibGit2.checkout!(repo,
string(LibGit2.GitHash(LibGit2.peel(LibGit2.GitCommit,
LibGit2.GitCommit(repo, "v" * string(info.vers.ver))))))
catch e
# FIXME: if this happens, return the commit too and use it in
the package
println(stderr, "Failed to checkout $(e), continuing")
end
end
run(pipeline(`guix hash -rx .`, stdout=out))
readline(out)
end
rm(wd, recursive = true)
hash
end
function makepackage(info; done = [])
if info === nothing
@warn "Could not find package (have you cloned the registry?)"
return
elseif info in done
return ""
end
push!(done, info)
deps = String[]
getdeps!(info.deps, info.vers.ver, true, deps)
# TODO: remove deps that are in base
deps = filter(x -> x !== nothing, getpackagebyuuid.(deps))
deplist = join(map(name -> "(\"$name\" ,$name)",
jlpkgname.(deps)), '\n')
packagedeps = join(makepackage.(deps, done = done), "")
hash = gethash(info)
hash == "" && return ""
"""
$packagedeps
(define-public $(jlpkgname(info))
(package
(name "$(jlpkgname(info))")
(version "$(info.vers.ver)")
(source
(origin
(method git-fetch)
(uri (git-reference
(url "$(info.repo)")
(commit (string-append "v" version))))
(file-name "$(info.name)")
(sha256
(base32 "$hash"))))
(propagated-inputs `($(deplist)))
(build-system julia-build-system)
(home-page "$(info.repo)")
(synopsis "")
(description "")
(license license:expat)))
"""
end
println.(makepackage.(getpackage.(ARGS)))
- [bug#45972] [PATCH] Add julia-json with dependencies, Nicolò Balzarotti, 2021/01/18
- [bug#45972] [PATCH] Add julia-json with dependencies, Ludovic Courtès, 2021/01/26
- [bug#45972] [PATCH] Add julia-json with dependencies, Nicolò Balzarotti, 2021/01/26
- [bug#45972] [PATCH] Add julia-json with dependencies, Ludovic Courtès, 2021/01/27
- [bug#45972] [PATCH] Add julia-json with dependencies, Nicolò Balzarotti, 2021/01/27
- [bug#45972] [PATCH] Add julia-json with dependencies, Ludovic Courtès, 2021/01/28
- [bug#45972] [PATCH] Add julia-json with dependencies, Nicolò Balzarotti, 2021/01/28
- bug#45972: [PATCH] Add julia-json with dependencies, Ludovic Courtès, 2021/01/30
- [bug#45972] Julia importer?, zimoun, 2021/01/30
- [bug#45972] Julia importer?,
Nicolò Balzarotti <=
- [bug#45972] Julia importer?, Nicolò Balzarotti, 2021/01/30
- [bug#45972] Julia importer?, zimoun, 2021/01/31
- [bug#45972] Julia importer?, Nicolò Balzarotti, 2021/01/31