[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[taler-grid5k] branch master updated (74d0f38 -> 2554db5)
|
From: |
gnunet |
|
Subject: |
[taler-grid5k] branch master updated (74d0f38 -> 2554db5) |
|
Date: |
Thu, 18 Nov 2021 14:49:01 +0100 |
This is an automated email from the git hooks/post-receive script.
marco-boss pushed a change to branch master
in repository grid5k.
from 74d0f38 -fix script
new 6deeecd add new version of image
new 3f223c4 rename jobs
new b8f8bb1 update image
new 6521a09 add dummy scripts and some etc files
new 4cfe3dd update init scripts
new ce67219 add env variable template
new 0edcdab update db script
new 3a39527 make pq accesible
new 5ac1249 fix overwriting
new 45a08a7 update bank config, add dns and bank scripts
new ba1f47b fix exch script
new b4fdafa fix sed command
new 40610a2 add bind
new e2fe77b add dyndns to setup script
new 1954d4c update scripts
new d88e7cd add var for bind
new a5cb0ef add nohup, add dnsutils to build image
new 9207559 update enviroment at start of script
new 5270540 add nameserver to /etc/resolv only if not there yet
new aa635a3 fix exchange script
new d098f8f remove default pg port
new ebcf853 add proxy
new efc1a9f fix scripts
new a2b01f7 add resolv.conf
new 1a0492e remove resolv.conf
new 423c61e undo removing grid5000 nameservers
new 3a22360 update dns config and scripts
new f1624ce updated dns
new ff6447c run dns on taler.net
new 9c51160 add script order
new 54177fd update scripts
new 9a248b0 add taler default env
new 857198e update path
new 15df966 set retry times to 1
new 056c68c add monitorig components
new d2c979b fix typo
new d303538 update image
new a006e3a update image - add prometheus + loki
new 74c413e first monitoring setup
new 0765e9d update configurations
new 5d58fcd fix prometheus
new 0ff3644 move exporters from db and nginx on monitor host
new e82cb7f dynamically update grafana datasources
new 4242ffa fix function name
new e1ff832 add rsyslog config for better performance with loki
new f83ab80 mulitple exch processes possible - syslog configuredwq
new 1c889ac fix promtail, add helper function script
new 9d4522f fix script imports, add net packages
new 58ef94a add node exporter
new fcccebe add node-exporters
new 3e1465e add tests with parallel
new 6b2c0b2 fix dns entries for wallets
new 8f105b3 clear dns updated
new cceb709 add step to build newest wallet from source
new bf8a634 add possibilty for dynamic amount of wallets
new 6fc2270 add debug to every taler service
new 533d636 configure nginx logging to syslog
new 57c70cb update wallet script
new d8b8f0c fix sed
new f93d251 fix rsyslog config and logger messages for wallet
new 1b8566e fix wallet script
new d572acb fix rsyslog and wallet log messages
new 056a482 add upstream response time
new d49185d update promtail - issues with too many open files when too
much labels are assigned
new 15ef828 update promtail config
new a6afcf5 increase wallet memory
new 15479d2 add merchant, some comments in scripts
new 0f49aff update clearing of dns
new da75db5 script aborts with ((i++)) ?? set to 'let "i+=1"'
new be60df6 update wallet logs
new 9f54e03 remove trhottle
new 0479465 configure postgres
new 57693bf switch branch in wallet build
new c7f448b install nodejs permanently
new 2db4851 add benchmark run for wallet
new 6d72dd6 modify nginx settings
new eba5116 update image build
new 076d8ed add jinja
new 40e45ff fix wallet build
new b73fdc3 add dist builds
new 1505008 build from source working with dist
new c909c5c add ping
new 2b23c67 update image - set install path to /usr instead of /usr/local
new 6c49fa6 set envfile in netdelay service
new a8f07a4 fix createuser script
new 9d4daab update scripts
new 907fccd fix timer to every five minutes
new bc0ef15 use wallet as a service
new 0a77c62 update configs and scripts
new e6160d9 change number of iterations to 10 - bigger values slow down
wallets over time
new dd3c262 change ts in proxy logs
new cf7e9ac update loki config
new 69617fe image build back to apt
new 3114373 add merchant configuration
new d5aa8e6 add merchant from source
new eea591d fix merchant path
new b029f29 fix variables in image script
new adcf0e3 add missing package for merchant build
new 1f05a24 add docker to build g5k image
new 1158bd9 update docker README
new 7004eb0 update merchant config
new d7ad99f update merchant configuration
new 277ce61 update wallet domain name settings
new 52fe8b8 update scripts and docker build
new 63174b2 more wallet hosts
new 64e5675 update scripts and docker build image
new 3f19a79 fix image config
new 1a169ae fix variable names
new 9aed5c1 update loki version
new e414c19 add no-copy flag
new c913916 move expect to top for pipeline to fail if not successful
new ba607fc no doc for gem
new c94f411 tune postgres config
new 380098f update configs
new 6ce1303 split rspec, change tags
new ecded79 add dnsmasq as stub resolver, tshark, update scripts
new 30abb59 start processes with argument to script
new 6a0c6d6 add wirewatch systemd template
new d0e1f34 update espec - add taler-perf utility
new f943a69 remove log levels for pipeline output
new da70ce1 cut log size of proxy
new 21a96a7 remove directories of taler after install
new bc07c54 update per stream limit in loki
new 0e3d064 use kameleon nfs image
new 0cb8f70 change permissions
new 234a7e1 add log backup and update dns config for nfs
new 9cefaca add log backup
new a66b20a add sanitizer and disable optimizer
new 346d08b add config vor exchange via environment
new 2c53fdf update image
new 593ffc8 fix image
new 3b7b779 change exchange initialisation script
new b63e3a7 use systemd-resolved instead of dnsmasq
new 67104d4 resolved does not work, switch back
new 212ba7b remove sanitizer again
new 5661f5e fix account
new 921535c add possibility for more than 100 exchanges
new 6a176c6 re-enable sanitizer
new fb8ab35 remove sanitizer
new aa81e6f move to master for wallet, enable logbackup for bank
new 2554db5 Node setup complete - multiple projects because of allocation
bug in jFed
The 141 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.gitignore | 1 +
README | 21 +-
at-boot.sh | 9 -
configs/etc/default/prometheus | 112 ++++++
configs/etc/default/prometheus-nginx-exporter | 38 ++
configs/etc/default/prometheus-postgres-exporter | 19 +
configs/etc/default/taler-exchange | 1 +
configs/etc/logrotate.d/taler | 12 +
configs/etc/monitor/loki.yaml | 65 +++
configs/etc/monitor/node-exporters.yaml.tpl | 26 ++
configs/etc/monitor/postgres-exporter.yaml | 275 +++++++++++++
configs/etc/monitor/prometheus.yaml | 23 ++
configs/etc/monitor/promtail.yaml | 21 +
configs/etc/nginx/nginx.conf | 62 +++
configs/etc/nginx/sites-available/proxy | 40 ++
configs/etc/rsyslog.d/taler.conf | 23 ++
configs/etc/taler/conf.d/exchange-business.conf | 36 ++
configs/etc/taler/conf.d/exchange-coins.conf | 158 ++++++++
configs/etc/taler/conf.d/exchange-system.conf | 10 +
configs/etc/taler/conf.d/fake-bank.conf | 5 +
configs/etc/taler/conf.d/merchant.conf | 15 +
configs/etc/taler/overrides.conf | 1 +
.../exchange-accountcredentials.secret.conf | 17 +
configs/etc/taler/secrets/exchange-db.secret.conf | 10 +
configs/etc/taler/secrets/merchant-db.secret.conf | 3 +
configs/etc/taler/taler.conf | 47 +++
configs/usr/lib/systemd/system/loki.service | 9 +
configs/usr/lib/systemd/system/nginx.service | 31 ++
configs/usr/lib/systemd/system/promtail.service | 9 +
.../system/taler-exchange-aggregator.service | 15 +
.../systemd/system/taler-exchange-closer.service | 16 +
.../systemd/system/taler-exchange-httpd.service | 22 +
.../lib/systemd/system/taler-exchange-httpd.socket | 14 +
.../systemd/system/taler-exchange-httpd@.service | 21 +
.../systemd/system/taler-exchange-httpd@.socket | 14 +
.../system/taler-exchange-secmod-eddsa.service | 17 +
.../system/taler-exchange-secmod-rsa.service | 16 +
.../systemd/system/taler-exchange-transfer.service | 16 +
.../system/taler-exchange-wirewatch.service | 16 +
.../system/taler-exchange-wirewatch@.service | 16 +
.../usr/lib/systemd/system/taler-exchange.target | 12 +
.../usr/lib/systemd/system/taler-fakebank.service | 12 +
.../usr/lib/systemd/system/taler-logbackup.service | 11 +
.../usr/lib/systemd/system/taler-logbackup.timer | 10 +
.../systemd/system/taler-merchant-httpd.service | 11 +
.../usr/lib/systemd/system/taler-netdelay.service | 11 +
.../usr/lib/systemd/system/taler-netdelay.timer | 10 +
.../usr/lib/systemd/system/taler-wallet@.service | 18 +
debian11-taler.yaml | 77 ----
default/from_scratch/x86_64/base.yaml | 138 -------
default/from_scratch/x86_64/debian-testing.yaml | 29 --
default/steps/aliases/defaults.yaml | 169 --------
default/steps/bootstrap/download_installer.yaml | 31 --
default/steps/bootstrap/start_http_server.yaml | 19 -
default/steps/bootstrap/start_qemu.yaml | 227 -----------
default/steps/data/helpers/export_appliance.py | 242 -----------
.../steps/data/helpers/netinstall_iso_finder.py | 163 --------
.../steps/data/preseed/debian-testing-preseed.cfg | 322 ---------------
default/steps/env/functions.sh | 201 ----------
docker/Dockerfile | 76 ++++
docker/README.md | 96 +++++
docker/docker-compose.yaml | 33 ++
docker/entrypoint.sh | 186 +++++++++
etc/.empty | 0
experiment/README | 21 +
experiment/cleardns.sh | 33 ++
experiment/env | 54 +++
experiment/experiment-specification.yml | 19 +
experiment/infra.rspec | 43 ++
experiment/run.sh | 85 ++++
experiment/scripts/bank.sh | 10 +
experiment/scripts/benchmark.sh | 23 ++
experiment/scripts/createusers.sh | 110 +++++
experiment/scripts/database.sh | 72 ++++
experiment/scripts/exchange.sh | 61 +++
experiment/scripts/helpers.sh | 47 +++
experiment/scripts/log-save.sh | 21 +
experiment/scripts/merchant.sh | 54 +++
experiment/scripts/monitor.sh | 81 ++++
experiment/scripts/ping.sh | 13 +
experiment/scripts/proxy.sh | 55 +++
experiment/scripts/wallet.sh | 54 +++
experiment/setup.sh | 60 +++
experiment/taler-perf.sh | 84 ++++
experiment/wallets.rspec | 63 +++
grid5000/debian11-x64-min.yaml | 27 --
grid5000/from_scratch/aarch64/base.yaml | 25 --
grid5000/from_scratch/aarch64/debian-base.yaml | 59 ---
grid5000/from_scratch/debian-base.yaml | 67 ----
.../bootstrap/debian/prepare_autoinstall.yaml | 11 -
grid5000/steps/bootstrap/prepare_appliance.yaml | 33 --
grid5000/steps/bootstrap/prepare_disk.yaml | 10 -
.../bootstrap/prepare_ssh_to_out_context.yaml | 23 --
grid5000/steps/checkpoints/simple.yaml | 21 -
grid5000/steps/data/helpers/simple_http_server.py | 129 ------
.../steps/data/preseed/debian-buster-preseed.cfg | 443 ---------------------
grid5000/steps/data/qemu-sendkeys.rb | 121 ------
.../steps/data/qemu-sendkeys/netinst-iso-debian | 1 -
.../env/files/std/oar/default_oar-node_site | 49 ---
.../manifests/base/configure_ip_over_infiniband.pp | 69 ----
.../modules/env/manifests/big/install_openmpi.pp | 58 ---
.../modules/env/manifests/big/install_smartd.pp | 26 --
grid5000/steps/disable_checkpoint.yaml | 3 -
grid5000/steps/enable_checkpoint.yaml | 5 -
grid5000/steps/env/bashrc | 23 --
grid5000/steps/export/export_vagrant_box.yaml | 42 --
grid5000/steps/export/save_appliance_VM.yaml | 23 --
grid5000/steps/setup/create_user.yaml | 11 -
grid5000/steps/setup/debian/clean_system.yaml | 34 --
.../setup/debian/clean_unnecessary_packages.yaml | 9 -
grid5000/steps/setup/debian/minimal_install.yaml | 6 -
grid5000/steps/setup/debian/setup_vagrant_box.yaml | 77 ----
gridboot.service | 13 -
image/.gitignore | 1 +
.../grid5000}/debian11-x64-common.yaml | 2 +-
.../grid5000/debian11-x64-nfs.yaml | 14 +-
.../grid5000}/from_scratch/base.yaml | 0
.../grid5000/from_scratch}/debian-base.yaml | 0
.../grid5000/from_scratch}/debian-bullseye.yaml | 0
.../grid5000}/steps/aliases/defaults.yaml | 0
.../bootstrap/debian/prepare_autoinstall.yaml | 0
.../steps/bootstrap/download_installer.yaml | 2 +-
.../steps/bootstrap/prepare_appliance.yaml | 0
.../grid5000}/steps/bootstrap/prepare_disk.yaml | 0
.../bootstrap/prepare_ssh_to_out_context.yaml | 0
.../steps/bootstrap/start_http_server.yaml | 2 +-
.../grid5000}/steps/bootstrap/start_qemu.yaml | 0
.../grid5000}/steps/checkpoints/simple.yaml | 0
.../steps/data/helpers/export_appliance.py | 2 +-
.../steps/data/helpers/netinstall_iso_finder.py | 6 +-
.../steps/data/helpers/simple_http_server.py | 2 +-
.../steps/data/preseed/debian-bullseye-preseed.cfg | 0
.../grid5000}/steps/data/qemu-sendkeys.rb | 0
.../steps/data/qemu-sendkeys/netinst-iso-debian | 0
.../grid5000}/steps/data/setup/hiera/hiera.yaml | 0
.../steps/data/setup/hiera/hieradata/defaults.yaml | 0
.../steps/data/setup/puppet/manifests/base.pp | 0
.../steps/data/setup/puppet/manifests/big.pp | 0
.../steps/data/setup/puppet/manifests/min.pp | 0
.../steps/data/setup/puppet/manifests/nfs.pp | 0
.../steps/data/setup/puppet/manifests/std.pp | 0
.../steps/data/setup/puppet/manifests/xen.pp | 0
.../modules/env/files/base/cpufreq/cpufrequtils | 0
.../modules/env/files/base/ganglia/gmond.conf | 0
.../modules/env/files/base/infiniband/90-ib.rules | 0
.../modules/env/files/base/infiniband/openib.conf | 0
.../modules/env/files/base/infiniband/openibd | 0
.../env/files/base/infiniband/openibd.service | 0
.../puppet/modules/env/files/base/kexec/kexec | 0
.../puppet/modules/env/files/base/mx/ip_over_mx | 0
.../modules/env/files/base/ndctl/ndctl.preset | 0
.../modules/env/files/base/sshfs/40-fuse.rules | 0
.../env/files/base/tuning/limits-grid5000.conf | 0
.../env/files/base/tuning/sysctl-00-grid5000.conf | 0
.../env/files/base/userns/sysctl-00-userns.conf | 0
.../modules/env/files/big/amd_gpu/70-amdgpu.rules | 0
.../modules/env/files/big/kvm/60-qemu-system.rules | 0
.../puppet/modules/env/files/big/kvm/create_tap | 0
.../puppet/modules/env/files/big/kvm/random_mac | 0
.../setup/puppet/modules/env/files/big/kvm/sudoers | 0
.../puppet/modules/env/files/big/mic/85-mic.rules | 0
.../setup/puppet/modules/env/files/big/mic/fstab | 0
.../puppet/modules/env/files/big/mic/mic0.filelist | 0
.../setup/puppet/modules/env/files/big/mic/mpss | 0
.../puppet/modules/env/files/big/nvidia/cuda.conf | 0
.../env/files/big/nvidia/dcgm-exporter.service | 0
.../env/files/big/nvidia/ganglia-monitor.service | 0
.../env/files/big/nvidia/modpython-nvidia.conf | 0
.../files/big/nvidia/nvidia-persistenced.service | 0
.../env/files/big/nvidia/nvidia-smi.service | 0
.../env/files/min/apt/grid5000-archive-key.asc | 0
.../env/files/min/cpu_microcode/amd64-microcode | 0
.../env/files/min/cpu_microcode/intel-microcode | 0
.../modules/env/files/min/image_versioning/git_tag | 0
.../env/files/min/image_versioning/postinst | 0
.../puppet/modules/env/files/min/locales/locale | 0
.../modules/env/files/min/locales/locale.gen | 0
.../env/files/min/network/g5k-update-host-name | 0
.../puppet/modules/env/files/min/network/hosts | 0
.../env/files/nfs/ldap/ca2019.grid5000.fr.cert | 0
.../puppet/modules/env/files/nfs/ldap/common-auth | 0
.../modules/env/files/nfs/ldap/common-password | 0
.../puppet/modules/env/files/nfs/ldap/ldap.conf | 0
.../modules/env/files/nfs/ldap/libnss-ldap.conf | 0
.../puppet/modules/env/files/nfs/ldap/nscd.conf | 0
.../puppet/modules/env/files/nfs/ldap/nslcd.conf | 0
.../modules/env/files/nfs/ldap/nsswitch.conf | 0
.../env/files/nfs/openiscsi/55-openiscsi.rules | 0
.../modules/env/files/nfs/openiscsi/iscsidev.sh | 0
.../files/std/g5k-manager/g5k-disk-manager-backend | 54 ++-
.../g5k-manager/g5k-disk-manager-backend.service | 0
.../env/files/std/g5k-manager/g5k-pmem-manager | 0
.../files/std/g5k-manager/g5k-pmem-manager.service | 0
.../env/files/std/g5k-manager/lib/g5k-manager.rb | 0
.../env/files/std/g5k_generator/g5k_generator | 0
.../env/files/std/g5kchecks/g5k-checks.conf | 0
.../puppet/modules/env/files/std/lvm/lvm.conf | 0
.../modules/env/files/std/net_access/iptables | 0
.../env/files/std/net_access/iptables.stretch | 0
.../modules/env/files/std/net_access/rsyslog.conf | 0
.../env/files/std/net_access/syslog_iptables.conf | 0
.../files/std/nvidia_configure/nvidia-reset-mig | 0
.../std/nvidia_configure/nvidia-reset-mig.service | 0
.../modules/env/files/std/oar/batch_job_bashrc | 0
.../env/files/std/oar/etc/security/access.conf | 0
.../modules/env/files/std/oar/oar-node-service | 33 +-
.../env/files/std/oar/oar-node.service.override | 2 +
.../modules/env/files/std/oar/oar_sshclient_config | 0
.../env/files/std/oar/var/lib/oar/access.conf | 0
.../modules/env/files/std/sudo-g5k/id_rsa_sudo-g5k | 0
.../env/files/std/sudo-g5k/id_rsa_sudo-g5k.pub | 0
.../data/setup/puppet/modules/env/files/version | 0
.../setup/puppet/modules/env/files/xen/xen/id_rsa | 0
.../puppet/modules/env/files/xen/xen/id_rsa.pub | 0
.../puppet/modules/env/files/xen/xen/random_mac | 0
.../setup/puppet/modules/env/files/xen/xen/xen-g5k | 0
.../modules/env/files/xen/xen/xen-g5k.service | 0
.../modules/env/files/xen/xen/xend-config.sxp | 0
.../env/lib/facter/installed_kernelreleases.rb | 0
.../setup/puppet/modules/env/manifests/base.pp | 0
.../modules/env/manifests/base/add_ca_grid5000.pp | 0
.../env/manifests/base/configure_dhclient.pp | 0
.../manifests/base/configure_ip_over_infiniband.pp | 75 ++++
.../modules/env/manifests/base/configure_kexec.pp | 0
.../env/manifests/base/configure_omnipath.pp | 10 +-
.../env/manifests/base/disable_ndctl_monitor.pp | 0
.../env/manifests/base/disable_nvme_multipath.pp | 0
.../modules/env/manifests/base/do_not_clean_tmp.pp | 0
.../enable_cpufreq_with_performance_governor.pp | 0
.../modules/env/manifests/base/enable_userns.pp | 0
.../env/manifests/base/increase_ssh_maxstartups.pp | 0
.../manifests/base/install_and_disable_ganglia.pp | 0
.../env/manifests/base/tcp_tuning_for_10gbe.pp | 0
.../base/unlimited_memlock_for_infiniband.pp | 0
.../data/setup/puppet/modules/env/manifests/big.pp | 0
.../modules/env/manifests/big/configure_amd_gpu.pp | 16 +-
.../env/manifests/big/configure_initramfs.pp | 0
.../modules/env/manifests/big/configure_kvm.pp | 0
.../env/manifests/big/configure_nvidia_gpu.pp | 0
.../env/manifests/big/configure_nvidia_gpu/cuda.pp | 8 +
.../manifests/big/configure_nvidia_gpu/drivers.pp | 0
.../manifests/big/configure_nvidia_gpu/ganglia.pp | 0
.../manifests/big/configure_nvidia_gpu/modules.pp | 0
.../big/configure_nvidia_gpu/prometheus.pp | 0
.../manifests/big/configure_nvidia_gpu/services.pp | 0
.../modules/env/manifests/big/configure_postfix.pp | 0
.../modules/env/manifests/big/configure_sshfs.pp | 0
.../modules/env/manifests/big/install_beegfs.pp | 0
.../env/manifests/big/install_g5k_jupyterlab.pp | 5 +-
.../modules/env/manifests/big/install_openmpi.pp | 95 +++++
.../manifests/big/install_prometheus_exporters.pp | 0
.../modules/env/manifests/big/install_smartd.pp | 28 ++
.../env/manifests/big/install_snmp_tools.pp | 0
.../manifests/big/prepare_kernel_module_build.pp | 0
.../modules/env/manifests/common/apt_pinning.pp | 0
.../modules/env/manifests/common/g5kpackages.pp | 0
.../env/manifests/common/software_versions.pp | 23 +-
.../puppet/modules/env/manifests/commonpackages.pp | 0
.../setup/puppet/modules/env/manifests/init.pp | 0
.../data/setup/puppet/modules/env/manifests/min.pp | 0
.../env/manifests/min/add_image_version_in_etc.pp | 0
.../configure_kernel_and_blacklist_some_modules.pp | 0
.../min/configure_network_and_install_drivers.pp | 0
.../modules/env/manifests/min/generate_etc_motd.pp | 0
.../manifests/min/install_and_configure_locales.pp | 0
.../env/manifests/min/install_and_configure_ssh.pp | 0
.../env/manifests/min/install_cpu_microcode.pp | 0
.../env/manifests/min/install_metapackage.pp | 0
.../modules/env/manifests/min/install_tgz_g5k.pp | 0
.../modules/env/manifests/min/kernel/initramfs.pp | 0
.../modules/env/manifests/min/kernel/modules.pp | 0
.../modules/env/manifests/min/kernel/remove_old.pp | 0
.../env/manifests/min/kernel/setup_links.pp | 0
.../modules/env/manifests/min/set_root_password.pp | 0
.../manifests/min/set_timezone_to_europe_paris.pp | 0
.../data/setup/puppet/modules/env/manifests/nfs.pp | 0
.../modules/env/manifests/nfs/configure_iscsi.pp | 0
.../modules/env/manifests/nfs/configure_ldap.pp | 0
.../env/manifests/nfs/configure_module_path.pp | 0
.../modules/env/manifests/nfs/configure_ntp.pp | 0
.../env/manifests/nfs/install_nfs_requirements.pp | 0
.../manifests/nfs/install_osirim_requirements.pp | 0
.../nfs/install_storage5k_requirements.pp | 0
.../data/setup/puppet/modules/env/manifests/std.pp | 2 +
.../env/manifests/std/add_g5kcode_to_path.pp | 0
.../std/configure_g5kdiskmanagerbackend.pp | 0
.../env/manifests/std/configure_g5kmanager.pp | 0
.../env/manifests/std/configure_g5kpmemmanager.pp | 0
.../env/manifests/std/configure_oar_client.pp | 23 +-
.../env/manifests/std/configure_rsyslog_remote.pp | 0
.../setup/puppet/modules/env/manifests/std/dell.pp | 0
.../modules/env/manifests/std/dell/params.pp | 0
.../env/manifests/std/disable_lvm_pvscan.pp | 0
.../modules/env/manifests/std/g5k_generator.pp | 0
.../modules/env/manifests/std/install_g5kchecks.pp | 12 +
.../env/manifests/std/install_g5ksubnets.pp | 0
.../env/manifests/std/install_hwraid_apt_source.pp | 0
.../modules/env/manifests/std/install_kameleon.pp | 21 +
.../manifests/std/install_libguestfs_backport.pp | 0
.../modules/env/manifests/std/install_megacli.pp | 0
.../modules/env/manifests/std/install_sudog5k.pp | 0
.../puppet/modules/env/manifests/std/ipmitool.pp | 0
.../modules/env/manifests/std/nvidia_reset_mig.pp | 0
.../data/setup/puppet/modules/env/manifests/xen.pp | 0
.../modules/env/manifests/xen/configure_xen.pp | 0
.../modules/env/manifests/xen/install_grub.pp | 0
.../env/templates/base/omnipath/scibian.key.erb | 0
.../modules/env/templates/common/apt_pinning.erb | 0
.../puppet/modules/env/templates/min/motd.erb | 0
.../env/templates/nfs/ldap/common-account.erb | 0
.../modules/env/templates/nfs/ntp/ntp.conf.erb | 0
.../env/templates/std/dell/linux.dell.com.key.erb | 0
.../std/hwraid/hwraid.le-vert.net.key.erb | 0
.../grid5000}/steps/disable_checkpoint.yaml | 0
.../grid5000}/steps/enable_checkpoint.yaml | 0
{default => image/grid5000}/steps/env/bashrc | 0
.../grid5000}/steps/env/functions.sh | 0
.../steps/export/debian/clean_dhcp_leases.yaml | 0
.../steps/export/do_qcow2_finish_works.yaml | 0
.../grid5000}/steps/export/export_g5k.yaml | 7 +-
.../grid5000}/steps/export/save_appliance_VM.yaml | 3 +-
.../grid5000}/steps/setup/debian/clean_system.yaml | 0
.../steps/setup/debian/configure_apt_sources.yaml | 0
.../steps/setup/debian/configure_system.yaml | 0
.../steps/setup/debian/install_packages.yaml | 0
.../steps/setup/debian/minimal_install.yaml | 0
.../steps/setup/debian/run_orchestrator.yaml | 0
.../steps/setup/debian/setup_orchestrator.yaml | 0
image/taler-debian11.yaml | 181 +++++++++
jfed/README | 2 -
jfed/twoboot.rspec | 17 -
notes.txt | 21 +-
steps/setup/taler_install.yaml | 8 -
steps/setup/taler_install.yaml~ | 2 -
update.sh | 11 -
335 files changed, 3398 insertions(+), 3159 deletions(-)
create mode 100644 .gitignore
delete mode 100644 at-boot.sh
create mode 100644 configs/etc/default/prometheus
create mode 100644 configs/etc/default/prometheus-nginx-exporter
create mode 100644 configs/etc/default/prometheus-postgres-exporter
create mode 100644 configs/etc/default/taler-exchange
create mode 100644 configs/etc/logrotate.d/taler
create mode 100644 configs/etc/monitor/loki.yaml
create mode 100644 configs/etc/monitor/node-exporters.yaml.tpl
create mode 100644 configs/etc/monitor/postgres-exporter.yaml
create mode 100644 configs/etc/monitor/prometheus.yaml
create mode 100644 configs/etc/monitor/promtail.yaml
create mode 100644 configs/etc/nginx/nginx.conf
create mode 100644 configs/etc/nginx/sites-available/proxy
create mode 100644 configs/etc/rsyslog.d/taler.conf
create mode 100755 configs/etc/taler/conf.d/exchange-business.conf
create mode 100755 configs/etc/taler/conf.d/exchange-coins.conf
create mode 100644 configs/etc/taler/conf.d/exchange-system.conf
create mode 100644 configs/etc/taler/conf.d/fake-bank.conf
create mode 100644 configs/etc/taler/conf.d/merchant.conf
create mode 100644 configs/etc/taler/overrides.conf
create mode 100755
configs/etc/taler/secrets/exchange-accountcredentials.secret.conf
create mode 100755 configs/etc/taler/secrets/exchange-db.secret.conf
create mode 100644 configs/etc/taler/secrets/merchant-db.secret.conf
create mode 100755 configs/etc/taler/taler.conf
create mode 100644 configs/usr/lib/systemd/system/loki.service
create mode 100644 configs/usr/lib/systemd/system/nginx.service
create mode 100644 configs/usr/lib/systemd/system/promtail.service
create mode 100644
configs/usr/lib/systemd/system/taler-exchange-aggregator.service
create mode 100644 configs/usr/lib/systemd/system/taler-exchange-closer.service
create mode 100755 configs/usr/lib/systemd/system/taler-exchange-httpd.service
create mode 100755 configs/usr/lib/systemd/system/taler-exchange-httpd.socket
create mode 100755 configs/usr/lib/systemd/system/taler-exchange-httpd@.service
create mode 100755 configs/usr/lib/systemd/system/taler-exchange-httpd@.socket
create mode 100644
configs/usr/lib/systemd/system/taler-exchange-secmod-eddsa.service
create mode 100644
configs/usr/lib/systemd/system/taler-exchange-secmod-rsa.service
create mode 100644
configs/usr/lib/systemd/system/taler-exchange-transfer.service
create mode 100644
configs/usr/lib/systemd/system/taler-exchange-wirewatch.service
create mode 100644
configs/usr/lib/systemd/system/taler-exchange-wirewatch@.service
create mode 100644 configs/usr/lib/systemd/system/taler-exchange.target
create mode 100644 configs/usr/lib/systemd/system/taler-fakebank.service
create mode 100644 configs/usr/lib/systemd/system/taler-logbackup.service
create mode 100644 configs/usr/lib/systemd/system/taler-logbackup.timer
create mode 100644 configs/usr/lib/systemd/system/taler-merchant-httpd.service
create mode 100644 configs/usr/lib/systemd/system/taler-netdelay.service
create mode 100644 configs/usr/lib/systemd/system/taler-netdelay.timer
create mode 100755 configs/usr/lib/systemd/system/taler-wallet@.service
delete mode 100644 debian11-taler.yaml
delete mode 100644 default/from_scratch/x86_64/base.yaml
delete mode 100644 default/from_scratch/x86_64/debian-testing.yaml
delete mode 100644 default/steps/aliases/defaults.yaml
delete mode 100644 default/steps/bootstrap/download_installer.yaml
delete mode 100644 default/steps/bootstrap/start_http_server.yaml
delete mode 100644 default/steps/bootstrap/start_qemu.yaml
delete mode 100644 default/steps/data/helpers/export_appliance.py
delete mode 100644 default/steps/data/helpers/netinstall_iso_finder.py
delete mode 100644 default/steps/data/preseed/debian-testing-preseed.cfg
delete mode 100644 default/steps/env/functions.sh
create mode 100644 docker/Dockerfile
create mode 100644 docker/README.md
create mode 100644 docker/docker-compose.yaml
create mode 100644 docker/entrypoint.sh
delete mode 100644 etc/.empty
create mode 100755 experiment/README
create mode 100644 experiment/cleardns.sh
create mode 100755 experiment/env
create mode 100755 experiment/experiment-specification.yml
create mode 100644 experiment/infra.rspec
create mode 100644 experiment/run.sh
create mode 100755 experiment/scripts/bank.sh
create mode 100755 experiment/scripts/benchmark.sh
create mode 100755 experiment/scripts/createusers.sh
create mode 100755 experiment/scripts/database.sh
create mode 100755 experiment/scripts/exchange.sh
create mode 100755 experiment/scripts/helpers.sh
create mode 100755 experiment/scripts/log-save.sh
create mode 100755 experiment/scripts/merchant.sh
create mode 100755 experiment/scripts/monitor.sh
create mode 100755 experiment/scripts/ping.sh
create mode 100755 experiment/scripts/proxy.sh
create mode 100755 experiment/scripts/wallet.sh
create mode 100644 experiment/setup.sh
create mode 100644 experiment/taler-perf.sh
create mode 100644 experiment/wallets.rspec
delete mode 100644 grid5000/debian11-x64-min.yaml
delete mode 100644 grid5000/from_scratch/aarch64/base.yaml
delete mode 100644 grid5000/from_scratch/aarch64/debian-base.yaml
delete mode 100644 grid5000/from_scratch/debian-base.yaml
delete mode 100644 grid5000/steps/bootstrap/debian/prepare_autoinstall.yaml
delete mode 100644 grid5000/steps/bootstrap/prepare_appliance.yaml
delete mode 100644 grid5000/steps/bootstrap/prepare_disk.yaml
delete mode 100644 grid5000/steps/bootstrap/prepare_ssh_to_out_context.yaml
delete mode 100644 grid5000/steps/checkpoints/simple.yaml
delete mode 100644 grid5000/steps/data/helpers/simple_http_server.py
delete mode 100644 grid5000/steps/data/preseed/debian-buster-preseed.cfg
delete mode 100644 grid5000/steps/data/qemu-sendkeys.rb
delete mode 100644 grid5000/steps/data/qemu-sendkeys/netinst-iso-debian
delete mode 100644
grid5000/steps/data/setup/puppet/modules/env/files/std/oar/default_oar-node_site
delete mode 100644
grid5000/steps/data/setup/puppet/modules/env/manifests/base/configure_ip_over_infiniband.pp
delete mode 100644
grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_openmpi.pp
delete mode 100644
grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_smartd.pp
delete mode 100644 grid5000/steps/disable_checkpoint.yaml
delete mode 100644 grid5000/steps/enable_checkpoint.yaml
delete mode 100644 grid5000/steps/env/bashrc
delete mode 100644 grid5000/steps/export/export_vagrant_box.yaml
delete mode 100644 grid5000/steps/export/save_appliance_VM.yaml
delete mode 100644 grid5000/steps/setup/create_user.yaml
delete mode 100644 grid5000/steps/setup/debian/clean_system.yaml
delete mode 100644 grid5000/steps/setup/debian/clean_unnecessary_packages.yaml
delete mode 100644 grid5000/steps/setup/debian/minimal_install.yaml
delete mode 100644 grid5000/steps/setup/debian/setup_vagrant_box.yaml
delete mode 100644 gridboot.service
create mode 100644 image/.gitignore
rename {grid5000 => image/grid5000}/debian11-x64-common.yaml (96%)
rename grid5000/from_scratch/debian-bullseye.yaml =>
image/grid5000/debian11-x64-nfs.yaml (56%)
rename {grid5000 => image/grid5000}/from_scratch/base.yaml (100%)
rename {default/from_scratch/x86_64 =>
image/grid5000/from_scratch}/debian-base.yaml (100%)
rename {grid5000/from_scratch/aarch64 =>
image/grid5000/from_scratch}/debian-bullseye.yaml (100%)
rename {grid5000 => image/grid5000}/steps/aliases/defaults.yaml (100%)
rename {default =>
image/grid5000}/steps/bootstrap/debian/prepare_autoinstall.yaml (100%)
rename {grid5000 => image/grid5000}/steps/bootstrap/download_installer.yaml
(90%)
rename {default => image/grid5000}/steps/bootstrap/prepare_appliance.yaml
(100%)
rename {default => image/grid5000}/steps/bootstrap/prepare_disk.yaml (100%)
rename {default =>
image/grid5000}/steps/bootstrap/prepare_ssh_to_out_context.yaml (100%)
rename {grid5000 => image/grid5000}/steps/bootstrap/start_http_server.yaml
(81%)
rename {grid5000 => image/grid5000}/steps/bootstrap/start_qemu.yaml (100%)
rename {default => image/grid5000}/steps/checkpoints/simple.yaml (100%)
rename {grid5000 => image/grid5000}/steps/data/helpers/export_appliance.py
(99%)
mode change 100644 => 100755
rename {grid5000 =>
image/grid5000}/steps/data/helpers/netinstall_iso_finder.py (98%)
mode change 100644 => 100755
rename {default => image/grid5000}/steps/data/helpers/simple_http_server.py
(99%)
mode change 100644 => 100755
rename {grid5000 =>
image/grid5000}/steps/data/preseed/debian-bullseye-preseed.cfg (100%)
rename {default => image/grid5000}/steps/data/qemu-sendkeys.rb (100%)
rename {default => image/grid5000}/steps/data/qemu-sendkeys/netinst-iso-debian
(100%)
rename {grid5000 => image/grid5000}/steps/data/setup/hiera/hiera.yaml (100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/hiera/hieradata/defaults.yaml (100%)
rename {grid5000 => image/grid5000}/steps/data/setup/puppet/manifests/base.pp
(100%)
rename {grid5000 => image/grid5000}/steps/data/setup/puppet/manifests/big.pp
(100%)
rename {grid5000 => image/grid5000}/steps/data/setup/puppet/manifests/min.pp
(100%)
rename {grid5000 => image/grid5000}/steps/data/setup/puppet/manifests/nfs.pp
(100%)
rename {grid5000 => image/grid5000}/steps/data/setup/puppet/manifests/std.pp
(100%)
rename {grid5000 => image/grid5000}/steps/data/setup/puppet/manifests/xen.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/base/cpufreq/cpufrequtils
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/base/ganglia/gmond.conf
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/base/infiniband/90-ib.rules
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/base/infiniband/openib.conf
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/base/infiniband/openibd
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/base/infiniband/openibd.service
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/base/kexec/kexec
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/base/mx/ip_over_mx
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/base/ndctl/ndctl.preset
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/base/sshfs/40-fuse.rules
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/base/tuning/limits-grid5000.conf
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/base/tuning/sysctl-00-grid5000.conf
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/base/userns/sysctl-00-userns.conf
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/big/amd_gpu/70-amdgpu.rules
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/big/kvm/60-qemu-system.rules
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/big/kvm/create_tap
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/big/kvm/random_mac
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/big/kvm/sudoers (100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/big/mic/85-mic.rules
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/big/mic/fstab (100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/big/mic/mic0.filelist
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/big/mic/mpss (100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/big/nvidia/cuda.conf
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/big/nvidia/dcgm-exporter.service
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/big/nvidia/ganglia-monitor.service
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/big/nvidia/modpython-nvidia.conf
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/big/nvidia/nvidia-persistenced.service
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/big/nvidia/nvidia-smi.service
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/min/apt/grid5000-archive-key.asc
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/min/cpu_microcode/amd64-microcode
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/min/cpu_microcode/intel-microcode
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/min/image_versioning/git_tag
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/min/image_versioning/postinst
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/min/locales/locale
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/min/locales/locale.gen
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/min/network/g5k-update-host-name
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/min/network/hosts
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/nfs/ldap/ca2019.grid5000.fr.cert
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/nfs/ldap/common-auth
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/nfs/ldap/common-password
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/nfs/ldap/ldap.conf
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/nfs/ldap/libnss-ldap.conf
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/nfs/ldap/nscd.conf
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/nfs/ldap/nslcd.conf
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/nfs/ldap/nsswitch.conf
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/nfs/openiscsi/55-openiscsi.rules
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/nfs/openiscsi/iscsidev.sh
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/std/g5k-manager/g5k-disk-manager-backend
(81%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/std/g5k-manager/g5k-disk-manager-backend.service
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/std/g5k-manager/g5k-pmem-manager
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/std/g5k-manager/g5k-pmem-manager.service
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/std/g5k-manager/lib/g5k-manager.rb
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/std/g5k_generator/g5k_generator
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/std/g5kchecks/g5k-checks.conf
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/std/lvm/lvm.conf
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/std/net_access/iptables
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/std/net_access/iptables.stretch
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/std/net_access/rsyslog.conf
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/std/net_access/syslog_iptables.conf
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/std/nvidia_configure/nvidia-reset-mig
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/std/nvidia_configure/nvidia-reset-mig.service
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/std/oar/batch_job_bashrc
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/std/oar/etc/security/access.conf
(100%)
rename
grid5000/steps/data/setup/puppet/modules/env/files/std/oar/default_oar-node =>
image/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/oar-node-service
(69%)
create mode 100644
image/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/oar-node.service.override
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/std/oar/oar_sshclient_config
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/std/oar/var/lib/oar/access.conf
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/std/sudo-g5k/id_rsa_sudo-g5k
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/std/sudo-g5k/id_rsa_sudo-g5k.pub
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/version (100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/xen/xen/id_rsa (100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/xen/xen/id_rsa.pub
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/xen/xen/random_mac
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/xen/xen/xen-g5k (100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/xen/xen/xen-g5k.service
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/files/xen/xen/xend-config.sxp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/lib/facter/installed_kernelreleases.rb
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/base.pp (100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/base/add_ca_grid5000.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/base/configure_dhclient.pp
(100%)
create mode 100644
image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/configure_ip_over_infiniband.pp
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/base/configure_kexec.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/base/configure_omnipath.pp
(85%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/base/disable_ndctl_monitor.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/base/disable_nvme_multipath.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/base/do_not_clean_tmp.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/base/enable_cpufreq_with_performance_governor.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/base/enable_userns.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/base/increase_ssh_maxstartups.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/base/install_and_disable_ganglia.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/base/tcp_tuning_for_10gbe.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/base/unlimited_memlock_for_infiniband.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/big.pp (100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/big/configure_amd_gpu.pp
(77%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/big/configure_initramfs.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/big/configure_kvm.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/cuda.pp
(92%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/drivers.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/ganglia.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/modules.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/prometheus.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/services.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/big/configure_postfix.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/big/configure_sshfs.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/big/install_beegfs.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/big/install_g5k_jupyterlab.pp
(63%)
create mode 100644
image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_openmpi.pp
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/big/install_prometheus_exporters.pp
(100%)
create mode 100644
image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_smartd.pp
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/big/install_snmp_tools.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/big/prepare_kernel_module_build.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/common/apt_pinning.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/common/g5kpackages.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/common/software_versions.pp
(74%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/commonpackages.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/init.pp (100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/min.pp (100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/min/add_image_version_in_etc.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/min/configure_kernel_and_blacklist_some_modules.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/min/configure_network_and_install_drivers.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/min/generate_etc_motd.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/min/install_and_configure_locales.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/min/install_and_configure_ssh.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/min/install_cpu_microcode.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/min/install_metapackage.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/min/install_tgz_g5k.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/min/kernel/initramfs.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/min/kernel/modules.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/min/kernel/remove_old.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/min/kernel/setup_links.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/min/set_root_password.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/min/set_timezone_to_europe_paris.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/nfs.pp (100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/nfs/configure_iscsi.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/nfs/configure_ldap.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/nfs/configure_module_path.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/nfs/configure_ntp.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/nfs/install_nfs_requirements.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/nfs/install_osirim_requirements.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/nfs/install_storage5k_requirements.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/std.pp (96%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/std/add_g5kcode_to_path.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/std/configure_g5kdiskmanagerbackend.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/std/configure_g5kmanager.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/std/configure_g5kpmemmanager.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/std/configure_oar_client.pp
(91%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/std/configure_rsyslog_remote.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/std/dell.pp (100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/std/dell/params.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/std/disable_lvm_pvscan.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/std/g5k_generator.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/std/install_g5kchecks.pp
(68%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/std/install_g5ksubnets.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/std/install_hwraid_apt_source.pp
(100%)
create mode 100644
image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_kameleon.pp
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/std/install_libguestfs_backport.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/std/install_megacli.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/std/install_sudog5k.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/std/ipmitool.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/std/nvidia_reset_mig.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/xen.pp (100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/xen/configure_xen.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/manifests/xen/install_grub.pp
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/templates/base/omnipath/scibian.key.erb
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/templates/common/apt_pinning.erb
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/templates/min/motd.erb
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/templates/nfs/ldap/common-account.erb
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/templates/nfs/ntp/ntp.conf.erb
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/templates/std/dell/linux.dell.com.key.erb
(100%)
rename {grid5000 =>
image/grid5000}/steps/data/setup/puppet/modules/env/templates/std/hwraid/hwraid.le-vert.net.key.erb
(100%)
rename {default => image/grid5000}/steps/disable_checkpoint.yaml (100%)
rename {default => image/grid5000}/steps/enable_checkpoint.yaml (100%)
rename {default => image/grid5000}/steps/env/bashrc (100%)
rename {grid5000 => image/grid5000}/steps/env/functions.sh (100%)
rename {grid5000 => image/grid5000}/steps/export/debian/clean_dhcp_leases.yaml
(100%)
rename {grid5000 => image/grid5000}/steps/export/do_qcow2_finish_works.yaml
(100%)
rename {grid5000 => image/grid5000}/steps/export/export_g5k.yaml (93%)
rename {default => image/grid5000}/steps/export/save_appliance_VM.yaml (85%)
rename {default => image/grid5000}/steps/setup/debian/clean_system.yaml (100%)
rename {grid5000 =>
image/grid5000}/steps/setup/debian/configure_apt_sources.yaml (100%)
rename {grid5000 => image/grid5000}/steps/setup/debian/configure_system.yaml
(100%)
rename {grid5000 => image/grid5000}/steps/setup/debian/install_packages.yaml
(100%)
rename {default => image/grid5000}/steps/setup/debian/minimal_install.yaml
(100%)
rename {grid5000 => image/grid5000}/steps/setup/debian/run_orchestrator.yaml
(100%)
rename {grid5000 => image/grid5000}/steps/setup/debian/setup_orchestrator.yaml
(100%)
create mode 100644 image/taler-debian11.yaml
delete mode 100644 jfed/README
delete mode 100644 jfed/twoboot.rspec
delete mode 100644 steps/setup/taler_install.yaml
delete mode 100644 steps/setup/taler_install.yaml~
delete mode 100644 update.sh
diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..4c49bd7
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1 @@
+.env
diff --git a/README b/README
index 3898e58..17f45f5 100644
--- a/README
+++ b/README
@@ -1,28 +1,23 @@
Use
-$ kameleon build debian11-taler.yaml --enable-cache
+$ kameleon build taler-debian11
to build the image.
-Note: theg5k_tgz_file option is not propagated to the ".dsc" file,
-so we must adjust it manually (in build/debian11-taler/debian11-taler.dsc)
-and set 'file' to:
-
- file: http://public.lyon.grid5000.fr/~grothoff/debian11-taler.tar.zst
-
+Note: Maybe the user needs to be adjusted in g5k_tar_path
+i.e. http://public.lyon.grid5000.fr/~<G5K_USER>/taler-debian11.tar.zst
Afterwards:
-$ cd build/debian11-taler
-$ scp debian11-taler.dsc debian11-taler.tar.zst \
- grothoff@access.grid5000.fr:lyon/public/
-
+$ cd build/taler-debian11
+$ scp taler-debian11.dsc taler-debian11.tar.zst \
+ <G5K_USER>@access.grid5000.fr:lyon/public/
Then we can use
- http://public.lyon.grid5000.fr/~grothoff/debian11-taler.dsc
+ http://public.lyon.grid5000.fr/~<G5K_USER>/taler-debian11.dsc
for the disk image in jfed.
-jFed example configurations are in the jFed/ directory.
+jFed example configurations are in the jfed/ directory.
diff --git a/at-boot.sh b/at-boot.sh
deleted file mode 100644
index 578425f..0000000
--- a/at-boot.sh
+++ /dev/null
@@ -1,9 +0,0 @@
-#!/bin/bash
-# Run by the gridboot.service at boot. Updates
-# the grid5k.git and then runs a script from that
-# Git.
-#
-cd /root/grid5k
-git pull
-chmod +x update.sh
-exec /root/grid5k/update.sh
diff --git a/configs/etc/default/prometheus b/configs/etc/default/prometheus
new file mode 100644
index 0000000..b9b82ca
--- /dev/null
+++ b/configs/etc/default/prometheus
@@ -0,0 +1,112 @@
+ARGS="--config.file=/etc/monitor/prometheus.yaml
--web.listen-address=0.0.0.0:8080"
+
+# Prometheus supports the following options:
+# --config.file="/etc/prometheus/prometheus.yml"
+# Prometheus configuration file path.
+# --web.listen-address="0.0.0.0:9090"
+# Address to listen on for UI, API, and telemetry.
+# --web.read-timeout=5m Maximum duration before timing out read of the
+# request, and closing idle connections.
+# --web.max-connections=512 Maximum number of simultaneous connections.
+# --web.external-url=<URL> The URL under which Prometheus is externally
+# reachable (for example, if Prometheus is served
+# via a reverse proxy). Used for generating
+# relative and absolute links back to Prometheus
+# itself. If the URL has a path portion, it will
+# be used to prefix all HTTP endpoints served by
+# Prometheus. If omitted, relevant URL components
+# will be derived automatically.
+# --web.route-prefix=<path> Prefix for the internal routes of web endpoints.
+# Defaults to path of --web.external-url.
+# --web.local-assets="/usr/share/prometheus/web/"
+# Path to static asset/templates directory.
+# --web.user-assets=<path> Path to user asset directory, available at
+# /user.
+# --web.enable-lifecycle Enable shutdown and reload via HTTP request.
+# --web.enable-admin-api Enable API endpoints for admin control actions.
+# --web.console.templates="/etc/prometheus/consoles"
+# Path to the console template directory,
+# available at /consoles.
+# --web.console.libraries="/etc/prometheus/console_libraries"
+# Path to the console library directory.
+# --web.page-title="Prometheus Time Series Collection and Processing Server"
+# Document title of Prometheus instance.
+# --web.cors.origin=".*" Regex for CORS origin. It is fully anchored.
+# Example: 'https?://(domain1|domain2)\.com'
+# --storage.tsdb.path="/var/lib/prometheus/metrics2/"
+# Base path for metrics storage.
+# --storage.tsdb.retention=15d
+# [DEPRECATED] How long to retain samples in
+# storage. This flag has been deprecated, use
+# "storage.tsdb.retention.time" instead
+# --storage.tsdb.retention.time=15d
+# How long to retain samples in storage. When this
+# flag is set it overrides
+# "storage.tsdb.retention".
+# If neither this flag nor "storage.tsdb.retention"
+# nor "storage.tsdb.retention.size" is set, the
+# retention time defaults to 15d.
+# Units Supported: y, w, d, h, m, s, ms.
+# --storage.tsdb.retention.size=
+# [EXPERIMENTAL] Maximum number of bytes that can
+# be stored for blocks. Units supported: KB, MB,
+# GB, TB, PB. This flag is experimental and can be
+# changed in future releases.
+# --storage.tsdb.use-lockfile
+# Create a lockfile in data directory.
+# --storage.tsdb.allow-overlapping-blocks
+# [EXPERIMENTAL] Allow overlapping blocks, which
+# in turn enables vertical compaction and
+# vertical query merge.
+# --storage.tsdb.wal-compression
+# Compress the tsdb WAL.
+# --storage.remote.flush-deadline=<duration>
+# How long to wait flushing sample on shutdown or
+# config reload.
+# --storage.remote.read-sample-limit=5e7
+# Maximum overall number of samples to return via
+# the remote read interface, in a single query. 0
+# means no limit. This limit is ignored for
+# streamed response types.
+# --storage.remote.read-concurrent-limit=10
+# Maximum number of concurrent remote read calls.
+# 0 means no limit.
+# --storage.remote.read-max-bytes-in-frame=1048576
+# Maximum number of bytes in a single frame for
+# streaming remote read response types before
+# marshalling. Note that client might have limit on
+# frame size as well. 1MB as recommended by
+# protobuf by default.
+# --rules.alert.for-outage-tolerance=1h
+# Max time to tolerate prometheus outage for
+# restoring "for" state of alert.
+# --rules.alert.for-grace-period=10m
+# Minimum duration between alert and restored "for"
+# state. This is maintained only for alerts with
+# configured "for" time greater than grace period.
+# --rules.alert.resend-delay=1m
+# Minimum amount of time to wait before resending
+# an alert to Alertmanager.
+# --alertmanager.notification-queue-capacity=10000
+# The capacity of the queue for pending
+# Alertmanager notifications.
+# --alertmanager.timeout=10s
+# Timeout for sending alerts to Alertmanager.
+# --query.lookback-delta=5m The maximum lookback duration for retrieving
+# metrics during expression evaluations and
+# federation.
+# --query.timeout=2m Maximum time a query may take before being
+# aborted.
+# --query.max-concurrency=20
+# Maximum number of queries executed concurrently.
+# --query.max-samples=50000000
+# Maximum number of samples a single query can load
+# into memory. Note that queries will fail if they
+# try to load more samples than this into memory,
+# so this also limits the number of samples a query
+# can return.
+# --log.level=info Only log messages with the given severity or
+# above. One of: [debug, info, warn, error]
+# --log.format=logfmt Output format of log messages. One of: [logfmt,
+# json]
+
diff --git a/configs/etc/default/prometheus-nginx-exporter
b/configs/etc/default/prometheus-nginx-exporter
new file mode 100644
index 0000000..57da070
--- /dev/null
+++ b/configs/etc/default/prometheus-nginx-exporter
@@ -0,0 +1,38 @@
+ARGS="-nginx.scrape-uri <PROXY_URL_HERE>"
+
+# Prometheus-nginx-exporter supports the following options:
+# -nginx.plus
+# Start the exporter for NGINX Plus. By default, the exporter is started
+# for NGINX. The default value can be overwritten by NGINX_PLUS environment
+# variable.
+# -nginx.retries uint
+# A number of retries the exporter will make on start to connect to the
+# NGINX stub_status page/NGINX Plus API before exiting with an error. The
+# default value can be overwritten by NGINX_RETRIES environment variable.
+# -nginx.retry-interval value
+# An interval between retries to connect to the NGINX stub_status
+# page/NGINX Plus API on start. The default value can be overwritten by
+# NGINX_RETRY_INTERVAL environment variable. (default 5s)
+# -nginx.scrape-uri string
+# A URI or unix domain socket path for scraping NGINX or NGINX Plus metrics.
+# For NGINX, the stub_status page must be available through the URI. For
+# NGINX Plus -- the API. The default value can be overwritten by SCRAPE_URI
+# environment variable. (default "http://127.0.0.1:8080/stub_status";)
+# -nginx.ssl-verify
+# Perform SSL certificate verification. The default value can be overwritten
+# by SSL_VERIFY environment variable. (default true)
+# -nginx.timeout value
+# A timeout for scraping metrics from NGINX or NGINX Plus. The default value
+# can be overwritten by TIMEOUT environment variable. (default 5s)
+# -prometheus.const-labels value
+# A comma separated list of constant labels that will be used in every
+# metric. Format is label1=value1,label2=value2... The default value can be
+# overwritten by CONST_LABELS environment variable.
+# -web.listen-address string
+# An address or unix domain socket path to listen on for web interface and
+# telemetry. The default value can be overwritten by LISTEN_ADDRESS
+# environment variable. (default ":9113")
+# -web.telemetry-path string
+# A path under which to expose metrics. The default value can be overwritten
+# by TELEMETRY_PATH environment variable. (default "/metrics")
+
diff --git a/configs/etc/default/prometheus-postgres-exporter
b/configs/etc/default/prometheus-postgres-exporter
new file mode 100644
index 0000000..15722ac
--- /dev/null
+++ b/configs/etc/default/prometheus-postgres-exporter
@@ -0,0 +1,19 @@
+# Connection string for the PostgreSQL database. You need to either connect as
+# superuser, or create a user with enough rights, as described in
+# /usr/share/doc/prometheus-postgres-exporter/README.Debian
+
+# DATA_SOURCE_NAME='postgresql://login:password@hostname:port/'
+# DATA_SOURCE_NAME='user=prometheus host=/run/postgresql dbname=postgres'
+DATA_SOURCE_NAME='<DB_URL_HERE>'
+
+# Set the command-line arguments to pass to the server.
+ARGS='--extend.query-path=/etc/monitor/postgres-exporter.yaml'
+
+# Available flags:
+# --web.listen-address=":9187" Address to listen on for web interface and
telemetry.
+# --web.telemetry-path="/metrics"
+# Path under which to expose metrics.
+# --extend.query-path="" Path to custom queries to run.
+# --log.level="info" Only log messages with the given severity or
above. Valid levels: [debug, info, warn, error, fatal]
+# --log.format="logger:stderr" Set the log target and format. Example:
"logger:syslog?appname=bob&local=7" or "logger:stdout?json=true"
+
diff --git a/configs/etc/default/taler-exchange
b/configs/etc/default/taler-exchange
new file mode 100644
index 0000000..adcb212
--- /dev/null
+++ b/configs/etc/default/taler-exchange
@@ -0,0 +1 @@
+ARGS="<ARGUMENTS_HERE>"
diff --git a/configs/etc/logrotate.d/taler b/configs/etc/logrotate.d/taler
new file mode 100644
index 0000000..bc8e16c
--- /dev/null
+++ b/configs/etc/logrotate.d/taler
@@ -0,0 +1,12 @@
+/tmp/taler/*log {
+ daily
+ missingok
+ rotate 5
+ notifempty
+ create
+ maxsize 1000M
+ compress
+ postrotate
+ systemctl kill -s HUP rsyslog.service
+ endscript
+}
diff --git a/configs/etc/monitor/loki.yaml b/configs/etc/monitor/loki.yaml
new file mode 100644
index 0000000..e00581b
--- /dev/null
+++ b/configs/etc/monitor/loki.yaml
@@ -0,0 +1,65 @@
+auth_enabled: false
+
+server:
+ http_listen_port: 80
+ grpc_listen_port: 9096
+
+querier:
+ max_concurrent: 40
+
+ingester:
+ wal:
+ enabled: true
+ dir: /tmp/wal
+ lifecycler:
+ address: 127.0.0.1
+ ring:
+ kvstore:
+ store: inmemory
+ replication_factor: 1
+ final_sleep: 0s
+ chunk_idle_period: 1h # Any chunk not receiving new logs in this time
will be flushed
+ max_chunk_age: 1h # All chunks will be flushed when they hit this
age, default is 1h
+ chunk_target_size: 1048576 # Loki will attempt to build chunks up to 1.5MB,
flushing first if chunk_idle_period or max_chunk_age is reached first
+ chunk_retain_period: 30s # Must be greater than index read cache TTL if
using an index cache (Default index read cache TTL is 5m)
+ max_transfer_retries: 0 # Chunk transfers disabled
+
+schema_config:
+ configs:
+ - from: 2020-10-24
+ store: boltdb-shipper
+ object_store: filesystem
+ schema: v11
+ index:
+ prefix: index_
+ period: 24h
+
+storage_config:
+ boltdb_shipper:
+ active_index_directory: /tmp/loki/boltdb-shipper-active
+ cache_location: /tmp/loki/boltdb-shipper-cache
+ cache_ttl: 24h
+ shared_store: filesystem
+ filesystem:
+ directory: /tmp/loki/chunks
+
+compactor:
+ working_directory: /tmp/loki/boltdb-shipper-compactor
+ shared_store: filesystem
+
+limits_config:
+ reject_old_samples: true
+ reject_old_samples_max_age: 168h
+ max_query_parallelism: 30
+ max_streams_per_user: 0 # unlimited
+ ingestion_burst_size_mb: 10000 # high enough to simulate unlimited
+ ingestion_rate_mb: 10000
+ per_stream_rate_limit: 1GB
+ per_stream_rate_limit_burst: 4GB
+
+chunk_store_config:
+ max_look_back_period: 0s
+
+table_manager:
+ retention_deletes_enabled: false
+ retention_period: 0s
diff --git a/configs/etc/monitor/node-exporters.yaml.tpl
b/configs/etc/monitor/node-exporters.yaml.tpl
new file mode 100644
index 0000000..99b79ed
--- /dev/null
+++ b/configs/etc/monitor/node-exporters.yaml.tpl
@@ -0,0 +1,26 @@
+ - job_name: 'nodes'
+ static_configs:
+ - labels:
+ component: 'database'
+ targets:
+ - 'db.perf.taler:9100'
+ - labels:
+ component: 'bank'
+ targets:
+ - 'bank.perf.taler:9100'
+ - labels:
+ component: 'exchange'
+ targets:
+ - 'exch.perf.taler:9100'
+ - labels:
+ component: 'proxy'
+ targets:
+ - 'proxy.perf.taler:9100'
+ - labels:
+ component: 'wallet'
+ targets:
+ # <WALLETS_HERE>
+ - labels:
+ component: 'monitor'
+ targets:
+ - 'monitor.perf.taler:9100'
diff --git a/configs/etc/monitor/postgres-exporter.yaml
b/configs/etc/monitor/postgres-exporter.yaml
new file mode 100644
index 0000000..e473b6a
--- /dev/null
+++ b/configs/etc/monitor/postgres-exporter.yaml
@@ -0,0 +1,275 @@
+pg_replication:
+ query: "SELECT CASE WHEN NOT pg_is_in_recovery() THEN 0 ELSE GREATEST (0,
EXTRACT(EPOCH FROM (now() - pg_last_xact_replay_timestamp()))) END AS lag"
+ master: true
+ metrics:
+ - lag:
+ usage: "GAUGE"
+ description: "Replication lag behind master in seconds"
+
+pg_postmaster:
+ query: "SELECT pg_postmaster_start_time as start_time_seconds from
pg_postmaster_start_time()"
+ master: true
+ metrics:
+ - start_time_seconds:
+ usage: "GAUGE"
+ description: "Time at which postmaster started"
+
+pg_stat_user_tables:
+ query: |
+ SELECT
+ current_database() datname,
+ schemaname,
+ relname,
+ seq_scan,
+ seq_tup_read,
+ idx_scan,
+ idx_tup_fetch,
+ n_tup_ins,
+ n_tup_upd,
+ n_tup_del,
+ n_tup_hot_upd,
+ n_live_tup,
+ n_dead_tup,
+ n_mod_since_analyze,
+ COALESCE(last_vacuum, '1970-01-01Z') as last_vacuum,
+ COALESCE(last_autovacuum, '1970-01-01Z') as last_autovacuum,
+ COALESCE(last_analyze, '1970-01-01Z') as last_analyze,
+ COALESCE(last_autoanalyze, '1970-01-01Z') as last_autoanalyze,
+ vacuum_count,
+ autovacuum_count,
+ analyze_count,
+ autoanalyze_count
+ FROM
+ pg_stat_user_tables
+ metrics:
+ - datname:
+ usage: "LABEL"
+ description: "Name of current database"
+ - schemaname:
+ usage: "LABEL"
+ description: "Name of the schema that this table is in"
+ - relname:
+ usage: "LABEL"
+ description: "Name of this table"
+ - seq_scan:
+ usage: "COUNTER"
+ description: "Number of sequential scans initiated on this table"
+ - seq_tup_read:
+ usage: "COUNTER"
+ description: "Number of live rows fetched by sequential scans"
+ - idx_scan:
+ usage: "COUNTER"
+ description: "Number of index scans initiated on this table"
+ - idx_tup_fetch:
+ usage: "COUNTER"
+ description: "Number of live rows fetched by index scans"
+ - n_tup_ins:
+ usage: "COUNTER"
+ description: "Number of rows inserted"
+ - n_tup_upd:
+ usage: "COUNTER"
+ description: "Number of rows updated"
+ - n_tup_del:
+ usage: "COUNTER"
+ description: "Number of rows deleted"
+ - n_tup_hot_upd:
+ usage: "COUNTER"
+ description: "Number of rows HOT updated (i.e., with no separate index
update required)"
+ - n_live_tup:
+ usage: "GAUGE"
+ description: "Estimated number of live rows"
+ - n_dead_tup:
+ usage: "GAUGE"
+ description: "Estimated number of dead rows"
+ - n_mod_since_analyze:
+ usage: "GAUGE"
+ description: "Estimated number of rows changed since last analyze"
+ - last_vacuum:
+ usage: "GAUGE"
+ description: "Last time at which this table was manually vacuumed (not
counting VACUUM FULL)"
+ - last_autovacuum:
+ usage: "GAUGE"
+ description: "Last time at which this table was vacuumed by the
autovacuum daemon"
+ - last_analyze:
+ usage: "GAUGE"
+ description: "Last time at which this table was manually analyzed"
+ - last_autoanalyze:
+ usage: "GAUGE"
+ description: "Last time at which this table was analyzed by the
autovacuum daemon"
+ - vacuum_count:
+ usage: "COUNTER"
+ description: "Number of times this table has been manually vacuumed
(not counting VACUUM FULL)"
+ - autovacuum_count:
+ usage: "COUNTER"
+ description: "Number of times this table has been vacuumed by the
autovacuum daemon"
+ - analyze_count:
+ usage: "COUNTER"
+ description: "Number of times this table has been manually analyzed"
+ - autoanalyze_count:
+ usage: "COUNTER"
+ description: "Number of times this table has been analyzed by the
autovacuum daemon"
+
+pg_statio_user_tables:
+ query: "SELECT current_database() datname, schemaname, relname,
heap_blks_read, heap_blks_hit, idx_blks_read, idx_blks_hit, toast_blks_read,
toast_blks_hit, tidx_blks_read, tidx_blks_hit FROM pg_statio_user_tables"
+ metrics:
+ - datname:
+ usage: "LABEL"
+ description: "Name of current database"
+ - schemaname:
+ usage: "LABEL"
+ description: "Name of the schema that this table is in"
+ - relname:
+ usage: "LABEL"
+ description: "Name of this table"
+ - heap_blks_read:
+ usage: "COUNTER"
+ description: "Number of disk blocks read from this table"
+ - heap_blks_hit:
+ usage: "COUNTER"
+ description: "Number of buffer hits in this table"
+ - idx_blks_read:
+ usage: "COUNTER"
+ description: "Number of disk blocks read from all indexes on this
table"
+ - idx_blks_hit:
+ usage: "COUNTER"
+ description: "Number of buffer hits in all indexes on this table"
+ - toast_blks_read:
+ usage: "COUNTER"
+ description: "Number of disk blocks read from this table's TOAST table
(if any)"
+ - toast_blks_hit:
+ usage: "COUNTER"
+ description: "Number of buffer hits in this table's TOAST table (if
any)"
+ - tidx_blks_read:
+ usage: "COUNTER"
+ description: "Number of disk blocks read from this table's TOAST table
indexes (if any)"
+ - tidx_blks_hit:
+ usage: "COUNTER"
+ description: "Number of buffer hits in this table's TOAST table
indexes (if any)"
+
+pg_database:
+ query: "SELECT pg_database.datname, pg_database_size(pg_database.datname) as
size_bytes FROM pg_database"
+ master: true
+ cache_seconds: 30
+ metrics:
+ - datname:
+ usage: "LABEL"
+ description: "Name of the database"
+ - size_bytes:
+ usage: "GAUGE"
+ description: "Disk space used by the database"
+
+pg_stat_statements:
+ query: "SELECT t2.rolname, t3.datname, queryid, calls, total_exec_time /
1000 as total_time_seconds, min_exec_time / 1000 as min_time_seconds,
max_exec_time / 1000 as max_time_seconds, mean_exec_time / 1000 as
mean_time_seconds, stddev_exec_time / 1000 as stddev_time_seconds, rows,
shared_blks_hit, shared_blks_read, shared_blks_dirtied, shared_blks_written,
local_blks_hit, local_blks_read, local_blks_dirtied, local_blks_written,
temp_blks_read, temp_blks_written, blk_read_time / 1000 a [...]
+ master: true
+ metrics:
+ - rolname:
+ usage: "LABEL"
+ description: "Name of user"
+ - datname:
+ usage: "LABEL"
+ description: "Name of database"
+ - queryid:
+ usage: "LABEL"
+ description: "Query ID"
+ - calls:
+ usage: "COUNTER"
+ description: "Number of times executed"
+ - total_time_seconds:
+ usage: "COUNTER"
+ description: "Total time spent in the statement, in milliseconds"
+ - min_time_seconds:
+ usage: "GAUGE"
+ description: "Minimum time spent in the statement, in milliseconds"
+ - max_time_seconds:
+ usage: "GAUGE"
+ description: "Maximum time spent in the statement, in milliseconds"
+ - mean_time_seconds:
+ usage: "GAUGE"
+ description: "Mean time spent in the statement, in milliseconds"
+ - stddev_time_seconds:
+ usage: "GAUGE"
+ description: "Population standard deviation of time spent in the
statement, in milliseconds"
+ - rows:
+ usage: "COUNTER"
+ description: "Total number of rows retrieved or affected by the
statement"
+ - shared_blks_hit:
+ usage: "COUNTER"
+ description: "Total number of shared block cache hits by the statement"
+ - shared_blks_read:
+ usage: "COUNTER"
+ description: "Total number of shared blocks read by the statement"
+ - shared_blks_dirtied:
+ usage: "COUNTER"
+ description: "Total number of shared blocks dirtied by the statement"
+ - shared_blks_written:
+ usage: "COUNTER"
+ description: "Total number of shared blocks written by the statement"
+ - local_blks_hit:
+ usage: "COUNTER"
+ description: "Total number of local block cache hits by the statement"
+ - local_blks_read:
+ usage: "COUNTER"
+ description: "Total number of local blocks read by the statement"
+ - local_blks_dirtied:
+ usage: "COUNTER"
+ description: "Total number of local blocks dirtied by the statement"
+ - local_blks_written:
+ usage: "COUNTER"
+ description: "Total number of local blocks written by the statement"
+ - temp_blks_read:
+ usage: "COUNTER"
+ description: "Total number of temp blocks read by the statement"
+ - temp_blks_written:
+ usage: "COUNTER"
+ description: "Total number of temp blocks written by the statement"
+ - blk_read_time_seconds:
+ usage: "COUNTER"
+ description: "Total time the statement spent reading blocks, in
milliseconds (if track_io_timing is enabled, otherwise zero)"
+ - blk_write_time_seconds:
+ usage: "COUNTER"
+ description: "Total time the statement spent writing blocks, in
milliseconds (if track_io_timing is enabled, otherwise zero)"
+
+pg_process_idle:
+ query: |
+ WITH
+ metrics AS (
+ SELECT
+ application_name,
+ SUM(EXTRACT(EPOCH FROM (CURRENT_TIMESTAMP -
state_change))::bigint)::float AS process_idle_seconds_sum,
+ COUNT(*) AS process_idle_seconds_count
+ FROM pg_stat_activity
+ WHERE state = 'idle'
+ GROUP BY application_name
+ ),
+ buckets AS (
+ SELECT
+ application_name,
+ le,
+ SUM(
+ CASE WHEN EXTRACT(EPOCH FROM (CURRENT_TIMESTAMP - state_change))
<= le
+ THEN 1
+ ELSE 0
+ END
+ )::bigint AS bucket
+ FROM
+ pg_stat_activity,
+ UNNEST(ARRAY[1, 2, 5, 15, 30, 60, 90, 120, 300]) AS le
+ GROUP BY application_name, le
+ ORDER BY application_name, le
+ )
+ SELECT
+ application_name,
+ process_idle_seconds_sum as seconds_sum,
+ process_idle_seconds_count as seconds_count,
+ ARRAY_AGG(le) AS seconds,
+ ARRAY_AGG(bucket) AS seconds_bucket
+ FROM metrics JOIN buckets USING (application_name)
+ GROUP BY 1, 2, 3
+ metrics:
+ - application_name:
+ usage: "LABEL"
+ description: "Application Name"
+ - seconds:
+ usage: "HISTOGRAM"
+ description: "Idle time of server processes"
+
diff --git a/configs/etc/monitor/prometheus.yaml
b/configs/etc/monitor/prometheus.yaml
new file mode 100644
index 0000000..88e61fd
--- /dev/null
+++ b/configs/etc/monitor/prometheus.yaml
@@ -0,0 +1,23 @@
+global:
+ scrape_interval: 5s
+ evaluation_interval: 5s
+
+rule_files: []
+
+scrape_configs:
+
+ # Self
+ - job_name: 'prometheus'
+ static_configs:
+ - targets: ['127.0.0.1:8080']
+
+ # DB Exporter
+ - job_name: 'database'
+ static_configs:
+ - targets: ['127.0.0.1:9187', '<MERCHANT_HOST_HERE>:9187']
+
+ # Exchange Proxy Exporter
+ - job_name: 'exchange-proxy'
+ static_configs:
+ - targets: ['127.0.0.1:9113']
+
diff --git a/configs/etc/monitor/promtail.yaml
b/configs/etc/monitor/promtail.yaml
new file mode 100644
index 0000000..2cd2a75
--- /dev/null
+++ b/configs/etc/monitor/promtail.yaml
@@ -0,0 +1,21 @@
+server:
+ http_listen_port: 9080
+ grpc_listen_port: 0
+
+positions:
+ filename: /tmp/positions.yaml
+
+clients:
+ - url: http://127.0.0.1/loki/api/v1/push
+
+scrape_configs:
+- job_name: taler
+ syslog:
+ listen_address: 0.0.0.0:1514
+ labels:
+ job: taler
+ relabel_configs:
+ - source_labels: [__syslog_message_app_name]
+ target_label: app
+ - source_labels: [__syslog_message_hostname]
+ target_label: host
diff --git a/configs/etc/nginx/nginx.conf b/configs/etc/nginx/nginx.conf
new file mode 100644
index 0000000..88861cf
--- /dev/null
+++ b/configs/etc/nginx/nginx.conf
@@ -0,0 +1,62 @@
+user www-data;
+worker_processes auto;
+worker_rlimit_nofile 500000;
+pid /run/nginx.pid;
+include /etc/nginx/modules-enabled/*.conf;
+
+events {
+ worker_connections 10000;
+ # multi_accept on;
+}
+
+http {
+
+ ##
+ # Basic Settings
+ ##
+
+ sendfile on;
+ tcp_nopush on;
+ types_hash_max_size 2048;
+ # server_tokens off;
+
+ # server_names_hash_bucket_size 64;
+ # server_name_in_redirect off;
+
+ include /etc/nginx/mime.types;
+ default_type application/octet-stream;
+
+ ##
+ # SSL Settings
+ ##
+
+ ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref:
POODLE
+ ssl_prefer_server_ciphers on;
+
+ ##
+ # Logging Settings
+ ##
+
+ access_log /var/log/nginx/access.log;
+ error_log /var/log/nginx/error.log;
+
+ ##
+ # Gzip Settings
+ ##
+
+ gzip on;
+
+ # gzip_vary on;
+ # gzip_proxied any;
+ # gzip_comp_level 6;
+ # gzip_buffers 16 8k;
+ # gzip_http_version 1.1;
+ # gzip_types text/plain text/css application/json
application/javascript text/xml application/xml application/xml+rss
text/javascript;
+
+ ##
+ # Virtual Host Configs
+ ##
+
+ include /etc/nginx/conf.d/*.conf;
+ include /etc/nginx/sites-enabled/*;
+}
diff --git a/configs/etc/nginx/sites-available/proxy
b/configs/etc/nginx/sites-available/proxy
new file mode 100644
index 0000000..4366d21
--- /dev/null
+++ b/configs/etc/nginx/sites-available/proxy
@@ -0,0 +1,40 @@
+upstream exchange {
+ least_conn;
+ # EXCHANGE_PROCESSES e.g.
+ # server exch.perf.taler:8080;
+ # server exch.perf.taler:80;
+ # <SERVERS_HERE>
+}
+
+log_format taler 'm=$request_method uri=$request_uri s=$status '
+ 'uct=$upstream_connect_time '
+ 'urt=$upstream_response_time '
+ 'rt=$request_time';
+
+map $request_method $log_line {
+ 'HEAD' 0;
+ default 1;
+}
+
+server {
+ listen 80;
+ listen [::]:80;
+
+ server_name localhost;
+
+ access_log
syslog:server=localhost,facility=user,tag=taler_proxy,severity=info taler
if=$log_line;
+
+ allow 172.16.0.0/12;
+ deny all;
+
+ location / {
+ proxy_pass http://exchange;
+ proxy_redirect off;
+ }
+
+ location /stub_status {
+ stub_status;
+ access_log off;
+ }
+}
+
diff --git a/configs/etc/rsyslog.d/taler.conf b/configs/etc/rsyslog.d/taler.conf
new file mode 100644
index 0000000..34bcc72
--- /dev/null
+++ b/configs/etc/rsyslog.d/taler.conf
@@ -0,0 +1,23 @@
+module(load="omprog")
+# Need to load due to parser error https://github.com/grafana/loki/issues/1783
+module(load="mmutf8fix")
+
+# Set the filename for the log backup
+$template FileName,"/tmp/taler/%programname%.log"
+
+if ($programname startswith 'taler' or $syslogtag startswith 'taler') then {
+ action(type="mmutf8fix" replacementChar="?")
+#
https://grafana.com/docs/loki/latest/clients/promtail/scraping/#rsyslog-output-configuration
+ action(type="omfwd"
+ protocol="tcp"
+ target="monitor.perf.taler"
+ port="1514"
+ Template="RSYSLOG_SyslogProtocol23Format"
+ TCP_Framing="octet-counted"
+ KeepAlive="on")
+ # Write all logs also to the file - will periodcally be rotaded by
taler-logbackup.service
+ *.* ?FileName
+ # Do not do any further steps with logs which matched this rule
+ & stop
+}
+
diff --git a/configs/etc/taler/conf.d/exchange-business.conf
b/configs/etc/taler/conf.d/exchange-business.conf
new file mode 100755
index 0000000..0e6dd1e
--- /dev/null
+++ b/configs/etc/taler/conf.d/exchange-business.conf
@@ -0,0 +1,36 @@
+# Configuration for business-level aspects of the exchange.
+
+[exchange]
+
+# Here you MUST add the master public key of the offline system
+# which you can get using `taler-exchange-offline setup`.
+# This is just an example, your key will be different!
+# MASTER_PUBLIC_KEY = YE6Q6TR1EDB7FD0S68TGDZGF1P0GHJD2S0XVV8R2S62MYJ6HJ4ZG
+MASTER_PUBLIC_KEY = <MASTER_KEY_HERE>
+
+# Publicly visible base URL of the exchange.
+# BASE_URL = https://example.com/
+BASE_URL = <BASE_URL_HERE>
+
+# For your terms of service and privacy policy, you should specify
+# an Etag that must be updated whenever there are significant
+# changes to either document. The format is up to you, what matters
+# is that the value is updated and never re-used. See the HTTP
+# specification on Etags.
+# TERMS_ETAG =
+# PRIVACY_ETAG =
+
+# Bank accounts used by the exchange should be specified here:
+[exchange-account-1]
+
+enable_credit = yes
+enable_debit = yes
+
+# Account identifier in the form of an RFC-8905 payto:// URI.
+# For SEPA, looks like payto://sepa/$IBAN?receiver-name=$NAME
+# Make sure to URL-encode spaces in $NAME!
+payto_uri = payto://x-taler-bank/<BANK_HOST_HERE>/Exchange
+
+# Credentials to access the account are in a separate
+# config file with restricted permissions.
+@inline-secret@ exchange-accountcredentials-1
../secrets/exchange-accountcredentials.secret.conf
diff --git a/configs/etc/taler/conf.d/exchange-coins.conf
b/configs/etc/taler/conf.d/exchange-coins.conf
new file mode 100755
index 0000000..f1c6f5c
--- /dev/null
+++ b/configs/etc/taler/conf.d/exchange-coins.conf
@@ -0,0 +1,158 @@
+# Coin configuration for the exchange.
+# Should be placed in "/etc/taler/conf.d/exchange-coins.conf".
+
+[COIN-KUDOS-n1-t1633183611]
+VALUE = KUDOS:0.01
+DURATION_WITHDRAW = 7 days
+DURATION_SPEND = 2 years
+DURATION_LEGAL = 6 years
+FEE_WITHDRAW = KUDOS:0
+FEE_DEPOSIT = KUDOS:0.01
+FEE_REFRESH = KUDOS:0
+FEE_REFUND = KUDOS:0
+RSA_KEYSIZE = 2048
+
+[COIN-KUDOS-n2-t1633183611]
+VALUE = KUDOS:0.02
+DURATION_WITHDRAW = 7 days
+DURATION_SPEND = 2 years
+DURATION_LEGAL = 6 years
+FEE_WITHDRAW = KUDOS:0
+FEE_DEPOSIT = KUDOS:0.01
+FEE_REFRESH = KUDOS:0
+FEE_REFUND = KUDOS:0
+RSA_KEYSIZE = 2048
+
+[COIN-KUDOS-n3-t1633183611]
+VALUE = KUDOS:0.04
+DURATION_WITHDRAW = 7 days
+DURATION_SPEND = 2 years
+DURATION_LEGAL = 6 years
+FEE_WITHDRAW = KUDOS:0
+FEE_DEPOSIT = KUDOS:0.01
+FEE_REFRESH = KUDOS:0
+FEE_REFUND = KUDOS:0
+RSA_KEYSIZE = 2048
+
+[COIN-KUDOS-n4-t1633183611]
+VALUE = KUDOS:0.08
+DURATION_WITHDRAW = 7 days
+DURATION_SPEND = 2 years
+DURATION_LEGAL = 6 years
+FEE_WITHDRAW = KUDOS:0
+FEE_DEPOSIT = KUDOS:0.01
+FEE_REFRESH = KUDOS:0
+FEE_REFUND = KUDOS:0
+RSA_KEYSIZE = 2048
+
+[COIN-KUDOS-n5-t1633183611]
+VALUE = KUDOS:0.16
+DURATION_WITHDRAW = 7 days
+DURATION_SPEND = 2 years
+DURATION_LEGAL = 6 years
+FEE_WITHDRAW = KUDOS:0
+FEE_DEPOSIT = KUDOS:0.01
+FEE_REFRESH = KUDOS:0
+FEE_REFUND = KUDOS:0
+RSA_KEYSIZE = 2048
+
+[COIN-KUDOS-n6-t1633183611]
+VALUE = KUDOS:0.32
+DURATION_WITHDRAW = 7 days
+DURATION_SPEND = 2 years
+DURATION_LEGAL = 6 years
+FEE_WITHDRAW = KUDOS:0
+FEE_DEPOSIT = KUDOS:0.01
+FEE_REFRESH = KUDOS:0
+FEE_REFUND = KUDOS:0
+RSA_KEYSIZE = 2048
+
+[COIN-KUDOS-n7-t1633183611]
+VALUE = KUDOS:0.64
+DURATION_WITHDRAW = 7 days
+DURATION_SPEND = 2 years
+DURATION_LEGAL = 6 years
+FEE_WITHDRAW = KUDOS:0
+FEE_DEPOSIT = KUDOS:0.01
+FEE_REFRESH = KUDOS:0
+FEE_REFUND = KUDOS:0
+RSA_KEYSIZE = 2048
+
+[COIN-KUDOS-n8-t1633183611]
+VALUE = KUDOS:1.28
+DURATION_WITHDRAW = 7 days
+DURATION_SPEND = 2 years
+DURATION_LEGAL = 6 years
+FEE_WITHDRAW = KUDOS:0
+FEE_DEPOSIT = KUDOS:0.01
+FEE_REFRESH = KUDOS:0
+FEE_REFUND = KUDOS:0
+RSA_KEYSIZE = 2048
+
+[COIN-KUDOS-n9-t1633183611]
+VALUE = KUDOS:2.56
+DURATION_WITHDRAW = 7 days
+DURATION_SPEND = 2 years
+DURATION_LEGAL = 6 years
+FEE_WITHDRAW = KUDOS:0
+FEE_DEPOSIT = KUDOS:0.01
+FEE_REFRESH = KUDOS:0
+FEE_REFUND = KUDOS:0
+RSA_KEYSIZE = 2048
+
+[COIN-KUDOS-n10-t1633183611]
+VALUE = KUDOS:5.12
+DURATION_WITHDRAW = 7 days
+DURATION_SPEND = 2 years
+DURATION_LEGAL = 6 years
+FEE_WITHDRAW = KUDOS:0
+FEE_DEPOSIT = KUDOS:0.01
+FEE_REFRESH = KUDOS:0
+FEE_REFUND = KUDOS:0
+RSA_KEYSIZE = 2048
+
+[COIN-KUDOS-n11-t1633183611]
+VALUE = KUDOS:10.24
+DURATION_WITHDRAW = 7 days
+DURATION_SPEND = 2 years
+DURATION_LEGAL = 6 years
+FEE_WITHDRAW = KUDOS:0
+FEE_DEPOSIT = KUDOS:0.01
+FEE_REFRESH = KUDOS:0
+FEE_REFUND = KUDOS:0
+RSA_KEYSIZE = 2048
+
+[COIN-KUDOS-n12-t1633183611]
+VALUE = KUDOS:20.48
+DURATION_WITHDRAW = 7 days
+DURATION_SPEND = 2 years
+DURATION_LEGAL = 6 years
+FEE_WITHDRAW = KUDOS:0
+FEE_DEPOSIT = KUDOS:0.01
+FEE_REFRESH = KUDOS:0
+FEE_REFUND = KUDOS:0
+RSA_KEYSIZE = 2048
+
+[COIN-KUDOS-n13-t1633183611]
+VALUE = KUDOS:40.96
+DURATION_WITHDRAW = 7 days
+DURATION_SPEND = 2 years
+DURATION_LEGAL = 6 years
+FEE_WITHDRAW = KUDOS:0
+FEE_DEPOSIT = KUDOS:0.01
+FEE_REFRESH = KUDOS:0
+FEE_REFUND = KUDOS:0
+RSA_KEYSIZE = 2048
+
+[COIN-KUDOS-n14-t1633183611]
+VALUE = KUDOS:81.92
+DURATION_WITHDRAW = 7 days
+DURATION_SPEND = 2 years
+DURATION_LEGAL = 6 years
+FEE_WITHDRAW = KUDOS:0
+FEE_DEPOSIT = KUDOS:0.01
+FEE_REFRESH = KUDOS:0
+FEE_REFUND = KUDOS:0
+RSA_KEYSIZE = 2048
+
+
diff --git a/configs/etc/taler/conf.d/exchange-system.conf
b/configs/etc/taler/conf.d/exchange-system.conf
new file mode 100644
index 0000000..75c670f
--- /dev/null
+++ b/configs/etc/taler/conf.d/exchange-system.conf
@@ -0,0 +1,10 @@
+# Configuration settings for system parameters of the exchange.
+
+# Read secret sections into configuration, but only
+# if we have permission to do so.
+@inline-secret@ exchangedb-postgres ../secrets/exchange-db.secret.conf
+
+[exchange]
+
+# Only supported database is Postgres right now.
+DATABASE = postgres
diff --git a/configs/etc/taler/conf.d/fake-bank.conf
b/configs/etc/taler/conf.d/fake-bank.conf
new file mode 100644
index 0000000..3b8634a
--- /dev/null
+++ b/configs/etc/taler/conf.d/fake-bank.conf
@@ -0,0 +1,5 @@
+[bank]
+HTTP_PORT = 80
+SERVE = http
+MAX_DEBT = KUDOS:100000000000.0
+MAX_DEBT_BANK = KUDOS:1000000000000000.0
diff --git a/configs/etc/taler/conf.d/merchant.conf
b/configs/etc/taler/conf.d/merchant.conf
new file mode 100644
index 0000000..0f7bd93
--- /dev/null
+++ b/configs/etc/taler/conf.d/merchant.conf
@@ -0,0 +1,15 @@
+@inline-secret@ merchantdb-postgres ../secrets/merchant-db.secret.conf
+
+[merchant]
+
+DATABASE = postgres
+SERVE = TCP
+PORT = 8080
+
+[merchant-exchange-perf]
+
+EXCHANGE_BASE_URL = <EXCHANGE_URL_HERE>
+MASTER_KEY = <EXCHANGE_MASTER_KEY_HERE>
+CURRENCY = KUDOS
+
+@inline-matching@ ../merchant-overrides.conf
diff --git a/configs/etc/taler/overrides.conf b/configs/etc/taler/overrides.conf
new file mode 100644
index 0000000..60296ea
--- /dev/null
+++ b/configs/etc/taler/overrides.conf
@@ -0,0 +1 @@
+# This configuration will be changed by tooling. Do not touch it manually.
diff --git a/configs/etc/taler/secrets/exchange-accountcredentials.secret.conf
b/configs/etc/taler/secrets/exchange-accountcredentials.secret.conf
new file mode 100755
index 0000000..e9f3e35
--- /dev/null
+++ b/configs/etc/taler/secrets/exchange-accountcredentials.secret.conf
@@ -0,0 +1,17 @@
+# This file contains the secret credentials
+# to access the Taler Wire Gateway API (usually
+# provided by LibEuFin) for the exchange accounts.
+#
+# Each exchange-account-* section should have a matching
+# exchange-accountcredentials-* section here.
+#
+# Each of those sections must be imported via @inline-secret@,
+# usually in conf.d/exchange-business.conf.
+
+[exchange-accountcredentials-1]
+
+wire_gateway_auth_method = basic
+password = x
+username = Exchange
+wire_gateway_url = http://<BANK_HOST_HERE>/Exchange/
+
diff --git a/configs/etc/taler/secrets/exchange-db.secret.conf
b/configs/etc/taler/secrets/exchange-db.secret.conf
new file mode 100755
index 0000000..cb52d0a
--- /dev/null
+++ b/configs/etc/taler/secrets/exchange-db.secret.conf
@@ -0,0 +1,10 @@
+# Database configuration for the Taler exchange.
+
+[exchangedb-postgres]
+
+# Typically, there should only be a single line here, of the form:
+
+CONFIG=<DB_URL_HERE>
+
+# The details of the URI depend on where the database lives and how
+# access control was configured.
diff --git a/configs/etc/taler/secrets/merchant-db.secret.conf
b/configs/etc/taler/secrets/merchant-db.secret.conf
new file mode 100644
index 0000000..3160b13
--- /dev/null
+++ b/configs/etc/taler/secrets/merchant-db.secret.conf
@@ -0,0 +1,3 @@
+[merchantdb-postgres]
+
+CONFIG=postgres:///taler-merchant
diff --git a/configs/etc/taler/taler.conf b/configs/etc/taler/taler.conf
new file mode 100755
index 0000000..111d109
--- /dev/null
+++ b/configs/etc/taler/taler.conf
@@ -0,0 +1,47 @@
+# Main entry point for the GNU Taler configuration.
+#
+# Structure:
+# - taler.conf is the main configuration entry point
+# used by all Taler components (the file you are currently
+# looking at.
+# - overrides.conf contains configuration overrides that are
+# set by some tools that help with the configuration,
+# and should not be edited by humans. Comments in this file
+# are not preserved.
+# - conf.d/ contains configuration files for
+# Taler components, which can be read by all
+# users of the system and are included by the main
+# configuration.
+# - secrets/ contains configuration snippets
+# with secrets for particular services.
+# These files should have restrictive permissions
+# so that only users of the relevant services
+# can read it. All files in it should end with
+# ".secret.conf".
+
+[taler]
+
+# Currency of the Taler deployment. This setting applies to all Taler
+# components that only support a single currency.
+currency = KUDOS
+
+# Smallest currency unit handled by the underlying bank system. Taler payments
+# can make payments smaller than this units, but interactions with external
+# systems is always rounded to this unit.
+currency_round_unit = KUDOS:0.01
+
+
+[paths]
+
+TALER_HOME = /var/lib/taler
+TALER_RUNTIME_DIR = /run/taler
+TALER_CACHE_HOME = /var/cache/taler
+TALER_CONFIG_HOME = /etc/taler
+TALER_DATA_HOME = /var/lib/taler
+
+
+# Inline configurations from all Taler components.
+@inline-matching@ conf.d/*.conf
+
+# Overrides from tools that help with configuration.
+@inline@ overrides.conf
diff --git a/configs/usr/lib/systemd/system/loki.service
b/configs/usr/lib/systemd/system/loki.service
new file mode 100644
index 0000000..8e5acdb
--- /dev/null
+++ b/configs/usr/lib/systemd/system/loki.service
@@ -0,0 +1,9 @@
+[Unit]
+Description=Grafana Loki instance
+
+[Service]
+Type=simple
+ExecStart=/usr/local/bin/loki --config.file=/etc/monitor/loki.yaml
+
+[Install]
+WantedBy=multi-user.target
diff --git a/configs/usr/lib/systemd/system/nginx.service
b/configs/usr/lib/systemd/system/nginx.service
new file mode 100644
index 0000000..9355ee0
--- /dev/null
+++ b/configs/usr/lib/systemd/system/nginx.service
@@ -0,0 +1,31 @@
+# Stop dance for nginx
+# =======================
+#
+# ExecStop sends SIGSTOP (graceful stop) to the nginx process.
+# If, after 5s (--retry QUIT/5) nginx is still running, systemd takes control
+# and sends SIGTERM (fast shutdown) to the main process.
+# After another 5s (TimeoutStopSec=5), and if nginx is alive, systemd sends
+# SIGKILL to all the remaining processes in the process group (KillMode=mixed).
+#
+# nginx signals reference doc:
+# http://nginx.org/en/docs/control.html
+#
+[Unit]
+Description=A high performance web server and a reverse proxy server
+Documentation=man:nginx(8)
+After=network.target nss-lookup.target
+
+[Service]
+Type=forking
+PIDFile=/run/nginx.pid
+ExecStartPre=/usr/sbin/nginx -t -q -g 'daemon on; master_process on;'
+ExecStart=/usr/sbin/nginx -g 'daemon on; master_process on;'
+ExecReload=/usr/sbin/nginx -g 'daemon on; master_process on;' -s reload
+ExecStop=-/sbin/start-stop-daemon --quiet --stop --retry QUIT/5 --pidfile
/run/nginx.pid
+TimeoutStopSec=5
+KillMode=mixed
+LimitNOFILE=500000
+
+[Install]
+WantedBy=multi-user.target
+
diff --git a/configs/usr/lib/systemd/system/promtail.service
b/configs/usr/lib/systemd/system/promtail.service
new file mode 100644
index 0000000..a0f971d
--- /dev/null
+++ b/configs/usr/lib/systemd/system/promtail.service
@@ -0,0 +1,9 @@
+[Unit]
+Description=Grafana Promtail instance
+
+[Service]
+Type=simple
+ExecStart=/usr/local/bin/promtail --config.file=/etc/monitor/promtail.yaml
+
+[Install]
+WantedBy=multi-user.target
diff --git a/configs/usr/lib/systemd/system/taler-exchange-aggregator.service
b/configs/usr/lib/systemd/system/taler-exchange-aggregator.service
new file mode 100644
index 0000000..44f4351
--- /dev/null
+++ b/configs/usr/lib/systemd/system/taler-exchange-aggregator.service
@@ -0,0 +1,15 @@
+[Unit]
+Description=GNU Taler payment system exchange aggregator service
+PartOf=taler-exchange.target
+
+[Service]
+User=taler-exchange-aggregator
+Type=simple
+Restart=on-failure
+EnvironmentFile=/etc/default/taler-exchange
+ExecStart=/usr/bin/taler-exchange-aggregator -c /etc/taler/taler.conf $ARGS
+StandardOutput=journal
+StandardError=journal
+PrivateTmp=yes
+PrivateDevices=yes
+ProtectSystem=full
diff --git a/configs/usr/lib/systemd/system/taler-exchange-closer.service
b/configs/usr/lib/systemd/system/taler-exchange-closer.service
new file mode 100644
index 0000000..f5bb6c6
--- /dev/null
+++ b/configs/usr/lib/systemd/system/taler-exchange-closer.service
@@ -0,0 +1,16 @@
+[Unit]
+Description=GNU Taler payment system exchange closer service
+PartOf=taler-exchange.target
+
+[Service]
+User=taler-exchange-closer
+Type=simple
+Restart=on-failure
+EnvironmentFile=/etc/default/taler-exchange
+ExecStart=/usr/bin/taler-exchange-closer -c /etc/taler/taler.conf $ARGS
+StandardOutput=journal
+StandardError=journal
+PrivateTmp=yes
+PrivateDevices=yes
+ProtectSystem=full
+
diff --git a/configs/usr/lib/systemd/system/taler-exchange-httpd.service
b/configs/usr/lib/systemd/system/taler-exchange-httpd.service
new file mode 100755
index 0000000..43a90c2
--- /dev/null
+++ b/configs/usr/lib/systemd/system/taler-exchange-httpd.service
@@ -0,0 +1,22 @@
+[Unit]
+Description=GNU Taler payment system exchange REST API
+AssertPathExists=/run/taler/exchange-httpd
+Requires=taler-exchange-httpd.socket taler-exchange-secmod-rsa.service
taler-exchange-secmod-eddsa.service
+After=postgres.service network.target taler-exchange-secmod-rsa.service
taler-exchange-secmod-eddsa.service
+PartOf=taler-exchange.target
+
+[Service]
+User=taler-exchange-httpd
+Type=simple
+Restart=on-failure
+EnvironmentFile=/etc/default/taler-exchange
+ExecStart=/usr/bin/taler-exchange-httpd -c /etc/taler/taler.conf $ARGS
+StandardOutput=journal
+StandardError=journal
+PrivateTmp=no
+PrivateDevices=yes
+ProtectSystem=full
+
+[Install]
+WantedBy=multi-user.target
+
diff --git a/configs/usr/lib/systemd/system/taler-exchange-httpd.socket
b/configs/usr/lib/systemd/system/taler-exchange-httpd.socket
new file mode 100755
index 0000000..59aa5be
--- /dev/null
+++ b/configs/usr/lib/systemd/system/taler-exchange-httpd.socket
@@ -0,0 +1,14 @@
+[Unit]
+Description=Taler Exchange Socket
+PartOf=taler-exchange-httpd.service
+
+[Socket]
+ListenStream=80
+Accept=no
+Service=taler-exchange-httpd.service
+SocketUser=taler-exchange-httpd
+SocketGroup=www-data
+SocketMode=0660
+
+[Install]
+WantedBy=sockets.target
diff --git a/configs/usr/lib/systemd/system/taler-exchange-httpd@.service
b/configs/usr/lib/systemd/system/taler-exchange-httpd@.service
new file mode 100755
index 0000000..d6ee28d
--- /dev/null
+++ b/configs/usr/lib/systemd/system/taler-exchange-httpd@.service
@@ -0,0 +1,21 @@
+[Unit]
+Description=GNU Taler payment system exchange REST API at %I
+AssertPathExists=/run/taler/exchange-httpd
+Requires=taler-exchange-httpd@%i.socket taler-exchange-secmod-rsa.service
taler-exchange-secmod-eddsa.service
+After=postgres.service network.target taler-exchange-secmod-rsa.service
taler-exchange-secmod-eddsa.service
+PartOf=taler-exchange.target
+
+[Service]
+User=taler-exchange-httpd
+Type=simple
+Restart=on-failure
+EnvironmentFile=/etc/default/taler-exchange
+ExecStart=/usr/bin/taler-exchange-httpd -c /etc/taler/taler.conf $ARGS
+StandardOutput=journal
+StandardError=journal
+PrivateTmp=no
+PrivateDevices=yes
+ProtectSystem=full
+
+[Install]
+WantedBy=multi-user.target
diff --git a/configs/usr/lib/systemd/system/taler-exchange-httpd@.socket
b/configs/usr/lib/systemd/system/taler-exchange-httpd@.socket
new file mode 100755
index 0000000..3f25eb3
--- /dev/null
+++ b/configs/usr/lib/systemd/system/taler-exchange-httpd@.socket
@@ -0,0 +1,14 @@
+[Unit]
+Description=Taler Exchange Socket at %I
+PartOf=taler-exchange-httpd@%i.service
+
+[Socket]
+ListenStream=%i
+Accept=no
+Service=taler-exchange-httpd@%i.service
+SocketUser=taler-exchange-httpd
+SocketGroup=www-data
+SocketMode=0660
+
+[Install]
+WantedBy=sockets.target
diff --git a/configs/usr/lib/systemd/system/taler-exchange-secmod-eddsa.service
b/configs/usr/lib/systemd/system/taler-exchange-secmod-eddsa.service
new file mode 100644
index 0000000..98d7ff0
--- /dev/null
+++ b/configs/usr/lib/systemd/system/taler-exchange-secmod-eddsa.service
@@ -0,0 +1,17 @@
+[Unit]
+Description=GNU Taler payment system exchange EdDSA security module
+AssertPathExists=/run/taler/exchange-secmod-eddsa
+PartOf=taler-exchange.target
+
+[Service]
+User=taler-exchange-secmod-eddsa
+Type=simple
+Restart=on-failure
+EnvironmentFile=/etc/default/taler-exchange
+ExecStart=/usr/bin/taler-exchange-secmod-eddsa -c /etc/taler/taler.conf $ARGS
+StandardOutput=journal
+StandardError=journal
+PrivateTmp=no
+PrivateDevices=yes
+ProtectSystem=full
+
diff --git a/configs/usr/lib/systemd/system/taler-exchange-secmod-rsa.service
b/configs/usr/lib/systemd/system/taler-exchange-secmod-rsa.service
new file mode 100644
index 0000000..fca5a3d
--- /dev/null
+++ b/configs/usr/lib/systemd/system/taler-exchange-secmod-rsa.service
@@ -0,0 +1,16 @@
+[Unit]
+Description=GNU Taler payment system exchange RSA security module
+AssertPathExists=/run/taler/exchange-secmod-rsa
+PartOf=taler-exchange.target
+
+[Service]
+User=taler-exchange-secmod-rsa
+Type=simple
+Restart=on-failure
+EnvironmentFile=/etc/default/taler-exchange
+ExecStart=/usr/bin/taler-exchange-secmod-rsa -c /etc/taler/taler.conf $ARGS
+StandardOutput=journal
+StandardError=journal
+PrivateTmp=no
+PrivateDevices=yes
+ProtectSystem=full
diff --git a/configs/usr/lib/systemd/system/taler-exchange-transfer.service
b/configs/usr/lib/systemd/system/taler-exchange-transfer.service
new file mode 100644
index 0000000..c8a2e9a
--- /dev/null
+++ b/configs/usr/lib/systemd/system/taler-exchange-transfer.service
@@ -0,0 +1,16 @@
+[Unit]
+Description=Taler Exchange Transfer Service
+After=network.target
+PartOf=taler-exchange.target
+
+[Service]
+User=taler-exchange-wire
+Type=simple
+Restart=on-failure
+EnvironmentFile=/etc/default/taler-exchange
+ExecStart=/usr/bin/taler-exchange-transfer -c /etc/taler/taler.conf $ARGS
+StandardOutput=journal
+StandardError=journal
+PrivateTmp=yes
+PrivateDevices=yes
+ProtectSystem=full
diff --git a/configs/usr/lib/systemd/system/taler-exchange-wirewatch.service
b/configs/usr/lib/systemd/system/taler-exchange-wirewatch.service
new file mode 100644
index 0000000..f8ab764
--- /dev/null
+++ b/configs/usr/lib/systemd/system/taler-exchange-wirewatch.service
@@ -0,0 +1,16 @@
+[Unit]
+Description=GNU Taler payment system exchange wirewatch service
+After=network.target
+PartOf=taler-exchange.target
+
+[Service]
+User=taler-exchange-wire
+Type=simple
+Restart=on-failure
+EnvironmentFile=/etc/default/taler-exchange
+ExecStart=/usr/bin/taler-exchange-wirewatch -c /etc/taler/taler.conf $ARGS
+StandardOutput=journal
+StandardError=journal
+PrivateTmp=yes
+PrivateDevices=yes
+ProtectSystem=full
diff --git a/configs/usr/lib/systemd/system/taler-exchange-wirewatch@.service
b/configs/usr/lib/systemd/system/taler-exchange-wirewatch@.service
new file mode 100644
index 0000000..f8ab764
--- /dev/null
+++ b/configs/usr/lib/systemd/system/taler-exchange-wirewatch@.service
@@ -0,0 +1,16 @@
+[Unit]
+Description=GNU Taler payment system exchange wirewatch service
+After=network.target
+PartOf=taler-exchange.target
+
+[Service]
+User=taler-exchange-wire
+Type=simple
+Restart=on-failure
+EnvironmentFile=/etc/default/taler-exchange
+ExecStart=/usr/bin/taler-exchange-wirewatch -c /etc/taler/taler.conf $ARGS
+StandardOutput=journal
+StandardError=journal
+PrivateTmp=yes
+PrivateDevices=yes
+ProtectSystem=full
diff --git a/configs/usr/lib/systemd/system/taler-exchange.target
b/configs/usr/lib/systemd/system/taler-exchange.target
new file mode 100644
index 0000000..eea37c8
--- /dev/null
+++ b/configs/usr/lib/systemd/system/taler-exchange.target
@@ -0,0 +1,12 @@
+[Unit]
+Description=GNU Taler exchange
+After=postgres.service network.target
+
+Wants=taler-exchange-httpd.service
+Wants=taler-exchange-wirewatch.service
+Wants=taler-exchange-aggregator.service
+Wants=taler-exchange-closer.service
+Wants=taler-exchange-transfer.service
+
+[Install]
+WantedBy=multi-user.target
diff --git a/configs/usr/lib/systemd/system/taler-fakebank.service
b/configs/usr/lib/systemd/system/taler-fakebank.service
new file mode 100644
index 0000000..f81ad53
--- /dev/null
+++ b/configs/usr/lib/systemd/system/taler-fakebank.service
@@ -0,0 +1,12 @@
+[Unit]
+Description=GNU Taler fake bank
+
+[Service]
+Type=simple
+EnvironmentFile=/etc/default/taler-exchange
+ExecStart=/usr/bin/taler-fakebank-run -c /etc/taler/taler.conf -n 100 $ARGS
+StandardOutput=journal
+StandardError=journal
+
+[Install]
+WantedBy=multi-user.target
diff --git a/configs/usr/lib/systemd/system/taler-logbackup.service
b/configs/usr/lib/systemd/system/taler-logbackup.service
new file mode 100644
index 0000000..ed81bc6
--- /dev/null
+++ b/configs/usr/lib/systemd/system/taler-logbackup.service
@@ -0,0 +1,11 @@
+[Unit]
+Description=Compresses and moves taler logs to the grid5000 nfs
+Wants=taler-logbackup.timer
+
+[Service]
+EnvironmentFile=/etc/environment
+Type=oneshot
+ExecStart=/root/scripts/log-save.sh
+
+[Install]
+WantedBy=multi-user.target
diff --git a/configs/usr/lib/systemd/system/taler-logbackup.timer
b/configs/usr/lib/systemd/system/taler-logbackup.timer
new file mode 100644
index 0000000..a602e2b
--- /dev/null
+++ b/configs/usr/lib/systemd/system/taler-logbackup.timer
@@ -0,0 +1,10 @@
+[Unit]
+Description=Compresses and moves taler logs to the grid5000 nfs
+Requires=taler-logbackup.service
+
+[Timer]
+Unit=taler-logbackup.service
+OnCalendar=*:0/10
+
+[Install]
+WantedBy=timers.target
diff --git a/configs/usr/lib/systemd/system/taler-merchant-httpd.service
b/configs/usr/lib/systemd/system/taler-merchant-httpd.service
new file mode 100644
index 0000000..4d98529
--- /dev/null
+++ b/configs/usr/lib/systemd/system/taler-merchant-httpd.service
@@ -0,0 +1,11 @@
+[Unit]
+Description=GNU Taler payment system merchant backend
+
+[Service]
+User=taler-merchant-httpd
+Type=simple
+Restart=on-failure
+ExecStart=/usr/bin/taler-merchant-httpd -c /etc/taler/taler.conf -L DEBUG
+
+[Install]
+WantedBy=multi-user.target
diff --git a/configs/usr/lib/systemd/system/taler-netdelay.service
b/configs/usr/lib/systemd/system/taler-netdelay.service
new file mode 100644
index 0000000..1dbfac0
--- /dev/null
+++ b/configs/usr/lib/systemd/system/taler-netdelay.service
@@ -0,0 +1,11 @@
+[Unit]
+Description=Loggs the network delay between experiment nodes
+Wants=taler-netdelay.timer
+
+[Service]
+EnvironmentFile=/etc/environment
+Type=oneshot
+ExecStart=/root/scripts/ping.sh <PING_DESTINATION>
+
+[Install]
+WantedBy=multi-user.target
diff --git a/configs/usr/lib/systemd/system/taler-netdelay.timer
b/configs/usr/lib/systemd/system/taler-netdelay.timer
new file mode 100644
index 0000000..f445635
--- /dev/null
+++ b/configs/usr/lib/systemd/system/taler-netdelay.timer
@@ -0,0 +1,10 @@
+[Unit]
+Description=Logs the network delay between experiment nodes
+Requires=taler-netdelay.service
+
+[Timer]
+Unit=taler-netdelay.service
+OnCalendar=*:0/5
+
+[Install]
+WantedBy=timers.target
diff --git a/configs/usr/lib/systemd/system/taler-wallet@.service
b/configs/usr/lib/systemd/system/taler-wallet@.service
new file mode 100755
index 0000000..419b666
--- /dev/null
+++ b/configs/usr/lib/systemd/system/taler-wallet@.service
@@ -0,0 +1,18 @@
+[Unit]
+Description=GNU Taler wallet benchmark
+# Always restart the process no matter how often it failed
+StartLimitIntervalSec=0
+
+[Service]
+Type=simple
+Restart=always
+RestartSec=5
+EnvironmentFile=/etc/environment
+ExecStart=/root/scripts/benchmark.sh %I
+
+StandardOutput=journal
+StandardError=journal
+SyslogIdentifier=taler-wallet-cli
+
+[Install]
+WantedBy=multi-user.target
diff --git a/debian11-taler.yaml b/debian11-taler.yaml
deleted file mode 100644
index fd84692..0000000
--- a/debian11-taler.yaml
+++ /dev/null
@@ -1,77 +0,0 @@
-#==============================================================================
-# vim: softtabstop=2 shiftwidth=2 expandtab fenc=utf-8 cc=81 tw=80
-#==============================================================================
-#
-# DESCRIPTION: Customization of a Debian 10 image with GNU Taler repositories.
-#
-#==============================================================================
-# This recipe extends another. To look at the step involed, run:
-# kameleon dryrun debian10_custom.yaml
-# To see the variables that you can override, use the following command:
-# kameleon info debian10_custom.yaml
----
-extend: grid5000/debian11-x64-min.yaml
-
-global:
- ### Uncomment and adapt the global variables below as needed
-
- ## Export format to generate
- # appliance_formats: qcow2 tar.zst
-
- ## Environment description customization
- ## Author
- g5k_author: "grothoff@gnu.org"
- ## Version
- g5k_version: 2
- ## Environment image path and compression
- # Note: unknown why this is not propagated to the ".dsc" file,
- # So we must adjust it manually!
- g5k_tgz_file: http://public.lyon.grid5000.fr/~grothoff/debian10-taler.tar.zst
- # g5k_tar_compression: "zstd"
- ## Environment postinstall path, compression, and script command
- # g5k_postinst_path: server:///grid5000/postinstalls/g5k-postinstall.tgz
- # g5k_postinst_compression: "gzip"
- # g5k_postinst_script: g5k-postinstall --net debian
- ## Environment kernel path and params
- # g5k_kernel_path: "/vmlinuz"
- # g5k_initrd_path: "/initrd.img"
- # g5k_kernel_params: ""
- ## Environment visibility
- # g5k_visibility: "shared"
- other_packages_no_clean: nginx postgresql-13 taler-exchange taler-auditor
taler-merchant taler-exchange-offline taler-wallet-cli sudo git zile
-
- ## Other parameters can be changed, see kameleon info debian10-taler.yaml
-
-bootstrap:
- ### The bootstrap section takes in charge the initial installation of the
- ## system (distribution installation). No modification should be needed here.
- - "@base"
-
-setup:
- ### The setup section is where customizations of the system take place.
- ## We can request steps from the extended recipe to be executed
- - "@base"
- - taler_install
- ## We add steps required by our customization after or before @base. Use
- ## kameleon dryrun debian10_custom.yaml to see the resulting steps in the
build.
- ## The following is given as example only, replace with your steps.
- - a_customization_step:
- - microstep1:
- - exec_in: |
- cd /root
- git clone git://git.taler.net/grid5k.git
- cp grid5k/gridboot.service /etc/systemd/system/gridboot.service
- chmod 640 /etc/systemd/system/gridboot.service
- cp grid5k/at-boot.sh /usr/local/bin/at-boot.sh
- chmod +x /usr/local/bin/at-boot.sh
- systemctl daemon-reload
- systemctl enable gridboot
-
- - microstep1:
- # This breakpoint will stop the build for inspecting the environment
- - breakpoint
-
-export:
- ### The export section takes in charge the export of your customized
Grid'5000
- ## environment. No modification should be needed here.
- - "@base"
diff --git a/default/from_scratch/x86_64/base.yaml
b/default/from_scratch/x86_64/base.yaml
deleted file mode 100644
index 777fdc4..0000000
--- a/default/from_scratch/x86_64/base.yaml
+++ /dev/null
@@ -1,138 +0,0 @@
-#==============================================================================
-# vim: softtabstop=2 shiftwidth=2 expandtab fenc=utf-8 cc=81 tw=80
-#==============================================================================
-#
-# DESCRIPTION: Base recipe template
-#
-#==============================================================================
----
-# Load qemu checkpoint
-checkpoint: simple.yaml
-# Loads some helpful aliases (this files are located in steps/aliases/
directory)
-aliases: defaults.yaml
-
-# Custom shell environement (this files are located in steps/env/ directory)
-env:
- - bashrc
- - functions.sh
-
-# Global variables use by Kameleon engine and the steps
-global:
- # Architecture for the target system
- arch: x86_64
- distrib: unknown
- release: unknown
- # Default hostname
- hostname: kameleon-$${distrib}
- # Default root password
- root_password: kameleon
-
- ## System variables. Required by kameleon engine
- # Include specific steps
- include_steps:
- - $${distrib}/$${release}
- - $${distrib}
-
- # If qemu_iso_path is set, boot from an iso, retrieved from the following
URL:
- installer_iso_arch: x86_64
- installer_iso_url:
- # or give an helper script to find out the iso URL:
- installer_iso_finder_helper:
- installer_iso_finder_args:
-
- # Otherwise, if qemu_kernel_path is set, boot from an kernel, initrd and
- # cmdline fetched from the URL defined below, and used directly in qemu:
- installer_kernel_url:
- installer_initrd_url:
- installer_cmdline:
-
- ## GPG keyserver (Waring: not all servers are reliable)
- gpg_keyserver: keyserver.ubuntu.com
-
- ## QEMU options
- qemu_enable_kvm: true
- qemu_uefi: false
- qemu_cpus: 2
- qemu_memory_size: 768
- qemu_monitor_socket: $${kameleon_cwd}/qemu_monitor.socket
- qemu_arch: $${arch}
- qemu_image_size: 10G
- qemu_pidfile: $${kameleon_cwd}/qemu.pid
- qemu_kernel_path: $${kameleon_cwd}/qemu_kernel
- qemu_initrd_path: $${kameleon_cwd}/qemu_initrd
- qemu_append_cmdline: $${installer_cmdline}
- qemu_iso_path: $${kameleon_cwd}/qemu.iso
-
- # rootfs options
- disk_device: /dev/vda
- rootfs: /rootfs
- filesystem_type: ext4
-
- # appliance options
- image_disk: $${kameleon_cwd}/base_$${kameleon_recipe_name}
- image_format: qcow2
-
- # Allowed formats are: tar.gz, tar.bz2, tar.xz, tar.lzo, qcow, qcow2, qed,
vdi, raw, vmdk
- appliance_formats: tar.xz
- appliance_filename: "$${kameleon_cwd}/$${kameleon_recipe_name}"
- appliance_tar_excludes: >-
- ./etc/fstab ./root/.bash_history ./root/kameleon_workdir ./root/.ssh
- ./var/tmp/* ./tmp/* ./dev/* ./proc/* ./run/*
- ./sys/* ./root/.rpmdb ./boot/extlinux ./boot/grub ./boot/grub2
- zerofree: false
-
- # GRUB
- grub_cmdline_linux: console=tty0 console=ttyS0,115200
-
- http_directory: $${kameleon_cwd}/http_dir
- http_pid: $${kameleon_cwd}/http.pid
-
- ssh_config_file: $${kameleon_cwd}/ssh_config
- local_ip: 10.0.2.2
-
- out_context:
- cmd: ssh -F $${ssh_config_file} $${kameleon_recipe_name} -t /bin/bash
- workdir: /root/kameleon_workdir
- proxy_cache: $${local_ip}
-
- in_context:
- cmd: ssh -F $${ssh_config_file} $${kameleon_recipe_name} -t /bin/bash
- workdir: /root/kameleon_workdir
- proxy_cache: $${local_ip}
-
-# Bootstrap the new system and create the 'in_context'
-bootstrap:
- - enable_checkpoint
- - download_installer
- - prepare_disk
- - prepare_autoinstall
- - start_http_server
- - start_qemu:
- - force_vm_shutdown: false
- - shutdown_vm_immediately: true
- - vm_cleanup_section: bootstrap
- - vm_expected_service:
- - boot_timeout: 5
- - prepare_ssh_to_out_context
- - prepare_appliance
- - start_qemu:
- - force_vm_shutdown: true
- - shutdown_vm_immediately: false
- - vm_cleanup_section: setup
- - vm_expected_server: ssh
- - boot_timeout: 100
- - qemu_iso_boot: false
- - qemu_iso_path: ""
- - qemu_kernel_path: ""
- - qemu_sendkeys_commands: ""
-
-# Install and configuration steps
-setup:
- - minimal_install
- - clean_system
-
-# Export the generated appliance in the format of your choice
-export:
- - disable_checkpoint
- - save_appliance_VM:
- - appliance_tar_compression_level: "9"
diff --git a/default/from_scratch/x86_64/debian-testing.yaml
b/default/from_scratch/x86_64/debian-testing.yaml
deleted file mode 100644
index 5c65ed8..0000000
--- a/default/from_scratch/x86_64/debian-testing.yaml
+++ /dev/null
@@ -1,29 +0,0 @@
-#==============================================================================
-# vim: softtabstop=2 shiftwidth=2 expandtab fenc=utf-8 cc=81 tw=80
-#==============================================================================
-#
-# DESCRIPTION: Debian testing recipe using the netinstall mechanism
-#
-#==============================================================================
----
-extend: debian-base.yaml
-# Global variables use by Kameleon engine and the steps
-global:
- # Distribution
- distrib: debian
- release: testing
- release_number: X
- # This URL may be invalid when the testing distribution is in its early
stage,
- # i.e after a recent release of a new Debian stable.
- # In this case, it is expected for this recipe to NOT work.
- # The debian-debootstrap-testing recipe may be prefered.
- installer_iso_url:
https://cdimage.debian.org/cdimage/weekly-builds/amd64/iso-cd/debian-testing-amd64-netinst.iso
-
-bootstrap:
- - "@base"
-
-setup:
- - "@base"
-
-export:
- - "@base"
diff --git a/default/steps/aliases/defaults.yaml
b/default/steps/aliases/defaults.yaml
deleted file mode 100644
index 6cf723b..0000000
--- a/default/steps/aliases/defaults.yaml
+++ /dev/null
@@ -1,169 +0,0 @@
-write_local:
- - exec_local: |
- mkdir -p $(dirname @1);
- cat >@1 <<EOF_KAMELEON_INTERNAL
- @2
- EOF_KAMELEON_INTERNAL
-
-write_in:
- - exec_in: |
- mkdir -p $(dirname @1);
- cat >@1 <<EOF_KAMELEON_INTERNAL
- @2
- EOF_KAMELEON_INTERNAL
-
-write_out:
- - exec_out: |
- mkdir -p $(dirname @1);
- cat >@1 <<EOF_KAMELEON_INTERNAL
- @2
- EOF_KAMELEON_INTERNAL
-
-append_local:
- - exec_local: |
- mkdir -p $(dirname @1);
- cat >>@1 <<EOF_KAMELEON_INTERNAL
- @2
- EOF_KAMELEON_INTERNAL
-
-append_in:
- - exec_in: |
- mkdir -p $(dirname @1);
- cat >>@1 <<EOF_KAMELEON_INTERNAL
- @2
- EOF_KAMELEON_INTERNAL
-
-append_out:
- - exec_out: |
- mkdir -p $(dirname @1);
- cat >>@1 <<EOF_KAMELEON_INTERNAL
- @2
- EOF_KAMELEON_INTERNAL
-
-write_raw_local:
- - exec_local: |
- mkdir -p $(dirname @1);
- cat >@1 <<'EOF_KAMELEON_INTERNAL'
- @2
- EOF_KAMELEON_INTERNAL
-
-write_raw_in:
- - exec_in: |
- mkdir -p $(dirname @1);
- cat >@1 <<'EOF_KAMELEON_INTERNAL'
- @2
- EOF_KAMELEON_INTERNAL
-
-write_raw_out:
- - exec_out: |
- mkdir -p $(dirname @1);
- cat >@1 <<'EOF_KAMELEON_INTERNAL'
- @2
- EOF_KAMELEON_INTERNAL
-
-append_raw_local:
- - exec_local: |
- mkdir -p $(dirname @1);
- cat >>@1 <<'EOF_KAMELEON_INTERNAL'
- @2
- EOF_KAMELEON_INTERNAL
-
-append_raw_in:
- - exec_in: |
- mkdir -p $(dirname @1);
- cat >>@1 <<'EOF_KAMELEON_INTERNAL'
- @2
- EOF_KAMELEON_INTERNAL
-
-append_raw_out:
- - exec_out: |
- mkdir -p $(dirname @1);
- cat >>@1 <<'EOF_KAMELEON_INTERNAL'
- @2
- EOF_KAMELEON_INTERNAL
-
-local2out:
- - exec_out: |
- mkdir -p $(dirname @2)
- - pipe:
- - exec_local: cat @1
- - exec_out: cat > @2
-
-local2in:
- - exec_in: mkdir -p $(dirname @2)
- - pipe:
- - exec_local: cat @1
- - exec_in: cat > @2
-
-out2local:
- - exec_local: mkdir -p $(dirname @2)
- - pipe:
- - exec_out: cat @1
- - exec_local: cat > @2
-
-out2in:
- - exec_in: mkdir -p $(dirname @2)
- - pipe:
- - exec_out: cat @1
- - exec_in: cat > @2
-
-in2local:
- - exec_local: mkdir -p $(dirname @2)
- - pipe:
- - exec_in: cat @1
- - exec_local: cat > @2
-
-in2out:
- - exec_out: mkdir -p $(dirname @2)
- - pipe:
- - exec_in: cat @1
- - exec_out: cat > @2
-
-check_cmd_out:
- - rescue:
- - exec_out: command -V @1 2> /dev/null
- - breakpoint: "@1 is missing from out_context"
-
-check_cmd_local:
- - on_bootstrap_init:
- - rescue:
- - exec_local: command -V @1 2> /dev/null
- - breakpoint: "@1 is missing from local_context"
-
-check_cmd_in:
- - rescue:
- - exec_in: command -V @1 2> /dev/null
- - breakpoint: "@1 is missing from in_context"
-
-umount_out:
- - exec_out: |
- echo "try umount @1..." ; mountpoint -q "@1" && umount -f -l "@1" || true
-
-umount_local:
- - exec_local: |
- echo "try umount @1..." ; mountpoint -q "@1" && umount -f -l "@1" || true
-
-umount_in:
- - exec_in: |
- echo "try umount @1..." ; mountpoint -q "@1" && umount -f -l "@1" || true
-
-download_file_in:
- - exec_in: __download "@1" "@2"
-
-download_file_out:
- - exec_out: __download "@1" "@2"
-
-download_file_local:
- - exec_local: __download "@1" "@2"
-
-download_recipe_build_local:
- - exec_local: __download_recipe_build "@1" "@2" "@3" "@4" "@5" "@6" "@7"
-
-download_kadeploy_environment_image_local:
- - exec_local: __download_kadeploy_environment_image "@1" "@2" "@3" "@4" "@5"
-
-apt-get_in:
- - exec_in: DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes @1 2>&1
-
-apt-get_out:
- - exec_out: DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes @1 2>&1
diff --git a/default/steps/bootstrap/download_installer.yaml
b/default/steps/bootstrap/download_installer.yaml
deleted file mode 100644
index f15f58c..0000000
--- a/default/steps/bootstrap/download_installer.yaml
+++ /dev/null
@@ -1,31 +0,0 @@
-- download_installer:
- - test:
- - exec_local: test -n "$${installer_iso_url}" -o -n
"$${installer_iso_finder_helper}"
- - group:
- - test:
- - exec_local: test -z "$${installer_iso_url}"
- - exec_local: |
- echo "Looking for the netinstall iso URL for
$${installer_iso_finder_args}"
- DOWNLOAD_SRC_URL=$(python2 $${installer_iso_finder_helper}
$${installer_iso_finder_args})
- - download_file_local:
- - $${installer_iso_url}
- - $${qemu_iso_path}
- - exec_local: unset DOWNLOAD_SRC_URL
- - group:
- - test:
- - exec_local: test -n "$${installer_kernel_url}"
- - download_file_local:
- - $${installer_kernel_url}
- - $${qemu_kernel_path}
- - test:
- - exec_local: test -n "$${installer_initrd_url}"
- - download_file_local:
- - $${installer_initrd_url}
- - $${qemu_initrd_path}
-
-- delete_installer:
- - on_checkpoint: skip
- - on_export_clean:
- - exec_local: rm -f $${qemu_iso_path}
- - exec_local: rm -f $${qemu_kernel_path}
- - exec_local: rm -f $${qemu_initrd_path}
diff --git a/default/steps/bootstrap/start_http_server.yaml
b/default/steps/bootstrap/start_http_server.yaml
deleted file mode 100644
index 59184c3..0000000
--- a/default/steps/bootstrap/start_http_server.yaml
+++ /dev/null
@@ -1,19 +0,0 @@
-- http_script: $${kameleon_data_dir}/helpers/simple_http_server.py
-
-- run_http_server:
- - exec_local: |
- HTTP_PORT=$(__find_free_port 8000 8100)
- echo "HTTP port: $HTTP_PORT"
- export HTTP_PORT
- - exec_local: python2 $${http_script} --root $${http_directory} --bind
0.0.0.0 --port $HTTP_PORT --daemon --pid $${http_pid}
- - on_bootstrap_clean:
- - exec_local: |
- if [ -f $${http_pid} ]; then
- HTTP_PID=$(cat $${http_pid})
- if ps -p $HTTP_PID > /dev/null; then
- echo "Killing HTTP server (pid: $HTTP_PID)..."
- kill -9 "$HTTP_PID"
- rm -f $${http_pid}
- fi
- rm -f $${http_pid}
- fi
diff --git a/default/steps/bootstrap/start_qemu.yaml
b/default/steps/bootstrap/start_qemu.yaml
deleted file mode 100644
index 4d47953..0000000
--- a/default/steps/bootstrap/start_qemu.yaml
+++ /dev/null
@@ -1,227 +0,0 @@
-# Require SSH_FWD_PORT bash environment variable to be set
-
-# This must be set if you want to boot an ISO image:
-- qemu_iso_path: ""
-- qemu_iso_boot: true
-# Else that can be set to boot from a kernel, initrd and cmdline:
-- qemu_kernel_path: ""
-- qemu_initrd_path: ""
-- qemu_append_cmdline: ""
-# Else boot from disk.
-
-- vm_expected_service: ssh
-- boot_timeout: 100
-- shutdown_timeout: 100
-- debug: false
-- telnet_port: ""
-- no_reboot: true
-- socat_monitor: socat - UNIX-CONNECT:$${qemu_monitor_socket}
-- qemu_sendkeys_script: $${kameleon_data_dir}/qemu-sendkeys.rb
-- qemu_sendkeys_commands:
-- vm_expected_service: ssh
-- vm_cleanup_section: setup
-- shutdown_vm_immediately: false
-- force_vm_shutdown: true
-- qemu_enable_kvm: true
-- qemu_cpus: 2
-- qemu_memory_size: 768
-- qemu_monitor_socket: $${kameleon_cwd}/qemu_monitor.socket
-- qemu_arch: $${arch}
-- qemu_image_size: 10G
-- qemu_pidfile: $${kameleon_cwd}/qemu.pid
-- qemu_uefi: false
-- qemu_uefi_code_path: /usr/share/AAVMF/AAVMF_CODE.fd
-- qemu_uefi_vars_path: /usr/share/AAVMF/AAVMF_VARS.fd
-- qemu_netdev_user_options:
-- disk_cache: unsafe
-
-- start_vm:
- - on_checkpoint: redo
- - check_cmd_local: qemu-system-$${qemu_arch}
- - check_cmd_local: socat
- - on_bootstrap_clean:
- - test:
- - exec_local: test "$${shutdown_vm_immediately}" == "false" -a
"$${vm_cleanup_section}" == "bootstrap"
- - group:
- - exec_local: &1 |
- if [ -f $${qemu_pidfile} ]; then
- _QEMU_PID=$(< $${qemu_pidfile})
- if ps -p $_QEMU_PID > /dev/null; then
- if [ "$${force_vm_shutdown}" == "true" ]; then
- if [ -S $${qemu_monitor_socket} ]; then
- echo "Executing a graceful shutdown of the qemu VM via the
monitor socket..."
- NEXT_WAIT_TIME=0
- echo system_powerdown | socat -
UNIX-CONNECT:$${qemu_monitor_socket} || true
- while ps -p $_QEMU_PID > /dev/null && [ $NEXT_WAIT_TIME
-lt $${shutdown_timeout} ];
- do
- sleep 1
- echo -en "\rWaiting for qemu virtual machine to
shutdown...($(( $${shutdown_timeout} - 1 - NEXT_WAIT_TIME++ ))s)"
- done
- fi
- else
- echo "Waiting for the VM to shutdown"
- echo "Run 'vncviewer :$VNC_PORT' to see what's happening in
the VM"
- while ps -p $_QEMU_PID > /dev/null;
- do
- sleep 2
- done
- fi
- fi
- fi
- - exec_local: &2 |
- if [ -f $${qemu_pidfile} ]; then
- _QEMU_PID=$(< $${qemu_pidfile})
- if ps -p $_QEMU_PID > /dev/null; then
- if [ -S $${qemu_monitor_socket} ]; then
- echo "The graceful shutdown of the qemu VM should have
failed (monitor socket is there)..."
- fi
- echo "Killing qemu (pid: $_QEMU_PID)."
- kill -9 "$_QEMU_PID"
- fi
- rm -f $${qemu_pidfile}
- fi
- rm -f $${qemu_monitor_socket}
- - on_setup_clean:
- - test:
- - exec_local: test "$${shutdown_vm_immediately}" == "false" -a
"$${vm_cleanup_section}" == "setup"
- - group:
- - exec_local: *1
- - exec_local: *2
- - on_export_clean:
- - test:
- - exec_local: test "$${shutdown_vm_immediately}" == "false" -a
"$${vm_cleanup_section}" == "export"
- - group:
- - exec_local: *1
- - exec_local: *2
- - exec_local: |
- if [ "$${shutdown_vm_immediately}" == "true" ]; then
- echo "Qemu VM shutdown: immediately"
- else
- echo "Qemu VM shutdown: in $${vm_cleanup_section} section cleaning"
- fi
- - exec_local: |
- if [ -r $${qemu_pidfile} ] && pgrep -F $${qemu_pidfile} > /dev/null; then
- echo "Qemu pid file found, with process running: killing it !" 1>&2
- pkill -F $${qemu_pidfile}
- sleep 0.5
- if pgrep -F $${qemu_pidfile} > /dev/null; then
- echo "Failed to kill qemu process." 1>&2
- exit 1
- fi
- fi
- - exec_local: |
- echo "Starting qemu..."
- if [ "$${qemu_enable_kvm}" == "true" ] && (/usr/sbin/kvm-ok > /dev/null
|| egrep '(vmx|svm)' /proc/cpuinfo > /dev/null) ; then # print warning if
/usr/sbin/kvm-ok is not installed
- if [ "$${qemu_arch}" == "aarch64" ]; then
- ENABLE_KVM="-enable-kvm -accel kvm -machine
virt,gic-version=host,accel=kvm:tcg -cpu host"
- #ENABLE_KVM="-global virtio-blk-pci.scsi=off -no-user-config
-enable-fips -machine virt,gic-version=host,accel=kvm:tcg -cpu host -rtc
driftfix=slew -object rng-random,filename=/dev/urandom,id=rng0 -device
virtio-rng-pci,rng=rng0"
- elif [ "$${qemu_arch}" == "ppc64" ]; then
- ENABLE_KVM="-enable-kvm -accel kvm -machine pseries,accel=kvm:tcg
-cpu host"
- else #X86_64
- ENABLE_KVM="-enable-kvm -cpu host"
- fi
- BOOT_TIMEOUT=$${boot_timeout}
- else
- echo "No KVM acceleration used"
- BOOT_TIMEOUT=$(($${boot_timeout}*2))
- fi
- if [ -f "vm_state_to_load.txt" ]
- then
- SAVED_STATE="$(< vm_state_to_load.txt)"
- LOADVM="-loadvm $SAVED_STATE"
- rm -f vm_state_to_load.txt
- fi
- if [ "$${debug}" == "true" ]; then
- VNC_OPT=""
- else
- # Find empty VNC port
- VNC_PORT=$(( $(__find_free_port 5900 5910) - 5900 ))
- echo "VNC port: $VNC_PORT"
- VNC_OPT="-vnc :$VNC_PORT"
- fi
- if [ -n "$${telnet_port}" ]; then
- SERIAL_TELNET="telnet:localhost:$${telnet_port},server"
- fi
- # Select disk
- QEMU_DRIVES="-drive
file=$${image_disk}.$${image_format},cache=$${disk_cache},media=disk,if=virtio,id=drive0"
- QEMU_BOOT=
- QEMU_APPEND_CMDLINE=
- if [ "$${qemu_uefi}" == "true" ]; then
- if [ ! -f $${kameleon_cwd}/qemu_uefi_vars.fd ]; then
- cp $${qemu_uefi_vars_path} $${kameleon_cwd}/qemu_uefi_vars.fd
- fi
- QEMU_BOOT="-drive
if=pflash,format=raw,readonly,file=$${qemu_uefi_code_path} -drive
if=pflash,format=raw,file=$${kameleon_cwd}/qemu_uefi_vars.fd"
- fi
- if [ -n "$${qemu_iso_path}" ]; then
- QEMU_DRIVES="-drive file=$${qemu_iso_path},readonly,media=cdrom
$QEMU_DRIVES"
- if [ "$${qemu_iso_boot}" == "true" ]; then
- QEMU_BOOT="$QEMU_BOOT -boot order=d"
- fi
- elif [ -n "$${qemu_kernel_path}" ]; then
- QEMU_BOOT="$QEMU_BOOT -kernel $${qemu_kernel_path}"
- if [ -n "$${qemu_initrd_path}" ]; then
- QEMU_BOOT="$QEMU_BOOT -initrd $${qemu_initrd_path}"
- fi
- if [ -n "$${qemu_append_cmdline}" ]; then
- QEMU_APPEND_CMDLINE="$${qemu_append_cmdline}"
- QEMU_APPEND_CMDLINE=${QEMU_APPEND_CMDLINE//%LOCAL_IP%/$${local_ip}}
- QEMU_APPEND_CMDLINE=${QEMU_APPEND_CMDLINE//%HTTP_PORT%/$HTTP_PORT}
- fi
- fi
- if [ -n "$${qemu_netdev_user_options}" ]; then
- QEMU_NETDEV_USER_OPTIONS=",$${qemu_netdev_user_options}"
- fi
- if [ "$${no_reboot}" == "true" ]; then
- NO_REBOOT="-no-reboot"
- fi
- if [ -n "${SSH_FWD_PORT}" ]; then
- HOSTFWD=",hostfwd=tcp::${SSH_FWD_PORT}-:22"
- fi
- qemu-system-$${qemu_arch} $ENABLE_KVM -smp $${qemu_cpus} -m
$${qemu_memory_size} -rtc base=localtime \
- -net nic,model=virtio -net user${QEMU_NETDEV_USER_OPTIONS}${HOSTFWD} \
- $QEMU_DRIVES \
- -monitor unix:$${qemu_monitor_socket},server,nowait -pidfile
$${qemu_pidfile} -daemonize \
- $QEMU_BOOT ${QEMU_APPEND_CMDLINE:+-append "$QEMU_APPEND_CMDLINE"}
$NO_REBOOT \
- $VNC_OPT $SERIAL_TELNET\
- $LOADVM
- - exec_local: |
- VM_AVAILABLE=0
- if [ "$${vm_expected_service}" == "ssh" ]; then
- TIMEOUT=$(( $(date +%s) + $BOOT_TIMEOUT ))
- until timeout 5 ssh -q -F $${ssh_config_file} -o ConnectionAttempts=1
$${kameleon_recipe_name} -t true && VM_AVAILABLE=1 || [ $(date +%s) -gt
$TIMEOUT ];
- do
- echo -en "\rWaiting for SSH to become available in VM for
out_context...($(( TIMEOUT - $(date +%s) ))s)"
- sleep 1
- done
- echo
- else
- TIMEOUT=$(( $(date +%s) + $BOOT_TIMEOUT ))
- until timeout 1 [ $(date +%s) -gt $TIMEOUT ];
- do
- echo -en "\rWaiting for VM to become available : ($(( TIMEOUT -
$(date +%s) ))s)"
- sleep 1
- done
- echo
- VM_AVAILABLE=1
- fi
- - rescue:
- - exec_local: test $VM_AVAILABLE -eq 1
- - breakpoint: |
- Failed to get VM up and running (expected service:
$${vm_expected_service}). Please verify the VM successfully booted with a vnc
client.
- - test:
- - exec_local: test -e "$${qemu_sendkeys_commands}" -a -s
"$${qemu_sendkeys_commands}"
- - exec_local: |
- echo "Sending keyboard commands to the VM: $${qemu_sendkeys_commands}"
- echo "(Local httpd server url: http://$${local_ip}:$HTTP_PORT)"
- ruby $${qemu_sendkeys_script} -d 0.05 "$(sed -e
s/%LOCAL_IP%/$${local_ip}/g -e s/%HTTP_PORT%/$HTTP_PORT/g
$${qemu_sendkeys_commands})" | $${socat_monitor} > /dev/null
- - exec_local: echo "No keyboard commands to send"
-
-- shutdown_vm:
- - on_checkpoint: redo
- - on_clean:
- - test:
- - exec_local: test "$${shutdown_vm_immediately}" == "true"
- - exec_local: *2
- - test:
- - exec_local: test "$${shutdown_vm_immediately}" == "true"
- - exec_local: *1
diff --git a/default/steps/data/helpers/export_appliance.py
b/default/steps/data/helpers/export_appliance.py
deleted file mode 100644
index 634b240..0000000
--- a/default/steps/data/helpers/export_appliance.py
+++ /dev/null
@@ -1,242 +0,0 @@
-#!/usr/bin/env python2
-# -*- coding: utf-8 -*-
-"""Convert a disk image to many others formats with guestfish."""
-from __future__ import division, unicode_literals
-
-import os
-# import time
-import os.path as op
-import sys
-import subprocess
-import argparse
-import logging
-
-
-logger = logging.getLogger(__name__)
-
-tar_formats = ('tar', 'tar.gz', 'tgz', 'tar.bz2', 'tbz', 'tar.xz', 'txz',
- 'tar.lzo', 'tzo')
-
-tar_options = ["--selinux", "--xattrs", "--xattrs-include=*",
"--numeric-owner", "--one-file-system"]
-
-disk_formats = ('qcow', 'qcow2', 'qed', 'vdi', 'raw', 'vmdk')
-
-
-def which(command):
- """Locate a command.
- Snippet from: http://stackoverflow.com/a/377028
- """
- def is_exe(fpath):
- return os.path.isfile(fpath) and os.access(fpath, os.X_OK)
-
- fpath, fname = os.path.split(command)
- if fpath:
- if is_exe(command):
- return command
- else:
- for path in os.environ["PATH"].split(os.pathsep):
- path = path.strip('"')
- exe_file = os.path.join(path, command)
- if is_exe(exe_file):
- return exe_file
-
- raise ValueError("Command '%s' not found" % command)
-
-
-def tar_convert(disk, output, excludes, compression_level):
- """Convert image to a tar rootfs archive."""
- if compression_level in ("best", "fast"):
- compression_level_opt = "--%s" % compression_level
- else:
- compression_level_opt = "-%s" % compression_level
-
- compr = ""
- if output.endswith(('tar.gz', 'tgz')):
- try:
- compr = "| %s %s" % (which("pigz"), compression_level_opt)
- except:
- compr = "| %s %s" % (which("gzip"), compression_level_opt)
- elif output.endswith(('tar.bz2', 'tbz')):
- compr = "| %s %s" % (which("bzip2"), compression_level_opt)
- elif output.endswith(('tar.xz', 'txz')):
- compr = "| {} {} -c --threads=0 -".format(
- which("xz"), compression_level_opt)
- elif output.endswith(('tar.lzo', 'tzo')):
- compr = "| %s %s -c -" % (which("lzop"), compression_level_opt)
-
- # NB: guestfish version >= 1.32 supports the special tar options, but not
available in Debian stable (jessie): do not use for now
- #tar_options_list = ["selinux:true", "acls:true", "xattrs:true",
- # "numericowner:true",
- # "excludes:\"%s\"" % ' '.join(excludes)]
- #tar_options_str = ' '.join(tar_options_list)
- #cmd = which("guestfish") + \
- # " --ro -i tar-out -a %s / - %s %s > %s"
- #cmd = cmd % (disk, tar_options_str, compr, output)
- #proc = subprocess.Popen(cmd_mount_tar, env=os.environ.copy(), shell=True)
- #proc.communicate()
- #if proc.returncode:
- # raise subprocess.CalledProcessError(proc.returncode, cmd)
-
- tar_options_str = ' '.join(tar_options + ['--exclude="%s"' % s for s in
excludes])
- # Necessary to have quick access to /etc (bug 12240) and also good for
reproducibility
- tar_options_str += ' --sort=name'
- directory = dir_path = os.path.dirname(os.path.realpath(disk))
- cmds = [
- which("mkdir") + " %s/.mnt" % directory,
- which("guestmount") + " --ro -i -a %s %s/.mnt" % (disk, directory),
- which("tar") + " -c %s -C %s/.mnt . %s > %s" % (tar_options_str,
directory, compr, output)
- ]
- cmd_mount_tar = " && ".join(cmds)
- proc = subprocess.Popen(cmd_mount_tar, env=os.environ.copy(), shell=True)
- proc.communicate()
- returncode_mount_tar = proc.returncode
-
- # try to umount even if the previous command failed
- cmds = [
- which("guestunmount") + " %s/.mnt" % directory,
- which("rmdir") + " %s/.mnt" % directory
- ]
- cmd_umount = " && ".join(cmds)
- proc = subprocess.Popen(cmd_umount, env=os.environ.copy(), shell=True)
- proc.communicate()
- returncode_umount = proc.returncode
-
- if returncode_mount_tar:
- raise subprocess.CalledProcessError(returncode_mount_tar,
cmd_mount_tar)
- elif returncode_umount:
- raise subprocess.CalledProcessError(returncode_umount, cmd_umount)
-
-
-def qemu_convert(disk, output_fmt, output_filename):
- """Convert the disk image filename to disk image output_filename."""
- binary = which("qemu-img")
- cmd = [binary, "convert", "-O", output_fmt, disk, output_filename]
- if output_fmt in ("qcow", "qcow2"):
- cmd.insert(2, "-c")
- proc = subprocess.Popen(cmd, env=os.environ.copy(), shell=False)
- proc.communicate()
- if proc.returncode:
- raise subprocess.CalledProcessError(proc.returncode, ' '.join(cmd))
-
-
-def run_guestfish_script(disk, script, mount=""):
- """
- Run guestfish script.
- Mount should be in ("read_only", "read_write", "ro", "rw")
- """
- args = [which("guestfish"), '-a', disk]
- if mount in ("read_only", "read_write", "ro", "rw"):
- args.append('-i')
- if mount in mount in ("read_only", "ro"):
- args.append('--ro')
- else:
- args.append('--rw')
- else:
- script = "run\n%s" % script
- proc = subprocess.Popen(args,
- stdin=subprocess.PIPE,
- env=os.environ.copy())
- proc.communicate(input=script.encode('utf-8'))
- if proc.returncode:
- raise subprocess.CalledProcessError(proc.returncode, ' '.join(args))
-
-
-def guestfish_zerofree(filename):
- """Fill free space with zero"""
- logger.info(guestfish_zerofree.__doc__)
- cmd = "virt-filesystems -a %s" % filename
- fs = subprocess.check_output(cmd.encode('utf-8'),
- stderr=subprocess.STDOUT,
- shell=True,
- env=os.environ.copy())
- list_fs = fs.decode('utf-8').split()
- logger.info('\n'.join((' `--> %s' % i for i in list_fs)))
- script = '\n'.join(('zerofree %s' % i for i in list_fs))
- run_guestfish_script(filename, script, mount="read_only")
-
-
-def convert_disk_image(args):
- """Convert disk to another format."""
- filename = op.abspath(args.file.name)
- output = op.abspath(args.output)
-
- os.environ['LIBGUESTFS_CACHEDIR'] = os.getcwd()
- if args.verbose:
- os.environ['LIBGUESTFS_DEBUG'] = '1'
-
- # sometimes guestfish fails because of other virtualization tools are
- # still running use a test and retry to wait for availability
- # attempts = 0
- # while attempts < 3:
- # try:
- # logger.info("Waiting for virtualisation to be available...")
- # run_guestfish_script(filename, "cat /etc/hostname", mount='ro')
- # break
- # except:
- # attempts += 1
- # time.sleep(1)
-
- if args.zerofree and (set(args.formats) & set(disk_formats)):
- guestfish_zerofree(filename)
-
- for fmt in args.formats:
- if fmt in (tar_formats + disk_formats):
- output_filename = "%s.%s" % (output, fmt)
- if output_filename == filename:
- continue
- logger.info("Creating %s" % output_filename)
- try:
- if fmt in tar_formats:
- tar_convert(filename, output_filename,
- args.tar_excludes,
- args.tar_compression_level)
- else:
- qemu_convert(filename, fmt, output_filename)
- except ValueError as exp:
- logger.error("Error: %s" % exp)
-
-
-if __name__ == '__main__':
- allowed_formats = tar_formats + disk_formats
- allowed_formats_help = 'Allowed values are ' + ', '.join(allowed_formats)
-
- allowed_levels = ["%d" % i for i in range(1, 10)] + ["best", "fast"]
- allowed_levels_helps = 'Allowed values are ' + ', '.join(allowed_levels)
-
- parser = argparse.ArgumentParser(
- description=sys.modules[__name__].__doc__,
- formatter_class=argparse.ArgumentDefaultsHelpFormatter
- )
- parser.add_argument('file', action="store", type=argparse.FileType('r'),
- help='Disk image filename')
- parser.add_argument('-F', '--formats', action="store", type=str, nargs='+',
- help='Output format. ' + allowed_formats_help,
- choices=allowed_formats, metavar='fmt', required=True)
- parser.add_argument('-o', '--output', action="store", type=str,
- help='Output filename (without file extension)',
- required=True, metavar='filename')
- parser.add_argument('--tar-compression-level', action="store", type=str,
- default="9", choices=allowed_levels, metavar='lvl',
- help="Compression level. " + allowed_levels_helps)
- parser.add_argument('--tar-excludes', action="store", type=str, nargs='+',
- help="Files to excluded from archive",
- metavar='pattern', default=[])
- parser.add_argument('--zerofree', action="store_true", default=False,
- help='Zero free unallocated blocks from ext2/3 '
- 'file-systems before export to reduce image size')
- parser.add_argument('--verbose', action="store_true", default=False,
- help='Enable very verbose messages')
- log_format = '%(levelname)s: %(message)s'
- level = logging.INFO
- args = parser.parse_args()
- if args.verbose:
- level = logging.DEBUG
-
- handler = logging.StreamHandler(sys.stdout)
- handler.setLevel(level)
- handler.setFormatter(logging.Formatter(log_format))
-
- logger.setLevel(level)
- logger.addHandler(handler)
-
- convert_disk_image(args)
diff --git a/default/steps/data/helpers/netinstall_iso_finder.py
b/default/steps/data/helpers/netinstall_iso_finder.py
deleted file mode 100644
index b4a135b..0000000
--- a/default/steps/data/helpers/netinstall_iso_finder.py
+++ /dev/null
@@ -1,163 +0,0 @@
-#!/usr/bin/env python2
-# -*- coding: utf-8 -*-
-"""Find the latest netinstall iso for a Debian version and system
architecture."""
-
-from html.parser import HTMLParser
-from urllib2 import urlopen
-from urlparse import urljoin
-import re
-import sys
-import argparse
-import logging
-
-logger = logging.getLogger(__name__)
-
-class LinkParser(HTMLParser):
- """Retrieve links (a hrefs) from a text/html document"""
- def __init__(self, url):
- HTMLParser.__init__(self)
- self.url = url
- self.links = set()
- response = urlopen(url)
- contentType = response.info().get('Content-Type')
- if not contentType:
- return
- logger.debug("url = " + url );
- logger.debug("contentType = " + contentType );
- if ';' in contentType:
- (mediaType,charset) = contentType.split(";")
- charset = charset.split("=")[1]
- else:
- mediaType = contentType
- # ISO-8859-1 is no longer the default charset, see
https://tools.ietf.org/html/rfc7231#appendix-B
- # Let's use UTF-8.
- charset = "utf-8"
- if mediaType =='text/html':
- htmlBytes = response.read()
- htmlString = htmlBytes.decode(charset)
- self.feed(htmlString)
-
- def handle_starttag(self, tag, attrs):
- if tag == 'a':
- for (key, value) in attrs:
- if key == 'href':
- new_url = urljoin(self.url,value)
- if re.match("^"+self.url, new_url):
- self.links.add(new_url)
-
- def get_links(self):
- """Returns all the collected links"""
- return self.links
-
-
-def url_find(to_visit_url_set,visited_url_set,found_url_set):
- """Recursively look for urls given a regex, a set of urls to visit, a set
of already visited urls, a set of already found urls. Returns the set of found
urls"""
- logger.debug("Progress: to_visit:{} visited:{}
found:{}".format(len(to_visit_url_set),len(visited_url_set),len(found_url_set)))
- assert(len(to_visit_url_set.intersection(visited_url_set)) == 0)
- assert(len(to_visit_url_set.intersection(found_url_set)) == 0)
- if (len(to_visit_url_set) == 0):
- return [visited_url_set,found_url_set]
- else:
- url = to_visit_url_set.pop()
- visited_url_set.add(url)
- if target_regex.match(url):
- found_url_set.add(url)
- return url_find(to_visit_url_set, visited_url_set, found_url_set)
- else:
- new_url_set = set([url for url in LinkParser(url).get_links() if
(logger.debug(url) or True) and url_regex.match(url)])
- new_url_set.difference_update(visited_url_set)
- to_visit_url_set.update(new_url_set)
- return url_find(to_visit_url_set, visited_url_set, found_url_set)
-
-def key_normalize(version_string):
- """"
- In order to perform a natural sorting, we normalize the version (X.Y.Z) as
a unique integer with the following formula: X*100 + Y*10 + Z
- For instance, it solves situations where "9.9.0" is greater than "9.9.11"
- """
- splitted_string = version_string.split('.')
- assert(len(splitted_string) == 3)
- return
int(splitted_string[0])*100+int(splitted_string[1])*10+int(splitted_string[2])
-
-if __name__ == '__main__':
- parser =
argparse.ArgumentParser(description=sys.modules[__name__].__doc__,
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
- parser.add_argument("distrib", metavar="DISTRIB", help="distribution")
- parser.add_argument("version", metavar="VERSION", help="version")
- parser.add_argument("arch", metavar="ARCH", help="architecture")
- parser.add_argument("mirror", metavar="MIRROR", help="mirror", nargs="?")
- parser.add_argument('--info', action="store_true", default=False,
help='print info messages')
- parser.add_argument('--debug', action="store_true", default=False,
help='print debug messages')
- args = parser.parse_args()
-
- handler = logging.StreamHandler()
- if args.debug:
- logger.setLevel(logging.DEBUG)
- handler.setLevel(logging.DEBUG)
- elif args.info:
- logger.setLevel(logging.INFO)
- handler.setLevel(logging.INFO)
- else:
- logger.setLevel(logging.WARNING)
- handler.setLevel(logging.WARNING)
- handler.setFormatter(logging.Formatter('%(levelname)s: %(message)s'))
- logger.addHandler(handler)
-
- try:
- visited = set([])
- found = set([])
- if (args.distrib.lower() == "debian"):
- if args.mirror == None:
- args.mirror = "http://cdimage.debian.org/";
- if not re.match("^\d+$",args.version):
- raise Exception("please give the Debian release number (e.g. 8
for Jessie)")
- if args.version == '10':
- url_regex =
re.compile("^"+args.mirror+"cdimage/release/(?:"+args.version+"\.\d+\.\d+/(?:"+args.arch+"/(?:iso-cd/(?:debian-"+args.version+"\.\d+\.\d+-"+args.arch+"-netinst\.iso)?)?)?)?$")
- else:
- url_regex =
re.compile("^"+args.mirror+"cdimage/archive/(?:"+args.version+"\.\d+\.\d+/(?:"+args.arch+"/(?:iso-cd/(?:debian-"+args.version+"\.\d+\.\d+-"+args.arch+"-netinst\.iso)?)?)?)?$")
- target_regex = re.compile("^.*-netinst\.iso$")
- [visited,found] = url_find(set([args.mirror+"cdimage/"+v+"/" for v
in ["release","archive"]]), set(), set())
- elif (args.distrib.lower() == "ubuntu"):
- if args.mirror == None:
- args.mirror = "http://(?:archive|old-releases).ubuntu.com/"
- servers = set(["http://"+s+".ubuntu.com/ubuntu/"; for s in
["old-releases","archive"]])
- else:
- servers = set([args.mirror])
- if not re.match("^\w+$",args.version):
- raise Exception("please give the Ubuntu release name")
- url_regex =
re.compile("^"+args.mirror+"ubuntu/dists/(?:"+args.version+"(?:-updates)?/(?:main/(?:installer-"+args.arch+"/(?:current/(?:(?:legacy-)?images/(?:netboot/(?:mini\.iso)?)?)?)?)?)?)?$")
- target_regex = re.compile("^.*/mini\.iso$")
- [visited,found] = url_find(servers, set(), set())
- elif (args.distrib.lower() == "centos"):
- if args.mirror == None:
- args.mirror = "http://mirror.in2p3.fr/linux/CentOS/";
- if not re.match("^\d+$",args.version):
- raise Exception("please give the CentOS release number (e.g. 7
for CentOS-7)")
- if args.version == '6':
- url_regex =
re.compile("^"+args.mirror+"(?:"+args.version+"/(?:isos/(?:"+args.arch+"/(?:CentOS-"+args.version+"(?:\.\d+)?-"+args.arch+"-netinstall\.iso)?)?)?)?$")
- target_regex =
re.compile("^.*CentOS-\d+(?:\.\d+)?-\w+-netinstall\.iso$")
- elif args.version == '7':
- url_regex =
re.compile("^"+args.mirror+"(?:"+args.version+"/(?:isos/(?:"+args.arch+"/(?:CentOS-"+args.version+"-"+args.arch+"-NetInstall-\d+\.iso)?)?)?)?$")
- target_regex =
re.compile("^.*CentOS-\d+-\w+-NetInstall-\d+\.iso$")
- else:
- url_regex =
re.compile("^"+args.mirror+"(?:"+args.version+"/(?:isos/(?:"+args.arch+"/(?:CentOS-"+args.version+"\.\d+\.\d+-"+args.arch+"-boot\.iso)?)?)?)?$")
- target_regex =
re.compile("^.*CentOS-\d+\.\d+\.\d+-\w+-boot\.iso$")
- [visited,found] = url_find(set([args.mirror]), set(), set())
- else:
- raise Exception("this distribution is not supported")
- logger.info("URL regex: "+url_regex.pattern)
- logger.info("Target regex: "+target_regex.pattern)
- logger.debug("Visited URLs:")
- for url in visited:
- logger.debug(url)
- logger.info("Found URLs:")
- for url in found:
- logger.info(url)
- if len(found) > 0:
- if (args.distrib.lower() == "debian"):
- print(sorted(found,key=lambda
x:key_normalize(re.sub(r".*/debian-(\d+).(\d+).(\d+)-"+args.arch+"-netinst\.iso$",r"\1.\2.\3",x)),reverse=True)[0])
- else:
- print(sorted(found, reverse=False)[0])
- else:
- raise Exception("no url found")
- except Exception as exc:
- sys.stderr.write(u"Error: %s\n" % exc)
- sys.exit(1)
diff --git a/default/steps/data/preseed/debian-testing-preseed.cfg
b/default/steps/data/preseed/debian-testing-preseed.cfg
deleted file mode 100644
index 5af0d99..0000000
--- a/default/steps/data/preseed/debian-testing-preseed.cfg
+++ /dev/null
@@ -1,322 +0,0 @@
-#### Contents of the preconfiguration file (for wheezy)
-### Localization
-# Locale sets language and country.
-d-i debian-installer/locale string en_US.UTF-8
-
-# Keyboard selection.
-#d-i keymap select us
-d-i keyboard-configuration/xkb-keymap select us
-
-### Network configuration
-# netcfg will choose an interface that has link if possible. This makes it
-# skip displaying a list if there is more than one interface.
-d-i netcfg/choose_interface select auto
-
-# To pick a particular interface instead:
-#d-i netcfg/choose_interface select eth1
-
-# If you have a slow dhcp server and the installer times out waiting for
-# it, this might be useful.
-#d-i netcfg/dhcp_timeout string 60
-
-# If you prefer to configure the network manually, uncomment this line and
-# the static network configuration below.
-#d-i netcfg/disable_dhcp boolean true
-
-# If you want the preconfiguration file to work on systems both with and
-# without a dhcp server, uncomment these lines and the static network
-# configuration below.
-#d-i netcfg/dhcp_failed note
-#d-i netcfg/dhcp_options select Configure network manually
-
-# Static network configuration.
-#d-i netcfg/get_nameservers string 192.168.1.1
-#d-i netcfg/get_ipaddress string 192.168.1.42
-#d-i netcfg/get_netmask string 255.255.255.0
-#d-i netcfg/get_gateway string 192.168.1.1
-#d-i netcfg/confirm_static boolean true
-
-# Any hostname and domain names assigned from dhcp take precedence over
-# values set here. However, setting the values still prevents the questions
-# from being shown, even if values come from dhcp.
-d-i netcfg/get_hostname string kameleon
-d-i netcfg/get_domain string kameleon
-
-# Disable that annoying WEP key dialog.
-d-i netcfg/wireless_wep string
-# The wacky dhcp hostname that some ISPs use as a password of sorts.
-#d-i netcfg/dhcp_hostname string radish
-
-# If non-free firmware is needed for the network or other hardware, you can
-# configure the installer to always try to load it, without prompting. Or
-# change to false to disable asking.
-#d-i hw-detect/load_firmware boolean true
-
-### Network console
-# Use the following settings if you wish to make use of the network-console
-# component for remote installation over SSH. This only makes sense if you
-# intend to perform the remainder of the installation manually.
-#d-i anna/choose_modules string network-console
-#d-i network-console/password password r00tme
-#d-i network-console/password-again password r00tme
-
-### Mirror settings
-# If you select ftp, the mirror/country string does not need to be set.
-#d-i mirror/protocol string ftp
-d-i mirror/country string manual
-d-i mirror/http/hostname string http.debian.net
-d-i mirror/http/directory string /debian
-d-i mirror/http/proxy string
-
-# Suite to install.
-d-i mirror/suite string testing
-# Suite to use for loading installer components (optional).
-d-i mirror/udeb/suite string unstable
-
-### Clock and time zone setup
-# Controls whether or not the hardware clock is set to UTC.
-d-i clock-setup/utc boolean true
-
-# You may set this to any valid setting for $TZ; see the contents of
-# /usr/share/zoneinfo/ for valid values.
-d-i time/zone string UTC
-
-# Controls whether to use NTP to set the clock during the install
-d-i clock-setup/ntp boolean true
-# NTP server to use. The default is almost always fine here.
-#d-i clock-setup/ntp-server string ntp.example.com
-
-### Partitioning
-# If the system has free space you can choose to only partition that space.
-#d-i partman-auto/init_automatically_partition select biggest_free
-
-# Alternatively, you can specify a disk to partition. The device name must
-# be given in traditional non-devfs format.
-# Note: A disk must be specified, unless the system has only one disk.
-# For example, to use the first SCSI/SATA hard disk:
-#d-i partman-auto/disk string /dev/sda
-# In addition, you'll need to specify the method to use.
-# The presently available methods are: "regular", "lvm" and "crypto"
-d-i partman-auto/method string regular
-
-# If one of the disks that are going to be automatically partitioned
-# contains an old LVM configuration, the user will normally receive a
-# warning. This can be preseeded away...
-d-i partman-lvm/device_remove_lvm boolean true
-# The same applies to pre-existing software RAID array:
-d-i partman-md/device_remove_md boolean true
-
-# And the same goes for the confirmation to write the lvm partitions.
-d-i partman-lvm/confirm boolean true
-d-i partman-lvm/confirm_nooverwrite boolean true
-
-
-d-i partman/choose_partition select finish
-d-i partman-auto-lvm/guided_size string max
-
-# You can choose one of the three predefined partitioning recipes:
-# - atomic: all files in one partition
-# - home: separate /home partition
-# - multi: separate /home, /usr, /var, and /tmp partitions
-d-i partman-auto/choose_recipe select atomic
-d-i partman/default_filesystem string ext4
-
-# Or provide a recipe of your own...
-# The recipe format is documented in the file devel/partman-auto-recipe.txt.
-# If you have a way to get a recipe file into the d-i environment, you can
-# just point at it.
-#d-i partman-auto/expert_recipe_file string /hd-media/recipe
-
-# If not, you can put an entire recipe into the preconfiguration file in one
-# (logical) line. This example creates a small /boot partition, suitable
-# swap, and uses the rest of the space for the root partition:
-#d-i partman-auto/expert_recipe string \
-# boot-root :: \
-# 40 50 100 ext3 \
-# $primary{ } $bootable{ } \
-# method{ format } format{ } \
-# use_filesystem{ } filesystem{ ext3 } \
-# mountpoint{ /boot } \
-# . \
-# 500 10000 1000000000 ext3 \
-# method{ format } format{ } \
-# use_filesystem{ } filesystem{ ext3 } \
-# mountpoint{ / } \
-# . \
-# 64 512 300% linux-swap \
-# method{ swap } format{ } \
-# .
-
-#The preseed line that "selects finish" needs to be in a certain order in your
preseed, the example-preseed does not follow this.
-#http://ubuntuforums.org/archive/index.php/t-1504045.html
-
-# This makes partman automatically partition without confirmation, provided
-# that you told it what to do using one of the methods above.
-d-i partman/confirm_write_new_label boolean true
-d-i partman/confirm boolean true
-d-i partman/confirm_nooverwrite boolean true
-
-
-### Base system installation
-# Select the initramfs generator used to generate the initrd for 2.6 kernels.
-#d-i base-installer/kernel/linux/initramfs-generators string yaird
-
-# The kernel image (meta) package to be installed; "none" can be used if no
-# kernel is to be installed.
-#d-i base-installer/kernel/image string linux-image-2.6-486
-
-### Account setup
-# Enable login to root account
-d-i passwd/root-login boolean true
-# Root password, either in clear text
-d-i passwd/root-password password kameleon
-d-i passwd/root-password-again password kameleon
-# or encrypted using an MD5 hash.
-#d-i passwd/root-password-crypted password [MD5 hash]
-
-# Skip creation of a normal user account.
-# d-i passwd/make-user boolean false
-
-# To create a normal user account.
-d-i passwd/user-fullname string Kameleon User
-d-i passwd/username string kameleon
-# Normal user's password, either in clear text
-d-i passwd/user-password password kameleon
-d-i passwd/user-password-again password kameleon
-# or encrypted using an MD5 hash.
-#d-i passwd/user-password-crypted password [MD5 hash]
-# Create the first user with the specified UID instead of the default.
-#d-i passwd/user-uid string 1010
-# d-i user-setup/encrypt-home boolean false
-# d-i user-setup/allow-password-weak boolean true
-
-# The user account will be added to some standard initial groups. To
-# override that, use this.
-d-i passwd/user-default-groups string audio cdrom video admin
-
-### Apt setup
-# You can choose to install non-free and contrib software.
-#d-i apt-setup/non-free boolean true
-#d-i apt-setup/contrib boolean true
-# Uncomment this if you don't want to use a network mirror.
-#d-i apt-setup/use_mirror boolean false
-# Select which update services to use; define the mirrors to be used.
-# Values shown below are the normal defaults.
-# FIXME : temporarily remove security repo while debian fixes the installer
(default value : d-i apt-setup/services-select multiselect security, volatile)
-d-i apt-setup/services-select multiselect
-#d-i apt-setup/security_host string security.debian.org
-#d-i apt-setup/volatile_host string volatile.debian.org
-
-# Scan another CD or DVD?
-d-i apt-setup/cdrom/set-first boolean false
-
-# By default the installer requires that repositories be authenticated
-# using a known gpg key. This setting can be used to disable that
-# authentication. Warning: Insecure, not recommended.
-#d-i debian-installer/allow_unauthenticated string true
-
-### Package selection
-tasksel tasksel/first multiselect none
-# If the desktop task is selected, install the kde and xfce desktops
-# instead of the default gnome desktop.
-#tasksel tasksel/desktop multiselect kde, xfce
-
-# Individual additional packages to install
-d-i pkgsel/include string openssh-server sudo rsync haveged
-
-# Whether to upgrade packages after debootstrap.
-# Allowed values: none, safe-upgrade, full-upgrade
-d-i pkgsel/upgrade select none
-
-# Some versions of the installer can report back on what software you have
-# installed, and what software you use. The default is not to report back,
-# but sending reports helps the project determine what software is most
-# popular and include it on CDs.
-popularity-contest popularity-contest/participate boolean false
-
-### Boot loader installation
-# Grub is the default boot loader (for x86). If you want lilo installed
-# instead, uncomment this:
-#d-i grub-installer/skip boolean true
-# To also skip installing lilo, and install no bootloader, uncomment this
-# too:
-#d-i lilo-installer/skip boolean true
-
-# This is fairly safe to set, it makes grub install automatically to the MBR
-# if no other operating system is detected on the machine.
-d-i grub-installer/only_debian boolean true
-
-# This one makes grub-installer install to the MBR if it also finds some other
-# OS, which is less safe as it might not be able to boot that other OS.
-d-i grub-installer/with_other_os boolean true
-
-# Alternatively, if you want to install to a location other than the mbr,
-# uncomment and edit these lines:
-#d-i grub-installer/only_debian boolean false
-#d-i grub-installer/with_other_os boolean false
-#d-i grub-installer/bootdev string (hd0,0)
-# To install grub to multiple disks:
-#d-i grub-installer/bootdev string (hd0,0) (hd1,0) (hd2,0)
-
-# Optional password for grub, either in clear text
-#d-i grub-installer/password password r00tme
-#d-i grub-installer/password-again password r00tme
-# or encrypted using an MD5 hash, see grub-md5-crypt(8).
-#d-i grub-installer/password-crypted password [MD5 hash]
-
-# GRUB install devices:
-# Choices: /dev/sda (21474 MB; VMware_Virtual_S), /dev/sda1 (21472 MB;
VMware_Virtual_S)
-grub-pc grub-pc/install_devices multiselect /dev/vda
-# Choices: Enter device manually, /dev/sda
-grub-installer grub-installer/choose_bootdev select /dev/vda
-
-### Finishing up the installation
-# During installations from serial console, the regular virtual consoles
-# (VT1-VT6) are normally disabled in /etc/inittab. Uncomment the next
-# line to prevent this.
-#d-i finish-install/keep-consoles boolean true
-
-# Avoid that last message about the install being complete.
-d-i finish-install/reboot_in_progress note
-
-# This will prevent the installer from ejecting the CD during the reboot,
-# which is useful in some situations.
-d-i cdrom-detect/eject boolean false
-
-# This is how to make the installer shutdown when finished, but not
-# reboot into the installed system.
-#d-i debian-installer/exit/halt boolean true
-# This will power off the machine instead of just halting it.
-d-i debian-installer/exit/poweroff boolean true
-
-### Preseeding other packages
-# Depending on what software you choose to install, or if things go wrong
-# during the installation process, it's possible that other questions may
-# be asked. You can preseed those too, of course. To get a list of every
-# possible question that could be asked during an install, do an
-# installation, and then run these commands:
-# debconf-get-selections --installer > file
-# debconf-get-selections >> file
-
-
-#### Advanced options
-### Running custom commands during the installation
-# d-i preseeding is inherently not secure. Nothing in the installer checks
-# for attempts at buffer overflows or other exploits of the values of a
-# preconfiguration file like this one. Only use preconfiguration files from
-# trusted locations! To drive that home, and because it's generally useful,
-# here's a way to run any shell command you'd like inside the installer,
-# automatically.
-
-# This first command is run as early as possible, just after
-# preseeding is read.
-# Prevent packaged version of VirtualBox Guest Additions being installed:
-#d-i preseed/early_command string sed -i \
-# '/in-target/idiscover(){/sbin/discover|grep -v VirtualBox;}' \
-# /usr/lib/pre-pkgsel.d/20install-hwpackages
-
-# This command is run just before the install finishes, but when there is
-# still a usable /target directory. You can chroot to /target and use it
-# directly, or use the apt-install and in-target commands to easily install
-# packages and run commands in the target system.
-
diff --git a/default/steps/env/functions.sh b/default/steps/env/functions.sh
deleted file mode 100644
index 1abcc38..0000000
--- a/default/steps/env/functions.sh
+++ /dev/null
@@ -1,201 +0,0 @@
-## functions
-
-function fail {
- echo $@ 1>&2
- false
-}
-
-export -f fail
-
-function __download {
- local src=$1
- local dst=$2
- if [ -n "$DOWNLOAD_SRC_URL" ]; then
- src="$DOWNLOAD_SRC_URL"
- fi
- if [ -z "$src" ]; then
- fail "No URL to download from"
- fi
- # If dst is unset or a directory, infers dst pathname from src
- if [ -z "$dst" -o "${dst: -1}" == "/" ]; then
- dst="$dst${src##*/}"
- dst="${dst%%\?*}"
- fi
- local dstdir=${dst%/*}
- if [ -n "$dstdir" -a "$dstdir" != "$dst" ]; then
- mkdir -p $dstdir
- fi
- echo -n "Downloading: $src..."
- # Put cURL first because it accept URIs (like file://...)
- if which curl >/dev/null; then
- echo " (cURL)"
- curl -S --fail -# -L --retry 999 --retry-max-time 0 "$src" -o "$dst"
2>&1
- elif which wget >/dev/null; then
- echo " (wget)"
- wget --retry-connrefused --progress=bar:force "$src" -O "$dst" 2>&1
- elif which python >/dev/null; then
- echo " (python)"
- python -c <<EOF
-import sys
-import time
-if sys.version_info >= (3,):
- import urllib.request as urllib
-else:
- import urllib
-
-
-def reporthook(count, block_size, total_size):
- global start_time
- if count == 0:
- start_time = time.time()
- return
- duration = time.time() - start_time
- progress_size = float(count * block_size)
- if duration != 0:
- if total_size == -1:
- total_size = block_size
- percent = 'Unknown size, '
- else:
- percent = '%.0f%%, ' % float(count * block_size * 100 / total_size)
- speed = int(progress_size / (1024 * duration))
- sys.stdout.write('\r%s%.2f MB, %d KB/s, %d seconds passed'
- % (percent, progress_size / (1024 * 1024), speed,
duration))
- sys.stdout.flush()
-
-urllib.urlretrieve('$src', '$dst', reporthook=reporthook)
-print('\n')
-EOF
- true
- else
- fail "No way to download $src"
- fi
-}
-
-export -f __download
-
-function __download_recipe_build() {
- set -e
- local recipe=$1
- local version=${2:-latest}
- local do_checksum=${3:-true}
- local do_checksign=${4:-false}
- local do_cache=${5:-false}
- local builds_url=${6:-http://kameleon.imag.fr/builds}
- local dest_dir="${7:-$recipe}"
- local dest=""
- mkdir -p $dest_dir
- pushd $dest_dir > /dev/null
- echo "Downloading $recipe ($version):"
- __download $builds_url/${recipe}_$version.manifest
- if [ "$do_checksign" == "true" ]; then
- __download $builds_url/${recipe}_$version.manifest.sign
- gpg --verify ${recipe}_$version.manifest{.sign,} || fail "Cannot
verify signature"
- fi
- for f in $(< ${recipe}_$version.manifest); do
- if [[ $f =~ ^$recipe-cache_ ]] && [ "$do_cache" != "true" ]; then
- continue
- fi
- if [[ $f =~ \.sha[[:digit:]]+sum$ ]]; then
- if [ "$do_checksum" == "true" ]; then
- __download $builds_url/$f
- ${f##*.} -c $f || fail "Cannot verify checksum"
- if [ "$do_checksign" == "true" ]; then
- __download $builds_url/$f.sign
- gpg --verify $f{.sign,} || fail "Cannot verify signature"
- fi
- fi
- else
- __download $builds_url/$f
- echo -n "Link to version-less filename: "
- dest=${f%_*}.tar.${f#*.tar.}
- ln -fv $f $dest
- fi
- done
- popd > /dev/null
- export UPSTREAM_TARBALL="$dest_dir/$dest"
- set +e
-}
-
-export -f __download_recipe_build
-
-function __download_kadeploy_environment_image() {
- set -e
- local kaenv_name=$1
- local kaenv_user=$2
- local kaenv_version=$3
- local remote=$4
- local dest_dir=${5:-$kaenv_name}
- mkdir -p $dest_dir
- echo "Retrieve image for Kadeploy environment $kaenv_name"
- ${remote:+ssh $remote }which kaenv3 > /dev/null || fail "kaenv3 command
not found (${remote:-localhost})"
- # retrieve image[file], image[kind] and image[compression] from kaenv3
- declare -A image
- __kaenv() { local k=${2%%:*}; image[$k]=${2#*:}; }
- mapfile -s 1 -t -c1 -C __kaenv < <(${remote:+ssh $remote
}kaenv3${kaenv_user:+ -u $kaenv_user}${kaenv_version:+ --env-version
$kaenv_version} -p $kaenv_name | grep -A3 -e '^image:' | sed -e 's/ //g')
- [ -n "${image[file]}" ] || fail "Failed to retrieve environment
$kaenv_name"
- if [ "${image[compression]}" == "gzip" ]; then
- image[compression]="gz"
- elif [ "${image[compression]}" == "bzip2" ]; then
- image[compression]="bz2"
- fi
- image[protocol]=${image[file]%%:*}
- image[path]=${image[file]#*://}
- image[filename]=${image[path]##*/}
- local
dest=$dest_dir/${image[filename]%%.*}.${image[kind]}.${image[compression]}
- if [ "${image[kind]}" == "tar" ]; then
- if [ "${image[protocol]}" == "http" -o "${image[protocol]}" == "https"
]; then
- __download ${image[file]} $dest
- else
- if [ "${image[protocol]}" == "server" ]; then
- # If server:// => see if available locally (NFS) or fail, same
as if local:// <=> ""
- echo "Image is server side, try and fetch it from local file
${image[path]}"
- fi
- [ -r ${image[path]} ] || fail "Cannot retrieve ${image[file]}"
- cp -v ${image[path]} $dest
- fi
- else # dd or whatever
- fail "Image format${image[kind]:+ ${image[kind]}} is not supported"
- fi
- export UPSTREAM_TARBALL=$dest
- set +e
-}
-
-export -f __download_kadeploy_environment_image
-
-function __find_linux_boot_device() {
- local PDEVICE=`stat -c %04D /boot`
- for file in $(find /dev -type b 2>/dev/null) ; do
- local CURRENT_DEVICE=$(stat -c "%02t%02T" $file)
- if [ $CURRENT_DEVICE = $PDEVICE ]; then
- ROOTDEVICE="$file"
- break;
- fi
- done
- echo "$ROOTDEVICE"
-}
-
-export -f __find_linux_boot_device
-
-
-function __find_free_port() {
- local begin_port=$1
- local end_port=$2
-
- local port=$begin_port
- local ret=$(nc -z 127.0.0.1 $port && echo in use || echo free)
- while [ $port -le $end_port ] && [ "$ret" == "in use" ]
- do
- local port=$[$port+1]
- local ret=$(nc -z 127.0.0.1 $port && echo in use || echo free)
- done
-
- # manage loop exits
- if [[ $port -gt $end_port ]]
- then
- fail "No free port available between $begin_port and $end_port"
- fi
-
- echo $port
-}
-
-export -f __find_free_port
diff --git a/docker/Dockerfile b/docker/Dockerfile
new file mode 100644
index 0000000..360c400
--- /dev/null
+++ b/docker/Dockerfile
@@ -0,0 +1,76 @@
+FROM debian:11
+
+ENV DEBIAN_FRONTEND=noninteractive
+
+ENV GRID5K_DEST=lyon,lille
+ENV GNUNET_COMMIT_SHA=master
+ENV EXCHANGE_COMMIT_SHA=master
+ENV MERCHANT_COMMIT_SHA=master
+ENV WALLET_COMMIT_SHA=master
+ENV GRID5K_COMMIT_SHA=master
+
+RUN apt update && \
+ apt upgrade -y && \
+ apt install -y \
+ recutils \
+ autoconf \
+ uncrustify \
+ autopoint \
+ libtool \
+ python3-pip \
+ libgcrypt20-dev \
+ libjansson-dev \
+ libcurl4-gnutls-dev \
+ libsodium-dev \
+ libidn2-dev \
+ libunistring-dev \
+ libmicrohttpd-dev \
+ libsqlite3-dev \
+ libqrencode-dev \
+ pkg-config \
+ gettext \
+ postgresql \
+ libpq-dev \
+ texinfo \
+ postgresql-contrib \
+ curl \
+ jq \
+ git \
+ make \
+ ruby-dev \
+ ruby-childprocess \
+ zstd \
+ libguestfs-tools \
+ linux-headers-amd64 \
+ socat \
+ qemu-utils \
+ ruby-dev \
+ ruby-childprocess \
+ pigz \
+ netcat \
+ eatmydata \
+ dirmngr \
+ gnupg \
+ gnupg-agent \
+ expect \
+ rsync \
+ nodejs \
+ npm && \
+ pip install jinja2 && \
+ npm install -g pnpm && \
+ gem install --no-document kameleon-builder
+
+RUN mkdir taler && cd taler && \
+ git clone https://git.gnunet.org/gnunet.git && \
+ git clone https://git.taler.net/exchange.git && \
+ git clone https://git.taler.net/merchant.git && \
+ git clone https://git.taler.net/wallet-core.git && \
+ git clone https://git.taler.net/grid5k.git
+
+COPY entrypoint.sh /entrypoint.sh
+RUN chmod +x entrypoint.sh
+
+VOLUME ["/taler"]
+
+ENTRYPOINT ["./entrypoint.sh"]
+CMD [""]
diff --git a/docker/README.md b/docker/README.md
new file mode 100644
index 0000000..bcccae7
--- /dev/null
+++ b/docker/README.md
@@ -0,0 +1,96 @@
+# Taler Grid5000 Build Image
+
+This docker image can be used to build the Grid5000 image for the taler
performance experiments
+
+## Build
+
+Build this image with `docker build . -t taler:build`
+Or alternatively with `docker-compose up --build` **NOTE** this will also run
the image, read below
+
+## Run the build
+
+Running the image will build GNUnet, Taler and the Grid5000 image from the
specified commits.
+
+The image will then be uploaded to the specified nodes Grid5000 public
directory using the certificate provided.
+
+### docker
+
+```bash
+docker run -it --rm \
+ --device=/dev/kvm:/dev/kvm \
+ --device=/dev/fuse:/dev/fuse \
+ --device=/dev/net/tun:/dev/net/tun \
+ -p 5900:5900 \
+ --cap-add NET_ADMIN \
+ --cap-add SYS_ADMIN \
+ --security-opt apparmor:unconfined \
+ -e GRID5K_USER=<user> \
+ -e GRID5K_CERT_PASSWD=<cert_passwd> \
+ -v <cert_path>:/root/cert.pem \
+ taler:build <ARGUMENTS>
+```
+
+### docker-compose
+
+Assuming an env file `.env` with the following contents:
+
+```bash
+GRID5K_USER=<user>
+GRID5K_CERT=<cert_path>
+```
+
+the build can be started with:
+
+```bash
+ARGUMENTS=<ARGUMENTS> GRID5K_CERT_PASSWD=<cert_passwd> docker-compose up
--build
+```
+
+Or place the password (also arguments) in .env too but make sure its protected.
+
+### Notes
+
+#### Environment Variables
+
+**GRID5K_USER`**: the user which `GRID5K_CERT` belongs to
+**GRID5K_CERT**: the certificate which is used to login to the Grid5000 nodes
(docker-compose only)
+**GRID5K_CERT_PASSWD**: the password to decrypt `GRID5K_CERT`
+**GRID5K_DEST**: comma separated list of where to copy the image to in the
grid (lille,lyon)
+**ARGUMENTS**: args to pass to entrypoint, one of
+ -c|--clean (run make clean)
+ -r|--rebuild (rebuild the image - automatically set if -c is used)
+ -n|--no-copy (do not copy the generated image to Grid5000 - make sure output
volume is mounted)
+ As per default, running the docker command again will not clean or rebuild
the image
+
+##### Additional
+
+**GNUNET_COMMIT_SHA**: Which commit to use of gnunet (master)
+**EXCHANGE_COMMIT_SHA**: Which commit to use of taler-exchange (master)
+**MERCHANT_COMMIT_SHA**: Which commit to use of taler-merchant (master)
+**WALLET_COMMIT_SHA**: Which commit to use of wallet-core (master)
+**GRID5K_COMMIT_SHA**: Which commit to use of this repo (master)
+
+#### VNC
+
+What happens during the build can be inspected via the logs from docker.
+However, kameleon uses qemu which can be inspected with `vncviewer` in certain
steps and when port 5900 is mapped:
+
+```bash
+apt install -y tigervnc-viewer
+```
+
+```bash
+vncviewer :0
+```
+
+#### Output
+
+The image will be published to the Grid5000's public directory on a specified
node.
+Additionally the generated image can also be mounted to the host by specifying
`-v <some_path>:/root/output`
+or
+
+```yaml
+volumes:
+ - <some_path>:/root/output
+```
+
+respectively
diff --git a/docker/docker-compose.yaml b/docker/docker-compose.yaml
new file mode 100644
index 0000000..48f8dc8
--- /dev/null
+++ b/docker/docker-compose.yaml
@@ -0,0 +1,33 @@
+version: "3"
+
+services:
+ taler-build:
+ image: taler:build
+ container_name: taler-g5k
+ build: .
+ ports:
+ - "5900:5900"
+ cap_add:
+ - NET_ADMIN
+ - SYS_ADMIN
+ devices:
+ - /dev/kvm
+ - /dev/fuse
+ - /dev/net/tun
+ environment:
+ GNUNET_COMMIT_SHA: master
+ EXCHANGE_COMMIT_SHA: master
+ MERCHANT_COMMIT_SHA: master
+ WALLET_COMMIT_SHA: master
+ GRID5K_COMMIT_SHA: node-setup
+ GRID5K_USER: ${GRID5K_USER}
+ GRID5K_CERT_PASSWD: ${GRID5K_CERT_PASSWD}
+ security_opt:
+ - apparmor:unconfined
+ volumes:
+ - build:/taler
+ - ${GRID5K_CERT}:/root/cert.pem
+ command: "${ARGUMENTS}"
+
+volumes:
+ build:
diff --git a/docker/entrypoint.sh b/docker/entrypoint.sh
new file mode 100644
index 0000000..a99ef00
--- /dev/null
+++ b/docker/entrypoint.sh
@@ -0,0 +1,186 @@
+#!/bin/bash
+set -e
+
+while [[ $# -gt 0 ]]; do
+ case "$1" in
+ -c|--clean)
+ CLEAN=true
+ REBUILD=true
+ echo "INFO will clean previous builds and image"
+ shift
+ ;;
+ -r|--rebuild)
+ REBUILD=true
+ echo "INFO will rebuild image"
+ shift
+ ;;
+ -n|--no-copy)
+ COPY=false
+ echo "INFO will not copy image to Grid5000"
+ shift
+ ;;
+ *)
+ shift
+ ;;
+ esac
+done
+
+TALER_HOME=/taler
+
+function prepare() {
+ cd "${TALER_HOME}/$1"
+ git checkout master > /dev/null && git pull > /dev/null 2>&1
+ git checkout "$2" > /dev/null && (git pull > /dev/null 2>&1 || true)
+}
+
+function build() {
+ prepare "$1" "$2"
+ if [ "$CLEAN" = true ]; then
+ echo "INFO cleaning previous build"
+ make clean > /dev/null 2>&1 || true
+ fi
+ if ! make > /dev/null 2>&1; then
+ echo "INFO running bootstrap and configure"
+ if ! ./bootstrap > bootstrap.log 2>&1; then
+ echo "ERROR bootstrap failed"
+ cat bootstrap.log
+ exit 1
+ fi
+ if ! (./configure --enable-logging=verbose --prefix=/usr > configure.log
2>&1 || \
+ ./configure > configure.log 2>&1); then
+ echo "ERROR configure failed"
+ cat configure.log
+ exit 1
+ fi
+ fi
+ if [ ! -z "$3" ]; then
+ echo "INFO running custom command '$3'"
+ /bin/bash -c "$3"
+ fi
+ echo "INFO running 'make dist'"
+ if ! make dist > dist.log 2>&1; then
+ echo "ERROR make dist failed"
+ cat dist.log
+ exit 1
+ fi
+}
+
+function install() {
+ build "$1" "$2" "$3"
+ echo "INFO installing"
+ if [[ $(ldconfig -p | grep "$1") == "" ]] || \
+ [ "$CLEAN" = true ]; then
+ if ! make install > install.log 2>&1; then
+ echo "ERROR install failed"
+ cat install.log
+ exit 1
+ fi
+ ldconfig
+ fi
+}
+
+if [ "$COPY" != false ];
+then
+ eval $(ssh-agent)
+
+ if [[ -f "/root/cert.pem" ]]; then
+
+ if [[ "${GRID5K_CERT_PASSWD}" != "" ]]; then
+ /usr/bin/expect -c "
+ spawn ssh-add /root/cert.pem;
+ expect {
+ \"Enter passphrase for /root/cert.pem: \" {
+ exp_send ${GRID5K_CERT_PASSWD}\n;
+ exp_continue
+ }
+ \"Identity added: /root/cert.pem (/root/cert.pem)\" {
+ exp_continue
+ }
+ eof {}
+ }" > /tmp/expect.log
+ else
+ ssh-add /root/cert.pem
+ fi
+
+ fi
+
+ if ! ssh-add -L > /dev/null ; then
+ echo "ERROR could not add certificate"
+ cat /tmp/expect.log
+ echo ""
+ exit 1
+ else
+ echo "INFO added certificate"
+ fi
+fi
+
+echo "INFO preparing Grid5000 repository"
+prepare grid5k \
+ "${GRID5K_COMMIT_SHA}"
+touch
${TALER_HOME}/grid5k/image/grid5000/steps/data/{gnunet,wallet,exchange,merchant}.tar.gz
+if ! kameleon dryrun ${TALER_HOME}/grid5k/image/taler-debian11.yaml >
dryrun.log 2>&1; then
+ echo "ERROR Grid5000 image configuration contains errors"
+ cat dryrun.log
+ exit 1
+else
+ rm ${TALER_HOME}/grid5k/image/grid5000/steps/data/*.tar.gz
+fi
+
+echo "INFO preparing GNUnet"
+install gnunet \
+ "${GNUNET_COMMIT_SHA}"
+
+echo "INFO preparing Taler Exchange"
+install exchange \
+ "${EXCHANGE_COMMIT_SHA}" \
+ 'find . -name "*Makefile*" -exec sed -i "/x-taler-bank.fee/d" {} \;'
+
+echo "INFO preparing Taler Merchant"
+install merchant \
+ "${MERCHANT_COMMIT_SHA}"
+
+echo "INFO preparing Taler Wallet"
+build wallet-core \
+ "${WALLET_COMMIT_SHA}"
+
+cd ${TALER_HOME}/grid5k/image
+
+if [ ! -f "build/taler-debian11/taler-debian11.tar.zst" ] || \
+ [ "$REBUILD" = true ];
+then
+ mv "${TALER_HOME}"/gnunet/gnunet*.tar.gz grid5000/steps/data/gnunet.tar.gz
+ mv "${TALER_HOME}"/exchange/taler-exchange*.tar.gz
grid5000/steps/data/exchange.tar.gz
+ mv "${TALER_HOME}"/merchant/taler-merchant*.tar.gz
grid5000/steps/data/merchant.tar.gz
+ mv "${TALER_HOME}"/wallet-core/taler-wallet*.tar.gz
grid5000/steps/data/wallet.tar.gz
+
+ rm -rf /tmp/taler-debian11 || true
+ echo "INFO building image"
+ yes r | kameleon build -b /tmp taler-debian11.yaml | tee build.log
+ mv /tmp/taler-debian11 ${TALER_HOME}
+fi
+
+cd ${TALER_HOME}/taler-debian11
+
+if [ -f "/root/cert.pem" ] && [ "$COPY" != false ]; then
+
+ mv taler-debian11.dsc taler-debian11.dsc.bak
+
+ IFS=, read -ra G5K_HOSTS <<< "${GRID5K_DEST}"
+
+ for G5K_HOST in "${G5K_HOSTS[@]}"; do
+ echo "Copying image to ${G5K_HOST}"
+
+ sed "s/<G5K_HOST>/${G5K_HOST}/g; s/<G5K_USER>/${GRID5K_USER}/g" \
+ taler-debian11.dsc.bak > taler-debian11.dsc
+
+ scp -o StrictHostKeyChecking=no taler-debian11.tar.zst taler-debian11.dsc \
+ "${GRID5K_USER}@access.grid5000.fr:${G5K_HOST}"/public/
+ done
+
+fi
+
+if [ -d "/root/output" ]; then
+ echo "INFO copying image to output (mounted volume)"
+ mv taler-debian11.dsc.bak taler-debian11.dsc || true
+ cp taler-debian11.tar.zst taler-debian11.dsc /root/output/
+fi
diff --git a/etc/.empty b/etc/.empty
deleted file mode 100644
index e69de29..0000000
diff --git a/experiment/README b/experiment/README
new file mode 100755
index 0000000..5caae65
--- /dev/null
+++ b/experiment/README
@@ -0,0 +1,21 @@
+All files needed to run the experiment can be found inside the jfed/ directory:
+ * experiment-specitication.yml: espec to run the experiment
+ * taler-perf.rspec: boots nodes based on the omnibus image into grid5000
+ * env: template file to add enviroment variables needed for the experiment
+
+Steps:
+ * cp env .env
+ * edit .env and define the specified variables
+ * DNS_HMAC_SHA256 can be generated with tsig-keygen from bind9 package (use
the secret only)
+ * Start jFed Experimenter GUI
+ * Load infra.rspec and click Run
+ * Specify the experiment name and time
+ * Wait until infra.rspec is allocated successfully
+ * Configure .env and set the hosts (maybe this can be fixed once we can use
one rspec only)
+ * Click (Re)Run ESpec for the infra job job
+ * Load wallets.rspec into jFed and click Run
+ * Click (Re)Run ESpec for the wallets job
+ * Run `talet-perf update prometheus` on any deployed node
+
+To add more wallet processes run `taler-perf start wallets <NUM>` on any node
+
diff --git a/experiment/cleardns.sh b/experiment/cleardns.sh
new file mode 100644
index 0000000..3d52bec
--- /dev/null
+++ b/experiment/cleardns.sh
@@ -0,0 +1,33 @@
+#!/bin/bash
+# Clears the DNS server for the new experiment
+set -eux
+
+source .env
+source ~/scripts/helpers.sh
+
+function del_ddn() {
+ nsupdate -k /root/ddns.key -v << EOF
+server ${DNS_HOST}
+zone ${DNS_ZONE}
+update delete ${1}
+send
+EOF
+}
+
+dig -t AXFR ${DNS_ZONE} ${DNS_HOST}
+
+# Delete all old dns entries of the previous experitment
+while IFS='=' read -r NAME VALUE; do
+ if [[ "${NAME}" == *"_DOMAIN" ]] && [[ "${VALUE}" == *".${DNS_ZONE}" ]];
then
+ if [[ "${VALUE}" == "wallet"* ]]; then
+ for WALLET in $(get_wallet_domains); do
+ del_ddn "wallet.${WALLET}.${DNS_ZONE}"
+ done
+ else
+ del_ddn "${VALUE}"
+ fi
+ fi
+done < .env
+
+dig -t AXFR ${DNS_ZONE} ${DNS_HOST}
+
diff --git a/experiment/env b/experiment/env
new file mode 100755
index 0000000..e0512d8
--- /dev/null
+++ b/experiment/env
@@ -0,0 +1,54 @@
+ BANK_HOSTS=nova-*.*
+DATABASE_HOSTS=gros-*.*
+EXCHANGE_HOSTS=grcinq-*.*
+MERCHANT_HOSTS=grvingt-*.*
+ MONITOR_HOSTS=econome-*.*
+ PROXY_HOSTS=grappe-*.*
+ WALLET_HOSTS=uvb-*.*
+
+DNS_HOST=taler.net
+DNS_ZONE=perf.taler
+
+ BANK_DOMAIN=bank.perf.taler
+DATABASE_DOMAIN=db.perf.taler
+EXCHANGE_DOMAIN=exch.perf.taler
+MERCHANT_DOMAIN=merch.perf.taler
+ MONITOR_DOMAIN=monitor.perf.taler
+ PROXY_DOMAIN=proxy.perf.taler
+
+# Wallet domain is also needed for cleardns.sh
+WALLET_DOMAIN=wallet.x.perf.taler
+
+EXCHANGE_ARGS="-L INFO"
+
+PROMETHEUS_DATASOURCE_NAME=Prometheus
+LOKI_DATASOURCE_NAME=Loki
+
+PROMETHEUS_G5K_PROXY_PORT=http8080
+LOKI_G5K_PROXY_PORT=http
+
+DB_NAME=taler-exchange
+DB_USER=taler
+DB_PASSWORD=
+
+NUM_WALLET_HOSTS=3
+
+# Initial number of wallets to start in parallel per host
+# This is a bulk size
+# running ./root/scripts/wallet.sh again will start another bulk
+# default = 10
+# Start another bunch with ./wallet.sh <NUN>
+NUM_WALLET_PROCESSES=10
+
+# Initial number of exchange processes to run
+# default = 10
+# Start another bunch with ./exchange.sh <NUM> AND ./proxy.sh <NUM>
+NUM_EXCHANGE_PROCESSES=1
+NUM_WIREWATTCH_PROCESSES=1
+
+# If prometheus node exporter should be enabled
+ENABLE_EXPORTERS=true
+
+DYNDNS_KEY=
+GRAFANA_HOST=
+GRAFANA_API_KEY=
diff --git a/experiment/experiment-specification.yml
b/experiment/experiment-specification.yml
new file mode 100755
index 0000000..11f8e3f
--- /dev/null
+++ b/experiment/experiment-specification.yml
@@ -0,0 +1,19 @@
+version: 1.0-basic
+rspec:
+ - bundled: infra.rspec
+upload:
+ - bundled: .env
+ - bundled: scripts
+ permission: "774"
+ - bundled: taler-perf.sh
+ permission: "700"
+ # Contains information about allocated nodes
+ - meta: experiment-info.json
+execute:
+ # Setup DNS and Environment config
+ - bundled: setup.sh
+ # Clear the DNS-Zone
+ - bundled: cleardns.sh
+ nodes: [Exchange]
+ # Execute node specific code
+ - bundled: run.sh
diff --git a/experiment/infra.rspec b/experiment/infra.rspec
new file mode 100644
index 0000000..01c2d7c
--- /dev/null
+++ b/experiment/infra.rspec
@@ -0,0 +1,43 @@
+<?xml version='1.0'?>
+<rspec xmlns="http://www.geni.net/resources/rspec/3"; type="request"
generated_by="jFed RSpec Editor" generated="2021-11-18T11:33:37.904+01:00"
xmlns:emulab="http://www.protogeni.net/resources/rspec/ext/emulab/1";
xmlns:delay="http://www.protogeni.net/resources/rspec/ext/delay/1";
xmlns:jfed-command="http://jfed.iminds.be/rspec/ext/jfed-command/1";
xmlns:client="http://www.protogeni.net/resources/rspec/ext/client/1";
xmlns:jfed-ssh-keys="http://jfed.iminds.be/rspec/ext/jfed-ssh-keys/1"; xmlns:
[...]
+ <node client_id="DB" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <hardware_type name="grvingt-nancy"/>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="156.0"
y="70.5"/>
+ </node>
+ <node client_id="Exchange" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <hardware_type name="grvingt-nancy"/>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="283.0"
y="127.5"/>
+ </node>
+ <node client_id="Bank" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <hardware_type name="grvingt-nancy"/>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="422.0"
y="72.5"/>
+ </node>
+ <node client_id="Proxy" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <hardware_type name="grvingt-nancy"/>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="284.5"
y="184.5"/>
+ </node>
+ <node client_id="Monitor" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="732.5"
y="156.5"/>
+ </node>
+ <node client_id="Merchant" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="554.5"
y="156.5"/>
+ </node>
+</rspec>
\ No newline at end of file
diff --git a/experiment/run.sh b/experiment/run.sh
new file mode 100644
index 0000000..85414fe
--- /dev/null
+++ b/experiment/run.sh
@@ -0,0 +1,85 @@
+#!/bin/bash
+# Run the experitment
+set -euax
+
+# They start the scripts with /bin/bash -c
+source .env
+
+# Set a dynamic domain name in our own dns
+# argument: the domain to be added
+function set_ddn() {
+ nsupdate -k /root/ddns.key -v << EOF
+server ${DNS_HOST}
+zone ${DNS_ZONE}
+update add $1 3600 A $(hostname -I)
+send
+EOF
+}
+
+function set_host() {
+ echo "TALER_HOST=$1" >> /etc/environment
+ echo "TALER_HOST=$1" >> /root/.env
+ export "TALER_HOST=$1"
+}
+
+function enable_netdelay() {
+ sed -i "s/<PING_DESTINATION>/${1}/g" \
+ /usr/lib/systemd/system/taler-netdelay.service
+ systemctl daemon-reload
+ systemctl restart taler-netdelay.timer
+}
+
+function enable_logbackup() {
+ systemctl restart taler-logbackup.timer
+}
+
+if [[ "${ENABLE_EXPORTERS}" == "true" ]];
+then
+ systemctl restart prometheus-node-exporter
+fi
+
+case "${HOSTNAME}" in
+ ${BANK_HOSTS})
+ set_host bank
+ set_ddn ${BANK_DOMAIN}
+ enable_logbackup
+ exec ~/scripts/bank.sh
+ ;;
+ ${DATABASE_HOSTS})
+ set_host database
+ set_ddn ${DATABASE_DOMAIN}
+ exec ~/scripts/database.sh init
+ ;;
+ ${EXCHANGE_HOSTS})
+ set_host exchange
+ set_ddn ${EXCHANGE_DOMAIN}
+ enable_netdelay ${DATABASE_DOMAIN}
+ enable_logbackup
+ exec ~/scripts/exchange.sh init
+ ;;
+ ${MERCHANT_HOSTS})
+ set_host merchant
+ set_ddn ${MERCHANT_DOMAIN}
+ exec ~/scripts/merchant.sh init
+ ;;
+ ${MONITOR_HOSTS})
+ set_host monitor
+ set_ddn ${MONITOR_DOMAIN}
+ exec ~/scripts/monitor.sh init
+ ;;
+ ${PROXY_HOSTS})
+ set_host proxy
+ set_ddn ${PROXY_DOMAIN}
+ enable_netdelay ${EXCHANGE_DOMAIN}
+ enable_logbackup
+ exec ~/scripts/proxy.sh init
+ ;;
+ ${WALLET_HOSTS})
+ set_host wallet
+ set_ddn "wallet.$(hostname | cut -d '.' -f1).${DNS_ZONE}"
+ enable_netdelay ${PROXY_DOMAIN}
+ exec ~/scripts/wallet.sh init
+ ;;
+esac
+
+exit 0
diff --git a/experiment/scripts/bank.sh b/experiment/scripts/bank.sh
new file mode 100755
index 0000000..4723a8c
--- /dev/null
+++ b/experiment/scripts/bank.sh
@@ -0,0 +1,10 @@
+#!/bin/bash
+set -eux
+
+source ~/scripts/helpers.sh
+
+restart_rsyslog
+
+systemctl restart taler-fakebank.service
+
+exit 0
diff --git a/experiment/scripts/benchmark.sh b/experiment/scripts/benchmark.sh
new file mode 100755
index 0000000..cf1aad3
--- /dev/null
+++ b/experiment/scripts/benchmark.sh
@@ -0,0 +1,23 @@
+#!/bin/bash
+
+LOG_LEVEL=ERROR
+
+case $1 in
+ *logging*)
+ LOG_LEVEL=INFO
+ ;;
+esac
+
+taler-wallet-cli \
+ -L ${LOG_LEVEL} \
+ advanced bench1 \
+ --config-json "
+{
+ \"exchange\": \"http://${PROXY_DOMAIN}/\";,
+ \"bank\": \"http://${BANK_DOMAIN}/\";,
+ \"currency\": \"KUDOS\",
+ \"payto\": \"payto://x-taler-bank/${BANK_DOMAIN}/foo\",
+ \"iterations\": 100000,
+ \"deposits\": 10,
+ \"restartAfter\": 2
+}"
diff --git a/experiment/scripts/createusers.sh
b/experiment/scripts/createusers.sh
new file mode 100755
index 0000000..a7de2a7
--- /dev/null
+++ b/experiment/scripts/createusers.sh
@@ -0,0 +1,110 @@
+#!/bin/bash
+set -e
+
+. /usr/share/debconf/confmodule
+
+TALER_HOME="/var/lib/taler"
+GROUPNAME=taler-exchange-secmod
+DBGROUPNAME=taler-exchange-db
+EUSERNAME=taler-exchange-httpd
+EGROUPNAME=www-data
+OUSERNAME=taler-exchange-offline
+OGROUPNAME=taler-exchange-offline
+CLOSERUSERNAME=taler-exchange-closer
+RSECUSERNAME=taler-exchange-secmod-rsa
+ESECUSERNAME=taler-exchange-secmod-eddsa
+AGGRUSERNAME=taler-exchange-aggregator
+WIREUSERNAME=taler-exchange-wire
+MERCHUSERNAME=taler-merchant-httpd
+MERCHGROUPNAME=www-data
+
+# Create taler groups as needed
+if ! getent group ${GROUPNAME} >/dev/null; then
+ addgroup --quiet --system ${GROUPNAME}
+fi
+if ! getent group ${DBGROUPNAME} >/dev/null; then
+ addgroup --quiet --system ${DBGROUPNAME}
+fi
+if ! getent group ${OGROUPNAME} >/dev/null; then
+ addgroup --quiet --system ${OGROUPNAME}
+fi
+
+# Create taler users if needed
+if ! getent passwd ${EUSERNAME} >/dev/null; then
+ adduser --quiet --system \
+ --no-create-home \
+ --ingroup ${GROUPNAME} \
+ --home ${TALER_HOME} ${EUSERNAME}
+ adduser --quiet ${EUSERNAME} ${DBGROUPNAME}
+fi
+if ! getent passwd ${RSECUSERNAME} >/dev/null; then
+ adduser --quiet --system \
+ --no-create-home \
+ --ingroup ${GROUPNAME} \
+ --home ${TALER_HOME} ${RSECUSERNAME}
+fi
+if ! getent passwd ${ESECUSERNAME} >/dev/null; then
+ adduser --quiet --system \
+ --no-create-home \
+ --ingroup ${GROUPNAME} \
+ --home ${TALER_HOME} ${ESECUSERNAME}
+fi
+if ! getent passwd ${WIREUSERNAME} >/dev/null; then
+ adduser --quiet --system \
+ --no-create-home \
+ --home ${TALER_HOME} ${WIREUSERNAME}
+ adduser --quiet ${WIREUSERNAME} ${DBGROUPNAME}
+fi
+if ! getent passwd ${CLOSERUSERNAME} >/dev/null; then
+ adduser --quiet --system \
+ --no-create-home \
+ --home ${TALER_HOME} ${CLOSERUSERNAME}
+ adduser --quiet ${CLOSERUSERNAME} ${DBGROUPNAME}
+fi
+if ! getent passwd ${AGGRUSERNAME} >/dev/null; then
+ adduser --quiet --system \
+ --no-create-home \
+ --home ${TALER_HOME} ${AGGRUSERNAME}
+ adduser --quiet ${AGGRUSERNAME} ${DBGROUPNAME}
+fi
+if ! getent passwd ${OUSERNAME} >/dev/null; then
+ adduser --quiet --system \
+ --ingroup ${OGROUPNAME} \
+ --no-create-home \
+ --home ${TALER_HOME} ${OUSERNAME}
+fi
+if ! getent passwd ${MERCHUSERNAME} >/dev/null; then
+ adduser --quiet --system \
+ --no-create-home \
+ --ingroup ${MERCHGROUPNAME} \
+ --home ${TALER_HOME} ${MERCHUSERNAME}
+fi
+
+install -d /var/lib/taler/exchange-offline -m 0700 -o ${OUSERNAME} -g
${OGROUPNAME}
+install -d /run/taler/exchange-secmod-rsa -m 0755 -o ${RSECUSERNAME} -g
${GROUPNAME}
+install -d /run/taler/exchange-secmod-eddsa -m 0755 -o ${ESECUSERNAME} -g
${GROUPNAME}
+install -d /run/taler/exchange-httpd -m 0750 -o ${EUSERNAME} -g ${EGROUPNAME}
+install -d /var/lib/taler/exchange-offline -m 0700 -o ${OUSERNAME} -g
${OGROUPNAME}
+install -d /var/lib/taler/exchange-secmod-rsa -m 0700 -o ${RSECUSERNAME} -g
${GROUPNAME}
+install -d /var/lib/taler/exchange-secmod-eddsa -m 0700 -o ${ESECUSERNAME} -g
${GROUPNAME}
+install -d /run/taler/merchant-httpd -m 0755 -o ${MERCHUSERNAME} -g
${MERCHGROUPNAME}
+
+if ! dpkg-statoverride --list
/etc/taler/secrets/exchange-accountcredentials.secret.conf >/dev/null 2>&1; then
+ dpkg-statoverride --add --update \
+ ${WIREUSERNAME} root 460 \
+ /etc/taler/secrets/exchange-accountcredentials.secret.conf
+fi
+
+if ! dpkg-statoverride --list /etc/taler/secrets/exchange-db.secret.conf
>/dev/null 2>&1; then
+ dpkg-statoverride --add --update \
+ root ${DBGROUPNAME} 660 \
+ /etc/taler/secrets/exchange-db.secret.conf
+fi
+
+if ! dpkg-statoverride --list /etc/taler/secrets/merchant-db.secret.conf
>/dev/null 2>&1; then
+ dpkg-statoverride --add --update \
+ ${MERCHUSERNAME} root 460 \
+ /etc/taler/secrets/merchant-db.secret.conf
+fi
+
+exit 0
diff --git a/experiment/scripts/database.sh b/experiment/scripts/database.sh
new file mode 100755
index 0000000..5bd2472
--- /dev/null
+++ b/experiment/scripts/database.sh
@@ -0,0 +1,72 @@
+#!/bin/bash
+set -eux
+
+source ~/scripts/helpers.sh
+
+# move to tmp to prevent change dir errors
+cd /tmp
+
+if [[ "$1" == "init" ]];
+then
+ sed -i "s\<DB_URL_HERE>\postgresql:///${DB_NAME}\g" \
+ /etc/taler/secrets/exchange-db.secret.conf
+
+ echo "
+ listen_addresses='*'
+ log_destination=syslog
+ syslog_ident='taler-database'
+ log_min_duration_statement=500
+ shared_preload_libraries='pg_stat_statements,auto_explain'
+
+ # use 25% of the available memory
+ # (https://www.postgresql.org/docs/13/runtime-config-resource.html)
+ shared_buffers=$(($(awk '/MemTotal/ {print $2}' /proc/meminfo) / 4 ))kB
+ effective_cache_size=$(($(awk '/MemTotal/ {print $2}' /proc/meminfo) *
3/4))kB
+ #
(https://www.postgresql.org/docs/current/runtime-config-wal.html#GUC-MAX-WAL-SIZE)
+ max_wal_size=2GB
+ max_worker_processes=16
+ max_connections=500
+ " >> /etc/postgresql/13/main/postgresql.conf
+
+ # Enable password for taler since this is commonly the case
+ # For the postgres user do not enable authentication (used in metrics)
+ echo "
+ host all ${DB_USER} 172.16.0.0/12 md5
+ host all postgres 172.16.0.0/12 trust
+ " >> /etc/postgresql/13/main/pg_hba.conf
+fi
+
+systemctl restart postgresql
+
+su postgres << EOF
+psql postgres -tAc "SELECT 1 FROM pg_roles WHERE
rolname='taler-exchange-httpd'" | \
+ grep -q 1 || \
+ createuser taler-exchange-httpd
+psql -tAc "SELECT 1 FROM pg_database WHERE datname='${DB_NAME}'" | \
+ grep -q 1 || \
+ createdb -O taler-exchange-httpd "${DB_NAME}"
+EOF
+
+if [[ "$1" == "init" ]];
+then
+ sudo -u taler-exchange-httpd taler-exchange-dbinit
+fi
+
+su postgres << EOF
+psql postgres -tAc "SELECT 1 FROM pg_roles WHERE rolname='${DB_USER}'" | \
+ grep -q 1 \
+ || psql << END
+ CREATE USER "${DB_USER}" with encrypted password '${DB_PASSWORD}';
+ CREATE EXTENSION pg_stat_statements;
+END
+EOF
+
+su taler-exchange-httpd -s /bin/bash << EOF
+psql -d "${DB_NAME}"
+GRANT SELECT,INSERT,UPDATE ON ALL TABLES IN SCHEMA public TO "${DB_USER}";
+GRANT USAGE ON ALL SEQUENCES IN SCHEMA public TO "${DB_USER}";
+EOF
+
+restart_rsyslog
+
+exit 0
diff --git a/experiment/scripts/exchange.sh b/experiment/scripts/exchange.sh
new file mode 100755
index 0000000..6a7656a
--- /dev/null
+++ b/experiment/scripts/exchange.sh
@@ -0,0 +1,61 @@
+#!/bin/bash
+set -eux
+
+if [[ "$1" == "init" ]];
+then
+ source ~/scripts/helpers.sh
+ restart_rsyslog
+
+ sed -i
"s\<DB_URL_HERE>\postgresql://${DB_USER}:${DB_PASSWORD}@${DATABASE_DOMAIN}/${DB_NAME}\g"
\
+ /etc/taler/secrets/exchange-db.secret.conf
+
+ sed -i "s/<BANK_HOST_HERE>/${BANK_DOMAIN}/g" \
+ /etc/taler/secrets/exchange-accountcredentials.secret.conf
+
+ MASTER_KEY=$(sudo -u taler-exchange-offline taler-exchange-offline setup)
+
+ sed -i -e "s/<BANK_HOST_HERE>/${BANK_DOMAIN}/g" \
+ -e "s\<BASE_URL_HERE>\http://${EXCHANGE_DOMAIN}/\g"; \
+ -e "s/<MASTER_KEY_HERE>/${MASTER_KEY}/g" \
+ /etc/taler/conf.d/exchange-business.conf
+
+ wait_for_db
+ # Wait another second to make sure user has permissions
+ sleep 5
+
+ NUM_PROCESSES=$((${NUM_EXCHANGE_PROCESSES:-10}-1))
+
+ systemctl restart taler-exchange.target
+
+ wait_for_keys "${EXCHANGE_DOMAIN}/management"
+
+ sleep 5
+
+ taler-exchange-offline download > sig-req.json
+ taler-exchange-offline sign < sig-req.json > sig-res.json
+ taler-exchange-offline enable-account
"payto://x-taler-bank/${BANK_DOMAIN}/Exchange" > acct-res.json
+ taler-exchange-offline wire-fee 2021 x-taler-bank KUDOS:0 KUDOS:0 >
fee-res.json
+ taler-exchange-offline upload < sig-res.json
+ taler-exchange-offline upload < acct-res.json
+ taler-exchange-offline upload < fee-res.json
+
+ let "NUM_WIREWATCH_PROCESSES-=1"
+ for i in $(seq ${NUM_WIREWATCH_PROCESSES});
+ do
+ systemctl restart taler-exchange-wirewatch@${i}.service
+ done
+
+else
+ NUM_PROCESSES=$1
+fi
+
+RUNNING=$(ps -aux | grep "[taler]-exchange-httpd" | wc -l)
+
+for i in $(seq ${NUM_PROCESSES});
+do
+ let "i+=${RUNNING}-1"
+ let "i+=10000"
+ systemctl restart taler-exchange-httpd@"${i}".socket
taler-exchange-httpd@"${i}".service
+done
+
+exit 0
diff --git a/experiment/scripts/helpers.sh b/experiment/scripts/helpers.sh
new file mode 100755
index 0000000..9cc0351
--- /dev/null
+++ b/experiment/scripts/helpers.sh
@@ -0,0 +1,47 @@
+#/!bin/bash
+
+function wait_for_db() {
+ until PGPASSWORD="${DB_PASSWORD}" psql \
+ -h "${DATABASE_DOMAIN}" \
+ -U "${DB_USER}" \
+ -d "${DB_NAME}" \
+ -c '\q';
+ do
+ echo "Database not ready yet"
+ sleep 2
+ done
+}
+
+# Arguments
+# 1: Domain to request /keys from
+function wait_for_keys() {
+ until wget http://${1}/keys \
+ --spider \
+ --timeout=5 \
+ --tries=1 \
+ --quiet;
+ do
+ echo "Exchange not ready yet"
+ sleep 5
+ done
+}
+
+function restart_rsyslog() {
+ # rsyslg fails to apply the taler rule if remote is not reachable
+ while ! nc -z ${MONITOR_DOMAIN} 1514;
+ do
+ echo "Waiting for promtail"
+ sleep 2
+ done
+
+ systemctl restart rsyslog
+}
+
+function get_wallet_domains() {
+ IFS=$'\n' read -r -d '' -a WALLETS < <(\
+ dig -t AXFR "${DNS_ZONE}" "@${DNS_HOST}" \
+ | grep wallet | awk '{print $1}' | cut -d '.' -f 2 \
+ )
+ echo ${WALLETS[@]}
+}
+
diff --git a/experiment/scripts/log-save.sh b/experiment/scripts/log-save.sh
new file mode 100755
index 0000000..bbdfe74
--- /dev/null
+++ b/experiment/scripts/log-save.sh
@@ -0,0 +1,21 @@
+#!/bin/bash
+# Runs logrotate on the logs written to LOG_DIR
+# Assumes rsyslog to be configured to log files there
+
+LOG_DIR=/tmp/taler
+
+EXPORT_DIR=/home/${G5K_USER}/taler-logs
+
+if [ ! -d "${EXPORT_DIR}" ]; then
+ mkdir "${EXPORT_DIR}"
+fi
+
+logrotate /etc/logrotate.d/taler
+
+TS=$(date +%s)
+
+if [ -d "${LOG_DIR}" ]; then
+ for FILE in $(find ${LOG_DIR} -iname "*.gz"); do
+ mv ${FILE} ${EXPORT_DIR}/$(basename -- ${FILE%.gz}.${TS}.gz)
+ done
+fi
diff --git a/experiment/scripts/merchant.sh b/experiment/scripts/merchant.sh
new file mode 100755
index 0000000..ceeaa2e
--- /dev/null
+++ b/experiment/scripts/merchant.sh
@@ -0,0 +1,54 @@
+#!/bin/bash
+set -eux
+
+source ~/scripts/helpers.sh
+
+cd /tmp
+
+restart_rsyslog
+
+if [[ "$1" == "init" ]];
+then
+ wait_for_keys "${PROXY_DOMAIN}"
+
+ MASTER_KEY=$(
+ curl -k -f \
+ "${PROXY_DOMAIN}/keys" \
+ | jq -r '.master_public_key'
+ )
+
+ sed -i -e "s\<EXCHANGE_URL_HERE>\http://${EXCHANGE_DOMAIN}/\g"; \
+ -e "s/<EXCHANGE_MASTER_KEY_HERE>/${MASTER_KEY}/g" \
+ /etc/taler/conf.d/merchant.conf
+
+ sed -i "s/local\s* all\s* postgres\s* peer/local all postgres trust/g" \
+ /etc/postgresql/13/main/pg_hba.conf
+
+ echo "
+ shared_preload_libraries='pg_stat_statements,auto_explain'
+ " >> /etc/postgresql/13/main/postgresql.conf
+fi
+
+systemctl restart postgresql
+
+su postgres << EOF
+psql postgres -tAc "SELECT 1 FROM pg_roles WHERE
rolname='taler-merchant-httpd'" | \
+ grep -q 1 || \
+ createuser taler-merchant-httpd
+psql -tAc "SELECT 1 FROM pg_database WHERE datname='taler-merchant'" | \
+ grep -q 1 || \
+ createdb -O taler-merchant-httpd taler-merchant
+psql
+CREATE EXTENSION pg_stat_statements;
+EOF
+
+if [[ "$1" == "init" ]];
+then
+ sudo -u taler-merchant-httpd taler-merchant-dbinit
+fi
+
+sed -i "s\<DB_URL_HERE>\user=postgres host=/run/postgresql
dbname=taler-merchant\g" \
+ /etc/default/prometheus-postgres-exporter
+
+systemctl restart taler-merchant-httpd \
+ prometheus-postgres-exporter
diff --git a/experiment/scripts/monitor.sh b/experiment/scripts/monitor.sh
new file mode 100755
index 0000000..0b15817
--- /dev/null
+++ b/experiment/scripts/monitor.sh
@@ -0,0 +1,81 @@
+#!/bin/bash
+set -ex
+
+source ~/scripts/helpers.sh
+
+if [[ "$1" == "init" ]];
+then
+ AUTH_HEADER="Authorization: Bearer ${GRAFANA_API_KEY}"
+ GRAFANA_API="https://$(host ${GRAFANA_HOST} | sed -n 1p | awk '{print
$4}'):3000/api"
+
+ if ! curl -k -f -H "${AUTH_HEADER}" \
+ "${GRAFANA_API}/datasources" \
+ -o ds.json ;
+ then
+ echo "Failed to retrieve datasources from Grafana"
+ exit $?
+ fi
+
+ function update_datasource() {
+ ID=$(jq --arg name "$1" '.[] | select(.name == $name) | .id' ds.json)
+
+ jq --arg url "https://$(hostname | cut -d "." -f 1,2
-).$2.proxy.grid5000.fr" \
+ --arg name "$1" \
+ '.[] | select(.name == $name) | .url = $url' \
+ ds.json | tee /dev/tty | curl -X PUT -k -f -d @- \
+ -H "${AUTH_HEADER}" \
+ -H "Content-Type: application/json" \
+ -H "Accept: application/json" \
+ "${GRAFANA_API}/datasources/${ID}"
+ }
+
+ update_datasource "${PROMETHEUS_DATASOURCE_NAME}"
"${PROMETHEUS_G5K_PROXY_PORT}"
+ update_datasource "${LOKI_DATASOURCE_NAME}" "${LOKI_G5K_PROXY_PORT}"
+
+ systemctl restart loki \
+ promtail
+
+ sed -i "s/<MERCHANT_HOST_HERE>/${MERCHANT_DOMAIN}/g" \
+ /etc/monitor/prometheus.yaml
+
+ sed -i "s\<DB_URL_HERE>\postgresql://postgres@${DATABASE_DOMAIN}:5432\g" \
+ /etc/default/prometheus-postgres-exporter
+
+ sed -i "s\<PROXY_URL_HERE>\http://${PROXY_DOMAIN}/stub_status\g"; \
+ /etc/default/prometheus-nginx-exporter
+
+ wait_for_db
+
+ # Initialize prometheus after the db is ready, then all dns records have
been set for sure
+ if [[ "${ENABLE_EXPORTERS}" == "true" ]];
+ then
+ cat /etc/monitor/node-exporters.yaml.tpl >> /etc/monitor/prometheus.yaml
+ for WALLET in $(get_wallet_domains);
+ do
+ sed -i "/<WALLETS_HERE>/a \ \ \ \ \ \ -
'wallet.${WALLET}.perf.taler:9100'" \
+ /etc/monitor/prometheus.yaml
+ done
+ fi
+
+ # Proxy takes longer to start
+ wait_for_keys "${PROXY_DOMAIN}"
+
+ systemctl restart prometheus-postgres-exporter \
+ prometheus-nginx-exporter
+else
+
+ if [[ "${ENABLE_EXPORTERS}" == "true" ]];
+ then
+ for WALLET in $(get_wallet_domains);
+ do
+ if ! grep -q "wallet.${WALLET}.perf.taler:9100"
/etc/monitor/prometheus.yaml;
+ then
+ sed -i "/<WALLETS_HERE>/a \ \ \ \ \ \ -
'wallet.${WALLET}.perf.taler:9100'" \
+ /etc/monitor/prometheus.yaml
+ fi
+ done
+ fi
+
+fi
+
+systemctl restart prometheus
diff --git a/experiment/scripts/ping.sh b/experiment/scripts/ping.sh
new file mode 100755
index 0000000..9f82ca0
--- /dev/null
+++ b/experiment/scripts/ping.sh
@@ -0,0 +1,13 @@
+#!/bin/bash
+
+DELAY=$(ping -c 2 $1 | sed -n "3p" | awk '{print $(NF-1)$NF}')
+
+if [[ "$DELAY" == "time="* ]];
+then
+ logger -s --tcp \
+ --port 1514 \
+ --server ${MONITOR_DOMAIN} \
+ --tag taler-network \
+ "src=${TALER_HOST} dst=${1} ${DELAY}" \
+ || true # Ignore errors (mostly because NXDOMAIN)
+fi
diff --git a/experiment/scripts/proxy.sh b/experiment/scripts/proxy.sh
new file mode 100755
index 0000000..5e04919
--- /dev/null
+++ b/experiment/scripts/proxy.sh
@@ -0,0 +1,55 @@
+#!/bin/bash
+set -eux
+
+if [[ "$1" == "init" ]];
+then
+ source ~/scripts/helpers.sh
+
+ sed -e "/<SERVERS_HERE>/a \ \ server ${EXCHANGE_DOMAIN}:80;" \
+ -e "s/<MONITOR_HOST_HERE>/${MONITOR_DOMAIN}/g" \
+ /etc/nginx/sites-available/proxy > /etc/nginx/sites-enabled/proxy
+
+ # We want n processes, one is already enabled on port 80
+ for (( i=1; i < ${NUM_EXCHANGE_PROCESSES}; i++ ))
+ do
+ let "PORT=i+10000"
+ sed -i "/<SERVERS_HERE>/a \ \ server ${EXCHANGE_DOMAIN}:${PORT};" \
+ /etc/nginx/sites-enabled/proxy
+ done
+
+ sed -i -e '/module(load="imudp")/s/^#//g' \
+ -e '/input(type="imudp" port="514")/s/^#//g' \
+ /etc/rsyslog.conf
+
+ systemctl restart rsyslog
+
+ echo "
+ fs.file-max=50000
+ " >> /etc/sysctl.conf
+
+ echo "
+ www-data soft nofile unlimited
+ www-data hard nofile unlimited
+ " >> /etc/security/limits.conf
+
+ sysctl -p
+
+ # Nginx does not start until the destination server is reachable - wait here
+ # nginx: [emerg] host not found in upstream "exch.perf.taler" ...
+ wait_for_keys "${EXCHANGE_DOMAIN}"
+
+ systemctl restart nginx
+else
+
+ ADDED=$(grep -r " server ${EXCHANGE_DOMAIN}:"
/etc/nginx/sites-enabled/proxy | wc -l)
+
+ for i in $(seq $1); do
+ let "i+=${ADDED}-1"
+ let "i+=10000"
+ sed -i "/<SERVERS_HERE>/a \ \ server ${EXCHANGE_DOMAIN}:${i};" \
+ /etc/nginx/sites-enabled/proxy
+ done
+
+fi
+
+systemctl reload nginx
diff --git a/experiment/scripts/wallet.sh b/experiment/scripts/wallet.sh
new file mode 100755
index 0000000..64c53b8
--- /dev/null
+++ b/experiment/scripts/wallet.sh
@@ -0,0 +1,54 @@
+#!/bin/bash
+set -eux
+
+if [[ "$1" == "init" ]];
+then
+ source ~/scripts/helpers.sh
+ restart_rsyslog
+ wait_for_keys "${PROXY_DOMAIN}"
+ sleep 5
+
+ NUM_PROCESSES=${NUM_WALLET_PROCESSES:-10}
+
+ if [ "$NUM_PROCESSES" -gt "0" ];
+ then
+ # Start one process with logging enabled
+ let "NUM_PROCESSES-=1"
+ systemctl restart taler-wallet@logging.service
+ fi
+else
+ NUM_PROCESSES=$2
+fi
+
+# count the running wallets
+RUNNING=$(ps -aux | grep "[wallet]-cli" | wc -l)
+
+case "$1" in
+ stop)
+ if [[ "${NUM_PROCESSES}" = "logging" ]]; then
+ systemctl stop taler-wallet@logging.service
+ else
+ let "START=${RUNNING}-${NUM_PROCESSES}+1"
+ for i in $(seq ${START} ${RUNNING})
+ do
+ systemctl stop taler-wallet@${i}.service
+ done
+ fi
+ ;;
+ start|init)
+ CPU_USAGE=$[100-$(vmstat 1 2 | tail -1 | awk '{print $15}')]
+ echo "CPU Usage: ${CPU_USAGE}%"
+ if [ "${CPU_USAGE}" -gt "75" ];
+ then
+ echo "Not starting any more wallets"
+ exit 1
+ fi
+ for i in $(seq ${NUM_PROCESSES})
+ do
+ let "i+=${RUNNING}"
+ systemctl restart taler-wallet@${i}.service
+ done
+ ;;
+esac
+
+exit 0
diff --git a/experiment/setup.sh b/experiment/setup.sh
new file mode 100644
index 0000000..4655a9e
--- /dev/null
+++ b/experiment/setup.sh
@@ -0,0 +1,60 @@
+#!/bin/bash
+# Setup nodes for the experiment
+set -euax
+
+source .env
+# Add the environment config for following shells
+cat .env | tee -a /etc/environment
+echo "G5K_USER=$(cat experiment-info.json | jq -r '.user.name')" >>
/etc/environment
+
+G5K_HOME=/root/taler/grid5k
+
+NS_IP=$(host ${DNS_HOST} | sed -n 1p | awk '{print $4}')
+
+# Temporarily checkout to the feature branch
+cd "${G5K_HOME}" && git checkout node-setup && git pull && cd
+
+# Remove default nginx config
+rm /etc/nginx/sites-enabled/default > /dev/null 2>&1 || true
+
+# Override default configurations with the one from this Git.
+cp -r "${G5K_HOME}"/configs/* /
+
+# Send all logs about taler to promtail on the monitoring node
+sed -i "s/<MONITOR_DOMAIN_HERE>/${MONITOR_DOMAIN}/g" \
+ /etc/rsyslog.d/taler.conf
+
+sed -i "s/<ARGUMENTS_HERE>/${EXCHANGE_ARGS}/g" \
+ /etc/default/taler-exchange
+
+# Add the dyndns key which will be used by the dns update
+cat << EOF > /root/ddns.key
+key "ddns-key.perf.taler" {
+ algorithm hmac-sha256;
+ secret "${DYNDNS_KEY}";
+};
+EOF
+
+if ! grep -Fxq "server=/${DNS_ZONE}/${NS_IP}" /etc/dnsmasq.conf ; then
+ echo "server=/${DNS_ZONE}/${NS_IP}" >> /etc/dnsmasq.conf
+ # Read grid5k DNS servers from resolv.conf and add them to dnsmasq
+ grep nameserver /etc/resolv.conf | \
+ awk '{print "server="$2}' \
+ >> /etc/dnsmasq.conf
+fi
+
+if ! grep -Fxq "nameserver 127.0.0.1" /etc/resolv.conf ; then
+ mv /etc/resolv.conf /etc/resolv.conf.bak
+ echo "nameserver 127.0.0.1" > /etc/resolv.conf
+fi
+
+systemctl daemon-reload
+systemctl restart dnsmasq
+
+if [ -f ~/taler-perf.sh ]; then
+ mv ~/taler-perf.sh /usr/local/bin/taler-perf
+fi
+
+exec ~/scripts/createusers.sh
+
+exit 0
diff --git a/experiment/taler-perf.sh b/experiment/taler-perf.sh
new file mode 100644
index 0000000..9a39540
--- /dev/null
+++ b/experiment/taler-perf.sh
@@ -0,0 +1,84 @@
+#!/bin/bash
+set -e
+
+source /etc/environment
+source ~/scripts/helpers.sh
+
+function start_wallets() {
+ for WALLET in $(get_wallet_domains); do
+ ssh -o StrictHostKeyChecking=no wallet.${WALLET}.perf.taler \
+ "/bin/bash /root/scripts/wallet.sh start
${1:-${NUM_WALLET_PROCESSES}}" &
+ done
+ wait
+}
+
+function stop_wallets() {
+ for WALLET in $(get_wallet_domains); do
+ ssh -o StrictHostKeyChecking=no wallet.${WALLET}.perf.taler \
+ "/bin/bash /root/scripts/wallet.sh stop
${1:-${NUM_WALLET_PROCESSES}}" &
+ done
+ wait
+}
+
+function start_exchanges() {
+ ssh -o StrictHostKeyChecking=no ${EXCHANGE_DOMAIN} \
+ "/bin/bash /root/scripts/exchange.sh ${1:-${NUM_EXCHANGE_PROCESSES}}"
+ ssh -o StrictHostKeyChecking=no ${PROXY_DOMAIN} \
+ "/bin/bash /root/scripts/proxy.sh ${1:-${NUM_EXCHANGE_PROCESSES}}"
+}
+
+function start_processes() {
+ case "$1" in
+ wallet)
+ start_wallets $2
+ ;;
+ exchange)
+ start_exchanges $2
+ ;;
+ *)
+ echo "Unknown argument '$1' for function ${FUNCNAME[0]}"
+ echo "Usage: start [wallet|exchange] NUM"
+ ;;
+ esac
+}
+
+function stop_processes() {
+ case "$1" in
+ wallet)
+ stop_wallets $2
+ ;;
+ *)
+ echo "Unknown argument '$1' for function ${FUNCNAME[0]}"
+ echo "Usage: stop [wallet] NUM"
+ ;;
+ esac
+}
+
+function update_processes() {
+ case "$1" in
+ prometheus)
+ ssh -o StrictHostKeyChecking=no ${MONITOR_DOMAIN} "/bin/bash -c
/root/scripts/monitor.sh"
+ ;;
+ *)
+ echo "Unknown argument '$1' for function ${FUNCNAME[0]}"
+ echo "Usage: update [prometheus]"
+ ;;
+ esac
+}
+
+case "$1" in
+ start)
+ shift
+ start_processes $@
+ ;;
+ stop)
+ shift
+ stop_processes $@
+ ;;
+ update)
+ shift
+ update_processes $@
+ ;;
+esac
+
+exit 0
diff --git a/experiment/wallets.rspec b/experiment/wallets.rspec
new file mode 100644
index 0000000..b80b714
--- /dev/null
+++ b/experiment/wallets.rspec
@@ -0,0 +1,63 @@
+<?xml version='1.0'?>
+<rspec xmlns="http://www.geni.net/resources/rspec/3"; type="request"
generated_by="jFed RSpec Editor" generated="2021-11-09T14:54:19.867+01:00"
xmlns:emulab="http://www.protogeni.net/resources/rspec/ext/emulab/1";
xmlns:delay="http://www.protogeni.net/resources/rspec/ext/delay/1";
xmlns:jfed-command="http://jfed.iminds.be/rspec/ext/jfed-command/1";
xmlns:client="http://www.protogeni.net/resources/rspec/ext/client/1";
xmlns:jfed-ssh-keys="http://jfed.iminds.be/rspec/ext/jfed-ssh-keys/1"; xmlns:
[...]
+ <node client_id="Wallet-1" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="130" y="87.0"/>
+ </node>
+ <node client_id="Wallet-2" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="280" y="87.0"/>
+ </node>
+ <node client_id="Wallet-3" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="430" y="87.0"/>
+ </node>
+ <node client_id="Wallet-4" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="580" y="87.0"/>
+ </node>
+ <node client_id="Wallet-5" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="730" y="87.0"/>
+ </node>
+ <node client_id="Wallet-6" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="130"
y="145.0"/>
+ </node>
+ <node client_id="Wallet-7" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="280"
y="145.0"/>
+ </node>
+ <node client_id="Wallet-8" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="430"
y="145.0"/>
+ </node>
+ <node client_id="Wallet-9" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="580"
y="145.0"/>
+ </node>
+ <node client_id="Wallet-10" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="730"
y="145.0"/>
+ </node>
+</rspec>
\ No newline at end of file
diff --git a/grid5000/debian11-x64-min.yaml b/grid5000/debian11-x64-min.yaml
deleted file mode 100644
index 969fd3b..0000000
--- a/grid5000/debian11-x64-min.yaml
+++ /dev/null
@@ -1,27 +0,0 @@
-#==============================================================================
-# vim: softtabstop=2 shiftwidth=2 expandtab fenc=utf-8 cc=81 tw=80
-#==============================================================================
-#
-# DESCRIPTION: Debian bullseye (11) x64 min Grid'5000 environment
-#
-#==============================================================================
----
-extend: ./debian11-x64-common.yaml
-
-global:
- # Grid'5000 environment variant
- g5k_variant: min
- # clean_unnecessary_packages step settings (packages debfoster must keep)
- default_packages_no_clean:
g5k-meta-packages-$${distrib}$${release_number}-$${g5k_variant} tgz-g5k gnupg
linux-image-$${deb_arch} console-setup rsync locales firmware-bnx2
firmware-bnx2x firmware-qlogic
- arch_packages_no_clean: grub-pc grub-efi-amd64-bin
- other_packages_no_clean:
-
-bootstrap:
- - "@base"
-
-setup:
- - clean_unnecessary_packages
- - "@base"
-
-export:
- - "@base"
diff --git a/grid5000/from_scratch/aarch64/base.yaml
b/grid5000/from_scratch/aarch64/base.yaml
deleted file mode 100644
index 3b6d58b..0000000
--- a/grid5000/from_scratch/aarch64/base.yaml
+++ /dev/null
@@ -1,25 +0,0 @@
-#==============================================================================
-# vim: softtabstop=2 shiftwidth=2 expandtab fenc=utf-8 cc=81 tw=80
-#==============================================================================
-#
-# DESCRIPTION: arm64 base recipe
-#
-#==============================================================================
----
-extend: ../base.yaml
-# Global variables use by Kameleon engine and the steps
-global:
- arch: aarch64
- installer_iso_arch: aarch64
- qemu_arch: aarch64
- qemu_enable_kvm: true
- qemu_uefi: true
-
-bootstrap:
- - "@base"
-
-setup:
- - "@base"
-
-export:
- - "@base"
diff --git a/grid5000/from_scratch/aarch64/debian-base.yaml
b/grid5000/from_scratch/aarch64/debian-base.yaml
deleted file mode 100644
index 4450198..0000000
--- a/grid5000/from_scratch/aarch64/debian-base.yaml
+++ /dev/null
@@ -1,59 +0,0 @@
-#==============================================================================
-# vim: softtabstop=2 shiftwidth=2 expandtab fenc=utf-8 cc=81 tw=80
-#==============================================================================
-#
-# DESCRIPTION: Debian generic recipe using the netinstall mechanim
-#
-# USAGE:
-# Select directly in this recipe: see usage example commented in the global
of
-# this recipe
-#
-# or, override the globals directly in CLI. For example:
-#
-# kameleon build --global distrib:debian,release:wheezy
-#
-# or extends this recipe with your own and override those variable in it.
-#
-#==============================================================================
----
-extend: base.yaml
-
-global:
- # Boilerplate values, so that `kameleon info' works with the recipe.
- # For a specific version of Debian, please see the dedicated recipe, as this
- # recipe is mainly meant as being extended.
- distrib: debian
- deb_arch: arm64
- release: buster
- release_number: 10
-
- # URL to retrieve packages from (sources.list)
- deb_mirror_hostname: deb.debian.org
- deb_mirror_directory: /debian
- deb_mirror_uri: http://$${deb_mirror_hostname}$${deb_mirror_directory}
- deb_security_hostname: security.debian.org
- deb_security_directory: /debian
- deb_components: main contrib non-free
-
- qemu_iso_path:
- installer_iso_finder_helper:
- # Debian netinstall iso refuses to get keyborad keys via the qemu sendkey
command on ARM64.
- # Because of that, we cannot set the preseed in the iso image cdrom boot in
qemu.
- # As a consequence, we use the netboot instead and boot kernel and initrd
directly
- installer_kernel_url:
$${deb_mirror_uri}/dists/$${release}/main/installer-$${deb_arch}/current/images/netboot/debian-installer/$${deb_arch}/linux
- installer_initrd_url:
$${deb_mirror_uri}/dists/$${release}/main/installer-$${deb_arch}/current/images/netboot/debian-installer/$${deb_arch}/initrd.gz
- # Debian arm64 netboot requires to give explicitly auto=true and
priority=critical
- # (the "auto" alias seems not to only be defined for amd64).
- # FIXME find the Debian documentation page which explains that.
- installer_cmdline: "auto=true priority=critical
url=http://%LOCAL_IP%:%HTTP_PORT%/preseed.cfg";
- base_preseed_path:
$${kameleon_data_dir}/preseed/$${distrib}-$${release}-preseed.cfg
- preseed_path: $${kameleon_cwd}/preseed.cfg
-
-bootstrap:
- - "@base"
-
-setup:
- - "@base"
-
-export:
- - "@base"
diff --git a/grid5000/from_scratch/debian-base.yaml
b/grid5000/from_scratch/debian-base.yaml
deleted file mode 100644
index 447e57d..0000000
--- a/grid5000/from_scratch/debian-base.yaml
+++ /dev/null
@@ -1,67 +0,0 @@
-#==============================================================================
-# vim: softtabstop=2 shiftwidth=2 expandtab fenc=utf-8 cc=81 tw=80
-#==============================================================================
-#
-# DESCRIPTION: Debian generic recipe using the netinstall mechanim
-#
-# USAGE:
-# Select directly in this recipe: see usage example commented in the global
of
-# this recipe
-#
-# or, override the globals directly in CLI. For example:
-#
-# kameleon build --global distrib:debian,release:wheezy
-#
-# or extends this recipe with your own and override those variable in it.
-#
-#==============================================================================
----
-extend: base.yaml
-
-global:
- # Boilerplate values, so that `kameleon info' works with the recipe.
- # For a specific version of Debian, please see the dedicated recipe, as this
- # recipe is mainly meant as being extended.
- distrib: debian
- deb_arch: amd64
- release: jessie
- release_number: 8
-
- # URL to retrieve packages from (sources.list)
- deb_mirror_hostname: deb.debian.org
- deb_mirror_directory: /debian
- deb_mirror_uri: http://$${deb_mirror_hostname}$${deb_mirror_directory}
- deb_security_hostname: security.debian.org
- deb_security_directory: /debian
- deb_components: main contrib non-free
-
- # Install from the installer's iso
- # The location of the Debian netinstall iso can be set manually or guessed
- # using a url finder helper script
- #installer_iso_filename: debian-$${release_number}-$${deb_arch}-netinst.iso
- #installer_iso_location: archive
- #installer_iso_release_version: 8.0.0
- #installer_iso_url:
http://cdimage.debian.org/cdimage/$${installer_iso_location}/$${installer_iso_release_version}/$${deb_arch}/iso-cd/$${installer_iso_filename}
- installer_iso_url:
- installer_iso_finder_helper:
$${kameleon_data_dir}/helpers/netinstall_iso_finder.py
- installer_iso_finder_args: $${distrib} $${release_number} $${deb_arch}
- qemu_iso_path: $${kameleon_cwd}/$${distrib}.iso
- # Or install from the netboot kernel and initrd directly
- #installer_kernel_url:
http://deb.debian.org/debian/dists/$${release}/main/installer-$${deb_arch}/current/images/netboot/debian-installer/$${deb_arch}/linux
- #installer_initrd_url:
http://deb.debian.org/debian/dists/$${release}/main/installer-$${deb_arch}/current/images/netboot/debian-installer/$${deb_arch}/initrd.gz
- #installer_cmdline: "auto url=http://%LOCAL_IP%:%HTTP_PORT%/preseed.cfg";
-
- base_preseed_path:
$${kameleon_data_dir}/preseed/$${distrib}-$${release}-preseed.cfg
- preseed_path: $${kameleon_cwd}/preseed.cfg
-
- qemu_sendkeys_commands:
$${kameleon_data_dir}/qemu-sendkeys/netinst-iso-$${distrib}
-
-
-bootstrap:
- - "@base"
-
-setup:
- - "@base"
-
-export:
- - "@base"
diff --git a/grid5000/steps/bootstrap/debian/prepare_autoinstall.yaml
b/grid5000/steps/bootstrap/debian/prepare_autoinstall.yaml
deleted file mode 100644
index f737d20..0000000
--- a/grid5000/steps/bootstrap/debian/prepare_autoinstall.yaml
+++ /dev/null
@@ -1,11 +0,0 @@
-- copy_autoinstall_script_to_http_directory:
- - exec_local: mkdir -p $${http_directory}
- - exec_local: cp $${base_preseed_path} $${http_directory}/preseed.cfg
-
-- customize_preseed:
- - exec_local: sed -i -e 's|\(d-i passwd/root-password password
\).*|\1$${root_password}|g' $${http_directory}/preseed.cfg
- - exec_local: sed -i -e 's|\(d-i passwd/root-password-again password
\).*|\1$${root_password}|g' $${http_directory}/preseed.cfg
- - exec_local: sed -i -e 's|\(d-i mirror/http/hostname string
\).*|\1$${deb_mirror_hostname}|g' $${http_directory}/preseed.cfg
- - exec_local: sed -i -e 's|\(d-i mirror/http/directory string
\).*|\1$${deb_mirror_directory}|g' $${http_directory}/preseed.cfg
- - exec_local: sed -i -e 's|\(d-i apt-setup/security_host string
\).*|\1$${deb_security_hostname}|g' $${http_directory}/preseed.cfg
- - exec_local: sed -i -e 's|\(d-i apt-setup/security_path string
\).*|\1$${deb_security_directory}|g' $${http_directory}/preseed.cfg
diff --git a/grid5000/steps/bootstrap/prepare_appliance.yaml
b/grid5000/steps/bootstrap/prepare_appliance.yaml
deleted file mode 100644
index 4f597c4..0000000
--- a/grid5000/steps/bootstrap/prepare_appliance.yaml
+++ /dev/null
@@ -1,33 +0,0 @@
-- insecure_ssh_key: $${kameleon_cwd}/insecure_ssh_key
-
-- generate_ssh_keys:
- - check_cmd_local: ssh-keygen
- - exec_local: echo -e 'y\n' | ssh-keygen -q -t rsa -b 4096 -f
$${insecure_ssh_key} -N ''
- - exec_local: cat $${insecure_ssh_key}
-
-- inject_ssh_private_key:
- - check_cmd_local: virt-customize
- - exec_local: |
- virt-customize \
- -a $${image_disk}.$${image_format} \
- --run-command 'mkdir -p /root/.ssh' \
- --upload $${insecure_ssh_key}.pub:/root/.ssh/.kameleon_authorized_keys
\
- --run-command 'touch /root/.ssh/authorized_keys' \
- --run-command 'cp /root/.ssh/authorized_keys
/root/.ssh/authorized_keys.bak' \
- --run-command 'cat /root/.ssh/.kameleon_authorized_keys >>
/root/.ssh/authorized_keys' \
- --run-command 'chmod 700 /root/.ssh' \
- --run-command 'chmod -R go-rw /root/.ssh' \
- --run-command 'chown -R root:root /root/.ssh'
- - on_export_init:
- - exec_local: |
- virt-customize \
- -a $${image_disk}.$${image_format} \
- --run-command 'mv /root/.ssh/authorized_keys.bak
/root/.ssh/authorized_keys' \
- --delete /root/.ssh/.kameleon_authorized_keys
-
-- add_insecure_key_to_ssh_config:
- - on_checkpoint: redo
- - exec_local: |
- cat <<EOF >> $${ssh_config_file}
- IdentityFile $${insecure_ssh_key}
- EOF
diff --git a/grid5000/steps/bootstrap/prepare_disk.yaml
b/grid5000/steps/bootstrap/prepare_disk.yaml
deleted file mode 100644
index 9c3dce4..0000000
--- a/grid5000/steps/bootstrap/prepare_disk.yaml
+++ /dev/null
@@ -1,10 +0,0 @@
-- create_initial_image:
- - check_cmd_local: qemu-img
- - exec_local: |
- rm -f $${image_disk}.$${image_format}
- qemu-img create -f qcow2 $${image_disk}.$${image_format}
$${qemu_image_size}
-
-- delete_initial_image:
- - on_checkpoint: skip
- - on_export_clean:
- - exec_local: rm -f $${image_disk}.$${image_format}
diff --git a/grid5000/steps/bootstrap/prepare_ssh_to_out_context.yaml
b/grid5000/steps/bootstrap/prepare_ssh_to_out_context.yaml
deleted file mode 100644
index 172f7a4..0000000
--- a/grid5000/steps/bootstrap/prepare_ssh_to_out_context.yaml
+++ /dev/null
@@ -1,23 +0,0 @@
-- select_empty_port:
- - on_checkpoint: redo
- - exec_local: |
- # Find empty SSH forwarding port
- SSH_FWD_PORT=$(__find_free_port 50000 60000)
- echo "SSH forwarding port: $SSH_FWD_PORT"
-- prepare_ssh_config:
- - on_checkpoint: redo
- - write_local:
- - $${ssh_config_file}
- - |
- Host $${kameleon_recipe_name}
- HostName 127.0.0.1
- Port ${SSH_FWD_PORT}
- User root
- UserKnownHostsFile /dev/null
- StrictHostKeyChecking no
- PasswordAuthentication no
- IdentitiesOnly yes
- LogLevel FATAL
- ForwardAgent yes
- Compression yes
- Protocol 2
diff --git a/grid5000/steps/checkpoints/simple.yaml
b/grid5000/steps/checkpoints/simple.yaml
deleted file mode 100644
index dbd60df..0000000
--- a/grid5000/steps/checkpoints/simple.yaml
+++ /dev/null
@@ -1,21 +0,0 @@
-enabled?:
- - exec_local: test -f $${kameleon_cwd}/checkpoint_enabled
-
-create:
- - exec_local: |
- echo @microstep_id >> $${kameleon_cwd}/checkpoints.list
-
-apply:
- - exec_local: |
- touch $${kameleon_cwd}/checkpoints.list
- grep -R @microstep_id $${kameleon_cwd}/checkpoints.list
-
-
-clear:
- - exec_local: |
- echo > $${kameleon_cwd}/checkpoints.list
-
-list:
- - exec_local: |
- touch $${kameleon_cwd}/checkpoints.list
- cat $${kameleon_cwd}/checkpoints.list | uniq
diff --git a/grid5000/steps/data/helpers/simple_http_server.py
b/grid5000/steps/data/helpers/simple_http_server.py
deleted file mode 100644
index 881343a..0000000
--- a/grid5000/steps/data/helpers/simple_http_server.py
+++ /dev/null
@@ -1,129 +0,0 @@
-#!/usr/bin/env python2
-"""Simple HTTP server"""
-from __future__ import unicode_literals
-import atexit
-import os
-import sys
-import argparse
-
-
-class HTTPServerDaemon(object):
-
- """A HTTP server daemon class."""
-
- def __init__(self, root=os.getcwd()):
- """ Initialize the object."""
- self.root = root
-
- def daemonize(self, pidfile):
- """Deamonize class. UNIX double fork mechanism."""
- try:
- pid = os.fork()
- if pid > 0:
- # exit first parent
- sys.exit(0)
- except OSError as err:
- sys.stderr.write('fork #1 failed: {0}\n'.format(err))
- sys.exit(1)
-
- # decouple from parent environment
- os.chdir(self.root)
- os.setsid()
- os.umask(0)
-
- # do second fork
- try:
- pid = os.fork()
- if pid > 0:
-
- # exit from second parent
- sys.exit(0)
- except OSError as err:
- sys.stderr.write('fork #2 failed: {0}\n'.format(err))
- sys.exit(1)
-
- # redirect standard file descriptors
- sys.stdout.flush()
- sys.stderr.flush()
- si = open(os.devnull, 'r')
- so = open(os.devnull, 'a+')
- se = open(os.devnull, 'a+')
-
- os.dup2(si.fileno(), sys.stdin.fileno())
- os.dup2(so.fileno(), sys.stdout.fileno())
- os.dup2(se.fileno(), sys.stderr.fileno())
-
- # Make sure pid file is removed if we quit
- @atexit.register
- def delpid(self):
- os.remove(pidfile)
-
- # write pidfile
- pid = str(os.getpid())
- with open(pidfile, 'w+') as f:
- f.write(pid + '\n')
-
- def start(self, pidfile, *args, **kwargs):
- """Start the daemon."""
- # Check for a pidfile to see if the daemon already runs
- try:
- with open(pidfile, 'r') as pf:
-
- pid = int(pf.read().strip())
- except IOError:
- pid = None
-
- if pid:
- message = "pidfile {0} already exist. " + \
- "Daemon already running?\n"
- sys.stderr.write(message.format(pidfile))
- sys.exit(1)
-
- # Start the daemon
- self.daemonize(pidfile)
- self.run(*args, **kwargs)
-
- def run(self, host, port):
- """ Run an HTTP server."""
- if sys.version_info[0] == 3:
- from http.server import HTTPServer, SimpleHTTPRequestHandler
- httpd = HTTPServer((host, port), SimpleHTTPRequestHandler)
- else:
- import SimpleHTTPServer
- import SocketServer
- handler = SimpleHTTPServer.SimpleHTTPRequestHandler
- httpd = SocketServer.TCPServer((host, port), handler)
-
- print("Running on http://%s:%s/"; % (host, port))
- os.chdir(self.root)
- try:
- httpd.serve_forever()
- except KeyboardInterrupt:
- sys.stderr.write(u"\nBye\n")
-
-
-if __name__ == '__main__':
- parser = argparse.ArgumentParser(
- description=sys.modules[__name__].__doc__,
- formatter_class=argparse.ArgumentDefaultsHelpFormatter
- )
- parser.add_argument('--port', action="store", default=9090, type=int,
- help='Set the listening port')
- parser.add_argument('--root', action="store", default=os.getcwd())
- parser.add_argument('--bind', action="store", default="0.0.0.0",
- help='Set the binding address')
- parser.add_argument('--daemon', action="store_true", default=False)
- parser.add_argument('--pid', action="store")
-
- try:
- args = parser.parse_args()
- http_server = HTTPServerDaemon(root=args.root)
- if args.daemon:
- if args.pid is None:
- parser.error("Need to set a pid file")
- http_server.start(args.pid, args.bind, args.port)
- else:
- http_server.run(args.bind, args.port)
- except Exception as exc:
- sys.stderr.write(u"\nError: %s\n" % exc)
- sys.exit(1)
diff --git a/grid5000/steps/data/preseed/debian-buster-preseed.cfg
b/grid5000/steps/data/preseed/debian-buster-preseed.cfg
deleted file mode 100644
index 434efb0..0000000
--- a/grid5000/steps/data/preseed/debian-buster-preseed.cfg
+++ /dev/null
@@ -1,443 +0,0 @@
-#### Contents of the preconfiguration file (for buster)
-### Localization
-# Preseeding only locale sets language, country and locale.
-d-i debian-installer/locale string en_US.UTF-8
-
-# The values can also be preseeded individually for greater flexibility.
-#d-i debian-installer/language string en
-#d-i debian-installer/country string NL
-#d-i debian-installer/locale string en_GB.UTF-8
-# Optionally specify additional locales to be generated.
-#d-i localechooser/supported-locales multiselect en_US.UTF-8, nl_NL.UTF-8
-
-# Keyboard selection.
-d-i keyboard-configuration/xkb-keymap select us
-# d-i keyboard-configuration/toggle select No toggling
-
-### Network configuration
-# Disable network configuration entirely. This is useful for cdrom
-# installations on non-networked devices where the network questions,
-# warning and long timeouts are a nuisance.
-#d-i netcfg/enable boolean false
-
-# netcfg will choose an interface that has link if possible. This makes it
-# skip displaying a list if there is more than one interface.
-d-i netcfg/choose_interface select auto
-
-# To pick a particular interface instead:
-#d-i netcfg/choose_interface select eth1
-
-# To set a different link detection timeout (default is 3 seconds).
-# Values are interpreted as seconds.
-#d-i netcfg/link_wait_timeout string 10
-
-# If you have a slow dhcp server and the installer times out waiting for
-# it, this might be useful.
-#d-i netcfg/dhcp_timeout string 60
-#d-i netcfg/dhcpv6_timeout string 60
-
-# If you prefer to configure the network manually, uncomment this line and
-# the static network configuration below.
-#d-i netcfg/disable_autoconfig boolean true
-
-# If you want the preconfiguration file to work on systems both with and
-# without a dhcp server, uncomment these lines and the static network
-# configuration below.
-#d-i netcfg/dhcp_failed note
-#d-i netcfg/dhcp_options select Configure network manually
-
-# Static network configuration.
-#
-# IPv4 example
-#d-i netcfg/get_ipaddress string 192.168.1.42
-#d-i netcfg/get_netmask string 255.255.255.0
-#d-i netcfg/get_gateway string 192.168.1.1
-#d-i netcfg/get_nameservers string 192.168.1.1
-#d-i netcfg/confirm_static boolean true
-#
-# IPv6 example
-#d-i netcfg/get_ipaddress string fc00::2
-#d-i netcfg/get_netmask string ffff:ffff:ffff:ffff::
-#d-i netcfg/get_gateway string fc00::1
-#d-i netcfg/get_nameservers string fc00::1
-#d-i netcfg/confirm_static boolean true
-
-# Any hostname and domain names assigned from dhcp take precedence over
-# values set here. However, setting the values still prevents the questions
-# from being shown, even if values come from dhcp.
-d-i netcfg/get_hostname string kameleon
-d-i netcfg/get_domain string kameleon
-
-# If you want to force a hostname, regardless of what either the DHCP
-# server returns or what the reverse DNS entry for the IP is, uncomment
-# and adjust the following line.
-#d-i netcfg/hostname string somehost
-
-# Disable that annoying WEP key dialog.
-d-i netcfg/wireless_wep string
-# The wacky dhcp hostname that some ISPs use as a password of sorts.
-#d-i netcfg/dhcp_hostname string radish
-
-# If non-free firmware is needed for the network or other hardware, you can
-# configure the installer to always try to load it, without prompting. Or
-# change to false to disable asking.
-#d-i hw-detect/load_firmware boolean true
-
-### Network console
-# Use the following settings if you wish to make use of the network-console
-# component for remote installation over SSH. This only makes sense if you
-# intend to perform the remainder of the installation manually.
-#d-i anna/choose_modules string network-console
-#d-i network-console/authorized_keys_url string http://10.0.0.1/openssh-key
-#d-i network-console/password password r00tme
-#d-i network-console/password-again password r00tme
-
-### Mirror settings
-# If you select ftp, the mirror/country string does not need to be set.
-#d-i mirror/protocol string ftp
-d-i mirror/country string manual
-d-i mirror/http/hostname string http.fr.debian.org
-d-i mirror/http/directory string /debian
-d-i mirror/http/proxy string
-
-# Suite to install.
-#d-i mirror/suite string testing
-# Suite to use for loading installer components (optional).
-#d-i mirror/udeb/suite string testing
-
-### Account setup
-# Skip creation of a root account (normal user account will be able to
-# use sudo).
-#d-i passwd/root-login boolean false
-# Alternatively, to skip creation of a normal user account.
-#d-i passwd/make-user boolean false
-# Enable login to root account
-d-i passwd/root-login boolean true
-
-# Root password, either in clear text
-d-i passwd/root-password password kameleon
-d-i passwd/root-password-again password kameleon
-# or encrypted using a crypt(3) hash.
-#d-i passwd/root-password-crypted password [crypt(3) hash]
-
-# To create a normal user account.
-d-i passwd/user-fullname string Kameleon User
-d-i passwd/username string kameleon
-# Normal user's password, either in clear text
-d-i passwd/user-password password kameleon
-d-i passwd/user-password-again password kameleon
-# or encrypted using a crypt(3) hash.
-#d-i passwd/user-password-crypted password [crypt(3) hash]
-# Create the first user with the specified UID instead of the default.
-#d-i passwd/user-uid string 1010
-
-# The user account will be added to some standard initial groups. To
-# override that, use this.
-#d-i passwd/user-default-groups string audio cdrom video
-d-i passwd/user-default-groups string audio cdrom video admin
-
-### Clock and time zone setup
-# Controls whether or not the hardware clock is set to UTC.
-d-i clock-setup/utc boolean true
-
-# You may set this to any valid setting for $TZ; see the contents of
-# /usr/share/zoneinfo/ for valid values.
-d-i time/zone string UTC
-
-# Controls whether to use NTP to set the clock during the install
-d-i clock-setup/ntp boolean true
-# NTP server to use. The default is almost always fine here.
-#d-i clock-setup/ntp-server string ntp.example.com
-
-### Partitioning
-## Partitioning example
-# If the system has free space you can choose to only partition that space.
-# This is only honoured if partman-auto/method (below) is not set.
-#d-i partman-auto/init_automatically_partition select biggest_free
-
-# Alternatively, you may specify a disk to partition. If the system has only
-# one disk the installer will default to using that, but otherwise the device
-# name must be given in traditional, non-devfs format (so e.g. /dev/sda
-# and not e.g. /dev/discs/disc0/disc).
-# For example, to use the first SCSI/SATA hard disk:
-#d-i partman-auto/disk string /dev/sda
-# In addition, you'll need to specify the method to use.
-# The presently available methods are:
-# - regular: use the usual partition types for your architecture
-# - lvm: use LVM to partition the disk
-# - crypto: use LVM within an encrypted partition
-d-i partman-auto/method string regular
-
-# If one of the disks that are going to be automatically partitioned
-# contains an old LVM configuration, the user will normally receive a
-# warning. This can be preseeded away...
-d-i partman-lvm/device_remove_lvm boolean true
-# The same applies to pre-existing software RAID array:
-d-i partman-md/device_remove_md boolean true
-# And the same goes for the confirmation to write the lvm partitions.
-d-i partman-lvm/confirm boolean true
-d-i partman-lvm/confirm_nooverwrite boolean true
-
-# You can choose one of the three predefined partitioning recipes:
-# - atomic: all files in one partition
-# - home: separate /home partition
-# - multi: separate /home, /var, and /tmp partitions
-d-i partman-auto/choose_recipe select atomic
-
-# Or provide a recipe of your own...
-# If you have a way to get a recipe file into the d-i environment, you can
-# just point at it.
-#d-i partman-auto/expert_recipe_file string /hd-media/recipe
-
-# If not, you can put an entire recipe into the preconfiguration file in one
-# (logical) line. This example creates a small /boot partition, suitable
-# swap, and uses the rest of the space for the root partition:
-#d-i partman-auto/expert_recipe string \
-# boot-root :: \
-# 40 50 100 ext3 \
-# $primary{ } $bootable{ } \
-# method{ format } format{ } \
-# use_filesystem{ } filesystem{ ext3 } \
-# mountpoint{ /boot } \
-# . \
-# 500 10000 1000000000 ext3 \
-# method{ format } format{ } \
-# use_filesystem{ } filesystem{ ext3 } \
-# mountpoint{ / } \
-# . \
-# 64 512 300% linux-swap \
-# method{ swap } format{ } \
-# .
-
-# The full recipe format is documented in the file partman-auto-recipe.txt
-# included in the 'debian-installer' package or available from D-I source
-# repository. This also documents how to specify settings such as file
-# system labels, volume group names and which physical devices to include
-# in a volume group.
-
-# This makes partman automatically partition without confirmation, provided
-# that you told it what to do using one of the methods above.
-d-i partman-partitioning/confirm_write_new_label boolean true
-d-i partman/choose_partition select finish
-d-i partman/confirm boolean true
-d-i partman/confirm_nooverwrite boolean true
-
-# When disk encryption is enabled, skip wiping the partitions beforehand.
-#d-i partman-auto-crypto/erase_disks boolean false
-
-## Partitioning using RAID
-# The method should be set to "raid".
-#d-i partman-auto/method string raid
-# Specify the disks to be partitioned. They will all get the same layout,
-# so this will only work if the disks are the same size.
-#d-i partman-auto/disk string /dev/sda /dev/sdb
-
-# Next you need to specify the physical partitions that will be used.
-#d-i partman-auto/expert_recipe string \
-# multiraid :: \
-# 1000 5000 4000 raid \
-# $primary{ } method{ raid } \
-# . \
-# 64 512 300% raid \
-# method{ raid } \
-# . \
-# 500 10000 1000000000 raid \
-# method{ raid } \
-# .
-
-# Last you need to specify how the previously defined partitions will be
-# used in the RAID setup. Remember to use the correct partition numbers
-# for logical partitions. RAID levels 0, 1, 5, 6 and 10 are supported;
-# devices are separated using "#".
-# Parameters are:
-# <raidtype> <devcount> <sparecount> <fstype> <mountpoint> \
-# <devices> <sparedevices>
-
-#d-i partman-auto-raid/recipe string \
-# 1 2 0 ext3 / \
-# /dev/sda1#/dev/sdb1 \
-# . \
-# 1 2 0 swap - \
-# /dev/sda5#/dev/sdb5 \
-# . \
-# 0 2 0 ext3 /home \
-# /dev/sda6#/dev/sdb6 \
-# .
-
-# For additional information see the file partman-auto-raid-recipe.txt
-# included in the 'debian-installer' package or available from D-I source
-# repository.
-
-# This makes partman automatically partition without confirmation.
-d-i partman-md/confirm boolean true
-d-i partman-partitioning/confirm_write_new_label boolean true
-d-i partman/choose_partition select finish
-d-i partman/confirm boolean true
-d-i partman/confirm_nooverwrite boolean true
-
-## Controlling how partitions are mounted
-# The default is to mount by UUID, but you can also choose "traditional" to
-# use traditional device names, or "label" to try filesystem labels before
-# falling back to UUIDs.
-#d-i partman/mount_style select uuid
-
-### Base system installation
-# Configure APT to not install recommended packages by default. Use of this
-# option can result in an incomplete system and should only be used by very
-# experienced users.
-#d-i base-installer/install-recommends boolean false
-
-# The kernel image (meta) package to be installed; "none" can be used if no
-# kernel is to be installed.
-#d-i base-installer/kernel/image string linux-image-686
-
-### Apt setup
-# You can choose to install non-free and contrib software.
-#d-i apt-setup/non-free boolean true
-#d-i apt-setup/contrib boolean true
-# Uncomment this if you don't want to use a network mirror.
-#d-i apt-setup/use_mirror boolean false
-# Select which update services to use; define the mirrors to be used.
-# Values shown below are the normal defaults.
-#d-i apt-setup/services-select multiselect security, updates
-#d-i apt-setup/security_host string security.debian.org
-
-# Additional repositories, local[0-9] available
-#d-i apt-setup/local0/repository string \
-# http://local.server/debian stable main
-#d-i apt-setup/local0/comment string local server
-# Enable deb-src lines
-#d-i apt-setup/local0/source boolean true
-# URL to the public key of the local repository; you must provide a key or
-# apt will complain about the unauthenticated repository and so the
-# sources.list line will be left commented out
-#d-i apt-setup/local0/key string http://local.server/key
-# Scan another CD or DVD?
-d-i apt-setup/cdrom/set-first boolean false
-
-# By default the installer requires that repositories be authenticated
-# using a known gpg key. This setting can be used to disable that
-# authentication. Warning: Insecure, not recommended.
-#d-i debian-installer/allow_unauthenticated boolean true
-
-# Uncomment this to add multiarch configuration for i386
-#d-i apt-setup/multiarch string i386
-
-
-### Package selection
-#tasksel tasksel/first multiselect standard, web-server, kde-desktop
-tasksel tasksel/first multiselect none
-
-# Individual additional packages to install
-#d-i pkgsel/include string openssh-server build-essential
-d-i pkgsel/include string openssh-server sudo rsync haveged
-# Whether to upgrade packages after debootstrap.
-# Allowed values: none, safe-upgrade, full-upgrade
-d-i pkgsel/upgrade select none
-
-# Some versions of the installer can report back on what software you have
-# installed, and what software you use. The default is not to report back,
-# but sending reports helps the project determine what software is most
-# popular and include it on CDs.
-popularity-contest popularity-contest/participate boolean false
-
-### Boot loader installation
-# Grub is the default boot loader (for x86). If you want lilo installed
-# instead, uncomment this:
-#d-i grub-installer/skip boolean true
-# To also skip installing lilo, and install no bootloader, uncomment this
-# too:
-#d-i lilo-installer/skip boolean true
-
-
-# This is fairly safe to set, it makes grub install automatically to the MBR
-# if no other operating system is detected on the machine.
-d-i grub-installer/only_debian boolean true
-
-# This one makes grub-installer install to the MBR if it also finds some other
-# OS, which is less safe as it might not be able to boot that other OS.
-d-i grub-installer/with_other_os boolean true
-
-# Due notably to potential USB sticks, the location of the MBR can not be
-# determined safely in general, so this needs to be specified:
-#d-i grub-installer/bootdev string /dev/sda
-# To install to the first device (assuming it is not a USB stick):
-#d-i grub-installer/bootdev string default
-
-# Alternatively, if you want to install to a location other than the mbr,
-# uncomment and edit these lines:
-#d-i grub-installer/only_debian boolean false
-#d-i grub-installer/with_other_os boolean false
-#d-i grub-installer/bootdev string (hd0,1)
-# To install grub to multiple disks:
-#d-i grub-installer/bootdev string (hd0,1) (hd1,1) (hd2,1)
-
-# Optional password for grub, either in clear text
-#d-i grub-installer/password password r00tme
-#d-i grub-installer/password-again password r00tme
-# or encrypted using an MD5 hash, see grub-md5-crypt(8).
-#d-i grub-installer/password-crypted password [MD5 hash]
-
-# Use the following option to add additional boot parameters for the
-# installed system (if supported by the bootloader installer).
-# Note: options passed to the installer will be added automatically.
-#d-i debian-installer/add-kernel-opts string nousb
-
-# GRUB install devices:
-# Choices: /dev/sda (21474 MB; VMware_Virtual_S), /dev/sda1 (21472 MB;
VMware_Virtual_S)
-grub-pc grub-pc/install_devices multiselect /dev/vda
-# Choices: Enter device manually, /dev/sda
-grub-installer grub-installer/choose_bootdev select /dev/vda
-
-### Finishing up the installation
-# During installations from serial console, the regular virtual consoles
-# (VT1-VT6) are normally disabled in /etc/inittab. Uncomment the next
-# line to prevent this.
-#d-i finish-install/keep-consoles boolean true
-
-# Avoid that last message about the install being complete.
-d-i finish-install/reboot_in_progress note
-
-# This will prevent the installer from ejecting the CD during the reboot,
-# which is useful in some situations.
-d-i cdrom-detect/eject boolean false
-
-# This is how to make the installer shutdown when finished, but not
-# reboot into the installed system.
-#d-i debian-installer/exit/halt boolean true
-# This will power off the machine instead of just halting it.
-d-i debian-installer/exit/poweroff boolean true
-
-### Preseeding other packages
-# Depending on what software you choose to install, or if things go wrong
-# during the installation process, it's possible that other questions may
-# be asked. You can preseed those too, of course. To get a list of every
-# possible question that could be asked during an install, do an
-# installation, and then run these commands:
-# debconf-get-selections --installer > file
-# debconf-get-selections >> file
-
-
-#### Advanced options
-### Running custom commands during the installation
-# d-i preseeding is inherently not secure. Nothing in the installer checks
-# for attempts at buffer overflows or other exploits of the values of a
-# preconfiguration file like this one. Only use preconfiguration files from
-# trusted locations! To drive that home, and because it's generally useful,
-# here's a way to run any shell command you'd like inside the installer,
-# automatically.
-
-# This first command is run as early as possible, just after
-# preseeding is read.
-#d-i preseed/early_command string anna-install some-udeb
-# This command is run immediately before the partitioner starts. It may be
-# useful to apply dynamic partitioner preseeding that depends on the state
-# of the disks (which may not be visible when preseed/early_command runs).
-#d-i partman/early_command \
-# string debconf-set partman-auto/disk "$(list-devices disk | head -n1)"
-# This command is run just before the install finishes, but when there is
-# still a usable /target directory. You can chroot to /target and use it
-# directly, or use the apt-install and in-target commands to easily install
-# packages and run commands in the target system.
-#d-i preseed/late_command string apt-install zsh; in-target chsh -s /bin/zsh
-
diff --git a/grid5000/steps/data/qemu-sendkeys.rb
b/grid5000/steps/data/qemu-sendkeys.rb
deleted file mode 100644
index d1bcb0f..0000000
--- a/grid5000/steps/data/qemu-sendkeys.rb
+++ /dev/null
@@ -1,121 +0,0 @@
-#!/usr/bin/env ruby
-# Translate a string to "sendkey" commands for QEMU.
-# Martin Vidner, MIT License
-
-# https://en.wikibooks.org/wiki/QEMU/Monitor#sendkey_keys
-# sendkey keys
-#
-# You can emulate keyboard events through sendkey command. The syntax is:
sendkey keys. To get a list of keys, type sendkey [tab]. Examples:
-#
-# sendkey a
-# sendkey shift-a
-# sendkey ctrl-u
-# sendkey ctrl-alt-f1
-#
-# As of QEMU 0.12.5 there are:
-# shift shift_r alt alt_r altgr altgr_r
-# ctrl ctrl_r menu esc 1 2
-# 3 4 5 6 7 8
-# 9 0 minus equal backspace tab
-# q w e r t y
-# u i o p ret a
-# s d f g h j
-# k l z x c v
-# b n m comma dot slash
-# asterisk spc caps_lock f1 f2 f3
-# f4 f5 f6 f7 f8 f9
-# f10 num_lock scroll_lock kp_divide kp_multiply kp_subtract
-# kp_add kp_enter kp_decimal sysrq kp_0 kp_1
-# kp_2 kp_3 kp_4 kp_5 kp_6 kp_7
-# kp_8 kp_9 < f11 f12 print
-# home pgup pgdn end left up
-# down right insert delete
-
-require "optparse"
-
-# incomplete! only what I need now.
-KEYS = {
- "%" => "shift-5",
- "/" => "slash",
- ":" => "shift-semicolon",
- "=" => "equal",
- "." => "dot",
- " " => "spc",
- "-" => "minus",
- "_" => "shift-minus",
- "*" => "asterisk",
- "," => "comma",
- "+" => "shift-equal",
- "|" => "shift-backslash",
- "\\" => "backslash",
-}
-
-class Main
- attr_accessor :command
- attr_accessor :delay_s
- attr_accessor :keystring
-
- def initialize
- self.command = nil
- self.delay_s = 0.1
-
- OptionParser.new do |opts|
- opts.banner = "Usage: sendkeys [-c command_to_pipe_to] STRING\n" +
- "Where STRING can be 'ls<enter>ls<gt>/dev/null<enter>'"
-
- opts.on("-c", "--command COMMAND",
- "Pipe sendkeys to this commands, individually") do |v|
- self.command = v
- end
- opts.on("-d", "--delay SECONDS", Float,
- "Delay SECONDS after each key (default: 0.1)") do |v|
- self.delay_s = v
- end
- end.parse!
- self.keystring = ARGV[0]
- end
-
- def sendkey(qemu_key_name)
- if qemu_key_name == "wait"
- sleep 1
- else
- if qemu_key_name =~ /[A-Za-z]/ && qemu_key_name == qemu_key_name.upcase
- key = "shift-#{qemu_key_name.downcase}"
- else
- key = qemu_key_name
- end
- qemu_cmd = "sendkey #{key}"
- if command
- system "echo '#{qemu_cmd}' | #{command}"
- else
- puts qemu_cmd
- $stdout.flush # important when we are piped
- end
- sleep delay_s
- end
- end
-
- PATTERN = /
- \G # where last match ended
- < [^>]+ >
- |
- \G
- .
- /x
- def run
- keystring.scan(PATTERN) do |match|
- if match[0] == "<"
- key_name = match.slice(1..-2)
- sendkey case key_name
- when "lt" then "shift-comma"
- when "gt" then "shift-dot"
- else key_name
- end
- else
- sendkey KEYS.fetch(match, match)
- end
- end
- end
-end
-
-Main.new.run
diff --git a/grid5000/steps/data/qemu-sendkeys/netinst-iso-debian
b/grid5000/steps/data/qemu-sendkeys/netinst-iso-debian
deleted file mode 100644
index 7705a44..0000000
--- a/grid5000/steps/data/qemu-sendkeys/netinst-iso-debian
+++ /dev/null
@@ -1 +0,0 @@
-<esc><wait>auto preseed/url=http://%LOCAL_IP%:%HTTP_PORT%/preseed.cfg<kp_enter>
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/default_oar-node_site
b/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/default_oar-node_site
deleted file mode 100644
index 273cf08..0000000
---
a/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/default_oar-node_site
+++ /dev/null
@@ -1,49 +0,0 @@
-# OARSERVER: machine where we remotely run oarnodesetting
-OARREMOTE="frontend"
-
-# The paths to oarnodecheckquery and oarnodecheckrun (check your installation)
-OARNODECHECKQUERY=/usr/bin/oarnodecheckquery
-OARNODECHECKRUN=/usr/lib/oar/oarnodecheckrun
-# Home directory of user oar
-OARHOME=/var/lib/oar
-
-# retry settings
-MODSLEEP=20
-MINSLEEP=10
-MAXRETRY=180
-# Ungly glitch do use the good oar key.
-SITE=$( hostname | cut -d'.' -f2)
-
-
-start_oar_node() {
- test -n "$OARREMOTE" || exit 0
- local retry=0
- local sleep=0
- local status=1
- until [ $status -eq 0 ]; do
- echo "oar-node: perform sanity checks"
- $OARNODECHECKRUN
- $OARNODECHECKQUERY
- status=$?
- [ $status -eq 0 ] && {
- echo "oar-node: set the ressources of this node to Alive"
- ssh -t -oStrictHostKeyChecking=no -oPasswordAuthentication=no -i
$OARHOME/.ssh/oarnodesetting_ssh_$SITE.key oar@$OARREMOTE -p 6667
- status=$?
- }
- [ $status -ne 0 ] && {
- if [ $((retry+=sleep)) -gt $MAXRETRY ]; then
- echo "oar-node: FAILED"
- return 1
- fi
- ((sleep = $RANDOM % $MODSLEEP + $MINSLEEP))
- echo "oar-node: retrying in $sleep seconds..."
- sleep $sleep
- }
- done
- return 0
-}
-
-stop_oar_node() {
- :
-}
-
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/base/configure_ip_over_infiniband.pp
b/grid5000/steps/data/setup/puppet/modules/env/manifests/base/configure_ip_over_infiniband.pp
deleted file mode 100644
index 50fd606..0000000
---
a/grid5000/steps/data/setup/puppet/modules/env/manifests/base/configure_ip_over_infiniband.pp
+++ /dev/null
@@ -1,69 +0,0 @@
-class env::base::configure_ip_over_infiniband (){
-
- if $::lsbdistcodename == 'stretch' {
-
- $infiniband_packages = ['qlvnictools']
-
- ensure_packages([$infiniband_packages], {'ensure' => 'installed'})
-
- Package[$infiniband_packages]
- ->Service['openibd']
-
- }
-
- # En suivant la doc https://wiki.debian.org/RDMA, vous n'avez pas besoin
d'installer opensm sur les environnements
- # Il risque de rentrer en conflit avec d'autres instances d'OpenSM présent
sur du matériel réseau, ou bien sur des clusters externes à Grid5000 (exemple :
https://intranet.grid5000.fr/bugzilla/show_bug.cgi?id=10747)
- service {
- 'openibd':
- provider => 'systemd',
- enable => true,
- require => [
- File['/etc/systemd/system/openibd.service']
- ];
- }
-
- file {
- '/etc/infiniband':
- ensure => directory,
- owner => root,
- group => root,
- mode => '0644';
- '/etc/infiniband/openib.conf':
- ensure => file,
- owner => root,
- group => root,
- mode => '0644',
- source => 'puppet:///modules/env/base/infiniband/openib.conf',
- require => File['/etc/infiniband'];
- '/etc/init.d/openibd':
- ensure => file,
- owner => root,
- group => root,
- mode => '0755',
- source => 'puppet:///modules/env/base/infiniband/openibd';
- '/etc/systemd/system/openibd.service':
- ensure => file,
- owner => root,
- group => root,
- mode => '0644',
- source => 'puppet:///modules/env/base/infiniband/openibd.service';
- '/lib/udev/rules.d/90-ib.rules':
- ensure => present,
- owner => root,
- group => root,
- mode => '0644',
- source => 'puppet:///modules/env/base/infiniband/90-ib.rules';
- }
-
- # Empeche que ibacm.service soit en status failed (voir #13013)
- if "${::lsbdistcodename}" == "bullseye" {
- file {
- '/etc/systemd/system/ibacm.service.d/':
- ensure => directory;
- '/etc/systemd/system/ibacm.service.d/override.conf':
- ensure => present,
- content =>
"[Service]\nType=exec\nExecStart=\nExecStart=-/usr/sbin/ibacm --systemd",
- require => File['/etc/systemd/system/ibacm.service.d/'];
- }
- }
-}
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_openmpi.pp
b/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_openmpi.pp
deleted file mode 100644
index b9af00e..0000000
---
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_openmpi.pp
+++ /dev/null
@@ -1,58 +0,0 @@
-class env::big::install_openmpi () {
-
- case "${::lsbdistcodename}" {
-
- "stretch" : {
- $openmpi_packages = [ 'libopenmpi-dev', 'openmpi-bin' ]
- $openmpi_deps_packages = [ 'librdmacm1', 'libgfortran3', 'libnuma1',
'blcr-util', 'libibverbs1-dbg', 'libibverbs-dev', 'libpsm2-dev', 'libhfi1-dev',
'libopamgt-dev' ]
- $openmpi_scibian_version = '2.0.2-2sci9+opa10.7u4'
-
- ensure_packages($openmpi_deps_packages, {
- ensure => present,
- require => Class['apt::update']
- })
-
- ensure_packages($openmpi_packages, {
- ensure => $openmpi_scibian_version,
- require => Class['apt::update']
- })
- }
-
- "buster", "bullseye" : {
- $openmpi_packages = [ 'libopenmpi-dev', 'openmpi-bin' ]
- $openmpi_deps_packages = [ 'libnuma1', 'libibverbs-dev' ]
- $openmpi_opa_packages = [ 'libpsm2-dev', 'libopamgt-dev' ]
-
- ensure_packages($openmpi_deps_packages, {
- ensure => present,
- require => Class['apt::update']
- })
-
- if $env::deb_arch == 'amd64' {
- ensure_packages($openmpi_opa_packages, {
- ensure => present,
- require => Class['apt::update']
- })
- }
-
- ensure_packages($openmpi_packages, {
- ensure => present,
- require => Class['apt::update']
- })
- }
- }
-
- if ($::lsbdistcodename == 'buster') {
- # The 'verbs' OFI provider is broken in OpenMPI 3.1.3. We disable it.
- # See https://intranet.grid5000.fr/bugzilla/show_bug.cgi?id=10918
- # and https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=941996
- # https://github.com/open-mpi/ompi/issues/7035
- # OpenMPI 4.x is not affected, so this can be removed after buster.
- # This does not affect OpenMPI when loaded using 'module'
- file_line { 'disable_verbs_ofi_provider':
- path => '/etc/openmpi/openmpi-mca-params.conf',
- line => 'mtl_ofi_provider_exclude = shm,sockets,tcp,udp,rstream,verbs',
- require => Package['openmpi-bin'];
- }
- }
-}
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_smartd.pp
b/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_smartd.pp
deleted file mode 100644
index 3d266d5..0000000
---
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_smartd.pp
+++ /dev/null
@@ -1,26 +0,0 @@
-class env::big::install_smartd {
-
- package {
- 'smartmontools':
- ensure => installed;
- }
-
- file {
- '/etc/systemd/system/smartd.service.d/':
- ensure => directory,
- require => Package['smartmontools'];
- '/etc/systemd/system/smartd.service.d/override.conf':
- ensure => present,
- content => "[Service]\nExecStartPre=mkdir -p /dev/discs",
- require => File['/etc/systemd/system/smartd.service.d/'];
- }
-
- file_line { 'smard.conf':
- ensure => present,
- require => Package['smartmontools'],
- path => '/etc/smartd.conf',
- line => 'DEVICESCAN -d nvme -d scsi -d ata -d sat -n standby -m root -M
exec /usr/share/smartmontools/smartd-runner',
- match => '^DEVICESCAN .*';
- }
-
-}
diff --git a/grid5000/steps/disable_checkpoint.yaml
b/grid5000/steps/disable_checkpoint.yaml
deleted file mode 100644
index cb571da..0000000
--- a/grid5000/steps/disable_checkpoint.yaml
+++ /dev/null
@@ -1,3 +0,0 @@
-- disable_checkpoint:
- - on_checkpoint: redo
- - exec_local: rm -f $${kameleon_cwd}/checkpoint_enabled
diff --git a/grid5000/steps/enable_checkpoint.yaml
b/grid5000/steps/enable_checkpoint.yaml
deleted file mode 100644
index 8ac4751..0000000
--- a/grid5000/steps/enable_checkpoint.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-- enable_checkpoint:
- - on_checkpoint: redo
- - on_bootstrap_init:
- - exec_local: rm -f $${kameleon_cwd}/checkpoint_enabled
- - exec_local: touch $${kameleon_cwd}/checkpoint_enabled
diff --git a/grid5000/steps/env/bashrc b/grid5000/steps/env/bashrc
deleted file mode 100644
index 6306e37..0000000
--- a/grid5000/steps/env/bashrc
+++ /dev/null
@@ -1,23 +0,0 @@
-## aliases
-# If not running interactively, don't do anything
-export USER=${USER:-"root"}
-export HOME=${HOME:-"/root"}
-export PATH=/usr/bin:/usr/sbin:/bin:/sbin:$PATH
-export LC_ALL=${LC_ALL:-"POSIX"}
-
-export DEBIAN_FRONTEND=noninteractive
-
-if [ -t 1 ] ; then
-export TERM=xterm
-# for fast typing
-alias h='history'
-alias g='git status'
-alias l='ls -lah'
-alias ll='ls -lh'
-alias la='ls -Ah'
-
-# for human readable output
-alias ls='ls -h'
-alias df='df -h'
-alias du='du -h'
-fi
diff --git a/grid5000/steps/export/export_vagrant_box.yaml
b/grid5000/steps/export/export_vagrant_box.yaml
deleted file mode 100644
index 6c048ef..0000000
--- a/grid5000/steps/export/export_vagrant_box.yaml
+++ /dev/null
@@ -1,42 +0,0 @@
-- virtualbox_vmid: $${kameleon_recipe_name}_$${kameleon_short_uuid}
-- virtualbox_disk_filename: $${appliance_filename}.$${appliance_formats}
-- virtualbox_os_type: "Debian_64"
-- vagrant_box_filename: $${kameleon_cwd}/$${kameleon_recipe_name}.box
-
-- create_vbox_machine:
- - on_export_clean:
- - exec_local: |
- if VBoxManage list vms | grep -q $${virtualbox_vmid}; then
- echo "Deleting VBox machine $${virtualbox_vmid}"
- VBoxManage unregistervm $${virtualbox_vmid} --delete
- fi
- - exec_local: |
- if [ -e $${virtualbox_disk_filename} ]; then
- echo "Deleting disk file $${virtualbox_disk_filename}"
- rm $${virtualbox_disk_filename}
- fi
- - exec_local: echo "Creating VBox machine $${virtualbox_vmid}"
- - exec_local: VBoxManage createvm --name $${virtualbox_vmid} --register
- - exec_local: VBoxManage modifyvm $${virtualbox_vmid} --ostype
$${virtualbox_os_type}
- - exec_local: VBoxManage modifyvm $${virtualbox_vmid} --boot1 disk
- - exec_local: VBoxManage modifyvm $${virtualbox_vmid} --memory 256
- - exec_local: VBoxManage modifyvm $${virtualbox_vmid} --acpi on
- - exec_local: VBoxManage modifyvm $${virtualbox_vmid} --nictype1 82540EM
- - exec_local: VBoxManage modifyvm $${virtualbox_vmid} --nictype2 82540EM
- - exec_local: VBoxManage modifyvm $${virtualbox_vmid} --nictype3 82540EM
- - exec_local: VBoxManage modifyvm $${virtualbox_vmid} --nictype4 82540EM
- - exec_local: VBoxManage modifyvm $${virtualbox_vmid} --nic1 nat
--cableconnected1 on
- - exec_local: VBoxManage storagectl $${virtualbox_vmid} --name "SATA
Controller" --add sata --controller IntelAHCI --hostiocache on
- - exec_local: |
- VBoxManage storageattach $${virtualbox_vmid} \
- --storagectl "SATA Controller" \
- --port 0 \
- --device 0 \
- --type hdd \
- --medium $${virtualbox_disk_filename}
-
-- save_box:
- - check_cmd_local: vagrant
- - exec_local: echo "Create vagrant box $${vagrant_box_filename}..."
- - exec_local: rm -f $${vagrant_box_filename}
- - exec_local: vagrant package --base $${virtualbox_vmid} --output
$${vagrant_box_filename}
diff --git a/grid5000/steps/export/save_appliance_VM.yaml
b/grid5000/steps/export/save_appliance_VM.yaml
deleted file mode 100644
index b064d02..0000000
--- a/grid5000/steps/export/save_appliance_VM.yaml
+++ /dev/null
@@ -1,23 +0,0 @@
-#
-# Save Appliance from virtual machine
-#
-- export_appliance_script: $${kameleon_data_dir}/helpers/export_appliance.py
-
-# Zero free unallocated blocks from ext2/3 file-systems before export to
-# reduce image size
-- zerofree: true
-
-- save_appliance:
- - check_cmd_local: python2
- - exec_local: |
- if [ "$${zerofree}" = "true" ]; then
- EXPORT_OPTS="--zerofree"
- else
- EXPORT_OPTS=""
- fi
- - exec_local: |
- python2 $${export_appliance_script} $${image_disk}.$${image_format} \
- -o $${appliance_filename} \
- --formats $${appliance_formats} \
- --tar-compression-level $${appliance_tar_compression_level} \
- --tar-excludes $${appliance_tar_excludes} $EXPORT_OPTS
diff --git a/grid5000/steps/setup/create_user.yaml
b/grid5000/steps/setup/create_user.yaml
deleted file mode 100644
index d7c75cf..0000000
--- a/grid5000/steps/setup/create_user.yaml
+++ /dev/null
@@ -1,11 +0,0 @@
-# Create User
-
-- shell: /bin/bash
-
-- add_user:
- - exec_in: useradd -m $${name} -s $${shell}
- - exec_in: echo -n '$${name}:$${password}' | chpasswd
-
-- add_to_groups:
- - exec_in: |
- usermod -G "$(echo $${groups} | tr ' ' ',')" $${name}
diff --git a/grid5000/steps/setup/debian/clean_system.yaml
b/grid5000/steps/setup/debian/clean_system.yaml
deleted file mode 100644
index 399c339..0000000
--- a/grid5000/steps/setup/debian/clean_system.yaml
+++ /dev/null
@@ -1,34 +0,0 @@
-- enable_lighten: false
-
-- clean_user:
- - on_setup_clean:
- - exec_in: |
- if id kameleon > /dev/null 2>&1; then
- echo "Removing the kameleon user"
- userdel -r kameleon 2> >(grep -v "userdel: kameleon mail spool
(/var/mail/kameleon) not found" )
- fi
-
-- clean_apt:
- - on_setup_clean:
- - apt-get_in: autoremove
- - apt-get_in: autoclean
- - apt-get_in: purge
- - apt-get_in: clean
- - exec_in: |
- if [ $${enable_lighten} = true ]; then
- rm -rf /var/lib/apt/lists/*
- rm -rf /usr/share/locale/*
- rm -rf /usr/share/man/*
- rm -rf /usr/share/doc/*
- fi
-
-- clean_network:
- - on_setup_clean:
- - exec_in: rm -rf /var/lib/dhcp/*
-
-- clean_udev:
- - on_setup_clean:
- - exec_in: rm -rf /etc/udev/rules.d/70-persistent-net.rules
- - exec_in: rm -rf /dev/.udev/
- - exec_in: touch /etc/udev/rules.d/70-persistent-net.rules
- - exec_in: rm -rf /lib/udev/rules.d/75-persistent-net-generator.rules
\ No newline at end of file
diff --git a/grid5000/steps/setup/debian/clean_unnecessary_packages.yaml
b/grid5000/steps/setup/debian/clean_unnecessary_packages.yaml
deleted file mode 100644
index f9cfa37..0000000
--- a/grid5000/steps/setup/debian/clean_unnecessary_packages.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-- default_packages_no_clean: gnupg linux-image-$${deb_arch} console-setup
rsync locales firmware-bnx2 firmware-bnx2x firmware-qlogic
-- arch_packages_no_clean: grub-pc grub-efi-amd64-bin
-- other_packages_no_clean:
-
-- clean_unnecessary_packages:
- - on_setup_clean:
- - exec_in: apt-get update && apt-get install -y debfoster
- - exec_in: yes | debfoster --quiet --force -o MaxPriority=standard
-oUseRecommends=yes $${default_packages_no_clean} $${arch_packages_no_clean}
$${other_packages_no_clean} || true
- - apt-get_in: clean
diff --git a/grid5000/steps/setup/debian/minimal_install.yaml
b/grid5000/steps/setup/debian/minimal_install.yaml
deleted file mode 100644
index d1cdc69..0000000
--- a/grid5000/steps/setup/debian/minimal_install.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-
-- set_root_password:
- - exec_in: echo -n 'root:$${root_password}' | chpasswd
-
-- upgrade_system:
- - apt-get_in: dist-upgrade
diff --git a/grid5000/steps/setup/debian/setup_vagrant_box.yaml
b/grid5000/steps/setup/debian/setup_vagrant_box.yaml
deleted file mode 100644
index fb1f827..0000000
--- a/grid5000/steps/setup/debian/setup_vagrant_box.yaml
+++ /dev/null
@@ -1,77 +0,0 @@
-- puppet_deb_source: "distib" #or "puppetlabs"
-- puppet_deb_url: "http://apt.puppetlabs.com/puppet-release-$${release}.deb";
-- virtualbox_deb_source: "distrib" #or "backports"
-
-- install_requirements:
- - apt-get_in: install rsync curl linux-headers-amd64
-
-- install_virtualbox:
- - test:
- - exec_in: test "$${virtualbox_deb_source}" = "backports"
- - group:
- - write_in:
- - /etc/apt/sources.list.d/virtualbox.list
- - deb $${deb_mirror_uri} $${release}-backports $${deb_components}
- - apt-get_in: update
- - apt-get_in: install virtualbox-guest-utils
- - exec_in: rm -f /etc/apt/sources.list.d/virtualbox.list
- - apt-get_in: update
- - apt-get_in: install virtualbox-guest-utils
-
-- enable_passwordless_sudo:
- - exec_in: |
- sed -i.bkp -e \
- 's/%sudo\s\+ALL=(ALL\(:ALL\)\?)\s\+ALL/%sudo ALL=NOPASSWD:ALL/g' \
- /etc/sudoers
-
-- install_puppet:
- - test:
- - exec_in: test "$${puppet_deb_source}" = "puppetlabs"
- - group:
- - download_file_in:
- - $${puppet_deb_url}
- - $KAMELEON_WORKDIR/puppet.deb
- - exec_in: dpkg -i $KAMELEON_WORKDIR/puppet.deb
- - apt-get_in: update
- - apt-get_in: install puppet
- - exec_in: rm -f $KAMELEON_WORKDIR/puppet.deb
-
-- copy_insecure_sshkey:
- - exec_in: mkdir -pm 700 /home/$${user_name}/.ssh/
- - download_file_in:
- - "https://raw.github.com/mitchellh/vagrant/master/keys/vagrant";
- - /home/$${user_name}/.ssh/id_rsa
- - download_file_in:
- - "https://raw.github.com/mitchellh/vagrant/master/keys/vagrant.pub";
- - /home/$${user_name}/.ssh/id_rsa.pub
- - exec_in: cp /home/$${user_name}/.ssh/id_rsa.pub
/home/$${user_name}/.ssh/authorized_keys
- - exec_in: chmod 0600 /home/$${user_name}/.ssh/*
-
-- config_ssh:
- - exec_in: echo "UseDNS no" >> /etc/ssh/sshd_config
- - write_in:
- - /home/$${user_name}/.ssh/config
- - |
- Host *
- ForwardX11 no
- StrictHostKeyChecking no
- PasswordAuthentication no
- AddressFamily inet
- - exec_in: chmod 0600 /home/$${user_name}/.ssh/config
- - exec_in: rsync -ah /home/$${user_name}/.ssh/ /root/.ssh
- - exec_in: |
- if [ -e /root/.ssh/.kameleon_authorized_keys ]; then
- cat /root/.ssh/.kameleon_authorized_keys >> /root/.ssh/authorized_keys
- fi
- - exec_in: chown "$${user_name}:$${user_name}" -R /home/$${user_name}
-
-- customize_motd:
- - exec_in: echo 'Welcome to your Vagrant-built virtual machine.' > /etc/motd
-
-- fix_network_interface_for_vbox:
- - exec_in: sed -i -e 's/ens3/enp0s3/g' /etc/network/interfaces
-
-- cleanup:
- - exec_in: |
- echo "Adding a 2 sec delay to the interface up, to make the dhclient
happy"
- echo "pre-up sleep 2" >> /etc/network/interfaces
diff --git a/gridboot.service b/gridboot.service
deleted file mode 100644
index 509a682..0000000
--- a/gridboot.service
+++ /dev/null
@@ -1,13 +0,0 @@
-[Unit]
-Description=Modify image based on grid5k Git on boot
-
-Wants=network-online.target
-After=syslog.target network.target network-online.target
-
-[Service]
-Type=oneshot
-ExecStart=/usr/local/bin/at-boot.sh
-RemainAfterExit=yes
-
-[Install]
-WantedBy=multi-user.target
diff --git a/image/.gitignore b/image/.gitignore
new file mode 100644
index 0000000..378eac2
--- /dev/null
+++ b/image/.gitignore
@@ -0,0 +1 @@
+build
diff --git a/grid5000/debian11-x64-common.yaml
b/image/grid5000/debian11-x64-common.yaml
similarity index 96%
rename from grid5000/debian11-x64-common.yaml
rename to image/grid5000/debian11-x64-common.yaml
index b2af6d3..6b53d71 100644
--- a/grid5000/debian11-x64-common.yaml
+++ b/image/grid5000/debian11-x64-common.yaml
@@ -24,7 +24,7 @@ global:
g5k_tar_compression: "zstd"
g5k_postinst_path: server:///grid5000/postinstalls/g5k-postinstall.tgz
g5k_postinst_compression: "gzip"
- g5k_postinst_script: g5k-postinstall --net debian
+ g5k_postinst_script: g5k-postinstall --net debian --disk-aliases
g5k_kernel_params: ""
deb_backports: true
# grub-efi-amd64-bin has to be installed if we want to support both mbr and
diff --git a/grid5000/from_scratch/debian-bullseye.yaml
b/image/grid5000/debian11-x64-nfs.yaml
similarity index 56%
rename from grid5000/from_scratch/debian-bullseye.yaml
rename to image/grid5000/debian11-x64-nfs.yaml
index a017aab..09a450d 100644
--- a/grid5000/from_scratch/debian-bullseye.yaml
+++ b/image/grid5000/debian11-x64-nfs.yaml
@@ -2,17 +2,17 @@
# vim: softtabstop=2 shiftwidth=2 expandtab fenc=utf-8 cc=81 tw=80
#==============================================================================
#
-# DESCRIPTION: Debian 11 (Bullseye) recipe using the netinstall mechanism
+# DESCRIPTION: Debian bullseye (11) x64 nfs Grid'5000 environment
#
#==============================================================================
---
-extend: debian-base.yaml
-# Global variables use by Kameleon engine and the steps
+extend: ./debian11-x64-common.yaml
+
global:
- # Distribution
- distrib: debian
- release: bullseye
- release_number: 11
+ # Grid'5000 environment variant
+ g5k_variant: nfs
+ # Grid'5000 Kadeploy environment parameters
+ g5k_postinst_script: g5k-postinstall --net debian --fstab nfs
--restrict-user current --disk-aliases
bootstrap:
- "@base"
diff --git a/grid5000/from_scratch/base.yaml
b/image/grid5000/from_scratch/base.yaml
similarity index 100%
rename from grid5000/from_scratch/base.yaml
rename to image/grid5000/from_scratch/base.yaml
diff --git a/default/from_scratch/x86_64/debian-base.yaml
b/image/grid5000/from_scratch/debian-base.yaml
similarity index 100%
rename from default/from_scratch/x86_64/debian-base.yaml
rename to image/grid5000/from_scratch/debian-base.yaml
diff --git a/grid5000/from_scratch/aarch64/debian-bullseye.yaml
b/image/grid5000/from_scratch/debian-bullseye.yaml
similarity index 100%
rename from grid5000/from_scratch/aarch64/debian-bullseye.yaml
rename to image/grid5000/from_scratch/debian-bullseye.yaml
diff --git a/grid5000/steps/aliases/defaults.yaml
b/image/grid5000/steps/aliases/defaults.yaml
similarity index 100%
rename from grid5000/steps/aliases/defaults.yaml
rename to image/grid5000/steps/aliases/defaults.yaml
diff --git a/default/steps/bootstrap/debian/prepare_autoinstall.yaml
b/image/grid5000/steps/bootstrap/debian/prepare_autoinstall.yaml
similarity index 100%
rename from default/steps/bootstrap/debian/prepare_autoinstall.yaml
rename to image/grid5000/steps/bootstrap/debian/prepare_autoinstall.yaml
diff --git a/grid5000/steps/bootstrap/download_installer.yaml
b/image/grid5000/steps/bootstrap/download_installer.yaml
similarity index 90%
rename from grid5000/steps/bootstrap/download_installer.yaml
rename to image/grid5000/steps/bootstrap/download_installer.yaml
index f15f58c..b57fdcc 100644
--- a/grid5000/steps/bootstrap/download_installer.yaml
+++ b/image/grid5000/steps/bootstrap/download_installer.yaml
@@ -6,7 +6,7 @@
- exec_local: test -z "$${installer_iso_url}"
- exec_local: |
echo "Looking for the netinstall iso URL for
$${installer_iso_finder_args}"
- DOWNLOAD_SRC_URL=$(python2 $${installer_iso_finder_helper}
$${installer_iso_finder_args})
+ DOWNLOAD_SRC_URL=$($${installer_iso_finder_helper}
$${installer_iso_finder_args})
- download_file_local:
- $${installer_iso_url}
- $${qemu_iso_path}
diff --git a/default/steps/bootstrap/prepare_appliance.yaml
b/image/grid5000/steps/bootstrap/prepare_appliance.yaml
similarity index 100%
rename from default/steps/bootstrap/prepare_appliance.yaml
rename to image/grid5000/steps/bootstrap/prepare_appliance.yaml
diff --git a/default/steps/bootstrap/prepare_disk.yaml
b/image/grid5000/steps/bootstrap/prepare_disk.yaml
similarity index 100%
rename from default/steps/bootstrap/prepare_disk.yaml
rename to image/grid5000/steps/bootstrap/prepare_disk.yaml
diff --git a/default/steps/bootstrap/prepare_ssh_to_out_context.yaml
b/image/grid5000/steps/bootstrap/prepare_ssh_to_out_context.yaml
similarity index 100%
rename from default/steps/bootstrap/prepare_ssh_to_out_context.yaml
rename to image/grid5000/steps/bootstrap/prepare_ssh_to_out_context.yaml
diff --git a/grid5000/steps/bootstrap/start_http_server.yaml
b/image/grid5000/steps/bootstrap/start_http_server.yaml
similarity index 81%
rename from grid5000/steps/bootstrap/start_http_server.yaml
rename to image/grid5000/steps/bootstrap/start_http_server.yaml
index 59184c3..04af024 100644
--- a/grid5000/steps/bootstrap/start_http_server.yaml
+++ b/image/grid5000/steps/bootstrap/start_http_server.yaml
@@ -5,7 +5,7 @@
HTTP_PORT=$(__find_free_port 8000 8100)
echo "HTTP port: $HTTP_PORT"
export HTTP_PORT
- - exec_local: python2 $${http_script} --root $${http_directory} --bind
0.0.0.0 --port $HTTP_PORT --daemon --pid $${http_pid}
+ - exec_local: $${http_script} --root $${http_directory} --bind 0.0.0.0
--port $HTTP_PORT --daemon --pid $${http_pid}
- on_bootstrap_clean:
- exec_local: |
if [ -f $${http_pid} ]; then
diff --git a/grid5000/steps/bootstrap/start_qemu.yaml
b/image/grid5000/steps/bootstrap/start_qemu.yaml
similarity index 100%
rename from grid5000/steps/bootstrap/start_qemu.yaml
rename to image/grid5000/steps/bootstrap/start_qemu.yaml
diff --git a/default/steps/checkpoints/simple.yaml
b/image/grid5000/steps/checkpoints/simple.yaml
similarity index 100%
rename from default/steps/checkpoints/simple.yaml
rename to image/grid5000/steps/checkpoints/simple.yaml
diff --git a/grid5000/steps/data/helpers/export_appliance.py
b/image/grid5000/steps/data/helpers/export_appliance.py
old mode 100644
new mode 100755
similarity index 99%
rename from grid5000/steps/data/helpers/export_appliance.py
rename to image/grid5000/steps/data/helpers/export_appliance.py
index 450ef47..5207e81
--- a/grid5000/steps/data/helpers/export_appliance.py
+++ b/image/grid5000/steps/data/helpers/export_appliance.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/python3
# -*- coding: utf-8 -*-
"""Convert a disk image to many others formats with guestfish."""
from __future__ import division, unicode_literals
diff --git a/grid5000/steps/data/helpers/netinstall_iso_finder.py
b/image/grid5000/steps/data/helpers/netinstall_iso_finder.py
old mode 100644
new mode 100755
similarity index 98%
rename from grid5000/steps/data/helpers/netinstall_iso_finder.py
rename to image/grid5000/steps/data/helpers/netinstall_iso_finder.py
index aa9a2e6..a3a98ab
--- a/grid5000/steps/data/helpers/netinstall_iso_finder.py
+++ b/image/grid5000/steps/data/helpers/netinstall_iso_finder.py
@@ -1,10 +1,10 @@
-#!/usr/bin/env python2
+#!/usr/bin/python3
# -*- coding: utf-8 -*-
"""Find the latest netinstall iso for a Debian version and system
architecture."""
from html.parser import HTMLParser
-from urllib2 import urlopen
-from urlparse import urljoin
+from urllib.request import urlopen
+from urllib.parse import urljoin
import re
import sys
import argparse
diff --git a/default/steps/data/helpers/simple_http_server.py
b/image/grid5000/steps/data/helpers/simple_http_server.py
old mode 100644
new mode 100755
similarity index 99%
rename from default/steps/data/helpers/simple_http_server.py
rename to image/grid5000/steps/data/helpers/simple_http_server.py
index 881343a..62c6215
--- a/default/steps/data/helpers/simple_http_server.py
+++ b/image/grid5000/steps/data/helpers/simple_http_server.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/python3
"""Simple HTTP server"""
from __future__ import unicode_literals
import atexit
diff --git a/grid5000/steps/data/preseed/debian-bullseye-preseed.cfg
b/image/grid5000/steps/data/preseed/debian-bullseye-preseed.cfg
similarity index 100%
rename from grid5000/steps/data/preseed/debian-bullseye-preseed.cfg
rename to image/grid5000/steps/data/preseed/debian-bullseye-preseed.cfg
diff --git a/default/steps/data/qemu-sendkeys.rb
b/image/grid5000/steps/data/qemu-sendkeys.rb
similarity index 100%
rename from default/steps/data/qemu-sendkeys.rb
rename to image/grid5000/steps/data/qemu-sendkeys.rb
diff --git a/default/steps/data/qemu-sendkeys/netinst-iso-debian
b/image/grid5000/steps/data/qemu-sendkeys/netinst-iso-debian
similarity index 100%
rename from default/steps/data/qemu-sendkeys/netinst-iso-debian
rename to image/grid5000/steps/data/qemu-sendkeys/netinst-iso-debian
diff --git a/grid5000/steps/data/setup/hiera/hiera.yaml
b/image/grid5000/steps/data/setup/hiera/hiera.yaml
similarity index 100%
rename from grid5000/steps/data/setup/hiera/hiera.yaml
rename to image/grid5000/steps/data/setup/hiera/hiera.yaml
diff --git a/grid5000/steps/data/setup/hiera/hieradata/defaults.yaml
b/image/grid5000/steps/data/setup/hiera/hieradata/defaults.yaml
similarity index 100%
rename from grid5000/steps/data/setup/hiera/hieradata/defaults.yaml
rename to image/grid5000/steps/data/setup/hiera/hieradata/defaults.yaml
diff --git a/grid5000/steps/data/setup/puppet/manifests/base.pp
b/image/grid5000/steps/data/setup/puppet/manifests/base.pp
similarity index 100%
rename from grid5000/steps/data/setup/puppet/manifests/base.pp
rename to image/grid5000/steps/data/setup/puppet/manifests/base.pp
diff --git a/grid5000/steps/data/setup/puppet/manifests/big.pp
b/image/grid5000/steps/data/setup/puppet/manifests/big.pp
similarity index 100%
rename from grid5000/steps/data/setup/puppet/manifests/big.pp
rename to image/grid5000/steps/data/setup/puppet/manifests/big.pp
diff --git a/grid5000/steps/data/setup/puppet/manifests/min.pp
b/image/grid5000/steps/data/setup/puppet/manifests/min.pp
similarity index 100%
rename from grid5000/steps/data/setup/puppet/manifests/min.pp
rename to image/grid5000/steps/data/setup/puppet/manifests/min.pp
diff --git a/grid5000/steps/data/setup/puppet/manifests/nfs.pp
b/image/grid5000/steps/data/setup/puppet/manifests/nfs.pp
similarity index 100%
rename from grid5000/steps/data/setup/puppet/manifests/nfs.pp
rename to image/grid5000/steps/data/setup/puppet/manifests/nfs.pp
diff --git a/grid5000/steps/data/setup/puppet/manifests/std.pp
b/image/grid5000/steps/data/setup/puppet/manifests/std.pp
similarity index 100%
rename from grid5000/steps/data/setup/puppet/manifests/std.pp
rename to image/grid5000/steps/data/setup/puppet/manifests/std.pp
diff --git a/grid5000/steps/data/setup/puppet/manifests/xen.pp
b/image/grid5000/steps/data/setup/puppet/manifests/xen.pp
similarity index 100%
rename from grid5000/steps/data/setup/puppet/manifests/xen.pp
rename to image/grid5000/steps/data/setup/puppet/manifests/xen.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/base/cpufreq/cpufrequtils
b/image/grid5000/steps/data/setup/puppet/modules/env/files/base/cpufreq/cpufrequtils
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/base/cpufreq/cpufrequtils
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/base/cpufreq/cpufrequtils
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/base/ganglia/gmond.conf
b/image/grid5000/steps/data/setup/puppet/modules/env/files/base/ganglia/gmond.conf
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/base/ganglia/gmond.conf
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/base/ganglia/gmond.conf
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/base/infiniband/90-ib.rules
b/image/grid5000/steps/data/setup/puppet/modules/env/files/base/infiniband/90-ib.rules
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/base/infiniband/90-ib.rules
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/base/infiniband/90-ib.rules
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/base/infiniband/openib.conf
b/image/grid5000/steps/data/setup/puppet/modules/env/files/base/infiniband/openib.conf
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/base/infiniband/openib.conf
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/base/infiniband/openib.conf
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/base/infiniband/openibd
b/image/grid5000/steps/data/setup/puppet/modules/env/files/base/infiniband/openibd
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/base/infiniband/openibd
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/base/infiniband/openibd
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/base/infiniband/openibd.service
b/image/grid5000/steps/data/setup/puppet/modules/env/files/base/infiniband/openibd.service
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/base/infiniband/openibd.service
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/base/infiniband/openibd.service
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/base/kexec/kexec
b/image/grid5000/steps/data/setup/puppet/modules/env/files/base/kexec/kexec
similarity index 100%
rename from grid5000/steps/data/setup/puppet/modules/env/files/base/kexec/kexec
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/base/kexec/kexec
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/base/mx/ip_over_mx
b/image/grid5000/steps/data/setup/puppet/modules/env/files/base/mx/ip_over_mx
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/base/mx/ip_over_mx
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/base/mx/ip_over_mx
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/base/ndctl/ndctl.preset
b/image/grid5000/steps/data/setup/puppet/modules/env/files/base/ndctl/ndctl.preset
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/base/ndctl/ndctl.preset
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/base/ndctl/ndctl.preset
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/base/sshfs/40-fuse.rules
b/image/grid5000/steps/data/setup/puppet/modules/env/files/base/sshfs/40-fuse.rules
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/base/sshfs/40-fuse.rules
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/base/sshfs/40-fuse.rules
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/base/tuning/limits-grid5000.conf
b/image/grid5000/steps/data/setup/puppet/modules/env/files/base/tuning/limits-grid5000.conf
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/base/tuning/limits-grid5000.conf
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/base/tuning/limits-grid5000.conf
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/base/tuning/sysctl-00-grid5000.conf
b/image/grid5000/steps/data/setup/puppet/modules/env/files/base/tuning/sysctl-00-grid5000.conf
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/base/tuning/sysctl-00-grid5000.conf
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/base/tuning/sysctl-00-grid5000.conf
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/base/userns/sysctl-00-userns.conf
b/image/grid5000/steps/data/setup/puppet/modules/env/files/base/userns/sysctl-00-userns.conf
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/base/userns/sysctl-00-userns.conf
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/base/userns/sysctl-00-userns.conf
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/big/amd_gpu/70-amdgpu.rules
b/image/grid5000/steps/data/setup/puppet/modules/env/files/big/amd_gpu/70-amdgpu.rules
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/big/amd_gpu/70-amdgpu.rules
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/big/amd_gpu/70-amdgpu.rules
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/big/kvm/60-qemu-system.rules
b/image/grid5000/steps/data/setup/puppet/modules/env/files/big/kvm/60-qemu-system.rules
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/big/kvm/60-qemu-system.rules
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/big/kvm/60-qemu-system.rules
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/big/kvm/create_tap
b/image/grid5000/steps/data/setup/puppet/modules/env/files/big/kvm/create_tap
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/big/kvm/create_tap
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/big/kvm/create_tap
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/big/kvm/random_mac
b/image/grid5000/steps/data/setup/puppet/modules/env/files/big/kvm/random_mac
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/big/kvm/random_mac
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/big/kvm/random_mac
diff --git a/grid5000/steps/data/setup/puppet/modules/env/files/big/kvm/sudoers
b/image/grid5000/steps/data/setup/puppet/modules/env/files/big/kvm/sudoers
similarity index 100%
rename from grid5000/steps/data/setup/puppet/modules/env/files/big/kvm/sudoers
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/big/kvm/sudoers
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/big/mic/85-mic.rules
b/image/grid5000/steps/data/setup/puppet/modules/env/files/big/mic/85-mic.rules
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/big/mic/85-mic.rules
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/big/mic/85-mic.rules
diff --git a/grid5000/steps/data/setup/puppet/modules/env/files/big/mic/fstab
b/image/grid5000/steps/data/setup/puppet/modules/env/files/big/mic/fstab
similarity index 100%
rename from grid5000/steps/data/setup/puppet/modules/env/files/big/mic/fstab
rename to image/grid5000/steps/data/setup/puppet/modules/env/files/big/mic/fstab
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/big/mic/mic0.filelist
b/image/grid5000/steps/data/setup/puppet/modules/env/files/big/mic/mic0.filelist
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/big/mic/mic0.filelist
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/big/mic/mic0.filelist
diff --git a/grid5000/steps/data/setup/puppet/modules/env/files/big/mic/mpss
b/image/grid5000/steps/data/setup/puppet/modules/env/files/big/mic/mpss
similarity index 100%
rename from grid5000/steps/data/setup/puppet/modules/env/files/big/mic/mpss
rename to image/grid5000/steps/data/setup/puppet/modules/env/files/big/mic/mpss
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/cuda.conf
b/image/grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/cuda.conf
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/cuda.conf
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/cuda.conf
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/dcgm-exporter.service
b/image/grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/dcgm-exporter.service
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/dcgm-exporter.service
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/dcgm-exporter.service
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/ganglia-monitor.service
b/image/grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/ganglia-monitor.service
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/ganglia-monitor.service
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/ganglia-monitor.service
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/modpython-nvidia.conf
b/image/grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/modpython-nvidia.conf
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/modpython-nvidia.conf
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/modpython-nvidia.conf
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/nvidia-persistenced.service
b/image/grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/nvidia-persistenced.service
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/nvidia-persistenced.service
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/nvidia-persistenced.service
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/nvidia-smi.service
b/image/grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/nvidia-smi.service
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/nvidia-smi.service
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/big/nvidia/nvidia-smi.service
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/min/apt/grid5000-archive-key.asc
b/image/grid5000/steps/data/setup/puppet/modules/env/files/min/apt/grid5000-archive-key.asc
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/min/apt/grid5000-archive-key.asc
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/min/apt/grid5000-archive-key.asc
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/min/cpu_microcode/amd64-microcode
b/image/grid5000/steps/data/setup/puppet/modules/env/files/min/cpu_microcode/amd64-microcode
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/min/cpu_microcode/amd64-microcode
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/min/cpu_microcode/amd64-microcode
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/min/cpu_microcode/intel-microcode
b/image/grid5000/steps/data/setup/puppet/modules/env/files/min/cpu_microcode/intel-microcode
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/min/cpu_microcode/intel-microcode
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/min/cpu_microcode/intel-microcode
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/min/image_versioning/git_tag
b/image/grid5000/steps/data/setup/puppet/modules/env/files/min/image_versioning/git_tag
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/min/image_versioning/git_tag
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/min/image_versioning/git_tag
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/min/image_versioning/postinst
b/image/grid5000/steps/data/setup/puppet/modules/env/files/min/image_versioning/postinst
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/min/image_versioning/postinst
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/min/image_versioning/postinst
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/min/locales/locale
b/image/grid5000/steps/data/setup/puppet/modules/env/files/min/locales/locale
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/min/locales/locale
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/min/locales/locale
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/min/locales/locale.gen
b/image/grid5000/steps/data/setup/puppet/modules/env/files/min/locales/locale.gen
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/min/locales/locale.gen
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/min/locales/locale.gen
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/min/network/g5k-update-host-name
b/image/grid5000/steps/data/setup/puppet/modules/env/files/min/network/g5k-update-host-name
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/min/network/g5k-update-host-name
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/min/network/g5k-update-host-name
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/min/network/hosts
b/image/grid5000/steps/data/setup/puppet/modules/env/files/min/network/hosts
similarity index 100%
rename from grid5000/steps/data/setup/puppet/modules/env/files/min/network/hosts
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/min/network/hosts
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/ca2019.grid5000.fr.cert
b/image/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/ca2019.grid5000.fr.cert
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/ca2019.grid5000.fr.cert
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/ca2019.grid5000.fr.cert
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/common-auth
b/image/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/common-auth
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/common-auth
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/common-auth
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/common-password
b/image/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/common-password
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/common-password
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/common-password
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/ldap.conf
b/image/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/ldap.conf
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/ldap.conf
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/ldap.conf
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/libnss-ldap.conf
b/image/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/libnss-ldap.conf
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/libnss-ldap.conf
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/libnss-ldap.conf
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/nscd.conf
b/image/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/nscd.conf
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/nscd.conf
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/nscd.conf
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/nslcd.conf
b/image/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/nslcd.conf
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/nslcd.conf
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/nslcd.conf
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/nsswitch.conf
b/image/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/nsswitch.conf
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/nsswitch.conf
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/nfs/ldap/nsswitch.conf
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/nfs/openiscsi/55-openiscsi.rules
b/image/grid5000/steps/data/setup/puppet/modules/env/files/nfs/openiscsi/55-openiscsi.rules
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/nfs/openiscsi/55-openiscsi.rules
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/nfs/openiscsi/55-openiscsi.rules
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/nfs/openiscsi/iscsidev.sh
b/image/grid5000/steps/data/setup/puppet/modules/env/files/nfs/openiscsi/iscsidev.sh
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/nfs/openiscsi/iscsidev.sh
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/nfs/openiscsi/iscsidev.sh
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/std/g5k-manager/g5k-disk-manager-backend
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/g5k-manager/g5k-disk-manager-backend
similarity index 81%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/std/g5k-manager/g5k-disk-manager-backend
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/std/g5k-manager/g5k-disk-manager-backend
index 97192ea..16c8e28 100644
---
a/grid5000/steps/data/setup/puppet/modules/env/files/std/g5k-manager/g5k-disk-manager-backend
+++
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/g5k-manager/g5k-disk-manager-backend
@@ -14,6 +14,9 @@ ENABLE_LAST_DELAY = 2
def main
options = parse_cmdline
+ if duplicate_scsi_ids?
+ error(1, 'Multiple disks have the same scsi_id in their by_path: we cannot
identify them correctly to run megacli')
+ end
start
if options[:on_boot]
startup_service(options[:force])
@@ -35,7 +38,7 @@ end
def parse_cmdline
options = {}
OptionParser.new do |opts|
- opts.banner = 'Usage: g5k-disk-manager-backend [--enable 1,2,3] [--disable
4,5]'
+ opts.banner = 'Usage: g5k-disk-manager-backend [--enable
disk1,disk2,disk3] [--disable disk4,disk5]'
opts.on('--on-boot', 'Enable all disks at boot time') do |v|
options[:on_boot] = v
end
@@ -57,7 +60,7 @@ def parse_cmdline
end
# The aim of this function is to activate all disks of the node in a predefined
-# order, so that sdb, sdc, ... devices names are always given to the same
+# order, so that disk1, disk2, ... devices names are always given to the same
# physical disks.
# It must be done just before g5k-checks is launched on the node, to avoid
# g5k-checks errors.
@@ -65,7 +68,7 @@ end
def startup_service(force)
_status, hostname = sh('hostname')
- unless reservable_disks?
+ if RESERVABLE_DISKS.empty?
notice "This cluster doesn't have reservable disks: exit service"
close
end
@@ -92,8 +95,8 @@ def startup_service(force)
close
end
- # Remove the first disk from the list (first disk is the main disk sda)
- physical_disks.shift
+ # Remove the main disk from the list
+ physical_disks.delete_if { |pd| pd[:device_id] ==
MAIN_DISK['by_path'].gsub(/.*\d+:\d+:(\d+):\d+$/,'\1').to_i }
# Disable then enable the disks
disable(physical_disks)
@@ -132,24 +135,20 @@ end
def disks_locations(physical_disks, ids)
return [] if ids.nil?
- ids = ids.split(',').map { |e| e.strip.to_i }
+ ids = ids.split(',').map { |e| e.strip }
+ error(4, "Too much disk ids in argument") if ids.length >=
physical_disks.length
disks = []
ids.each do |id|
- # id == 0 corresponds to the main disk sda
- error(4, "Wrong disk id: #{id}") if id <= 0 || id >= physical_disks.length
- disks.push(physical_disks[id])
+ # id == disk0 corresponds to the main disk
+ error(4, "Wrong disk id: #{id}") if id == 'disk0'
+ # We assume that the device ID used by megacli is this captured digit in
by_path
+ # See:
https://github.com/eLvErDe/hwraid/blob/master/wrapper-scripts/megaclisas-status#L301
+ device_id =
RESERVABLE_DISKS[id]['by_path'].gsub(/.*\d+:\d+:(\d+):\d+$/,'\1').to_i
+ disks.push(physical_disks.select { |x| x[:device_id] == device_id }.first)
end
disks
end
-# Clusters with reservable disks are clusters whose
-# reference-repository storage_devices property contains property
-# reservation: true
-def reservable_disks?
- ref_api = File.read('/etc/grid5000/ref-api.json')
- JSON.parse(ref_api)['storage_devices'].select { |sd| sd['reservation'] }.any?
-end
-
def megacli_compliant?
# Get the number or RAID controllers supported by megacli
# The return code of the command is the number of controllers supported
@@ -276,9 +275,30 @@ def disable(physical_disks)
num_disable_errors
end
+# Clusters with reservable disks are clusters whose
+# reference-repository storage_devices property contains property
+# reservation: true
+# The main disk is the one with id 'disk0'
+def g5k_disks
+ ref_api = File.read('/etc/grid5000/ref-api.json')
+ reservable_disks = JSON.parse(ref_api)['storage_devices'].select { |sd|
sd['reservation'] }
+ .map { |sd| [sd['id'], sd] }.to_h
+ main_disk = JSON.parse(ref_api)['storage_devices'].select { |sd| sd['id'] ==
'disk0' }.first
+ [reservable_disks, main_disk]
+end
+
+# We assume that the by-path of a disk is containing the device ID used by
megacli
+# to enable or disable it. However, if for some reason this ID can be the same
+# for multiple disks, we prefer to fail and investiguate the issue.
+def duplicate_scsi_ids?
+ ids = RESERVABLE_DISKS.map { |_id, rd|
rd['by_path'].gsub(/.*\d+:\d+:(\d+):\d+$/,'\1') }
+ (ids - ids.uniq).any?
+end
+
# Main program
MEGACLI = '/usr/sbin/megacli'.freeze
+RESERVABLE_DISKS, MAIN_DISK = g5k_disks
_status, TMPDIR = sh('mktemp -d /tmp/tmp.g5k-disk-manager-backend.XXXXXX')
main
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/std/g5k-manager/g5k-disk-manager-backend.service
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/g5k-manager/g5k-disk-manager-backend.service
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/std/g5k-manager/g5k-disk-manager-backend.service
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/std/g5k-manager/g5k-disk-manager-backend.service
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/std/g5k-manager/g5k-pmem-manager
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/g5k-manager/g5k-pmem-manager
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/std/g5k-manager/g5k-pmem-manager
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/std/g5k-manager/g5k-pmem-manager
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/std/g5k-manager/g5k-pmem-manager.service
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/g5k-manager/g5k-pmem-manager.service
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/std/g5k-manager/g5k-pmem-manager.service
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/std/g5k-manager/g5k-pmem-manager.service
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/std/g5k-manager/lib/g5k-manager.rb
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/g5k-manager/lib/g5k-manager.rb
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/std/g5k-manager/lib/g5k-manager.rb
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/std/g5k-manager/lib/g5k-manager.rb
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/std/g5k_generator/g5k_generator
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/g5k_generator/g5k_generator
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/std/g5k_generator/g5k_generator
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/std/g5k_generator/g5k_generator
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/std/g5kchecks/g5k-checks.conf
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/g5kchecks/g5k-checks.conf
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/std/g5kchecks/g5k-checks.conf
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/std/g5kchecks/g5k-checks.conf
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/std/lvm/lvm.conf
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/lvm/lvm.conf
similarity index 100%
rename from grid5000/steps/data/setup/puppet/modules/env/files/std/lvm/lvm.conf
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/std/lvm/lvm.conf
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/std/net_access/iptables
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/net_access/iptables
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/std/net_access/iptables
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/std/net_access/iptables
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/std/net_access/iptables.stretch
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/net_access/iptables.stretch
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/std/net_access/iptables.stretch
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/std/net_access/iptables.stretch
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/std/net_access/rsyslog.conf
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/net_access/rsyslog.conf
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/std/net_access/rsyslog.conf
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/std/net_access/rsyslog.conf
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/std/net_access/syslog_iptables.conf
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/net_access/syslog_iptables.conf
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/std/net_access/syslog_iptables.conf
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/std/net_access/syslog_iptables.conf
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/std/nvidia_configure/nvidia-reset-mig
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/nvidia_configure/nvidia-reset-mig
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/std/nvidia_configure/nvidia-reset-mig
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/std/nvidia_configure/nvidia-reset-mig
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/std/nvidia_configure/nvidia-reset-mig.service
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/nvidia_configure/nvidia-reset-mig.service
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/std/nvidia_configure/nvidia-reset-mig.service
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/std/nvidia_configure/nvidia-reset-mig.service
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/batch_job_bashrc
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/batch_job_bashrc
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/std/oar/batch_job_bashrc
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/batch_job_bashrc
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/etc/security/access.conf
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/etc/security/access.conf
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/std/oar/etc/security/access.conf
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/etc/security/access.conf
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/default_oar-node
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/oar-node-service
similarity index 69%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/std/oar/default_oar-node
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/oar-node-service
index b8a6fc0..3ce7360 100644
---
a/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/default_oar-node
+++
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/oar-node-service
@@ -1,3 +1,8 @@
+#!/bin/sh
+
+# This script is called when oar-node service is started.
+# You can change this to use another method to switch the node into the Alive
state
+
# OARSERVER: machine where we remotely run oarnodesetting
OARREMOTE="oar"
@@ -10,8 +15,7 @@ OARHOME=/var/lib/oar
# retry settings
MODSLEEP=20
MINSLEEP=10
-MAXRETRY=180
-
+MAXRETRY=300
start_oar_node() {
test -n "$OARREMOTE" || exit 0
@@ -38,16 +42,39 @@ start_oar_node() {
# not provide $RANDOM
if [ "x$random" = "x" ]; then
random=$(bash -c 'echo $RANDOM')
- fi
+ fi
sleep=$(($random % $MODSLEEP + $MINSLEEP))
echo "oar-node: retrying in $sleep seconds..."
sleep $sleep
}
done
+
return 0
}
+# This function is called when oar-node service is stopped.
+# You can change this to use another method to switch the node into the Absent
state
+
stop_oar_node() {
:
}
+usage() {
+ echo "Usage: $0 start|stop"
+}
+
+if [ $# -eq 0 ]; then
+ echo "Missing operation!"
+ usage
+ exit 1
+fi
+
+if [ $1 = "start" ]; then
+ start_oar_node
+elif [ $1 = "stop" ]; then
+ stop_oar_node
+else
+ echo "Unknown operation!"
+ usage
+ exit 1
+fi
diff --git
a/image/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/oar-node.service.override
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/oar-node.service.override
new file mode 100644
index 0000000..352c851
--- /dev/null
+++
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/oar-node.service.override
@@ -0,0 +1,2 @@
+[Service]
+TimeoutStartSec=300
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/oar_sshclient_config
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/oar_sshclient_config
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/std/oar/oar_sshclient_config
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/oar_sshclient_config
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/var/lib/oar/access.conf
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/var/lib/oar/access.conf
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/std/oar/var/lib/oar/access.conf
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/std/oar/var/lib/oar/access.conf
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/std/sudo-g5k/id_rsa_sudo-g5k
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/sudo-g5k/id_rsa_sudo-g5k
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/std/sudo-g5k/id_rsa_sudo-g5k
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/std/sudo-g5k/id_rsa_sudo-g5k
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/std/sudo-g5k/id_rsa_sudo-g5k.pub
b/image/grid5000/steps/data/setup/puppet/modules/env/files/std/sudo-g5k/id_rsa_sudo-g5k.pub
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/std/sudo-g5k/id_rsa_sudo-g5k.pub
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/std/sudo-g5k/id_rsa_sudo-g5k.pub
diff --git a/grid5000/steps/data/setup/puppet/modules/env/files/version
b/image/grid5000/steps/data/setup/puppet/modules/env/files/version
similarity index 100%
rename from grid5000/steps/data/setup/puppet/modules/env/files/version
rename to image/grid5000/steps/data/setup/puppet/modules/env/files/version
diff --git a/grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/id_rsa
b/image/grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/id_rsa
similarity index 100%
rename from grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/id_rsa
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/id_rsa
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/id_rsa.pub
b/image/grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/id_rsa.pub
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/id_rsa.pub
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/id_rsa.pub
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/random_mac
b/image/grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/random_mac
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/random_mac
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/random_mac
diff --git a/grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/xen-g5k
b/image/grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/xen-g5k
similarity index 100%
rename from grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/xen-g5k
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/xen-g5k
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/xen-g5k.service
b/image/grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/xen-g5k.service
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/xen-g5k.service
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/xen-g5k.service
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/xend-config.sxp
b/image/grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/xend-config.sxp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/xend-config.sxp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/files/xen/xen/xend-config.sxp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/lib/facter/installed_kernelreleases.rb
b/image/grid5000/steps/data/setup/puppet/modules/env/lib/facter/installed_kernelreleases.rb
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/lib/facter/installed_kernelreleases.rb
rename to
image/grid5000/steps/data/setup/puppet/modules/env/lib/facter/installed_kernelreleases.rb
diff --git a/grid5000/steps/data/setup/puppet/modules/env/manifests/base.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/base.pp
similarity index 100%
rename from grid5000/steps/data/setup/puppet/modules/env/manifests/base.pp
rename to image/grid5000/steps/data/setup/puppet/modules/env/manifests/base.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/base/add_ca_grid5000.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/add_ca_grid5000.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/base/add_ca_grid5000.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/add_ca_grid5000.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/base/configure_dhclient.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/configure_dhclient.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/base/configure_dhclient.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/configure_dhclient.pp
diff --git
a/image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/configure_ip_over_infiniband.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/configure_ip_over_infiniband.pp
new file mode 100644
index 0000000..7b34f73
--- /dev/null
+++
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/configure_ip_over_infiniband.pp
@@ -0,0 +1,75 @@
+class env::base::configure_ip_over_infiniband (){
+
+ if $::lsbdistcodename == 'stretch' {
+
+ $infiniband_packages = ['qlvnictools']
+
+ ensure_packages([$infiniband_packages], {'ensure' => 'installed'})
+
+ Package[$infiniband_packages]
+ ->Service['openibd']
+
+ }
+
+ if $::lsbdistcodename != 'bullseye' {
+ # En suivant la doc https://wiki.debian.org/RDMA, vous n'avez pas besoin
d'installer opensm sur les environnements
+ # Il risque de rentrer en conflit avec d'autres instances d'OpenSM présent
sur du matériel réseau, ou bien sur des clusters externes à Grid5000 (exemple :
https://intranet.grid5000.fr/bugzilla/show_bug.cgi?id=10747)
+ service {
+ 'openibd':
+ provider => 'systemd',
+ enable => true,
+ require => [
+ File['/etc/systemd/system/openibd.service']
+ ];
+ }
+
+ file {
+ '/etc/infiniband':
+ ensure => directory,
+ owner => root,
+ group => root,
+ mode => '0644';
+ '/etc/infiniband/openib.conf':
+ ensure => file,
+ owner => root,
+ group => root,
+ mode => '0644',
+ source => 'puppet:///modules/env/base/infiniband/openib.conf',
+ require => File['/etc/infiniband'];
+ '/etc/init.d/openibd':
+ ensure => file,
+ owner => root,
+ group => root,
+ mode => '0755',
+ source => 'puppet:///modules/env/base/infiniband/openibd';
+ '/etc/systemd/system/openibd.service':
+ ensure => file,
+ owner => root,
+ group => root,
+ mode => '0644',
+ source => 'puppet:///modules/env/base/infiniband/openibd.service';
+ '/lib/udev/rules.d/90-ib.rules':
+ ensure => present,
+ owner => root,
+ group => root,
+ mode => '0644',
+ source => 'puppet:///modules/env/base/infiniband/90-ib.rules';
+ }
+ }
+
+ if "${::lsbdistcodename}" == "bullseye" {
+ package {
+ 'rdma-core':
+ ensure => installed;
+ }
+ # Empeche que ibacm.service soit en status failed (voir #13013)
+ file {
+ '/etc/systemd/system/ibacm.service.d/':
+ ensure => directory;
+ '/etc/systemd/system/ibacm.service.d/override.conf':
+ ensure => present,
+ content =>
"[Service]\nType=exec\nExecStart=\nExecStart=-/usr/sbin/ibacm --systemd",
+ require => File['/etc/systemd/system/ibacm.service.d/'];
+ }
+ }
+}
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/base/configure_kexec.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/configure_kexec.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/base/configure_kexec.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/configure_kexec.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/base/configure_omnipath.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/configure_omnipath.pp
similarity index 85%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/base/configure_omnipath.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/configure_omnipath.pp
index 9cf899a..35defd9 100644
---
a/grid5000/steps/data/setup/puppet/modules/env/manifests/base/configure_omnipath.pp
+++
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/configure_omnipath.pp
@@ -5,8 +5,9 @@ class env::base::configure_omnipath(){
$opapackages = ['opa-address-resolution', 'opa-fastfabric',
'libopamgt0', 'libopasadb1',
'opa-basic-tools', 'firmware-misc-nonfree']
+ # Note: G5K libfabric1 packages installed in install_openmpi.pp
$rdmapackages = ['qperf', 'libibverbs1', 'librdmacm1', 'libibmad5',
'libibumad3', 'ibverbs-providers',
- 'rdmacm-utils', 'infiniband-diags', 'libfabric1',
'ibverbs-utils']
+ 'rdmacm-utils', 'infiniband-diags', 'ibverbs-utils']
if $env::deb_arch == 'amd64' {
ensure_packages([$opapackages, $rdmapackages], {
@@ -26,6 +27,13 @@ class env::base::configure_omnipath(){
ensure => present
})
}
+ file {
+ # Fix PSM2, see #13470
+ '/lib/udev/rules.d/60-rdma-persistent-naming.rules':
+ ensure => 'file',
+ content => 'ACTION=="add", SUBSYSTEM=="infiniband",
PROGRAM="rdma_rename %k NAME_KERNEL"',
+ require => Package['rdma-core'];
+ }
}
'buster': {
$opapackages = ['opa-address-resolution', 'opa-fastfabric',
'libopamgt0', 'libopasadb1',
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/base/disable_ndctl_monitor.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/disable_ndctl_monitor.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/base/disable_ndctl_monitor.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/disable_ndctl_monitor.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/base/disable_nvme_multipath.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/disable_nvme_multipath.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/base/disable_nvme_multipath.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/disable_nvme_multipath.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/base/do_not_clean_tmp.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/do_not_clean_tmp.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/base/do_not_clean_tmp.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/do_not_clean_tmp.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/base/enable_cpufreq_with_performance_governor.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/enable_cpufreq_with_performance_governor.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/base/enable_cpufreq_with_performance_governor.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/enable_cpufreq_with_performance_governor.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/base/enable_userns.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/enable_userns.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/base/enable_userns.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/enable_userns.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/base/increase_ssh_maxstartups.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/increase_ssh_maxstartups.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/base/increase_ssh_maxstartups.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/increase_ssh_maxstartups.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/base/install_and_disable_ganglia.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/install_and_disable_ganglia.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/base/install_and_disable_ganglia.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/install_and_disable_ganglia.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/base/tcp_tuning_for_10gbe.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/tcp_tuning_for_10gbe.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/base/tcp_tuning_for_10gbe.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/tcp_tuning_for_10gbe.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/base/unlimited_memlock_for_infiniband.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/unlimited_memlock_for_infiniband.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/base/unlimited_memlock_for_infiniband.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/base/unlimited_memlock_for_infiniband.pp
diff --git a/grid5000/steps/data/setup/puppet/modules/env/manifests/big.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big.pp
similarity index 100%
rename from grid5000/steps/data/setup/puppet/modules/env/manifests/big.pp
rename to image/grid5000/steps/data/setup/puppet/modules/env/manifests/big.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_amd_gpu.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_amd_gpu.pp
similarity index 77%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_amd_gpu.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_amd_gpu.pp
index 3a3b9be..f5dd40f 100644
---
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_amd_gpu.pp
+++
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_amd_gpu.pp
@@ -6,8 +6,8 @@ class env::big::configure_amd_gpu () {
apt::source {
'repo.radeon.com':
comment => 'Repo for AMD ROCM packages',
- location => 'https://repo.radeon.com/rocm/apt/debian/',
- release => 'xenial',
+ location =>
"https://repo.radeon.com/rocm/apt/${::env::common::software_versions::rocm_version}/";,
+ release => 'ubuntu',
repos => 'main',
architecture => 'amd64',
key => {
@@ -28,17 +28,23 @@ class env::big::configure_amd_gpu () {
require => [Apt::Source['repo.radeon.com'],
Exec['apt_update']];
}
+ exec {
+ 'add_rocm_symlink':
+ command => "/bin/ln -s /opt/rocm-*/ /opt/rocm",
+ require => Package['rocm-smi-lib'];
+ }
+
file_line {
'rocm_etc_profile_path':
path => '/etc/profile',
- line => 'export PATH=$PATH:/opt/rocm-4.2.0/bin';
+ line => 'export PATH=$PATH:/opt/rocm/bin';
}
file {
'/usr/local/bin/rocm-smi':
ensure => link,
- target => '/opt/rocm-4.2.0/bin/rocm-smi',
- require => Package['rocm-smi-lib'];
+ target => '/opt/rocm/bin/rocm-smi',
+ require => [Package['rocm-smi-lib'], Exec['add_rocm_symlink']];
'/etc/udev/rules.d/70-amdgpu.rules':
ensure => present,
owner => root,
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_initramfs.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_initramfs.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_initramfs.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_initramfs.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_kvm.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_kvm.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_kvm.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_kvm.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/cuda.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/cuda.pp
similarity index 92%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/cuda.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/cuda.pp
index 9e886bf..9ea131a 100644
---
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/cuda.pp
+++
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/cuda.pp
@@ -95,6 +95,10 @@ class env::big::configure_nvidia_gpu::cuda () {
ensure => installed;
} -> package {
'libhwloc-contrib-plugins':
+ install_options => ['-t', 'bullseye-backports'],
+ ensure => installed;
+ 'hwloc':
+ install_options => ['-t', 'bullseye-backports'],
ensure => installed;
}
}
@@ -108,6 +112,10 @@ class env::big::configure_nvidia_gpu::cuda () {
ensure => installed;
} -> package {
'libhwloc-contrib-plugins':
+ install_options => ['-t', 'bullseye-backports'],
+ ensure => installed;
+ 'hwloc':
+ install_options => ['-t', 'bullseye-backports'],
ensure => installed;
}
}
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/drivers.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/drivers.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/drivers.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/drivers.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/ganglia.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/ganglia.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/ganglia.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/ganglia.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/modules.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/modules.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/modules.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/modules.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/prometheus.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/prometheus.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/prometheus.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/prometheus.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/services.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/services.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/services.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_nvidia_gpu/services.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_postfix.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_postfix.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_postfix.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_postfix.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_sshfs.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_sshfs.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_sshfs.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/configure_sshfs.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_beegfs.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_beegfs.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_beegfs.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_beegfs.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_g5k_jupyterlab.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_g5k_jupyterlab.pp
similarity index 63%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_g5k_jupyterlab.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_g5k_jupyterlab.pp
index 8cc0fd5..cb200a2 100644
---
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_g5k_jupyterlab.pp
+++
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_g5k_jupyterlab.pp
@@ -1,13 +1,14 @@
class env::big::install_g5k_jupyterlab {
case $operatingsystem {
'Debian': {
- if "${::lsbdistcodename}" != "stretch" {
+ if "${::lsbdistcodename}" != 'stretch' {
include env::common::software_versions
env::common::g5kpackages {
'g5k-jupyterlab':
- ensure => $::env::common::software_versions::g5k_jupyterlab;
+ ensure => "${::env::common::software_versions::g5k_jupyterlab}",
+ release => "${::lsbdistcodename}";
}
}
}
diff --git
a/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_openmpi.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_openmpi.pp
new file mode 100644
index 0000000..be2a5bd
--- /dev/null
+++
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_openmpi.pp
@@ -0,0 +1,95 @@
+class env::big::install_openmpi () {
+
+ case "${::lsbdistcodename}" {
+
+ "stretch" : {
+ $openmpi_packages = [ 'libopenmpi-dev', 'openmpi-bin' ]
+ $openmpi_deps_packages = [ 'librdmacm1', 'libgfortran3', 'libnuma1',
'blcr-util', 'libibverbs1-dbg', 'libibverbs-dev', 'libpsm2-dev', 'libhfi1-dev',
'libopamgt-dev' ]
+ $openmpi_scibian_version = '2.0.2-2sci9+opa10.7u4'
+
+ ensure_packages($openmpi_deps_packages, {
+ ensure => present,
+ require => Class['apt::update']
+ })
+
+ ensure_packages($openmpi_packages, {
+ ensure => $openmpi_scibian_version,
+ require => Class['apt::update']
+ })
+ }
+
+ "buster" : {
+ $openmpi_packages = [ 'libopenmpi-dev', 'openmpi-bin' ]
+ $openmpi_deps_packages = [ 'libnuma1', 'libibverbs-dev' ]
+ $openmpi_opa_packages = [ 'libpsm2-dev', 'libopamgt-dev' ]
+
+ ensure_packages($openmpi_deps_packages, {
+ ensure => present,
+ require => Class['apt::update']
+ })
+
+ if $env::deb_arch == 'amd64' {
+ ensure_packages($openmpi_opa_packages, {
+ ensure => present,
+ require => Class['apt::update']
+ })
+ }
+
+ ensure_packages($openmpi_packages, {
+ ensure => present,
+ require => Class['apt::update']
+ })
+
+ # The 'verbs' OFI provider is broken in OpenMPI 3.1.3. We disable it.
+ # See https://intranet.grid5000.fr/bugzilla/show_bug.cgi?id=10918
+ # and https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=941996
+ # https://github.com/open-mpi/ompi/issues/7035
+ # OpenMPI 4.x is not affected, so this can be removed after buster.
+ # This does not affect OpenMPI when loaded using 'module'
+ file_line { 'disable_verbs_ofi_provider':
+ path => '/etc/openmpi/openmpi-mca-params.conf',
+ line => 'mtl_ofi_provider_exclude = shm,sockets,tcp,udp,rstream,verbs',
+ require => Package['openmpi-bin'];
+ }
+
+ }
+
+ "bullseye" : {
+ $openmpi_packages = [ 'libopenmpi-dev', 'openmpi-bin', 'ucx-utils',
'libfabric-bin' ]
+ $openmpi_deps_packages = [ 'libnuma1', 'libibverbs-dev' ]
+ $openmpi_opa_packages = [ 'libpsm2-dev', 'libopamgt-dev' ]
+
+ ensure_packages($openmpi_deps_packages, {
+ ensure => present,
+ require => Class['apt::update']
+ })
+
+ if $env::deb_arch == 'amd64' {
+ ensure_packages($openmpi_opa_packages, {
+ ensure => present,
+ require => Class['apt::update']
+ })
+ }
+
+ ensure_packages($openmpi_packages, {
+ ensure => present,
+ require => Class['apt::update']
+ })
+
+ # libfabric packages : G5K rebuild with efa provider disabled
+ # See Bug #13260
+ env::common::g5kpackages {
+ 'libfabric1':
+ packages => 'libfabric1';
+ }
+
+ # Debian11 disables many providers by default. We restore UCX and Fabric,
+ # while keeping openib disabled to avoid useless warnings
+ file { '/etc/openmpi/openmpi-mca-params.conf':
+ content => "#Managed by Grid'5000 environments
recipes\nbtl_base_warn_component_unused=0\nbtl = ^openib",
+ require => Package['openmpi-bin'];
+ }
+ }
+ }
+
+}
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_prometheus_exporters.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_prometheus_exporters.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_prometheus_exporters.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_prometheus_exporters.pp
diff --git
a/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_smartd.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_smartd.pp
new file mode 100644
index 0000000..f94394f
--- /dev/null
+++
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_smartd.pp
@@ -0,0 +1,28 @@
+class env::big::install_smartd {
+
+ if "$::lsbdistcodename" != "stretch" {
+
+ package {
+ 'smartmontools':
+ ensure => installed;
+ }
+
+ file {
+ '/etc/systemd/system/smartd.service.d/':
+ ensure => directory,
+ require => Package['smartmontools'];
+ '/etc/systemd/system/smartd.service.d/override.conf':
+ ensure => present,
+ content => "[Service]\nExecStartPre=mkdir -p /dev/discs",
+ require => File['/etc/systemd/system/smartd.service.d/'];
+ }
+
+ file_line { 'smard.conf':
+ ensure => present,
+ require => Package['smartmontools'],
+ path => '/etc/smartd.conf',
+ line => 'DEVICESCAN -d nvme -d scsi -d ata -d sat -n standby -m root
-M exec /usr/share/smartmontools/smartd-runner',
+ match => '^DEVICESCAN .*';
+ }
+ }
+}
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_snmp_tools.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_snmp_tools.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_snmp_tools.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/install_snmp_tools.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/big/prepare_kernel_module_build.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/prepare_kernel_module_build.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/big/prepare_kernel_module_build.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/big/prepare_kernel_module_build.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/common/apt_pinning.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/common/apt_pinning.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/common/apt_pinning.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/common/apt_pinning.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/common/g5kpackages.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/common/g5kpackages.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/common/g5kpackages.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/common/g5kpackages.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/common/software_versions.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/common/software_versions.pp
similarity index 74%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/common/software_versions.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/common/software_versions.pp
index 2f968be..0a6ae11 100644
---
a/grid5000/steps/data/setup/puppet/modules/env/manifests/common/software_versions.pp
+++
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/common/software_versions.pp
@@ -2,28 +2,35 @@
class env::common::software_versions {
$g5k_subnets = '1.4.2'
- $g5k_meta_packages = '0.7.45'
+ $g5k_meta_packages = '0.7.46'
$tgz_g5k = '2.0.17'
- $g5k_checks = '0.11.2'
- $sudo_g5k = '1.11'
+ $g5k_checks = '0.11.5'
+ $sudo_g5k = '1.12'
$ganglia_monitor = '3.6.0-7.1'
$libguestfs_backport_arm64 = '1:1.40.2-7~bpog5k10+1'
$libguestfs_backport_ppc64el = '1:1.40.2-7~bpog5k10+1'
$lmod = '6.6-0.3g5k1'
- $g5k_jupyterlab = '0.6'
+ $g5k_jupyterlab = '0.9'
+ $rocm_version = '4.3.1'
- case "$env::deb_arch" {
+ case "${env::deb_arch}" {
'amd64': {
$nvidia_driver_arch = 'x86_64'
case $lsbdistcodename {
- 'stretch', 'buster': {
+ 'stretch': {
$nvidia_driver = '450.119.04'
$nvidia_cuda = '10.1.243_418.87.00_linux'
$datacenter_gpu_manager = '1:1.7.2'
$dcgm_exporter = '2.0.0-rc.11'
}
+ 'buster': {
+ $nvidia_driver = '460.91.03'
+ $nvidia_cuda = '10.1.243_418.87.00_linux'
+ $datacenter_gpu_manager = '1:1.7.2'
+ $dcgm_exporter = '2.0.0-rc.11'
+ }
'bullseye': {
- $nvidia_driver = '460.73.01'
+ $nvidia_driver = '460.91.03'
$nvidia_cuda = '11.2.2_460.32.03_linux'
$datacenter_gpu_manager = '1:2.1.4'
$dcgm_exporter = '2.3.0-1'
@@ -44,7 +51,7 @@ class env::common::software_versions {
}
'bullseye': {
$nvidia_driver = '418.197.02'
- $nvidia_user_driver = '460.73.01'
+ $nvidia_user_driver = '460.91.03'
$nvidia_cuda = '11.2.2_460.32.03_linux_ppc64le'
$datacenter_gpu_manager = '1:2.0.15'
$dcgm_exporter = '2.3.0-1'
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/commonpackages.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/commonpackages.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/commonpackages.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/commonpackages.pp
diff --git a/grid5000/steps/data/setup/puppet/modules/env/manifests/init.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/init.pp
similarity index 100%
rename from grid5000/steps/data/setup/puppet/modules/env/manifests/init.pp
rename to image/grid5000/steps/data/setup/puppet/modules/env/manifests/init.pp
diff --git a/grid5000/steps/data/setup/puppet/modules/env/manifests/min.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/min.pp
similarity index 100%
rename from grid5000/steps/data/setup/puppet/modules/env/manifests/min.pp
rename to image/grid5000/steps/data/setup/puppet/modules/env/manifests/min.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/min/add_image_version_in_etc.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/add_image_version_in_etc.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/min/add_image_version_in_etc.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/add_image_version_in_etc.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/min/configure_kernel_and_blacklist_some_modules.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/configure_kernel_and_blacklist_some_modules.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/min/configure_kernel_and_blacklist_some_modules.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/configure_kernel_and_blacklist_some_modules.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/min/configure_network_and_install_drivers.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/configure_network_and_install_drivers.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/min/configure_network_and_install_drivers.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/configure_network_and_install_drivers.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/min/generate_etc_motd.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/generate_etc_motd.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/min/generate_etc_motd.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/generate_etc_motd.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/min/install_and_configure_locales.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/install_and_configure_locales.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/min/install_and_configure_locales.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/install_and_configure_locales.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/min/install_and_configure_ssh.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/install_and_configure_ssh.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/min/install_and_configure_ssh.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/install_and_configure_ssh.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/min/install_cpu_microcode.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/install_cpu_microcode.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/min/install_cpu_microcode.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/install_cpu_microcode.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/min/install_metapackage.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/install_metapackage.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/min/install_metapackage.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/install_metapackage.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/min/install_tgz_g5k.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/install_tgz_g5k.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/min/install_tgz_g5k.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/install_tgz_g5k.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/min/kernel/initramfs.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/kernel/initramfs.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/min/kernel/initramfs.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/kernel/initramfs.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/min/kernel/modules.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/kernel/modules.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/min/kernel/modules.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/kernel/modules.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/min/kernel/remove_old.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/kernel/remove_old.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/min/kernel/remove_old.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/kernel/remove_old.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/min/kernel/setup_links.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/kernel/setup_links.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/min/kernel/setup_links.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/kernel/setup_links.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/min/set_root_password.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/set_root_password.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/min/set_root_password.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/set_root_password.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/min/set_timezone_to_europe_paris.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/set_timezone_to_europe_paris.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/min/set_timezone_to_europe_paris.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/min/set_timezone_to_europe_paris.pp
diff --git a/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs.pp
similarity index 100%
rename from grid5000/steps/data/setup/puppet/modules/env/manifests/nfs.pp
rename to image/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/configure_iscsi.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/configure_iscsi.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/configure_iscsi.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/configure_iscsi.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/configure_ldap.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/configure_ldap.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/configure_ldap.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/configure_ldap.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/configure_module_path.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/configure_module_path.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/configure_module_path.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/configure_module_path.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/configure_ntp.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/configure_ntp.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/configure_ntp.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/configure_ntp.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/install_nfs_requirements.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/install_nfs_requirements.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/install_nfs_requirements.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/install_nfs_requirements.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/install_osirim_requirements.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/install_osirim_requirements.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/install_osirim_requirements.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/install_osirim_requirements.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/install_storage5k_requirements.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/install_storage5k_requirements.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/install_storage5k_requirements.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/nfs/install_storage5k_requirements.pp
diff --git a/grid5000/steps/data/setup/puppet/modules/env/manifests/std.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std.pp
similarity index 96%
rename from grid5000/steps/data/setup/puppet/modules/env/manifests/std.pp
rename to image/grid5000/steps/data/setup/puppet/modules/env/manifests/std.pp
index 99d3611..1f71a7e 100644
--- a/grid5000/steps/data/setup/puppet/modules/env/manifests/std.pp
+++ b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std.pp
@@ -55,4 +55,6 @@ class env::std ( $variant = "big", $parent_parameters = {} ){
if $env::deb_arch == 'arm64' or $env::deb_arch == 'ppc64el' {
class { 'env::std::install_libguestfs_backport': }
}
+ # kameleon
+ class { 'env::std::install_kameleon': }
}
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/std/add_g5kcode_to_path.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/add_g5kcode_to_path.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/std/add_g5kcode_to_path.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/add_g5kcode_to_path.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/std/configure_g5kdiskmanagerbackend.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/configure_g5kdiskmanagerbackend.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/std/configure_g5kdiskmanagerbackend.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/configure_g5kdiskmanagerbackend.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/std/configure_g5kmanager.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/configure_g5kmanager.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/std/configure_g5kmanager.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/configure_g5kmanager.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/std/configure_g5kpmemmanager.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/configure_g5kpmemmanager.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/std/configure_g5kpmemmanager.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/configure_g5kpmemmanager.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/std/configure_oar_client.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/configure_oar_client.pp
similarity index 91%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/std/configure_oar_client.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/configure_oar_client.pp
index d73964d..67b9957 100644
---
a/grid5000/steps/data/setup/puppet/modules/env/manifests/std/configure_oar_client.pp
+++
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/configure_oar_client.pp
@@ -11,11 +11,11 @@ class env::std::configure_oar_client {
$oar_repos_release = "stretch-backports_beta"
}
'buster' : {
- $oar_version = "2.5.10~g5k8-1";
+ $oar_version = "2.5.10~g5k10-2";
$oar_repos = "g5k"
}
'bullseye' : {
- $oar_version = "2.5.10~g5k8-1";
+ $oar_version = "2.5.10~g5k10-2";
$oar_repos = "g5k"
}
default : {
@@ -168,13 +168,26 @@ class env::std::configure_oar_client {
mode => '0644',
content => $hiera['oar_authorized_keys'],
require => Package[$oar_packages];
- '/etc/default/oar-node':
+ '/etc/oar/oar-node-service':
ensure => present,
owner => root,
group => root,
- mode => '0644',
- source => 'puppet:///modules/env/std/oar/default_oar-node',
+ mode => '0755',
+ source => 'puppet:///modules/env/std/oar/oar-node-service',
require => Package[$oar_packages];
+ '/etc/systemd/system/oar-node.service.d':
+ ensure => directory,
+ owner => root,
+ group => root,
+ mode => '0755',
+ require => Package[$oar_packages];
+ '/etc/systemd/system/oar-node.service.d/override.conf':
+ ensure => present,
+ owner => root,
+ group => root,
+ mode => '0644',
+ source => 'puppet:///modules/env/std/oar/oar-node.service.override',
+ require => [Package[$oar_packages],
File['/etc/systemd/system/oar-node.service.d']];
}
augeas {
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/std/configure_rsyslog_remote.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/configure_rsyslog_remote.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/std/configure_rsyslog_remote.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/configure_rsyslog_remote.pp
diff --git a/grid5000/steps/data/setup/puppet/modules/env/manifests/std/dell.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/dell.pp
similarity index 100%
rename from grid5000/steps/data/setup/puppet/modules/env/manifests/std/dell.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/dell.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/std/dell/params.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/dell/params.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/std/dell/params.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/dell/params.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/std/disable_lvm_pvscan.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/disable_lvm_pvscan.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/std/disable_lvm_pvscan.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/disable_lvm_pvscan.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/std/g5k_generator.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/g5k_generator.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/std/g5k_generator.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/g5k_generator.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_g5kchecks.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_g5kchecks.pp
similarity index 68%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_g5kchecks.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_g5kchecks.pp
index 871045f..000c9bb 100644
---
a/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_g5kchecks.pp
+++
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_g5kchecks.pp
@@ -27,6 +27,18 @@ class env::std::install_g5kchecks {
source => "puppet:///modules/env/std/g5kchecks/g5k-checks.conf",
require => Package["g5k-checks"];
}
+
+ # Use refrepo specific branch for testing
+ if "${::lsbdistcodename}" == "bullseye" {
+ file_line { 'g5k-checks_conf':
+ ensure => present,
+ match => "^(#)?branch:*",
+ path => '/etc/g5k-checks.conf',
+ line => 'branch: debian11_std',
+ replace => true,
+ require => File['/etc/g5k-checks.conf'];
+ }
+ }
}
default: {
err "${operatingsystem} not supported."
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_g5ksubnets.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_g5ksubnets.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_g5ksubnets.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_g5ksubnets.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_hwraid_apt_source.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_hwraid_apt_source.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_hwraid_apt_source.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_hwraid_apt_source.pp
diff --git
a/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_kameleon.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_kameleon.pp
new file mode 100644
index 0000000..6e3be85
--- /dev/null
+++
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_kameleon.pp
@@ -0,0 +1,21 @@
+class env::std::install_kameleon {
+
+ case $operatingsystem {
+ 'Debian': {
+ case "${lsbdistcodename}" {
+ 'buster', 'bullseye': {
+ env::common::g5kpackages {
+ 'kameleon':
+ release => "${lsbdistcodename}";
+ }
+ }
+ default: {
+ err "${lsbdistcodename} not supported."
+ }
+ }
+ }
+ default: {
+ err "${operatingsystem} not supported."
+ }
+ }
+}
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_libguestfs_backport.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_libguestfs_backport.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_libguestfs_backport.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_libguestfs_backport.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_megacli.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_megacli.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_megacli.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_megacli.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_sudog5k.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_sudog5k.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_sudog5k.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/install_sudog5k.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/std/ipmitool.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/ipmitool.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/std/ipmitool.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/ipmitool.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/std/nvidia_reset_mig.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/nvidia_reset_mig.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/std/nvidia_reset_mig.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/std/nvidia_reset_mig.pp
diff --git a/grid5000/steps/data/setup/puppet/modules/env/manifests/xen.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/xen.pp
similarity index 100%
rename from grid5000/steps/data/setup/puppet/modules/env/manifests/xen.pp
rename to image/grid5000/steps/data/setup/puppet/modules/env/manifests/xen.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/xen/configure_xen.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/xen/configure_xen.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/xen/configure_xen.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/xen/configure_xen.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/manifests/xen/install_grub.pp
b/image/grid5000/steps/data/setup/puppet/modules/env/manifests/xen/install_grub.pp
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/manifests/xen/install_grub.pp
rename to
image/grid5000/steps/data/setup/puppet/modules/env/manifests/xen/install_grub.pp
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/templates/base/omnipath/scibian.key.erb
b/image/grid5000/steps/data/setup/puppet/modules/env/templates/base/omnipath/scibian.key.erb
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/templates/base/omnipath/scibian.key.erb
rename to
image/grid5000/steps/data/setup/puppet/modules/env/templates/base/omnipath/scibian.key.erb
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/templates/common/apt_pinning.erb
b/image/grid5000/steps/data/setup/puppet/modules/env/templates/common/apt_pinning.erb
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/templates/common/apt_pinning.erb
rename to
image/grid5000/steps/data/setup/puppet/modules/env/templates/common/apt_pinning.erb
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/templates/min/motd.erb
b/image/grid5000/steps/data/setup/puppet/modules/env/templates/min/motd.erb
similarity index 100%
rename from grid5000/steps/data/setup/puppet/modules/env/templates/min/motd.erb
rename to
image/grid5000/steps/data/setup/puppet/modules/env/templates/min/motd.erb
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/templates/nfs/ldap/common-account.erb
b/image/grid5000/steps/data/setup/puppet/modules/env/templates/nfs/ldap/common-account.erb
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/templates/nfs/ldap/common-account.erb
rename to
image/grid5000/steps/data/setup/puppet/modules/env/templates/nfs/ldap/common-account.erb
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/templates/nfs/ntp/ntp.conf.erb
b/image/grid5000/steps/data/setup/puppet/modules/env/templates/nfs/ntp/ntp.conf.erb
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/templates/nfs/ntp/ntp.conf.erb
rename to
image/grid5000/steps/data/setup/puppet/modules/env/templates/nfs/ntp/ntp.conf.erb
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/templates/std/dell/linux.dell.com.key.erb
b/image/grid5000/steps/data/setup/puppet/modules/env/templates/std/dell/linux.dell.com.key.erb
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/templates/std/dell/linux.dell.com.key.erb
rename to
image/grid5000/steps/data/setup/puppet/modules/env/templates/std/dell/linux.dell.com.key.erb
diff --git
a/grid5000/steps/data/setup/puppet/modules/env/templates/std/hwraid/hwraid.le-vert.net.key.erb
b/image/grid5000/steps/data/setup/puppet/modules/env/templates/std/hwraid/hwraid.le-vert.net.key.erb
similarity index 100%
rename from
grid5000/steps/data/setup/puppet/modules/env/templates/std/hwraid/hwraid.le-vert.net.key.erb
rename to
image/grid5000/steps/data/setup/puppet/modules/env/templates/std/hwraid/hwraid.le-vert.net.key.erb
diff --git a/default/steps/disable_checkpoint.yaml
b/image/grid5000/steps/disable_checkpoint.yaml
similarity index 100%
rename from default/steps/disable_checkpoint.yaml
rename to image/grid5000/steps/disable_checkpoint.yaml
diff --git a/default/steps/enable_checkpoint.yaml
b/image/grid5000/steps/enable_checkpoint.yaml
similarity index 100%
rename from default/steps/enable_checkpoint.yaml
rename to image/grid5000/steps/enable_checkpoint.yaml
diff --git a/default/steps/env/bashrc b/image/grid5000/steps/env/bashrc
similarity index 100%
rename from default/steps/env/bashrc
rename to image/grid5000/steps/env/bashrc
diff --git a/grid5000/steps/env/functions.sh
b/image/grid5000/steps/env/functions.sh
similarity index 100%
rename from grid5000/steps/env/functions.sh
rename to image/grid5000/steps/env/functions.sh
diff --git a/grid5000/steps/export/debian/clean_dhcp_leases.yaml
b/image/grid5000/steps/export/debian/clean_dhcp_leases.yaml
similarity index 100%
rename from grid5000/steps/export/debian/clean_dhcp_leases.yaml
rename to image/grid5000/steps/export/debian/clean_dhcp_leases.yaml
diff --git a/grid5000/steps/export/do_qcow2_finish_works.yaml
b/image/grid5000/steps/export/do_qcow2_finish_works.yaml
similarity index 100%
rename from grid5000/steps/export/do_qcow2_finish_works.yaml
rename to image/grid5000/steps/export/do_qcow2_finish_works.yaml
diff --git a/grid5000/steps/export/export_g5k.yaml
b/image/grid5000/steps/export/export_g5k.yaml
similarity index 93%
rename from grid5000/steps/export/export_g5k.yaml
rename to image/grid5000/steps/export/export_g5k.yaml
index 04b1358..3362908 100644
--- a/grid5000/steps/export/export_g5k.yaml
+++ b/image/grid5000/steps/export/export_g5k.yaml
@@ -10,6 +10,7 @@
- g5k_destructive: "false"
- g5k_tar_compression: "gzip"
- g5k_postinst_compression: "gzip"
+- g5k_description: $${distrib} $${release_number} ($${release}) for
$${g5k_image_arch} - $${g5k_variant}
# - save_as_tgz:
# - check_cmd_local: guestfish
@@ -28,8 +29,9 @@
cat << EOF > $${output}.dsc
$${dashes}
name: $${kameleon_recipe_name}
+ arch: $${arch}
version: $${g5k_version}
- description: $${distrib} $${release_number} ($${release}) for
$${g5k_image_arch} - $${g5k_variant}
+ description: $${g5k_description}
author: $${g5k_author}
visibility: $${g5k_visibility}
destructive: $${g5k_destructive}
@@ -54,8 +56,9 @@
cat << EOF > $${output}.dsc
$${dashes}
name: $${kameleon_recipe_name}
+ arch: $${arch}
version: $${g5k_version}
- description: $${distrib} $${release_number} ($${release}) for
$${g5k_image_arch} - $${g5k_variant}
+ description: $${g5k_description}
author: $${g5k_author}
visibility: $${g5k_visibility}
destructive: $${g5k_destructive}
diff --git a/default/steps/export/save_appliance_VM.yaml
b/image/grid5000/steps/export/save_appliance_VM.yaml
similarity index 85%
rename from default/steps/export/save_appliance_VM.yaml
rename to image/grid5000/steps/export/save_appliance_VM.yaml
index b064d02..e755113 100644
--- a/default/steps/export/save_appliance_VM.yaml
+++ b/image/grid5000/steps/export/save_appliance_VM.yaml
@@ -8,7 +8,6 @@
- zerofree: true
- save_appliance:
- - check_cmd_local: python2
- exec_local: |
if [ "$${zerofree}" = "true" ]; then
EXPORT_OPTS="--zerofree"
@@ -16,7 +15,7 @@
EXPORT_OPTS=""
fi
- exec_local: |
- python2 $${export_appliance_script} $${image_disk}.$${image_format} \
+ $${export_appliance_script} $${image_disk}.$${image_format} \
-o $${appliance_filename} \
--formats $${appliance_formats} \
--tar-compression-level $${appliance_tar_compression_level} \
diff --git a/default/steps/setup/debian/clean_system.yaml
b/image/grid5000/steps/setup/debian/clean_system.yaml
similarity index 100%
rename from default/steps/setup/debian/clean_system.yaml
rename to image/grid5000/steps/setup/debian/clean_system.yaml
diff --git a/grid5000/steps/setup/debian/configure_apt_sources.yaml
b/image/grid5000/steps/setup/debian/configure_apt_sources.yaml
similarity index 100%
rename from grid5000/steps/setup/debian/configure_apt_sources.yaml
rename to image/grid5000/steps/setup/debian/configure_apt_sources.yaml
diff --git a/grid5000/steps/setup/debian/configure_system.yaml
b/image/grid5000/steps/setup/debian/configure_system.yaml
similarity index 100%
rename from grid5000/steps/setup/debian/configure_system.yaml
rename to image/grid5000/steps/setup/debian/configure_system.yaml
diff --git a/grid5000/steps/setup/debian/install_packages.yaml
b/image/grid5000/steps/setup/debian/install_packages.yaml
similarity index 100%
rename from grid5000/steps/setup/debian/install_packages.yaml
rename to image/grid5000/steps/setup/debian/install_packages.yaml
diff --git a/default/steps/setup/debian/minimal_install.yaml
b/image/grid5000/steps/setup/debian/minimal_install.yaml
similarity index 100%
rename from default/steps/setup/debian/minimal_install.yaml
rename to image/grid5000/steps/setup/debian/minimal_install.yaml
diff --git a/grid5000/steps/setup/debian/run_orchestrator.yaml
b/image/grid5000/steps/setup/debian/run_orchestrator.yaml
similarity index 100%
rename from grid5000/steps/setup/debian/run_orchestrator.yaml
rename to image/grid5000/steps/setup/debian/run_orchestrator.yaml
diff --git a/grid5000/steps/setup/debian/setup_orchestrator.yaml
b/image/grid5000/steps/setup/debian/setup_orchestrator.yaml
similarity index 100%
rename from grid5000/steps/setup/debian/setup_orchestrator.yaml
rename to image/grid5000/steps/setup/debian/setup_orchestrator.yaml
diff --git a/image/taler-debian11.yaml b/image/taler-debian11.yaml
new file mode 100644
index 0000000..e7d424c
--- /dev/null
+++ b/image/taler-debian11.yaml
@@ -0,0 +1,181 @@
+#==============================================================================
+# vim: softtabstop=2 shiftwidth=2 expandtab fenc=utf-8 cc=81 tw=80
+#==============================================================================
+#
+# DESCRIPTION: Recipe extending a Grid'5000 environment recipe supported by
+# the technical team. The build will be similar to the one of the supported
+# environments, involving puppet notably.
+#
+#==============================================================================
+# This recipe extends another. To look at the step involed, run:
+# kameleon dryrun taler-debian11.yaml
+# To see the variables that you can override, use the following command:
+# kameleon info taler-debian11.yaml
+---
+extend: grid5000/debian11-x64-nfs.yaml
+
+global:
+ ### Uncomment and adapt the global variables below as needed
+
+ ## Export format to generate
+ # appliance_formats: qcow2 tar.zst
+
+ ## Environment description customization
+ ## Author
+ g5k_author: "bossm8@bfh.ch"
+ ## Version
+ g5k_version: 2
+ ## Environment image path and compression
+ # Note: setup for the docker build image - please replace G5K_HOST and
G5K_USER for manual builds
+ g5k_tar_path:
"http://public.<G5K_HOST>.grid5000.fr/~<G5K_USER>/taler-debian11.tar.zst"
+ # g5k_tar_compression: "zstd"
+ ## Environment postinstall path, compression, and script command
+ # g5k_postinst_path: server:///grid5000/postinstalls/g5k-postinstall.tgz
+ # g5k_postinst_compression: "gzip"
+ # g5k_postinst_script: g5k-postinstall --net debian
+ ## Environment kernel path and params
+ # g5k_kernel_path: "/vmlinuz"
+ # g5k_initrd_path: "/initrd.img"
+ # g5k_kernel_params: ""
+ ## Environment visibility
+ # g5k_visibility: "shared"
+ taler_build_packages: "recutils autoconf uncrustify autopoint libtool
python3-pip libgcrypt20-dev libjansson-dev libcurl4-gnutls-dev libsodium-dev
libidn2-dev libunistring-dev libmicrohttpd-dev libsqlite3-dev libqrencode-dev
valgrind libpq-dev texinfo gdb make npm zip python3-distutils pkg-config"
+ taler_packages: "nginx postgresql-13 postgresql-contrib curl jq bc sudo git
zile dnsutils prometheus-postgres-exporter prometheus-nginx-exporter net-tools
netcat parallel nodejs tshark dnsmasq"
+ taler_packages_no_recommends: "prometheus prometheus-node-exporter"
+ taler_disable_services: "nginx postgresql prometheus
prometheus-postgres-exporter prometheus-nginx-exporter prometheus-node-exporter"
+ taler_loki_version: "v2.4.0"
+ gnunet_tar: gnunet.tar.gz
+ exchange_tar: exchange.tar.gz
+ merchant_tar: merchant.tar.gz
+ wallet_tar: wallet.tar.gz
+ taler_path: /root/taler
+ gnunet_path: $${taler_path}/$${gnunet_tar}
+ exchange_path: $${taler_path}/$${exchange_tar}
+ merchant_path: $${taler_path}/$${merchant_tar}
+ wallet_path: $${taler_path}/$${wallet_tar}
+ ## Other parameters can be changed, see kameleon info debian10-taler.yaml
+
+bootstrap:
+ ### The bootstrap section takes in charge the initial installation of the
+ ## system (distribution installation). No modification should be needed here.
+ - "@base"
+
+setup:
+ ### The setup section is where customizations of the system take place.
+ ## We can request steps from the extended recipe to be executed
+ - "@base"
+ ## We add steps required by our customization after or before @base. Use
+ ## kameleon dryrun debian10_custom.yaml to see the resulting steps in the
build.
+ ## The following is given as example only, replace with your steps.
+
+ - import:
+ - import_gnunet:
+ # kameleon_data_dir=./grid5000/steps/data
+ - local2in:
+ - $${kameleon_data_dir}/$${gnunet_tar}
+ - $${gnunet_path}
+ - import_exchange:
+ - local2in:
+ - $${kameleon_data_dir}/$${exchange_tar}
+ - $${exchange_path}
+ - import_merchant:
+ - local2in:
+ - $${kameleon_data_dir}/$${merchant_tar}
+ - $${merchant_path}
+ - import_wallet:
+ - local2in:
+ - $${kameleon_data_dir}/$${wallet_tar}
+ - $${wallet_path}
+
+ - install:
+ - packages:
+ - exec_in: |
+ echo "deb https://deb.taler.net/apt/debian bullseye main" >
/etc/apt/sources.list.d/taler.list
+ wget -O - https://taler.net/taler-systems.gpg.key | apt-key add -
+ apt update
+ apt upgrade
+ apt install -y $${taler_packages} $${taler_build_packages}
+ apt install -y --no-install-recommends
$${taler_packages_no_recommends}
+
+ - gnunet:
+ - exec_in: |
+ cd $${taler_path}
+
+ tar -xvf $${gnunet_tar}
+ cd gnunet-*
+ CFLAGS="-O0 -g" ./configure --enable-logging=verbose --prefix=/usr #
--enable-sanitizer
+ make install
+ ldconfig
+
+ cd .. && rm -rf gnunet-*
+
+ - exchange:
+ - exec_in: |
+ cd $${taler_path}
+
+ pip install jinja2
+ tar -xvf $${exchange_tar}
+ cd taler-exchange*
+ CFLAGS="-O0 -g" ./configure --enable-logging=verbose --prefix=/usr #
--enable-sanitizer
+ make install
+ ldconfig
+
+ cd .. && rm -rf taler-exchange-*
+
+ - merchant:
+ - exec_in: |
+ cd $${taler_path}
+
+ tar -xvf $${merchant_tar}
+ cd taler-merchant*
+ ./configure --enable-logging=verbose --prefix=/usr #
--enable-sanitizer
+ make install
+ ldconfig
+
+ cd .. && rm -rf taler-merchant-*
+
+ - wallet:
+ - exec_in: |
+ cd $${taler_path}
+
+ tar -xvf $${wallet_tar}
+ cd taler-wallet*
+ npm install -g pnpm
+ ./configure
+ make install
+
+ cd .. && rm -rf taler-wallet-*
+
+ - grid5k:
+ - exec_in: |
+ cd $${taler_path}
+
+ git clone git://git.taler.net/grid5k.git
+
+ - loki-promtail:
+ - exec_in: |
+ apt-get install -y unzip curl
+
+ curl -O -L
"https://github.com/grafana/loki/releases/download/$${taler_loki_version}/loki-linux-amd64.zip";
+ unzip loki-linux-amd64.zip
+ rm loki-linux-amd64.zip
+ mv loki-linux-amd64 /usr/local/bin/loki
+ chmod 770 /usr/local/bin/loki
+
+ curl -O -L
"https://github.com/grafana/loki/releases/download/$${taler_loki_version}/promtail-linux-amd64.zip";
+ unzip promtail-linux-amd64.zip
+ rm promtail-linux-amd64.zip
+ mv promtail-linux-amd64 /usr/local/bin/promtail
+ chmod 770 /usr/local/bin/promtail
+
+ - disable_services:
+ - exec_in: |
+ systemctl daemon-reload
+ systemctl stop $${taler_disable_services}
+ systemctl disable $${taler_disable_services}
+
+export:
+ ### The export section takes in charge the export of your customized
Grid'5000
+ ## environment. No modification should be needed here.
+ - "@base"
+
diff --git a/jfed/README b/jfed/README
deleted file mode 100644
index a9bf655..0000000
--- a/jfed/README
+++ /dev/null
@@ -1,2 +0,0 @@
-twoboot: simply boot two hosts in different regions with our master image
-
diff --git a/jfed/twoboot.rspec b/jfed/twoboot.rspec
deleted file mode 100644
index 3a3c1fb..0000000
--- a/jfed/twoboot.rspec
+++ /dev/null
@@ -1,17 +0,0 @@
-<?xml version='1.0'?>
-<rspec xmlns="http://www.geni.net/resources/rspec/3"; type="request"
generated_by="jFed RSpec Editor" generated="2021-09-05T13:16:08.333+02:00"
xmlns:emulab="http://www.protogeni.net/resources/rspec/ext/emulab/1";
xmlns:delay="http://www.protogeni.net/resources/rspec/ext/delay/1";
xmlns:jfed-command="http://jfed.iminds.be/rspec/ext/jfed-command/1";
xmlns:client="http://www.protogeni.net/resources/rspec/ext/client/1";
xmlns:jfed-ssh-keys="http://jfed.iminds.be/rspec/ext/jfed-ssh-keys/1"; xmlns:
[...]
- <node client_id="node0" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
- <sliver_type name="raw-pc">
- <disk_image
name="http://public.lyon.grid5000.fr/~grothoff/debian11-taler.dsc"/>
- </sliver_type>
- <hardware_type name="nova-lyon"/>
- <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1";
x="354.33333333333337" y="89.66666666666666"/>
- </node>
- <node client_id="node1" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
- <sliver_type name="raw-pc">
- <disk_image
name="http://public.lyon.grid5000.fr/~grothoff/debian11-taler.dsc"/>
- </sliver_type>
- <hardware_type name="uvb-sophia"/>
- <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1";
x="347.33333333333337" y="161.66666666666669"/>
- </node>
-</rspec>
\ No newline at end of file
diff --git a/notes.txt b/notes.txt
index 48966ed..d4200b4 100644
--- a/notes.txt
+++ b/notes.txt
@@ -1,3 +1,22 @@
* option "other_packages_no_clean" in global for yaml image builder
not properly documented, forced us to hunt down why explictily
- installed packages were removed again after the setup step
+ installed packages were removed again after the setup step - no problem
anymore in
+ (found in grid5000/steps/setup/debian/clean_unnecessary_packages.yaml)
+* scripts located in grid5000/steps/data/helpers/ are not executable which
results in
+ Permission denied when running kameleon build
+* jFed: Reserve not working for grid5000 resources
+* jFed: Error do not clearly show which nodes failed allocating
+ its hard to detect problems when using many nodes - should be clearer
+ bsp error: Some requiered resource was not available at this time. (which?!)
+* only show nodes which are available for users - e.g. not neovise
+* jFed: Changing Node or HW type does sometimes not apply and is not saved
+* Online status monitoing not up to date
+* Very common error:
+ <h1>Proxy Error</h1>
+ <p>The proxy server received an invalid
+ response from an upstream server.<br />
+ The proxy server could not handle the request<p>Reason: <strong>Error
reading from remote server</strong></p></p>
+* SSH login from rpesc viewer not possible
+* SSH login often fails - must provide password for cert and user (user
password is not the one I speciefied, neither the default for the root user of
the image) - or connection closed ... - BUG in allocation code according to
email
+* espec - possibility to rerun and override previously uploaded files
+* ansible not working with debian 11 - install script fails - but even when
installed manually it hangs in preparing ansible dir
diff --git a/steps/setup/taler_install.yaml b/steps/setup/taler_install.yaml
deleted file mode 100644
index 2e55b44..0000000
--- a/steps/setup/taler_install.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-- install_taler:
- - exec_in : |
- echo "deb https://deb.taler.net/apt/debian bullseye main" >
/etc/apt/sources.list.d/taler.list
- wget -O - https://taler.net/taler-systems.gpg.key | apt-key add -
- apt-get update
- apt-upgrade
- apt-get install -y nginx postgresql-13 taler-exchange taler-auditor
taler-merchant taler-exchange-offline taler-wallet-cli
- apt-get install -y git zile
diff --git a/steps/setup/taler_install.yaml~ b/steps/setup/taler_install.yaml~
deleted file mode 100644
index 75f58bd..0000000
--- a/steps/setup/taler_install.yaml~
+++ /dev/null
@@ -1,2 +0,0 @@
-- install_ffmpeg:
- - exec_in : apt-get update && apt-get install -y libsodium-dev
\ No newline at end of file
diff --git a/update.sh b/update.sh
deleted file mode 100644
index 2e8b0df..0000000
--- a/update.sh
+++ /dev/null
@@ -1,11 +0,0 @@
-#!/bin/bash
-# Script run when a machine boots. (Always this
-# latest version from Git).
-set -eu
-
-touch /root/at-boot-worked
-
-# Override default /etc/ with the one from this Git.
-cp -r /root/grid5k/etc/* /etc
-
-exit 0
--
To stop receiving notification emails like this one, please contact
gnunet@gnunet.org.
- [taler-grid5k] branch master updated (74d0f38 -> 2554db5),
gnunet <=
- [taler-grid5k] 12/141: fix sed command, gnunet, 2021/11/18
- [taler-grid5k] 06/141: add env variable template, gnunet, 2021/11/18
- [taler-grid5k] 08/141: make pq accesible, gnunet, 2021/11/18
- [taler-grid5k] 11/141: fix exch script, gnunet, 2021/11/18
- [taler-grid5k] 09/141: fix overwriting, gnunet, 2021/11/18
- [taler-grid5k] 01/141: add new version of image, gnunet, 2021/11/18
- [taler-grid5k] 02/141: rename jobs, gnunet, 2021/11/18
- [taler-grid5k] 07/141: update db script, gnunet, 2021/11/18
- [taler-grid5k] 10/141: update bank config, add dns and bank scripts, gnunet, 2021/11/18
- [taler-grid5k] 04/141: add dummy scripts and some etc files, gnunet, 2021/11/18