[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[taler-grid5k] branch master updated (d6cc557 -> 0901fad)
|
From: |
gnunet |
|
Subject: |
[taler-grid5k] branch master updated (d6cc557 -> 0901fad) |
|
Date: |
Tue, 14 Dec 2021 15:09:42 +0100 |
This is an automated email from the git hooks/post-receive script.
marco-boss pushed a change to branch master
in repository grid5k.
from d6cc557 DNS included in experiment
new f317984 working well unless dns
new fc1e9b5 dns should work now
new 5f62e61 fix unbound variable in helper
new ab67393 run 'createuser' in image build - try to fix dmsasq issue
new 96eb57a add configuration option for cmd prefix
new d36f490 cleanup setup script - add completition
new 720427e update doc in default env
new 737ca46 remove executable bits
new fe1c693 add config options for all exchange processes
new 593c101 update service files
new 0ddd5d9 switch back to dahu
new 5615333 increase max_locks_per_transaction
new 503f1dc add first grafana exports
new 2fb9f2a update dashboard exports
new 2f28f21 build wallet from source since dist does not work
new a3edb4e fix wallet build - broken on master
new 22e1918 still not working on master, use previous commit
new 8fdeb73 wallet core still not building in one step
new 240986e running make two times seems to work
new ec94d62 issue in logbackup
new 85dcb07 switch back to wallet master
new e4e1342 add pgbouncer, tune pg config
new a2abe2d disable pgbouncer per default
new 01793c1 cleanup with functions - try work_mem and idle_in_transaction
new 6bbc920 enable postgres-exporter again
new e0ff0fc add pgbouncer config
new 5974781 rename userlist
new 6ef6814 increase pgbouncer max connections
new 6b404f5 switch to transaction mode since otherwise its all the same
as when pgbouncer not used at all
new 31cd61c increase default pool size
new 6bf03a7 pgbouncer still occupying 6432 even when stopped
new be4d5be fix typo
new 4aad1ae add merchant to exporter
new 334e9f0 exporter for merchant now on monitor too
new 4ae70bb some more pgbouncer config
new 600c678 Update image build, add install script to install taler
binaries in running experiment
new ca32ef0 bootstrap seems to work with sudo
new 29059d6 speedup image build
new 26ffa69 fix typo
new 2a31f3e document variables
new 44fb492 update installer script
new 2353f52 fix setup.sh
new 6e8f9fd update config of db and exch
new 57a6288 -resolve conflict
new d1312a5 take postgres config to seperate file
new cfd4a85 remove previous builds
new 682015d fix stupid bug and fix arguments for bank
new ea45216 update install script
new dd94049 monitor dns too, change db settings
new 83312a5 note about postgres huge pages
new 28618d0 add auto explain - not tested yet
new 3252295 possible to configure max_requests with env file
new 7e9c2bd add more description about huge pages
new 760069f enable huge_pages, implement exchange scrape logic
new d8cabd2 db disk mount
new 40c077f fix startup
new 92f2f38 improve wallet script
new 93009e2 sanitize scripts
new 4ca1029 fix taler-perf.sh, improve other scripts
new feb7d12 fix logbackup so it gets rotated correctly
new 31178d9 refactor db.sh
new c802de8 fix monitor
new acdeab2 fix
new 2f71d0e comment scripts and add help
new 56e059b filter wallet getRecord messages - they spam the logs
new 0dc773f update dashboard exports
new 65e72ca update dashboards
new 8902748 update grafana dashboards, add option for ram mount in db
new d251330 update plotter script
new 396e680 update plotter README and dashboards
new 202f987 remove unused rpecs and handled notes
new f3837cb add possibility to define grid5k repo branch for experiments
new 0901fad scripts documented and commented
The 73 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
README.md | 18 +-
additional/README.md | 47 +-
additional/config.yaml | 20 -
additional/grafana/database.json | 3577 ++++++++++++++++++++
additional/grafana/load-statistics.json | 1314 +++++++
additional/grafana/logs.json | 202 ++
additional/grafana/request-statistics.json | 1386 ++++++++
additional/grafana/taler-performance.json | 176 +
additional/grafana/transactions.json | 1810 ++++++++++
additional/plot.sh | 38 -
additional/plots/config.yaml | 29 +
additional/plots/plot.sh | 66 +
configs/etc/bind/named.conf.options | 1 -
configs/etc/default/prometheus-postgres-exporter | 2 +-
configs/etc/default/taler-exchange | 1 -
configs/etc/monitor/exchange-exporters.yaml.tpl | 6 +
configs/etc/monitor/node-exporters.yaml.tpl | 16 +-
configs/etc/monitor/prometheus.yaml | 2 +-
.../nginx/{sites-available => sites-enabled}/proxy | 0
configs/etc/pgbouncer/pgbouncer.ini | 25 +
configs/etc/pgbouncer/userlist.txt | 1 +
configs/etc/rsyslog.d/taler.conf | 2 +-
configs/etc/taler/conf.d/exchange-business.conf | 2 +
configs/etc/taler/conf.d/exchange-coins.conf | 0
.../exchange-accountcredentials.secret.conf | 0
configs/etc/taler/secrets/exchange-db.secret.conf | 0
configs/etc/taler/taler.conf | 0
.../system/taler-exchange-aggregator.service | 4 +-
.../systemd/system/taler-exchange-closer.service | 4 +-
.../systemd/system/taler-exchange-httpd.service | 11 +-
.../lib/systemd/system/taler-exchange-httpd.socket | 0
.../systemd/system/taler-exchange-httpd@.service | 10 +-
.../systemd/system/taler-exchange-httpd@.socket | 0
.../system/taler-exchange-secmod-eddsa.service | 4 +-
.../system/taler-exchange-secmod-rsa.service | 4 +-
.../systemd/system/taler-exchange-transfer.service | 4 +-
.../system/taler-exchange-wirewatch.service | 4 +-
.../system/taler-exchange-wirewatch@.service | 7 +-
.../usr/lib/systemd/system/taler-fakebank.service | 4 +-
...r-logbackup.service => taler-logrotate.service} | 2 +-
...taler-logbackup.timer => taler-logrotate.timer} | 4 +-
.../usr/lib/systemd/system/taler-wallet@.service | 0
configs/var/lib/bind/perf.taler | 2 +-
docker/Dockerfile | 20 +-
docker/README.md | 21 +-
docker/entrypoint.sh | 145 +-
experiment/README.md | 54 +-
additional/grafana/tbd => experiment/TODO | 0
experiment/env | 88 +-
experiment/experiment-specification.yml | 22 +-
experiment/infra.rspec | 50 -
experiment/scripts/bank.sh | 26 +-
experiment/scripts/benchmark.sh | 58 +-
experiment/scripts/createusers.sh | 8 +-
experiment/scripts/database.sh | 181 +-
experiment/scripts/dns.sh | 12 -
experiment/scripts/exchange.sh | 124 +-
experiment/scripts/helpers.sh | 141 +-
experiment/scripts/install.sh | 74 +
experiment/scripts/merchant.sh | 82 +-
experiment/scripts/monitor.sh | 200 +-
experiment/scripts/ping.sh | 24 +-
experiment/scripts/proxy.sh | 105 +-
experiment/scripts/run.sh | 61 +-
experiment/scripts/setup.sh | 223 +-
experiment/scripts/taler-perf.sh | 87 +-
experiment/scripts/wallet.sh | 92 +-
experiment/taler.rspec | 170 +
experiment/wallets.rspec | 63 -
image/README.md | 38 +-
image/taler-debian11.yaml | 93 +-
notes.txt | 25 -
72 files changed, 10245 insertions(+), 847 deletions(-)
delete mode 100644 additional/config.yaml
create mode 100644 additional/grafana/database.json
create mode 100644 additional/grafana/load-statistics.json
create mode 100644 additional/grafana/logs.json
create mode 100644 additional/grafana/request-statistics.json
create mode 100644 additional/grafana/taler-performance.json
create mode 100644 additional/grafana/transactions.json
delete mode 100755 additional/plot.sh
create mode 100644 additional/plots/config.yaml
create mode 100755 additional/plots/plot.sh
delete mode 100644 configs/etc/default/taler-exchange
create mode 100644 configs/etc/monitor/exchange-exporters.yaml.tpl
rename configs/etc/nginx/{sites-available => sites-enabled}/proxy (100%)
create mode 100644 configs/etc/pgbouncer/pgbouncer.ini
create mode 100644 configs/etc/pgbouncer/userlist.txt
mode change 100755 => 100644 configs/etc/taler/conf.d/exchange-business.conf
mode change 100755 => 100644 configs/etc/taler/conf.d/exchange-coins.conf
mode change 100755 => 100644
configs/etc/taler/secrets/exchange-accountcredentials.secret.conf
mode change 100755 => 100644 configs/etc/taler/secrets/exchange-db.secret.conf
mode change 100755 => 100644 configs/etc/taler/taler.conf
mode change 100755 => 100644
configs/usr/lib/systemd/system/taler-exchange-httpd.service
mode change 100755 => 100644
configs/usr/lib/systemd/system/taler-exchange-httpd.socket
mode change 100755 => 100644
configs/usr/lib/systemd/system/taler-exchange-httpd@.service
mode change 100755 => 100644
configs/usr/lib/systemd/system/taler-exchange-httpd@.socket
rename configs/usr/lib/systemd/system/{taler-logbackup.service =>
taler-logrotate.service} (86%)
rename configs/usr/lib/systemd/system/{taler-logbackup.timer =>
taler-logrotate.timer} (68%)
mode change 100755 => 100644
configs/usr/lib/systemd/system/taler-wallet@.service
rename additional/grafana/tbd => experiment/TODO (100%)
delete mode 100644 experiment/infra.rspec
delete mode 100644 experiment/scripts/dns.sh
create mode 100755 experiment/scripts/install.sh
create mode 100644 experiment/taler.rspec
delete mode 100644 experiment/wallets.rspec
diff --git a/README.md b/README.md
index bf0d198..3f170a0 100644
--- a/README.md
+++ b/README.md
@@ -9,7 +9,7 @@ which describes how the parts in there can be used.
Contains all configurations and scripts needed
for the experiment to run in the grid. They are setup to
-work with (jFed)[https://jfed.ilabt.imec.be/].
+work with [jFed](https://jfed.ilabt.imec.be/).
### Image
@@ -25,11 +25,23 @@ No need to install the required build tools on your machine.
### Additional
-Scripts and configurations which can be used to e.g. persist
+Additional resources such as grafana dashboards and
+scripts/configurations which can be used to e.g. persist
data of experiments.
### Configs
Contains the configurations for the applications in the environment.
-They will be copied and adjusted once an experiment is started.
+They will be adjusted copied to '/' (some make sure to add the correct
directory strucutre)
+once an experiment is started.
+## Quick Start
+
+To run an experiment, you must
+
+* (optionally) have a grafana instance
+* Make sure the environment exists in the public direcory which is configured
in
+ `experiment/taler.rspec`. If its not in the grid, use `image/README.md` or
`docker/README.md`
+ to see how to build such an environment.
+* Read `experiment/README.md` for instructions on how to run an experiment
inside the grid
+
diff --git a/additional/README.md b/additional/README.md
index 28278cf..25919f3 100644
--- a/additional/README.md
+++ b/additional/README.md
@@ -1,10 +1,49 @@
# Additional Resources for Experiments
-## plot.sh
+## plots
-Create png plots from the grafana experiment dashboards using [Grafana
Dashboard Plotter](https://github.com/bossm8/grafana-dashboard-plotter).
+### plot.sh
+
+Create png plots from the grafana experiment dashboards using
+[Grafana Dashboard
Plotter](https://github.com/bossm8/grafana-dashboard-plotter).
+
+To use this script, the `.env` file located in `experiment` needs to be
configured
+since grafana configuration is read from there.
+
+Read config.yaml, plot.sh and the README of the dashboard plotter for help.
+
+Basically the steps are the following (inside plots directory):
+
+* Export the experiment start time as a unix timestamp:
+ `export FROM=$(date --date="today 08:30:00" +%s)`
+ If not done, now-4h is taken as default (override in config.yaml).
+* Optionally export the end of the experiment:
+ `export TO=$(date +%s)`
+ If not done, now is taken as default.
+* Run the script:
+ `./plot.sh <OPTIONAL_NAME>`
+ `OPTIONAL_NAME` can be a name for the experiment, it is used
+ in the archive which is created once the plots are finished.
+ Default is `TO`.
## grafana
-**TBD**
-Will contain grafana dashboards once finished
+### Custom
+
+Contains all *custom* dashboards for the experiments.
+Import them vie `Create->Import->Upload JSON` (plus sign)
+
+The database dashboard is a combination of the following two (plus some custom
custom panels):
+
+* [Postgres Overview](https://grafana.com/grafana/dashboards/455): 455
+* [PostgreSQL Statistics](https://grafana.com/grafana/dashboards/6742): 6742
+
+### Library
+
+Additional ones needed can be imported from the library.
+In your Grafana instance head to `Create->Import->Import via grafana.com`
+and copy those ID one after another:
+
+* [Node Exporter Full](https://grafana.com/grafana/dashboards/1860): 1860
+* [Nginx Exporter](https://grafana.com/grafana/dashboards/12708): 12708
+
diff --git a/additional/config.yaml b/additional/config.yaml
deleted file mode 100644
index 8a13227..0000000
--- a/additional/config.yaml
+++ /dev/null
@@ -1,20 +0,0 @@
-grafana:
- admin_api_key: ${ADMIN_API_KEY}
- base_url: ${BASE_URL}
- default_time_range: 14400 # 4 hours
- tls_verify: false
-
-dashboards:
- - uid: 83vvgKKnk # Transacions
- - uid: 2FTtdeOnk # Database
- variables: ['db', 'instance']
- - uid: MsjffzSZz # Proxy
- - uid: WcfSXqDnk # Request Statistics
- variables: ['endpoint']
- - uid: rYdddlPWk # Nodes
- variables: ['node']
- ignore: 'wallet*|monitor*|bank*|merch*'
-
-prometheus:
- node_exporter_job_name: nodes
-
diff --git a/additional/grafana/database.json b/additional/grafana/database.json
new file mode 100644
index 0000000..b81c6d2
--- /dev/null
+++ b/additional/grafana/database.json
@@ -0,0 +1,3577 @@
+{
+ "__inputs": [
+ {
+ "name": "DS_PROMETHEUS",
+ "label": "Prometheus",
+ "description": "",
+ "type": "datasource",
+ "pluginId": "prometheus",
+ "pluginName": "Prometheus"
+ },
+ {
+ "name": "DS_LOKI",
+ "label": "Loki",
+ "description": "",
+ "type": "datasource",
+ "pluginId": "loki",
+ "pluginName": "Loki"
+ }
+ ],
+ "__requires": [
+ {
+ "type": "panel",
+ "id": "bargauge",
+ "name": "Bar gauge",
+ "version": ""
+ },
+ {
+ "type": "grafana",
+ "id": "grafana",
+ "name": "Grafana",
+ "version": "8.2.5"
+ },
+ {
+ "type": "panel",
+ "id": "graph",
+ "name": "Graph (old)",
+ "version": ""
+ },
+ {
+ "type": "datasource",
+ "id": "loki",
+ "name": "Loki",
+ "version": "1.0.0"
+ },
+ {
+ "type": "datasource",
+ "id": "prometheus",
+ "name": "Prometheus",
+ "version": "1.0.0"
+ },
+ {
+ "type": "panel",
+ "id": "stat",
+ "name": "Stat",
+ "version": ""
+ },
+ {
+ "type": "panel",
+ "id": "table",
+ "name": "Table",
+ "version": ""
+ },
+ {
+ "type": "panel",
+ "id": "timeseries",
+ "name": "Time series",
+ "version": ""
+ }
+ ],
+ "annotations": {
+ "list": [
+ {
+ "builtIn": 1,
+ "datasource": "-- Grafana --",
+ "enable": true,
+ "hide": true,
+ "iconColor": "rgba(0, 211, 255, 1)",
+ "name": "Annotations & Alerts",
+ "target": {
+ "limit": 100,
+ "matchAny": false,
+ "tags": [],
+ "type": "dashboard"
+ },
+ "type": "dashboard"
+ }
+ ]
+ },
+ "description": "Dashboard for PostgreSQL Statistics.",
+ "editable": true,
+ "fiscalYearStartMonth": 0,
+ "gnetId": 6742,
+ "graphTooltip": 1,
+ "id": null,
+ "iteration": 1639300468278,
+ "links": [],
+ "liveNow": false,
+ "panels": [
+ {
+ "collapsed": false,
+ "datasource": null,
+ "gridPos": {
+ "h": 1,
+ "w": 24,
+ "x": 0,
+ "y": 0
+ },
+ "id": 34,
+ "panels": [],
+ "title": "Settings",
+ "type": "row"
+ },
+ {
+ "cacheTimeout": null,
+ "datasource": "${DS_PROMETHEUS}",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "fixedColor": "text",
+ "mode": "fixed"
+ },
+ "mappings": [
+ {
+ "options": {
+ "match": "null",
+ "result": {
+ "text": "N/A"
+ }
+ },
+ "type": "special"
+ }
+ ],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ },
+ "unit": "none"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 3,
+ "x": 0,
+ "y": 1
+ },
+ "id": 2,
+ "interval": null,
+ "links": [],
+ "maxDataPoints": 100,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "none",
+ "justifyMode": "auto",
+ "orientation": "horizontal",
+ "reduceOptions": {
+ "calcs": [
+ "mean"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "name"
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "pg_static{instance=~\"$instance\", server=\"$server\"}",
+ "format": "time_series",
+ "instant": true,
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "{{short_version}}",
+ "refId": "A"
+ }
+ ],
+ "title": "Version",
+ "type": "stat"
+ },
+ {
+ "cacheTimeout": null,
+ "datasource": "${DS_PROMETHEUS}",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [
+ {
+ "options": {
+ "match": "null",
+ "result": {
+ "text": "N/A"
+ }
+ },
+ "type": "special"
+ }
+ ],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "none"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 3,
+ "x": 3,
+ "y": 1
+ },
+ "id": 54,
+ "interval": null,
+ "links": [],
+ "maxDataPoints": 100,
+ "options": {
+ "colorMode": "none",
+ "graphMode": "none",
+ "justifyMode": "auto",
+ "orientation": "horizontal",
+ "reduceOptions": {
+ "calcs": [
+ "mean"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "pg_settings_max_connections{instance=\"$instance\",
server=\"$server\"}",
+ "format": "time_series",
+ "instant": true,
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "Max Connections",
+ "type": "stat"
+ },
+ {
+ "cacheTimeout": null,
+ "datasource": "${DS_PROMETHEUS}",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [
+ {
+ "options": {
+ "match": "null",
+ "result": {
+ "text": "N/A"
+ }
+ },
+ "type": "special"
+ }
+ ],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "bytes"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 3,
+ "x": 6,
+ "y": 1
+ },
+ "id": 56,
+ "interval": null,
+ "links": [],
+ "maxDataPoints": 100,
+ "options": {
+ "colorMode": "none",
+ "graphMode": "none",
+ "justifyMode": "auto",
+ "orientation": "horizontal",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "pg_settings_shared_buffers_bytes{instance=\"$instance\",
server=\"$server\"}",
+ "format": "time_series",
+ "instant": true,
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "Shared Buffers",
+ "type": "stat"
+ },
+ {
+ "cacheTimeout": null,
+ "datasource": "${DS_PROMETHEUS}",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [
+ {
+ "options": {
+ "match": "null",
+ "result": {
+ "text": "N/A"
+ }
+ },
+ "type": "special"
+ }
+ ],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "bytes"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 3,
+ "x": 9,
+ "y": 1
+ },
+ "id": 58,
+ "interval": null,
+ "links": [],
+ "maxDataPoints": 100,
+ "options": {
+ "colorMode": "none",
+ "graphMode": "none",
+ "justifyMode": "auto",
+ "orientation": "horizontal",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr":
"pg_settings_effective_cache_size_bytes{instance=\"$instance\",
server=\"$server\"}",
+ "format": "time_series",
+ "instant": true,
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "Effective Cache",
+ "type": "stat"
+ },
+ {
+ "cacheTimeout": null,
+ "datasource": "${DS_PROMETHEUS}",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [
+ {
+ "options": {
+ "match": "null",
+ "result": {
+ "text": "N/A"
+ }
+ },
+ "type": "special"
+ }
+ ],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "bytes"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 3,
+ "x": 12,
+ "y": 1
+ },
+ "id": 60,
+ "interval": null,
+ "links": [],
+ "maxDataPoints": 100,
+ "options": {
+ "colorMode": "none",
+ "graphMode": "none",
+ "justifyMode": "auto",
+ "orientation": "horizontal",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr":
"pg_settings_maintenance_work_mem_bytes{instance=\"$instance\",
server=\"$server\"}",
+ "format": "time_series",
+ "instant": true,
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "Maintenance Work Mem",
+ "type": "stat"
+ },
+ {
+ "cacheTimeout": null,
+ "datasource": "${DS_PROMETHEUS}",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [
+ {
+ "options": {
+ "match": "null",
+ "result": {
+ "text": "N/A"
+ }
+ },
+ "type": "special"
+ }
+ ],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "bytes"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 3,
+ "x": 15,
+ "y": 1
+ },
+ "id": 66,
+ "interval": null,
+ "links": [],
+ "maxDataPoints": 100,
+ "options": {
+ "colorMode": "none",
+ "graphMode": "none",
+ "justifyMode": "auto",
+ "orientation": "horizontal",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "pg_settings_work_mem_bytes{instance=\"$instance\",
server=\"$server\"}",
+ "format": "time_series",
+ "instant": true,
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "Work Mem",
+ "type": "stat"
+ },
+ {
+ "cacheTimeout": null,
+ "datasource": "${DS_PROMETHEUS}",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "decimals": 1,
+ "mappings": [
+ {
+ "options": {
+ "match": "null",
+ "result": {
+ "text": "N/A"
+ }
+ },
+ "type": "special"
+ }
+ ],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "bytes"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 3,
+ "x": 18,
+ "y": 1
+ },
+ "id": 32,
+ "interval": null,
+ "links": [],
+ "maxDataPoints": 100,
+ "options": {
+ "colorMode": "none",
+ "graphMode": "none",
+ "justifyMode": "auto",
+ "orientation": "horizontal",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "pg_settings_min_wal_size_bytes{instance=\"$instance\",
server=\"$server\"}",
+ "format": "time_series",
+ "instant": true,
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "Min WAL Size",
+ "type": "stat"
+ },
+ {
+ "cacheTimeout": null,
+ "datasource": "${DS_PROMETHEUS}",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "decimals": 1,
+ "mappings": [
+ {
+ "options": {
+ "match": "null",
+ "result": {
+ "text": "N/A"
+ }
+ },
+ "type": "special"
+ }
+ ],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "bytes"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 3,
+ "x": 21,
+ "y": 1
+ },
+ "id": 86,
+ "interval": null,
+ "links": [],
+ "maxDataPoints": 100,
+ "options": {
+ "colorMode": "none",
+ "graphMode": "none",
+ "justifyMode": "auto",
+ "orientation": "horizontal",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "pg_settings_max_wal_size_bytes{instance=\"$instance\",
server=\"$server\"}",
+ "format": "time_series",
+ "instant": true,
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "Max WAL Size",
+ "type": "stat"
+ },
+ {
+ "cacheTimeout": null,
+ "datasource": "${DS_PROMETHEUS}",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [
+ {
+ "options": {
+ "match": "null",
+ "result": {
+ "text": "N/A"
+ }
+ },
+ "type": "special"
+ }
+ ],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "none"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 3,
+ "x": 0,
+ "y": 4
+ },
+ "id": 62,
+ "interval": null,
+ "links": [],
+ "maxDataPoints": 100,
+ "options": {
+ "colorMode": "none",
+ "graphMode": "none",
+ "justifyMode": "auto",
+ "orientation": "horizontal",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "pg_settings_random_page_cost{instance=\"$instance\",
server=\"$server\"}",
+ "format": "time_series",
+ "instant": true,
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "Random Page Cost",
+ "type": "stat"
+ },
+ {
+ "cacheTimeout": null,
+ "datasource": "${DS_PROMETHEUS}",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [
+ {
+ "options": {
+ "match": "null",
+ "result": {
+ "text": "N/A"
+ }
+ },
+ "type": "special"
+ }
+ ],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "none"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 3,
+ "x": 3,
+ "y": 4
+ },
+ "id": 70,
+ "interval": null,
+ "links": [],
+ "maxDataPoints": 100,
+ "options": {
+ "colorMode": "none",
+ "graphMode": "none",
+ "justifyMode": "auto",
+ "orientation": "horizontal",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "pg_settings_seq_page_cost{instance=~\"$instance\",
server=\"$server\"}",
+ "format": "time_series",
+ "instant": true,
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "Seq Page Cost",
+ "type": "stat"
+ },
+ {
+ "cacheTimeout": null,
+ "datasource": "${DS_PROMETHEUS}",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [
+ {
+ "options": {
+ "match": "null",
+ "result": {
+ "text": "N/A"
+ }
+ },
+ "type": "special"
+ }
+ ],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "none"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 3,
+ "x": 6,
+ "y": 4
+ },
+ "id": 64,
+ "interval": null,
+ "links": [],
+ "maxDataPoints": 100,
+ "options": {
+ "colorMode": "none",
+ "graphMode": "none",
+ "justifyMode": "auto",
+ "orientation": "horizontal",
+ "reduceOptions": {
+ "calcs": [
+ "mean"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "pg_settings_max_worker_processes{instance=\"$instance\",
server=\"$server\"}",
+ "format": "time_series",
+ "instant": true,
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "Max Worker Processes",
+ "type": "stat"
+ },
+ {
+ "cacheTimeout": null,
+ "datasource": "${DS_PROMETHEUS}",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [
+ {
+ "options": {
+ "match": "null",
+ "result": {
+ "text": "N/A"
+ }
+ },
+ "type": "special"
+ }
+ ],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "none"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 3,
+ "x": 9,
+ "y": 4
+ },
+ "id": 68,
+ "interval": null,
+ "links": [],
+ "maxDataPoints": 100,
+ "options": {
+ "colorMode": "none",
+ "graphMode": "none",
+ "justifyMode": "auto",
+ "orientation": "horizontal",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "pg_settings_max_parallel_workers{instance=\"$instance\",
server=\"$server\"}",
+ "format": "time_series",
+ "instant": true,
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "Max Parallel Workers",
+ "type": "stat"
+ },
+ {
+ "datasource": null,
+ "description": "",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "fixedColor": "text",
+ "mode": "fixed"
+ },
+ "mappings": [],
+ "min": 13,
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ },
+ "unit": "none"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 3,
+ "x": 12,
+ "y": 4
+ },
+ "id": 83,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "none",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr":
"pg_settings_max_locks_per_transaction{instance=\"$instance\",
server=\"$server\"}",
+ "instant": true,
+ "interval": "",
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "Max Locks per Transaction",
+ "type": "stat"
+ },
+ {
+ "datasource": null,
+ "description": "",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "fixedColor": "text",
+ "mode": "fixed"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ },
+ "unit": "s"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 3,
+ "x": 15,
+ "y": 4
+ },
+ "id": 82,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "none",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr":
"pg_settings_log_min_duration_statement_seconds{instance=\"$instance\",
server=\"$server\"}",
+ "instant": true,
+ "interval": "",
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "Log Min Duration Statement",
+ "type": "stat"
+ },
+ {
+ "datasource": null,
+ "description": "",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "fixedColor": "text",
+ "mode": "fixed"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ },
+ "unit": "s"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 3,
+ "x": 18,
+ "y": 4
+ },
+ "id": 90,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "none",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr":
"pg_settings_auto_explain_log_min_duration_seconds{instance=\"$instance\",
server=\"$server\"}",
+ "instant": true,
+ "interval": "",
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "Auto Explain Log Min Duration",
+ "type": "stat"
+ },
+ {
+ "cacheTimeout": null,
+ "datasource": "${DS_PROMETHEUS}",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "decimals": 1,
+ "mappings": [
+ {
+ "options": {
+ "match": "null",
+ "result": {
+ "index": 0,
+ "text": "N/A"
+ }
+ },
+ "type": "special"
+ }
+ ],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ },
+ "unit": "s"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 3,
+ "x": 21,
+ "y": 4
+ },
+ "id": 87,
+ "interval": null,
+ "links": [],
+ "maxDataPoints": 100,
+ "options": {
+ "colorMode": "none",
+ "graphMode": "none",
+ "justifyMode": "auto",
+ "orientation": "horizontal",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr":
"pg_settings_idle_in_transaction_session_timeout_seconds{instance=\"$instance\",
server=\"$server\"}",
+ "format": "time_series",
+ "instant": true,
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "Idle in Transaction Timeout",
+ "type": "stat"
+ },
+ {
+ "collapsed": false,
+ "datasource": null,
+ "gridPos": {
+ "h": 1,
+ "w": 24,
+ "x": 0,
+ "y": 7
+ },
+ "id": 85,
+ "panels": [],
+ "title": "Failure Statistics",
+ "type": "row"
+ },
+ {
+ "datasource": null,
+ "description": "Serialization Errors per Second and Request-Type",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "axisLabel": "Serialization Errors / Second",
+ "axisPlacement": "auto",
+ "barAlignment": 0,
+ "drawStyle": "line",
+ "fillOpacity": 0,
+ "gradientMode": "none",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ },
+ "lineInterpolation": "linear",
+ "lineWidth": 1,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "auto",
+ "spanNulls": false,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
+ }
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ },
+ "unit": "none"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 9,
+ "w": 10,
+ "x": 0,
+ "y": 8
+ },
+ "id": 92,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "list",
+ "placement": "bottom"
+ },
+ "tooltip": {
+ "mode": "single"
+ }
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "sum by (type)
(rate(taler_exchange_serialization_failures{}[1m]))",
+ "instant": false,
+ "interval": "",
+ "legendFormat": "{{type}}",
+ "refId": "A"
+ }
+ ],
+ "title": "Serialization Errors",
+ "transformations": [],
+ "type": "timeseries"
+ },
+ {
+ "datasource": null,
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "fixedColor": "text",
+ "mode": "fixed"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 3,
+ "x": 10,
+ "y": 8
+ },
+ "id": 76,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "none",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "sum(pg_stat_database_xact_commit{instance=~\"$instance\",
server=\"$server\", datname=\"$db\"})",
+ "instant": true,
+ "interval": "",
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "Count of Commited Transactions",
+ "type": "stat"
+ },
+ {
+ "datasource": null,
+ "description": "Number of serialization errors by request endpoint
(type) since the last exchange restarts",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "text",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 100
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 9,
+ "w": 6,
+ "x": 13,
+ "y": 8
+ },
+ "id": 94,
+ "options": {
+ "displayMode": "basic",
+ "orientation": "horizontal",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "showUnfilled": true,
+ "text": {}
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "sort_desc (sum by(type)
(taler_exchange_serialization_failures{}))",
+ "instant": true,
+ "interval": "",
+ "legendFormat": "{{type}}",
+ "refId": "A"
+ }
+ ],
+ "timeFrom": null,
+ "timeShift": null,
+ "title": "Serialization Errors by Request Type",
+ "transformations": [],
+ "type": "bargauge"
+ },
+ {
+ "datasource": null,
+ "description": "Number of serialization errors per request since the
last exchange restarts",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [
+ {
+ "options": {
+ "match": "nan",
+ "result": {
+ "index": 0,
+ "text": "-"
+ }
+ },
+ "type": "special"
+ }
+ ],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "text",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 2
+ }
+ ]
+ },
+ "unit": "percentunit"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 9,
+ "w": 5,
+ "x": 19,
+ "y": 8
+ },
+ "id": 95,
+ "options": {
+ "displayMode": "basic",
+ "orientation": "horizontal",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "showUnfilled": true,
+ "text": {}
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "sort_desc(sum by(type)
(rate(taler_exchange_serialization_failures{}[10m]))\n/\nsum by(type)
(rate(taler_exchange_received_requests{}[10m])))",
+ "instant": true,
+ "interval": "",
+ "legendFormat": "{{type}}",
+ "refId": "A"
+ }
+ ],
+ "timeFrom": null,
+ "timeShift": null,
+ "title": "Serialization Errors / Request",
+ "type": "bargauge"
+ },
+ {
+ "datasource": null,
+ "description": "Total number of serialization errors that happened since
last exchange restarts",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "fixedColor": "text",
+ "mode": "fixed"
+ },
+ "mappings": [],
+ "noValue": "0",
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 3,
+ "x": 10,
+ "y": 11
+ },
+ "id": 74,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "none",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "sum(taler_exchange_serialization_failures{})",
+ "interval": "",
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "Total Serialization Errors",
+ "type": "stat"
+ },
+ {
+ "datasource": "${DS_LOKI}",
+ "description": "Number of queries which took longer than the configured
duration",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "fixedColor": "text",
+ "mode": "fixed"
+ },
+ "mappings": [],
+ "noValue": "0",
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 3,
+ "x": 10,
+ "y": 14
+ },
+ "id": 72,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "none",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "expr": "sum(count_over_time({app=\"taler-database\"} |~ \"duration:
(.*) plan:\" [$__range]))",
+ "instant": true,
+ "range": false,
+ "refId": "A"
+ }
+ ],
+ "title": "Total Slow Queries",
+ "type": "stat"
+ },
+ {
+ "datasource": "${DS_LOKI}",
+ "description": "",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "custom": {
+ "align": "auto",
+ "displayMode": "json-view",
+ "filterable": false
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": [
+ {
+ "matcher": {
+ "id": "byName",
+ "options": "Count"
+ },
+ "properties": [
+ {
+ "id": "custom.width",
+ "value": 100
+ }
+ ]
+ }
+ ]
+ },
+ "gridPos": {
+ "h": 7,
+ "w": 24,
+ "x": 0,
+ "y": 17
+ },
+ "id": 89,
+ "options": {
+ "frameIndex": 1,
+ "showHeader": true,
+ "sortBy": []
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "expr": "sum by (query) (count_over_time({app=\"taler-database\"} |~
\"011Query Text:\" | regexp \".*011Query Text: (?P<query>.*)\" [$__range]))",
+ "instant": true,
+ "range": false,
+ "refId": "A"
+ }
+ ],
+ "title": "Slow Queries",
+ "transformations": [
+ {
+ "id": "groupBy",
+ "options": {
+ "fields": {
+ "Query": {
+ "aggregations": [
+ "count"
+ ],
+ "operation": "groupby"
+ },
+ "Value": {
+ "aggregations": [
+ "lastNotNull"
+ ],
+ "operation": "aggregate"
+ },
+ "Value #A": {
+ "aggregations": [
+ "lastNotNull"
+ ],
+ "operation": "aggregate"
+ },
+ "app": {
+ "aggregations": [
+ "count"
+ ],
+ "operation": "aggregate"
+ },
+ "query": {
+ "aggregations": [],
+ "operation": "groupby"
+ }
+ }
+ }
+ },
+ {
+ "id": "organize",
+ "options": {
+ "excludeByName": {},
+ "indexByName": {},
+ "renameByName": {
+ "Value #A (lastNotNull)": "Count",
+ "Value (lastNotNull)": "Count",
+ "app (count)": "Count",
+ "query": "Query"
+ }
+ }
+ }
+ ],
+ "type": "table"
+ },
+ {
+ "collapsed": false,
+ "datasource": null,
+ "gridPos": {
+ "h": 1,
+ "w": 24,
+ "x": 0,
+ "y": 24
+ },
+ "id": 36,
+ "panels": [],
+ "title": "Connection / Transaction Statistics",
+ "type": "row"
+ },
+ {
+ "datasource": "${DS_PROMETHEUS}",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "axisLabel": "",
+ "axisPlacement": "auto",
+ "barAlignment": 0,
+ "drawStyle": "line",
+ "fillOpacity": 10,
+ "gradientMode": "none",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ },
+ "lineInterpolation": "linear",
+ "lineWidth": 1,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "never",
+ "spanNulls": true,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
+ }
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "short"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 10,
+ "w": 12,
+ "x": 0,
+ "y": 25
+ },
+ "id": 6,
+ "links": [],
+ "options": {
+ "legend": {
+ "calcs": [
+ "mean",
+ "lastNotNull",
+ "max",
+ "min"
+ ],
+ "displayMode": "table",
+ "placement": "bottom"
+ },
+ "tooltip": {
+ "mode": "single"
+ }
+ },
+ "pluginVersion": "8.2.1",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "pg_stat_activity_count{instance=\"$instance\",
datname=\"$db\", server=\"$server\"}",
+ "format": "time_series",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "{{state}}",
+ "refId": "A"
+ }
+ ],
+ "timeFrom": null,
+ "timeShift": null,
+ "title": "Connections",
+ "type": "timeseries"
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": "${DS_PROMETHEUS}",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 10,
+ "w": 12,
+ "x": 12,
+ "y": 25
+ },
+ "hiddenSeries": false,
+ "id": 8,
+ "legend": {
+ "alignAsTable": false,
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "8.2.5",
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "irate(pg_stat_database_xact_commit{instance=\"$instance\",
datname=\"$db\", server=\"$server\"}[5m])",
+ "format": "time_series",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "commits",
+ "refId": "A"
+ },
+ {
+ "exemplar": true,
+ "expr":
"irate(pg_stat_database_xact_rollback{instance=\"$instance\", datname=\"$db\",
server=\"$server\"}[5m])",
+ "format": "time_series",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "rollbacks",
+ "refId": "B"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Transactions",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:573",
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "$$hashKey": "object:574",
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": "${DS_PROMETHEUS}",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 12,
+ "x": 0,
+ "y": 35
+ },
+ "hiddenSeries": false,
+ "id": 18,
+ "legend": {
+ "alignAsTable": true,
+ "avg": true,
+ "current": true,
+ "max": true,
+ "min": true,
+ "show": true,
+ "total": false,
+ "values": true
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "8.2.5",
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "expr": "irate(pg_stat_database_tup_fetched{instance=\"$instance\",
datname=\"$db\"}[5m])",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "legendFormat": "SELECT (index scan)",
+ "refId": "A"
+ },
+ {
+ "expr": "irate(pg_stat_database_tup_returned{instance=\"$instance\",
datname=\"$db\"}[5m])",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "legendFormat": "SELECT (table scan)",
+ "refId": "B"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Read Stats",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "decimals": null,
+ "format": "short",
+ "label": "Rows",
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": "${DS_PROMETHEUS}",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 12,
+ "x": 12,
+ "y": 35
+ },
+ "hiddenSeries": false,
+ "id": 20,
+ "legend": {
+ "alignAsTable": true,
+ "avg": true,
+ "current": true,
+ "max": true,
+ "min": true,
+ "show": true,
+ "total": false,
+ "values": true
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "8.2.5",
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "expr": "irate(pg_stat_database_tup_inserted{instance=\"$instance\",
datname=\"$db\"}[5m])",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "legendFormat": "INSERT",
+ "refId": "A"
+ },
+ {
+ "expr": "irate(pg_stat_database_tup_updated{instance=\"$instance\",
datname=\"$db\"}[5m])",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "legendFormat": "UPDATE",
+ "refId": "B"
+ },
+ {
+ "expr": "irate(pg_stat_database_tup_deleted{instance=\"$instance\",
datname=\"$db\"}[5m])",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "legendFormat": "DELETE",
+ "refId": "C"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Change Stats",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "format": "short",
+ "label": "Rows",
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": "${DS_PROMETHEUS}",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 12,
+ "x": 0,
+ "y": 43
+ },
+ "hiddenSeries": false,
+ "id": 42,
+ "legend": {
+ "alignAsTable": true,
+ "avg": true,
+ "current": true,
+ "hideEmpty": true,
+ "hideZero": true,
+ "max": true,
+ "min": true,
+ "show": true,
+ "total": false,
+ "values": true
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "8.2.5",
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "expr": "pg_stat_activity_max_tx_duration{instance=\"$instance\",
datname=\"$db\"}",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "legendFormat": "max_tx_duration [{{state}}]",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Longest Transaction",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "format": "s",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": "${DS_PROMETHEUS}",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 12,
+ "x": 12,
+ "y": 43
+ },
+ "hiddenSeries": false,
+ "id": 44,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": false,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "8.2.5",
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "pg_stat_database_blks_hit{instance=\"$instance\",
datname=\"$db\", server=\"$server\"} /
(pg_stat_database_blks_read{instance=\"$instance\", datname=\"$db\",
server=\"$server\"} + pg_stat_database_blks_hit{instance=\"$instance\",
datname=\"$db\", server=\"$server\"})",
+ "format": "time_series",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "Cache Hit Rate",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Cache Hit Rate",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:494",
+ "decimals": 4,
+ "format": "percentunit",
+ "label": "",
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "$$hashKey": "object:495",
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "collapsed": false,
+ "datasource": null,
+ "gridPos": {
+ "h": 1,
+ "w": 24,
+ "x": 0,
+ "y": 51
+ },
+ "id": 50,
+ "panels": [],
+ "title": "Misc",
+ "type": "row"
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": "${DS_PROMETHEUS}",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 6,
+ "w": 17,
+ "x": 0,
+ "y": 52
+ },
+ "hiddenSeries": false,
+ "id": 46,
+ "legend": {
+ "alignAsTable": true,
+ "avg": true,
+ "current": true,
+ "max": true,
+ "min": true,
+ "rightSide": true,
+ "show": true,
+ "total": false,
+ "values": true
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "8.2.5",
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "exemplar": true,
+ "expr":
"irate(pg_stat_bgwriter_buffers_backend{instance=\"$instance\",
server=\"$server\"}[5m])",
+ "format": "time_series",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "buffers_backend",
+ "refId": "A"
+ },
+ {
+ "exemplar": true,
+ "expr":
"irate(pg_stat_bgwriter_buffers_alloc{instance=\"$instance\",
server=\"$server\"}[5m])",
+ "format": "time_series",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "buffers_alloc",
+ "refId": "B"
+ },
+ {
+ "exemplar": true,
+ "expr":
"irate(pg_stat_bgwriter_buffers_backend_fsync{instance=\"$instance\",
server=\"$server\"}[5m])",
+ "format": "time_series",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "backend_fsync",
+ "refId": "C"
+ },
+ {
+ "exemplar": true,
+ "expr":
"irate(pg_stat_bgwriter_buffers_checkpoint{instance=\"$instance\",
server=\"$server\"}[5m])",
+ "format": "time_series",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "buffers_checkpoint",
+ "refId": "D"
+ },
+ {
+ "exemplar": true,
+ "expr":
"irate(pg_stat_bgwriter_buffers_clean{instance=\"$instance\",
server=\"$server\"}[5m])",
+ "format": "time_series",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "buffers_clean",
+ "refId": "E"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Buffers (bgwriter)",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:97",
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "$$hashKey": "object:98",
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": "${DS_PROMETHEUS}",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 6,
+ "w": 7,
+ "x": 17,
+ "y": 52
+ },
+ "hiddenSeries": false,
+ "id": 28,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "8.2.5",
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "irate(pg_stat_database_conflicts{instance=\"$instance\",
datname=\"$db\", server=\"$server\"}[5m])",
+ "format": "time_series",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "conflicts",
+ "refId": "B"
+ },
+ {
+ "exemplar": true,
+ "expr": "irate(pg_stat_database_deadlocks{instance=\"$instance\",
datname=\"$db\", server=\"$server\"}[5m])",
+ "format": "time_series",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "deadlocks",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Conflicts/Deadlocks",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:257",
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "$$hashKey": "object:258",
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": "${DS_PROMETHEUS}",
+ "decimals": 0,
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 7,
+ "w": 17,
+ "x": 0,
+ "y": 58
+ },
+ "hiddenSeries": false,
+ "id": 30,
+ "legend": {
+ "alignAsTable": true,
+ "avg": true,
+ "current": true,
+ "max": true,
+ "min": true,
+ "rightSide": true,
+ "show": true,
+ "total": false,
+ "values": true
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "8.2.5",
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "pg_locks_count{instance=\"$instance\", datname=\"$db\",
server=\"$server\"}",
+ "format": "time_series",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "{{mode}}",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Lock Tables",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:97",
+ "decimals": 0,
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "$$hashKey": "object:98",
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": "${DS_PROMETHEUS}",
+ "description": "Total amount of data written to temporary files by
queries in this database. All temporary files are counted, regardless of why
the temporary file was created, and regardless of the log_temp_files setting.",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 7,
+ "w": 7,
+ "x": 17,
+ "y": 58
+ },
+ "hiddenSeries": false,
+ "id": 40,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": false,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "8.2.5",
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "expr": "irate(pg_stat_database_temp_bytes{instance=\"$instance\",
datname=\"$db\"}[5m])",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "legendFormat": "Temp Bytes",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Temp File",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "format": "bytes",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": "${DS_PROMETHEUS}",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 9,
+ "w": 24,
+ "x": 0,
+ "y": 65
+ },
+ "hiddenSeries": false,
+ "id": 38,
+ "legend": {
+ "alignAsTable": true,
+ "avg": true,
+ "current": true,
+ "max": true,
+ "min": true,
+ "show": true,
+ "total": false,
+ "values": true
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "8.2.5",
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "exemplar": true,
+ "expr":
"irate(pg_stat_bgwriter_checkpoint_write_time{instance=\"$instance\",
server=\"$server\"}[5m])",
+ "format": "time_series",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "write_time - Total amount of time that has been
spent in the portion of checkpoint processing where files are written to disk.",
+ "refId": "B"
+ },
+ {
+ "exemplar": true,
+ "expr":
"irate(pg_stat_bgwriter_checkpoint_sync_time{instance=\"$instance\",
server=\"$server\"}[5m])",
+ "format": "time_series",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "sync_time - Total amount of time that has been
spent in the portion of checkpoint processing where files are synchronized to
disk.",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Checkpoint Stats",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:336",
+ "format": "ms",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "$$hashKey": "object:337",
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": "${DS_PROMETHEUS}",
+ "editable": true,
+ "error": false,
+ "fill": 1,
+ "fillGradient": 0,
+ "grid": {},
+ "gridPos": {
+ "h": 7,
+ "w": 20,
+ "x": 0,
+ "y": 74
+ },
+ "hiddenSeries": false,
+ "id": 80,
+ "isNew": true,
+ "legend": {
+ "alignAsTable": true,
+ "avg": true,
+ "current": false,
+ "max": true,
+ "min": true,
+ "rightSide": true,
+ "show": true,
+ "total": false,
+ "values": true
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "connected",
+ "options": {
+ "alertThreshold": true
+ },
+ "percentage": false,
+ "pluginVersion": "8.2.5",
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "alias": "fetched",
+ "dsType": "prometheus",
+ "exemplar": true,
+ "expr":
"sum(irate(pg_stat_database_tup_fetched{datname=~\"$db\",instance=~\"$instance\",
server=\"$server\"}[5m]))",
+ "format": "time_series",
+ "groupBy": [
+ {
+ "params": [
+ "$interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "fetched",
+ "measurement": "postgresql",
+ "policy": "default",
+ "refId": "A",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "tup_fetched"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ },
+ {
+ "params": [
+ "10s"
+ ],
+ "type": "non_negative_derivative"
+ }
+ ]
+ ],
+ "step": 120,
+ "tags": [
+ {
+ "key": "host",
+ "operator": "=~",
+ "value": "/^$host$/"
+ }
+ ]
+ },
+ {
+ "alias": "fetched",
+ "dsType": "prometheus",
+ "exemplar": true,
+ "expr":
"sum(irate(pg_stat_database_tup_returned{datname=~\"$db\",instance=~\"$instance\",
server=\"$server\"}[5m]))",
+ "format": "time_series",
+ "groupBy": [
+ {
+ "params": [
+ "$interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "returned",
+ "measurement": "postgresql",
+ "policy": "default",
+ "refId": "B",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "tup_fetched"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ },
+ {
+ "params": [
+ "10s"
+ ],
+ "type": "non_negative_derivative"
+ }
+ ]
+ ],
+ "step": 120,
+ "tags": [
+ {
+ "key": "host",
+ "operator": "=~",
+ "value": "/^$host$/"
+ }
+ ]
+ },
+ {
+ "alias": "fetched",
+ "dsType": "prometheus",
+ "exemplar": true,
+ "expr":
"sum(irate(pg_stat_database_tup_inserted{datname=~\"$db\",instance=~\"$instance\",
server=\"$server\"}[5m]))",
+ "format": "time_series",
+ "groupBy": [
+ {
+ "params": [
+ "$interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "inserted",
+ "measurement": "postgresql",
+ "policy": "default",
+ "refId": "C",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "tup_fetched"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ },
+ {
+ "params": [
+ "10s"
+ ],
+ "type": "non_negative_derivative"
+ }
+ ]
+ ],
+ "step": 120,
+ "tags": [
+ {
+ "key": "host",
+ "operator": "=~",
+ "value": "/^$host$/"
+ }
+ ]
+ },
+ {
+ "alias": "fetched",
+ "dsType": "prometheus",
+ "exemplar": true,
+ "expr":
"sum(irate(pg_stat_database_tup_updated{datname=~\"$db\",instance=~\"$instance\",
server=\"$server\"}[5m]))",
+ "format": "time_series",
+ "groupBy": [
+ {
+ "params": [
+ "$interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "updated",
+ "measurement": "postgresql",
+ "policy": "default",
+ "refId": "D",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "tup_fetched"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ },
+ {
+ "params": [
+ "10s"
+ ],
+ "type": "non_negative_derivative"
+ }
+ ]
+ ],
+ "step": 120,
+ "tags": [
+ {
+ "key": "host",
+ "operator": "=~",
+ "value": "/^$host$/"
+ }
+ ]
+ },
+ {
+ "alias": "fetched",
+ "dsType": "prometheus",
+ "exemplar": true,
+ "expr":
"sum(irate(pg_stat_database_tup_deleted{datname=~\"$db\",instance=~\"$instance\",
server=\"$server\"}[5m]))",
+ "format": "time_series",
+ "groupBy": [
+ {
+ "params": [
+ "$interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "deleted",
+ "measurement": "postgresql",
+ "policy": "default",
+ "refId": "E",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "tup_fetched"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ },
+ {
+ "params": [
+ "10s"
+ ],
+ "type": "non_negative_derivative"
+ }
+ ]
+ ],
+ "step": 120,
+ "tags": [
+ {
+ "key": "host",
+ "operator": "=~",
+ "value": "/^$host$/"
+ }
+ ]
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Rows",
+ "tooltip": {
+ "msResolution": true,
+ "shared": true,
+ "sort": 0,
+ "value_type": "cumulative"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "$$hashKey": "object:97",
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "$$hashKey": "object:98",
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "cacheTimeout": null,
+ "datasource": "${DS_PROMETHEUS}",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "fixedColor": "rgb(31, 120, 193)",
+ "mode": "fixed"
+ },
+ "decimals": 0,
+ "mappings": [
+ {
+ "options": {
+ "match": "null",
+ "result": {
+ "text": "N/A"
+ }
+ },
+ "type": "special"
+ }
+ ],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "none"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 7,
+ "w": 4,
+ "x": 20,
+ "y": 74
+ },
+ "id": 78,
+ "interval": null,
+ "links": [],
+ "maxDataPoints": 100,
+ "options": {
+ "colorMode": "none",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "horizontal",
+ "reduceOptions": {
+ "calcs": [
+ "mean"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "dsType": "prometheus",
+ "exemplar": true,
+ "expr":
"sum(irate(pg_stat_database_xact_commit{datname=\"$db\",instance=~\"$instance\",
server=\"$server\"}[5m])) +
sum(irate(pg_stat_database_xact_rollback{datname=\"$db\",instance=~\"$instance\",
server=\"$server\"}[5m]))",
+ "format": "time_series",
+ "groupBy": [
+ {
+ "params": [
+ "$interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "interval": "",
+ "intervalFactor": 2,
+ "legendFormat": "",
+ "measurement": "postgresql",
+ "policy": "default",
+ "refId": "A",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "xact_commit"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ },
+ {
+ "params": [
+ "10s"
+ ],
+ "type": "non_negative_derivative"
+ }
+ ]
+ ],
+ "step": 1800,
+ "tags": [
+ {
+ "key": "host",
+ "operator": "=~",
+ "value": "/^$host$/"
+ }
+ ]
+ }
+ ],
+ "title": "QPS",
+ "type": "stat"
+ }
+ ],
+ "refresh": "1m",
+ "schemaVersion": 32,
+ "style": "dark",
+ "tags": [
+ "performance",
+ "postgres"
+ ],
+ "templating": {
+ "list": [
+ {
+ "allValue": null,
+ "current": {},
+ "datasource": "${DS_PROMETHEUS}",
+ "definition": "",
+ "description": null,
+ "error": null,
+ "hide": 0,
+ "includeAll": false,
+ "label": "Instance",
+ "multi": false,
+ "name": "instance",
+ "options": [],
+ "query": {
+ "query": "label_values(pg_up, instance)",
+ "refId": "Prometheus-instance-Variable-Query"
+ },
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "sort": 1,
+ "tagValuesQuery": "",
+ "tagsQuery": "",
+ "type": "query",
+ "useTags": false
+ },
+ {
+ "allValue": null,
+ "current": {},
+ "datasource": "${DS_PROMETHEUS}",
+ "definition": "",
+ "description": null,
+ "error": null,
+ "hide": 0,
+ "includeAll": false,
+ "label": "Database",
+ "multi": false,
+ "name": "db",
+ "options": [],
+ "query": {
+ "query":
"label_values(pg_stat_database_tup_fetched{datname!~\"template.*|postgres\"},datname)",
+ "refId": "Prometheus-db-Variable-Query"
+ },
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "sort": 1,
+ "tagValuesQuery": "",
+ "tagsQuery": "",
+ "type": "query",
+ "useTags": false
+ },
+ {
+ "allValue": null,
+ "current": {},
+ "datasource": "${DS_PROMETHEUS}",
+ "definition": "label_values(pg_settings_array_nulls{}, server)",
+ "description": null,
+ "error": null,
+ "hide": 0,
+ "includeAll": false,
+ "label": "Server",
+ "multi": false,
+ "name": "server",
+ "options": [],
+ "query": {
+ "query": "label_values(pg_settings_array_nulls{}, server)",
+ "refId": "Prometheus-server-Variable-Query"
+ },
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "sort": 0,
+ "type": "query"
+ }
+ ]
+ },
+ "time": {
+ "from": "now-1h",
+ "to": "now"
+ },
+ "timepicker": {
+ "refresh_intervals": [
+ "5s",
+ "10s",
+ "30s",
+ "1m",
+ "5m",
+ "15m",
+ "30m",
+ "1h",
+ "2h",
+ "1d"
+ ],
+ "time_options": [
+ "5m",
+ "15m",
+ "1h",
+ "6h",
+ "12h",
+ "24h",
+ "2d",
+ "7d",
+ "30d"
+ ]
+ },
+ "timezone": "",
+ "title": "Database",
+ "uid": "2FTtdeOnk",
+ "version": 77
+}
\ No newline at end of file
diff --git a/additional/grafana/load-statistics.json
b/additional/grafana/load-statistics.json
new file mode 100644
index 0000000..c335d36
--- /dev/null
+++ b/additional/grafana/load-statistics.json
@@ -0,0 +1,1314 @@
+{
+ "__inputs": [
+ {
+ "name": "DS_PROMETHEUS",
+ "label": "Prometheus",
+ "description": "",
+ "type": "datasource",
+ "pluginId": "prometheus",
+ "pluginName": "Prometheus"
+ },
+ {
+ "name": "DS_LOKI",
+ "label": "Loki",
+ "description": "",
+ "type": "datasource",
+ "pluginId": "loki",
+ "pluginName": "Loki"
+ }
+ ],
+ "__elements": [],
+ "__requires": [
+ {
+ "type": "grafana",
+ "id": "grafana",
+ "name": "Grafana",
+ "version": "8.3.2"
+ },
+ {
+ "type": "datasource",
+ "id": "loki",
+ "name": "Loki",
+ "version": "1.0.0"
+ },
+ {
+ "type": "datasource",
+ "id": "prometheus",
+ "name": "Prometheus",
+ "version": "1.0.0"
+ },
+ {
+ "type": "panel",
+ "id": "timeseries",
+ "name": "Time series",
+ "version": ""
+ }
+ ],
+ "annotations": {
+ "list": [
+ {
+ "builtIn": 1,
+ "datasource": "-- Grafana --",
+ "enable": true,
+ "hide": true,
+ "iconColor": "rgba(0, 211, 255, 1)",
+ "name": "Annotations & Alerts",
+ "target": {
+ "limit": 100,
+ "matchAny": false,
+ "tags": [],
+ "type": "dashboard"
+ },
+ "type": "dashboard"
+ }
+ ]
+ },
+ "editable": true,
+ "fiscalYearStartMonth": 0,
+ "graphTooltip": 1,
+ "id": null,
+ "links": [],
+ "liveNow": false,
+ "panels": [
+ {
+ "collapsed": false,
+ "gridPos": {
+ "h": 1,
+ "w": 24,
+ "x": 0,
+ "y": 0
+ },
+ "id": 13,
+ "panels": [],
+ "title": "Exchange",
+ "type": "row"
+ },
+ {
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "axisLabel": "",
+ "axisPlacement": "auto",
+ "barAlignment": 0,
+ "drawStyle": "line",
+ "fillOpacity": 0,
+ "gradientMode": "none",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ },
+ "lineInterpolation": "linear",
+ "lineWidth": 2,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "auto",
+ "spanNulls": false,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
+ }
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": [
+ {
+ "matcher": {
+ "id": "byRegexp",
+ "options": "CPU Busy .*"
+ },
+ "properties": [
+ {
+ "id": "custom.axisPlacement",
+ "value": "right"
+ },
+ {
+ "id": "unit",
+ "value": "percent"
+ }
+ ]
+ },
+ {
+ "matcher": {
+ "id": "byName",
+ "options": "Requests"
+ },
+ "properties": [
+ {
+ "id": "custom.axisLabel",
+ "value": "Requests / Second"
+ }
+ ]
+ }
+ ]
+ },
+ "gridPos": {
+ "h": 9,
+ "w": 12,
+ "x": 0,
+ "y": 1
+ },
+ "id": 2,
+ "maxDataPoints": 500,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "list",
+ "placement": "bottom"
+ },
+ "tooltip": {
+ "mode": "single"
+ }
+ },
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "sum(rate(nginx_http_requests_total{} [$__rate_interval]))",
+ "instant": false,
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "Requests",
+ "refId": "A"
+ },
+ {
+ "exemplar": true,
+ "expr": "avg by (mode) (sum by (mode, instance)
(rate(node_cpu_seconds_total{mode=~\"system|user\",
component=\"exchange\"}[$__rate_interval])))",
+ "hide": false,
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "CPU Busy {{mode}}",
+ "refId": "B"
+ }
+ ],
+ "title": " [Exchange] Requests vs CPU",
+ "type": "timeseries"
+ },
+ {
+ "description": "",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "axisLabel": "",
+ "axisPlacement": "auto",
+ "barAlignment": 0,
+ "drawStyle": "line",
+ "fillOpacity": 0,
+ "gradientMode": "none",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ },
+ "lineInterpolation": "linear",
+ "lineWidth": 2,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "auto",
+ "spanNulls": false,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
+ }
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": [
+ {
+ "matcher": {
+ "id": "byRegexp",
+ "options": "Network (.*)"
+ },
+ "properties": [
+ {
+ "id": "custom.axisPlacement",
+ "value": "right"
+ },
+ {
+ "id": "unit",
+ "value": "bps"
+ }
+ ]
+ },
+ {
+ "matcher": {
+ "id": "byName",
+ "options": "Requests"
+ },
+ "properties": [
+ {
+ "id": "custom.axisLabel",
+ "value": "Requests / Second"
+ }
+ ]
+ }
+ ]
+ },
+ "gridPos": {
+ "h": 9,
+ "w": 12,
+ "x": 12,
+ "y": 1
+ },
+ "id": 7,
+ "maxDataPoints": 500,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "list",
+ "placement": "bottom"
+ },
+ "tooltip": {
+ "mode": "single"
+ }
+ },
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "sum(rate(nginx_http_requests_total{} [$__rate_interval]))",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "Requests",
+ "refId": "A"
+ },
+ {
+ "exemplar": true,
+ "expr":
"sum(rate(node_network_receive_bytes_total{component=\"exchange\"}[$__rate_interval])
+ rate(node_network_transmit_bytes_total{component=\"exchange\"}
[$__rate_interval]))*8",
+ "hide": false,
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "Network Bandwidth",
+ "refId": "B"
+ }
+ ],
+ "title": "[Exchange] Requests vs Network Traffic",
+ "type": "timeseries"
+ },
+ {
+ "collapsed": false,
+ "gridPos": {
+ "h": 1,
+ "w": 24,
+ "x": 0,
+ "y": 10
+ },
+ "id": 15,
+ "panels": [],
+ "title": "Database",
+ "type": "row"
+ },
+ {
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "axisLabel": "",
+ "axisPlacement": "auto",
+ "barAlignment": 0,
+ "drawStyle": "line",
+ "fillOpacity": 0,
+ "gradientMode": "none",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ },
+ "lineInterpolation": "linear",
+ "lineWidth": 2,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "auto",
+ "spanNulls": false,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
+ }
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": [
+ {
+ "matcher": {
+ "id": "byRegexp",
+ "options": "CPU Busy .*"
+ },
+ "properties": [
+ {
+ "id": "custom.axisPlacement",
+ "value": "right"
+ },
+ {
+ "id": "unit",
+ "value": "percent"
+ }
+ ]
+ },
+ {
+ "matcher": {
+ "id": "byName",
+ "options": "Requests"
+ },
+ "properties": [
+ {
+ "id": "custom.axisLabel",
+ "value": "Requests / Second"
+ }
+ ]
+ }
+ ]
+ },
+ "gridPos": {
+ "h": 9,
+ "w": 12,
+ "x": 0,
+ "y": 11
+ },
+ "id": 3,
+ "maxDataPoints": 500,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "list",
+ "placement": "bottom"
+ },
+ "tooltip": {
+ "mode": "single"
+ }
+ },
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "sum(rate(nginx_http_requests_total{} [$__rate_interval]))",
+ "interval": "",
+ "legendFormat": "Requests",
+ "refId": "A"
+ },
+ {
+ "exemplar": true,
+ "expr": "avg by (mode) (sum by (mode,
instance)(rate(node_cpu_seconds_total{mode=~\"system|user|iowait\",
component=\"database\"}[$__rate_interval])) )",
+ "hide": false,
+ "interval": "",
+ "legendFormat": "CPU Busy {{mode}}",
+ "refId": "B"
+ }
+ ],
+ "title": "[Database] Requests vs CPU",
+ "type": "timeseries"
+ },
+ {
+ "description": "",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "axisLabel": "",
+ "axisPlacement": "auto",
+ "barAlignment": 0,
+ "drawStyle": "line",
+ "fillOpacity": 0,
+ "gradientMode": "none",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ },
+ "lineInterpolation": "linear",
+ "lineWidth": 2,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "auto",
+ "spanNulls": false,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
+ }
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": [
+ {
+ "matcher": {
+ "id": "byRegexp",
+ "options": "Network (.*)"
+ },
+ "properties": [
+ {
+ "id": "custom.axisPlacement",
+ "value": "right"
+ },
+ {
+ "id": "unit",
+ "value": "bps"
+ }
+ ]
+ },
+ {
+ "matcher": {
+ "id": "byName",
+ "options": "Requests"
+ },
+ "properties": [
+ {
+ "id": "custom.axisLabel",
+ "value": "Requests / Second"
+ }
+ ]
+ }
+ ]
+ },
+ "gridPos": {
+ "h": 9,
+ "w": 12,
+ "x": 12,
+ "y": 11
+ },
+ "id": 8,
+ "maxDataPoints": 500,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "list",
+ "placement": "bottom"
+ },
+ "tooltip": {
+ "mode": "single"
+ }
+ },
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "sum(rate(nginx_http_requests_total{} [$__rate_interval]))",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "Requests",
+ "refId": "A"
+ },
+ {
+ "exemplar": true,
+ "expr":
"sum(rate(node_network_receive_bytes_total{component=\"database\"}[$__rate_interval])
+ rate(node_network_transmit_bytes_total{component=\"database\"}
[$__rate_interval]))*8",
+ "hide": false,
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "Network Bandwidth",
+ "refId": "B"
+ }
+ ],
+ "title": "[Database] Requests vs Network Traffic",
+ "type": "timeseries"
+ },
+ {
+ "description": "",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "axisLabel": "",
+ "axisPlacement": "auto",
+ "barAlignment": 0,
+ "drawStyle": "line",
+ "fillOpacity": 0,
+ "gradientMode": "none",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ },
+ "lineInterpolation": "linear",
+ "lineWidth": 2,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "auto",
+ "spanNulls": false,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
+ }
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": [
+ {
+ "matcher": {
+ "id": "byName",
+ "options": "Database Size"
+ },
+ "properties": [
+ {
+ "id": "custom.axisPlacement",
+ "value": "right"
+ },
+ {
+ "id": "unit",
+ "value": "bytes"
+ }
+ ]
+ },
+ {
+ "matcher": {
+ "id": "byName",
+ "options": "Total Requests"
+ },
+ "properties": [
+ {
+ "id": "unit",
+ "value": "short"
+ }
+ ]
+ }
+ ]
+ },
+ "gridPos": {
+ "h": 9,
+ "w": 12,
+ "x": 0,
+ "y": 20
+ },
+ "id": 5,
+ "maxDataPoints": 500,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "list",
+ "placement": "bottom"
+ },
+ "tooltip": {
+ "mode": "single"
+ }
+ },
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "nginx_http_requests_total{}",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "Requests",
+ "refId": "A"
+ },
+ {
+ "exemplar": true,
+ "expr": "sum(pg_database_size_bytes{instance=\"127.0.0.1:9187\"})",
+ "hide": false,
+ "interval": "",
+ "legendFormat": "Database Size",
+ "refId": "C"
+ }
+ ],
+ "title": "Total Requests vs Disk Space used",
+ "type": "timeseries"
+ },
+ {
+ "description": "",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "axisLabel": "",
+ "axisPlacement": "auto",
+ "barAlignment": 0,
+ "drawStyle": "line",
+ "fillOpacity": 0,
+ "gradientMode": "none",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ },
+ "lineInterpolation": "linear",
+ "lineWidth": 2,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "auto",
+ "spanNulls": false,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
+ }
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": [
+ {
+ "matcher": {
+ "id": "byName",
+ "options": "I/O Utilization"
+ },
+ "properties": [
+ {
+ "id": "custom.axisPlacement",
+ "value": "right"
+ },
+ {
+ "id": "unit",
+ "value": "percentunit"
+ }
+ ]
+ },
+ {
+ "matcher": {
+ "id": "byName",
+ "options": "Requests"
+ },
+ "properties": [
+ {
+ "id": "custom.axisLabel",
+ "value": "Requests / Second"
+ }
+ ]
+ }
+ ]
+ },
+ "gridPos": {
+ "h": 9,
+ "w": 12,
+ "x": 12,
+ "y": 20
+ },
+ "id": 9,
+ "maxDataPoints": 500,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "list",
+ "placement": "bottom"
+ },
+ "tooltip": {
+ "mode": "single"
+ }
+ },
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "sum(rate(nginx_http_requests_total{} [$__rate_interval]))",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "Requests",
+ "refId": "A"
+ },
+ {
+ "exemplar": true,
+ "expr":
"sum(rate(node_disk_io_time_seconds_total{component=\"database\"}
[$__rate_interval]))",
+ "hide": false,
+ "interval": "",
+ "legendFormat": "I/O Utilization",
+ "refId": "C"
+ }
+ ],
+ "title": "Requests vs I/O Utilization",
+ "type": "timeseries"
+ },
+ {
+ "description": "",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "axisLabel": "",
+ "axisPlacement": "auto",
+ "barAlignment": 0,
+ "drawStyle": "line",
+ "fillOpacity": 0,
+ "gradientMode": "none",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ },
+ "lineInterpolation": "linear",
+ "lineWidth": 2,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "auto",
+ "spanNulls": false,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
+ }
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": [
+ {
+ "matcher": {
+ "id": "byName",
+ "options": "Cache Hit Rate"
+ },
+ "properties": [
+ {
+ "id": "custom.axisPlacement",
+ "value": "right"
+ },
+ {
+ "id": "unit",
+ "value": "percentunit"
+ }
+ ]
+ },
+ {
+ "matcher": {
+ "id": "byName",
+ "options": "Requests"
+ },
+ "properties": [
+ {
+ "id": "custom.axisLabel",
+ "value": "Requests / Second"
+ }
+ ]
+ }
+ ]
+ },
+ "gridPos": {
+ "h": 9,
+ "w": 12,
+ "x": 0,
+ "y": 29
+ },
+ "id": 10,
+ "maxDataPoints": 500,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "list",
+ "placement": "bottom"
+ },
+ "tooltip": {
+ "mode": "single"
+ }
+ },
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "sum(rate(nginx_http_requests_total{} [$__rate_interval]))",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "Requests",
+ "refId": "A"
+ },
+ {
+ "exemplar": true,
+ "expr": "pg_stat_database_blks_hit{instance=\"127.0.0.1:9187\",
server=\"db.perf.taler:5432\", datname!~\"postgres|template.*|\"} /
(pg_stat_database_blks_read{instance=\"127.0.0.1:9187\",
server=\"db.perf.taler:5432\", datname!~\"postgres|template.*|\"} +
pg_stat_database_blks_hit{instance=\"127.0.0.1:9187\",
server=\"db.perf.taler:5432\", datname!~\"postgres|template.*|\"})",
+ "hide": false,
+ "interval": "",
+ "legendFormat": "Cache Hit Rate",
+ "refId": "C"
+ }
+ ],
+ "title": "Requests vs Cache Hit Rate",
+ "type": "timeseries"
+ },
+ {
+ "datasource": {
+ "type": "datasource",
+ "uid": "-- Mixed --"
+ },
+ "description": "",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "axisLabel": "",
+ "axisPlacement": "auto",
+ "barAlignment": 0,
+ "drawStyle": "line",
+ "fillOpacity": 0,
+ "gradientMode": "none",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ },
+ "lineInterpolation": "linear",
+ "lineWidth": 2,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "auto",
+ "spanNulls": false,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
+ }
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": [
+ {
+ "matcher": {
+ "id": "byName",
+ "options": "Slow Queries"
+ },
+ "properties": [
+ {
+ "id": "custom.axisPlacement",
+ "value": "right"
+ },
+ {
+ "id": "custom.axisLabel",
+ "value": "Slow Queries / Second"
+ },
+ {
+ "id": "unit",
+ "value": "none"
+ },
+ {
+ "id": "custom.lineStyle",
+ "value": {
+ "fill": "solid"
+ }
+ },
+ {
+ "id": "custom.fillOpacity",
+ "value": 15
+ }
+ ]
+ },
+ {
+ "matcher": {
+ "id": "byName",
+ "options": "Requests"
+ },
+ "properties": [
+ {
+ "id": "custom.axisLabel",
+ "value": "Requests / Second"
+ }
+ ]
+ }
+ ]
+ },
+ "gridPos": {
+ "h": 9,
+ "w": 12,
+ "x": 12,
+ "y": 29
+ },
+ "id": 11,
+ "maxDataPoints": 500,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "list",
+ "placement": "bottom"
+ },
+ "tooltip": {
+ "mode": "single"
+ }
+ },
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "exemplar": true,
+ "expr": "sum(rate(nginx_http_requests_total{} [$__rate_interval]))",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "Requests",
+ "refId": "A"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "expr": "rate({app=\"taler-database\"} |~ \"duration:\" [1m])",
+ "hide": false,
+ "legendFormat": "Slow Queries",
+ "refId": "C"
+ }
+ ],
+ "title": "Requests vs Slow Queries",
+ "type": "timeseries"
+ },
+ {
+ "description": "Serialization Errors per Second and Request-Type",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "axisLabel": "",
+ "axisPlacement": "auto",
+ "barAlignment": 0,
+ "drawStyle": "line",
+ "fillOpacity": 0,
+ "gradientMode": "opacity",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ },
+ "lineInterpolation": "linear",
+ "lineWidth": 2,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "auto",
+ "spanNulls": false,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
+ }
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ },
+ "unit": "none"
+ },
+ "overrides": [
+ {
+ "matcher": {
+ "id": "byName",
+ "options": "Serialization Errors"
+ },
+ "properties": [
+ {
+ "id": "custom.axisPlacement",
+ "value": "right"
+ },
+ {
+ "id": "custom.axisLabel",
+ "value": "Serialization Errors / Second"
+ },
+ {
+ "id": "unit"
+ }
+ ]
+ },
+ {
+ "matcher": {
+ "id": "byName",
+ "options": "Requests"
+ },
+ "properties": [
+ {
+ "id": "custom.axisLabel",
+ "value": "Requests / Second"
+ }
+ ]
+ }
+ ]
+ },
+ "gridPos": {
+ "h": 9,
+ "w": 12,
+ "x": 0,
+ "y": 38
+ },
+ "id": 18,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "list",
+ "placement": "bottom"
+ },
+ "tooltip": {
+ "mode": "single"
+ }
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "sum(rate(nginx_http_requests_total{} [$__rate_interval]))",
+ "hide": false,
+ "interval": "",
+ "legendFormat": "Requests",
+ "refId": "B"
+ },
+ {
+ "exemplar": true,
+ "expr": "sum (rate(taler_exchange_serialization_failures{}[1m]))",
+ "instant": false,
+ "interval": "",
+ "legendFormat": "Serialization Errors",
+ "refId": "A"
+ }
+ ],
+ "title": "Requests vs Serialization Errors",
+ "transformations": [],
+ "type": "timeseries"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "axisLabel": "",
+ "axisPlacement": "auto",
+ "barAlignment": 0,
+ "drawStyle": "line",
+ "fillOpacity": 0,
+ "gradientMode": "none",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ },
+ "lineInterpolation": "linear",
+ "lineWidth": 2,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "auto",
+ "spanNulls": false,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
+ }
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ },
+ "unit": "s"
+ },
+ "overrides": [
+ {
+ "matcher": {
+ "id": "byName",
+ "options": "Slow Queries"
+ },
+ "properties": [
+ {
+ "id": "custom.axisPlacement",
+ "value": "right"
+ },
+ {
+ "id": "custom.axisLabel",
+ "value": "Slow Queries / Second"
+ },
+ {
+ "id": "unit",
+ "value": "none"
+ },
+ {
+ "id": "custom.drawStyle",
+ "value": "line"
+ },
+ {
+ "id": "custom.lineStyle"
+ },
+ {
+ "id": "custom.fillOpacity",
+ "value": 15
+ }
+ ]
+ }
+ ]
+ },
+ "gridPos": {
+ "h": 9,
+ "w": 12,
+ "x": 12,
+ "y": 38
+ },
+ "id": 16,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "list",
+ "placement": "bottom"
+ },
+ "tooltip": {
+ "mode": "single"
+ }
+ },
+ "targets": [
+ {
+ "expr": "avg_over_time(({app=\"taler_proxy\"} | logfmt |
__error__!=\"LogfmtParserErr\" | unwrap rt ) [1m]) by (app)",
+ "legendFormat": "Average Response time ",
+ "refId": "A",
+ "resolution": 5
+ },
+ {
+ "expr": "rate({app=\"taler-database\"} |~ \"duration:\" [1m])",
+ "hide": false,
+ "legendFormat": "Slow Queries",
+ "refId": "C",
+ "resolution": 1
+ }
+ ],
+ "title": "Proxy Response Time vs Slow Queries",
+ "type": "timeseries"
+ }
+ ],
+ "refresh": "1m",
+ "schemaVersion": 33,
+ "style": "dark",
+ "tags": [
+ "taler",
+ "performance"
+ ],
+ "templating": {
+ "list": []
+ },
+ "time": {
+ "from": "now-1h",
+ "to": "now"
+ },
+ "timepicker": {},
+ "timezone": "",
+ "title": "Load Statistics",
+ "uid": "rkyhDAt7z",
+ "version": 48,
+ "weekStart": ""
+}
\ No newline at end of file
diff --git a/additional/grafana/logs.json b/additional/grafana/logs.json
new file mode 100644
index 0000000..012b501
--- /dev/null
+++ b/additional/grafana/logs.json
@@ -0,0 +1,202 @@
+{
+ "__inputs": [
+ {
+ "name": "DS_LOKI",
+ "label": "Loki",
+ "description": "",
+ "type": "datasource",
+ "pluginId": "loki",
+ "pluginName": "Loki"
+ }
+ ],
+ "__requires": [
+ {
+ "type": "grafana",
+ "id": "grafana",
+ "name": "Grafana",
+ "version": "8.2.5"
+ },
+ {
+ "type": "panel",
+ "id": "logs",
+ "name": "Logs",
+ "version": ""
+ },
+ {
+ "type": "datasource",
+ "id": "loki",
+ "name": "Loki",
+ "version": "1.0.0"
+ }
+ ],
+ "annotations": {
+ "list": [
+ {
+ "builtIn": 1,
+ "datasource": "-- Grafana --",
+ "enable": true,
+ "hide": true,
+ "iconColor": "rgba(0, 211, 255, 1)",
+ "name": "Annotations & Alerts",
+ "target": {
+ "limit": 100,
+ "matchAny": false,
+ "tags": [],
+ "type": "dashboard"
+ },
+ "type": "dashboard"
+ }
+ ]
+ },
+ "editable": true,
+ "fiscalYearStartMonth": 0,
+ "gnetId": null,
+ "graphTooltip": 0,
+ "id": null,
+ "iteration": 1639255473609,
+ "links": [],
+ "liveNow": false,
+ "panels": [
+ {
+ "datasource": "${DS_LOKI}",
+ "description": "",
+ "gridPos": {
+ "h": 13,
+ "w": 21,
+ "x": 1,
+ "y": 0
+ },
+ "id": 2,
+ "options": {
+ "dedupStrategy": "none",
+ "enableLogDetails": true,
+ "prettifyLogMessage": false,
+ "showCommonLabels": false,
+ "showLabels": false,
+ "showTime": false,
+ "sortOrder": "Descending",
+ "wrapLogMessage": false
+ },
+ "repeat": "app",
+ "repeatDirection": "v",
+ "targets": [
+ {
+ "expr": "{app=\"$app\"} |~ \"(?i)$filter\" !~
\"(?i)${invertfilter:quote}|asdasdzzzzz\" |~ \"$predef_filter\"",
+ "refId": "A"
+ }
+ ],
+ "title": "Logs: $app ",
+ "type": "logs"
+ }
+ ],
+ "refresh": "",
+ "schemaVersion": 32,
+ "style": "dark",
+ "tags": [
+ "performance",
+ "taler"
+ ],
+ "templating": {
+ "list": [
+ {
+ "allValue": null,
+ "current": {},
+ "datasource": "${DS_LOKI}",
+ "definition": "label_values({}, app)",
+ "description": null,
+ "error": null,
+ "hide": 0,
+ "includeAll": true,
+ "label": "App",
+ "multi": true,
+ "name": "app",
+ "options": [],
+ "query": "label_values({}, app)",
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "sort": 0,
+ "type": "query"
+ },
+ {
+ "current": {
+ "selected": false,
+ "text": "",
+ "value": ""
+ },
+ "description": null,
+ "error": null,
+ "hide": 0,
+ "label": "filter",
+ "name": "filter",
+ "options": [
+ {
+ "selected": true,
+ "text": "",
+ "value": ""
+ }
+ ],
+ "query": "",
+ "skipUrlSync": false,
+ "type": "textbox"
+ },
+ {
+ "current": {
+ "selected": false,
+ "text": "",
+ "value": ""
+ },
+ "description": null,
+ "error": null,
+ "hide": 0,
+ "label": "invertfilter",
+ "name": "invertfilter",
+ "options": [
+ {
+ "selected": true,
+ "text": "",
+ "value": ""
+ }
+ ],
+ "query": "",
+ "skipUrlSync": false,
+ "type": "textbox"
+ },
+ {
+ "allValue": null,
+ "current": {
+ "selected": true,
+ "text": [],
+ "value": []
+ },
+ "description": "Some predefined filters to apply to logs",
+ "error": null,
+ "hide": 0,
+ "includeAll": false,
+ "label": "Predefined Filter",
+ "multi": true,
+ "name": "predef_filter",
+ "options": [
+ {
+ "selected": true,
+ "text": "slow query analyzation",
+ "value": "#011"
+ }
+ ],
+ "query": "slow query analyzation : #011",
+ "queryValue": "",
+ "skipUrlSync": false,
+ "type": "custom"
+ }
+ ]
+ },
+ "time": {
+ "from": "now-1h",
+ "to": "now"
+ },
+ "timepicker": {},
+ "timezone": "",
+ "title": "Logs",
+ "uid": "eCk_1vdnk",
+ "version": 23
+}
\ No newline at end of file
diff --git a/additional/grafana/request-statistics.json
b/additional/grafana/request-statistics.json
new file mode 100644
index 0000000..4eb95cc
--- /dev/null
+++ b/additional/grafana/request-statistics.json
@@ -0,0 +1,1386 @@
+{
+ "__inputs": [
+ {
+ "name": "DS_LOKI",
+ "label": "Loki",
+ "description": "",
+ "type": "datasource",
+ "pluginId": "loki",
+ "pluginName": "Loki"
+ }
+ ],
+ "__elements": [],
+ "__requires": [
+ {
+ "type": "grafana",
+ "id": "grafana",
+ "name": "Grafana",
+ "version": "8.3.2"
+ },
+ {
+ "type": "datasource",
+ "id": "loki",
+ "name": "Loki",
+ "version": "1.0.0"
+ },
+ {
+ "type": "panel",
+ "id": "stat",
+ "name": "Stat",
+ "version": ""
+ },
+ {
+ "type": "panel",
+ "id": "timeseries",
+ "name": "Time series",
+ "version": ""
+ }
+ ],
+ "annotations": {
+ "list": [
+ {
+ "builtIn": 1,
+ "datasource": "-- Grafana --",
+ "enable": true,
+ "hide": true,
+ "iconColor": "rgba(0, 211, 255, 1)",
+ "name": "Annotations & Alerts",
+ "target": {
+ "limit": 100,
+ "matchAny": false,
+ "tags": [],
+ "type": "dashboard"
+ },
+ "type": "dashboard"
+ }
+ ]
+ },
+ "description": "",
+ "editable": true,
+ "fiscalYearStartMonth": 0,
+ "graphTooltip": 1,
+ "id": null,
+ "iteration": 1639471729428,
+ "links": [],
+ "liveNow": false,
+ "panels": [
+ {
+ "collapsed": false,
+ "gridPos": {
+ "h": 1,
+ "w": 24,
+ "x": 0,
+ "y": 0
+ },
+ "id": 12,
+ "panels": [],
+ "title": "Network",
+ "type": "row"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "round-trip-time wallet to proxy",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "s"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 4,
+ "w": 3,
+ "x": 7,
+ "y": 1
+ },
+ "id": 162,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "expr": "avg_over_time({app=\"taler-network\"} | logfmt |
src=\"wallet\" | unwrap duration(time) [$__range])",
+ "instant": true,
+ "range": false,
+ "refId": "A"
+ }
+ ],
+ "title": "RTT Wallet-Proxy",
+ "type": "stat"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "round-trip-time proxy to exchange",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "s"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 4,
+ "w": 3,
+ "x": 10,
+ "y": 1
+ },
+ "id": 176,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "expr": "avg_over_time({app=\"taler-network\"} | logfmt |
src=\"proxy\" | unwrap duration(time) [$__range])",
+ "instant": true,
+ "range": false,
+ "refId": "A"
+ }
+ ],
+ "title": "RTT Proxy-Exchange",
+ "type": "stat"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "round-trip-time exchange to database",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "s"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 4,
+ "w": 3,
+ "x": 13,
+ "y": 1
+ },
+ "id": 178,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "expr": "avg_over_time({app=\"taler-network\"} | logfmt |
src=\"exchange\" | unwrap duration(time) [$__range])",
+ "instant": true,
+ "range": false,
+ "refId": "A"
+ }
+ ],
+ "title": "RTT Exchange-Database",
+ "type": "stat"
+ },
+ {
+ "collapsed": false,
+ "gridPos": {
+ "h": 1,
+ "w": 24,
+ "x": 0,
+ "y": 5
+ },
+ "id": 230,
+ "panels": [],
+ "title": "Related Panels",
+ "type": "row"
+ },
+ {
+ "description": "Serialization Errors per Second and Request-Type",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "axisLabel": "Serialization Errors / Second",
+ "axisPlacement": "auto",
+ "barAlignment": 0,
+ "drawStyle": "line",
+ "fillOpacity": 0,
+ "gradientMode": "none",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ },
+ "lineInterpolation": "linear",
+ "lineWidth": 1,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "auto",
+ "spanNulls": false,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
+ }
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ },
+ "unit": "none"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 7,
+ "w": 12,
+ "x": 0,
+ "y": 6
+ },
+ "id": 280,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "list",
+ "placement": "bottom"
+ },
+ "tooltip": {
+ "mode": "single"
+ }
+ },
+ "pluginVersion": "8.2.5",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "sum by (type)
(rate(taler_exchange_serialization_failures{}[1m]))",
+ "instant": false,
+ "interval": "",
+ "legendFormat": "{{type}}",
+ "refId": "A"
+ }
+ ],
+ "title": "Serialization Errors",
+ "transformations": [],
+ "type": "timeseries"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "custom": {
+ "axisLabel": "Slow Queries / Second",
+ "axisPlacement": "auto",
+ "barAlignment": 0,
+ "drawStyle": "line",
+ "fillOpacity": 19,
+ "gradientMode": "hue",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ },
+ "lineInterpolation": "smooth",
+ "lineWidth": 1,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "auto",
+ "spanNulls": false,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
+ }
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "#EAB839",
+ "value": 5
+ },
+ {
+ "color": "red",
+ "value": 10
+ }
+ ]
+ },
+ "unit": "SQ/s"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 7,
+ "w": 12,
+ "x": 12,
+ "y": 6
+ },
+ "id": 228,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "hidden",
+ "placement": "bottom"
+ },
+ "tooltip": {
+ "mode": "single"
+ }
+ },
+ "targets": [
+ {
+ "expr": "rate({app=\"taler-database\"} |~ \"duration:\" [1m])",
+ "refId": "A"
+ }
+ ],
+ "title": "Slow Queries",
+ "type": "timeseries"
+ },
+ {
+ "collapsed": false,
+ "gridPos": {
+ "h": 1,
+ "w": 24,
+ "x": 0,
+ "y": 13
+ },
+ "id": 20,
+ "panels": [],
+ "repeat": "endpoint",
+ "title": "Request Times ($endpoint)",
+ "type": "row"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "Minimal response time for successful (200) requests to
$endpoint",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "s"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 6,
+ "w": 3,
+ "x": 0,
+ "y": 14
+ },
+ "id": 25,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "none",
+ "justifyMode": "center",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "expr": "min_over_time(({app=\"taler_proxy\"} |~ \"s=200\" | logfmt
|~ \"${endpoint:dublequote} \" | unwrap rt) [$interval]) by (app)",
+ "instant": true,
+ "legendFormat": "",
+ "range": false,
+ "refId": "A",
+ "resolution": 2
+ }
+ ],
+ "title": "Minimal [$interval]",
+ "type": "stat"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "Average request time of successful (200) requests to
$endpoint",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "continuous-GrYlRd"
+ },
+ "mappings": [],
+ "max": 5,
+ "min": 0,
+ "thresholds": {
+ "mode": "percentage",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "#EAB839",
+ "value": 60
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "s"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 6,
+ "w": 3,
+ "x": 3,
+ "y": 14
+ },
+ "id": 18,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "expr": "avg_over_time(({app=\"taler_proxy\"} |~ \"s=200\" | logfmt
|~ \"${endpoint:dublequote} \" | unwrap rt) [$interval]) by (app)",
+ "instant": true,
+ "range": false,
+ "refId": "A",
+ "resolution": 2
+ }
+ ],
+ "title": "Average [$interval]",
+ "type": "stat"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "Maximal response time for successful (200) requests to
$endpoint",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "s"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 6,
+ "w": 3,
+ "x": 6,
+ "y": 14
+ },
+ "id": 37,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "none",
+ "justifyMode": "center",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "expr": "max_over_time({app=\"taler_proxy\"} |~ \"s=200\" | logfmt
|~ \"${endpoint:dublequote} \" | unwrap rt [$interval]) by (app)",
+ "hide": false,
+ "instant": true,
+ "legendFormat": "",
+ "range": false,
+ "refId": "B",
+ "resolution": 2
+ }
+ ],
+ "title": "Maximal [$interval]",
+ "type": "stat"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "Number of successful requests (=200) to $endpoint",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "fixedColor": "semi-dark-orange",
+ "mode": "thresholds"
+ },
+ "mappings": [],
+ "noValue": "0",
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 3,
+ "x": 9,
+ "y": 14
+ },
+ "id": 148,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "none",
+ "justifyMode": "center",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "expr": "sum by (app) (count_over_time({app=\"taler_proxy\"} |~
\"s=200\" |~ \"${endpoint:dublequote} \"[$interval]))",
+ "refId": "A"
+ }
+ ],
+ "title": "Successful Requests [$interval]",
+ "type": "stat"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "Average response time for requests to $endpoint",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "axisLabel": "",
+ "axisPlacement": "auto",
+ "barAlignment": 0,
+ "drawStyle": "line",
+ "fillOpacity": 86,
+ "gradientMode": "opacity",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ },
+ "lineInterpolation": "smooth",
+ "lineWidth": 1,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "never",
+ "spanNulls": false,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
+ }
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "s"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 6,
+ "w": 12,
+ "x": 12,
+ "y": 14
+ },
+ "id": 14,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "list",
+ "placement": "bottom"
+ },
+ "tooltip": {
+ "mode": "single"
+ }
+ },
+ "pluginVersion": "8.2.1",
+ "targets": [
+ {
+ "expr": "avg_over_time(({app=\"taler_proxy\"} | logfmt | s=200 |~
\"${endpoint:dublequote} \" | unwrap rt) [$interval]) by (app)",
+ "instant": false,
+ "legendFormat": "200",
+ "range": true,
+ "refId": "A",
+ "resolution": 10
+ }
+ ],
+ "title": "Request Time AVG [$interval]",
+ "type": "timeseries"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "Number of failed requests (!=200) to $endpoint",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [],
+ "noValue": "0",
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "orange",
+ "value": 1
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 3,
+ "x": 9,
+ "y": 17
+ },
+ "id": 31,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "none",
+ "justifyMode": "center",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "expr": "sum by (app) (count_over_time({app=\"taler_proxy\"} !~
\"s=200\" |~ \"${endpoint:dublequote} \" [$interval]))",
+ "refId": "A"
+ }
+ ],
+ "title": "Failed Requests [$interval]",
+ "type": "stat"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "Median response time for successful (200) requests to
$endpoint",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "s"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 6,
+ "w": 3,
+ "x": 0,
+ "y": 20
+ },
+ "id": 41,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "none",
+ "justifyMode": "center",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "expr": "quantile_over_time(0.50, ({app=\"taler_proxy\"} |~
\"s=200\" | logfmt |~ \"${endpoint:dublequote} \" | unwrap rt) [$interval]) by
(app)",
+ "hide": false,
+ "instant": true,
+ "legendFormat": "",
+ "range": false,
+ "refId": "F",
+ "resolution": 2
+ }
+ ],
+ "title": "Median [$interval]",
+ "type": "stat"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "90% of requests to $endpoint complete successfully (200)
below this time",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "s"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 6,
+ "w": 3,
+ "x": 3,
+ "y": 20
+ },
+ "id": 40,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "none",
+ "justifyMode": "center",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "expr": "quantile_over_time(0.90, ({app=\"taler_proxy\"} |~
\"s=200\" | logfmt |~ \"${endpoint:dublequote} \" | unwrap rt) [$interval]) by
(app)",
+ "hide": false,
+ "instant": true,
+ "legendFormat": "90% Quantile ",
+ "range": false,
+ "refId": "E",
+ "resolution": 2
+ }
+ ],
+ "title": "90th Percentile [$interval]",
+ "type": "stat"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "99% of requests to $endpoint complete successfully (200)
below this time",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "s"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 6,
+ "w": 3,
+ "x": 6,
+ "y": 20
+ },
+ "id": 39,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "none",
+ "justifyMode": "center",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "expr": "quantile_over_time(0.99, ({app=\"taler_proxy\"} |~
\"s=200\" | logfmt |~ \"${endpoint:dublequote} \" | unwrap rt) [$interval]) by
(app)",
+ "hide": false,
+ "instant": true,
+ "legendFormat": "",
+ "range": false,
+ "refId": "D",
+ "resolution": 2
+ }
+ ],
+ "title": "99th Percentile [$interval]",
+ "type": "stat"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "Standard deviation of response times for successful
(200) requests to $endpoint",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "s"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 6,
+ "w": 3,
+ "x": 9,
+ "y": 20
+ },
+ "id": 38,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "none",
+ "justifyMode": "center",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "expr": "stddev_over_time(({app=\"taler_proxy\"} |~ \"s=200\" |
logfmt |~ \"${endpoint:dublequote} \" | unwrap rt) [$interval]) by (app)",
+ "hide": false,
+ "instant": true,
+ "legendFormat": "",
+ "range": false,
+ "refId": "C",
+ "resolution": 2
+ }
+ ],
+ "title": "Standard Deviation [$interval]",
+ "type": "stat"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "axisLabel": "",
+ "axisPlacement": "auto",
+ "barAlignment": 0,
+ "drawStyle": "line",
+ "fillOpacity": 0,
+ "gradientMode": "none",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ },
+ "lineInterpolation": "linear",
+ "lineWidth": 1,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "never",
+ "spanNulls": false,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
+ }
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 6,
+ "w": 12,
+ "x": 12,
+ "y": 20
+ },
+ "id": 107,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "list",
+ "placement": "bottom"
+ },
+ "tooltip": {
+ "mode": "single"
+ }
+ },
+ "targets": [
+ {
+ "expr": "sum by (s) (count_over_time(({app=\"taler_proxy\"} | logfmt
|~ \"${endpoint:dublequote} \") [$interval]))",
+ "legendFormat": "{{s}}",
+ "refId": "A",
+ "resolution": 10
+ }
+ ],
+ "title": "Number of Requests [$interval]",
+ "type": "timeseries"
+ }
+ ],
+ "refresh": "1m",
+ "schemaVersion": 33,
+ "style": "dark",
+ "tags": [
+ "performance",
+ "taler"
+ ],
+ "templating": {
+ "list": [
+ {
+ "auto": false,
+ "auto_count": 30,
+ "auto_min": "10s",
+ "current": {
+ "selected": true,
+ "text": "1m",
+ "value": "1m"
+ },
+ "hide": 0,
+ "label": "interval",
+ "name": "interval",
+ "options": [
+ {
+ "selected": false,
+ "text": "30s",
+ "value": "30s"
+ },
+ {
+ "selected": true,
+ "text": "1m",
+ "value": "1m"
+ },
+ {
+ "selected": false,
+ "text": "2m",
+ "value": "2m"
+ },
+ {
+ "selected": false,
+ "text": "3m",
+ "value": "3m"
+ },
+ {
+ "selected": false,
+ "text": "5m",
+ "value": "5m"
+ },
+ {
+ "selected": false,
+ "text": "10m",
+ "value": "10m"
+ },
+ {
+ "selected": false,
+ "text": "15m",
+ "value": "15m"
+ },
+ {
+ "selected": false,
+ "text": "20m",
+ "value": "20m"
+ },
+ {
+ "selected": false,
+ "text": "30m",
+ "value": "30m"
+ },
+ {
+ "selected": false,
+ "text": "1h",
+ "value": "1h"
+ }
+ ],
+ "query": "30s,1m,2m,3m,5m,10m,15m,20m,30m,1h",
+ "queryValue": "",
+ "refresh": 2,
+ "skipUrlSync": false,
+ "type": "interval"
+ },
+ {
+ "current": {
+ "selected": false,
+ "text": "All",
+ "value": "$__all"
+ },
+ "hide": 0,
+ "includeAll": true,
+ "label": "endpoint",
+ "multi": false,
+ "name": "endpoint",
+ "options": [
+ {
+ "selected": true,
+ "text": "All",
+ "value": "$__all"
+ },
+ {
+ "selected": false,
+ "text": "/withdraw",
+ "value": "/withdraw"
+ },
+ {
+ "selected": false,
+ "text": "/reveal",
+ "value": "/reveal"
+ },
+ {
+ "selected": false,
+ "text": "/melt",
+ "value": "/melt"
+ },
+ {
+ "selected": false,
+ "text": "/deposit",
+ "value": "/deposit"
+ },
+ {
+ "selected": false,
+ "text": "/reserves/[A-Z0-9]+",
+ "value": "/reserves/[A-Z0-9]+"
+ }
+ ],
+ "query": "/withdraw,/reveal,/melt,/deposit,/reserves/[A-Z0-9]+",
+ "queryValue": "",
+ "skipUrlSync": false,
+ "type": "custom"
+ }
+ ]
+ },
+ "time": {
+ "from": "now-1h",
+ "to": "now"
+ },
+ "timepicker": {
+ "hidden": false,
+ "refresh_intervals": [
+ "5s",
+ "10s",
+ "30s",
+ "1m",
+ "5m",
+ "15m",
+ "30m",
+ "1h",
+ "2h",
+ "1d"
+ ],
+ "time_options": [
+ "5m",
+ "15m",
+ "1h",
+ "6h",
+ "12h",
+ "24h",
+ "2d",
+ "7d",
+ "30d"
+ ],
+ "type": "timepicker"
+ },
+ "timezone": "browser",
+ "title": "Request Statistics",
+ "uid": "WcfSXqDnk",
+ "version": 131,
+ "weekStart": ""
+}
\ No newline at end of file
diff --git a/additional/grafana/taler-performance.json
b/additional/grafana/taler-performance.json
new file mode 100644
index 0000000..282dcca
--- /dev/null
+++ b/additional/grafana/taler-performance.json
@@ -0,0 +1,176 @@
+{
+ "__inputs": [],
+ "__requires": [
+ {
+ "type": "panel",
+ "id": "dashlist",
+ "name": "Dashboard list",
+ "version": ""
+ },
+ {
+ "type": "grafana",
+ "id": "grafana",
+ "name": "Grafana",
+ "version": "8.2.5"
+ },
+ {
+ "type": "panel",
+ "id": "text",
+ "name": "Text",
+ "version": ""
+ }
+ ],
+ "annotations": {
+ "list": [
+ {
+ "builtIn": 1,
+ "datasource": "-- Grafana --",
+ "enable": true,
+ "hide": true,
+ "iconColor": "rgba(0, 211, 255, 1)",
+ "name": "Annotations & Alerts",
+ "target": {
+ "limit": 100,
+ "matchAny": false,
+ "tags": [],
+ "type": "dashboard"
+ },
+ "type": "dashboard"
+ }
+ ]
+ },
+ "editable": false,
+ "fiscalYearStartMonth": 0,
+ "gnetId": null,
+ "graphTooltip": 0,
+ "id": null,
+ "links": [
+ {
+ "asDropdown": false,
+ "icon": "dashboard",
+ "includeVars": false,
+ "keepTime": false,
+ "tags": [],
+ "targetBlank": false,
+ "title": "Experiment Snapshots",
+ "tooltip": "Snapshots",
+ "type": "link",
+ "url": "https://147.87.255.221:3000/dashboard/snapshots";
+ }
+ ],
+ "liveNow": false,
+ "panels": [
+ {
+ "datasource": null,
+ "description": "",
+ "gridPos": {
+ "h": 5,
+ "w": 4,
+ "x": 5,
+ "y": 0
+ },
+ "id": 8,
+ "options": {
+ "content": "<center>\n <img
src=\"https://taler.net/images/logo-2021.svg\"; width=\"200\">\n</center>\n
",
+ "mode": "markdown"
+ },
+ "pluginVersion": "8.2.5",
+ "title": " ",
+ "transparent": true,
+ "type": "text"
+ },
+ {
+ "datasource": null,
+ "description": "",
+ "gridPos": {
+ "h": 7,
+ "w": 5,
+ "x": 9,
+ "y": 0
+ },
+ "id": 6,
+ "options": {
+ "content": "<center>\n<h1>Taler Performance Tests</h1>\n\nGoal: Reach
100'000 Transactions per second [TPS]\n\nTestbed:
[Grid500](https://www.grid5000.fr/w/Grid5000:Home) \n\nRepository: [Taler
Git](https://git.taler.net/grid5k.git/)\n</center>",
+ "mode": "markdown"
+ },
+ "pluginVersion": "8.2.5",
+ "title": " ",
+ "transparent": true,
+ "type": "text"
+ },
+ {
+ "datasource": null,
+ "description": "",
+ "gridPos": {
+ "h": 6,
+ "w": 4,
+ "x": 14,
+ "y": 0
+ },
+ "id": 9,
+ "options": {
+ "content": "<center>\n <img
src=\"https://www.grid5000.fr/mediawiki/resources/assets/logo.png?2c43b\";
width=\"200\">\n</center>\n ",
+ "mode": "markdown"
+ },
+ "pluginVersion": "8.2.5",
+ "title": " ",
+ "transparent": true,
+ "type": "text"
+ },
+ {
+ "datasource": null,
+ "gridPos": {
+ "h": 13,
+ "w": 13,
+ "x": 5,
+ "y": 7
+ },
+ "id": 4,
+ "options": {
+ "folderId": 0,
+ "maxItems": 10,
+ "query": "",
+ "showHeadings": false,
+ "showRecentlyViewed": false,
+ "showSearch": true,
+ "showStarred": false,
+ "tags": [
+ "performance"
+ ]
+ },
+ "pluginVersion": "8.2.5",
+ "title": "Dashboards",
+ "transparent": true,
+ "type": "dashlist"
+ }
+ ],
+ "schemaVersion": 32,
+ "style": "dark",
+ "tags": [],
+ "templating": {
+ "list": []
+ },
+ "time": {
+ "from": "now-5m",
+ "to": "now"
+ },
+ "timepicker": {
+ "hidden": true,
+ "refresh_intervals": [
+ "5s",
+ "10s",
+ "30s",
+ "1m",
+ "5m",
+ "15m",
+ "30m",
+ "1h",
+ "2h",
+ "1d"
+ ]
+ },
+ "timezone": "",
+ "title": "Taler Performance",
+ "uid": "YwfzrVd7z",
+ "version": 22
+}
\ No newline at end of file
diff --git a/additional/grafana/transactions.json
b/additional/grafana/transactions.json
new file mode 100644
index 0000000..7b7d4f8
--- /dev/null
+++ b/additional/grafana/transactions.json
@@ -0,0 +1,1810 @@
+{
+ "__inputs": [
+ {
+ "name": "DS_LOKI",
+ "label": "Loki",
+ "description": "",
+ "type": "datasource",
+ "pluginId": "loki",
+ "pluginName": "Loki"
+ }
+ ],
+ "__elements": [],
+ "__requires": [
+ {
+ "type": "panel",
+ "id": "gauge",
+ "name": "Gauge",
+ "version": ""
+ },
+ {
+ "type": "grafana",
+ "id": "grafana",
+ "name": "Grafana",
+ "version": "8.3.2"
+ },
+ {
+ "type": "datasource",
+ "id": "loki",
+ "name": "Loki",
+ "version": "1.0.0"
+ },
+ {
+ "type": "panel",
+ "id": "stat",
+ "name": "Stat",
+ "version": ""
+ },
+ {
+ "type": "panel",
+ "id": "timeseries",
+ "name": "Time series",
+ "version": ""
+ }
+ ],
+ "annotations": {
+ "list": [
+ {
+ "builtIn": 1,
+ "datasource": "-- Grafana --",
+ "enable": true,
+ "hide": true,
+ "iconColor": "rgba(0, 211, 255, 1)",
+ "name": "Annotations & Alerts",
+ "target": {
+ "limit": 100,
+ "matchAny": false,
+ "tags": [],
+ "type": "dashboard"
+ },
+ "type": "dashboard"
+ }
+ ]
+ },
+ "description": "",
+ "editable": true,
+ "fiscalYearStartMonth": 0,
+ "graphTooltip": 1,
+ "id": null,
+ "links": [],
+ "liveNow": false,
+ "panels": [
+ {
+ "collapsed": false,
+ "gridPos": {
+ "h": 1,
+ "w": 24,
+ "x": 0,
+ "y": 0
+ },
+ "id": 28,
+ "panels": [],
+ "title": "General Information",
+ "type": "row"
+ },
+ {
+ "description": "Number of wallets running",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [],
+ "noValue": "0",
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "semi-dark-red",
+ "value": null
+ },
+ {
+ "color": "green",
+ "value": 1
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 4,
+ "w": 3,
+ "x": 0,
+ "y": 1
+ },
+ "id": 10,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr":
"sum(node_systemd_unit_state{name=~\"taler-wallet@(.*).service\",
state=\"active\"})",
+ "instant": true,
+ "interval": "",
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "# Wallet",
+ "type": "stat"
+ },
+ {
+ "description": "Number of exchange processes running",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 4,
+ "w": 3,
+ "x": 6,
+ "y": 1
+ },
+ "id": 8,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr":
"sum(node_systemd_unit_state{name=~\"taler-exchange-http(.*).service\",
state=\"active\"})",
+ "instant": true,
+ "interval": "",
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "# Exchange",
+ "type": "stat"
+ },
+ {
+ "description": "Number of wirewatch processes running",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 4,
+ "w": 3,
+ "x": 9,
+ "y": 1
+ },
+ "id": 41,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr":
"sum(node_systemd_unit_state{name=~\"taler-exchange-wirewatch(.*).service\",
state=\"active\"})",
+ "instant": true,
+ "interval": "",
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "# Wirewatch",
+ "type": "stat"
+ },
+ {
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [
+ {
+ "options": {
+ "0": {
+ "color": "red",
+ "index": 1,
+ "text": "DOWN"
+ },
+ "1": {
+ "color": "green",
+ "index": 0,
+ "text": "OK"
+ }
+ },
+ "type": "value"
+ }
+ ],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 4,
+ "w": 3,
+ "x": 12,
+ "y": 1
+ },
+ "id": 46,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr":
"node_systemd_unit_state{name=\"taler-exchange-aggregator.service\",
state=\"active\"}",
+ "instant": true,
+ "interval": "",
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "# Aggregator",
+ "type": "stat"
+ },
+ {
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [
+ {
+ "options": {
+ "0": {
+ "color": "red",
+ "index": 1,
+ "text": "DOWN"
+ },
+ "1": {
+ "color": "green",
+ "index": 0,
+ "text": "OK"
+ }
+ },
+ "type": "value"
+ }
+ ],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 2,
+ "w": 2,
+ "x": 15,
+ "y": 1
+ },
+ "id": 50,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr":
"node_systemd_unit_state{name=\"taler-exchange-transfer.service\",
state=\"active\"}",
+ "instant": true,
+ "interval": "",
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "Transfer",
+ "type": "stat"
+ },
+ {
+ "description": "",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [
+ {
+ "options": {
+ "0": {
+ "color": "red",
+ "index": 1,
+ "text": "DOWN"
+ },
+ "1": {
+ "color": "green",
+ "index": 0,
+ "text": "OK"
+ }
+ },
+ "type": "value"
+ }
+ ],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 2,
+ "w": 2,
+ "x": 17,
+ "y": 1
+ },
+ "id": 49,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr":
"node_systemd_unit_state{name=\"taler-exchange-closer.service\",
state=\"active\"}",
+ "instant": true,
+ "interval": "",
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "Closer",
+ "type": "stat"
+ },
+ {
+ "description": "",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [
+ {
+ "options": {
+ "0": {
+ "color": "red",
+ "index": 1,
+ "text": "DOWN"
+ },
+ "1": {
+ "color": "green",
+ "index": 0,
+ "text": "OK"
+ }
+ },
+ "type": "value"
+ }
+ ],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 2,
+ "w": 2,
+ "x": 22,
+ "y": 1
+ },
+ "id": 47,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "node_systemd_unit_state{name=\"loki.service\",
state=\"active\"}",
+ "instant": true,
+ "interval": "",
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "Loki",
+ "type": "stat"
+ },
+ {
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [
+ {
+ "options": {
+ "0": {
+ "color": "red",
+ "index": 1,
+ "text": "DOWN"
+ },
+ "1": {
+ "color": "green",
+ "index": 0,
+ "text": "OK"
+ }
+ },
+ "type": "value"
+ }
+ ],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 2,
+ "w": 2,
+ "x": 15,
+ "y": 3
+ },
+ "id": 52,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr":
"node_systemd_unit_state{name=\"taler-exchange-secmod-rsa.service\",
state=\"active\"}",
+ "instant": true,
+ "interval": "",
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "RSA",
+ "type": "stat"
+ },
+ {
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [
+ {
+ "options": {
+ "0": {
+ "color": "red",
+ "index": 1,
+ "text": "DOWN"
+ },
+ "1": {
+ "color": "green",
+ "index": 0,
+ "text": "OK"
+ }
+ },
+ "type": "value"
+ }
+ ],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 2,
+ "w": 2,
+ "x": 17,
+ "y": 3
+ },
+ "id": 51,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr":
"node_systemd_unit_state{name=\"taler-exchange-secmod-eddsa.service\",
state=\"active\"}",
+ "instant": true,
+ "interval": "",
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "EdDSA",
+ "type": "stat"
+ },
+ {
+ "description": "",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [
+ {
+ "options": {
+ "0": {
+ "color": "red",
+ "index": 1,
+ "text": "DOWN"
+ },
+ "1": {
+ "color": "green",
+ "index": 0,
+ "text": "OK"
+ }
+ },
+ "type": "value"
+ }
+ ],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 2,
+ "w": 2,
+ "x": 22,
+ "y": 3
+ },
+ "id": 48,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "node_systemd_unit_state{name=\"promtail.service\",
state=\"active\"}",
+ "instant": true,
+ "interval": "",
+ "legendFormat": "",
+ "refId": "A"
+ }
+ ],
+ "title": "Promtail",
+ "type": "stat"
+ },
+ {
+ "collapsed": false,
+ "gridPos": {
+ "h": 1,
+ "w": 24,
+ "x": 0,
+ "y": 5
+ },
+ "id": 22,
+ "panels": [],
+ "title": "Total TPS (Sucessful Requests)",
+ "type": "row"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "Total successful (200) requests to /deposit and
/withdraw per second, measured at the nginx proxy and over the whole displayed
time range.",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "continuous-RdYlGr"
+ },
+ "mappings": [],
+ "max": 100000,
+ "min": 0,
+ "noValue": "0",
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 9,
+ "w": 5,
+ "x": 0,
+ "y": 6
+ },
+ "id": 2,
+ "options": {
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "showThresholdLabels": false,
+ "showThresholdMarkers": true,
+ "text": {}
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "expr": "sum by (app) (rate({app=\"taler_proxy\"} |~
\"(/deposit|/withdraw)\" |~ \"s=200\" [2m] ))",
+ "instant": true,
+ "range": false,
+ "refId": "A"
+ }
+ ],
+ "title": "Current TPS ",
+ "transformations": [],
+ "type": "gauge"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "Total successful (200) requests per second to /withdraw
per second, measured at the nginx proxy and over the whole displayed time
range.",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "continuous-RdYlGr"
+ },
+ "mappings": [],
+ "max": 100000,
+ "min": 0,
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 5,
+ "x": 5,
+ "y": 6
+ },
+ "id": 4,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "none",
+ "justifyMode": "center",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "expr": "sum by (app) (rate({app=\"taler_proxy\"} |~ \"/withdraw\"
|~ \"s=200\" [1h] ))",
+ "instant": true,
+ "range": false,
+ "refId": "A"
+ }
+ ],
+ "title": "Withdrawals per Second ",
+ "transformations": [],
+ "type": "stat"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "Total number of successful requests to /deposit and
/withdraw per second, measured at the nginx proxy and over a 5 minute
interval.",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "axisLabel": "TPS",
+ "axisPlacement": "auto",
+ "barAlignment": 0,
+ "drawStyle": "line",
+ "fillOpacity": 47,
+ "gradientMode": "opacity",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ },
+ "lineInterpolation": "smooth",
+ "lineStyle": {
+ "fill": "solid"
+ },
+ "lineWidth": 1,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "never",
+ "spanNulls": true,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
+ }
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ },
+ "unit": "none"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 9,
+ "w": 14,
+ "x": 10,
+ "y": 6
+ },
+ "id": 34,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "hidden",
+ "placement": "bottom"
+ },
+ "tooltip": {
+ "mode": "single"
+ }
+ },
+ "targets": [
+ {
+ "expr": "sum by (app) (count_over_time({app=\"taler_proxy\"} |~
\"s=200\" |~ \"(/withdraw|/deposit)\" | logfmt [2m])) / (2*60)",
+ "legendFormat": "TPS",
+ "refId": "A",
+ "resolution": 10
+ }
+ ],
+ "title": "TPS History",
+ "type": "timeseries"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "Total successful (200) requests to /deposit per second,
measured at the nginx proxy and over the whole displayed time range.",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "continuous-RdYlGr"
+ },
+ "mappings": [],
+ "max": 100000,
+ "min": 0,
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 5,
+ "x": 5,
+ "y": 9
+ },
+ "id": 30,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "none",
+ "justifyMode": "center",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "expr": "sum by (app) (rate({app=\"taler_proxy\"} |~ \"/deposit\" |~
\"s=200\" [1h] ))",
+ "instant": true,
+ "range": false,
+ "refId": "A"
+ }
+ ],
+ "title": "Payments per Second",
+ "transformations": [],
+ "type": "stat"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 5,
+ "x": 5,
+ "y": 12
+ },
+ "id": 54,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "expr": "(sum(count_over_time({app=\"taler_proxy\"} |~ \"/deposit\"
|~ \"s=200\" [1h]))) / (sum (count_over_time({app=\"taler_proxy\"} |~
\"/withdraw\" |~ \"s=200\" [1h])))",
+ "instant": true,
+ "range": false,
+ "refId": "A"
+ }
+ ],
+ "title": "Payments / Withdraw",
+ "type": "stat"
+ },
+ {
+ "collapsed": true,
+ "gridPos": {
+ "h": 1,
+ "w": 24,
+ "x": 0,
+ "y": 15
+ },
+ "id": 20,
+ "panels": [
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "Total successful requests to withdraw a whole
reserve and to /deposit per second, measured at the nginx proxy and over the
whole displayed time range. (Requests to /withdraw are grouped into reserves
they belong to before they get counted.)",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "continuous-RdYlGr"
+ },
+ "mappings": [],
+ "max": 100000,
+ "min": 0,
+ "noValue": "0",
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 9,
+ "w": 5,
+ "x": 0,
+ "y": 16
+ },
+ "id": 14,
+ "options": {
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "showThresholdLabels": false,
+ "showThresholdMarkers": true,
+ "text": {}
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "expr": "count without(uri) (sum by(uri)
(count_over_time({app=\"taler_proxy\"} |~ \"s=200\" |~ \"(/withdraw|/deposit)\"
| logfmt [2m]))) / 120",
+ "instant": true,
+ "range": false,
+ "refId": "A"
+ }
+ ],
+ "title": "Current TPS (Grouped)",
+ "type": "gauge"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "Total successful requests to withdraw all coins of a
reserve per second, measured at the nginx proxy and over the whole displayed
time range. (Requests to /withdraw are grouped into reserves they belong to
before they get counted.)",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "continuous-RdYlGr"
+ },
+ "mappings": [],
+ "max": 100000,
+ "min": 0,
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 5,
+ "x": 5,
+ "y": 16
+ },
+ "id": 16,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "none",
+ "justifyMode": "center",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "expr": "(count without(uri) (sum by(uri)
(count_over_time({app=\"taler_proxy\"} |~ \"/withdraw\" |~ \"s=200\" | logfmt
[1h])))) / 3600",
+ "instant": true,
+ "range": false,
+ "refId": "A"
+ }
+ ],
+ "title": "Withdrawals per Second (Grouped)",
+ "transformations": [],
+ "type": "stat"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "Total successful requests to withdraw a whole
reserve and to /deposit per second, measured at the nginx proxy and over the
whole displayed time range. (Requests to /withdraw are grouped into reserves
they belong to before they get counted.)",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "axisLabel": "TPS",
+ "axisPlacement": "auto",
+ "barAlignment": 0,
+ "drawStyle": "line",
+ "fillOpacity": 47,
+ "gradientMode": "opacity",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ },
+ "lineInterpolation": "smooth",
+ "lineStyle": {
+ "fill": "solid"
+ },
+ "lineWidth": 1,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "never",
+ "spanNulls": true,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
+ }
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ },
+ "unit": "none"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 9,
+ "w": 14,
+ "x": 10,
+ "y": 16
+ },
+ "id": 26,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "hidden",
+ "placement": "bottom"
+ },
+ "tooltip": {
+ "mode": "single"
+ }
+ },
+ "targets": [
+ {
+ "expr": "count without(uri) (sum by(uri)
(count_over_time({app=\"taler_proxy\"} |~ \"s=200\" |~
\"(/withdraw|/deposit)\" | logfmt [2m]))) / (2*60)",
+ "legendFormat": "TPS",
+ "refId": "A",
+ "resolution": 10
+ }
+ ],
+ "title": "TPS History (Grouped)",
+ "type": "timeseries"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "continuous-RdYlGr"
+ },
+ "mappings": [],
+ "max": 100000,
+ "min": 0,
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 5,
+ "x": 5,
+ "y": 19
+ },
+ "id": 6,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "none",
+ "justifyMode": "center",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "expr": "sum by (app) (rate({app=\"taler_proxy\"} |~
\"/deposit\" |~ \"s=200\" [1h] ))",
+ "instant": true,
+ "range": false,
+ "refId": "A"
+ }
+ ],
+ "title": "Payments per Second (Grouped)",
+ "transformations": [],
+ "type": "stat"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 5,
+ "x": 5,
+ "y": 22
+ },
+ "id": 12,
+ "options": {
+ "colorMode": "value",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "text": {},
+ "textMode": "auto"
+ },
+ "pluginVersion": "8.3.2",
+ "targets": [
+ {
+ "expr": "(sum(count_over_time({app=\"taler_proxy\"} |~
\"/deposit\" |~ \"s=200\" [1h]))) / (count without(uri) (sum by(uri)
(count_over_time({app=\"taler_proxy\"} |~ \"/withdraw\" |~ \"s=200\" | logfmt
[1h]))))",
+ "instant": true,
+ "range": false,
+ "refId": "A"
+ }
+ ],
+ "title": "Payments / Reserve (Grouped)",
+ "type": "stat"
+ }
+ ],
+ "title": "Total TPS (Grouped by Reserves)",
+ "type": "row"
+ },
+ {
+ "collapsed": false,
+ "gridPos": {
+ "h": 1,
+ "w": 24,
+ "x": 0,
+ "y": 16
+ },
+ "id": 40,
+ "panels": [],
+ "title": "Other ",
+ "type": "row"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "Average time to withdraw all reserved coins in one
iteration, measured by the wallets.",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "axisLabel": "",
+ "axisPlacement": "auto",
+ "barAlignment": 0,
+ "drawStyle": "line",
+ "fillOpacity": 0,
+ "gradientMode": "none",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ },
+ "lineInterpolation": "linear",
+ "lineWidth": 1,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "auto",
+ "spanNulls": false,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
+ }
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "ms"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 10,
+ "w": 10,
+ "x": 0,
+ "y": 17
+ },
+ "id": 44,
+ "maxDataPoints": 500,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "hidden",
+ "placement": "bottom"
+ },
+ "tooltip": {
+ "mode": "single"
+ }
+ },
+ "targets": [
+ {
+ "expr": "avg_over_time({app=\"taler-wallet-cli\"} |~ \"time=\" |~
\"withdraw\" | logfmt | unwrap time [10m]) by (app)",
+ "instant": false,
+ "legendFormat": "withdraw",
+ "range": true,
+ "refId": "A",
+ "resolution": 5
+ }
+ ],
+ "title": "Average Withdraw Duration (Reservation)",
+ "type": "timeseries"
+ },
+ {
+ "description": "",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "axisLabel": "",
+ "axisPlacement": "auto",
+ "barAlignment": 0,
+ "drawStyle": "line",
+ "fillOpacity": 0,
+ "gradientMode": "none",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ },
+ "lineInterpolation": "linear",
+ "lineWidth": 1,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "auto",
+ "spanNulls": false,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
+ }
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 10,
+ "w": 14,
+ "x": 10,
+ "y": 17
+ },
+ "id": 38,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "list",
+ "placement": "bottom"
+ },
+ "tooltip": {
+ "mode": "single"
+ }
+ },
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "sum by (state)
(node_systemd_unit_state{name=~\"taler-wallet(.*)\"})",
+ "interval": "",
+ "legendFormat": "{{state}}",
+ "refId": "A"
+ }
+ ],
+ "title": "Wallets over Time",
+ "type": "timeseries"
+ },
+ {
+ "datasource": {
+ "type": "loki",
+ "uid": "${DS_LOKI}"
+ },
+ "description": "Average time to deposit coins measured by wallet - for
all deposits in one iteration",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "axisLabel": "",
+ "axisPlacement": "auto",
+ "barAlignment": 0,
+ "drawStyle": "line",
+ "fillOpacity": 0,
+ "gradientMode": "none",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ },
+ "lineInterpolation": "linear",
+ "lineWidth": 1,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "auto",
+ "spanNulls": false,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
+ }
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "ms"
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 10,
+ "w": 10,
+ "x": 0,
+ "y": 27
+ },
+ "id": 43,
+ "maxDataPoints": 500,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "hidden",
+ "placement": "bottom"
+ },
+ "tooltip": {
+ "mode": "single"
+ }
+ },
+ "targets": [
+ {
+ "expr": "avg_over_time({app=\"taler-wallet-cli\"} |~ \"time=\" |~
\"deposit\" | logfmt | unwrap time [10m]) by (app)",
+ "legendFormat": "deposit",
+ "refId": "A",
+ "resolution": 5
+ }
+ ],
+ "title": "Average Deposit Duration (Iteration)",
+ "type": "timeseries"
+ },
+ {
+ "description": "",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "axisLabel": "",
+ "axisPlacement": "auto",
+ "barAlignment": 0,
+ "drawStyle": "line",
+ "fillOpacity": 0,
+ "gradientMode": "none",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ },
+ "lineInterpolation": "linear",
+ "lineWidth": 1,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "auto",
+ "spanNulls": false,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
+ }
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 10,
+ "w": 14,
+ "x": 10,
+ "y": 27
+ },
+ "id": 55,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "list",
+ "placement": "bottom"
+ },
+ "tooltip": {
+ "mode": "single"
+ }
+ },
+ "targets": [
+ {
+ "exemplar": true,
+ "expr": "sum by (state)
(node_systemd_unit_state{name=~\"taler-exchange-httpd(.*).service\"})",
+ "interval": "",
+ "legendFormat": "{{state}}",
+ "refId": "A"
+ }
+ ],
+ "title": "Exchanges over Time",
+ "type": "timeseries"
+ }
+ ],
+ "refresh": "1m",
+ "schemaVersion": 33,
+ "style": "dark",
+ "tags": [
+ "performance",
+ "taler"
+ ],
+ "templating": {
+ "list": []
+ },
+ "time": {
+ "from": "now-1h",
+ "to": "now"
+ },
+ "timepicker": {},
+ "timezone": "",
+ "title": "Transactions",
+ "uid": "83vvgKKnk",
+ "version": 84,
+ "weekStart": ""
+}
\ No newline at end of file
diff --git a/additional/plot.sh b/additional/plot.sh
deleted file mode 100755
index 068ff91..0000000
--- a/additional/plot.sh
+++ /dev/null
@@ -1,38 +0,0 @@
-#!/bin/bash
-# Create plots from the grafana dashboard
-set -eu
-
-PLOTTER_DIR=plotter
-
-source ../experiment/.env
-
-if [ ! -d ${PLOTTER_DIR} ]; then
- git clone https://github.com/bossm8/grafana-dashboard-plotter.git
${PLOTTER_DIR}
-fi
-
-cd ${PLOTTER_DIR} && git pull
-
-if [ ! -d venv ]; then
- python3 -m venv venv
-fi
-
-source venv/bin/activate
-pip install -r requirements.txt > /dev/null
-
-sed -e "s|\${ADMIN_API_KEY}|${GRAFANA_API_KEY}|g" \
- -e "s|\${BASE_URL}|${GRAFANA_HOST}|g" \
- ../config.yaml > config.yaml
-
-set +u
-
-TO=${TO:-$(date +%s)}
-
-if [ -z "${FROM}" ]; then
- python3 plots.py --to=${TO}
-else
- python3 plots.py --from=${FROM} --to=${TO}
-fi
-
-tar -czvf ../plots-${TO}.tar.gz plots
-
-rm -rf plots
diff --git a/additional/plots/config.yaml b/additional/plots/config.yaml
new file mode 100644
index 0000000..743ad4e
--- /dev/null
+++ b/additional/plots/config.yaml
@@ -0,0 +1,29 @@
+# Configuration for the Grafana Dashboard Plotter
+# https://github.com/bossm8/grafana-dashboard-plotter.git
+
+grafana:
+ admin_api_key: ${ADMIN_API_KEY}
+ base_url: ${BASE_URL}
+ # Default recording duration = 4 hours
+ # Can be overriden with --from
+ # With plot.sh this can be done by exporting FROM
+ # export FROM=$(date --date="today 08:30:00" +%s)
+ default_time_range: 14400
+ tls_verify: false
+
+dashboards:
+ - uid: 83vvgKKnk # Transacions
+ collapsed: true
+ - uid: 2FTtdeOnk # Database
+ variables: ['db', 'server']
+ - uid: MsjffzSZz # Proxy
+ - uid: WcfSXqDnk # Request Statistics
+ variables: ['endpoint']
+ - uid: rYdddlPWk # Nodes
+ variables: ['node']
+ ignore: 'wallet*|monitor*|bank*|merch*|ns*|proxy*'
+ - uid: rkyhDAt7z # Load Statistics
+
+prometheus:
+ node_exporter_job_name: nodes
+
diff --git a/additional/plots/plot.sh b/additional/plots/plot.sh
new file mode 100755
index 0000000..11e0e9c
--- /dev/null
+++ b/additional/plots/plot.sh
@@ -0,0 +1,66 @@
+#!/bin/bash
+# Create plots from the grafana dashboard
+# $1: [optional] name of the experiment
+#
+# Additional env: FROM, from when the plots should start
+# e.g. export FROM=$(date --date="today 08:30:00" +%s)
+# Additional env: TO, OPTIONAL, to when the plots should end
+# e.g. export TO=$(date +%s)
+set -eu
+
+# Setup the plotter from source
+PLOTTER_DIR=plotter
+
+source ../../experiment/.env
+
+if [ ! -d ${PLOTTER_DIR} ]; then
+ git clone https://github.com/bossm8/grafana-dashboard-plotter.git
${PLOTTER_DIR}
+fi
+
+cd ${PLOTTER_DIR} && git pull
+
+if [ ! -d venv ]; then
+ python3 -m venv venv
+fi
+
+source venv/bin/activate
+pip install -r requirements.txt > /dev/null
+
+# Apply the configuration for the plotter by using the experiments env config
+sed -e "s|\${ADMIN_API_KEY}|${GRAFANA_API_KEY}|g" \
+ -e "s|\${BASE_URL}|${GRAFANA_HOST}|g" \
+ ../config.yaml > config.yaml
+
+set +u
+
+TO=${TO:-$(date +%s)}
+
+if [ -z "${FROM}" ]; then
+ # Start with the configured default range (config.yaml - 4h)
+ python3 plots.py --to=${TO}
+else
+ python3 plots.py --from=${FROM} --to=${TO}
+fi
+
+NAME=${1:-${TO}}
+
+# Slugify the name
+NAME=$(\
+ echo "$NAME" | \
+ sed -r s/[~\^]+//g | \
+ sed -r s/[^a-zA-Z0-9]+/-/g | \
+ sed -r s/^-+\|-+$//g | \
+ tr A-Z a-z \
+)
+
+# Make sure no existing archive is overridden
+N=1
+while [ -f ../plots-${NAME}.tar.gz ]; do
+ NAME="${NAME}-${N}"
+ ((N++))
+done
+
+# Create an archive and remove the plots created
+tar -czvf ../plots-${NAME}.tar.gz plots
+
+rm -rf plots
diff --git a/configs/etc/bind/named.conf.options
b/configs/etc/bind/named.conf.options
index cae6567..bceecea 100644
--- a/configs/etc/bind/named.conf.options
+++ b/configs/etc/bind/named.conf.options
@@ -20,4 +20,3 @@ options {
<GRID_DNS_HERE>
};
};
-
diff --git a/configs/etc/default/prometheus-postgres-exporter
b/configs/etc/default/prometheus-postgres-exporter
index 15722ac..fe82513 100644
--- a/configs/etc/default/prometheus-postgres-exporter
+++ b/configs/etc/default/prometheus-postgres-exporter
@@ -4,7 +4,7 @@
# DATA_SOURCE_NAME='postgresql://login:password@hostname:port/'
# DATA_SOURCE_NAME='user=prometheus host=/run/postgresql dbname=postgres'
-DATA_SOURCE_NAME='<DB_URL_HERE>'
+DATA_SOURCE_NAME='<EXCHANGE_DB_URL_HERE>,<MERCHANT_DB_URL_HERE>'
# Set the command-line arguments to pass to the server.
ARGS='--extend.query-path=/etc/monitor/postgres-exporter.yaml'
diff --git a/configs/etc/default/taler-exchange
b/configs/etc/default/taler-exchange
deleted file mode 100644
index adcb212..0000000
--- a/configs/etc/default/taler-exchange
+++ /dev/null
@@ -1 +0,0 @@
-ARGS="<ARGUMENTS_HERE>"
diff --git a/configs/etc/monitor/exchange-exporters.yaml.tpl
b/configs/etc/monitor/exchange-exporters.yaml.tpl
new file mode 100644
index 0000000..74a4992
--- /dev/null
+++ b/configs/etc/monitor/exchange-exporters.yaml.tpl
@@ -0,0 +1,6 @@
+ - job_name: 'taler'
+ static_configs:
+ - labels:
+ component: 'exchange'
+ targets:
+ # <EXCHANGES_HERE>
diff --git a/configs/etc/monitor/node-exporters.yaml.tpl
b/configs/etc/monitor/node-exporters.yaml.tpl
index 09cbcaa..d8b69ac 100644
--- a/configs/etc/monitor/node-exporters.yaml.tpl
+++ b/configs/etc/monitor/node-exporters.yaml.tpl
@@ -1,21 +1,25 @@
- job_name: 'nodes'
static_configs:
+ - labels:
+ component: 'dns'
+ targets:
+ - 'ns1.${DNS_ZONE}:9100'
- labels:
component: 'database'
targets:
- - 'db.perf.taler:9100'
+ - '${DATABASE_DOMAIN}:9100'
- labels:
component: 'bank'
targets:
- - 'bank.perf.taler:9100'
+ - '${BANK_DOMAIN}:9100'
- labels:
component: 'exchange'
targets:
- - 'exch.perf.taler:9100'
+ - '${EXCHANGE_DOMAIN}:9100'
- labels:
component: 'proxy'
targets:
- - 'proxy.perf.taler:9100'
+ - '${PROXY_DOMAIN}:9100'
- labels:
component: 'wallet'
targets:
@@ -23,8 +27,8 @@
- labels:
component: 'monitor'
targets:
- - 'monitor.perf.taler:9100'
+ - '${MONITOR_DOMAIN}:9100'
- labels:
component: 'merchant'
targets:
- - 'merch.perf.taler:9100'
+ - '${MERCHANT_DOMAIN}:9100'
diff --git a/configs/etc/monitor/prometheus.yaml
b/configs/etc/monitor/prometheus.yaml
index 88e61fd..37f9773 100644
--- a/configs/etc/monitor/prometheus.yaml
+++ b/configs/etc/monitor/prometheus.yaml
@@ -14,7 +14,7 @@ scrape_configs:
# DB Exporter
- job_name: 'database'
static_configs:
- - targets: ['127.0.0.1:9187', '<MERCHANT_HOST_HERE>:9187']
+ - targets: ['127.0.0.1:9187'] #, '<MERCHANT_HOST_HERE>:9187']
# Exchange Proxy Exporter
- job_name: 'exchange-proxy'
diff --git a/configs/etc/nginx/sites-available/proxy
b/configs/etc/nginx/sites-enabled/proxy
similarity index 100%
rename from configs/etc/nginx/sites-available/proxy
rename to configs/etc/nginx/sites-enabled/proxy
diff --git a/configs/etc/pgbouncer/pgbouncer.ini
b/configs/etc/pgbouncer/pgbouncer.ini
new file mode 100644
index 0000000..b2b15c1
--- /dev/null
+++ b/configs/etc/pgbouncer/pgbouncer.ini
@@ -0,0 +1,25 @@
+[databases]
+
+<DB_NAME_HERE> = dbname=<DB_NAME_HERE> host=localhost
+
+[pgbouncer]
+
+listen_addr = *
+listen_port = 6432
+
+auth_type = md5
+auth_file = /etc/pgbouncer/userlist.txt
+
+max_client_conn = 500
+
+pool_mode = transaction
+default_pool_size = 50
+reserve_pool_size = 20
+reserve_pool_timeout = 1
+
+server_idle_timeout = 60
+
+logfile = /var/log/postgresql/pgbouncer.log
+pidfile = /var/run/postgresql/pgbouncer.pid
+
+unix_socket_dir = /var/run/postgresql
diff --git a/configs/etc/pgbouncer/userlist.txt
b/configs/etc/pgbouncer/userlist.txt
new file mode 100644
index 0000000..1436a38
--- /dev/null
+++ b/configs/etc/pgbouncer/userlist.txt
@@ -0,0 +1 @@
+"<DB_USER_HERE>" "<DB_PASSWORD_HERE>"
diff --git a/configs/etc/rsyslog.d/taler.conf b/configs/etc/rsyslog.d/taler.conf
index cbc8e7a..848afe4 100644
--- a/configs/etc/rsyslog.d/taler.conf
+++ b/configs/etc/rsyslog.d/taler.conf
@@ -10,7 +10,7 @@ if ($programname startswith 'taler' or $syslogtag startswith
'taler') then {
#
https://grafana.com/docs/loki/latest/clients/promtail/scraping/#rsyslog-output-configuration
action(type="omfwd"
protocol="tcp"
- target="monitor.perf.taler"
+ target="<MONITOR_DOMAIN_HERE>"
port="1514"
Template="RSYSLOG_SyslogProtocol23Format"
TCP_Framing="octet-counted"
diff --git a/configs/etc/taler/conf.d/exchange-business.conf
b/configs/etc/taler/conf.d/exchange-business.conf
old mode 100755
new mode 100644
index 0e6dd1e..c5e6e17
--- a/configs/etc/taler/conf.d/exchange-business.conf
+++ b/configs/etc/taler/conf.d/exchange-business.conf
@@ -12,6 +12,8 @@ MASTER_PUBLIC_KEY = <MASTER_KEY_HERE>
# BASE_URL = https://example.com/
BASE_URL = <BASE_URL_HERE>
+MAX_REQUESTS = <MAX_REQUESTS_HERE>
+
# For your terms of service and privacy policy, you should specify
# an Etag that must be updated whenever there are significant
# changes to either document. The format is up to you, what matters
diff --git a/configs/etc/taler/conf.d/exchange-coins.conf
b/configs/etc/taler/conf.d/exchange-coins.conf
old mode 100755
new mode 100644
diff --git a/configs/etc/taler/secrets/exchange-accountcredentials.secret.conf
b/configs/etc/taler/secrets/exchange-accountcredentials.secret.conf
old mode 100755
new mode 100644
diff --git a/configs/etc/taler/secrets/exchange-db.secret.conf
b/configs/etc/taler/secrets/exchange-db.secret.conf
old mode 100755
new mode 100644
diff --git a/configs/etc/taler/taler.conf b/configs/etc/taler/taler.conf
old mode 100755
new mode 100644
diff --git a/configs/usr/lib/systemd/system/taler-exchange-aggregator.service
b/configs/usr/lib/systemd/system/taler-exchange-aggregator.service
index 44f4351..9aba5a4 100644
--- a/configs/usr/lib/systemd/system/taler-exchange-aggregator.service
+++ b/configs/usr/lib/systemd/system/taler-exchange-aggregator.service
@@ -6,8 +6,8 @@ PartOf=taler-exchange.target
User=taler-exchange-aggregator
Type=simple
Restart=on-failure
-EnvironmentFile=/etc/default/taler-exchange
-ExecStart=/usr/bin/taler-exchange-aggregator -c /etc/taler/taler.conf $ARGS
+EnvironmentFile=/etc/environment
+ExecStart=/usr/bin/taler-exchange-aggregator -c /etc/taler/taler.conf
$TALER_ARGS $AGGREGATOR_ARGS
StandardOutput=journal
StandardError=journal
PrivateTmp=yes
diff --git a/configs/usr/lib/systemd/system/taler-exchange-closer.service
b/configs/usr/lib/systemd/system/taler-exchange-closer.service
index f5bb6c6..0728ce2 100644
--- a/configs/usr/lib/systemd/system/taler-exchange-closer.service
+++ b/configs/usr/lib/systemd/system/taler-exchange-closer.service
@@ -6,8 +6,8 @@ PartOf=taler-exchange.target
User=taler-exchange-closer
Type=simple
Restart=on-failure
-EnvironmentFile=/etc/default/taler-exchange
-ExecStart=/usr/bin/taler-exchange-closer -c /etc/taler/taler.conf $ARGS
+EnvironmentFile=/etc/environment
+ExecStart=/usr/bin/taler-exchange-closer -c /etc/taler/taler.conf $TALER_ARGS
$CLOSER_ARGS
StandardOutput=journal
StandardError=journal
PrivateTmp=yes
diff --git a/configs/usr/lib/systemd/system/taler-exchange-httpd.service
b/configs/usr/lib/systemd/system/taler-exchange-httpd.service
old mode 100755
new mode 100644
index 43a90c2..8be8049
--- a/configs/usr/lib/systemd/system/taler-exchange-httpd.service
+++ b/configs/usr/lib/systemd/system/taler-exchange-httpd.service
@@ -8,9 +8,13 @@ PartOf=taler-exchange.target
[Service]
User=taler-exchange-httpd
Type=simple
-Restart=on-failure
-EnvironmentFile=/etc/default/taler-exchange
-ExecStart=/usr/bin/taler-exchange-httpd -c /etc/taler/taler.conf $ARGS
+# Depending on the configuration, the service suicides and then
+# needs to be restarted.
+Restart=always
+# Do not dally on restarts.
+RestartSec=1ms
+EnvironmentFile=/etc/environment
+ExecStart=<CMD_PREFIX_HERE>/usr/bin/taler-exchange-httpd -c
/etc/taler/taler.conf $TALER_ARGS $EXCHANGE_ARGS
StandardOutput=journal
StandardError=journal
PrivateTmp=no
@@ -19,4 +23,3 @@ ProtectSystem=full
[Install]
WantedBy=multi-user.target
-
diff --git a/configs/usr/lib/systemd/system/taler-exchange-httpd.socket
b/configs/usr/lib/systemd/system/taler-exchange-httpd.socket
old mode 100755
new mode 100644
diff --git a/configs/usr/lib/systemd/system/taler-exchange-httpd@.service
b/configs/usr/lib/systemd/system/taler-exchange-httpd@.service
old mode 100755
new mode 100644
index d6ee28d..03477ad
--- a/configs/usr/lib/systemd/system/taler-exchange-httpd@.service
+++ b/configs/usr/lib/systemd/system/taler-exchange-httpd@.service
@@ -8,9 +8,13 @@ PartOf=taler-exchange.target
[Service]
User=taler-exchange-httpd
Type=simple
-Restart=on-failure
-EnvironmentFile=/etc/default/taler-exchange
-ExecStart=/usr/bin/taler-exchange-httpd -c /etc/taler/taler.conf $ARGS
+# Depending on the configuration, the service suicides and then
+# needs to be restarted.
+Restart=always
+# Do not dally on restarts.
+RestartSec=1ms
+EnvironmentFile=/etc/environment
+ExecStart=<CMD_PREFIX_HERE>/usr/bin/taler-exchange-httpd -c
/etc/taler/taler.conf $TALER_ARGS $EXCHANGE_ARGS
StandardOutput=journal
StandardError=journal
PrivateTmp=no
diff --git a/configs/usr/lib/systemd/system/taler-exchange-httpd@.socket
b/configs/usr/lib/systemd/system/taler-exchange-httpd@.socket
old mode 100755
new mode 100644
diff --git a/configs/usr/lib/systemd/system/taler-exchange-secmod-eddsa.service
b/configs/usr/lib/systemd/system/taler-exchange-secmod-eddsa.service
index 98d7ff0..f96bee3 100644
--- a/configs/usr/lib/systemd/system/taler-exchange-secmod-eddsa.service
+++ b/configs/usr/lib/systemd/system/taler-exchange-secmod-eddsa.service
@@ -7,8 +7,8 @@ PartOf=taler-exchange.target
User=taler-exchange-secmod-eddsa
Type=simple
Restart=on-failure
-EnvironmentFile=/etc/default/taler-exchange
-ExecStart=/usr/bin/taler-exchange-secmod-eddsa -c /etc/taler/taler.conf $ARGS
+EnvironmentFile=/etc/environment
+ExecStart=/usr/bin/taler-exchange-secmod-eddsa -c /etc/taler/taler.conf
$TALER_ARGS $EDDSA_ARGS
StandardOutput=journal
StandardError=journal
PrivateTmp=no
diff --git a/configs/usr/lib/systemd/system/taler-exchange-secmod-rsa.service
b/configs/usr/lib/systemd/system/taler-exchange-secmod-rsa.service
index fca5a3d..433fd5a 100644
--- a/configs/usr/lib/systemd/system/taler-exchange-secmod-rsa.service
+++ b/configs/usr/lib/systemd/system/taler-exchange-secmod-rsa.service
@@ -7,8 +7,8 @@ PartOf=taler-exchange.target
User=taler-exchange-secmod-rsa
Type=simple
Restart=on-failure
-EnvironmentFile=/etc/default/taler-exchange
-ExecStart=/usr/bin/taler-exchange-secmod-rsa -c /etc/taler/taler.conf $ARGS
+EnvironmentFile=/etc/environment
+ExecStart=/usr/bin/taler-exchange-secmod-rsa -c /etc/taler/taler.conf
$TALER_ARGS $RSA_ARGS
StandardOutput=journal
StandardError=journal
PrivateTmp=no
diff --git a/configs/usr/lib/systemd/system/taler-exchange-transfer.service
b/configs/usr/lib/systemd/system/taler-exchange-transfer.service
index c8a2e9a..325a37a 100644
--- a/configs/usr/lib/systemd/system/taler-exchange-transfer.service
+++ b/configs/usr/lib/systemd/system/taler-exchange-transfer.service
@@ -7,8 +7,8 @@ PartOf=taler-exchange.target
User=taler-exchange-wire
Type=simple
Restart=on-failure
-EnvironmentFile=/etc/default/taler-exchange
-ExecStart=/usr/bin/taler-exchange-transfer -c /etc/taler/taler.conf $ARGS
+EnvironmentFile=/etc/environment
+ExecStart=/usr/bin/taler-exchange-transfer -c /etc/taler/taler.conf
$TALER_ARGS $TRANSFER_ARGS
StandardOutput=journal
StandardError=journal
PrivateTmp=yes
diff --git a/configs/usr/lib/systemd/system/taler-exchange-wirewatch.service
b/configs/usr/lib/systemd/system/taler-exchange-wirewatch.service
index f8ab764..7efe7de 100644
--- a/configs/usr/lib/systemd/system/taler-exchange-wirewatch.service
+++ b/configs/usr/lib/systemd/system/taler-exchange-wirewatch.service
@@ -7,8 +7,8 @@ PartOf=taler-exchange.target
User=taler-exchange-wire
Type=simple
Restart=on-failure
-EnvironmentFile=/etc/default/taler-exchange
-ExecStart=/usr/bin/taler-exchange-wirewatch -c /etc/taler/taler.conf $ARGS
+EnvironmentFile=/etc/environment
+ExecStart=/usr/bin/taler-exchange-wirewatch -c /etc/taler/taler.conf
$TALER_ARGS $WIREWATCH_ARGS
StandardOutput=journal
StandardError=journal
PrivateTmp=yes
diff --git a/configs/usr/lib/systemd/system/taler-exchange-wirewatch@.service
b/configs/usr/lib/systemd/system/taler-exchange-wirewatch@.service
index f8ab764..6c98764 100644
--- a/configs/usr/lib/systemd/system/taler-exchange-wirewatch@.service
+++ b/configs/usr/lib/systemd/system/taler-exchange-wirewatch@.service
@@ -6,9 +6,10 @@ PartOf=taler-exchange.target
[Service]
User=taler-exchange-wire
Type=simple
-Restart=on-failure
-EnvironmentFile=/etc/default/taler-exchange
-ExecStart=/usr/bin/taler-exchange-wirewatch -c /etc/taler/taler.conf $ARGS
+Restart=always
+RestartSec=100ms
+EnvironmentFile=/etc/environment
+ExecStart=/usr/bin/taler-exchange-wirewatch -c /etc/taler/taler.conf
$TALER_ARGS $WIREWATCH_ARGS
StandardOutput=journal
StandardError=journal
PrivateTmp=yes
diff --git a/configs/usr/lib/systemd/system/taler-fakebank.service
b/configs/usr/lib/systemd/system/taler-fakebank.service
index f81ad53..074e917 100644
--- a/configs/usr/lib/systemd/system/taler-fakebank.service
+++ b/configs/usr/lib/systemd/system/taler-fakebank.service
@@ -3,8 +3,8 @@ Description=GNU Taler fake bank
[Service]
Type=simple
-EnvironmentFile=/etc/default/taler-exchange
-ExecStart=/usr/bin/taler-fakebank-run -c /etc/taler/taler.conf -n 100 $ARGS
+EnvironmentFile=/etc/environment
+ExecStart=/usr/bin/taler-fakebank-run -c /etc/taler/taler.conf -n 100
$TALER_ARGS $BANK_ARGS
StandardOutput=journal
StandardError=journal
diff --git a/configs/usr/lib/systemd/system/taler-logbackup.service
b/configs/usr/lib/systemd/system/taler-logrotate.service
similarity index 86%
rename from configs/usr/lib/systemd/system/taler-logbackup.service
rename to configs/usr/lib/systemd/system/taler-logrotate.service
index 45001c2..0dd22af 100644
--- a/configs/usr/lib/systemd/system/taler-logbackup.service
+++ b/configs/usr/lib/systemd/system/taler-logrotate.service
@@ -1,6 +1,6 @@
[Unit]
Description=Compresses and moves taler logs to the grid5000 nfs
-Wants=taler-logbackup.timer
+Wants=taler-logrotate.timer
[Service]
Type=oneshot
diff --git a/configs/usr/lib/systemd/system/taler-logbackup.timer
b/configs/usr/lib/systemd/system/taler-logrotate.timer
similarity index 68%
rename from configs/usr/lib/systemd/system/taler-logbackup.timer
rename to configs/usr/lib/systemd/system/taler-logrotate.timer
index fe33a6a..7cb2ae3 100644
--- a/configs/usr/lib/systemd/system/taler-logbackup.timer
+++ b/configs/usr/lib/systemd/system/taler-logrotate.timer
@@ -1,9 +1,9 @@
[Unit]
Description=Compresses and moves taler logs to the grid5000 nfs
-Requires=taler-logbackup.service
+Requires=taler-logrotate.service
[Timer]
-Unit=taler-logbackup.service
+Unit=taler-logrotate.service
OnCalendar=*:0/15
[Install]
diff --git a/configs/usr/lib/systemd/system/taler-wallet@.service
b/configs/usr/lib/systemd/system/taler-wallet@.service
old mode 100755
new mode 100644
diff --git a/configs/var/lib/bind/perf.taler b/configs/var/lib/bind/perf.taler
index 81ea22d..afa5ad3 100644
--- a/configs/var/lib/bind/perf.taler
+++ b/configs/var/lib/bind/perf.taler
@@ -9,4 +9,4 @@ $TTL 3600 ; 1 hour
)
NS ns1.<DNS_ZONE_HERE>.
$ORIGIN <DNS_ZONE_HERE>.
-ns1 A 127.0.0.1
+ns1 A <NS_IP_HERE>
diff --git a/docker/Dockerfile b/docker/Dockerfile
index 8805d3e..5c29514 100644
--- a/docker/Dockerfile
+++ b/docker/Dockerfile
@@ -1,13 +1,21 @@
FROM debian:11
+LABEL author="Boss Marco" \
+ company="Taler" \
+ description="Image to automate Grid5000 image build for the \
+Taler performance experiments. \
+This image contains all dev libraries used to also \
+build the binaries directly"
+
ENV DEBIAN_FRONTEND=noninteractive
-ENV GRID5K_DEST=lyon,lille
-ENV GNUNET_COMMIT_SHA=master
-ENV EXCHANGE_COMMIT_SHA=master
-ENV MERCHANT_COMMIT_SHA=master
-ENV WALLET_COMMIT_SHA=master
-ENV GRID5K_COMMIT_SHA=master
+ENV GRID5K_DEST=lyon,lille \
+ GNUNET_COMMIT_SHA=master \
+ EXCHANGE_COMMIT_SHA=master \
+ MERCHANT_COMMIT_SHA=master \
+ WALLET_COMMIT_SHA=master \
+ GRID5K_COMMIT_SHA=master \
+ TALER_HOME=/taler
RUN apt update && \
apt upgrade -y && \
diff --git a/docker/README.md b/docker/README.md
index 0e7b579..270db82 100644
--- a/docker/README.md
+++ b/docker/README.md
@@ -33,6 +33,15 @@ docker run -it --rm \
**NOTE** about the port 5900, this one can be used for vncviewer to see whats
happening inside the image which
will be created. Run `vncviewver :0`.
+#### Manual Build
+
+To get an interactive shell into the image override the entrypoint by adding
the following argument
+before the last line in the command above:
+
+```bash
+--entrypoint=/bin/bash
+```
+
### docker-compose
Assuming an env file `.env` with the following contents:
@@ -50,6 +59,14 @@ ARGUMENTS=<ARGUMENTS> GRID5K_CERT_PASSWD=<cert_passwd>
docker-compose up --build
Or place the password (also arguments) in .env too but make sure its protected.
+#### Manual Build
+
+You can also use the docker compose to get an interactive shell into the image:
+
+```bash
+docker-compose run --entrypoint /bin/bash taler-build
+```
+
### Notes
#### Environment Variables
@@ -59,10 +76,8 @@ Or place the password (also arguments) in .env too but make
sure its protected.
**GRID5K_CERT_PASSWD**: the password to decrypt `GRID5K_CERT`
**GRID5K_DEST**: comma separated list of where to copy the image to in the
grid (default: lille,lyon)
**ARGUMENTS**: args to pass to entrypoint, one of
- -c|--clean (run make clean)
- -r|--rebuild (rebuild the image - automatically set if -c is used)
+ -r|--rebuild (rebuild the image)
-n|--no-copy (do not copy the generated image to Grid5000 - make sure output
volume is mounted)
- -v|--verbose (enable verbose logging)
As per default, running the docker command again will not clean or rebuild
the image
##### Additional
diff --git a/docker/entrypoint.sh b/docker/entrypoint.sh
index 2862722..b1f3eef 100644
--- a/docker/entrypoint.sh
+++ b/docker/entrypoint.sh
@@ -1,14 +1,12 @@
#!/bin/bash
+# Docker Entrypoint script
+# Automatically builds the Taler performance
+# environment image for Grid5000
+
set -e
while [[ $# -gt 0 ]]; do
case "$1" in
- -c|--clean)
- CLEAN=true
- REBUILD=true
- echo "INFO will clean previous builds and image"
- shift
- ;;
-r|--rebuild)
REBUILD=true
echo "INFO will rebuild image"
@@ -19,91 +17,36 @@ while [[ $# -gt 0 ]]; do
echo "INFO will not copy image to Grid5000"
shift
;;
- -v|--verbose)
- VERBOSE=true
- echo "INFO verbose output enabled"
- shift
- ;;
*)
shift
;;
esac
done
-TALER_HOME=/taler
+# Run a command and fail when there was an error
+# $1: Command to run
+# $2: Log file to write to (default /dev/null)
+function run_command() {
+ if ! $(eval ${1} > "${2:-/dev/null}" 2>&1) ; then
+ echo "ERROR cmd ${1} failed"
+ cat "${2}" || true
+ exit 1
+ fi
+}
+# Prepare a git repo
+# $1: Git repo name
+# $2: Git commit to checkout to
function prepare() {
cd "${TALER_HOME}/$1"
git checkout master > /dev/null && (git pull > /dev/null 2>&1 || true)
git checkout "$2" > /dev/null && (git pull > /dev/null 2>&1 || true)
}
-function _make() {
- echo "INFO running bootstrap and configure"
- if ! ./bootstrap > bootstrap.log 2>&1; then
- echo "ERROR bootstrap failed"
- cat bootstrap.log
- exit 1
- fi
- if [ -f contrib/gana.sh ]; then
- ./contrib/gana.sh > gana.log 2>&1
- fi
- if ! (./configure --enable-logging=verbose --prefix=/usr > configure.log
2>&1 || \
- ./configure > configure.log 2>&1); then
- echo "ERROR configure failed"
- cat configure.log
- exit 1
- fi
- if [ "$VERBOSE" = true ]; then
- cat bootstrap.log || true
- cat gana.log || true
- cat configure.log || true
- fi
-}
-
-function build() {
- prepare "$1" "$2"
- if [ "$CLEAN" = true ]; then
- echo "INFO cleaning previous build"
- make clean > /dev/null 2>&1 || true
- fi
- if ! make > /dev/null 2>&1; then
- _make
- fi
- if [ ! -z "$3" ]; then
- echo "INFO running custom command '$3'"
- /bin/bash -c "$3"
- fi
- echo "INFO running 'make dist'"
- if ! make dist > dist.log 2>&1; then
- echo "ERROR make dist failed"
- cat dist.log
- exit 1
- fi
- if [ "$VERBOSE" = true ]; then
- cat dist.log || true
- fi
-}
-
-function install() {
- build "$1" "$2" "$3"
- echo "INFO installing"
- if [[ $(ldconfig -p | grep "$1") == "" ]] || \
- [ "$CLEAN" = true ]; then
- if ! make install > install.log 2>&1; then
- echo "ERROR install failed"
- cat install.log
- exit 1
- fi
- ldconfig
- fi
- if [ "$VERBOSE" = true ]; then
- cat install.log || true
- fi
-}
-
if [ "$COPY" != false ];
then
+ # Add the certificate to the agent
+ # no more passwords are needed when it succeeds
eval $(ssh-agent)
if [[ -f "/root/cert.pem" ]]; then
@@ -138,47 +81,29 @@ then
fi
echo "INFO preparing Grid5000 repository"
-prepare grid5k \
- "${GRID5K_COMMIT_SHA}"
-touch
${TALER_HOME}/grid5k/image/grid5000/steps/data/{gnunet,wallet,exchange,merchant}.tar.gz
-if ! kameleon dryrun ${TALER_HOME}/grid5k/image/taler-debian11.yaml >
dryrun.log 2>&1; then
- echo "ERROR Grid5000 image configuration contains errors"
- cat dryrun.log
- exit 1
-else
- rm ${TALER_HOME}/grid5k/image/grid5000/steps/data/*.tar.gz
-fi
+prepare grid5k "${GRID5K_COMMIT_SHA}"
-echo "INFO preparing GNUnet"
-install gnunet \
- "${GNUNET_COMMIT_SHA}"
-
-echo "INFO preparing Taler Exchange"
-install exchange \
- "${EXCHANGE_COMMIT_SHA}" \
- 'find . -name "*Makefile*" -exec sed -i "/x-taler-bank.fee/d" {} \;'
-
-echo "INFO preparing Taler Merchant"
-install merchant \
- "${MERCHANT_COMMIT_SHA}"
-
-echo "INFO preparing Taler Wallet"
-build wallet-core \
- "${WALLET_COMMIT_SHA}"
+# Test if the image specification is correct
+run_command \
+ "kameleon dryrun -g g5k_user:${G5K_USER}
${TALER_HOME}/grid5k/image/taler-debian11.yaml" \
+ "dryrun.log"
cd ${TALER_HOME}/grid5k/image
if [ ! -f "build/taler-debian11/taler-debian11.tar.zst" ] || \
[ "$REBUILD" = true ];
then
- mv "${TALER_HOME}"/gnunet/gnunet*.tar.gz grid5000/steps/data/gnunet.tar.gz
- mv "${TALER_HOME}"/exchange/taler-exchange*.tar.gz
grid5000/steps/data/exchange.tar.gz
- mv "${TALER_HOME}"/merchant/taler-merchant*.tar.gz
grid5000/steps/data/merchant.tar.gz
- mv "${TALER_HOME}"/wallet-core/taler-wallet*.tar.gz
grid5000/steps/data/wallet.tar.gz
-
- rm -rf /tmp/taler-debian11 || true
+ rm -rf /tmp/taler-debian11 ${TALER_HOME}/taler-debian11 || true
echo "INFO building image"
- yes r | kameleon build -b /tmp taler-debian11.yaml | tee build.log
+ yes r | kameleon build \
+ -b /tmp taler-debian11.yaml \
+ -g gnunet_commit_sha:${GNUNET_COMMIT_SHA} \
+ exchange_commit_sha:${EXCHANGE_COMMIT_SHA} \
+ merchant_commit_sha:${MERCHANT_COMMIT_SHA} \
+ wallet_commit_sha:${WALLET_COMMIT_SHA} \
+ grid5k_commit_sha:${GRID5K_COMMIT_SHA} \
+ g5k_user:${GRID5K_USER} \
+ | tee build.log
mv /tmp/taler-debian11 ${TALER_HOME}
fi
@@ -193,7 +118,7 @@ if [ -f "/root/cert.pem" ] && [ "$COPY" != false ]; then
for G5K_HOST in "${G5K_HOSTS[@]}"; do
echo "Copying image to ${G5K_HOST}"
- sed "s/<G5K_HOST>/${G5K_HOST}/g; s/<G5K_USER>/${GRID5K_USER}/g" \
+ sed "/g5k_tar_path/s/lyon/${G5K_HOST}/g" \
taler-debian11.dsc.bak > taler-debian11.dsc
scp -o StrictHostKeyChecking=no taler-debian11.tar.zst taler-debian11.dsc \
diff --git a/experiment/README.md b/experiment/README.md
index 599beb3..0517538 100644
--- a/experiment/README.md
+++ b/experiment/README.md
@@ -3,8 +3,9 @@
## Files
* experiment-specitication.yml: [ESpec](https://jfed.ilabt.imec.be/espec/) for
jFed
-* infra.rspec: Contains all nodes for the Taler infrastructure
-* wallets.rspec: Contains a set of nodes for wallets
+* taler.rspec: Complete set of nodes to run an experiment
+* infra.rspec: Contains all nodes for the Taler infrastructure only
+* wallets.rspec: Contains an additional set of nodes for wallets
* env: template file to add enviroment variables needed for the experiment
* scripts: Bash scripts which will be run in the experiment
@@ -20,20 +21,43 @@ To successfully run an experiment the following steps must
be made:
* Copy the environment default configuration `env` to `.env`
* Read through `.env` and define the missing variables
* Start jFed Experimenter GUI
-* Load infra.rspec and click Run
+* Load taler.rspec and click Run
* Specify the experiment name and time
-* Wait until infra.rspec is allocated successfully
-* Configure `.env` and set the hosts - the allocated nodes can be found in the
tab `RSpec Viewer`
- (this can be fixed once we can use one rspec only)
-* Wait until the nodes are ready
-* Click (Re)Run ESpec for the infra job (use the type Directory and select
this directory (experiment))
+* Wait until taler.rspec is allocated successfully and nodes are ready
+* Click (Re)Run ESpec for the job (use the type Directory and select this
directory (experiment))
+* If any error ocurr just press (Re)Run Espec again because sometimes there
are still unindentidfied errors
+
+### Add addtional Wallet Hosts
+
+TODO - DOES NOT WORK YET - DNS host needs to be known ..
+
* Load wallets.rspec into jFed and click Run
-* Click (Re)Run ESpec for the wallets job (type directory)
+* Click (Re)Run ESpec for the wallet job (type directory)
* Run `talet-perf update prometheus` on any deployed node to make prometheus
aware of the freshly added wallets
-**NOTE** If for some reason the espec needs to be rerun, all uploaded files
need to be deleted currently.
- This can be achieved on all nodes by pressing `Multi Command` and
specifying the command:
- `rm -rf scripts .env`
+**NOTE** On `taler-perf`, when not using a terminal opened from jFed make sure
to forward the ssh-agent
+ to make the script work. E.g. `ssh -A graoully-3.nancy.g5k`
+
+## Rebuild Taler Binaries
+
+To quickly test fixes of new commits in gnunet,exchange,merchant and wallet,
there is a script `scripts/instal.sh`
+which can be run inside a running experiment rather than rebuilding the whole
image.
+
+To do so copy the following snippet into the `Multi Command` window in jFed:
+Please adjust commits as needed, the ones which are not defined or empty will
not be built.
+The ones which are a space only (" ") will build on master.
+
+```bash
+#!/bin/bash
+
+export GNUNET_COMMIT_SHA=master
+export EXCHANGE_COMMIT_SHA=master
+export MERCHANT_COMMIT_SHA=master
+export WALLET_COMMIT_SHA=master
+
+/bin/bash /root/taler/grid5k/experiment/scripts/install.sh
+# /bin/bash /root/scripts/install.sh
+```
## Actions in running Experiment
@@ -54,8 +78,4 @@ plots based on a configuration. Please refer to the README
located in the specif
The logs of the taler applications will be moved periodically to
`/home/<G5K_USER>/taler-logs`.
They can then be retrieved using scp to the respective site they where running
on
(e.g. nancy/taler-logs for grvingt and others).
-Make sure to take them away before restarting an experiment, since they will
be deleted in a new experiment.
-
-## Notes
-
-Hopefully the bind dns gets useless once the experiment can be run in one
espec.
+Make sure to back them up before restarting an experiment, since they will be
deleted in a new one.
diff --git a/additional/grafana/tbd b/experiment/TODO
similarity index 100%
rename from additional/grafana/tbd
rename to experiment/TODO
diff --git a/experiment/env b/experiment/env
index f08ffdc..a0f3e9d 100644
--- a/experiment/env
+++ b/experiment/env
@@ -1,37 +1,54 @@
-# Place hosts which are allocated by jFed here
-# They can be found in the tab 'RSpec Viewer' once the
-# nodes are allocated.
-# Once the allocation bug is fixed these configurations
-# will be removed.
-# The DNS_HOST must be set completely, others can contain
-# a wildcard after the node name to match the site automatically.
-# Wallet hosts can be specified as a wildcard only since
-# all other hosts are matched before.
- BANK_HOSTS=dahu-x.*
-DATABASE_HOSTS=dahu-y.*
- DNS_HOST=dahu-12.grenoble.grid5000.fr
-EXCHANGE_HOSTS=dahu-z.*
-MERCHANT_HOSTS=grvingt-x.*
- MONITOR_HOSTS=econome-y.*
- PROXY_HOSTS=dahu-u.*
- WALLET_HOSTS=*
+# Important node names, DO NOT Adjust unless you know
+# what you are doing, it most certain needs adjustings
+# in run.sh and other scripts since variable names depend on it.
+# Make sure to name all hosts with the following format in jFed:
+# 'Role'-N, add only 'Role' here, setup.sh will determine the rest.
+# Nodes listed here will be parsed and exported in
+# <ROLE>_HOSTS=<Grid5000-node[s]>
+NODES="Bank DB DNS Exchange Merchant Monitor Proxy"
+# DNS Zone to setup inside the grid for the experiment
DNS_ZONE=perf.taler
-# These values specify the domain names for
-# the nodes, they most certain do not need to be adjusted.
-# DO NOT change wallet domain, since this is currently needed
-# for cleardns.sh
- BANK_DOMAIN=bank.perf.taler
-DATABASE_DOMAIN=db.perf.taler
-EXCHANGE_DOMAIN=exch.perf.taler
-MERCHANT_DOMAIN=merch.perf.taler
- MONITOR_DOMAIN=monitor.perf.taler
- PROXY_DOMAIN=proxy.perf.taler
- WALLET_DOMAIN=wallet.*.perf.taler
+# Where this repository is located inside the image
+# Should not need to be changed unless the image itself was adjusted
+G5K_HOME=/root/taler/grid5k
+# Which branch of this repository to checkout for
+# configuration files
+G5K_COMMIT_SHA=master
-# Arguments to pass to taler-exchange-* binaries at startup.
-EXCHANGE_ARGS="-L INFO"
+# Domain names for the hosts inside the grid.
+# The most certainly do not need to be adjusted.
+# If you do anyway make sure the wallets contain a wildcard (*)
+# Otherwise the experiment will not work
+ BANK_DOMAIN=bank.${DNS_ZONE}
+DATABASE_DOMAIN=db.${DNS_ZONE}
+ DNS_DOMAIN=ns1.${DNS_ZONE}
+EXCHANGE_DOMAIN=exchange.${DNS_ZONE}
+MERCHANT_DOMAIN=merchant.${DNS_ZONE}
+ MONITOR_DOMAIN=monitor.${DNS_ZONE}
+ PROXY_DOMAIN=proxy.${DNS_ZONE}
+ WALLET_DOMAIN=wallet.*.${DNS_ZONE}
+
+# Arguments to pass to all taler-exchange-* binaries at startup.
+# E.g. -L INFO
+ TALER_ARGS="-L INFO"
+# Service specific arguments to pass at startup
+AGGREGATOR_ARGS="-y"
+ BANK_ARGS=""
+ CLOSER_ARGS=""
+ EXCHANGE_ARGS=""
+ TRANSFER_ARGS=""
+ WIREWATCH_ARGS=""
+ EDDSA_ARGS=""
+ RSA_ARGS=""
+# Additional arguments to prepend to taler-exchange-httpd.service
+# files ExecStart command, e.g. /usr/bin/valgrind
--log-file=/tmp/exchange-%N.log
+EXCHANGE_CMD_PREFIX=""
+
+# Maximal number of requests an exchange process should handle
+# before it kills itself
+EXCHANGE_MAX_REQUESTS=10000
# Datasource names registered in the grafana instance
PROMETHEUS_DATASOURCE_NAME=Prometheus
@@ -47,7 +64,10 @@ LOKI_G5K_PROXY_PORT=http
# Exchange database configuration
DB_NAME=taler-exchange
DB_USER=taler
-DB_PASSWORD=
+# Database password
+DB_PASSWORD=taler
+# If exchanges should connect to pgBouncer rather than to the db directly
+USE_PGBOUNCER=false
# Initial number of wallets to start in parallel per host.
# This is a bulk size with default = 10
@@ -63,12 +83,18 @@ NUM_EXCHANGE_PROCESSES=1
# This setting is static, default = 1
NUM_WIREWATTCH_PROCESSES=1
+TALER_WALLET_SYNC_CRYPTO=1
+TALER_WALLET_PRIMITIVE_WORKER=1
+TALER_WALLET_INSECURE_TRUST_EXCHANGE=1
+
# If prometheus node exporter should be enabled
# Most certainly this is true, since otherwise no overview
# of services and running wallets can be generated in grafana.
ENABLE_EXPORTERS=true
# Host where grafana is reachable including scheme and port.
+# If this one or the key is left empty, no grafana instance
+# is assumed, Loki and Prometheus will run anyway since they are used by
rsyslog.
GRAFANA_HOST=https://147.87.255.221:3000
# API key for grafana
# Needs admin level since datasources need to be updated.
diff --git a/experiment/experiment-specification.yml
b/experiment/experiment-specification.yml
index 6881aea..908a4cf 100644
--- a/experiment/experiment-specification.yml
+++ b/experiment/experiment-specification.yml
@@ -1,18 +1,30 @@
+# Experiment Speification
+# To be loaded into jFed using (Re)Run Espec
version: 1.0-basic
rspec:
- - bundled: infra.rspec
+ - bundled: taler.rspec
upload:
- bundled: .env
permission: "664"
- - bundled: scripts
- path: ~/scripts
+ path: /tmp/.env
+ - bundled: scripts/
+ path: /tmp/scripts/
permission: "775"
# Contains information about allocated nodes
- meta: experiment-info.json
execute:
+ # Clear previous uploads and move new ones to ~/
+ # This step is necessary since dir uploads are not overriden
+ # which results in an error otherwise
+ - direct: |
+ #!/bin/bash
+ rm -rf /root/{scripts,.env,nodes.json} || true
+ mv /tmp/{scripts,.env} ~/
+ rm -rf ~/'execute[0].sh'
+ # Reset possible DNS issues created by our scripts
+ mv /etc/resolv.conf.bak /etc/resolv.conf || true
+ log: /dev/null
# Setup DNS and Environment config
- path: ~/scripts/setup.sh
- - path: ~/scripts/dns.sh
- nodes: [DNS]
# Execute node specific code
- path: ~/scripts/run.sh
diff --git a/experiment/infra.rspec b/experiment/infra.rspec
deleted file mode 100644
index b6a8e6d..0000000
--- a/experiment/infra.rspec
+++ /dev/null
@@ -1,50 +0,0 @@
-<?xml version='1.0'?>
-<rspec xmlns="http://www.geni.net/resources/rspec/3"; type="request"
generated_by="jFed RSpec Editor" generated="2021-11-22T19:20:29.068+01:00"
xmlns:emulab="http://www.protogeni.net/resources/rspec/ext/emulab/1";
xmlns:delay="http://www.protogeni.net/resources/rspec/ext/delay/1";
xmlns:jfed-command="http://jfed.iminds.be/rspec/ext/jfed-command/1";
xmlns:client="http://www.protogeni.net/resources/rspec/ext/client/1";
xmlns:jfed-ssh-keys="http://jfed.iminds.be/rspec/ext/jfed-ssh-keys/1"; xmlns:
[...]
- <node client_id="DB" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
- <sliver_type name="raw-pc">
- <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
- </sliver_type>
- <hardware_type name="grvingt-nancy"/>
- <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="156.0" y="70"/>
- </node>
- <node client_id="Exchange" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
- <sliver_type name="raw-pc">
- <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
- </sliver_type>
- <hardware_type name="grvingt-nancy"/>
- <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="283.0"
y="127.5"/>
- </node>
- <node client_id="Bank" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
- <sliver_type name="raw-pc">
- <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
- </sliver_type>
- <hardware_type name="grvingt-nancy"/>
- <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="422.0" y="70"/>
- </node>
- <node client_id="Proxy" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
- <sliver_type name="raw-pc">
- <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
- </sliver_type>
- <hardware_type name="grvingt-nancy"/>
- <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="284.5"
y="184.5"/>
- </node>
- <node client_id="Monitor" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
- <sliver_type name="raw-pc">
- <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
- </sliver_type>
- <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="732.5"
y="156.5"/>
- </node>
- <node client_id="Merchant" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
- <sliver_type name="raw-pc">
- <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
- </sliver_type>
- <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="554.5"
y="156.5"/>
- </node>
- <node client_id="DNS" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
- <sliver_type name="raw-pc">
- <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
- </sliver_type>
- <hardware_type name="grvingt-nancy"/>
- <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="638.0" y="70"/>
- </node>
-</rspec>
\ No newline at end of file
diff --git a/experiment/scripts/bank.sh b/experiment/scripts/bank.sh
index 4723a8c..feabff3 100755
--- a/experiment/scripts/bank.sh
+++ b/experiment/scripts/bank.sh
@@ -1,10 +1,30 @@
#!/bin/bash
-set -eux
+INFO_MSG="
+Setup the bank node for the experiments
+(Start the taler-fakebank)
+"
+OPT_MSG="
+init:
+ Configure and start the taler-fakebank
+"
+set -eux
source ~/scripts/helpers.sh
-restart_rsyslog
+# Start the taler-fakebank
+function init_bank() {
+ restart_rsyslog
+
+ systemctl restart taler-fakebank.service
+}
-systemctl restart taler-fakebank.service
+case $1 in
+ init)
+ init_bank
+ ;;
+ *)
+ taler_perf_help $0 "$INFO_MSG" "$OPT_MSG"
+ ;;
+esac
exit 0
diff --git a/experiment/scripts/benchmark.sh b/experiment/scripts/benchmark.sh
index cf1aad3..f2cb7b2 100755
--- a/experiment/scripts/benchmark.sh
+++ b/experiment/scripts/benchmark.sh
@@ -1,23 +1,47 @@
#!/bin/bash
+INFO_MSG="
+Start a wallet benchmark loop with 100'000 iterations
+"
+OPT_MSG="
+<N>:
+ Any number
+ If it is dividable by 10 then the wallet will log in INFO level
+"
-LOG_LEVEL=ERROR
+set -eu
+source ~/scripts/helpers.sh
+
+# Start a wallet benchmark loop
+function start_wallet_bench() {
+ LOG_LEVEL=ERROR
+
+ # Every thenth wallet should log messages
+ if ! (($1 % 10)) || [ $1 == "1" ]; then
+ LOG_LEVEL=INFO
+ fi
+
+ taler-wallet-cli \
+ -L ${LOG_LEVEL} \
+ advanced bench1 \
+ --config-json "
+ {
+ \"exchange\": \"http://${PROXY_DOMAIN}/\";,
+ \"bank\": \"http://${BANK_DOMAIN}/\";,
+ \"currency\": \"KUDOS\",
+ \"payto\": \"payto://x-taler-bank/${BANK_DOMAIN}/foo\",
+ \"iterations\": 100000,
+ \"deposits\": 10,
+ \"restartAfter\": 2
+ }" | grep -v getRecords
+}
case $1 in
- *logging*)
- LOG_LEVEL=INFO
- ;;
+ [0-9]*)
+ start_wallet_bench $1
+ ;;
+ *)
+ taler_perf_help $0 "$INFO_MSG" "$OPT_MSG"
+ ;;
esac
-taler-wallet-cli \
- -L ${LOG_LEVEL} \
- advanced bench1 \
- --config-json "
-{
- \"exchange\": \"http://${PROXY_DOMAIN}/\";,
- \"bank\": \"http://${BANK_DOMAIN}/\";,
- \"currency\": \"KUDOS\",
- \"payto\": \"payto://x-taler-bank/${BANK_DOMAIN}/foo\",
- \"iterations\": 100000,
- \"deposits\": 10,
- \"restartAfter\": 2
-}"
+exit 0
diff --git a/experiment/scripts/createusers.sh
b/experiment/scripts/createusers.sh
index a7de2a7..a5b3705 100755
--- a/experiment/scripts/createusers.sh
+++ b/experiment/scripts/createusers.sh
@@ -1,9 +1,15 @@
#!/bin/bash
+# Create all necessary users and groups
+# for the taler applications.
+# (normaly done automatically when installing from packages)
+#
+# Usage: ./createusers.sh
set -e
-. /usr/share/debconf/confmodule
+source /usr/share/debconf/confmodule
TALER_HOME="/var/lib/taler"
+
GROUPNAME=taler-exchange-secmod
DBGROUPNAME=taler-exchange-db
EUSERNAME=taler-exchange-httpd
diff --git a/experiment/scripts/database.sh b/experiment/scripts/database.sh
index 5bd2472..4b1c4eb 100755
--- a/experiment/scripts/database.sh
+++ b/experiment/scripts/database.sh
@@ -1,44 +1,149 @@
#!/bin/bash
-set -eux
+INFO_MSG="
+Setup the database node (start postgresql)
+"
+OPT_MSG="
+init:
+ Initialize and start the taler database
+"
+set -eux
source ~/scripts/helpers.sh
-# move to tmp to prevent change dir errors
+# move to tmp to prevent change directory errors
cd /tmp
-if [[ "$1" == "init" ]];
-then
+function setup_disks() {
+ if [[ "$(hostname)" =~ "dahu" ]]; then
+ mkdir /mnt/sdb || true
+ mount /dev/sdb /mnt/sdb || true
+ if [ ! -L /var/lib/postgresql/13/main/pg_wal ]; then
+ rm -rf /mnt/sdb/pg_wal || true
+ mv /var/lib/postgresql/13/main/pg_wal/ /mnt/sdb
+ ln -s /mnt/sdb/pg_wal /var/lib/postgresql/13/main/pg_wal
+ chown -R postgres:postgres /var/lib/postgresql/13/main/pg_wal/
+ fi
+ fi
+}
+
+function setup_ram_storage() {
+ SIZE=$(($(awk '/MemTotal/ {print $2}' /proc/meminfo) / 10))
+ if [ ! -L /var/lib/postgresql ]; then
+ mv /var/lib/postgresql /var/lib/postgresql.bak
+ mkdir /var/lib/postgresql
+ chown postgres:postgres /var/lib/postgresql
+ mount -t tmpfs -o size=${SIZE}k pgdata /var/lib/postgresql
+ cp -rp /var/lib/postgresql.bak/* /var/lib/postgresql
+ fi
+}
+
+# Setup the postgresql configuration
+function setup_config() {
sed -i "s\<DB_URL_HERE>\postgresql:///${DB_NAME}\g" \
/etc/taler/secrets/exchange-db.secret.conf
+ # Enable password for taler since this is the case in real world deployments
+ # For the postgres user do not enable authentication (used in metrics)
+ if ! grep -q "host all ${DB_USER} 127.16.0.0/12 md5" \
+ /etc/postgresql/13/main/pg_hba.conf; then
+ echo "
+ host all ${DB_USER} 172.16.0.0/12 md5
+ host all postgres 172.16.0.0/12 trust
+ " >> /etc/postgresql/13/main/pg_hba.conf
+ fi
+
+ # Get hardware info to tune in postgresql.conf
+ SHARED_MEM=$(($(awk '/MemTotal/ {print $2}' /proc/meminfo) / 3 ))
+ CACHE_SIZE=$(($(awk '/MemTotal/ {print $2}' /proc/meminfo) * 3/4))
+ NUM_CPU=$(lscpu | grep "CPU(s)" | head -n 1 | awk '{print $2}')
+
+ # Enable huge pages
+ # Size for huge_pages =~ shared_buffers * 1.25 so that there is enough
+ VM_PEAK=$((${SHARED_MEM} * 10/8))
+
+ HUGE_PAGES_SIZE=$(grep ^Hugepagesize /proc/meminfo | awk '{print $2}')
+ NUM_PAGES=$((${VM_PEAK} / ${HUGE_PAGES_SIZE}))
+
+ if ! grep -q "vm.nr_hugepages'" /etc/sysctl.conf; then
+ echo "vm.nr_hugepages=${NUM_PAGES}" >> /etc/sysctl.conf
+ sysctl -p
+ fi
+
+ # Configure postgres with an additional file and include this
+ # in the main configuration
echo "
listen_addresses='*'
log_destination=syslog
syslog_ident='taler-database'
log_min_duration_statement=500
shared_preload_libraries='pg_stat_statements,auto_explain'
+ auto_explain.log_min_duration='300ms'
+ auto_explain.log_analyze=true
+ auto_explain.log_buffers=true
+ auto_explain.log_wal=true
# use 25% of the available memory
# (https://www.postgresql.org/docs/13/runtime-config-resource.html)
- shared_buffers=$(($(awk '/MemTotal/ {print $2}' /proc/meminfo) / 4 ))kB
- effective_cache_size=$(($(awk '/MemTotal/ {print $2}' /proc/meminfo) *
3/4))kB
+ shared_buffers=${SHARED_MEM}kB
+ effective_cache_size=${CACHE_SIZE}kB
+
+ huge_pages=on
+
#
(https://www.postgresql.org/docs/current/runtime-config-wal.html#GUC-MAX-WAL-SIZE)
- max_wal_size=2GB
- max_worker_processes=16
+ min_wal_size=4GB
+ max_wal_size=8GB
+ wal_buffers=16MB
+
+ # Hopefully reduce disk IO
+ # https://www.postgresql.org/docs/12/wal-configuration.html
+ checkpoint_completion_target=0.9
+ random_page_cost=1.1
+
+ max_worker_processes=${NUM_CPU}
+ max_parallel_workers=${NUM_CPU}
max_connections=500
- " >> /etc/postgresql/13/main/postgresql.conf
+
+ max_parallel_maintenance_workers=4
- # Enable password for taler since this is commonly the case
- # For the postgres user do not enable authentication (used in metrics)
- echo "
- host all ${DB_USER} 172.16.0.0/12 md5
- host all postgres 172.16.0.0/12 trust
- " >> /etc/postgresql/13/main/pg_hba.conf
-fi
+ # out of shared memory
+ max_locks_per_transaction=85
-systemctl restart postgresql
+ # Increase work mem to lower I/O utilization (max used =~ work_mem *
max_connections)
+ # NOTE: This formula is not completely correct
+ work_mem=265MB
+ maintenance_work_mem=2GB
+ # 1 min
+ idle_in_transaction_session_timeout=60000
+ " > /etc/postgresql/13/main/exchange.conf
+
+ if ! grep -q "include = 'exchange.conf'" \
+ /etc/postgresql/13/main/postgresql.conf; then
+ echo "include = 'exchange.conf'" >> \
+ /etc/postgresql/13/main/postgresql.conf
+ fi
+}
-su postgres << EOF
+# Configure and start pgBouncer if $USE_PGBOUNCER is true
+function setup_pgbouncer() {
+ if [ "${USE_PGBOUNCER}" = "true" ]; then
+ sed -i -e "s/<DB_USER_HERE>/${DB_USER}/g" \
+ -e "s/<DB_PASSWORD_HERE>/${DB_PASSWORD}/g" \
+ /etc/pgbouncer/userlist.txt
+ sed -i -e "s/<DB_NAME_HERE>/${DB_NAME}/g" \
+ /etc/pgbouncer/pgbouncer.ini
+ # pgbouncer does not cleanup those sometimes
+ rm -f /var/run/postgresql/pgbouncer.pid
+ rm -f /var/run/postgresql/.s.PGSQL.6432
+ systemctl restart pgbouncer
+ fi
+}
+
+# Initialize the database for taler exchange
+function init_db() {
+ systemctl restart postgresql
+
+ # Create the role taler-exchange-httpd and the database
+ su postgres << EOF
psql postgres -tAc "SELECT 1 FROM pg_roles WHERE
rolname='taler-exchange-httpd'" | \
grep -q 1 || \
createuser taler-exchange-httpd
@@ -46,27 +151,41 @@ psql -tAc "SELECT 1 FROM pg_database WHERE
datname='${DB_NAME}'" | \
grep -q 1 || \
createdb -O taler-exchange-httpd "${DB_NAME}"
EOF
-
-if [[ "$1" == "init" ]];
-then
+
+ sudo -u taler-exchange-httpd taler-exchange-dbinit -r || true
+ sudo -u taler-exchange-httpd taler-exchange-dbinit -s || true
sudo -u taler-exchange-httpd taler-exchange-dbinit
-fi
-
-su postgres << EOF
+
+ # Create the remote user "$DB_USER" and load pg_stat_statements for metrics
+ su postgres << EOF
psql postgres -tAc "SELECT 1 FROM pg_roles WHERE rolname='${DB_USER}'" | \
- grep -q 1 \
- || psql << END
- CREATE USER "${DB_USER}" with encrypted password '${DB_PASSWORD}';
- CREATE EXTENSION pg_stat_statements;
+ grep -q 1 || \
+ psql << END
+ CREATE USER "${DB_USER}" with encrypted password '${DB_PASSWORD}';
+ CREATE EXTENSION pg_stat_statements;
END
EOF
-
-su taler-exchange-httpd -s /bin/bash << EOF
+
+ # Grant access to the databse to the remote user
+ su taler-exchange-httpd -s /bin/bash << EOF
psql -d "${DB_NAME}"
GRANT SELECT,INSERT,UPDATE ON ALL TABLES IN SCHEMA public TO "${DB_USER}";
GRANT USAGE ON ALL SEQUENCES IN SCHEMA public TO "${DB_USER}";
EOF
+}
-restart_rsyslog
+case ${1} in
+ init)
+ setup_config
+ #setup_disks
+ setup_ram_storage
+ init_db
+ setup_pgbouncer
+ restart_rsyslog
+ ;;
+ *)
+ taler_perf_help $0 "$INFO_MSG" "$OPT_MSG"
+ ;;
+esac
exit 0
diff --git a/experiment/scripts/dns.sh b/experiment/scripts/dns.sh
deleted file mode 100644
index 06c01ff..0000000
--- a/experiment/scripts/dns.sh
+++ /dev/null
@@ -1,12 +0,0 @@
-#!/bin/bash
-set -eux
-
-# Stop dnsmasq just to be save
-systemctl stop dnsmasq
-systemctl restart named
-
-# Make sure the dns is ready
-sleep 5
-
-exit 0
-
diff --git a/experiment/scripts/exchange.sh b/experiment/scripts/exchange.sh
index c27399c..7277211 100755
--- a/experiment/scripts/exchange.sh
+++ b/experiment/scripts/exchange.sh
@@ -1,12 +1,32 @@
#!/bin/bash
+INFO_MSG="
+Setup the Exchange node
+Start taler-exchnage-*
+-----
+ could change later to single nodes
+ for aggregator, wirewatch etc..
+-----
+Each exchange-http daemon, will get its own port starting from 10001,
+unless the first, this one will get port 80
+"
+OPT_MSG="
+init:
+ Initialize the applications and start them
+ uses NUM_EXCHANGE_PROCESSES and NUM_WIREWATCH_PROCESSES
+
+start NUM:
+ Start another NUM exchange-http daemons
+
+stop NUM:
+ Stop NUM exchange-httpd daemons
+"
+
set -eux
+source ~/scripts/helpers.sh
-if [[ "$1" == "init" ]];
-then
- source ~/scripts/helpers.sh
- restart_rsyslog
-
- sed -i
"s\<DB_URL_HERE>\postgresql://${DB_USER}:${DB_PASSWORD}@${DATABASE_DOMAIN}/${DB_NAME}\g"
\
+# Setup the configuration in /etc/taler
+function setup_config() {
+ sed -i
"s\<DB_URL_HERE>\postgresql://${DB_USER}:${DB_PASSWORD}@${DATABASE_DOMAIN}:${DB_PORT}/${DB_NAME}\g"
\
/etc/taler/secrets/exchange-db.secret.conf
sed -i "s/<BANK_HOST_HERE>/${BANK_DOMAIN}/g" \
@@ -17,14 +37,12 @@ then
sed -i -e "s/<BANK_HOST_HERE>/${BANK_DOMAIN}/g" \
-e "s\<BASE_URL_HERE>\http://${EXCHANGE_DOMAIN}/\g"; \
-e "s/<MASTER_KEY_HERE>/${MASTER_KEY}/g" \
+ -e "s/<MAX_REQUESTS_HERE>/${EXCHANGE_MAX_REQUESTS:-8192}/g" \
/etc/taler/conf.d/exchange-business.conf
-
- wait_for_db
- # Wait another second to make sure user has permissions
- sleep 5
-
- NUM_PROCESSES=$((${NUM_EXCHANGE_PROCESSES:-10}-1))
+}
+# Setup the exchange with the taler-exchange-offline signing procedure
+function setup_exchange() {
systemctl restart taler-exchange.target
wait_for_keys "${EXCHANGE_DOMAIN}/management"
@@ -38,25 +56,73 @@ then
taler-exchange-offline upload < sig-res.json
taler-exchange-offline upload < acct-res.json
taler-exchange-offline upload < fee-res.json
+}
- # || true is needed since when 1-1 is run let returns an error
- let "NUM_WIREWATCH_PROCESSES-=1" || true
- for i in $(seq ${NUM_WIREWATCH_PROCESSES});
- do
- systemctl restart taler-exchange-wirewatch@${i}.service
- done
+# Initialize all stuff needed
+# logs, configs, exchanges
+function init_exchanges() {
+ restart_rsyslog
-else
- NUM_PROCESSES=$1
-fi
+ setup_config
-RUNNING=$(ps -aux | grep "[taler]-exchange-httpd" | wc -l)
-
-for i in $(seq ${NUM_PROCESSES});
-do
- let "i+=${RUNNING}-1" || true
- let "i+=10000"
- systemctl restart taler-exchange-httpd@"${i}".socket
taler-exchange-httpd@"${i}".service
-done
+ wait_for_db
+ # Wait another second to make sure user has permissions
+ sleep 5
+
+ setup_exchange
+}
+
+# Start N new exchange-http daemons
+# $1: N - number of new exchanges to start
+function start_exchanges() {
+ # Get all currently running ones so the next free
+ # port can be calculated - 10000 + RUNNING
+ RUNNING=$(ps -aux | grep "[taler]-exchange-httpd" | wc -l)
+
+ # We cant do seq 0 n, if n=0 it would yield 0, thus do seq n and decrement
+ # by one, then seq 0 yields nothing
+ for i in $(seq ${1}); do
+ # seq starts at 1 - substract it
+ let "i+=${RUNNING}-1" || true
+ let "i+=10000"
+ systemctl restart taler-exchange-httpd@"${i}".socket \
+ taler-exchange-httpd@"${i}".service
+ # Wait so they have some small delay in between their routines from the
start
+ sleep 0.05
+ done
+}
+
+# Start N new exchange-wirewatch processes
+# $1: N - number of new wirewatchers to start
+# NOTE: only for init purposes currently
+function start_wirewatchers() {
+ for i in $(seq ${1}); do
+ systemctl restart taler-exchange-wirewatch@"${i}".service
+ done
+}
+
+# Stop N exchange daemons
+# $1: N -- number of exchanges to stop
+function stop_exchanges() {
+ stop_numbered_services "taler-exchange-httpd" $1
+}
+
+case $1 in
+ init)
+ init_exchanges
+ # TODO adjust when started manually and not with .target
+ start_exchanges "$((${NUM_EXCHANGE_PROCESSES}-1))"
+ start_wirewatchers "$((${NUM_WIREWATCH_PROCESSES}-1))"
+ ;;
+ start)
+ start_exchanges $2
+ ;;
+ stop)
+ stop_exchanges $2
+ ;;
+ *)
+ taler_perf_help $0 "$INFO_MSG" "$OPT_MSG"
+ ;;
+esac
exit 0
diff --git a/experiment/scripts/helpers.sh b/experiment/scripts/helpers.sh
index 9cc0351..413043e 100755
--- a/experiment/scripts/helpers.sh
+++ b/experiment/scripts/helpers.sh
@@ -1,10 +1,84 @@
#/!bin/bash
+# Useful functions which are neeeded across
+# multiple scripts
+#
+# Usage: source ./helpers.sh
+# Disable verbose output when loading .env so secret
+# Variables are not in logs unecessarily
+set +x
+source ~/.env
+set -x
+
+# Set a dynamic domain name in our own dns
+# IP address will be resolved on the node this function
+# is executed on
+# $1: The domain to be added
+function set_ddn() {
+ # There are still issues sometimes
+ # hostname -i ... failure syntax error
+ # Even when looping with while ! hostname -i the next one
+ # possibly may fail. Thus we try to fix it with eval
+ while ! eval $(echo IP=$(hostname -i)) ; do
+ sleep 5;
+ systemctl restart dnsmasq
+ done
+ nsupdate -v << EOF
+server ${DNS_HOSTS}
+zone ${DNS_ZONE}
+update add ${1} 3600 A ${IP}
+send
+EOF
+}
+
+# Set the host (role) which the node executing this
+# function is assigned to in the experiments
+# This can later be used to setup log directories per role or in
+# log entries - such as net-delay (ping.sh) for example
+function set_host() {
+ # Add it to all sources and export it to the current shell
+ echo "TALER_HOST=${1}" >> /etc/environment
+ echo "TALER_HOST=${1}" >> /root/.env
+ export "TALER_HOST=${1}"
+}
+
+# Enable the service which logs round trip times between nodes
+# $1: Destination to ping
+# NOTE: currenly only one destination is supported
+function enable_netdelay() {
+ sed -i "s/<PING_DESTINATION>/${1}/g" \
+ /usr/lib/systemd/system/taler-netdelay.service
+ systemctl daemon-reload
+ systemctl restart taler-netdelay.timer
+}
+
+# Setup the log directiories in the Grid5000 shared home directory (NFS)
+function setup_log() {
+ HOST_LOG_DIR=${LOG_DIR}/${TALER_HOST}
+ # || true is sill needed when e.g. wallets want to create the same
+ # directory at the same time
+ test -d ${HOST_LOG_DIR} || mkdir ${HOST_LOG_DIR} || true
+ # Send all logs about taler to promtail on the monitoring node
+ sed -i -e "s/<MONITOR_DOMAIN_HERE>/${MONITOR_DOMAIN}/g" \
+ -e "s|<LOG_DIR_HERE>|${HOST_LOG_DIR}|g" \
+ /etc/rsyslog.d/taler.conf
+ # Enable log rotating for all logs in the host log dir on NFS
+ sed -i "s|<LOG_DIR_HERE>|${HOST_LOG_DIR}|g" \
+ /etc/logrotate.d/taler
+}
+
+# Enable the logbackup (NFS) - logrotation service
+function enable_logrotate() {
+ systemctl restart taler-logrotate.timer
+}
+
+# Wait for the database to be acessible by $DB_USER from remote
function wait_for_db() {
until PGPASSWORD="${DB_PASSWORD}" psql \
-h "${DATABASE_DOMAIN}" \
-U "${DB_USER}" \
-d "${DB_NAME}" \
+ -p "${DB_PORT}" \
-c '\q';
do
echo "Database not ready yet"
@@ -12,8 +86,8 @@ function wait_for_db() {
done
}
-# Arguments
-# 1: Domain to request /keys from
+# Wait until the exchange is ready to present key materials
+# $1: Domain to request /keys from (exch.perf.taler</management>)
function wait_for_keys() {
until wget http://${1}/keys \
--spider \
@@ -26,22 +100,77 @@ function wait_for_keys() {
done
}
+# Restart (enable) the rsyslog to send the logs to the monitoring
+# Node and to the shared log dir (NFS)
function restart_rsyslog() {
# rsyslg fails to apply the taler rule if remote is not reachable
while ! nc -z ${MONITOR_DOMAIN} 1514;
do
echo "Waiting for promtail"
- sleep 2
+ # There are issues when dnsmasq tries to resolve before the service
+ # is ready, when it gets restarted it works
+ systemctl restart dnsmasq
+ sleep 5
done
systemctl restart rsyslog
}
-function get_wallet_domains() {
+# Stop services which have the form name@number.service
+# will stop the services starting from the biggest numbers
+# $1: service base name, e.g. taler-exchange-httpd
+# $2: amount of services to stop
+function stop_numbered_services() {
+ # Get all running processes of the service $1,
+ # extract their number, limit by number of
+ # to stop and stop those
+ N=$(\
+ systemctl status "${1}"@*.service | \
+ grep -E "${1}@[0-9]+.service -" | \
+ awk '{print $2}' | \
+ sed 's/[^0-9]*//g' | \
+ sort -n -r | \
+ head -n $2 \
+ )
+ for i in ${N}; do
+ systemctl stop ${1}@${i}.service
+ done
+}
+
+# Get all Grid5000 hosts which host wallets
+# Returns only the Grid5000 node - e.g. graoully-1
+function get_wallet_hosts() {
IFS=$'\n' read -r -d '' -a WALLETS < <(\
- dig -t AXFR "${DNS_ZONE}" "@${DNS_HOST}" \
- | grep wallet | awk '{print $1}' | cut -d '.' -f 2 \
+ dig -t AXFR "${DNS_ZONE}" "@${DNS_HOSTS}" \
+ | grep ${WALLET_DOMAIN} | awk '{print $1}' | cut -d '.' -f 2 \
)
echo ${WALLETS[@]}
}
+# Get all exchanges which are registered in the nginx proxy config
+# Returns the whole domain - e.g. exchange.perf.taler:80
+function get_exchanges() {
+ IFS=$'\n' read -r -d '' -a EXCHANGES < <(\
+ ssh -o StrictHostKeyChecking=no ${PROXY_DOMAIN} \
+ grep -E "^[[:space:]]*server[[:space:]]exch"
/etc/nginx/sites-enabled/proxy \
+ | cut -d ";" -f 1 | cut -d " " -f 4 \
+ )
+ echo ${EXCHANGES[@]}
+}
+
+# Display a help message and exit
+# $1: script name to display help for
+# $2: info message about the script to display
+# $3: option description to display
+function taler_perf_help() {
+ set +x
+ echo "=================== Taler Performance Help ===================="
+ echo "$2"
+ echo "Usage: $1 OPTIONS"
+ echo ""
+ echo "OPTIONS"
+ echo ""
+ echo "$3"
+ echo "==============================================================="
+ exit 2
+}
diff --git a/experiment/scripts/install.sh b/experiment/scripts/install.sh
new file mode 100755
index 0000000..86d8ed6
--- /dev/null
+++ b/experiment/scripts/install.sh
@@ -0,0 +1,74 @@
+#!/bin/bash
+# Rebuild the taler binaries from source
+# Requires the following optional variables to be set,
+# if not set the corresponding repo will not be rebuilt.
+# <GUNET|EXCHANGE|MERCHANT|WALLET>_COMMIT_SHA
+
+TALER_HOME=~/taler
+
+# Prepare the repository
+# $1: Git repo to clone
+# $2: Commit to checkout to
+function prepare() {
+ DIR="${TALER_HOME}/$(basename ${1%.*})"
+ test -d "${DIR}" || git clone "${1}" "${DIR}"
+ cd "${DIR}"
+ git checkout master > /dev/null && \
+ (git pull > /dev/null 2>&1 || true)
+ git checkout "$2" > /dev/null && \
+ (git pull > /dev/null 2>&1 || true)
+}
+
+# Build the binaries in the current directory with make
+# (runs ./bootstrap & ./configure)
+function build() {
+ echo "INFO running bootstrap and configure"
+ ./bootstrap
+ if [ -f contrib/gana.sh ]; then
+ ./contrib/gana.sh
+ fi
+ ./configure --enable-logging=verbose --prefix=/usr || ./configure
+ make
+}
+
+# Install from a git repo
+# $1: Git repo to clone
+# $2: Commit to checkout to
+function install() {
+ prepare "$1" "$2"
+ build
+ echo "INFO installing"
+ make install
+ ldconfig
+}
+
+if [ ! -d "${TALER_HOME}" ]; then
+ mkdir "${TALER_HOME}"
+fi
+
+# Use ! -z since -n would be false for ""
+if [ ! -z "${GNUNET_COMMIT_SHA}" ]; then
+ echo "INFO installing GNUnet"
+ install "https://git.gnunet.org/gnunet.git"; \
+ "${GNUNET_COMMIT_SHA:-master}"
+fi
+
+if [ ! -z "${EXCHANGE_COMMIT_SHA}" ]; then
+ echo "INFO installing Taler Exchange"
+ install "https://git.taler.net/exchange.git"; \
+ "${EXCHANGE_COMMIT_SHA:-master}"
+fi
+
+if [ ! -z "${MERCHANT_COMMIT_SHA}" ]; then
+ echo "INFO installing Taler Merchant"
+ install "https://git.taler.net/merchant.git"; \
+ "${MERCHANT_COMMIT_SHA:-master}"
+fi
+
+if [ ! -z "${WALLET_COMMIT_SHA}" ]; then
+ echo "INFO installing Taler Wallet"
+ install "https://git.taler.net/wallet-core.git"; \
+ "${WALLET_COMMIT_SHA:-master}"
+fi
+
+exit 0
diff --git a/experiment/scripts/merchant.sh b/experiment/scripts/merchant.sh
index ceeaa2e..62d1ba3 100755
--- a/experiment/scripts/merchant.sh
+++ b/experiment/scripts/merchant.sh
@@ -1,14 +1,21 @@
#!/bin/bash
set -eux
-
source ~/scripts/helpers.sh
-cd /tmp
+INFO_MSG="
+Setup the merchant node
+(Start taler-merchant-httpd)
+"
+OPT_MSG="
+init:
+ Configure and start the merchant together with its database
+"
-restart_rsyslog
+# Prevent change directory errors
+cd /tmp
-if [[ "$1" == "init" ]];
-then
+# Configurre the merchants files in /etc/taler
+function configure_merchant() {
wait_for_keys "${PROXY_DOMAIN}"
MASTER_KEY=$(
@@ -20,35 +27,56 @@ then
sed -i -e "s\<EXCHANGE_URL_HERE>\http://${EXCHANGE_DOMAIN}/\g"; \
-e "s/<EXCHANGE_MASTER_KEY_HERE>/${MASTER_KEY}/g" \
/etc/taler/conf.d/merchant.conf
+}
+
+# Setup the merchants db on the same host
+function configure_db() {
+ # Allow the remote hosts (monitor) access with the postgres user
+ echo "
+ host all postgres 172.16.0.0/12 trust
+ " >> /etc/postgresql/13/main/pg_hba.conf
- sed -i "s/local\s* all\s* postgres\s* peer/local all postgres trust/g" \
- /etc/postgresql/13/main/pg_hba.conf
-
+ # Listen on all interfaces so the monitors db exporter can reach the db
echo "
+ listen_addresses='*'
shared_preload_libraries='pg_stat_statements,auto_explain'
" >> /etc/postgresql/13/main/postgresql.conf
-fi
+}
+
+# Start postgres an initialize the merchant's database
+function init_db() {
+ systemctl restart postgresql
-systemctl restart postgresql
-
-su postgres << EOF
-psql postgres -tAc "SELECT 1 FROM pg_roles WHERE
rolname='taler-merchant-httpd'" | \
- grep -q 1 || \
- createuser taler-merchant-httpd
-psql -tAc "SELECT 1 FROM pg_database WHERE datname='taler-merchant'" | \
- grep -q 1 || \
- createdb -O taler-merchant-httpd taler-merchant
-psql
-CREATE EXTENSION pg_stat_statements;
+ su postgres << EOF
+ psql postgres -tAc "SELECT 1 FROM pg_roles WHERE
rolname='taler-merchant-httpd'" | \
+ grep -q 1 || \
+ createuser taler-merchant-httpd
+ psql -tAc "SELECT 1 FROM pg_database WHERE datname='taler-merchant'" | \
+ grep -q 1 || \
+ createdb -O taler-merchant-httpd taler-merchant
+ psql
+ CREATE EXTENSION pg_stat_statements;
EOF
-if [[ "$1" == "init" ]];
-then
sudo -u taler-merchant-httpd taler-merchant-dbinit
-fi
+}
+
+# Initialize the merchant
+function init_merchant() {
+ restart_rsyslog
+ configure_merchant
+ configure_db
+ init_db
+ systemctl restart taler-merchant-httpd
+}
-sed -i "s\<DB_URL_HERE>\user=postgres host=/run/postgresql
dbname=taler-merchant\g" \
- /etc/default/prometheus-postgres-exporter
+case $1 in
+ init)
+ init_merchant
+ ;;
+ *)
+ taler_perf_help $0 "$INFO_MSG" "$OPT_MSG"
+ ;;
+esac
-systemctl restart taler-merchant-httpd \
- prometheus-postgres-exporter
+exit 0
diff --git a/experiment/scripts/monitor.sh b/experiment/scripts/monitor.sh
index f003a32..292b4ff 100755
--- a/experiment/scripts/monitor.sh
+++ b/experiment/scripts/monitor.sh
@@ -1,11 +1,48 @@
#!/bin/bash
-set -ex
+INFO_MSG="
+Setup and start the experiment monitoring infrastructure such as
+Loki, Promtail, Prometheus and various Prometheus exporters
+"
+OPT_MSG="
+init
+ Initialize and start all services
+
+start:
+ Automatically update wallet / exchange nodes which are monitored
+ (does add but not remove)
+stop-exchanges NUM:
+ Stop NUM exchanges from being monitored
+"
+
+set -ex
source ~/scripts/helpers.sh
-source ~/.env
-if [[ "$1" == "init" ]];
-then
+# Update a data source on the external grafana instance
+# $1: Data-source name (configured in .env)
+# $2: Grid5000 external url for the node where the data source is hosted
+# - https://www.grid5000.fr/w/HTTP/HTTPs_access
+function update_datasource() {
+ ID=$(jq --arg name "$1" '.[] | select(.name == $name) | .id' ds.json)
+
+ jq --arg url "https://$(hostname | cut -d "." -f 1,2
-).$2.proxy.grid5000.fr" \
+ --arg name "$1" \
+ '.[] | select(.name == $name) | .url = $url' \
+ ds.json | tee /dev/tty | curl -X PUT -k -f -d @- \
+ -H "${AUTH_HEADER}" \
+ -H "Content-Type: application/json" \
+ -H "Accept: application/json" \
+ "${GRAFANA_API}/datasources/${ID}"
+}
+
+# Update the external grafana instance and tell it
+# about the nodes which should be queried
+# If GRAFANA_HOST or GRAFANA_API_KEY are empty this
+# step is skipped - requires admin level api key to update data sources
+function update_grafana() {
+ if [ -z "${GRAFANA_HOST}" ] || [ -z ${GRAFANA_API_KEY} ]; then
+ return
+ fi
AUTH_HEADER="Authorization: Bearer ${GRAFANA_API_KEY}"
GRAFANA_API="${GRAFANA_HOST}/api"
@@ -17,66 +54,125 @@ then
exit $?
fi
- function update_datasource() {
- ID=$(jq --arg name "$1" '.[] | select(.name == $name) | .id' ds.json)
-
- jq --arg url "https://$(hostname | cut -d "." -f 1,2
-).$2.proxy.grid5000.fr" \
- --arg name "$1" \
- '.[] | select(.name == $name) | .url = $url' \
- ds.json | tee /dev/tty | curl -X PUT -k -f -d @- \
- -H "${AUTH_HEADER}" \
- -H "Content-Type: application/json" \
- -H "Accept: application/json" \
- "${GRAFANA_API}/datasources/${ID}"
- }
-
update_datasource "${PROMETHEUS_DATASOURCE_NAME}"
"${PROMETHEUS_G5K_PROXY_PORT}"
update_datasource "${LOKI_DATASOURCE_NAME}" "${LOKI_G5K_PROXY_PORT}"
-
- systemctl restart loki \
- promtail
-
+}
+
+# Configure all exporters which run on this host
+function configure_prometheus_and_exporters() {
sed -i "s/<MERCHANT_HOST_HERE>/${MERCHANT_DOMAIN}/g" \
/etc/monitor/prometheus.yaml
- sed -i "s\<DB_URL_HERE>\postgresql://postgres@${DATABASE_DOMAIN}:5432\g" \
- /etc/default/prometheus-postgres-exporter
+ sed -i -e
"s\<EXCHANGE_DB_URL_HERE>\postgresql://postgres@${DATABASE_DOMAIN}:5432\g" \
+ -e
"s\<MERCHANT_DB_URL_HERE>\postgresql://postgres@${MERCHANT_DOMAIN}:5432\g" \
+ /etc/default/prometheus-postgres-exporter
sed -i "s\<PROXY_URL_HERE>\http://${PROXY_DOMAIN}/stub_status\g"; \
- /etc/default/prometheus-nginx-exporter
-
- wait_for_db
-
- # Initialize prometheus after the db is ready, then all dns records have
been set for sure
- if [[ "${ENABLE_EXPORTERS}" == "true" ]];
- then
- cat /etc/monitor/node-exporters.yaml.tpl >> /etc/monitor/prometheus.yaml
- for WALLET in $(get_wallet_domains);
- do
- sed -i "/<WALLETS_HERE>/a \ \ \ \ \ \ -
'wallet.${WALLET}.perf.taler:9100'" \
- /etc/monitor/prometheus.yaml
+ /etc/default/prometheus-nginx-exporter
+}
+
+# Add wallet nodes to be monitored
+# Requires no argument since wallet nodes are retrieved from the DNS
+function add_wallet_nodes_to_prometheus() {
+ for WALLET in $(get_wallet_hosts); do
+ if ! grep -q "${WALLET_DOMAIN//\*/${WALLET}}:9100"
/etc/monitor/prometheus.yaml;
+ then
+ sed -i "/<WALLETS_HERE>/a \ \ \ \ \ \ -
'${WALLET_DOMAIN//\*/${WALLET}}:9100'" \
+ /etc/monitor/prometheus.yaml
+ fi
+ done
+}
+
+# Add new exchanges to be monitored
+# Requires no argument since exchanges are retrieved from the proxy
+function add_exchanges_to_prometheus() {
+ if ! grep -q "${EXCHANGE_DOMAIN}:80" /etc/monitor/prometheus.yaml; then
+ # unfortunately we do not have access to the other hosts when running from
ESpec
+ # But when initializing there are no hosts listed in the yaml yet, thus
+ # it's sufficient to search for the exchange from the target
+ sed -i "/<EXCHANGES_HERE>/a \ \ \ \ \ \ - '${EXCHANGE_DOMAIN}:80'" \
+ /etc/monitor/prometheus.yaml
+ for i in $(seq $((${NUM_EXCHANGE_PROCESSES:-10}-1)) ); do
+ let "i+=10000"
+ sed -i "/<EXCHANGES_HERE>/a \ \ \ \ \ \ - '${EXCHANGE_DOMAIN}:${i}'" \
+ /etc/monitor/prometheus.yaml
done
+ return
fi
+ for EXCH in $(get_exchanges); do
+ if ! grep -q "${EXCH}" /etc/monitor/prometheus.yaml;
+ then
+ sed -i "/<EXCHANGES_HERE>/a \ \ \ \ \ \ - '${EXCH}'" \
+ /etc/monitor/prometheus.yaml
+ fi
+ done
+}
+
+# Remove N exchanges which have been stopped
+# from the config so they will not be scraped anymore
+# $1: N - number of exchanges to remove
+function remove_exchanges_from_prometheus() {
+ TO_STOP=$(\
+ get_exchanges | \
+ tr " " "\n" | \
+ sort -t ":" -k 2 -n -r | \
+ head -n ${1}\
+ )
+
+ for EXCH in ${TO_STOP}; do
+ sed -i "/${EXCH}/d" \
+ /etc/monitor/prometheus.yaml
+ done
+}
+
+# Initialize all applications needed for the
+# observation of the experiments
+function init_monitor() {
+
+ update_grafana
+
+ systemctl restart loki \
+ promtail
+
+ configure_prometheus_and_exporters
- # Proxy takes longer to start
wait_for_keys "${PROXY_DOMAIN}"
- systemctl restart prometheus-postgres-exporter \
- prometheus-nginx-exporter
-else
-
- if [[ "${ENABLE_EXPORTERS}" == "true" ]];
- then
- for WALLET in $(get_wallet_domains);
- do
- if ! grep -q "wallet.${WALLET}.perf.taler:9100"
/etc/monitor/prometheus.yaml;
- then
- sed -i "/<WALLETS_HERE>/a \ \ \ \ \ \ -
'wallet.${WALLET}.perf.taler:9100'" \
- /etc/monitor/prometheus.yaml
- fi
- done
+ if [[ "${ENABLE_EXPORTERS}" == "true" ]]; then
+ if ! grep -q "job_name: 'nodes'" /etc/monitor/prometheus.yaml; then
+ cat /etc/monitor/node-exporters.yaml.tpl | \
+ envsubst >> /etc/monitor/prometheus.yaml
+ add_wallet_nodes_to_prometheus
+ fi
fi
-fi
+ if ! grep -q "job_name: 'taler'" /etc/monitor/prometheus.yaml; then
+ cat /etc/monitor/exchange-exporters.yaml.tpl >>
/etc/monitor/prometheus.yaml
+ add_exchanges_to_prometheus
+ fi
+
+ systemctl restart prometheus-nginx-exporter \
+ prometheus-postgres-exporter \
+ prometheus
+
+ exit 0
+}
+
+case $1 in
+ init)
+ init_monitor
+ ;;
+ start)
+ add_wallet_nodes_to_prometheus
+ add_exchanges_to_prometheus
+ ;;
+ stop-exchanges)
+ remove_exchanges_from_prometheus $2
+ ;;
+ *)
+ taler_perf_help $0 "$INFO_MSG" "$OPT_MSG"
+ ;;
+esac
-systemctl restart prometheus
+systemctl reload prometheus
+exit 0
diff --git a/experiment/scripts/ping.sh b/experiment/scripts/ping.sh
index 9f82ca0..a766a7d 100755
--- a/experiment/scripts/ping.sh
+++ b/experiment/scripts/ping.sh
@@ -1,13 +1,31 @@
#!/bin/bash
+INFO_MSG="
+Measure and log round-trip-times between experiment nodes
+Logs to promtail directly
+"
+OPT_MSG="
+<DEST>:
+ Destination host to ping
+"
-DELAY=$(ping -c 2 $1 | sed -n "3p" | awk '{print $(NF-1)$NF}')
+source ~/scripts/helpers.sh
-if [[ "$DELAY" == "time="* ]];
+if [ -z "${1}" ]; then
+ taler_perf_help $0 "$INFO_MSG" "$OPT_MSG"
+fi
+
+RTT=$(\
+ ping -c 2 ${1} | \
+ sed -n "3p" | \
+ awk '{print $(NF-1)$NF}' \
+)
+
+if [[ "${RTT}" == "time="* ]];
then
logger -s --tcp \
--port 1514 \
--server ${MONITOR_DOMAIN} \
--tag taler-network \
- "src=${TALER_HOST} dst=${1} ${DELAY}" \
+ "src=${TALER_HOST} dst=${1} ${RTT}" \
|| true # Ignore errors (mostly because NXDOMAIN)
fi
diff --git a/experiment/scripts/proxy.sh b/experiment/scripts/proxy.sh
index 5e04919..0c13038 100755
--- a/experiment/scripts/proxy.sh
+++ b/experiment/scripts/proxy.sh
@@ -1,28 +1,58 @@
#!/bin/bash
+INFO_MSG="
+Confiure and start the nginx proxy server
+"
+OPT_MSG="
+init:
+ Initialize and start the proxy with NUM_EXCHANGE_PROCESSES upstreams
+
+start NUM:
+ Add another NUM exchanges to the list of upstreams
+
+stop NUM:
+ Remove NUM exchanges from the list of upstreams
+"
+
set -eux
+source ~/scripts/helpers.sh
-if [[ "$1" == "init" ]];
-then
- source ~/scripts/helpers.sh
+# Add N exchanges to the upstream servers in the proxy configuration
+# $1: Number of exchanges to add
+function add_exchanges() {
+ # Determine the number of exchanges which are already added
+ # and calculate new ports based on that info
+ ADDED=$(\
+ grep -r " server ${EXCHANGE_DOMAIN}:" /etc/nginx/sites-enabled/proxy | \
+ wc -l
+ )
- sed -e "/<SERVERS_HERE>/a \ \ server ${EXCHANGE_DOMAIN}:80;" \
- -e "s/<MONITOR_HOST_HERE>/${MONITOR_DOMAIN}/g" \
- /etc/nginx/sites-available/proxy > /etc/nginx/sites-enabled/proxy
-
- # We want n processes, one is already enabled on port 80
- for (( i=1; i < ${NUM_EXCHANGE_PROCESSES}; i++ ))
- do
- let "PORT=i+10000"
- sed -i "/<SERVERS_HERE>/a \ \ server ${EXCHANGE_DOMAIN}:${PORT};" \
- /etc/nginx/sites-enabled/proxy
+ for i in $(seq ${1}); do
+ if [[ "${ADDED}" -eq "0" ]] && [[ "${i}" -eq "1" ]]; then
+ # The first exchange to add is the default one from the target on port 80
+ i="80"
+ else
+ # seq starts at 1 so remove it again
+ # (0 N cannot be used since seq 0 0 would yield 0 while seq N=0 yields
nothing)
+ let "i+=${ADDED}-1"
+ let "i+=10000"
+ fi
+ sed -i "/<SERVERS_HERE>/a \ \ server ${EXCHANGE_DOMAIN}:${i};" \
+ /etc/nginx/sites-enabled/proxy
done
+}
+
+# Setup the node and proxy configuration
+function setup_config() {
+ add_exchanges ${NUM_EXCHANGE_PROCESSES}
+ # Nginx will log to our rsyslog directly an then from there it will be
+ # redirected to promtail - there was an issue doing it directly that's
+ # why it is done this way
sed -i -e '/module(load="imudp")/s/^#//g' \
-e '/input(type="imudp" port="514")/s/^#//g' \
/etc/rsyslog.conf
- systemctl restart rsyslog
-
+ # Allow enough files to be opened by nginx (www-data)
echo "
fs.file-max=50000
" >> /etc/sysctl.conf
@@ -33,23 +63,50 @@ then
" >> /etc/security/limits.conf
sysctl -p
+}
+
+# Initialize and start the proxy
+function init_proxy() {
+ setup_config
+
+ restart_rsyslog
# Nginx does not start until the destination server is reachable - wait here
# nginx: [emerg] host not found in upstream "exch.perf.taler" ...
wait_for_keys "${EXCHANGE_DOMAIN}"
systemctl restart nginx
-else
+}
- ADDED=$(grep -r " server ${EXCHANGE_DOMAIN}:"
/etc/nginx/sites-enabled/proxy | wc -l)
-
- for i in $(seq $1); do
- let "i+=${ADDED}-1"
- let "i+=10000"
- sed -i "/<SERVERS_HERE>/a \ \ server ${EXCHANGE_DOMAIN}:${i};" \
- /etc/nginx/sites-enabled/proxy
+# Remove N exchanges from the upstream list
+# $1: N - number of exchanges to remove
+function remove_exchanges() {
+ TO_STOP=$(\
+ get_exchanges | \
+ tr " " "\n" | \
+ sort -t ":" -k 2 -n -r | \
+ head -n ${1}\
+ )
+
+ for EXCH in ${TO_STOP}; do
+ sed -i "/${EXCH};/d" \
+ /etc/nginx/sites-enabled/proxy
done
+}
-fi
+case $1 in
+ init)
+ init_proxy
+ ;;
+ start)
+ add_exchanges $2
+ ;;
+ stop)
+ remove_exchanges $2
+ ;;
+ *)
+ taler_perf_help $0 "$INFO_MSG" "$OPT_MSG"
+ ;;
+esac
systemctl reload nginx
diff --git a/experiment/scripts/run.sh b/experiment/scripts/run.sh
index 2f08fb6..69a451e 100644
--- a/experiment/scripts/run.sh
+++ b/experiment/scripts/run.sh
@@ -1,37 +1,9 @@
#!/bin/bash
-# Run the experitment
+# Run the experiment
+# Will determine the role of the node
+# based on its hostname and start role specific scripts
set -euax
-
-# They start the scripts with /bin/bash -c
-source ~/.env
-
-# Set a dynamic domain name in our own dns
-# argument: the domain to be added
-function set_ddn() {
- nsupdate -v << EOF
-server ${DNS_HOST}
-zone ${DNS_ZONE}
-update add $1 3600 A $(hostname -I)
-send
-EOF
-}
-
-function set_host() {
- echo "TALER_HOST=$1" >> /etc/environment
- echo "TALER_HOST=$1" >> /root/.env
- export "TALER_HOST=$1"
-}
-
-function enable_netdelay() {
- sed -i "s/<PING_DESTINATION>/${1}/g" \
- /usr/lib/systemd/system/taler-netdelay.service
- systemctl daemon-reload
- systemctl restart taler-netdelay.timer
-}
-
-function enable_logbackup() {
- systemctl restart taler-logbackup.timer
-}
+source ~/scripts/helpers.sh
if [[ "${ENABLE_EXPORTERS}" == "true" ]];
then
@@ -42,24 +14,30 @@ case "${HOSTNAME}" in
${BANK_HOSTS})
set_host bank
set_ddn ${BANK_DOMAIN}
- enable_logbackup
- exec ~/scripts/bank.sh
+ setup_log
+ enable_logrotate
+ exec ~/scripts/bank.sh init
;;
- ${DATABASE_HOSTS})
+ ${DB_HOSTS})
set_host database
set_ddn ${DATABASE_DOMAIN}
+ setup_log
+ enable_logrotate
exec ~/scripts/database.sh init
;;
${EXCHANGE_HOSTS})
set_host exchange
set_ddn ${EXCHANGE_DOMAIN}
+ setup_log
+ enable_logrotate
enable_netdelay ${DATABASE_DOMAIN}
- enable_logbackup
exec ~/scripts/exchange.sh init
;;
${MERCHANT_HOSTS})
set_host merchant
set_ddn ${MERCHANT_DOMAIN}
+ setup_log
+ enable_logrotate
exec ~/scripts/merchant.sh init
;;
${MONITOR_HOSTS})
@@ -71,15 +49,20 @@ case "${HOSTNAME}" in
set_host proxy
set_ddn ${PROXY_DOMAIN}
enable_netdelay ${EXCHANGE_DOMAIN}
- enable_logbackup
+ setup_log
+ enable_logrotate
exec ~/scripts/proxy.sh init
;;
- ${DNS_HOST})
+ ${DNS_HOSTS})
+ set_host dns
+ setup_log
;;
${WALLET_HOSTS})
set_host wallet
- set_ddn "wallet.$(hostname | cut -d '.' -f1).${DNS_ZONE}"
+ HOST=$(hostname | cut -d '.' -f1)
+ set_ddn "${WALLET_DOMAIN//\*/${HOST}}"
enable_netdelay ${PROXY_DOMAIN}
+ setup_log
exec ~/scripts/wallet.sh init
;;
esac
diff --git a/experiment/scripts/setup.sh b/experiment/scripts/setup.sh
index 1021cb8..6d5b544 100644
--- a/experiment/scripts/setup.sh
+++ b/experiment/scripts/setup.sh
@@ -1,80 +1,159 @@
#!/bin/bash
# Setup nodes for the experiment
-set -eua
+# Set the current user
echo "G5K_USER=$(cat ~/experiment-info.json | jq -r '.user.name')" >> ~/.env
source ~/.env
-# Add the environment config for following shells
-cat ~/.env | grep -v API_KEY | tee -a /etc/environment
-
-set -x
-
-if [[ "${DNS_HOST}" != *-*.*.grid5000.fr ]]; then
- echo "DNS_HOST must be set completely!"
-fi
-
-LOG_DIR=/home/${G5K_USER}/taler-logs
-
-if [ -d ${LOG_DIR} ]; then
- rm -rf ${LOG_DIR}/*
-elif [ -d /home/${G5K_USER} ]; then
- mkdir ${LOG_DIR}
-else
- LOG_DIR=/tmp/taler && mkdir ${LOG_DIR}
-fi
-
-G5K_HOME=/root/taler/grid5k
-
-NS_IP=$(host ${DNS_HOST} | sed -n 1p | awk '{print $4}')
-
-# Temporarily checkout to the feature branch
-cd "${G5K_HOME}" && git checkout node-setup && git pull && cd
-
-# Remove default nginx config
-rm /etc/nginx/sites-enabled/default > /dev/null 2>&1 || true
-
-# Override default configurations with the one from this Git.
-cp -r "${G5K_HOME}"/configs/* /
-
-# Send all logs about taler to promtail on the monitoring node
-sed -i -e "s/<MONITOR_DOMAIN_HERE>/${MONITOR_DOMAIN}/g" \
- -e "s|<LOG_DIR_HERE>|${LOG_DIR}|g" \
- /etc/rsyslog.d/taler.conf
-
-sed -i "s|<LOG_DIR_HERE>|${LOG_DIR}|g" \
- /etc/logrotate.d/taler
-
-sed -i "s/<ARGUMENTS_HERE>/${EXCHANGE_ARGS}/g" \
- /etc/default/taler-exchange
-
-if ! grep -Fxq "server=${NS_IP}" /etc/dnsmasq.conf ; then
- echo "server=${NS_IP}" >> /etc/dnsmasq.conf
-fi
-
-if ! grep -Fxq "nameserver 127.0.0.1" /etc/resolv.conf ; then
- mv /etc/resolv.conf /etc/resolv.conf.bak
- echo "nameserver 127.0.0.1" > /etc/resolv.conf
-fi
-
-BIND_SERVERS=$(grep nameserver /etc/resolv.conf.bak | awk '{print $2}' ORS=';
')
-sed -i "s/<DNS_ZONE_HERE>/${DNS_ZONE}/g" \
- /etc/bind/named.conf.local
-sed -i "s/<GRID_DNS_HERE>/${BIND_SERVERS}/g" \
- /etc/bind/named.conf.options
-sed -i "s/<DNS_ZONE_HERE>/${DNS_ZONE}/g" \
- /var/lib/bind/perf.taler
-
-systemctl daemon-reload
-
-if [[ "${HOSTNAME}" != "${DNS_HOST}" ]]; then
- systemctl restart dnsmasq
-fi
-
-if [ -f ~/scripts/taler-perf.sh ]; then
- mv ~/scripts/taler-perf.sh /usr/local/bin/taler-perf
-fi
+set -euax
+
+# Parse and export the experiment nodes specified in the meta file
+# experiment-info.json
+function parse_experiment_nodes() {
+ # Get the nodes and add them to nodes.json in the form:
+ # {node: jFed node-name, host: Grid5000 node}
+ if [ ! -f ~/nodes.json ]; then
+ cat ~/experiment-info.json | \
+ jq '.nodes | to_entries | .[] | {node: .key, host:
.value.ssh_login[1].hostname }' | \
+ jq -s \
+ > ~/nodes.json
+ fi
+
+ # Read the jFed node-names which are defined in the NODES env
+ # and export their Grid5000 hostname to .env
+ # Assume the jFed nodes Exchange-1 Exchange-2, then Exchange must be added
in NODES
+ # The env will then be the following:
+ # EXCHANGE_HOSTS=node-1.site-1.grid5000.fr|node-2.site-2.grid5000.fr
+ # When jFed has only Exchange, the env will be just
+ # EXCHANGE_HOSTS=node-1.site-1.grid5000.fr
+ # This will be used in run.sh to determine which role script to execute
+ for NODE in ${NODES}; do
+ echo "${NODE^^}_HOSTS=\"$(\
+ cat ~/nodes.json | \
+ jq --arg NODE ${NODE}.* -r 'map(select(.node | test($NODE)) | .host) |
join("|")'
+ )\"" \
+ >> ~/.env
+ done
+
+ echo "WALLET_HOSTS=*" >> ~/.env
+}
+
+# Determine and create the base log directory
+# If NFS exists, it will be created in the users home dir on the NFS
+function setup_log_dir() {
+ LOG_DIR=/home/${G5K_USER}/taler-logs
+
+ if [ -d ${LOG_DIR} ]; then
+ # If multiple nodes want to delete the same dir we run into errors
+ # Let it fail safely with || true
+ rm -rf ${LOG_DIR}/* || true
+ elif [ -d /home/${G5K_USER} ]; then
+ mkdir ${LOG_DIR} | true
+ else
+ LOG_DIR=/tmp/taler && mkdir ${LOG_DIR}
+ fi
+
+ echo "LOG_DIR=${LOG_DIR}" >> ~/.env
+}
+
+# Setup the environment configuration
+function setup_environment() {
+ set +x
+
+ # Determine the port the taler-exchange-* should use
+ if [ "$USE_PGBOUNCER" = "true" ]; then
+ echo "DB_PORT=6432" >> ~/.env
+ else
+ echo "DB_PORT=5432" >> ~/.env
+ fi
+
+ echo "START_TIME=$(date +%s)" >> ~/.env
+
+ # Needed for envsubst to work
+ export DNS_ZONE=${DNS_ZONE}
+ # Set the hostnames completely with substituting ${DNS_ZONE}
+ # Important: dont use cat env | envsubst > env - this will truncate env
before its read
+ # and thus it will be empty in the end
+ cat ~/.env | envsubst > /tmp/.env && mv /tmp/.env ~/.env
+ # Add the environment config for following shells
+ cat ~/.env | grep -v API_KEY | tee /etc/environment
+
+ # Reload the env since HOST_* and *_DOMAIN was added
+ source ~/.env
+
+ set -x
+}
+
+# Setup shared configurations such as the ones from configs/*
+function setup_config() {
+ # Temporarily checkout to the feature branch
+ cd "${G5K_HOME}" && git checkout "${G5K_COMMIT_SHA}" && git pull && cd
+
+ # Remove default nginx config
+ rm /etc/nginx/sites-enabled/default > /dev/null 2>&1 || true
+
+ # Override default configurations with the one from this Git.
+ cp -r "${G5K_HOME}"/configs/* /
+
+ find /usr/lib/systemd/system/ -iname taler-exchange-httpd*.service \
+ -exec sed -i "s|<CMD_PREFIX_HERE>|${EXCHANGE_CMD_PREFIX} |g" {} \;
+
+ if [ -f ~/scripts/taler-perf.sh ]; then
+ mv ~/scripts/taler-perf.sh /usr/local/bin/taler-perf
+ fi
+}
+
+# Configure the experiments DNS
+function setup_dns() {
+ NS_IP=$(host ${DNS_HOSTS} | sed -n 1p | awk '{print $4}')
+
+ # Set our DNS to be the only DNS to query by the stub resolver
+ if ! grep -Fxq "server=${NS_IP}" /etc/dnsmasq.conf ; then
+ echo "server=${NS_IP}" >> /etc/dnsmasq.conf
+ fi
+
+ # Set dnsmasq as our only resolver (stub)
+ if ! grep -Fxq "nameserver 127.0.0.1" /etc/resolv.conf ; then
+ mv /etc/resolv.conf /etc/resolv.conf.bak
+ echo "nameserver 127.0.0.1" > /etc/resolv.conf
+ fi
+
+ # Add the Grid5000 DNS servers as forwarders to our DNS so grid5000 stuff
+ # gets resolved correctly
+ BIND_SERVERS=$(grep nameserver /etc/resolv.conf.bak | awk '{print $2}'
ORS='; ')
+ sed -i "s/<DNS_ZONE_HERE>/${DNS_ZONE}/g" \
+ /etc/bind/named.conf.local
+ sed -i "s/<GRID_DNS_HERE>/${BIND_SERVERS}/g" \
+ /etc/bind/named.conf.options
+ sed -i -e "s/<DNS_ZONE_HERE>/${DNS_ZONE}/g" \
+ -e "s/<NS_IP_HERE>/${NS_IP}/g" \
+ /var/lib/bind/perf.taler
+
+ # Remove potentionally expired zone journals so bind will not complain
+ # (this can happen when the Espec - or just this script are executed
multiple times)
+ if [ -f /var/lib/bind/perf.taler.jnl ]; then
+ rm -rf /var/lib/bind/perf.taler.jnl
+ fi
+
+ systemctl daemon-reload
+
+ if ! [[ "${HOSTNAME}" =~ ${DNS_HOSTS} ]]; then
+ # Wait for named to be ready before starting dnsmasq
+ sleep 5
+ systemctl restart dnsmasq
+ # Wait again in hope that the hostname error is fixed
+ # hostname -I couldn't get address for 'x': failure syntax error
+ sleep 5
+ else
+ # Start the DNS when we are the DNS host
+ systemctl restart named
+ fi
+}
+
+parse_experiment_nodes
+setup_log_dir
+setup_environment
+setup_config
+setup_dns
exec ~/scripts/createusers.sh
-
-exit 0
diff --git a/experiment/scripts/taler-perf.sh b/experiment/scripts/taler-perf.sh
index 9a39540..e1ff428 100644
--- a/experiment/scripts/taler-perf.sh
+++ b/experiment/scripts/taler-perf.sh
@@ -1,20 +1,32 @@
#!/bin/bash
set -e
-source /etc/environment
source ~/scripts/helpers.sh
+function update_processes() {
+ case "$1" in
+ prometheus)
+ ssh -A -o StrictHostKeyChecking=no ${MONITOR_DOMAIN} \
+ "/bin/bash /root/scripts/monitor.sh start"
+ ;;
+ *)
+ echo "Unknown argument '$1' for function ${FUNCNAME[0]}"
+ echo "Usage: update [prometheus]"
+ ;;
+ esac
+}
+
function start_wallets() {
- for WALLET in $(get_wallet_domains); do
- ssh -o StrictHostKeyChecking=no wallet.${WALLET}.perf.taler \
+ for WALLET in $(get_wallet_hosts); do
+ ssh -o StrictHostKeyChecking=no ${WALLET_DOMAIN//\*/${WALLET}} \
"/bin/bash /root/scripts/wallet.sh start
${1:-${NUM_WALLET_PROCESSES}}" &
done
wait
}
function stop_wallets() {
- for WALLET in $(get_wallet_domains); do
- ssh -o StrictHostKeyChecking=no wallet.${WALLET}.perf.taler \
+ for WALLET in $(get_wallet_hosts); do
+ ssh -o StrictHostKeyChecking=no ${WALLET_DOMAIN//\*/${WALLET}} \
"/bin/bash /root/scripts/wallet.sh stop
${1:-${NUM_WALLET_PROCESSES}}" &
done
wait
@@ -22,9 +34,22 @@ function stop_wallets() {
function start_exchanges() {
ssh -o StrictHostKeyChecking=no ${EXCHANGE_DOMAIN} \
- "/bin/bash /root/scripts/exchange.sh ${1:-${NUM_EXCHANGE_PROCESSES}}"
+ "/bin/bash /root/scripts/exchange.sh start
${1:-${NUM_EXCHANGE_PROCESSES}}"
ssh -o StrictHostKeyChecking=no ${PROXY_DOMAIN} \
- "/bin/bash /root/scripts/proxy.sh ${1:-${NUM_EXCHANGE_PROCESSES}}"
+ "/bin/bash /root/scripts/proxy.sh start
${1:-${NUM_EXCHANGE_PROCESSES}}"
+ update_processes "prometheus"
+}
+
+function stop_exchanges() {
+ # must remove the exchange form the monitor host before the one from the
nginx
+ # since helpers.sh gets all exchanges from the nginx config
+ ssh -A -o StrictHostKeyChecking=no ${MONITOR_DOMAIN} \
+ "/bin/bash /root/scripts/monitor.sh stop-exchanges
${1:-${NUM_EXCHANGE_PROCESSES}}"
+ ssh -A -o StrictHostKeyChecking=no ${PROXY_DOMAIN} \
+ "/bin/bash /root/scripts/proxy.sh stop
${1:-${NUM_EXCHANGE_PROCESSES}}"
+ sleep 5
+ ssh -o StrictHostKeyChecking=no ${EXCHANGE_DOMAIN} \
+ "/bin/bash /root/scripts/exchange.sh stop
${1:-${NUM_EXCHANGE_PROCESSES}}"
}
function start_processes() {
@@ -47,6 +72,9 @@ function stop_processes() {
wallet)
stop_wallets $2
;;
+ exchange)
+ stop_exchanges $2
+ ;;
*)
echo "Unknown argument '$1' for function ${FUNCNAME[0]}"
echo "Usage: stop [wallet] NUM"
@@ -54,16 +82,37 @@ function stop_processes() {
esac
}
-function update_processes() {
- case "$1" in
- prometheus)
- ssh -o StrictHostKeyChecking=no ${MONITOR_DOMAIN} "/bin/bash -c
/root/scripts/monitor.sh"
- ;;
- *)
- echo "Unknown argument '$1' for function ${FUNCNAME[0]}"
- echo "Usage: update [prometheus]"
- ;;
- esac
+function rebuild() {
+ while [[ $# -gt 0 ]]; do
+ case "$1" in
+ --exchange|-e)
+ EXCHANGE_COMMIT_SHA=$2
+ shift 2
+ ;;
+ --gnunet|-g)
+ GNUNET_COMMIT_SHA=$2
+ shift 2
+ ;;
+ --wallet|-w)
+ WALLET_COMMIT_SHA=$2
+ shift 2
+ ;;
+ --merchant|-m)
+ MERCHANT_COMMIT_SHA=$2
+ shift 2
+ ;;
+ *)
+ echo "Unkown argument $1"
+ echo "Usage rebuilt [target]"
+ echo "Targets: "
+ echo "-e|--exchange <commit-sha>"
+ echo "-g|--gnunet <commit-sha>"
+ echo "-w|--wallet <commit-sha>"
+ echo "-m|--merchant <commit-sha>"
+ exit 1
+ esac
+ done
+ # TODO
}
case "$1" in
@@ -79,6 +128,10 @@ case "$1" in
shift
update_processes $@
;;
+ rebuild)
+ shift
+ rebuild
+ ;;
esac
exit 0
diff --git a/experiment/scripts/wallet.sh b/experiment/scripts/wallet.sh
index 64c53b8..45ba4f4 100755
--- a/experiment/scripts/wallet.sh
+++ b/experiment/scripts/wallet.sh
@@ -1,53 +1,73 @@
#!/bin/bash
+INFO_MSG="
+Configure and start taler-wallets
+"
+OPT_MSG="
+init:
+ Initialize the wallets and start an inital NUM_WALLET_PROCESSES
+
+start NUM:
+ Start NUM new wallet benchmark processes
+
+stop NUM:
+ Stop NUM wallet benchmark processes
+"
+
set -eux
+source ~/scripts/helpers.sh
-if [[ "$1" == "init" ]];
-then
- source ~/scripts/helpers.sh
+# Enable rsyslog and wait until the exchange is ready
+function init_wallets() {
restart_rsyslog
wait_for_keys "${PROXY_DOMAIN}"
sleep 5
+}
- NUM_PROCESSES=${NUM_WALLET_PROCESSES:-10}
+# Start N wallets
+# If CPU load is over 75%, no wallets are started here
+# $1 Number of wallets to start
+function start_wallets() {
+ CPU_USAGE=$[100-$(vmstat 1 2 | tail -1 | awk '{print $15}')]
+ echo "CPU Usage: ${CPU_USAGE}%"
- if [ "$NUM_PROCESSES" -gt "0" ];
- then
- # Start one process with logging enabled
- let "NUM_PROCESSES-=1"
- systemctl restart taler-wallet@logging.service
+ if [ "${CPU_USAGE}" -gt "75" ]; then
+ echo "Not starting any more wallets"
+ exit 1
fi
-else
- NUM_PROCESSES=$2
-fi
-# count the running wallets
-RUNNING=$(ps -aux | grep "[wallet]-cli" | wc -l)
+ # count the running wallets so that numbers can be increased
+ RUNNING=$(ps -aux | grep "[wallet]-cli" | wc -l)
+
+ for i in $(seq ${1}); do
+ let "i+=${RUNNING}"
+ systemctl restart taler-wallet@${i}.service
+ sleep 0.5
+ done
+}
+
+# Stop N wallets
+# $1: N - number of wallets to stop or "all"
+function stop_wallets() {
+ if [[ "$1" == "all" ]]; then
+ systemctl stop taler-wallet@*.service
+ else
+ stop_numbered_services "taler-wallet" $1
+ fi
+}
case "$1" in
+ init)
+ init_wallets
+ start_wallets ${NUM_WALLET_PROCESSES:-10}
+ ;;
stop)
- if [[ "${NUM_PROCESSES}" = "logging" ]]; then
- systemctl stop taler-wallet@logging.service
- else
- let "START=${RUNNING}-${NUM_PROCESSES}+1"
- for i in $(seq ${START} ${RUNNING})
- do
- systemctl stop taler-wallet@${i}.service
- done
- fi
+ stop_wallets $2
+ ;;
+ start)
+ start_wallets $2
;;
- start|init)
- CPU_USAGE=$[100-$(vmstat 1 2 | tail -1 | awk '{print $15}')]
- echo "CPU Usage: ${CPU_USAGE}%"
- if [ "${CPU_USAGE}" -gt "75" ];
- then
- echo "Not starting any more wallets"
- exit 1
- fi
- for i in $(seq ${NUM_PROCESSES})
- do
- let "i+=${RUNNING}"
- systemctl restart taler-wallet@${i}.service
- done
+ *)
+ taler_perf_help $0 "$INFO_MSG" "$OPT_MSG"
;;
esac
diff --git a/experiment/taler.rspec b/experiment/taler.rspec
new file mode 100644
index 0000000..462f39a
--- /dev/null
+++ b/experiment/taler.rspec
@@ -0,0 +1,170 @@
+<?xml version='1.0'?>
+<rspec xmlns="http://www.geni.net/resources/rspec/3"; type="request"
generated_by="jFed RSpec Editor" generated="2021-11-25T12:41:28.205+01:00"
xmlns:emulab="http://www.protogeni.net/resources/rspec/ext/emulab/1";
xmlns:delay="http://www.protogeni.net/resources/rspec/ext/delay/1";
xmlns:jfed-command="http://jfed.iminds.be/rspec/ext/jfed-command/1";
xmlns:client="http://www.protogeni.net/resources/rspec/ext/client/1";
xmlns:jfed-ssh-keys="http://jfed.iminds.be/rspec/ext/jfed-ssh-keys/1"; xmlns:
[...]
+ <node client_id="DB" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <hardware_type name="dahu-grenoble"/>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="156.0"
y="70.0"/>
+ </node>
+ <node client_id="Exchange" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <hardware_type name="dahu-grenoble"/>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="283.0"
y="127.5"/>
+ </node>
+ <node client_id="Bank" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <hardware_type name="dahu-grenoble"/>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="422.0"
y="70.0"/>
+ </node>
+ <node client_id="Proxy" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <hardware_type name="dahu-grenoble"/>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="284.5"
y="184.5"/>
+ </node>
+ <node client_id="Monitor" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="732.5"
y="156.5"/>
+ </node>
+ <node client_id="Merchant" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="554.5"
y="156.5"/>
+ </node>
+ <node client_id="DNS" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <hardware_type name="dahu-grenoble"/>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="638.0"
y="70.0"/>
+ </node>
+ <node client_id="Wallet-1" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="140.0"
y="300.0"/>
+ </node>
+ <node client_id="Wallet-2" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="280.0"
y="300.0"/>
+ </node>
+ <node client_id="Wallet-3" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="420.0"
y="300.0"/>
+ </node>
+ <node client_id="Wallet-4" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="560.0"
y="300.0"/>
+ </node>
+ <node client_id="Wallet-5" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="700.0"
y="300.0"/>
+ </node>
+ <node client_id="Wallet-6" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="140.0"
y="340.0"/>
+ </node>
+ <node client_id="Wallet-7" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="280.0"
y="340.0"/>
+ </node>
+ <node client_id="Wallet-8" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="420.0"
y="340.0"/>
+ </node>
+ <node client_id="Wallet-9" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="560.0"
y="340.0"/>
+ </node>
+ <node client_id="Wallet-10" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="700.0"
y="340.0"/>
+ </node>
+ <node client_id="Wallet-11" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="140.0"
y="380.0"/>
+ </node>
+ <node client_id="Wallet-12" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="280.0"
y="380.0"/>
+ </node>
+ <node client_id="Wallet-13" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="420.0"
y="380.0"/>
+ </node>
+ <node client_id="Wallet-14" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="560.0"
y="380.0"/>
+ </node>
+ <node client_id="Wallet-15" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="700.0"
y="380.0"/>
+ </node>
+ <node client_id="Wallet-16" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="140.0"
y="420.0"/>
+ </node>
+ <node client_id="Wallet-17" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="280.0"
y="420.0"/>
+ </node>
+ <node client_id="Wallet-18" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="420.0"
y="420.0"/>
+ </node>
+ <node client_id="Wallet-19" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="560.0"
y="420.0"/>
+ </node>
+ <node client_id="Wallet-20" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+ <sliver_type name="raw-pc">
+ <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+ </sliver_type>
+ <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="700.0"
y="420.0"/>
+ </node>
+</rspec>
\ No newline at end of file
diff --git a/experiment/wallets.rspec b/experiment/wallets.rspec
deleted file mode 100644
index b80b714..0000000
--- a/experiment/wallets.rspec
+++ /dev/null
@@ -1,63 +0,0 @@
-<?xml version='1.0'?>
-<rspec xmlns="http://www.geni.net/resources/rspec/3"; type="request"
generated_by="jFed RSpec Editor" generated="2021-11-09T14:54:19.867+01:00"
xmlns:emulab="http://www.protogeni.net/resources/rspec/ext/emulab/1";
xmlns:delay="http://www.protogeni.net/resources/rspec/ext/delay/1";
xmlns:jfed-command="http://jfed.iminds.be/rspec/ext/jfed-command/1";
xmlns:client="http://www.protogeni.net/resources/rspec/ext/client/1";
xmlns:jfed-ssh-keys="http://jfed.iminds.be/rspec/ext/jfed-ssh-keys/1"; xmlns:
[...]
- <node client_id="Wallet-1" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
- <sliver_type name="raw-pc">
- <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
- </sliver_type>
- <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="130" y="87.0"/>
- </node>
- <node client_id="Wallet-2" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
- <sliver_type name="raw-pc">
- <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
- </sliver_type>
- <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="280" y="87.0"/>
- </node>
- <node client_id="Wallet-3" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
- <sliver_type name="raw-pc">
- <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
- </sliver_type>
- <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="430" y="87.0"/>
- </node>
- <node client_id="Wallet-4" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
- <sliver_type name="raw-pc">
- <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
- </sliver_type>
- <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="580" y="87.0"/>
- </node>
- <node client_id="Wallet-5" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
- <sliver_type name="raw-pc">
- <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
- </sliver_type>
- <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="730" y="87.0"/>
- </node>
- <node client_id="Wallet-6" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
- <sliver_type name="raw-pc">
- <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
- </sliver_type>
- <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="130"
y="145.0"/>
- </node>
- <node client_id="Wallet-7" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
- <sliver_type name="raw-pc">
- <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
- </sliver_type>
- <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="280"
y="145.0"/>
- </node>
- <node client_id="Wallet-8" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
- <sliver_type name="raw-pc">
- <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
- </sliver_type>
- <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="430"
y="145.0"/>
- </node>
- <node client_id="Wallet-9" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
- <sliver_type name="raw-pc">
- <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
- </sliver_type>
- <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="580"
y="145.0"/>
- </node>
- <node client_id="Wallet-10" exclusive="true"
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
- <sliver_type name="raw-pc">
- <disk_image
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
- </sliver_type>
- <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="730"
y="145.0"/>
- </node>
-</rspec>
\ No newline at end of file
diff --git a/image/README.md b/image/README.md
index 6cc5523..cc52c66 100644
--- a/image/README.md
+++ b/image/README.md
@@ -8,39 +8,41 @@ Official documentation can be found on these links:
## Manual Build
-### Prerequisites
+Replace `<G5K_USER>` with your Grid5000 username.
+This variable is required, if not specified the build will fail.
-To build the image, the following repositories need to be added as dist
archives in
-`grid5000/steps/data`
+```bash
+kameleon build -g g5k_user:<G5K_USER> taler-debian11
+```
+
+**NOTE** Make sure that all dependencies listed in `Grid5000 Environment` are
installed
-* [gnunet](https://git.gnunet.org/gnunet.git): gnunet.tar.gz
-* [taler-exchange](https://git.taler.net/exchange.git): exchange.tar.gz
-* [taler-merchant](https://git.taler.net/merchant.git): merchant.tar.gz
-* [taler-wallet-core](https://git.taler.net/wallet-core.git): wallet.tar.gz
+### Additional Variables
-Replace `<G5K_USER> `and `<G5K_HOST>` in `taler-debian11.yaml` in the
following line:
-`g5k_tar_path:
"http://public.<G5K_HOST>.grid5000.fr/~<G5K_USER>/taler-debian11.tar.zst"`
+The Taler binaries are built from source. You have the possiblity to override
the commit from
+which should be built with the following variables (default `master`):
-**G5K_USER**: Your Grid5000 username.
-**G5K_HOST**: The host where the image will be copied to in the last step.
+`gnunet_commit_sha`, `exchange_commit_sha`, `merchant_commit_sha`,
`wallet_commit_sha` and `grid5k_commit_sha`
-Alternatively they can be replaced after the build in
`build/taler-debian11/taler-debian11.dsc`
+To override them you must add them to the `-g` option of `kameleon build`:
-### Build
-`kameleon build taler-debian11`
+```bash
+kameleon build -g g5k_user:<G5K_USER> gnunet_commit_sha:master
grid5k_commit_sha:node-setup taler-debian11
+```
-**NOTE** Make sure that all dependencies listed in Grid5000 Environments are
installed
+For more information please run `kameleon build --help`
### Deploy
-Copy the image to a Grid5000 node:
+Copy the image to a Grid5000 site:
```bash
cd build/taler-debian11
-scp taler-debian11.* <G5K_USER>@access.grid5000.fr:<G5K_NODE>/public/
+scp taler-debian11.* <G5K_USER>@access.grid5000.fr:<G5K_SITE>/public/
```
-**NOTE** G5K_USER and G5K_HOST should match the ones in taler-debian11.yaml
+**NOTE** G5K_USER and G5K_SITE should match the ones in taler-debian11.dsc
+G5K_SITE defaults to `lyon`.
## Usage
diff --git a/image/taler-debian11.yaml b/image/taler-debian11.yaml
index dec6fb8..323eea1 100644
--- a/image/taler-debian11.yaml
+++ b/image/taler-debian11.yaml
@@ -27,7 +27,7 @@ global:
g5k_version: 2
## Environment image path and compression
# Note: setup for the docker build image - please replace G5K_HOST and
G5K_USER for manual builds
- g5k_tar_path:
"http://public.<G5K_HOST>.grid5000.fr/~<G5K_USER>/taler-debian11.tar.zst"
+ g5k_tar_path:
"http://public.lyon.grid5000.fr/~$${g5k_user}/taler-debian11.tar.zst";
# g5k_tar_compression: "zstd"
## Environment postinstall path, compression, and script command
# g5k_postinst_path: server:///grid5000/postinstalls/g5k-postinstall.tgz
@@ -40,19 +40,16 @@ global:
## Environment visibility
# g5k_visibility: "shared"
taler_build_packages: "recutils autoconf uncrustify autopoint libtool
python3-pip libgcrypt20-dev libjansson-dev libcurl4-gnutls-dev libsodium-dev
libidn2-dev libunistring-dev libmicrohttpd-dev libsqlite3-dev libqrencode-dev
valgrind libpq-dev texinfo gdb make npm zip python3-distutils pkg-config"
- taler_packages: "nginx postgresql-13 postgresql-contrib curl jq bc sudo git
zile dnsutils prometheus-postgres-exporter prometheus-nginx-exporter net-tools
netcat parallel nodejs tshark dnsmasq bind9"
- taler_packages_no_recommends: "prometheus prometheus-node-exporter"
- taler_disable_services: "nginx postgresql prometheus
prometheus-postgres-exporter prometheus-nginx-exporter prometheus-node-exporter
named"
+ taler_packages: "nginx postgresql-13 postgresql-contrib curl jq bc sudo git
zile dnsutils prometheus-postgres-exporter prometheus-nginx-exporter net-tools
netcat parallel nodejs tshark dnsmasq bind9 systemd-coredump bash-completion
pgbouncer pgstat"
+ taler_packages_no_recommends: "prometheus prometheus-node-exporter
prometheus-pgbouncer-exporter"
+ taler_disable_services: "nginx postgresql prometheus
prometheus-postgres-exporter prometheus-nginx-exporter prometheus-node-exporter
named gettext-base dnsmasq prometheus-pgbouncer-exporter pgbouncer"
taler_loki_version: "v2.4.0"
- gnunet_tar: gnunet.tar.gz
- exchange_tar: exchange.tar.gz
- merchant_tar: merchant.tar.gz
- wallet_tar: wallet.tar.gz
taler_path: /root/taler
- gnunet_path: $${taler_path}/$${gnunet_tar}
- exchange_path: $${taler_path}/$${exchange_tar}
- merchant_path: $${taler_path}/$${merchant_tar}
- wallet_path: $${taler_path}/$${wallet_tar}
+ gnunet_commit_sha: master
+ exchange_commit_sha: master
+ merchant_commit_sha: master
+ wallet_commit_sha: master
+ grid5k_commit_sha: master
## Other parameters can be changed, see kameleon info debian10-taler.yaml
bootstrap:
@@ -68,24 +65,6 @@ setup:
## kameleon dryrun debian10_custom.yaml to see the resulting steps in the
build.
## The following is given as example only, replace with your steps.
- - import:
- - import_gnunet:
- # kameleon_data_dir=./grid5000/steps/data
- - local2in:
- - $${kameleon_data_dir}/$${gnunet_tar}
- - $${gnunet_path}
- - import_exchange:
- - local2in:
- - $${kameleon_data_dir}/$${exchange_tar}
- - $${exchange_path}
- - import_merchant:
- - local2in:
- - $${kameleon_data_dir}/$${merchant_tar}
- - $${merchant_path}
- - import_wallet:
- - local2in:
- - $${kameleon_data_dir}/$${wallet_tar}
- - $${wallet_path}
- install:
- packages:
@@ -97,60 +76,81 @@ setup:
apt install -y $${taler_packages} $${taler_build_packages}
apt install -y --no-install-recommends
$${taler_packages_no_recommends}
+ mkdir -p $${taler_path}
+
- gnunet:
- exec_in: |
cd $${taler_path}
- tar -xvf $${gnunet_tar}
- cd gnunet-*
+ git clone https://git.gnunet.org/gnunet.git
+ cd gnunet
+ git checkout $${gnunet_commit_sha} || true
+ git pull || true
+
+ sudo ./bootstrap
CFLAGS="-O0 -g" ./configure --enable-logging=verbose --prefix=/usr #
--enable-sanitizer
+ make || true
make install
ldconfig
- cd .. && rm -rf gnunet-*
-
- exchange:
- exec_in: |
cd $${taler_path}
pip install jinja2
- tar -xvf $${exchange_tar}
- cd taler-exchange*
+
+ git clone https://git.taler.net/exchange.git
+ cd exchange
+ git checkout $${exchange_commit_sha} || true
+ git pull || true
+
+ sudo ./bootstrap
CFLAGS="-O0 -g" ./configure --enable-logging=verbose --prefix=/usr #
--enable-sanitizer
+ make || true
make install
ldconfig
- cd .. && rm -rf taler-exchange-*
-
- merchant:
- exec_in: |
cd $${taler_path}
- tar -xvf $${merchant_tar}
- cd taler-merchant*
+ git clone https://git.taler.net/merchant.git
+ cd merchant
+ git checkout $${merchant_commit_sha} || true
+ git pull || true
+
+ sudo ./bootstrap
+
./configure --enable-logging=verbose --prefix=/usr #
--enable-sanitizer
+ make || true
make install
ldconfig
- cd .. && rm -rf taler-merchant-*
-
- wallet:
- exec_in: |
cd $${taler_path}
- tar -xvf $${wallet_tar}
- cd taler-wallet*
+ git clone https://git.taler.net/wallet-core.git
+ cd wallet-core
+ git checkout $${wallet_commit_sha} || true
+ git pull || true
+
+ sudo ./bootstrap
+
npm install -g pnpm
./configure
+ make || true
make install
- cd .. && rm -rf taler-wallet-*
-
- grid5k:
- exec_in: |
cd $${taler_path}
git clone git://git.taler.net/grid5k.git
+ cd grid5k
+ git checkout $${grid5k_commit_sha} || true
+
+ ./experiment/scripts/createusers.sh
- loki-promtail:
- exec_in: |
@@ -170,6 +170,7 @@ setup:
- disable_services:
- exec_in: |
+ /etc/init.d/pgbouncer stop
systemctl daemon-reload
systemctl stop $${taler_disable_services}
systemctl disable $${taler_disable_services}
diff --git a/notes.txt b/notes.txt
index d98530f..e69de29 100644
--- a/notes.txt
+++ b/notes.txt
@@ -1,25 +0,0 @@
-* option "other_packages_no_clean" in global for yaml image builder
- not properly documented, forced us to hunt down why explictily
- installed packages were removed again after the setup step - no problem
anymore in
- (found in grid5000/steps/setup/debian/clean_unnecessary_packages.yaml)
-* scripts located in grid5000/steps/data/helpers/ are not executable which
results in
- Permission denied when running kameleon build
-* jFed: Reserve not working for grid5000 resources
-* jFed: Error do not clearly show which nodes failed allocating
- its hard to detect problems when using many nodes - should be clearer
- bsp error: Some requiered resource was not available at this time. (which?!)
-* only show nodes which are available for users - e.g. not neovise
-* jFed: Changing Node or HW type does sometimes not apply and is not saved
-* Online status monitoing not up to date
-* Very common error:
- <h1>Proxy Error</h1>
- <p>The proxy server received an invalid
- response from an upstream server.<br />
- The proxy server could not handle the request<p>Reason: <strong>Error
reading from remote server</strong></p></p>
-* SSH login from rpesc viewer not possible
-* SSH login often fails - must provide password for cert and user (user
password is not the one I speciefied, neither the default for the root user of
the image) - or connection closed ... - BUG in allocation code according to
email
-* espec documentation incomplete and contains unknown key, exaple:
execute.local dir.sudo
-* espec dir creation does not work - hangs at Dir "null" Creations at ...
-* espec - possibility to rerun and override previously uploaded files
(directory type)
-* espec - states 'permissions' as keyword where it is really 'permission' only
-* ansible not working with debian 11 - install script fails - but even when
installed manually it hangs in preparing ansible dir
--
To stop receiving notification emails like this one, please contact
gnunet@gnunet.org.
- [taler-grid5k] branch master updated (d6cc557 -> 0901fad),
gnunet <=
- [taler-grid5k] 02/73: dns should work now, gnunet, 2021/12/14
- [taler-grid5k] 03/73: fix unbound variable in helper, gnunet, 2021/12/14
- [taler-grid5k] 05/73: add configuration option for cmd prefix, gnunet, 2021/12/14
- [taler-grid5k] 04/73: run 'createuser' in image build - try to fix dmsasq issue, gnunet, 2021/12/14
- [taler-grid5k] 06/73: cleanup setup script - add completition, gnunet, 2021/12/14
- [taler-grid5k] 13/73: add first grafana exports, gnunet, 2021/12/14
- [taler-grid5k] 14/73: update dashboard exports, gnunet, 2021/12/14
- [taler-grid5k] 09/73: add config options for all exchange processes, gnunet, 2021/12/14
- [taler-grid5k] 01/73: working well unless dns, gnunet, 2021/12/14
- [taler-grid5k] 11/73: switch back to dahu, gnunet, 2021/12/14