[taler-grid5k] 11/16: switch to in experiment dns for simplicity

[gnunet]

This is an automated email from the git hooks/post-receive script.

marco-boss pushed a commit to branch master
in repository grid5k.

commit 18c88565550529cd39b357906257088d0372dc02
Author: Boss Marco <bossm8@bfh.ch>
AuthorDate: Mon Nov 22 20:59:01 2021 +0100

    switch to in experiment dns for simplicity
---
 additional/README.md                       |  7 ++---
 additional/configs/bind/named.conf.local   | 13 ----------
 additional/configs/bind/named.conf.options | 19 --------------
 additional/configs/bind/perf.taler         | 12 ---------
 additional/{configs => }/grafana/tbd       |  0
 experiment/README.md                       | 21 +++++++++++----
 experiment/env                             |  8 +++---
 experiment/experiment-specification.yml    |  5 ++--
 experiment/infra.rspec                     | 13 +++++++---
 experiment/scripts/cleardns.sh             | 33 ------------------------
 experiment/scripts/dns.sh                  | 10 ++++++++
 experiment/scripts/run.sh                  |  4 ++-
 experiment/scripts/setup.sh                | 41 ++++++++++++++++++------------
 13 files changed, 73 insertions(+), 113 deletions(-)

diff --git a/additional/README.md b/additional/README.md
index ad7863b..28278cf 100644
--- a/additional/README.md
+++ b/additional/README.md
@@ -2,8 +2,9 @@
 
 ## plot.sh
 
-Create png plots from the grafana experiment dashboards using [Grafan 
Dashboard Plotter](https://github.com/bossm8/grafana-dashboard-plotter).
+Create png plots from the grafana experiment dashboards using [Grafana 
Dashboard Plotter](https://github.com/bossm8/grafana-dashboard-plotter).
 
-## configs
+## grafana
 
-Contains configurations and examples for additional resources needed which are 
not located inside the grid.
+**TBD**
+Will contain grafana dashboards once finished
diff --git a/additional/configs/bind/named.conf.local 
b/additional/configs/bind/named.conf.local
deleted file mode 100644
index 929caca..0000000
--- a/additional/configs/bind/named.conf.local
+++ /dev/null
@@ -1,13 +0,0 @@
-key "ddns-key.perf.taler" {
-  algorithm hmac-sha256;
-  secret "<SOME_SECRET_KEY>
-};
-
-zone "perf.taler" {
-  type master;
-  file "/var/lib/bind/perf.taler";
-  update-policy {
-    grant ddns-key.perf.taler subdomain perf.taler. ANY;
-  };
-};
-
diff --git a/additional/configs/bind/named.conf.options 
b/additional/configs/bind/named.conf.options
deleted file mode 100644
index 22f11c1..0000000
--- a/additional/configs/bind/named.conf.options
+++ /dev/null
@@ -1,19 +0,0 @@
-acl grid5000 {
-        194.254.60.35;
-        194.254.60.13;
-       127.0.0.0/8;
-};
-
-options {
-        directory "/var/cache/bind";
-
-        allow-query { grid5000; };
-        allow-update { grid5000; };
-
-        dnssec-validation auto;
-
-        rrset-order { order random; };
-
-        listen-on-v6 { any; };
-};
-
diff --git a/additional/configs/bind/perf.taler 
b/additional/configs/bind/perf.taler
deleted file mode 100644
index ab36c57..0000000
--- a/additional/configs/bind/perf.taler
+++ /dev/null
@@ -1,12 +0,0 @@
-$ORIGIN .
-$TTL 60        ; 1 minute
-perf.taler             IN SOA  ns1.perf.taler. admin.perf.taler. (
-                               202112574  ; serial
-                               604800     ; refresh (1 week)
-                               86400      ; retry (1 day)
-                               2419200    ; expire (4 weeks)
-                               604800     ; minimum (1 week)
-                               )
-                       NS      ns1.perf.taler.
-$ORIGIN perf.taler.
-ns1                    A       127.0.0.1
diff --git a/additional/configs/grafana/tbd b/additional/grafana/tbd
similarity index 100%
rename from additional/configs/grafana/tbd
rename to additional/grafana/tbd
diff --git a/experiment/README.md b/experiment/README.md
index 1260dea..599beb3 100644
--- a/experiment/README.md
+++ b/experiment/README.md
@@ -12,11 +12,10 @@
 
 To successfully run an experiment the following steps must be made:
 
-**NOTE** Two external nodes are required to run this experiment:
-         * Bind9 DNS server with the Zone specified in `.env` - example config 
can be found in `additional`
-         * Grafana instance with Taler Performance Dashboards - dashboards can 
be found in `additional`
-           Install instructions can be found on 
[grafana.com](https://grafana.com/docs/grafana/latest/installation/)
-           Once installed, two datasources must be added - Prometheus and 
Loki, they will be updated from the experiment
+**NOTE** An external Grafana instance with Taler Performance Dashboards is 
needed
+         Dashboards can be found in `additional` (TBD)
+         Install instructions can be found on 
[grafana.com](https://grafana.com/docs/grafana/latest/installation/)
+         Once installed, two datasources must be added - Prometheus and Loki, 
they will be updated from the experiment
 
 * Copy the environment default configuration `env` to `.env`
 * Read through `.env` and define the missing variables
@@ -45,6 +44,18 @@ They can also be stopped in the same way: `taler-perf stop 
<KIND> <NUM>`
 To stop the wallet processes which are logging, specify `taler-perf stop 
wallet logging`, they will not 
 be stopped otherwise.
 
+### Grafana Dashboard Plotting
+ 
+To persists the dashboards as png plots, there is a script in `../additional` 
which creates png
+plots based on a configuration. Please refer to the README located in the 
specified directory.
+
+### Application Logs
+
+The logs of the taler applications will be moved periodically to 
`/home/<G5K_USER>/taler-logs`. 
+They can then be retrieved using scp to the respective site they where running 
on
+(e.g. nancy/taler-logs for grvingt and others).
+Make sure to take them away before restarting an experiment, since they will 
be deleted in a new experiment.
+
 ## Notes
 
 Hopefully the bind dns gets useless once the experiment can be run in one 
espec.
diff --git a/experiment/env b/experiment/env
index ef3524f..f08ffdc 100644
--- a/experiment/env
+++ b/experiment/env
@@ -3,22 +3,20 @@
 # nodes are allocated.
 # Once the allocation bug is fixed these configurations
 # will be removed.
+# The DNS_HOST must be set completely, others can contain
+# a wildcard after the node name to match the site automatically.
 # Wallet hosts can be specified as a wildcard only since
 # all other hosts are matched before.
     BANK_HOSTS=dahu-x.*
 DATABASE_HOSTS=dahu-y.*
+      DNS_HOST=dahu-12.grenoble.grid5000.fr
 EXCHANGE_HOSTS=dahu-z.*
 MERCHANT_HOSTS=grvingt-x.*
  MONITOR_HOSTS=econome-y.*
    PROXY_HOSTS=dahu-u.*
   WALLET_HOSTS=*
 
-DNS_HOST=taler.net
 DNS_ZONE=perf.taler
-# The key to update the dns zone dynamically, base64 encoded.
-# Can be generated with tsig-keygen from bind9 package
-# (make sure the dns server knows about it)
-DYNDNS_KEY=
 
 # These values specify the domain names for
 # the nodes, they most certain do not need to be adjusted.
diff --git a/experiment/experiment-specification.yml 
b/experiment/experiment-specification.yml
index 7f68075..6881aea 100644
--- a/experiment/experiment-specification.yml
+++ b/experiment/experiment-specification.yml
@@ -12,8 +12,7 @@ upload:
 execute:
   # Setup DNS and Environment config
   - path: ~/scripts/setup.sh
-  # Clear the DNS-Zone
-  - path: ~/scripts/cleardns.sh
-    nodes: [Exchange]
+  - path: ~/scripts/dns.sh
+    nodes: [DNS]
   # Execute node specific code
   - path: ~/scripts/run.sh
diff --git a/experiment/infra.rspec b/experiment/infra.rspec
index 01c2d7c..b6a8e6d 100644
--- a/experiment/infra.rspec
+++ b/experiment/infra.rspec
@@ -1,11 +1,11 @@
 <?xml version='1.0'?>
-<rspec xmlns="http://www.geni.net/resources/rspec/3"; type="request" 
generated_by="jFed RSpec Editor" generated="2021-11-18T11:33:37.904+01:00" 
xmlns:emulab="http://www.protogeni.net/resources/rspec/ext/emulab/1"; 
xmlns:delay="http://www.protogeni.net/resources/rspec/ext/delay/1"; 
xmlns:jfed-command="http://jfed.iminds.be/rspec/ext/jfed-command/1"; 
xmlns:client="http://www.protogeni.net/resources/rspec/ext/client/1"; 
xmlns:jfed-ssh-keys="http://jfed.iminds.be/rspec/ext/jfed-ssh-keys/1"; xmlns: 
[...]
+<rspec xmlns="http://www.geni.net/resources/rspec/3"; type="request" 
generated_by="jFed RSpec Editor" generated="2021-11-22T19:20:29.068+01:00" 
xmlns:emulab="http://www.protogeni.net/resources/rspec/ext/emulab/1"; 
xmlns:delay="http://www.protogeni.net/resources/rspec/ext/delay/1"; 
xmlns:jfed-command="http://jfed.iminds.be/rspec/ext/jfed-command/1"; 
xmlns:client="http://www.protogeni.net/resources/rspec/ext/client/1"; 
xmlns:jfed-ssh-keys="http://jfed.iminds.be/rspec/ext/jfed-ssh-keys/1"; xmlns: 
[...]
   <node client_id="DB" exclusive="true" 
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
     <sliver_type name="raw-pc">
       <disk_image 
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
     </sliver_type>
     <hardware_type name="grvingt-nancy"/>
-    <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="156.0" 
y="70.5"/>
+    <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="156.0" y="70"/>
   </node>
   <node client_id="Exchange" exclusive="true" 
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
     <sliver_type name="raw-pc">
@@ -19,7 +19,7 @@
       <disk_image 
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
     </sliver_type>
     <hardware_type name="grvingt-nancy"/>
-    <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="422.0" 
y="72.5"/>
+    <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="422.0" y="70"/>
   </node>
   <node client_id="Proxy" exclusive="true" 
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
     <sliver_type name="raw-pc">
@@ -40,4 +40,11 @@
     </sliver_type>
     <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="554.5" 
y="156.5"/>
   </node>
+  <node client_id="DNS" exclusive="true" 
component_manager_id="urn:publicid:IDN+am.grid5000.fr+authority+am">
+    <sliver_type name="raw-pc">
+      <disk_image 
name="http://public.lille.grid5000.fr/~bfhch01/taler-debian11.dsc"/>
+    </sliver_type>
+    <hardware_type name="grvingt-nancy"/>
+    <location xmlns="http://jfed.iminds.be/rspec/ext/jfed/1"; x="638.0" y="70"/>
+  </node>
 </rspec>
\ No newline at end of file
diff --git a/experiment/scripts/cleardns.sh b/experiment/scripts/cleardns.sh
deleted file mode 100644
index b30cde2..0000000
--- a/experiment/scripts/cleardns.sh
+++ /dev/null
@@ -1,33 +0,0 @@
-#!/bin/bash
-# Clears the DNS server for the new experiment
-set -eux
-
-source ~/.env
-source ~/scripts/helpers.sh
-
-function del_ddn() {
-  nsupdate -k /root/ddns.key -v << EOF
-server ${DNS_HOST}
-zone ${DNS_ZONE}
-update delete ${1}
-send
-EOF
-}
-
-dig -t AXFR ${DNS_ZONE} ${DNS_HOST}
-
-# Delete all old dns entries of the previous experitment
-while IFS='=' read -r NAME VALUE; do
-  if [[ "${NAME}" == *"_DOMAIN" ]] && [[ "${VALUE}" == *".${DNS_ZONE}" ]]; 
then 
-    if [[ "${VALUE}" =~ "${WALLET_DOMAIN}" ]]; then 
-      for WALLET in $(get_wallet_domains); do
-        del_ddn "${WALLET_DOMAIN//\*/${WALLET}}"
-      done
-    else
-      del_ddn "${VALUE}"
-    fi
-  fi
-done < ~/.env
-
-dig -t AXFR ${DNS_ZONE} ${DNS_HOST}
-
diff --git a/experiment/scripts/dns.sh b/experiment/scripts/dns.sh
new file mode 100644
index 0000000..274d85b
--- /dev/null
+++ b/experiment/scripts/dns.sh
@@ -0,0 +1,10 @@
+#!/bin/bash
+set -eux
+
+systemctl restart named
+
+# Make sure the dns is ready
+sleep 5
+
+exit 0
+
diff --git a/experiment/scripts/run.sh b/experiment/scripts/run.sh
index 8ecc77c..2f08fb6 100644
--- a/experiment/scripts/run.sh
+++ b/experiment/scripts/run.sh
@@ -8,7 +8,7 @@ source ~/.env
 # Set a dynamic domain name in our own dns
 # argument: the domain to be added
 function set_ddn() {
-  nsupdate -k /root/ddns.key -v << EOF
+  nsupdate -v << EOF
 server ${DNS_HOST}
 zone ${DNS_ZONE}
 update add $1 3600 A $(hostname -I)
@@ -74,6 +74,8 @@ case "${HOSTNAME}" in
     enable_logbackup
     exec ~/scripts/proxy.sh init
     ;;
+  ${DNS_HOST})
+    ;;
   ${WALLET_HOSTS}) 
     set_host wallet
     set_ddn "wallet.$(hostname | cut -d '.' -f1).${DNS_ZONE}"
diff --git a/experiment/scripts/setup.sh b/experiment/scripts/setup.sh
index 48f5457..20732ee 100644
--- a/experiment/scripts/setup.sh
+++ b/experiment/scripts/setup.sh
@@ -7,6 +7,12 @@ source ~/.env
 cat ~/.env | tee -a /etc/environment
 echo "G5K_USER=$(cat ~/experiment-info.json | jq -r '.user.name')" >> 
/etc/environment
 
+if [ -d /home${G5K_USER}/taler-logs ]; then
+  rm -rf /home/${G5K_USER}/taler-logs/*
+else 
+  mkdir /home/${G5K_USER}/taler-logs
+fi
+
 G5K_HOME=/root/taler/grid5k
 
 NS_IP=$(host ${DNS_HOST} | sed -n 1p | awk '{print $4}')
@@ -21,26 +27,18 @@ rm /etc/nginx/sites-enabled/default > /dev/null 2>&1 || true
 cp -r "${G5K_HOME}"/configs/* /
 
 # Send all logs about taler to promtail on the monitoring node
-sed -i "s/<MONITOR_DOMAIN_HERE>/${MONITOR_DOMAIN}/g" \
+sed -i -e "s/<MONITOR_DOMAIN_HERE>/${MONITOR_DOMAIN}/g" \
+         "s/<G5K_USER_HERE>/${G5K_USER}/g" \
        /etc/rsyslog.d/taler.conf
 
+sed -i "s/<G5K_USER_HERE>/${G5K_USER}/g" \
+        /etc/logrotate.d/taler
+
 sed -i "s/<ARGUMENTS_HERE>/${EXCHANGE_ARGS}/g" \
         /etc/default/taler-exchange
 
-# Add the dyndns key which will be used by the dns update
-cat << EOF > /root/ddns.key 
-key "ddns-key.perf.taler" {
-  algorithm hmac-sha256;
-  secret "${DYNDNS_KEY}";
-};
-EOF
-
-if ! grep -Fxq "server=/${DNS_ZONE}/${NS_IP}" /etc/dnsmasq.conf ; then
-  echo "server=/${DNS_ZONE}/${NS_IP}" >> /etc/dnsmasq.conf 
-  # Read grid5k DNS servers from resolv.conf and add them to dnsmasq
-  grep nameserver /etc/resolv.conf | \
-    awk '{print "server="$2}' \
-    >> /etc/dnsmasq.conf
+if ! grep -Fxq "server=${NS_IP}" /etc/dnsmasq.conf ; then
+  echo "server=${NS_IP}" >> /etc/dnsmasq.conf 
 fi
 
 if ! grep -Fxq "nameserver 127.0.0.1" /etc/resolv.conf ; then
@@ -48,8 +46,19 @@ if ! grep -Fxq "nameserver 127.0.0.1" /etc/resolv.conf ; then
   echo "nameserver 127.0.0.1" > /etc/resolv.conf
 fi
 
+BIND_SERVERS=$(grep nameserver /etc/resolv.conf.bak | awk '{print $2}' ORS='; 
')
+sed -i -e "s/<GRID_DNS_HERE>/${BIND_SERVERS}/g" \
+       -e "s/<DNS_ZONE_HERE>/${DNS_ZONE}/g" \
+           /etc/bind/named.conf.local
+
+sed -i "s/<DNS_ZONE_HERE>/${DNS_ZONE}/g" \
+        /var/lib/bind/perf.taler
+
 systemctl daemon-reload
-systemctl restart dnsmasq
+
+if [[ "${HOSTNAME}" != "${DNS_HOST}" ]]; then
+  systemctl restart dnsmasq
+fi
 
 if [ -f ~/scripts/taler-perf.sh ]; then
   mv ~/scripts/taler-perf.sh /usr/local/bin/taler-perf

-- 
To stop receiving notification emails like this one, please contact
gnunet@gnunet.org.