Re: NTPsec reports excessive jitter from GPSD/KPPS SHM

[Paul Theodoropoulos]

    Seems I sent this only to Nick, when it was meant for the list.
    
    
    
    Hi Nick,
    
    It's been a while since I set it up, let me dig around a
    bit...........
    
    okay:
    
    Here's my ntp.conf, obviously much can be ignored:
> root@ A-NTPsec: ~ # cat /etc/ntp.conf
>       refclock shm unit 1 refid PPS minpoll 1 maxpoll 4 flag4 1
>         prefer
>       refclock shm unit 0 refid GPS minpoll 1 maxpoll 4 flag4 1
>         noselect
>       restrict default kod limited nomodify nopeer noquery
>       restrict 0.0.0.0/0 kod limited nomodify nopeer noquery
>       restrict 127.0.0.1
>       restrict 192.168.1.0/24
>       restrict 98.153.238.202 ignore 
>       leapfile /var/lib/ntp/leap-seconds.list
>       logfile /var/log/ntp.log
>       driftfile /var/lib/ntp/ntp.drift
>       statsdir /var/log/ntpstats/
>       statistics loopstats peerstats protostats clockstats
>       filegen loopstats file loopstats type day enable
>       filegen peerstats file peerstats type day enable
>       filegen protostats file protostats type day enable
>       filegen clockstats file clockstats type day enable
>       server c-ntpsec                    minpoll 3 maxpoll  6 iburst
>         # u-blox
>       server b-ntpsec                    minpoll 3 maxpoll  6 iburst
>         # adafruit
>       server clock.fmt.he.net            minpoll 6 maxpoll 10 iburst
>         # Fremont
>       server clock.sjc.he.net            minpoll 6 maxpoll 10 iburst
>         # San Jose
>       server time.weber.edu              minpoll 6 maxpoll 10 iburst
>         # San Jose
>       #server gpstime.la-archdiocese.net  minpoll 6 maxpoll 10
>         iburst # Los Angeles
>       #server tock.usshc.com              minpoll 6 maxpoll 10
>         iburst # Monticello,IA
>       #server ntp2.net.berkeley.edu       minpoll 6 maxpoll 10
>         iburst # Berkeley, duh
>       
>       nts enable
>       nts key
>         /etc/letsencrypt/live/ntpsec.anastrophe.com/privkey.pem
>       nts cert
>         /etc/letsencrypt/live/ntpsec.anastrophe.com/fullchain.pem
>       nts cookie /var/lib/ntp/nts-keys
>       server ntpmon.dcs1.biz nts
>       server pi4.rellim.com nts
>       server ntp1.glypnod.com nts
>       server ntp2.glypnod.com nts
>     
    I have of course created /var/log/ntpstats dir, with owner/group
    ntp. Since I have three raspi timeservers, and generating the graphs
    is quite cpu intensive, i ship the logs to a separate dedicated
    ntpviz machine (merely a raspi 4). 
    
    On that server for ntpviz,
> apt install gnuplot
>       apt install fonts-liberation
>       apt install fonts-liberation2
>     
    I *think* that's all that's necessary for initial setup, other than
    setting up the webserver appropriately, and configuring the 'header'
    and 'optionfile' for each time-interval of graphs you want - but I
    could be overlooking scads of intermediate steps. 
> root@ A-ntpVIZ: /var/www/a-1hr # cat header
>       <!-- You may customize your top of page nav here. -->
>       <meta name="robots" content="noindex">
>       <link rel="stylesheet" href=""
>         type="text/css" />
>           <style type="text/css">
>             .headsup { background: #ffcc99; }
>       <style>
>       dt {
>       font-weight: bold;
>       margin-left: 20px;
>       }
>       dd {
>       margin-top: 4px;
>       margin-bottom: 10px;
>       }
>       .section {
>       color: #000000;
>       text-decoration: none;
>       .site-title:visited {
>       color: #000000; }
>       text-decoration: none;
>       }
>       <style>
>         body {
>         }
>         main {
>           margin-bottom: 4%;
>         }
>         .floating-menu {
>           background: slategrey;
>           padding: 5px;
>           width:120px;
>           z-index: 100;
>           position: fixed;
>           top: 10px;
>           right: 10px;
>         }
>         .floating-menu a,
>         .floating-menu h3 {
>           font-size: 0.9em;
>           display: inline-block;
>           margin: 0 0.5em;
>           color: white;
>         }
>       </style>
>       <main>
>       <nav class="floating-menu">
>       <a href="" class="moz-txt-link-rfc2396E"
>           href="https://ntpsec.anastrophe.com" moz-do-not-send="true">"https://ntpsec.anastrophe.com">Main</a>
>       <br>&nbsp;&nbsp;1 hour Stats
>       <a href="">3 hour Stats</a>
>       <a href="">6 hour Stats</a>
>       <a href="">12 hour Stats</a>
>       <a href="">1 day Stats</a>
>       <a href="">3 day Stats</a>
>       <a href="">7 day Stats</a>
>       <a href="">14 day Stats</a>
>       <hr/>
>       <a href="">Top</a>
>       <a href="">RMS Time
>         Jitter</a>
>       <a href="">RMS Freq
>         Jitter</a>
>       <a
>         href="">Freq/Temp</a>
>       <a href="">GPS</a>
>       </nav>
>       </main>
>     
    
> root@ A-ntpVIZ: /var/www/a-1hr # cat optionfile 
>       --name=A-ntpsec-1-hour-stats
>       --datadir /var/log/a-ntpstats
>       --outdir /var/www/a-1hr
>       --period=0.042
>       #--clip
>     
    
    Since I have three timeservers, it's a lot of manual configuration
    to get swaths of graphs set up, but once its in place, all that's
    left is an hourly cronjob:
    
> root@ A-ntpVIZ: /var/www/a-1hr # cat
>           /usr/local/bin/ntpviz-ALL
>       #!/bin/bash
>       LOG="/var/log/rsync.log"
>       USER="rsync"
>       DEBUG=0
>       A_NTPSEC=192.168.1.10
>       B_NTPSEC=192.168.1.11
>       C_NTPSEC=192.168.1.12
>       OPTS="--delete-before --verbose -raAXHPv "
>       DATE=`date`
>       #
>       # copy stats here from A-ntpsec, B-ntpsec, C-ntpsec
>       /usr/bin/rsync -e 'ssh -p 10 -i /home/rsync/.ssh/ed25519'
>         ${OPTS} --rsync-path="sudo rsync"
>         ${USER}@${A_NTPSEC}:/var/log/ntpstats/ /var/log/a-ntpstats
>         >> ${LOG}
>       /usr/bin/rsync -e 'ssh -p 11 -i /home/rsync/.ssh/ed25519'
>         ${OPTS} --rsync-path="sudo rsync"
>         ${USER}@${B_NTPSEC}:/var/log/ntpstats/ /var/log/b-ntpstats
>         >> ${LOG}
>       /usr/bin/rsync -e 'ssh -p 12 -i /home/rsync/.ssh/ed25519'
>         ${OPTS} --rsync-path="sudo rsync"
>         ${USER}@${C_NTPSEC}:/var/log/ntpstats/ /var/log/c-ntpstats
>         >> ${LOG}
>       echo "End transfers: $DATE" >> ${LOG}
>       #
>       cd /var/www
>       INTERVAL=( 1hr 3hr 6hr 12hr 24hr 72hr 7day 14day )
>       UNIT=( a b c )
>       #
>       for INTERVALID in "${INTERVAL[@]}"
>       do
>           for UNITID in "${UNIT[@]}"
>           do
>               /usr/local/bin/ntpviz --debug "${DEBUG}"
>         @"${UNITID}"-"${INTERVALID}"/optionfile
>               REPPUB=`date +"%a %b %d %X %Y %Z "`
>               REPSTRING="<b>Report\ published:</b>
>         ${REPPUB}<br>"
>               sed -i '/End\ Time.* /a '"${REPSTRING}"''
>         "${UNITID}"-"${INTERVALID}"/index.html
>           done
>       done
>     
    The section above with 'REPPUB'/'REPSTRING' is just because I'd
    rather have an easy reference to the _local_ time the graphs were
    run.
    
    Hopefully the bits of formatting I did above for readability will
    leak through...
    
    cheers,
    paul

On 3/6/2020 0:14 AM, Nick Burkitt wrote:

Hi Paul.

Thanks - care to share your favorite ntpd logging playlists and ntpviz command lines?

My first go only got me this:

root@MPM-4006:/etc/ntp.d# ntpviz

ntpviz: WARNING: liberation truetype fonts not found

ntpviz: ERROR: /var/log/ntpstats is not a directory

I'm using the "use-minimal-logging" configuration file, which contains only the line

driftfile /var/lib/ntp/ntp.drift

And /var/lib/ntp doesn't exist (now it does).

-Nick

------ Original Message ------

From: "Paul Theodoropoulos via" <address@hidden>

To: address@hidden

Sent: 3/6/2020 12:04:50 AM

Subject: Re: NTPsec reports excessive jitter from GPSD/KPPS SHM

On 3/5/2020 19:51 PM, Gary E. Miller wrote:

Here's a few minutes worth.

Which is not useful. The long time constants in ntpd take hours

to settle. And not helpful without seeing the other chimers.

 

If I may interject here - this is where ntpviz is akin to singing 'I was blind, but now I see'. For me it was revelatory, I thought my nifty Raspi Pi was kicking out some slick ntp riffs, and I was badly mistaken. Visualizing what's happening with graphs reveals so much more depth to what is going on. My timeserver(s) are  an order of magnitude "better" than what I'd initially put together several years ago.

 

Highly recommended.

 

-- Paul Theodoropoulos

http://www.anastrophe.com

 

 

-- 
Paul Theodoropoulos
www.anastrophe.com