Re: gpsd time service: PPS-synchronised SHM samples rare, both sources gradually less and less frequent

[Marek Szuba]

On 2020-09-19 00:44, Gary E. Miller wrote:

> Which sounds like a hardware problem on the gpio.  Nothing in gpsd 
> can make pulses on gpio go away.

I am starting to suspect just that. Or, seeing as the weird PPS signal
appears on two different boards from two different manufacturers with
two different SoCs, possibly the pps-gpio driver.

>>> Well, that would be a harware or kernel problem.  Hard to say 
>>> since it is gone.
>> 
>> I wish it were that simple...
> 
> I am sure I never said simple.

I know, what you've said is still quoted above. I meant that it would
have been so much simpler had the problem simply disappeared on its own.

>> Unfortunately what I meant by "ppstest looked okay" was that 
>> ppstest produced the expected output regardless of how gpsd was 
>> doing.
> 
> But just above you said " values look okay at first
>> but stop updating after a while".  Which is it?

No, it was NTP samples in SHM that stopped updating. ppstest continued
reporting asserts.

>> - having shut down gpsd with the receiver still powered and 
>> maintaining the fix (confirmed by looking directly at NMEA messages
>> arriving at the serial port).
> 
> You keep saying you see things, but never show what you see. hard for
> anyone to see what you are missing when we can't see what you see.

I do not particularly feel like sharing my location data with the
Internet. Okay, seeing as you appear to be under the assumption I do not
know what I am doing (which truth be told is not a bad approach online),
let me clarify that a bit: even with gpsd shut down, GGA sentences
received from the receiver via the serial port show correct co-ordinates
and the quality flag shows the fix is valid; before getting the fix both
co-ordinates are something like 99.999 and the quality flag is 0.

> Do you know you always have a good fix?

See above. Or do you mean that it never disappears? In that case the
skyview is good but of course I cannot guarantee that I never lose the fix.

>> 1. cgps: shows a 3D fix with 13/30 satellites, reported UTC time is
>> accurate. The initial set of messages (i.e. before "TPV" start 
>> showing up is
> 
> Which is the least intersting log output you could show us.

And yet that's the output that already shows the difference between
working and non-working case, via the bps value.

>> and if I understand the documentation correctly M8L without 
>> automotive sensors connected to it will act like a M8N.
> 
> I dout that.  The only external connection to an M8L is the wheel 
> pulse.

So you are saying that M8L without the wheel pulse connected to it will
NOT act like a M8N?

>> 2. ppstest /dev/pps0: produces bursts of about 2,000 messages every
>> second.
> 
> Once again, back to hardware problems.  If you can't make ppstest 
> behave, then software that uses PPS will of course fail.
[...]
>> 3. ntpshmmon:
> 
> No point.  Always stop at the first failure.  If ppstest fails, not 
> point looking downstream.

Actually, it would have helped me immensely had you mentioned right away
that more than 2 ppstest lines per second (I did mention there being
many lines per second already in my first post) shows there is something
wrong. Then I would have indeed concentrated on debugging PPS.

>> Notice how 'bps' now reports 4,800 instead of the expected 9,600?
> 
> Either you rebooted, or something else touched the port.  That is not
> the work of gpsd.

Nope, I am pretty sure it's gpsd. I am quite confident that even on the
production server nothing even tries to touch either the serial or the
pps device (everything else aside they're restricted to root), and I am
*sure* nothing does that on the test

>> Which gave me an idea for something to try - instead of restarting
>>  anything, I simply ran
>> 
>> stty -F /dev/ttyAMA0 9600
>> 
>> on another console. Et voila, cgps immediately began reporting 
>> having a 3D fix and NTP0 samples reappeared in ntpshmmon output.
> 
> Or, you could just wait for gpsd to autobaud.

In light of the fact that gpsd had by then failed to re-establish the
connection to the receiver for over 7.5 hours, I think it was safe to
assume it wouldn't come back to 9,600 bps by itself.

>> And all this time, 'ppstest /dev/pps0' continues to work.
> 
> And yet eariler you said it failed.

No, it worked - the data kept coming in, without timeouts. That the data
has turned out to be wrong is a completely different story.

>> Is there any way of forcing it to stay in specific serial mode?
> 
> The "-s" option.  But that only masks your hardware issue.

Noted for future reference, thanks.

> You have never saaid where your u-blox came from?  Chinese parts are 
> often fakes or factory rejects.  Maybe spend the $15 and get a
> second one to test?

It's from our local hackspace, which in turn got it from a place selling
parts for RC planes. No idea where THEY in turn had got it from. Then
again, while it is possible that the receiver itself is faulty I feel
the fact it only misbehaves on GPIO (I've had it running over the
USB-TTL converter for over 24 hours straight now with no issues)
suggests to me that the problem lies elsewhere in the hardware.

>> Yet another update: two hours ago I connected the same receiver (no
>> factory reset, no reconfiguration) to the same system using a USB-TTL
>> converter (with the PPS line connected to DCD),
>
> You realize that the NEO-M8L does not output TTL levels?  Trying to
> use TTL levels will cause weird issues.

It does not? The u-blox hardware integration guide for that model
explicitly mentions UART levels between 0 and Vcc. Not to mention that
having had the oscilloscope hooked up to the receiver's TX and TIMEPULSE
lines for quite a while I logged no levels other than 0 V, 3.3 V, and
the edges, on either line. So, regardless of whether my receiver is an
L, N or some other letter, it very much appears to use 3.3V TTL levels
for its serial output.

>> Native TTL + pps-gpio, on two different systems: unusable.
>
> As expected.  No Raspberry Pi is TTL.  In fact, TTL can burn out gpio
> pins.  I have done so.

Fun. Good thing I have generally not bought into the Pi hype, then! It
SHOULD work on the other system though because that one explicitly says
its UARTs support TTL levels, will do my further testing there once I've
figured out what is wrong with pps-gpio.

-- 
MS