Re: Duplicate or near-duplicate messages on u-blox M8

[Gary E. Miller]

Yo Luke!

On Wed, 26 Feb 2020 17:45:37 -0700
Luke Hutchison <address@hidden> wrote:

> On Wed, Feb 26, 2020 at 4:30 PM Gary E. Miller <address@hidden> wrote:
> 
> > That is info on the board, we need the GPS model.  You can find it
> > this way with gpsd running:
> >        ubxtool -p MON-VER
> >  
> 
> Here you go.. warnings and all :-)

Warnings that are not present in 3.20 or git head.  That is what happens
when you mix new (python 3.8), old (gpsd 3.19) and really old (gps_udm)
code.

I guess the thoughtful devs at Fedora thought Python deserved updating,
but the matching gpsd did not.  Feel free to file a bug report with
Fedora.


> UBX-MON-VER:
>  swVersion EXT CORE 3.01 (107900)
>  hwVersion 00080000
>  extension ROM BASE 3.01 (107888)
>  extension FWVER=SPG 3.01
>  extension PROTVER=18.00
>  extension MOD=NEO-M8N-0
>  extension FIS=0xEF4015 (100111)
>  extension GPS;GLO;GAL;BDS
>  extension SBAS;IMES;QZSS

So, a plain NEO-M8N, with flash.

> Yup.  Such are the problems with NMEA.  That is why u-blox binary
> > mode is preferred.
> >  
> 
> Sure, but surely cgps could simply not generate location updates with
> SKY messages?

You confuse TPV with mere location.  The data in the SKY is used to
provide additional error estimates to the TPV data.

> Either way, the GPS should always be in binary mode as far as I am
> aware, if that is gpsd's default.

Your choice.  When gpsd tries to enforce receiver settings we get
complaints.  Everyone's needs are different.

> However, I just tried running `gpspipe -R -n 100 > /tmp/raw2.log`
> again, and this time the log is in NMEA mode. See attachment. I
> didn't even touch the receiver, other than maybe unplugging it and
> plugging it back in.

Oh, you ONLY did a hard reset?  And you do not expect that to change
things?

> So gpsd is not even reliably selecting binary
> mode for some reason.

gpsd does not select the mode, it detects the current mode.

> Restarting the gpsd daemon and unplugging /
> replugging the GPS receiver switched it back to binary mode.

And I'll ask again, how are you starting the gpsd daemon?  With what
options?

> > The slight drift in location during these  
> > > non-location updates is probably due to Kalman filtering or
> > > something.  
> >
> > Nope.  gpsd does no filtering.
> >  
> 
> Then why would a SKY message, which contains no location information,
> generate a location update that was not the same as, but a tiny bit
> offset from, the previous location?

You are confused on the order of things.  SKY is an output from gpsd,
not an input.  So SKY can't affect the TPV in any way.

> Here's a plot of lat/long over
> time for a few seconds of GPS data.

Useless without the accompanying raw receiver data and the method that
you used to create the plot.

> Note the near-double-ups (or
> triple-ups). (This is a plot of the lat/long/altitude table I sent in
> my previous message):

And still useless.

> This combination of responses was a little bewildering:

Welcome to the world of GNSS.

> > I'll ask again for you geoid separation.
> > This shows us what is in your raw.log:
> >     ubxtool -f raw.log -v 2  
> 
> 
> As I said I didn't know what geoid separation was or how to find it.
> I did send the log as requested. You even opened the log attachment
> and found the geoid separation :-D

Yup.  Pay attention and remember how I showed you to calcualte it.

> > 3.20 is the latest release.  No idea here how Fedora may have
> > miunged it. 
> 
> gpsd 3.20 was released on 2020-01-01. Fedora 31 was released on
> 2019-10-29. Fedora usually only updates package versions between
> releases if they need to address critical bugfixes.

And yet, they updated to Python 3.8, which "broke" gpsd as you showed
above.  But that is between you and them.

> How are you starting pgsd?  With systemd?  Or?  Are you using the "-n"
> > option.  These affect gpsd operation.
> >  
> 
> With systemd:

Oh.  Another mess.  I don't do systemd, that recipe is not part of gpsd.
So I'll just say it is wrong, and suggest that you not do that.  If
a systemd person ever figures out how to reliably use gpsd with systemd I
hope they help you.


> > Are you sure?  Did you reset the device?  Many receivers get changed
> > from the defaults before shipping.
> >  
> 
> In that case no, I'm not sure the settings are default. I am using the
> device as shipped, except that I updated the firmware using the
> u-blox tool to the latest (3.01) release. But you also now have the
> output of `ubxtool -f raw.log -v 2`, so you should be able to see
> what the settings are -- right?

Yeah, except now you say your receiver is sending NMEA, so not correlated
to your current settings.  Even if nothing changed, that raw log is
only indicative of your settings, very few of the ettings can be deduced
from your raw log.  But, as that is not directly related to your problem
we can table that for now.


> > That looks very different than you sent last time.  You previously
> > sent NMEA as the output of your receiver, and you just sent u-blox
> > binary.  NMEA != binary.  So either you did not collect what you
> > thought you collected, or you changed something.
> >  
> 
> As I mentioned, last time I just used gpscat to capture the log,

Without mentioning exactly how you used it.

> because that's what your FAQ suggested doing:

My point stands.  Something is changing on your end.

>  In my previous message I followed the request from your previous
> message, and instead used `gpspipe -R -n 100 > raw.log`. Hence the
> difference. You need to update that part of the FAQ too.

gpspipe did not change your recevier into binary mode.  Something else
did.  I expect systemd is part of the problem...

> Of course, I am trying to follow your instructions closely, but they
> were different from what I sent initially because I followed the FAQ
> closely the first time around, before posting to this mailing list.

The details are less important that you understand what is going on.
Otherwise I would take the time to point out where you differed from
the troubleshooting doc, but that would be a distraction.

> Please be patient as I catch up, I'm totally new to the GPS device
> and protocol world.

Patience is not really what you want from us, you want answers, and I'm
trying to give those to you.  Many of us are happy to take the time
to help people get it right.  GNSS do not really work the way you think
they do, so try to be flexible.

> OK, thanks for explaining. I just tested this:

I'm unclear what you tested and how?

> height = 1332.800
> hMSL = 1351.616
> geoid separation = -18.816 (same exact value as before for the same
> location, despite a drift in altitude measurement)

Where did you get that?  cgps, xgps, ubxtool, gps_udm?

It helps if you explain how you get things, otherwise I'm lost w/o
context.

> Now the altitude is stable between messages,

And nothing changed?  That is really random.

> but the epv, epx, epy
> values are oscillating.

Yes, that is very normal.  As more data comes in, gpsd updates the
error estimates.  Don't take them seriously, what they actually mean
is undocumentned by u-blox.  Use them as a figure of merit.

> Here is the output of `gpspipe -w` for two seconds:

Sigh, useless.  Send the raw data (gpspipe -R) not the processed data.
With the raw data I can then recerate the processed data and see
how they match.

> Notice that the message order for a single update is SKY -> TPV ->
> GST -> TPV. The two TPV messages differ from each other in ep*
> values, e.g. epv keeps switching back and forth between 28.060 and
> 9.600.

Yes, this is normal.  As I previously explained, the first TPV is sent
as soon as minimum data is available, the last TPV, with updates, after
all the cycle data is collected.  This is normal.

> Previously I assume the two TPV messages also differed in
> altitude, since xgps showed the altitude oscillating between the two
> at the same time as these error messages were also fluctuating. I
> don't know why altitude is consistent now.

More randomness...  At a minimum we know you have been flipping
between NMEA and u-blox binary.  They behave very differently.  Plus
the randomness from systemd(ung).

> > Dunno what that means...
> >  
> 
> ROS is the Robot OS, it's a pub-sub message bus for robotics
> components to talk to each other. GPS is widely used in the ROS
> world, and unfortunately the gps_umd driver is the standard way to
> talk to a GPS device.

I looked at the gps_udm code.  Last update 7 years ago!  Any robot using
that will soon have 3rd law violations and be sent to the scrap heap.
You use that old broken stuff, that is your problem, not gpsd's.

> > They do not really support gpsd 3.19.
> >  
> 
> The code is junk, for sure. Do you have any specific issues you can
> point out, and/or pointers to a good (simple) gpsd client that could
> be studied to produce a better ROS gpsd client?

I have many specific issues with that code, but I have more than enough
issues with the gpsd to care.  I see a 4 year old MR waiting on the
gps_udm code.  Not gonna waste my time on that.

> I say "simple" because all most ROS users care about is lat, long,
> altitude, and possibly covariance measures for each axis, and they
> just want everything to work out of the box.

Scary.  That is how we get robots (Teslas) that kill people.

It ie really important to know you have a valid fix before acting on it.
It ie really important to know the error bounds on a fix before acting
on any fix data.

Start by removing "altitude" from your vocabulary.  It is far too vague
a term to be used by anyone when failure has consequences.  Use
"haight above ellipsoid" or "altitude MSL".  Then a matching datum.  A
height without a datum is useless.

RGDS
GARY
---------------------------------------------------------------------------
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
        address@hidden  Tel:+1 541 382 8588

            Veritas liberabit vos. -- Quid est veritas?
    "If you can't measure it, you can't improve it." - Lord Kelvin

pgpKz_g8paRzw.pgp
Description: OpenPGP digital signature