Re: [gpsd-users] using the python gps module without scons install

[shouldbe q931]

Hi Gary,

On Thu, Mar 7, 2019 at 11:04 PM Gary E. Miller <address@hidden> wrote:
>
> Yo shouldbe!
>
> On Thu, 7 Mar 2019 22:53:33 +0000
> shouldbe q931 <address@hidden> wrote:
>
> > >
> > I did not have any issue building gpsd, which obviously needs scons to
> > be run in the directory that gpsd has been cloned to or untarred to.
>
> Uh, no.  I never have sscons in my gpsd source directory.  Being in
> your PATH is sufficient.

To clarify the point, I was not suggesting that he scons "application"
needed to be in the gpsd/ directory, perhaps the below wil be clearer
to parse ?

I did not have any issue building gpsd, which obviously needs the
scons commands (commands that are either "scons" on its own, or
"scons" followed by a space and then further words such as "scons
timeservice=yes", or "scons check" or "scons install") to be run in
the directory that gpsd has been cloned to or untarred to.

> > Because I had misunderstood from the python documentation what path I
> > needed to provide in $PYTHONPATH for ntpsec to "build" ntploggps,
>
> Uh, no.  PYTHONPATH is not required to "build" ntploggps.  You just
> need the prerequisites in your sys.path.  Which is easy if you
> installed your distro packages for the prerequisites.

The distro package for the python gps module also installs gpsd (its a
dependancy), which would run counter to what I am attempting to
acheive, yes I am aware that I am able to force the install without
the dependancy, but I would prefer to build from source.

> > I
> > was under the misunderstanding that "something else" needed to be done
> > after building gpsd, rather than as Rob pointed out, simply using the
> > gpsd directory in $PYTHONPATH instead of gpsd/gps
>
> Correct.  That is the clockmaker way.  Not the normal way.

The clockmaker script does not build ntploggps, from your previous,
and my reponse above, you would appear to be suggesting that to build
ntploggps, one should install a distro gpsd rather than build gpsd
from source, have I misunderstood something here ?

> > > Try not to confuse how you install and use gpsd with clockmaker,
> > > versus the normal install and useage.
> > >
> > I don't see a difference between the two apart from just calling scons
> > on its own and calling scons with several "configure alike" statements
>
> Yes, you do not see the differences.  scons is NOT one of them.

One of what ?

> Try a normal gpsd install, using the build.txt file, and youll see
> the differences.

Well, yes, as per my original email on the subject, I'm very aware
that if I run scons install it will copy the files for the gps module
into /usr/local/lib/python2.7/dist-packages/gps/ . And as per my
original email, I was attempting to avoid adding another step that
required root

> > > > is waf/python also searching for a matching
> > > > directory name, or is there something else in the gpsd directory
> > > > that it is using ?
> > >
> > > Lost me.  There is no waf/python.
> > >
> > Apologies, I used "waf/python"  instead of "waf and/or python"
>
> Still lost me.  waf is a python program,  there is no "or".

I used and/or as I was unsure as to the correct terminology, waf is
searching for the module, but waf is a python program, and using
python to search, so in an attempt to reduce any frustration by my
using the incorrect term, I provided both.

The text that I was asking the question about is "When a module named
spam is imported, the interpreter first searches for a built-in module
with that name. If not found, it then searches for a file named
spam.py in a list of directories given by the variable sys.path"

To highlight
1. searches for a .py file
2. in list of directories given by the variable sys.path

The python gps module appears (from the presence of the _init__.py) to
be in the gpsd/gps directory, not in the gpsd directory.

 is python also searching for a matching directory name, or is there
something else in the gpsd directory that it is using ?

> > > When you run a python program in the location that clockmaker tells
> > > you to, then the gps is in that directory (.), which is in sys.path.
> > >
> > In the clockmaker script, ./waf configure (with some additional
> > options) is run in ntpsec/, not in gpsd/
>
> Uh, no.  Look again.  A lot of "cd"ing going on.

I disagree.

There are three instance of waf in clockmaker
Line 338  and 339 are clearly run in ntpsec/

    os.chdir("ntpsec")
    os.system("./waf configure --refclock=nmea,pps,shm,gpsd")
    os.system("./waf build")
    os.chdir("..")

Line 371 is also cleary run in ntpsec/
    os.system("(cd ntpsec && ./waf install) && cp ntp.conf /etc")

Just in case you meant scons instead of waf, there are also three
instances of scons in clockmaker

Line 214 when it is being installed

Line 328 where it is clearly being run in gpsd/
    os.chdir("gpsd")
    os.system("scons timeservice=yes magic_hat=yes nmea0183=yes ublox=yes "
              "mtk3301=yes fixed_port_speed=9600 fixed_stop_bits=1")
    os.chdir("..")

Line 364 where again it is clearly being run in gpsd/
    os.system("(cd gpsd && scons install)")

> > I'm not trying to make a "better" clockmaker script, I'm using it as
> > an initial base to learn how to be able to a scripted install of gpsd
> > and ntpsec running as separate services, and to have the logging for
> > ntpviz installed as well
>
> Which is to misunderstand the purpose of clockmaker.  That has led, and
> will lead, to much confusion.

Purpose of clockmaker "Configure an SBC as a single-purpose timeserver."

My purposes, add logging of TDOP and number of visible satellites to
clockmaker, learn what makes the clockmaker script "tick" (yes I know
it's an appaling pun)

> Do it the "clockmaker" way, or do it the normal way.  Mixing them up
> is confusing you.

What was confusing me was a misunderstanding on the path that was
required for $PYTHONPATH

In an attempt to get everything in one email, I am also going to add
below from your previous reply.

> > I have to disagree, when "scons install" is run as root it puts files
> > in several paths that are not in the user's home directory
> > /usr/local/, /usr/local/lib/
> > /usr/local/lib/python2.7/dist-packages/gps, and /usr/local/include
> > I have seen this bahaviour under rasbian on a pi and in the Ubuntu VM
>
> Which is why the clockmaker script does not tell you to do that!

from the howto
This step can be automated with "./clockmaker --install" done as root.
# cd gpsd; scons install

if the instructions from build.txt are followed, the same paths have
files copied to them, build.txt includes "The default value is
"/usr/local"", could it be that you are using a different distro or
have other configuration options provide different results ? such as
"scons prefix=/home/pi"

> Don't mix up recipes until you are a good chef.  The clockmaker
> purpose and install and not the same as the normal purpose and install.

1. I would say that taking something that works one way, and learning
how to modify it to work a different way, is a key part of becoming a
chef, another key part is experimention.
2. I presume you meant "and install are not the same", to which I
would say that the original purpose has been subverted.

> Or just use "scons uninstall".  Can't get easier than that.

I had not seen "scons uninstall" in build.txt

> > > I see an ntpsec configure, but no gpsd configure?
> > >
> > Sorry, you've lost me, I was under the obviously mistaken
> > understanding that scons without install was a combined configure and
> > build, with the configure "options" being passed on the command line.
>
> Yes, mistaken.  Not how you think. You sent me your ntpsec config, not your
> gpsd config.  We are talking about gpsd here.

Ok, so the "timeservice=yes" are build options, not configure options,
there is no equivalent to ./waf configure in either build.txt or
clockmaker, I do not understand what you asked to see considering that
I had also posted the bash script I had used, which I would have hoped
made it obvious exactly what commands I was running to duplicate what
I was seeing.

Just to reiterate, I was not suggesting that either gpsd or ntpsec had
a fault, I was trying to understand why what I had thought would work
did not work, and from the reply from Rob, I now have something that
works.

Now I have found
https://chrisyeh96.github.io/2017/08/08/definitive-guide-python-imports.html#basics-of-the-python-import-and-syspath
which has in it

"Technically, Python’s documentation is incomplete. The interpreter
will not only look for a file (i.e. module) named spam.py, it will
also look for a folder (i.e. package) named spam"

So I made three mistakes, a newbie mistake to expect the python.org
documentation to be complete, a typo in just the wrong place in the
first post, and not to show how I could reproduce what I was seeing on
a reduced system (such as the bash script) in my first post. I have
learnt from these.

I can now go back to hacking around on clockmaker (-:

Thanks!

Arne