Re: [Discuss-gnuradio] 3.7.9.2 Win64 binaries ready + Build scripts

[Anon Lister]

You can run powershell in VS? Neat. I've been doing it all in the
powershell terminal, (and I fell back to a cygwin terminal when I had
a moment of weakness trying to tail -f something and not wanting to
type gc -wai -ta1 <file>) To be honest I haven't even opened VS.
However, when in Rome...lets see, I had to install an extension, also
found they have a vim extension. Tab completion is nice. Will try
this.

Yeah I saw that in the comment at the top of the script. For debugging
I actually copied the Python script and removed all the other sections
but wx, because you weren't kidding about it taking a while. I guess
from what I read nmake doesn't do parallel builds. Shame. Looks like 4
and 5 completed successfully though.


So on step 6... ran into:
"glfw...shallow cloned
cloning GNURadio...fatal: Not a git repository (or any of the parent
directories): .git
fatal: Not a git repository (or any of the parent directories): .git
complete"

Cryptic. Anywho. Log:
"Submodule 'volk' (https://github.com/gnuradio/volk.git) registered
for path 'volk'
Cloning into 'volk'...
error: no such remote ref fcfc49dbb6e1e861c4fbdb02502bf18599b2fd0c
Fetched in submodule path 'volk', but it did not contain
fcfc49dbb6e1e861c4fbdb02502bf18599b2fd0c. Direct fetching of that
commit failed."

Its odd, I can get to that path on github,
https://github.com/gnuradio/volk/tree/fcfc49dbb6e1e861c4fbdb02502bf18599b2fd0c
but it doesn't show up in git log, and wont work in git checkout. It
looks like this commit's parent was merged by Nathan West on Jan 29.
Perhaps he squashed the commit somehow when he did the merge? The code
changes in that commit do appear to be in place in maint. Anyway, I'll
pick this up tonight, but I'll probably just checkout the latest, or
at least your latest commit from maint and build with that, assuming
that you will eventually point the submodule to a newer commit anyway
as this seems to be just a git issue.

-Anon

On Apr 18, 2016 10:11 AM, "Geof Nieboer" <address@hidden> wrote:
>
> Good news.  The out-file fix must have missed a commit somehow, I distinctly 
> remember making and fixing that.  Oh well, thanks.
> The other is interesting, I'm not sure why it worked on mine then.  I'm 
> already patching that file to recognize VC14, so just need to amend the patch.
>
> As you continue to experiment, in case you haven't already figured it out, 
> once you open the scripts project the first time in VS 2015, just run 
> Setup.ps1 once to set the environment, then you can run any of the scripts 
> individually, and by setting the $configuration variable (shortcuts at bottom 
> of several scripts) you can usually just run any individual "sub-build" on 
> it's own to save time and incrementally move forward.  But it looks like 
> you've already gotten past the toughest part, so that's excellent..
>
> On Mon, Apr 18, 2016 at 2:27 PM, Anon Lister <address@hidden> wrote:
>>
>> Aha. Found the error. I don't know how you didn't encounter it in your
>> builds, but in the build-wxpython.py script, line 443 fails, it's up
>> in the log. It sets install_dir to some variables defined in a if not
>> windows block higher up. This causes the script to bomb at install due
>> to an undefined variable. I just set install_dir to "" and validate
>> was successful.
>>
>> Thanks for the tip on looking for the _core_.pyd, that's what lead me
>> to it, it wasn't present anywhere but in the build directory (and a
>> few other places on my box, but all from timestamps way before the
>> build). Anyway, that step of the script is rebuilding now, just to be
>> sure it works.
>>
>> I dug into the Step5-consolidation error a bit, it looks like its failing 
>> with
>>
>> "Consolidating Qt...Out-File : A parameter cannot be found that
>> matches parameter name 'path'.
>> At C:\gr-build\scripts\Step5-ConsolidateLibs.ps1:67 char:23
>> +     "[Paths]" | out-file -path $root/build/$configuration/bin/qt.conf ...
>> +                          ~~~~~
>>     + CategoryInfo          : InvalidArgument: (:) [Out-File],
>> ParentContainsErrorRecordException
>>     + FullyQualifiedErrorId :
>> NamedParameterNotFound,Microsoft.PowerShell.Commands.OutFileCommand"
>>
>> According to [1] out-file accepts -FilePath as an option. I switched
>> to that (actually just ran that line on console) and it seemed to
>> work, so that one may be an easy fix. I'll kick it off once the python
>> builds finish. Probably wont get back to it till tonight.
>>
>> Yeah my goal here is mainly to reproduce the build, (and use it as a
>> motivating project to learn a little powershell), then hopefully
>> contribute to getting my favorite OOT's building. So no taking the
>> cheap route. :) I'm still amazed you got all this stuff building on
>> windows, most of it is a pain to build even on its native platform.
>> heh.
>>
>> -Anon
>>
>>
>> [1] https://technet.microsoft.com/en-us/library/hh849882.aspx
>>
>> On Apr 18, 2016 4:02 AM, "Geof Nieboer" <address@hidden> wrote:
>> >
>> > Anon,
>> >
>> >>
>> >> which seems to just be a sanity check that something we expected to build 
>> >> was not
>> >
>> >
>> > Exactly right.  It's not perfect but usually works.
>> >
>> >> ran setup.ps1 before I saw the ~RUNME_FIRST.ps1
>> >
>> >
>> > ~RUNME_FIRST creates the basic directory tree for the build in the 
>> > location you specify. You will want to re-run that with I:/gr-build as the 
>> > root to ensure all the directories you need were created (but that's 
>> > likely not the problem with WX but might be related to the consolidation 
>> > fail).  You can abort it after it starts running the Step1 script.
>> >
>> >>  just symlinked C:/gr-build to I:/gr-build
>> >
>> >
>> > Hmm, you shouldn't have had to do that, I will dig deeper there.
>> >
>> >> whats special about wx
>> >
>> >
>> > WX is just plain special.  It was problematic to get to build 
>> > consistently, so I'm not surprised that it is where your build failed, 
>> > unfortunately.
>> >
>> >> I did see in the comments something about debug builds not working 
>> >> correctly,  and there's a workaround
>> >
>> >
>> > Yes, that comment is about building WX itself in a debug mode, not the 
>> > overall build.  The workaround is already in place which just to always 
>> > builds WX in release.
>> >
>> >> Any ideas what could be wrong?
>> >
>> >
>> > Well, like you said, the build logs look clean, which is certainly odd.  I 
>> > think I would start with trying to search the computer for _core_.pyd and 
>> > see where copies of it are, perhaps it's installing the package in the 
>> > wrong location?  Then I'd also look for the slew of Wx DLLs, they should 
>> > be in $pythonroot/DLLs  I'm fairly certain.
>> >
>> > Later when I'm back at the machine I will rebuild mine and compare the 
>> > logs side by side and perhaps something with jump out.  Obviously a cheap 
>> > workaround is to download the dependency pack and manually copy over the 
>> > WX-related dirs in site-packages and the DLLs into /DLLs.  But that 
>> > doesn't fix the underlying issue.
>> >
>> > Geof
>> >
>> > On Sun, Apr 17, 2016 at 9:20 PM, Anon Lister <address@hidden> wrote:
>> >>
>> >> Yep, my bad on UHD, used to doing git builds and seeing 3.10. Cool yeah 
>> >> gqrx would be the priority, baz is just a nice to have (and its a bit 
>> >> dated as its all in WX which seems to be going the route of deprecation).
>> >> So for some feedback, I installed the binaries and everything seems to 
>> >> work good. Hooked up the UHD and was able to receive a decent sample rate 
>> >> without issue. Only small issue is was a popup on first run that said:
>> >>
>> >> "The xterm executable 'xterm' is missing.
>> >>
>> >> You can change this setting in your gnuradio.conf, in section [grc], 
>> >> 'xterm_executable'.
>> >>
>> >> (This message is shown only once)"
>> >>
>> >> Which is understandable, as xterm is not built.  Anyway, after clicking 
>> >> ok everything works.
>> >>
>> >> On the build side, first off, I'll say while I'm quite proficient on the 
>> >> Linux side, on the windows side I'm pretty noobish, so the first errors I 
>> >> received (missing io.h header, among other errors), appeared to be due to 
>> >> me having old VS 2012 only half removed which confused 14.. Anywho, after 
>> >> figuring that out and cleaning it up everything went good till the WX 
>> >> python build. But first I'll throw in that I did install VS2014 into an 
>> >> alternate path, this caused several issues with checks, I ended up 
>> >> creating the windows equivalent of a symlink (or maybe multi-disk 
>> >> hardlink, think its called a junction) and that solved that issue, but 
>> >> also the scripts ask for a location to place its build files, I tried 
>> >> I:/gr-build but received errors where directories in C:/gr-build were 
>> >> missing, I just symlinked C:/gr-build to I:/gr-build and things seemed 
>> >> fine. I only bring it up since the build scripts asked for a location. 
>> >> Perhaps it was something I did as I first ran setup.ps1 before I saw the 
>> >> ~RUNME_FIRST.ps1. Anyway, back to the WX error:
>> >>
>> >> Got the error "Validation Failed, 
>> >> C:/gr-build\src-stage2-python\gr-python27\lib\site-packages\wx-3.0-msw\wx\_core_.pyd
>> >>  was not found and is required" which seems to just be a sanity check 
>> >> that something we expected to build was not(i did independently verify 
>> >> its not there, theres no wx* in site-packages), I've uploaded the log for 
>> >> wx-python[1], let me know if you need any more info.
>> >>
>> >> The build appears to stop there. Not sure where to go with debugging as 
>> >> there appears to be no error, only some warnings? I'm also not familiar 
>> >> with wheel, though one would assume it produces the whl installer files. 
>> >> I did test removing the WX block and Step4 runs to completion. So it 
>> >> appears to be something with WX. I did see in the comments something 
>> >> about debug builds not working correctly,  and there's a workaround. For 
>> >> completeness, I went ahead and commented out the Verify line and all 
>> >> builds (AVX, ReleaseDLL, Debug ) appear to generate this same log (except 
>> >> for folder names and such), and all do not generate the appropriate pyd 
>> >> file.
>> >>
>> >> Any ideas what could be wrong? Or maybe whats special about wx as 
>> >> compared to the other python modules? If it was a bunch I would suspect 
>> >> something in my env may just be incorrect(Which, very well may be the 
>> >> case), but it seems just that one library.
>> >>
>> >> P.S. I did just try to bypass it for debugging purposes, but I ended up 
>> >> with a different error on the consolidation step(not with WX) but lets 
>> >> take those one at a time.
>> >>
>> >> Also, if you want, I can take this to github/issues.
>> >>
>> >> -Anon
>> >>
>> >> [1] http://filebin.ca/2e5LIv6ROV29/39-ReleaseDLLwxpython.txt
>> >>
>> >> On Fri, Apr 15, 2016 at 3:43 AM, Geof Nieboer <address@hidden> wrote:
>> >>>
>> >>> Anon,
>> >>>
>> >>> UHD 3.9.3 is the most current release from Ettus.
>> >>>
>> >>> gqrx is on my to-do list, and in fact there are code stubs in the 
>> >>> scripts already for it.  What stopped me is that it needs qt5 built, 
>> >>> whereas GR is on qt4 still, so I wanted to get out what I had first 
>> >>> before re-attacking that one.  But once I have that done (and Qt5 
>> >>> appears way easier to build than 4), then I'll get cracking on it.  
>> >>> gr-baz I have not taken a look at, but I will do so.
>> >>>
>> >>> On the downloads page in the website, I tried to keep track of which 
>> >>> library required what other libraries as I was going through it, so 
>> >>> you'll see that listed, perhaps that might be useful in creating a 
>> >>> dependency graph.  I don't guarantee it's 100% complete, though.
>> >>>
>> >>> Yes, please let me know how the build goes.  Take a look at the "Issues" 
>> >>> in the readme.  Biggest recurring irritation I had was downloads that 
>> >>> would fail partway through then appear later as build failures.  I would 
>> >>> like to move to a hash verification step after the download, but for the 
>> >>> moment it is what it is.  And once the downloads have been successful 
>> >>> once, you should be smooth sailing as it will keep them cached in the 
>> >>> 'packages' subdir.
>> >>>
>> >>> Geof
>> >>>
>> >>> On Fri, Apr 15, 2016 at 9:41 AM, Anon Lister <address@hidden> wrote:
>> >>>>
>> >>>> This is awesome, I will test out the build process this weekend on 10. 
>> >>>> Any reason for the slightly older uhd release? I'd love to get gr-baz 
>> >>>> and gqrx working on Win.
>> >>>>
>> >>>> I'm also somewhat interested in stealing a pre-compiled list of 
>> >>>> dependencies for my somewhat crazy project of building GR + some OOTs 
>> >>>> on RHEL 6. Talk about dependency hell. ;p
>> >>>>
>> >>>> On Apr 14, 2016 6:24 AM, "Geof Nieboer" <address@hidden> wrote:
>> >>>>>
>> >>>>> All,
>> >>>>>
>> >>>>> Some may recall in the fall I posted a link to a beta windows 
>> >>>>> installer.  I'm happy to report that I'm releasing new versions today 
>> >>>>> for the 3.7.9.2 release, compatible with Windows 7/8/10.  All 
>> >>>>> dependencies are included, and all are built natively using MSVC 2015, 
>> >>>>> no Cygwin or MinGW required. It's about a 300MB package download.
>> >>>>>
>> >>>>> I've also refactored the entire build process used to make the msi's 
>> >>>>> and gotten it down to a series of Powershell scripts that can either:
>> >>>>> 1- Build the entire GNURadio windows dependency chain from source and 
>> >>>>> then build GNURadio itself.
>> >>>>> 2- Download a prebuilt "dependency pack" as binaries and then build 
>> >>>>> GNURadio and a couple OOT modules
>> >>>>>
>> >>>>> The binaries (for both GR and the dependencies) can be found at 
>> >>>>> http://www.gcndevelopment.com/gnuradio.  The scripts themselves are 
>> >>>>> hosted at http://github.com/gnieboer/gnuradio_windows_build_scripts.  
>> >>>>> While the binaries have no dependencies, the build scripts have 
>> >>>>> several, but all mandatory dependencies are free to install.  The 
>> >>>>> various patches required to make everything build on Win32/MSVC are 
>> >>>>> either workarounds built into the scripts, patches downloadable on the 
>> >>>>> website, or forked repos on my github account.  For the most part pull 
>> >>>>> requests have been submitted upstream.
>> >>>>>
>> >>>>> All GR components except gr-comedi are installed, and several OOT 
>> >>>>> blocks are also included by default, including UHD 3.9.3, gr-fosphor, 
>> >>>>> and gr-osmosdr with most drivers.  The windows audio sink has also 
>> >>>>> been refactored to double buffer to avoid the skipping others have 
>> >>>>> reported.
>> >>>>>
>> >>>>> It uses OpenBLAS for numpy/scipy to stay GPLv3 compliant...users can 
>> >>>>> replace it with an MKL-based version as a wheel from the downloads 
>> >>>>> page should more performance be desired.
>> >>>>>
>> >>>>> More information is available on the website.  I hope both the 
>> >>>>> binaries and scripts are useful and look forward to feedback.
>> >>>>>
>> >>>>> Geof
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> _______________________________________________
>> >>>>> Discuss-gnuradio mailing list
>> >>>>> address@hidden
>> >>>>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>> >>>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> Discuss-gnuradio mailing list
>> >>>> address@hidden
>> >>>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>> >>>>
>> >>>
>> >>
>> >
>>
>