Re: [Discuss-gnuradio] [USRP-users] Gnu Radio apps freezes (locks up)

[Nick Foster]

On Tue, Apr 10, 2012 at 1:16 PM, Rickard <address@hidden> wrote:

On 31 mar 2012, at 15.43, Tom Rondeau wrote:

> On Thu, Mar 29, 2012 at 6:04 AM, Rickard Radio <address@hidden> wrote:
>>
>> On Mar 29, 2012, at 3:26 AM, Tom Rondeau wrote:
>>
>>> On Wed, Mar 28, 2012 at 10:02 AM, Rickard Radio <address@hidden> wrote:
>>>> Hi list,
>>>>
>>>> After I upgraded to latest Gnu Radio 3.5.2, and latest UHD (and images), GR applications just freeze when running. No warnings, error messages or overflows etc. Just freeze.
>>>> A simple FFT plot directly on received samples from the USRPN210 just freezes after some seconds, or minutes (depending on the sample rate), although the load on the machine is not high. Need to kill GR-app and restart it, with the same problem occurring again.
>>>>
>>>> This has never been a problem with earlier versions of GR/UHD, from about 6 months ago.
>>>> The freezing happens quicker with high sample rate setting but also with lower, eventually. No overflows happen (which was possible to get before with too high sample rates or load, etc.)
>>>>
>>>> The USRPN210 stops sending samples to the computer at the same moment as GnuRadio freezes (as observed on the system monitor).
>>>>
>>>> Same thing happen on two identical laptops running Ubuntu 10.10 (also upgraded it from 10.04). Not sure if its a strict GnuRadio problem (since it worked before), UHD, or some problem with the Ubuntu Linux 10.10. It work(ed) flawlessly with another machine on OSX (before I tried to upgrade GR on it but then got stuck...) with identical UHD version and images.
>>>>
>>>> Installation of UHD+GnuRadio with the automatic linux script runs without any problems, as before, no errors or warnings.
>>>>
>>>> Any de-freezing help or clues appreciated!
>>>>
>>>> Rickard
>>>
>>> Rickard,
>>>
>>> Just to be clear. When you install 3.5.2 from the tarball, it freezes.
>>> When you use the build-gnuradio, everything works fine?
>>>
>>> What's your machine?
>>>
>>> Tom
>>
>> Tom,
>>
>> The installation with build-gnuradio script works just fine, as before (no tarballs).
>> Same result on both laptops, Acer Aspire TimelineX with i3 processors (2.26 GHz),  running Ubuntu 10.10.
>> I did not have this problem earlier with Ubuntu 10.04. Or on a Mac with OS X (with the source from git).  Could Ubuntu 10.10 cause the problem somehow?
>>
>> Note: Halting/freezing only happens when running an application (with the N210) as a receiver. (Not transmitter, see below.)
>> The flow of receiving samples just halts after a while and the application freezes/halts (as a consequence).
>> This happen sooner with high sampling rate (after a few seconds with 25MSPS), but eventually also with a bit lower sample rates. The CPU's are not overwhelmed (< 50%).
>> It happens even if the UHD usrp source is connected directly to a null sink only. I do not get any overflows before the halt.
>> In fact, I cannot even provoke a continuos stream of overflows since the reception just halts instead of producing overflows, which was the result earlier.
>>  GnuRadio itself does not freeze as a whole (like the grc in the background), just the running application, which I then need to abort.
>>
>> Strangely enough, this "freezing/halting" does NOT happen when transmitting, correspondingly, even with high transmit sample rates such as 25 MSPS (or now possible even with 50MSPS with 8 bit/samples!). Then it works just fine - even without underruns (when using just a file source).
>>
>>
>> / Rickard
>
>
> Rickard,
>
> Thanks for the data. Unfortunately, I have no idea what to make of
> this. There isn't that much difference between the last release
> (3.5.2.1) and what you get using the build-gnuradio script. That just
> grabs the latest master version from our git repo, and we haven't done
> all that much since the release. That really doesn't make a lot of
> sense.
>
> How's your git? If you're comfortable doing so, can you check out the
> v3.5.2.1 tag on git and try that one instead of the tarball release?
>
> Thanks,
> Tom

Thanks for your info.  My current take on this interrupt issue:

The problem may not be gnuradio or uhd's "fault", I now suspect. Instead somehow the network connection and its settings might cause the interrupts. However, I am no linux guru so I am learning at the same time as I am doing.

First, I've updated Ubuntu, from release 10.10 to 11.04, but then nothing worked (just got a completely blank screen without any gui), so fast-forward upgraded to 11.10 and then both laptops came right back on track. I then also updated the gnuradio+uhd to latest (3.6.0) version using the excellent build-gnuradio script (as before), which itself went just fine (also as before).

Unfortunately, the exact same problem with interrupts (total halt) in the receiver at high sample rates persists. Note, as before, this happens (only?) with very high rates over the Ethernet (about 400 mbit/s or more, the higher rate the sooner it halts, typically just a few secs) although the computer display no overruns or other errors.

Then by jacking around with the MTU setting (100,500, 1500,5000, etc), increasing the default too low (and initially also gnuradio complaining) buffer settings (net.core.rmem_max, net.core.wmem_max), disabling the Ubuntu network manager (via the menu) and removing the automatic network configuration when USRPN210 connects and instead setting up the network connection manually with "sudo ifconfig eth1 192.168.10.1" ,  I *sometimes* can get the Ethernet connection into a state to work with the N210 at high sampling rates without any interruptions at all !

In that case, a beautiful continuous flow of samples to the crunching computer (like a fft-plot), at highest possible rate 50 MSPS, 8 bit/samples over the wire. This *can* happen with a MTU above 1500 (or more), buffers increased to recommended settings by UHD, and when this works the Ubuntu system manager tells me that some 834 Mbit/samples flows through the Ethernet cable, and about 50% load on the CPU-cores, very nice. Then it also works for repeated runs, not just one "lucky" one, and for a prolonged period of time (more than an hour).  In the working network state I can also easily provoke nice expected overruns, strings of 'ooooooooooo':hs, which isn't possible when the Ethernet connection is in the "wrong" and "interrupted" state - since then it just halts/stops without further info.

However, finding this working network state is not just a matter of setting the particular network parameters as I hoped it to be...  I suspect some other things are happening behind the scenes, maybe some other settings etc (I do not yet have full knowledge of ethernets full functionality and behavior, there may be more influencing parameters ?) which prevent me finding the working network state in a consistent manner. Quite weird.

I have USRP-N210 rev2 (says sticker on the back) but now noticed that when I burned the latest fw and fpga images with the net burner tool it prints "Hardware type: n210_r3" although I selected the fpga version image for rev2 corresponding to my version. Could this be related to my issue? Haven't noticed that inconsistent message earlier, though.

If some of this rings any bells, please give me some further advise.
Sorry for long post.

Thanks,
Rickard

My first guess is that network-manager sucks, and it's cutting your connection. I've seen this on Ubuntu on a semi-regular basis even on my own laptop, when using ifconfig to set the adapter address manually, sometimes network-manager decides to bring the interface down just because. To diagnose this, look to see the network light on the N210 go out after the samples stop coming. The solution is to go into the network configuration and set eth0 (or whatever network adapter it is) to use a static IP address instead of "auto".

--n

 

_______________________________________________
Discuss-gnuradio mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio