Re: [Discuss-gnuradio] Re: UHD Announcement - February 25rd 2011

[Feng Andrew Ge]

Josh,

As predicted, changing from "RECV_MODE_FULL_BUFF" to "RECV_MODE_ONE_PACKET" and then "RECV_MODE_FULL_BUFF" reduced the latency significantly. My ping RTT time was >17ms in average, now it is 10.5ms. This is GREAT :-)

Nonetheless, 10.5ms still includes quite some overhead because the communication time for my ping message is less than 3.9 ms in total. Above 6ms delay still exists somewhere.

In your description below, "for the source, the work function waits for the entire RX buffer" (now it is one UDP packet of samples first and then the entire RX buffer), do you mean the buffer size determined by "noutput_items"? Likewise, is the entire TX buffer at the sink determined by "ninput_items"?

One UDP packet of samples is only 1472/4= 368 samples and  1472/4/2/8 = 23 B of user data (given BPSK and 2 SPS). For a ping ICMP message ( 42B, which is tiny), plus 19B of frame overhead,  it still must wait for "waits for the entire RX buffer" which may be possibly 10s of thousands of samples.  

Is there some way that we can further optimize the behavior? For example, can we limit the size of noutput_items? Is this purely GNU Radio scheduler's job?

One previous question I had is this: why recv_buff_size  is needed for benchmark_tx.py which involves only gr_uhd_usrp_sink.cc? I thought only send_buff_siz is needed.

Andrew


On 03/01/2011 08:20 PM, Josh Blum wrote:

On 03/01/2011 04:39 PM, Feng Andrew Ge wrote:

Josh,

Your explanation makes sense.  Is there a quick fix for me to bypass
this problem temporarily while you are working on the modification.

http://gnuradio.org/cgit/jblum.git/commit/?id=75538e12300cb0d593792a986841ba2df9997c54

:-)

Further,  following your current bandwidth optimization implementation,
is the code trying to fill the buffer in both uhd_single_usrp_sink
(sending buffer) and uhd_single_usrp_source (receiving buffer)?

For the source, the work function waits for the entire RX buffer to be
filled with samples. I strongly believe this is the cause of the latency.

For the sink, the work function sends the entire buffer. This is correct
behavior for optimum latency and bandwidth.

When I started uhd_benchmark_tx.py, it also asked for specification of
*recv_buff_size, where is it used? *

http://www.ettus.com/uhd_docs/manual/html/transport.html#resize-socket-buffers

-Josh

On 03/01/2011 07:01 PM, Josh Blum wrote:

On 03/01/2011 03:25 PM, Feng Andrew Ge wrote:

Josh,

Everything is in the attachment, look for python-based UHD_* code.

The performance before and after Feb.25 was both poor. At one point
before Feb. 25, the latency was even worse, up to>30ms. But I forgot
the time.

In terms of GNU Radio, for running UHD I checked out next from GNU Radio
git.  However, for raw Ethernet, I only ran GNU Radio 3.3.0.  I haven't
found time to check GNU Radio changes yet, what might cause such huge
performance drop.

Andrew,

Here is an idea that may explain your problem:

When the raw ethernet source calls into work(), it does not attempt to
fill the entire buffer (noutput_items). Rather, it waits at least one
packet to become available and then copies only the data that is
available immediately into the buffer.

In contrast, the UHD work function is bandwidth optimized and tries to
fill the entire buffer. At your sample rate (500ksps), this will impose
serious delays for very large noutput_items (10s of thousands). I hope
that explains the issue you see.

I am going to attempt a modification to the work function, where we recv
a single packet with timeout, and then anything else that is available
without waiting.

I will let you know when I post a branch with changes.
-Josh

Andrew

On 03/01/2011 06:13 PM, Josh Blum wrote:

On 03/01/2011 02:21 PM, Feng Andrew Ge wrote:

Josh,

First of all, I am aware of what you pointed out and I did use the
code
latency_test.cpp for measuring latency between USRP2 and a host.  The
latency is negligible.

Ok, i see. You were measuring the ping time over the tunnel. :-)

Can you tell me: Is this a new problem with UHD since the " February
25rd 2011" announcement. That is, was it working properly for you
previously?

I think I was not clear enough in my previous email.
My setting is this: host_1--USRP2_1   talks to  host_2--USRP2_2.  The
latency I measured is based on GNU Radio-created wireless network
interface, e.g., gr0.  I started tunnel.py and created a digital link
between host_1 and host_2; then I compared ping RTT time performance
between using the UHD code and using the Raw_Ethernet code base.  UHD
introduced 9 ms of overhead and I am really puzzled about this. Since

I am puzzled as well. 9ms sounds pretty bad. Is this a port of
tunnel.py
to UHD, can you share it?

USRP2 sends samples immediately out and the latency between the
host and
USRP2 is negligible, the likely place I can think of is the interface
between UHD and GNU Radio, for example, can UDP packet sending be
preempted by other threads? Each time how many UDP packets can be
possibly sent out?

The work function is called with a randomly sized buffer determined by
the scheduler. The number of packets received or sent depends on the
size of the buffer when the work() function is called.

I think this is exactly the same for the raw_ethernet driver.

It may be helpful to print the number of items in the work function. It
seems to be in the 10s of thousands of samples last I looked.

When you compared UHD vs raw_ethernet driver, it was all the same
version of gnuradio, correct?

Another possibility is how you allocate CPU resources in handling UHD
and what the impact might be.

The third possibility is buffer management: how do you handle buffer
management in UHD for sending and receiving? How do data stay in those
buffers and how are data are processed, by FIFO or LIFO? If overflow
happens, will newly coming packets get simply dropped?

Nothing gets buffered in the UHD in the usrp2/n210 implementation.
However, there is a kernel socket buffer on receive that has enough
buffering for a second of samples. Once this buffer fills, newer
packets
are dropped.

I also believe that this is the same on the raw ethernet driver.

-josh