Re: [Discuss-gnuradio] UHD USRP Source for B2x0 overflows File Sink

[Murphy, John]

Date: Sun, 10 May 2015 17:18:58 -0400
From: "Marcus D. Leech" <address@hidden>
You could try a simple experiment that tests your disk subsystem write
performance without SDR stuff at all.  Something like:
time dd if=/dev/zero of=some-file-on-your-disk bs=2000000 count=15000
This should write a 30GB file.   The 'time' command will report how long
it took.  Then it's just a simple matter of math to figure out what your
achievable long-term write rate is.

address@hidden Documents]$ /usr/bin/time dd if=/dev/zero
of=./thrutest bs=2000000 count=15000
15000+0 records in
15000+0 records out
30000000000 bytes (30 GB) copied, 54.9445 s, 546 MB/s
0.00user 15.17system 0:54.95elapsed 27%CPU (0avgtext+0avgdata 3964maxresident)k
96inputs+58593760outputs (0major+564minor)pagefaults 0swaps

That is 30GBytes written in about 55 seconds. That is 545 MBytes per
second, which it actually says in the command response.

So... I still do not see any long term or average throughput problem
here. Although even if it had said 69 MBytes/sec I would still think
that is enough to handle 56 MBytes/sec average rate under the right
circumstances.
So... do I now need to do some sort of buffer setup to handle the data
flow during periods the system is too busy to write to the disk, or
the disk is too busy to be written to?
I already have the UHD USRP Source with num_recv_frames=1024 (have
also tried 512). In this version of UHD (3.8.1) you have to enter this
in the Device Arguments param space without quotes.
I also see the advanced tab of all the blocks that has the minimum and
maximum output buffer sizes.
But... I know nothing about how any of these work, certainly not in a
quantitative sense anyhow.

Or am I still missing some reason why this will never work at all?

- John

John Murphy
address@hidden


On Sat, May 9, 2015 at 9:02 AM, Murphy, John <address@hidden> wrote:
> Marcus et al,
>
> Had to drop this to do some work on another project yesterday, but
> still want to pursue this just a little further if you don't mind,
> because the numbers you are giving all look to me like it should be
> able to be made to work.
>
> You found my SDD sequential sustained write speed of 69 MBytes/sec.
> If I attempt to save data at 14 MSamp/sec to disk with complex 16-bit
> integers I believe there is an average long-term rate of 56 MBytes/sec
> going to the disk.
>
> So I am not understanding - it seems to me like I have plenty of
> sustained throughput overhead to make this work, with the right
> buffering to take up the temporary slack.
>
> With 16 GBytes of RAM (the system is using some, but still) I would
> expect that I can buffer up something like 4 minutes of data at the
> required 56 MBytes/sec rate - seems like with the proper setup there
> should be plenty of capability to ride through whatever other kernel
> operations etc are momentarily stalling the disk writes.
>
> Thanks, I appreciate you taking the time and list bandwidth to help me
> understand this,
> - John
>
> Date: Thu, 07 May 2015 17:35:57 -0400
> From: "Marcus D. Leech" <address@hidden>
>
>     The basic problem is that if the long-term-average offered-load on your
>     write medium (your SSD in this case) is higher than it can sustain, it
>     doesn't
>        matter how much buffering you add in front of it, eventually, the
>     piper has to be paid.   Buffering is useful to meeting short-term
>     short-falls in
>        throughput capacity.  They *cannot* help if offered load, on average,
>     exceeds the long-term capacity of the resource.  Now, having said that,
>        if you only need to record for a short time, consider adding more
>     RAM, and creating a ramdisk, then stage the ramdisk out to your hard disk.
>        But this ONLY WORKS if you don't need to record continuously,
>     otherwise, you're back to the buffering issue....
>     But at 8Msps, and 4-bytes per sample, that's 32Mbyte/second, you have
>     about 30 seconds/gigabyte.
>
>> On Thu, May 7, 2015 at 3:43 PM,  <address@hidden> wrote:
>
>>> I looked at their blurb on that drive, and its *sustained* rate comes out to
>>> about 69Mbyte/second.  Sure, it'll take bursts at screaming-fast rates,
>>> because, like the Linux kernel, it has a whacking great write-behind buffer.