Mallesham,
you should __really__ look into what kind of data you need to store. Are you sure what you want to do in the end is analyze 24 h * 3600 s/h * 32·10⁶ S/s * 8B/s = 20 Terabyte of data? That is what you get when you store the FFTs of one day worth of samples. I know that you're probably tuning in between, but again, as we all stressed: the FFT does *not* reduce the amount of data! It is just a transform into a different base system.
Maybe you can take a step back and describe your *application*. What is it that you need to *do*? I think your system requirements are not fully developed, yet.
Best regards,
Marcus
On 01/20/2017 09:15 PM, Mallesham Dasari wrote:
Hi All,
Sorry, it was a typo. The sample rate is 32MHz. Thank you very much for the information. I will look for the larger storage or some other mechanism.
Thanks!
On Fri, Jan 20, 2017 at 3:01 PM, Kyeong Su Shin <address@hidden> wrote:
To whom it may concern:
As like other people have explained, amount of information carried by the FFT data equals that of the inputted raw I-Q data. At 32GS/s, that would be 64Gbit per a second even if the ADCs are 1-bit. (I am actually not aware of such fast SDR transceivers.. Is it possible?)
However, do you really need all that data? What are you trying to achieve? What I do (or some FFT-related blocks do) is duty-cycling the data collection. For an example, you can do something like storing only 1 FFT vector per every 1000 generated and still get a decent information about the channel. If you really need to have all that data, then no, I am not aware of any feasible implementation.
Regards,Kyeong Su Shin______________________________
On Fri, Jan 20, 2017 at 11:42 AM, Mallesham Dasari <address@hidden> wrote:
Hi Marcus,
Thanks for the quick response. I am recording the FFT samples continuously. But, I am getting overflow after some time when the file size has become huge. My sample rate is high (32GHz) and hence writing to the file takes so long and hence the usrp_spectrum_sense getting overflow.______________________________
On Fri, Jan 20, 2017 at 2:33 PM, Marcus Müller <address@hidden> wrote:
Hello Mallesham,
I'm afraid not, since I'm afraid that to my current understanding, what you want is mathematically impossible. Either you want much data – and that seems to be the case, since you want to record 24h of raw IQ data – or you can store it in what comparably little RAM modern computers have.
Maybe, however, we haven't fully understood the problem. Can you, mathematically, define what you want to observe and record?
Best regards,
Marcus
On 01/20/2017 08:28 PM, Mallesham Dasari wrote:
Hello everyone,
Can anyone give some solution for this? Even writing to the ramdisk is not enough for running the flow graph for so long. I am facing the same issue.
Thank you!
On Thu, Jan 12, 2017 at 5:41 PM, Hasini Abeywickrama <address@hidden> wrote:
Hi all,
Thank you very much for the informative responses.
My requirement is to run the flowgraph for a long time (ideally 24 hours) and store the FFT data in the memory (ramdisk) to they can be processed later or in chunks, not everything at the same time.
So far, I have increased the size of the ramdisk and it works fine for a few hours. But it still is not the solution I'm looking for.
Regards,Hasini
On Thu, Jan 12, 2017 at 8:30 PM, Marcus Müller <address@hidden> wrote:
______________________________But if you do a single 1024-FFT, you'd only operate on 1024 of the input samples!
And: the FFT doesn't just give you power values, but complex values; mathematically, the FFT is a DFT, and the DFT is an invertible linear operator
:
which maps complex vectors to complex vectors of size
. It is, in fact, representable as square matrix with column (and row) vectors being samples of the orthogonal complex sinusoids $e^{j\frac{2\pi}N nk},\, k=0,\ldots,N-1$; that is, it can also be understood as a base change matrix, that just represents the "input vector" according to a different base, orthogonal base.
In the physical sense: the input vector base was represented by the standard basis $\mathbf e_N$, meaning that each base vector represents a single point in time – the sample time of the respective entry; the "output" of the transform is represented on a base of orthogonal frequencies. This is an invertible operation – really just another way to look at the same signal. I think this is really important to keep in mind:
The Fourier transforms are not magical by any means. What they do is represent the same signal from a different point of view. It can be interpreted as transform between time and frequency domain (or space and impulse, or...). The DFT is still just a boring, old, square, orthogonal, invertible matrix that produces output of the same dimensionality as it takes input.
As you can see, the DFT/FFT itself never reduces the amount of data.
What you might be referring to is some kind PSD estimate done by first |·|² a lot of DFTed vectors and then averaging them. The data reduction here lies in the magnitude square operation and the average, not in the DFT.
The point here is that you're throwing away a whole lot of information, and I'm not convinced that's what Hasini needs!
Best regards,
Marcus
On 12.01.2017 05:54, Mallesham Dasari wrote:
Hi Marcus,
Raw IQ samples take lots of memory because each sample will be around 8Bytes. Suppose, if we 1Msps sample rate, just for 10 minutes of data, we get 10*60*1M*8B = 4.8GB data. On the other hand, if you store just FFT with 1024 bin, we get 4.8GB/1024 power values right (which has very less size)?
Please correct me if I am wrong.
Thanks
On Wed, Jan 11, 2017 at 7:32 AM, Marcus Müller <address@hidden> wrote:
______________________________Hi Mallesham,
I don't understand – the raw IQ samples and their FFT have the same size, and data type.
Maybe you've understood something that I (and Martin) didn't – could you elaborate?
Best regards,
Marcus
On 01/11/2017 12:56 AM, Mallesham Dasari wrote:
Hi Hasini,
If you are trying to print just the FFT, it should not be an issue. If you print raw iq samples, then you will run out of memory. By long, you mean how long? Days?
On Tue, Jan 10, 2017 at 3:16 PM, Martin Braun <address@hidden> wrote:
Hasini,
can you please re-state what you're trying to do? That might help you
getting some answers. It is not quite clear from this email.
Cheers,
Martin> ______________________________
On 01/02/2017 09:16 PM, Hasini Abeywickrama wrote:
> Hi all,
>
> I have a flowgraph that reads a signal and writes its FFT samples to a
> file. I need to run this continuously (for a long time), without running
> out of memory.
>
> I tired deleting the earlier FFT samples from the file but that messes
> up with reading the data. I also tried starting writing to a different
> file after some time so the initial file can be completely deleted. But
> it did not work as well.
>
> What would be the best approach for this? Any thought would be very much
> appreciated.
>
> Regards,
> Hasini
>
>
_________________
> Discuss-gnuradio mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>
_______________________________________________
Discuss-gnuradio mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
--
Best Regards,Mallesham DasariDepartment of Computer ScienceStony Brook UniversityUSA - 11794
_______________________________________________ Discuss-gnuradio mailing list address@hidden https://lists.gnu.org/mailman/ listinfo/discuss-gnuradio _________________ Discuss-gnuradio mailing list address@hidden https://lists.gnu.org/mailman/ listinfo/discuss-gnuradio --Best Regards,Mallesham DasariDepartment of Computer ScienceStony Brook UniversityUSA - 11794_________________ Discuss-gnuradio mailing list address@hidden https://lists.gnu.org/mailman/ listinfo/discuss-gnuradio --Best Regards,Mallesham DasariDepartment of Computer ScienceStony Brook UniversityUSA - 11794--Best Regards,Mallesham DasariDepartment of Computer ScienceStony Brook UniversityUSA - 11794_________________ Discuss-gnuradio mailing list address@hidden https://lists.gnu.org/mailman/ listinfo/discuss-gnuradio _________________ Discuss-gnuradio mailing list address@hidden https://lists.gnu.org/mailman/ listinfo/discuss-gnuradio --Best Regards,Mallesham DasariDepartment of Computer ScienceStony Brook UniversityUSA - 11794
