discuss-gnuradio
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Having problems using forecast method


From: Martin Luelf
Subject: Re: Having problems using forecast method
Date: Tue, 6 Jul 2021 17:12:33 +0200
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0

Dear George,

please always reply to the mailing list (or have the list in CC), rather than only to the person you are replying to. This way other people that will find your original question in the future can benefit from the discussion as well.

From your example I see that you are using python rather than C++. Python will automatically convert the ratio between two integers to a floating point number if that is required, so you can ignore my comment on casting everything to float.

For your small example with just 12 samples the way GNURadio works might seem overly convoluted and complicated, but GNURadio can't know whether this flowgraph will run for just 12 samples, or for hours with millions and millions of samples. In the latter case it is impossible to process everything in one go, so GNURadio is processing everything in smaller chunks. Your code assumes it will process the entire data set within one call of general_work, rather than processing just the part that GNURadio currently wants processed. Or at least it assumes that GNURadio will always feed you input data and output buffer size in the correct ratio. But that is not correct. GNURadio will provide you as much input data as it has right now, even if that is more than you asked for.

It will ask a block beforehand, "Hey block, how much samples do you need from me to give me a certain number of output samples" (this is done in the forecast method). It might actually call forecast multiple times with a different number of outputs until it arrives at a situation where the requested number of input samples are actually available in a buffer from a previous block (it might also run the previous block a couple of times before coming back to the current block to make sure there is now enough input).

Then it calls the general_work function and tells it, here are len(input_items) inputs and here is a buffer where you can write your output to. Since this buffer needs to be allocated first you cannot write an arbitrary amount of data into that buffer. general_work must produce *at most* as many output samples, as output_items[0] is large. general_work can chose to produce exactly that amount of outputs, less, or none at all (even though it told GNURadio something else in forecast).

Then the scheduler will run another time. Usually by first calling forecast (maybe multiple times) and eventually general_work where you can produce additional output. This will repeat until you have processed all the data.

The bug in your code is that you determine the number of loop iterations only based on the number of input samples, not on the available output buffer size. To better understand this behavior I suggest you add a print statement into the forecast method that prints the requested number of output samples and your computed number of input samples and a print statement in the general_work method that prints the length of the input_items[0] and output_items[0] arrays (before you change any of them). With these prints you should be able to follow my description of the scheduler and understand what you need in your code to fix the error.

Yours
Martin


On 06.07.21 14:00, George Edwards wrote:
Hi Martin,

Thank you very much for the detailed response. Like I indicated, I experimented and found a work around by leaving out the forecast method and in the general_work method, I simply get the value of the number of input samples and scale it to get the precise number of outputs and reset Gnuradio noutput_items[0] to match and it works. But I would love to learn how to also use the forecast method and have the same OOT work.

Here is a simplified version of my OOT code using the forecast method and it works sometimes in the QA test which is not good. In this OOT I am feeding in a stream of data and output a stream. In this exampleI I choose M=4 and N=2 to be small numbers and M/N =2 was an integer (my true algorithm has M and N in the hundreds and M/N is not an integer, but would be a decimal value). This simple algorithm, reads the input in group of 4 values and output the first two values in the group. I will list my results when I run the QA test!  Here is the Python code:

In the general_work method I have:
in = input_items[0]
out = output_items[0]
jj =0
for n in range(0, len(in)-4, 4):   # read in 4 samples at a time so M=4
      input = in[n:n+4]
      for k in range(0,2)
          out[jj] = reg[k]                # write to output to the first 2 input samples, N=2
           jj = jj+1
self.consume(0, len(input_items[0]))
return len(output_items[0]) ,

In the forecast method for loop, I set input_items_required[i] = noutput_items*M/N   where M=4 and N =2

I ran the QA test with input vectors [0,1,2,3], then [0,1,2,3,4,5,6,7] and the outputs were [0,1] and [0,1,4,5], respectively which are correct. When I changed the QA input vector to [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] (so now the output should be [0,1,4,5,8,9]). The QA test bombs and stated: handler caught exception: index 4 is out of bounds for axis 0 which is size 4. I inserted some print statements inside the general_work method to see the values for ninput_items and noutput_items and the ninput_items was 12 (the size of the input vector in the QA test) and noutput_items was 4. noutput_items was 4 explains why the program bombs because out[jj] with jj = 4 means I am attempting to place a value in out[4] which does not exist because noutput_items is 4. Based on the settings in the forecast method, I was expecting that if the scheduler set noutput_items to 4, then it would set ninput_items to 4*M/N = 8 and the program would not bomb, it would work perfectly. Then in the next scheduler round, the last 4 samples would be brought in for processing.

Any ideas on how I can fix the simple program above so I can use the forecast method and get it to work?

Thanks again for your help.
George



On Tue, Jul 6, 2021 at 1:04 AM Martin Luelf <mail@mluelf.de <mailto:mail@mluelf.de>> wrote:

    Dear George,

    what specifically does not work with your test? Any error messages, or
    is it not producing the result you are expecting. And if so, what is
    your block input, what is the output and what output are you expecting?

    Keep in mind that the forecast method tells the scheduler how much
    input
    you *likely* need. This is not binding, meaning the general_work method
    does not have to produce this number of outputs, nor does GNURadio have
    to give you that exact amount of input data. In the general_work method
    you need to tell the scheduler how many samples you used (using the
    consume method) and how many samples you created (by returning that
    number).

    Also be aware of the data types in C++. noutput_items is an int. If M
    and N are integer types as well, C++ will round M/N to the nearest
    integer before multiplying it to noutput_items and you have effectively
    created an integer decimating block. To avoid that you want to cast
    M, N
    and noutput_items to a floating point number type (if you know the C++
    specifics you can probably get away with just casting one type and have
    the compiler cast the others, but just to be sure I would cast them all
    to float). With that you will get a floating point number of samples
    that your block needs. Since GNURadio can only deal with an integer
    number of samples, it is up to you and your algorithm to bring that
    to a
    sensible integer number. Most likely you will want to round that number
    up or down and feed this integer number of samples back into
    ninput_items_required[0].

    Yours
    Martin


    On 05.07.21 22:51, George Edwards wrote:
     > Hello,
     >
     > I played with it and got it to work. I left out the forecast
    method and
     > forced my will on the scheduler in terms of the input to output
     > sample relationship.
     >
     > However, I am still open to hearing from anyone who has used the
    general
     > block with the forecast method to solve this problem, so as to
    have the
     > perspective on how to use the forecast method.
     >
     > Thanks!
     >
     > George
     >
     > On Mon, Jul 5, 2021 at 2:44 PM George Edwards
    <gedwards.eng@gmail.com <mailto:gedwards.eng@gmail.com>
     > <mailto:gedwards.eng@gmail.com <mailto:gedwards.eng@gmail.com>>>
    wrote:
     >
     >     Good afternoon GNURadio community!
     >
     >     I am having a problem using the forecast method in my OOT model.
     >
     >     In my model, I have one input port and one output port with
     >     streaming data. My signal processing algorithm converts every M
     >     input samples into N output samples where the ratio of M to N
    is a
     >     floating point number, soI cannot use the sync  block (if
    M=N) nor
     >     the interpolator or decimator OOT blocks.
     >
     >     I am forced to use the general or basic block which has
    theforecast
     >     method where one has to inform the Scheduler of the relationship
     >     between the input and output samples.
     >
     >     In the forecast method, I set up the relationship as:
     >     ninput_items_required[i] = noutput_items*M/N
     >     which is the precise relationship. However, on running the QA
    test I
     >     cannot get the OOT module to work properly.
     >
     >     Will appreciate any suggestions!
     >
     >     Regards,
     >     George
     >




reply via email to

[Prev in Thread] Current Thread [Next in Thread]