[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Having problems using forecast method
Re: Having problems using forecast method
Tue, 6 Jul 2021 17:12:33 +0200
Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0
please always reply to the mailing list (or have the list in CC), rather
than only to the person you are replying to. This way other people that
will find your original question in the future can benefit from the
discussion as well.
From your example I see that you are using python rather than C++.
Python will automatically convert the ratio between two integers to a
floating point number if that is required, so you can ignore my comment
on casting everything to float.
For your small example with just 12 samples the way GNURadio works might
seem overly convoluted and complicated, but GNURadio can't know whether
this flowgraph will run for just 12 samples, or for hours with millions
and millions of samples. In the latter case it is impossible to process
everything in one go, so GNURadio is processing everything in smaller
chunks. Your code assumes it will process the entire data set within one
call of general_work, rather than processing just the part that GNURadio
currently wants processed. Or at least it assumes that GNURadio will
always feed you input data and output buffer size in the correct ratio.
But that is not correct. GNURadio will provide you as much input data as
it has right now, even if that is more than you asked for.
It will ask a block beforehand, "Hey block, how much samples do you need
from me to give me a certain number of output samples" (this is done in
the forecast method). It might actually call forecast multiple times
with a different number of outputs until it arrives at a situation where
the requested number of input samples are actually available in a buffer
from a previous block (it might also run the previous block a couple of
times before coming back to the current block to make sure there is now
Then it calls the general_work function and tells it, here are
len(input_items) inputs and here is a buffer where you can write your
output to. Since this buffer needs to be allocated first you cannot
write an arbitrary amount of data into that buffer. general_work must
produce *at most* as many output samples, as output_items is large.
general_work can chose to produce exactly that amount of outputs, less,
or none at all (even though it told GNURadio something else in forecast).
Then the scheduler will run another time. Usually by first calling
forecast (maybe multiple times) and eventually general_work where you
can produce additional output. This will repeat until you have processed
all the data.
The bug in your code is that you determine the number of loop iterations
only based on the number of input samples, not on the available output
buffer size. To better understand this behavior I suggest you add a
print statement into the forecast method that prints the requested
number of output samples and your computed number of input samples and a
print statement in the general_work method that prints the length of the
input_items and output_items arrays (before you change any of
them). With these prints you should be able to follow my description of
the scheduler and understand what you need in your code to fix the error.
On 06.07.21 14:00, George Edwards wrote:
Thank you very much for the detailed response. Like I indicated, I
experimented and found a work around by leaving out the forecast method
and in the general_work method, I simply get the value of the number of
input samples and scale it to get the precise number of outputs and
reset Gnuradio noutput_items to match and it works. But I would love
to learn how to also use the forecast method and have the same OOT work.
Here is a simplified version of my OOT code using the forecast method
and it works sometimes in the QA test which is not good. In this OOT I
am feeding in a stream of data and output a stream. In this exampleI I
choose M=4 and N=2 to be small numbers and M/N =2 was an integer (my
true algorithm has M and N in the hundreds and M/N is not an
integer, but would be a decimal value). This simple algorithm, reads the
input in group of 4 values and output the first two values in the group.
I will list my results when I run the QA test! Here is the Python code:
In the general_work method I have:
in = input_items
out = output_items
for n in range(0, len(in)-4, 4): # read in 4 samples at a time so M=4
input = in[n:n+4]
for k in range(0,2)
out[jj] = reg[k] # write to output to the
first 2 input samples, N=2
jj = jj+1
return len(output_items) ,
In the forecast method for loop, I set input_items_required[i] =
noutput_items*M/N where M=4 and N =2
I ran the QA test with input vectors [0,1,2,3], then [0,1,2,3,4,5,6,7]
and the outputs were [0,1] and [0,1,4,5], respectively which are correct.
When I changed the QA input vector to [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
11] (so now the output should be [0,1,4,5,8,9]). The QA test bombs and
stated: handler caught exception: index 4 is out of bounds for axis 0
which is size 4. I inserted some print statements inside
the general_work method to see the values for ninput_items and
noutput_items and the ninput_items was 12 (the size of the input vector
in the QA test) and noutput_items was 4. noutput_items was
4 explains why the program bombs because out[jj] with jj = 4 means I am
attempting to place a value in out which does not exist because
noutput_items is 4. Based on the settings in the forecast method, I was
expecting that if the scheduler set noutput_items to 4, then it would
set ninput_items to 4*M/N = 8 and the program would not bomb, it would
work perfectly. Then in the next scheduler round, the last 4 samples
would be brought in for processing.
Any ideas on how I can fix the simple program above so I can use the
forecast method and get it to work?
Thanks again for your help.
On Tue, Jul 6, 2021 at 1:04 AM Martin Luelf <firstname.lastname@example.org
what specifically does not work with your test? Any error messages, or
is it not producing the result you are expecting. And if so, what is
your block input, what is the output and what output are you expecting?
Keep in mind that the forecast method tells the scheduler how much
you *likely* need. This is not binding, meaning the general_work method
does not have to produce this number of outputs, nor does GNURadio have
to give you that exact amount of input data. In the general_work method
you need to tell the scheduler how many samples you used (using the
consume method) and how many samples you created (by returning that
Also be aware of the data types in C++. noutput_items is an int. If M
and N are integer types as well, C++ will round M/N to the nearest
integer before multiplying it to noutput_items and you have effectively
created an integer decimating block. To avoid that you want to cast
and noutput_items to a floating point number type (if you know the C++
specifics you can probably get away with just casting one type and have
the compiler cast the others, but just to be sure I would cast them all
to float). With that you will get a floating point number of samples
that your block needs. Since GNURadio can only deal with an integer
number of samples, it is up to you and your algorithm to bring that
sensible integer number. Most likely you will want to round that number
up or down and feed this integer number of samples back into
On 05.07.21 22:51, George Edwards wrote:
> I played with it and got it to work. I left out the forecast
> forced my will on the scheduler in terms of the input to output
> sample relationship.
> However, I am still open to hearing from anyone who has used the
> block with the forecast method to solve this problem, so as to
> perspective on how to use the forecast method.
> On Mon, Jul 5, 2021 at 2:44 PM George Edwards
> <mailto:email@example.com <mailto:firstname.lastname@example.org>>>
> Good afternoon GNURadio community!
> I am having a problem using the forecast method in my OOT model.
> In my model, I have one input port and one output port with
> streaming data. My signal processing algorithm converts every M
> input samples into N output samples where the ratio of M to N
> floating point number, soI cannot use the sync block (if
> the interpolator or decimator OOT blocks.
> I am forced to use the general or basic block which has
> method where one has to inform the Scheduler of the relationship
> between the input and output samples.
> In the forecast method, I set up the relationship as:
> ninput_items_required[i] = noutput_items*M/N
> which is the precise relationship. However, on running the QA
> cannot get the OOT module to work properly.
> Will appreciate any suggestions!