|
| From: | George Edwards |
| Subject: | Re: Having problems using forecast method |
| Date: | Tue, 6 Jul 2021 18:20:05 -0500 |
Dear George,
please always reply to the mailing list (or have the list in CC), rather
than only to the person you are replying to. This way other people that
will find your original question in the future can benefit from the
discussion as well.
From your example I see that you are using python rather than C++.
Python will automatically convert the ratio between two integers to a
floating point number if that is required, so you can ignore my comment
on casting everything to float.
For your small example with just 12 samples the way GNURadio works might
seem overly convoluted and complicated, but GNURadio can't know whether
this flowgraph will run for just 12 samples, or for hours with millions
and millions of samples. In the latter case it is impossible to process
everything in one go, so GNURadio is processing everything in smaller
chunks. Your code assumes it will process the entire data set within one
call of general_work, rather than processing just the part that GNURadio
currently wants processed. Or at least it assumes that GNURadio will
always feed you input data and output buffer size in the correct ratio.
But that is not correct. GNURadio will provide you as much input data as
it has right now, even if that is more than you asked for.
It will ask a block beforehand, "Hey block, how much samples do you need
from me to give me a certain number of output samples" (this is done in
the forecast method). It might actually call forecast multiple times
with a different number of outputs until it arrives at a situation where
the requested number of input samples are actually available in a buffer
from a previous block (it might also run the previous block a couple of
times before coming back to the current block to make sure there is now
enough input).
Then it calls the general_work function and tells it, here are
len(input_items) inputs and here is a buffer where you can write your
output to. Since this buffer needs to be allocated first you cannot
write an arbitrary amount of data into that buffer. general_work must
produce *at most* as many output samples, as output_items[0] is large.
general_work can chose to produce exactly that amount of outputs, less,
or none at all (even though it told GNURadio something else in forecast).
Then the scheduler will run another time. Usually by first calling
forecast (maybe multiple times) and eventually general_work where you
can produce additional output. This will repeat until you have processed
all the data.
The bug in your code is that you determine the number of loop iterations
only based on the number of input samples, not on the available output
buffer size. To better understand this behavior I suggest you add a
print statement into the forecast method that prints the requested
number of output samples and your computed number of input samples and a
print statement in the general_work method that prints the length of the
input_items[0] and output_items[0] arrays (before you change any of
them). With these prints you should be able to follow my description of
the scheduler and understand what you need in your code to fix the error.
Yours
Martin
On 06.07.21 14:00, George Edwards wrote:
> Hi Martin,
>
> Thank you very much for the detailed response. Like I indicated, I
> experimented and found a work around by leaving out the forecast method
> and in the general_work method, I simply get the value of the number of
> input samples and scale it to get the precise number of outputs and
> reset Gnuradio noutput_items[0] to match and it works. But I would love
> to learn how to also use the forecast method and have the same OOT work.
>
> Here is a simplified version of my OOT code using the forecast method
> and it works sometimes in the QA test which is not good. In this OOT I
> am feeding in a stream of data and output a stream. In this exampleI I
> choose M=4 and N=2 to be small numbers and M/N =2 was an integer (my
> true algorithm has M and N in the hundreds and M/N is not an
> integer, but would be a decimal value). This simple algorithm, reads the
> input in group of 4 values and output the first two values in the group.
> I will list my results when I run the QA test! Here is the Python code:
>
> In the general_work method I have:
> in = input_items[0]
> out = output_items[0]
> jj =0
> for n in range(0, len(in)-4, 4): # read in 4 samples at a time so M=4
> input = in[n:n+4]
> for k in range(0,2)
> out[jj] = reg[k] # write to output to the
> first 2 input samples, N=2
> jj = jj+1
> self.consume(0, len(input_items[0]))
> return len(output_items[0]) ,
>
> In the forecast method for loop, I set input_items_required[i] =
> noutput_items*M/N where M=4 and N =2
>
> I ran the QA test with input vectors [0,1,2,3], then [0,1,2,3,4,5,6,7]
> and the outputs were [0,1] and [0,1,4,5], respectively which are correct.
> When I changed the QA input vector to [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
> 11] (so now the output should be [0,1,4,5,8,9]). The QA test bombs and
> stated: handler caught exception: index 4 is out of bounds for axis 0
> which is size 4. I inserted some print statements inside
> the general_work method to see the values for ninput_items and
> noutput_items and the ninput_items was 12 (the size of the input vector
> in the QA test) and noutput_items was 4. noutput_items was
> 4 explains why the program bombs because out[jj] with jj = 4 means I am
> attempting to place a value in out[4] which does not exist because
> noutput_items is 4. Based on the settings in the forecast method, I was
> expecting that if the scheduler set noutput_items to 4, then it would
> set ninput_items to 4*M/N = 8 and the program would not bomb, it would
> work perfectly. Then in the next scheduler round, the last 4 samples
> would be brought in for processing.
>
> Any ideas on how I can fix the simple program above so I can use the
> forecast method and get it to work?
>
> Thanks again for your help.
> George
>
>
>
> On Tue, Jul 6, 2021 at 1:04 AM Martin Luelf <mail@mluelf.de
> <mailto:mail@mluelf.de>> wrote:
>
> Dear George,
>
> what specifically does not work with your test? Any error messages, or
> is it not producing the result you are expecting. And if so, what is
> your block input, what is the output and what output are you expecting?
>
> Keep in mind that the forecast method tells the scheduler how much
> input
> you *likely* need. This is not binding, meaning the general_work method
> does not have to produce this number of outputs, nor does GNURadio have
> to give you that exact amount of input data. In the general_work method
> you need to tell the scheduler how many samples you used (using the
> consume method) and how many samples you created (by returning that
> number).
>
> Also be aware of the data types in C++. noutput_items is an int. If M
> and N are integer types as well, C++ will round M/N to the nearest
> integer before multiplying it to noutput_items and you have effectively
> created an integer decimating block. To avoid that you want to cast
> M, N
> and noutput_items to a floating point number type (if you know the C++
> specifics you can probably get away with just casting one type and have
> the compiler cast the others, but just to be sure I would cast them all
> to float). With that you will get a floating point number of samples
> that your block needs. Since GNURadio can only deal with an integer
> number of samples, it is up to you and your algorithm to bring that
> to a
> sensible integer number. Most likely you will want to round that number
> up or down and feed this integer number of samples back into
> ninput_items_required[0].
>
> Yours
> Martin
>
>
> On 05.07.21 22:51, George Edwards wrote:
> > Hello,
> >
> > I played with it and got it to work. I left out the forecast
> method and
> > forced my will on the scheduler in terms of the input to output
> > sample relationship.
> >
> > However, I am still open to hearing from anyone who has used the
> general
> > block with the forecast method to solve this problem, so as to
> have the
> > perspective on how to use the forecast method.
> >
> > Thanks!
> >
> > George
> >
> > On Mon, Jul 5, 2021 at 2:44 PM George Edwards
> <gedwards.eng@gmail.com <mailto:gedwards.eng@gmail.com>
> > <mailto:gedwards.eng@gmail.com <mailto:gedwards.eng@gmail.com>>>
> wrote:
> >
> > Good afternoon GNURadio community!
> >
> > I am having a problem using the forecast method in my OOT model.
> >
> > In my model, I have one input port and one output port with
> > streaming data. My signal processing algorithm converts every M
> > input samples into N output samples where the ratio of M to N
> is a
> > floating point number, soI cannot use the sync block (if
> M=N) nor
> > the interpolator or decimator OOT blocks.
> >
> > I am forced to use the general or basic block which has
> theforecast
> > method where one has to inform the Scheduler of the relationship
> > between the input and output samples.
> >
> > In the forecast method, I set up the relationship as:
> > ninput_items_required[i] = noutput_items*M/N
> > which is the precise relationship. However, on running the QA
> test I
> > cannot get the OOT module to work properly.
> >
> > Will appreciate any suggestions!
> >
> > Regards,
> > George
> >
>
| [Prev in Thread] | Current Thread | [Next in Thread] |