Re: [Discuss-gnuradio] custom tagged_stream_block basetype and swig

[Miklos Maroti]

Hi Marcus,

On Mon, Jul 31, 2017 at 1:38 AM, Marcus Müller <address@hidden> wrote:
> Hi Miklos,
>
> what surprises me a bit is that you're introducing a new block
> scheduling semantic in an OOT – wouldn't you have to modify the
> scheduler to make it do things differently when encountering a
> tagged_stream_block2 ?
>
> Also, maybe we'd want to have a short chat about what your application
> needs and why a modification of the tagged_stream_block behaviour is
> what you want in that case. I must admit, I'm not the biggest fan of the
> technology behind TSBs myself, and if I'd be reimplementing stuff right
> now, I wouldn't do it TSB-alike. Now, in fact, I'm considering how to
> improve (read: in many places, rewrite) the current GNU Radio scheduler,
> so I'd be very interested in the feedback you could offer about what GR
> is lacking for your current use case.

I have kept the tagged stream block semantics, so no need to modify
the scheduler. The change is focused on adding a standard dictionary
to the packet that is carried along the data automatically just like
packet_len is. Processing blocks can query this dictionary (e.g. what
modulation is used, what was the SNR, etc) or add to it. These
dictionaries are automatically merged from the inputs and the same
dictionary is used for all outputs. Here is the header:

class MYMODULE_API tagged_stream_block2 : public gr::block {
protected:
  const pmt::pmt_t block_name_sym;
  const int max_output_length;

  std::vector<int> input_lengths;
  std::vector<pmt::pmt_t> input_dicts;
  pmt::pmt_t output_dict;

protected:
  // allows pure virtual interface sub-classes
  tagged_stream_block2(void) : max_output_length(0) {}
  tagged_stream_block2(const std::string &name,
                       gr::io_signature::sptr input_signature,
                       gr::io_signature::sptr output_signature,
                       int max_output_length);

  // reads input dictionaries
  bool has_input_long(int input, const pmt::pmt_t &key);
  long get_input_long(int input, const pmt::pmt_t &key, long val);

  // writes output dictionary
  void set_output_long(const pmt::pmt_t &key, long val);
  void set_output_float(const pmt::pmt_t &key, float val);
  void set_output_complex_vector(const pmt::pmt_t &key,
                                 const std::vector<std::complex<float>> &val);

public:
  void forecast(int noutput_items, gr_vector_int &ninput_items_required);

  int general_work(int noutput_items, gr_vector_int &ninput_items,
                   gr_vector_const_void_star &input_items,
                   gr_vector_void_star &output_items);

  // you have to implement this
  virtual int work(int noutput_items, gr_vector_int &ninput_items,
                   gr_vector_const_void_star &input_items,
                   gr_vector_void_star &output_items) = 0;
};

I also changed somewhat how the output buffer length is calculated.
The problem with calculate_output_stream_length is that it might work
in a test scenario, but fail in production: when I design the flow
graph I need to think about how much data this block might produce and
make sure that there is enough buffer space available for that. WIth
the original tagged_stream_block you might return your output stream
length which is larger than allocated buffer, then you are stuck,
because the scheduler cannot create more space than it has originally
allocated. So I decided to set_min_noutput_items and
set_min_output_buffer right from the constructor and always verify
that you do not produce more than you have actually promised. I have
kept the tricks in forecast, that works fine, although I completely
rewrote the logic.

I have struggled with packet based processing before, trying out
various methods, for example using output_multiple (but that fails if
your packet length is e.g. 1023 because the scheduler wants to
allocate a multiple of that which is huge). Messages do not work
because they do not provide back pressure, so now I am experimenting
with tagged stream blocks. What changes do you have in mind for the
scheduler?

Best,
Miklos