Re: [Discuss-gnuradio] Flow Graph Flow Control (was: Re: Code Reuse Question)

[Johnathan Corgan]

On Thu, May 12, 2016 at 2:53 PM, Marcus Müller <address@hidden> wrote:

Hi Johnathan,

blocks can't side effect each other except through really strictly controlled, well-defined ways

... and that was still very clear and dominant to the user under GR 3.4, I agree – then we "softened" that concept by adding stream tags, message passing, and if you look at things like the ofdm_rx.grc, you'll notice that there's extensive use of shared state through using the same equalizer object for payload and header and so on.

​The desire to minimize side effects between blocks is motivated by the principle that block authors shouldn't have to know anything about the other blocks they connect to or that are connected to them, and flowgraph designers shouldn't have to know anything about how the internals of blocks work other than their stated operation.​  This decoupling is crucial to the building block approach to application design.

​There are five ways blocks can side-effect each other (and I've been pushing to remove the fifth for a very long time):

1) Produce data on an output port in work() => causes downstream block scheduler(s) to wake up and evaluate whether to run their own work()

2) Consume data on an input port in work() => causes upstream block scheduler to wake up and evaluate whether to run its own work()

​3) Add/remove stream tags in work() => communicates out-of-band data synchronous to a stream flow downstream

4) ​Post asynchronous messages to an output message port => queues the message at listening blocks, causes their schedulers to wake up and call registered message handler functions

5) Call a function on a block from another block or from a thread outside GNU Radio

The nice thing about 1) through 4) is that they are all mediated by the scheduler, and therefore are fire-and-forget actions that make it hard to cause race conditions, deadlocks, or memory corruption.

With 5) the calling thread is always different than the called block's own execution thread, so without adding synchronization primitives, it is very easy to cause any of the above without a skilled understanding of multithreaded programming.  Also, the calling code has to have some sort of in memory pointer to the called block to call it as a function.

The shared object issue in ofdm_rx.grc (and elsewhere) is a result of this--passing shared objects via function calls (constructors) between different objects in different threads. It's not the greatest way to do things, but a lot of that code was written before there was anything like a message passing mechanism in GNU Radio and before we had much experience with the stream tag mechanism.

In addition, calling functions on blocks requires that the caller and callee be in the same process memory, limiting the flowgraph scope to a single application running in one environment. The alternative is to implement remote procedure calls and that way lies CORBA and madness (but I repeat myself.)

We've been exploring the idea of using asynchronous messages instead of function calls for a while now.  Several blocks can now have their configuration parameters updated via a message port, and we just recently added a QTGUI block that generates asynchronous messages when updated by the user.  Also, QTGUI sinks can now have their display updated via receiving async messages instead of stream ports.

An immediate consequence is that, if a block can only communicate or be communicated with via a messages or stream(s)+tags, those are very easy to extend to distributed flowgraphs in a way that is completely decoupled from the blocks themselves.  Two blocks passing messages or stream tags to each other can be on opposite sides of the Internet and only need to agree themselves on what the messages mean, while the transport can be message independent.  We're starting to get some experience with this in a purely manual way with the ZMQ stream and message blocks.

​All this goes to say that there has been fundamental architectural considerations in GNU Radio design that have driven its evolution, and even more has to happen to evolve GNU Radio into more heterogeneous environments.

So, there's two conflicting opinions I have about that

​[...]​

 

 

* The whole python variables/callback/GRC-generated code thing is great for ad-hoc GUI things, but it has led to numerous cases of people being in confusion on how to make blocks "talk" to each other. It's unclean, and maybe we should look into how we can reduce usage of that in favour of replacing it either by explicit application of e.g. message passing, or implicit RPC (which we'd still have to come up with a consistent method for).

​Yes, as outlined above.​

 

OK. Time for lucid dreaming:

​I need a little time to think through what you have written.  Agree it is ripe for discussion.​

​-Johnathan​