Re: [Discuss-gnuradio] [GSOC] A HTML-based GUI for GNU Radio: Draft of proposal

[Kartik Patel]

Hello all, 

Thank you very much for the discussions and feedbacks. I will try to answer all of them one-by-one. 

Mail from Marcus:

• said server has a REST API, i.e. is polled from the client only? Or can there be pushing data from server -> client? What is the model you'd prefer? Your proposal says "stream to the clients", but I can't find that in Bokeh (but then again: no experience on interactive Web dev on my side at all, so this might be really easy for you, if possible).

Although server has a REST API, the bokeh use sockets to transfer the data. Websockets are like pipelines of data which can be used by both server and client. This is not based on typical request-response model of the web. "Stream to the client" means server will sent data to the client without client asking it. The data transfer can be continuous until socket is not broken (from either side). For an example: Video streaming asks for the video once, and then a simple socket will stream the data in frames (without client asking for more data at every frame).

• I've no idea how to talk GR data -> bokeh-server. Maybe you could elaborate on how you do that?

Consider the #L62 in time_sink_f.py in my prototype [1]. The function stream of the DataSource object will stream data to respective socket. This data source is initialised at #L50 in same file and connected to the plot in the line #L51. So, every time after processing the data through GR, it can be streamed through the function to the server. This streaming can be handled by Bokeh library.

• the JSON is relatively verbose, and contains basically all imaginable settings for a plot. However, I presume a data update from the bokeh-server would only consist of the model data, which, for a 5-point line graph would consist of (this is pretty-printed, Bokeh omits all the whitespace/line breaks):

Correct.

• Hence, if performance of the python server is doubted, or integration into GR is complicated, you could also just write a C++ block that takes numerical values from GNU Radio and serves them via Websocket (how does one do that?), and point BokehJS at that instead? (example from [0], found the embed_items declaration in [1])
  

Although we can do that, but I don't think it is necessary. We are not doing anything in Python except the streaming. All processing is done by C++. But I have never used Python where its speed is the constraint. So, may be you can suggest me more on this. I don't think that the Socket will limit the dataflow. Also, since for now we are limiting the use-case to the local network, the speed of network will not be the problem.

• However, when running and changing the data source, I see all the plot JSON being transferred, the ColumnDataSource object just being about half of the 12 kB response object (amongst dozens of requests made to react to this change of model, but that might be an effect of this being a feature-rich plot).
  

Although I did not understand the point you are making, but considering our use-case, we don't need to change the data source.

• Can you please explain the user work flow? I would have imagined that the users design their "plot layout" either
• implicitly (by just using your sinks in their flow graph), so that gr-bokehgui builds a HTML page containing the plots they want on its own, or
• explicitly by designing an HTML page with <div> placeholders on their own and tell gr-bokehgui "hey, use that; the plot <div> IDs are [list of IDs]",
• but I'm not sure you'd support the second method, since you mention the ability to add labels and such, which would be relatively redundant with that.

The user will add a sink from gr-bokehgui. On the user side, everything is exactly same as using the current QT GUI sinks. About layout and all, I agree on the first point. Again, it will be similar to what gr-qtgui is doing as of now. It will automatically adjust the layout of the plots. We can have some CSS or _javascript_ tricks on the frontend to adjust the plots layout and all. But that can be added later.

Now the things I'm not sure about:

• Bokeh is Python library that produces plots
• It's plotting frontend is HTML & _javascript_
• This enables interactive plotting (ie. you can zoom in, pan etc without updating the dataset visualized)
• The frontend _javascript_ is a library (BokehJS) used in a "standard" piece of code that you feed two things:
• The ID of the HTML <div> tag to be filled with the graph
• The data and plot settings to be visualized as JSON
• Bokeh (the python library) comes with a Python-implemented server that you're planning to use

All of these points are correct.

I hope these queries are probably solved. Let me know if you have more queries. I will also address these in my proposal because I think it is the fundamental concept of the module.

The mail from Sebastian:

While bokeh (and plotly) were mentioned in the proposal only because the idea was inspired from these libraries, they maybe not the be a natural fit for us after all - at least on the server side. Maybe, at this point, it would be useful to specify what we actually want out this and then look at what Bokeh and others provide us to get there. Even if that means, that this module will not have all features after 3 months of coding that we want long term.

Point noted. So, we may slightly compromise with number of blocks implemented at the cost of having a strong core of the module.

My rough idea of this is
- a server that runs an event-loop to push push data to clients and receives/handles events from them

Correct. There will not be request from the client. Server will push data to clients. Websockets can do this.

Also, we don't need to use separate event-loop. The function work() in the blocks will do that.

- a single websocket connection per client which the data (for all plots) is multiplexed using messages

A single websocket connection per server. All the clients will be connected to same socket. That will ensure the simultaneous changes in all clients.

- each sink block holds a number plot data buffers ("O"s may happen) that the event loop async pulls from and sends to clients

I don't think that we will need a separate data buffer in Python. At every work() call, the updated data will be streamed through the websocket. (just like at every work() call, the QT windows is updated in gr-qtgui).

 

- metadata is a separate message?

Some metadata can be sent through socket (if those are initiated from server) and others can be sent as a response to the request from clients.

As far as I can tell this is pretty much what Bokeh does, except that we want a more direct path from our blocks to the websocket stream.  I'm not sure if the Bokeh server has an lower-level API that we could we. 

Bokeh does more than this. It maintains a common Document instance which ensures simultaneous changes in all clients. If I change something on the client side, it will be reflected on the others. The manual websocket stream may need to be configured to do that. On the other hand, if I change something on the server side, one common socket for all clients will ensure that every server stream will go to all clients.

The other thing I am unsure about is the flow of data, that Marcus mentioned. Ideally we could have something with the event loop in Python that allowed us to stream data stored in buffer object (build in at the C level). 

As I have mentioned in Marcus's question above:

"Consider the #L62 in time_sink_f.py in my prototype [1]. The function stream of the DataSource object will stream data to respective socket. This data source is initialised at #L50 in same file and connected to the plot in the line #L51. So, every time after processing the data through GR, it can be streamed through the function to the server. This streaming can be handled by Bokeh library."

In addition to that, since we are only focusing on streaming the data from Python, and in general bottleneck of streaming will be network speed, I am not sure using C level buffer will be of much help.

Kartik, I think you should include a more detailed description on how you plan to control the data flow and maybe, expand on the planned options of the placement of widgets.

Thanks for the suggestion. I will update the proposal with the working flow and data flow, both from developer and user side, and add these layout details.

I hope this carries the discussion forward and answers the queries till now. This is the first time I have had this serious discussion on any project before starting the project. It is getting exciting.

[1] https://github.com/kartikp1995/gr-htmlgui/blob/master/python/time_sink_f.py#L62