Re: [Discuss-gnuradio] How do I know when my flow graph has finished in a non-blocking way?

[Eskil Varenius]

Dear Marcus,
Thank you very much for that advice. After thinking and reading a bit more based on what you wrote, I think I managed to solve my problem! I attach a brief example I wrote to try it out, including only the essential things + simple plotting.

Regarding the examples you mentioned: In my system (Linux Mint 17), to be able to check the examples you mentioned, I had to install the docs by "sudo aptget python-qt4-doc" and they were then accessible at "/usr/share/doc/python-qt4-doc/examples/network/threadedfortuneserver.py". I learned some good things here, but regarding threading I found this page even more useful (which I found once I realised what I was looking for, thanks to you):
http://stackoverflow.com/questions/6783194/background-thread-with-qthread-in-pyqt

I also went ahead and implemented a simple graphics window as well, with help of this page:
http://stackoverflow.com/questions/12459811/how-to-embed-matplotib-in-pyqt-for-dummies

I am relatively new to Qt and threading, so I am sure I make mistakes and have bad coding style. But, since this simple test program works, I thought I might as well share it anyway. Since I myself find much information as text online, I include in this email also below the code I wrote as text (not as attached files), with hopes that it might be useful for someone searching for related things online.

Best regards,
Eskil

------------------------------------------------------------------------------------------------------

SUMMARY of the test program:
Shows a window on screen where the user can click a button to start a GNUradio flowgraph. The flowgraph will run in a separate thread to not disturb the UI. When data has been generated by GNUradio, the user can plot the data on screen. Tested on Linux Mint 17 computer using gnuradio 3.7.2 (as available via apt-get) and relevant Python2/PyQt4 modules.

List of files included:
Main.py -- The main program to be run by the user.
FlowGraph.grc -- Flowgraph made with GNUradio-companion. Used to generate (in GRC) the file top_block.py.
top_block.py -- File generated automatically by GRC. This contains all GNURadio code for the flowgraph. Used in Main.py. When the flowgraph is run, it will generate a file called "threadingdata.dat" with samples from a signal generator.
GUI.ui -- Graphical user interface created with QtDesigner. Used to generate GUI.py.
GUI.py -- Generated from GUI.ui by running "pyuic4 GUI.ui > GUI.py" in terminal. Used in Main.py.

PASTED CODE FOR EACH FILE BELOW:
-------Main.py-------
#!/usr/bin/env python
from matplotlib.backends.backend_qt4agg import FigureCanvasQTAgg as FigureCanvas
from matplotlib.backends.backend_qt4agg import NavigationToolbar2QTAgg as NavigationToolbar
import matplotlib.pyplot as plt
from GUI import Ui_Dialog
from PyQt4 import QtGui, QtCore
import sys
from top_block import *
import numpy as np

# Define object used to run GNUradio in a separate QThread, according to:
# http://stackoverflow.com/questions/6783194/background-thread-with-qthread-in-pyqt
class Worker(QtCore.QObject):
    finished = QtCore.pyqtSignal()

    @QtCore.pyqtSlot()
    def runflowgraph(self):
        self.tb.start()
        self.tb.wait()
        self.finished.emit()

# Implement custom Thread class, according to:
# http://stackoverflow.com/questions/6783194/background-thread-with-qthread-in-pyqt
class Thread(QtCore.QThread):
    def __init__(self, parent=None):
        QtCore.QThread.__init__(self, parent)

    def start(self):
        QtCore.QThread.start(self)

    def run(self):
        QtCore.QThread.run(self)

# Define GUI, using the code generated by QtDesigner.
# Made by running in terminal "pyuic4 GUI.ui > GUI.py" to get GUI.py containing class Ui_Dialog.
class main_Dialog(Ui_Dialog, QtGui.QDialog):
    def __init__(self):
        super(main_Dialog, self).__init__()
        self.setupUi(self)
        self.init_Ui()

    # Set states of interface and connect button signals
    # to relevant slots.
    def init_Ui(self):
        self.testGUIbutton.clicked.connect(self.showGUIisalive)
        self.getdatabutton.clicked.connect(self.runGNURadio)
        self.plotresbutton.clicked.connect(self.plot)

        # Do not allow plotting before data has been taken
        self.plotresbutton.setEnabled(False)

        # ADD MATPLOTLIB CANVAS, based on:
        # http://stackoverflow.com/questions/12459811/how-to-embed-matplotib-in-pyqt-for-dummies
        # a figure instance to plot on
        self.figure = plt.figure()
        # this is the Canvas Widget that displays the `figure`
        # it takes the `figure` instance as a parameter to __init__
        self.canvas = FigureCanvas(self.figure)
        self.canvas.setParent(self)
        # set the layout
        # Position as left, top, width, height
        self.canvas.setGeometry(QtCore.QRect(400, 160, 271, 300))

    # Show that GUI is responsive when this slot is activated by GUI.
    def showGUIisalive(self):
        self.logmsg('GUI alive!')

    # Create worker object and QThread to handle GNURadio execution in the background
    # without blocking the UI. This will run a simple flowgraph with a
    # signal generator, generating a file with floating
    # point numbers that can later be plotted.
    # Based on http://stackoverflow.com/questions/6783194/background-thread-with-qthread-in-pyqt
    def runGNURadio(self):
        self.plotresbutton.setEnabled(False)
        self.getdatabutton.setEnabled(False)
        self.obj = Worker()
        self.obj.tb = top_block()
        self.thread = Thread() # Create thread to run GNURadio in background
        self.obj.moveToThread(self.thread)
        self.obj.finished.connect(self.thread.quit)
        self.thread.finished.connect(self.GNUradiofinished)
        self.thread.started.connect(self.obj.runflowgraph)
        self.logmsg('GNURadio started.')
        self.thread.start() # Will run the GNU radio flow graph
   
    # Things to do when GNURadio is finished, such as enable plotting.
    def GNUradiofinished(self):
        self.logmsg('GNURadio finished.')
        self.getdatabutton.setEnabled(True)
        self.plotresbutton.setEnabled(True)

    # Put message in textwindow
    def logmsg(self, msg):
        self.plainTextEdit.appendPlainText(QtCore.QDateTime.currentDateTime().toString() + ': ' + msg )

    # Plot data generated by GNUradio
    def plot(self):
        self.logmsg('Plotting 100 samples of threadingdata.dat')
        # create an axis
        ax = self.figure.add_subplot(111)
        # discards the old graph
        ax.hold(False)
        # Read data from file saved by GNUradio (floats). Need to use correct
        # datatype to read numbers properly from GNUradio output.
        data = "" mode = 'r', dtype=np.float32)
        # plot data, only first part to make it clear
        ax.plot(data[0:100], '*-')
        # refresh canvas
        self.canvas.draw()

# Run main event loop and show GUI on screen.
def main():
    app = QtGui.QApplication(sys.argv)
    dialog = main_Dialog()
    dialog.show()
    sys.exit(app.exec_())

# If this file is run (e.g. from terminal by "python Main.py", then run code, else act as library.
if __name__ == '__main__':
    main()

-------FlowGraph.grc-------
<?xml version='1.0' encoding='ASCII'?>
<flow_graph>
  <timestamp>Thu Jul  3 09:48:33 2014</timestamp>
  <block>
    <key>options</key>
    <param>
      <key>id</key>
      <value>top_block</value>
    </param>
    <param>
      <key>_enabled</key>
      <value>True</value>
    </param>
    <param>
      <key>title</key>
      <value></value>
    </param>
    <param>
      <key>author</key>
      <value></value>
    </param>
    <param>
      <key>description</key>
      <value></value>
    </param>
    <param>
      <key>window_size</key>
      <value>1280, 1024</value>
    </param>
    <param>
      <key>generate_options</key>
      <value>no_gui</value>
    </param>
    <param>
      <key>category</key>
      <value>Custom</value>
    </param>
    <param>
      <key>run_options</key>
      <value>run</value>
    </param>
    <param>
      <key>run</key>
      <value>True</value>
    </param>
    <param>
      <key>max_nouts</key>
      <value>0</value>
    </param>
    <param>
      <key>realtime_scheduling</key>
      <value></value>
    </param>
    <param>
      <key>_coordinate</key>
      <value>(10, 10)</value>
    </param>
    <param>
      <key>_rotation</key>
      <value>0</value>
    </param>
  </block>
  <block>
    <key>variable</key>
    <param>
      <key>id</key>
      <value>time_seconds</value>
    </param>
    <param>
      <key>_enabled</key>
      <value>True</value>
    </param>
    <param>
      <key>value</key>
      <value>5</value>
    </param>
    <param>
      <key>_coordinate</key>
      <value>(299, 19)</value>
    </param>
    <param>
      <key>_rotation</key>
      <value>0</value>
    </param>
  </block>
  <block>
    <key>variable</key>
    <param>
      <key>id</key>
      <value>samp_rate</value>
    </param>
    <param>
      <key>_enabled</key>
      <value>True</value>
    </param>
    <param>
      <key>value</key>
      <value>32000</value>
    </param>
    <param>
      <key>_coordinate</key>
      <value>(200, 16)</value>
    </param>
    <param>
      <key>_rotation</key>
      <value>0</value>
    </param>
  </block>
  <block>
    <key>analog_sig_source_x</key>
    <param>
      <key>id</key>
      <value>analog_sig_source_x_0</value>
    </param>
    <param>
      <key>_enabled</key>
      <value>True</value>
    </param>
    <param>
      <key>type</key>
      <value>float</value>
    </param>
    <param>
      <key>samp_rate</key>
      <value>samp_rate</value>
    </param>
    <param>
      <key>waveform</key>
      <value>analog.GR_COS_WAVE</value>
    </param>
    <param>
      <key>freq</key>
      <value>1000</value>
    </param>
    <param>
      <key>amp</key>
      <value>1</value>
    </param>
    <param>
      <key>offset</key>
      <value>0</value>
    </param>
    <param>
      <key>affinity</key>
      <value></value>
    </param>
    <param>
      <key>minoutbuf</key>
      <value>0</value>
    </param>
    <param>
      <key>maxoutbuf</key>
      <value>0</value>
    </param>
    <param>
      <key>_coordinate</key>
      <value>(16, 112)</value>
    </param>
    <param>
      <key>_rotation</key>
      <value>0</value>
    </param>
  </block>
  <block>
    <key>blocks_throttle</key>
    <param>
      <key>id</key>
      <value>blocks_throttle_0</value>
    </param>
    <param>
      <key>_enabled</key>
      <value>True</value>
    </param>
    <param>
      <key>type</key>
      <value>float</value>
    </param>
    <param>
      <key>samples_per_second</key>
      <value>samp_rate</value>
    </param>
    <param>
      <key>vlen</key>
      <value>1</value>
    </param>
    <param>
      <key>affinity</key>
      <value></value>
    </param>
    <param>
      <key>minoutbuf</key>
      <value>0</value>
    </param>
    <param>
      <key>maxoutbuf</key>
      <value>0</value>
    </param>
    <param>
      <key>_coordinate</key>
      <value>(211, 104)</value>
    </param>
    <param>
      <key>_rotation</key>
      <value>0</value>
    </param>
  </block>
  <block>
    <key>blocks_head</key>
    <param>
      <key>id</key>
      <value>blocks_head_0</value>
    </param>
    <param>
      <key>_enabled</key>
      <value>True</value>
    </param>
    <param>
      <key>type</key>
      <value>float</value>
    </param>
    <param>
      <key>num_items</key>
      <value>time_seconds*samp_rate</value>
    </param>
    <param>
      <key>vlen</key>
      <value>1</value>
    </param>
    <param>
      <key>affinity</key>
      <value></value>
    </param>
    <param>
      <key>minoutbuf</key>
      <value>0</value>
    </param>
    <param>
      <key>maxoutbuf</key>
      <value>0</value>
    </param>
    <param>
      <key>_coordinate</key>
      <value>(240, 190)</value>
    </param>
    <param>
      <key>_rotation</key>
      <value>0</value>
    </param>
  </block>
  <block>
    <key>blocks_file_sink</key>
    <param>
      <key>id</key>
      <value>blocks_file_sink_0</value>
    </param>
    <param>
      <key>_enabled</key>
      <value>True</value>
    </param>
    <param>
      <key>file</key>
      <value>threadingdata.dat</value>
    </param>
    <param>
      <key>type</key>
      <value>float</value>
    </param>
    <param>
      <key>vlen</key>
      <value>1</value>
    </param>
    <param>
      <key>unbuffered</key>
      <value>False</value>
    </param>
    <param>
      <key>append</key>
      <value>False</value>
    </param>
    <param>
      <key>affinity</key>
      <value></value>
    </param>
    <param>
      <key>_coordinate</key>
      <value>(202, 261)</value>
    </param>
    <param>
      <key>_rotation</key>
      <value>0</value>
    </param>
  </block>
  <connection>
    <source_block_id>blocks_head_0</source_block_id>
    <sink_block_id>blocks_file_sink_0</sink_block_id>
    <source_key>0</source_key>
    <sink_key>0</sink_key>
  </connection>
  <connection>
    <source_block_id>analog_sig_source_x_0</source_block_id>
    <sink_block_id>blocks_throttle_0</sink_block_id>
    <source_key>0</source_key>
    <sink_key>0</sink_key>
  </connection>
  <connection>
    <source_block_id>blocks_throttle_0</source_block_id>
    <sink_block_id>blocks_head_0</sink_block_id>
    <source_key>0</source_key>
    <sink_key>0</sink_key>
  </connection>
</flow_graph>
-------top_block.py-------
#!/usr/bin/env python
##################################################
# Gnuradio Python Flow Graph
# Title: Top Block
# Generated: Thu Jul  3 09:48:35 2014
##################################################

from gnuradio import analog
from gnuradio import blocks
from gnuradio import eng_notation
from gnuradio import gr
from gnuradio.eng_option import eng_option
from gnuradio.filter import firdes
from optparse import OptionParser

class top_block(gr.top_block):

    def __init__(self):
        gr.top_block.__init__(self, "Top Block")

        ##################################################
        # Variables
        ##################################################
        self.time_seconds = time_seconds = 5
        self.samp_rate = samp_rate = 32000

        ##################################################
        # Blocks
        ##################################################
        self.blocks_throttle_0 = blocks.throttle(gr.sizeof_float*1, samp_rate)
        self.blocks_head_0 = blocks.head(gr.sizeof_float*1, time_seconds*samp_rate)
        self.blocks_file_sink_0 = blocks.file_sink(gr.sizeof_float*1, "threadingdata.dat", False)
        self.blocks_file_sink_0.set_unbuffered(False)
        self.analog_sig_source_x_0 = analog.sig_source_f(samp_rate, analog.GR_COS_WAVE, 1000, 1, 0)

        ##################################################
        # Connections
        ##################################################
        self.connect((self.blocks_head_0, 0), (self.blocks_file_sink_0, 0))
        self.connect((self.analog_sig_source_x_0, 0), (self.blocks_throttle_0, 0))
        self.connect((self.blocks_throttle_0, 0), (self.blocks_head_0, 0))


# QT sink close method reimplementation

    def get_time_seconds(self):
        return self.time_seconds

    def set_time_seconds(self, time_seconds):
        self.time_seconds = time_seconds

    def get_samp_rate(self):
        return self.samp_rate

    def set_samp_rate(self, samp_rate):
        self.samp_rate = samp_rate
        self.analog_sig_source_x_0.set_sampling_freq(self.samp_rate)
        self.blocks_throttle_0.set_sample_rate(self.samp_rate)

if __name__ == '__main__':
    parser = OptionParser(option_class=eng_option, usage="%prog: [options#]")
    (options, args) = parser.parse_args()
    tb = top_block()
    tb.start()
    tb.wait()
-------GUI.ui-------
<?xml version="1.0" encoding="UTF-8"?>
<ui version="4.0">
 <class>Dialog</class>
 <widget class="QDialog" name="Dialog">
  <property name="geometry">
   <rect>
    <x>0</x>
    <y>0</y>
    <width>724</width>
    <height>503</height>
   </rect>
  </property>
  <property name="windowTitle">
   <string>Dialog</string>
  </property>
  <widget class="QPushButton" name="getdatabutton">
   <property name="geometry">
    <rect>
     <x>50</x>
     <y>50</y>
     <width>201</width>
     <height>41</height>
    </rect>
   </property>
   <property name="text">
    <string>Run GNU Radio in background</string>
   </property>
  </widget>
  <widget class="QPushButton" name="testGUIbutton">
   <property name="geometry">
    <rect>
     <x>40</x>
     <y>140</y>
     <width>131</width>
     <height>31</height>
    </rect>
   </property>
   <property name="text">
    <string>Test GUI (print)</string>
   </property>
  </widget>
  <widget class="QPushButton" name="plotresbutton">
   <property name="geometry">
    <rect>
     <x>380</x>
     <y>50</y>
     <width>131</width>
     <height>31</height>
    </rect>
   </property>
   <property name="text">
    <string>Plotdata</string>
   </property>
  </widget>
  <widget class="QLabel" name="textlabel">
   <property name="geometry">
    <rect>
     <x>40</x>
     <y>230</y>
     <width>71</width>
     <height>16</height>
    </rect>
   </property>
   <property name="text">
    <string>Textoutput</string>
   </property>
  </widget>
  <widget class="QLabel" name="graphlabel">
   <property name="geometry">
    <rect>
     <x>390</x>
     <y>110</y>
     <width>58</width>
     <height>15</height>
    </rect>
   </property>
   <property name="text">
    <string>Graph</string>
   </property>
  </widget>
  <widget class="QPlainTextEdit" name="plainTextEdit">
   <property name="geometry">
    <rect>
     <x>40</x>
     <y>260</y>
     <width>311</width>
     <height>171</height>
    </rect>
   </property>
  </widget>
 </widget>
 <resources/>
 <connections/>
</ui>
-------GUI.py-------
# -*- coding: utf-8 -*-

# Form implementation generated from reading ui file 'GUI.ui'
#
# Created: Thu Jul  3 11:17:09 2014
#      by: PyQt4 UI code generator 4.10.4
#
# WARNING! All changes made in this file will be lost!

from PyQt4 import QtCore, QtGui

try:
    _fromUtf8 = QtCore.QString.fromUtf8
except AttributeError:
    def _fromUtf8(s):
        return s

try:
    _encoding = QtGui.QApplication.UnicodeUTF8
    def _translate(context, text, disambig):
        return QtGui.QApplication.translate(context, text, disambig, _encoding)
except AttributeError:
    def _translate(context, text, disambig):
        return QtGui.QApplication.translate(context, text, disambig)

class Ui_Dialog(object):
    def setupUi(self, Dialog):
        Dialog.setObjectName(_fromUtf8("Dialog"))
        Dialog.resize(724, 503)
        self.getdatabutton = QtGui.QPushButton(Dialog)
        self.getdatabutton.setGeometry(QtCore.QRect(50, 50, 201, 41))
        self.getdatabutton.setObjectName(_fromUtf8("getdatabutton"))
        self.testGUIbutton = QtGui.QPushButton(Dialog)
        self.testGUIbutton.setGeometry(QtCore.QRect(40, 140, 131, 31))
        self.testGUIbutton.setObjectName(_fromUtf8("testGUIbutton"))
        self.plotresbutton = QtGui.QPushButton(Dialog)
        self.plotresbutton.setGeometry(QtCore.QRect(380, 50, 131, 31))
        self.plotresbutton.setObjectName(_fromUtf8("plotresbutton"))
        self.textlabel = QtGui.QLabel(Dialog)
        self.textlabel.setGeometry(QtCore.QRect(40, 230, 71, 16))
        self.textlabel.setObjectName(_fromUtf8("textlabel"))
        self.graphlabel = QtGui.QLabel(Dialog)
        self.graphlabel.setGeometry(QtCore.QRect(390, 110, 58, 15))
        self.graphlabel.setObjectName(_fromUtf8("graphlabel"))
        self.plainTextEdit = QtGui.QPlainTextEdit(Dialog)
        self.plainTextEdit.setGeometry(QtCore.QRect(40, 260, 311, 171))
        self.plainTextEdit.setObjectName(_fromUtf8("plainTextEdit"))

        self.retranslateUi(Dialog)
        QtCore.QMetaObject.connectSlotsByName(Dialog)

    def retranslateUi(self, Dialog):
        Dialog.setWindowTitle(_translate("Dialog", "Dialog", None))
        self.getdatabutton.setText(_translate("Dialog", "Run GNU Radio in background", None))
        self.testGUIbutton.setText(_translate("Dialog", "Test GUI (print)", None))
        self.plotresbutton.setText(_translate("Dialog", "Plotdata", None))
        self.textlabel.setText(_translate("Dialog", "Textoutput", None))
        self.graphlabel.setText(_translate("Dialog", "Graph", None))

2014-07-02 21:07 GMT+02:00 Marcus Müller <address@hidden>:

Hi Eskil,

QT has its own threading library that you'll need to use to spawn a new thread to do the blocking calls in:
http://pyqt.sourceforge.net/Docs/PyQt4/qthread.html (sadly, that page is still C++ centric)

PyQT4 usually comes with a set of nice examples, which have been useful for me over the last days; for me, those installed to
/usr/share/doc/PyQt4-devel-4.10.1/examples
and for your threading intents, you might want to look at
/usr/share/doc/PyQt4-devel-4.10.1/examples/network/threadedfortuneserver.py

Greetings, and happy hacking,
Marcus

On 02.07.2014 20:27, Eskil Varenius wrote:

Dear friends of GNU Radio,
I have a question about blocking, threads and GUI. Thing is, I am writing
an GUI application (in PyQT4) where the user can start a GNU Radio
flowgraph by a button. The flowgraph records data from an USRP and some
additional blocks. Now, I want only a finite amount of data so I have added
a "head" block to my flowgraph. The obvious way to run this is as the
following pseudo code program:
# Define flowgraph
class USRP(gr.top_block)
...connect USRP -> head_block_> file_sink
# Define GUI
GUI__init__()
    gui.button.clicked.connect(startUSRP) ...
# Define action to be taken when button clicked, i.e. start flowgraph
startUSRP()
  tb=USRP()
  tb.start()
  tb.wait() # Will block everything until finished
#  define and run GUI
main()
    app = QtGui.QApplication(sys.argv)
    window = main_window()
    window.show()
    sys.exit(app.exec_())
if __name__ == '__main__':
    main()

My code runs as expected, in the sense data is produced, but tb.wait is a
blocking call. This means that my GUI will freeze until the flowgraph has
ended. The GUI must not be blocked during flowgraph execution. Also, I
would like to track the progress of the flowgraph (the number of elements
processed / total elements requested). I tried to search online, and there
is some related information, but unfortunately I have not managed to
understand how to put the pieces together yet. I imagine something like
using QTimer to periodically check the status of the flowgraph, by asking
the flowgraph in some way.
This flowgraph ends with writing to a file, after the head_block. Hence, I
suspect that just checking the head_block might not give me all data?
Either I need to add a small extra delay to make sure the file sink gets
all data, or check the number of items passed to the file sink itself? So I
guess the short question is:
How to periodically check if a started flowgraph has finished, without
using a blocking call such as tb.wait?

I would be very grateful for any hints. I don't know much about C++ and
threading, which makes finding clues from the C++ API more difficult.
However, I am eager to learn if anyone would enlighten me on this.

Best regards,
Eskil

_______________________________________________
Discuss-gnuradio mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

_______________________________________________
Discuss-gnuradio mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio