Re: [Discuss-gnuradio] [GSOC17 - Draft proposal] Implement SigMF functionality for GNU Radio

[Ben Hilburn]

Hi Kostis -

Great proposal! I'm really excited to see someone tackling the SigMF item on the Ideas List.

I'll respond in-line to Bastian, and then add some of my own:

- You mention that sources and sinks will handle PDUs and Tagged
Streams. I wonder what would be one PDU. A SigMF "capture segment" or
some other chunk of samples. Also are you sure you want to use Tagged
Streams? Maybe you meant normal streams with tags? With Tagged Streams,
you would have to fit the whole capture segment in the input buffer of
the block. That might not be possible.

- You mention that RapidJSON code is messy. Does it auto-generate code?
That could even be an advantage, since SigMF might well go through some
more iterations. Did you plan to also create wrappers for Python?

- The goal of SigMF seems to be to provide a very minimal spec, and I
think many users will have to rely on custom extension in namespaces.
Maybe you could describe a bit how source/sinks and the GUI could deal
with these extensions. 

- You will provide the means to filter data. Did you only plan to filter
specific capture segments or also arbitrary samples based on
annotations? Since you mentioned modulation, which is, AFAIK, currently
not specified, did you plan to allow filtering on custom properties?

So, thinking about this a bit, if Kostis' implementation enables filtering on fields in the `core` namespace, it shouldn't really be any additional effort to support non-`core` namespaced fields. Since any field would be in a capture / annotation segment, which thus would be mapped to a sample index / number of samples, Kostis' software doesn't have to care *what* that field is in order to "select it" in the tool.

That's how I read the proposal, anyway. Kostis, it might be good to clarify this in the proposal.

 

- Regarding the GUI, it seems as if you plan to develop something from
scratch. Did you consider extending projects like "such samples" from
Tim or Inspectrum?

(Seth just responded while I was writing this message, but my comments are similar.)

I agree with Seth and Bastian, here. Generally, I'm a fan of extending existing functionality rather than creating something from scratch. That said, using one of the existing things will also require development.

In the "such samples" case, it's actually just itself a flowgraph (https://github.com/osh/gr-pyqt/blob/master/apps/such_samples2.grc). In order to integrate SigMF into it (which I don't think is a bad idea), new blocks / code will need to be written. Using "Inspectrum" is actually more problematic, especially since it doesn't use GNU Radio under-the-hood. You would effectively be doing work for a different project on GNU Radio GSoC money, which I don't think makes sense.

It might be worth looking at what blocks / code would be necessary to allow the existing Qt GUI features (e.g., "such samples") to make use of the functionality you will be writing in GNU Radio. Ideally, any work you do there can be used in other flowgraphs (not just "such samples"), but using "such samples" would be a great way to reduce your own burden. If you determine that there is really no way to build on things that already exist, then implementing a new tool (like the one in your proposal, now) that is integrated with GRC (e.g., the filter design tool) might indeed be the best way forward. It would be good to explain the justification behind the final decision in the proposal.

(It's worth noting that I would really like to see SigMF support in Inspectrum, and I don't think SigMF support there means it wouldn't be useful here, as well. Inspectrum does more than just visualize data, and one of the key goals of SigMF is portability between tools & workflows.)

 

- I wonder what would be the workflow when creating a SigMF recording.
An issue might be that you cannot immediately write the SigMF meta file
during recording. (For example, the hash has to be calculated after the
recording. Or the annotations have to be written after the capture
segments.). Did you plan to create the meta data in memory and write the
file when the flow graph shuts down (which might cause data loss when
you experience a segfault) or do you write meta data to disk in an
intermediate format that you have to post-process manually to create the
final SigMF files.

So the SigMF spec is designed specifically to enable writing the metadata and dataset files in a streaming manner. The spec is careful not to dictate how applications create SigMF recordings, so it's not a problem if you write a bunch of stuff and then after you finish streaming to the files you open them up and add some more stuff to finish them off.

Bastian's point, though, is a good one. In the use-case where you are streaming metadata and samples into files and then you close your application, do you plan on needing to go back to add more information to the metadata file? If so, how do you do that in an automatic way if the flowgraph has stopped?

- We are working hard to get the SigMF spec into a stable state as soon as possible, but as you can see from the issue tracker (which is where we are hosting our discussions) there are aspects of the spec that are still being debated. Many of these would impact your implementation. I think it would be good to explain your plan to (a) participate in the SigMF spec discussions, especially with the insight you gain from implementing support for it and (b) deal with pieces of the spec that may be a moving target.

- Since this will create a dependency on RapidJSON (or whatever you use), we'll want to be sure that *not* having that installed doesn't prevent building anything not related to SigMF within GNU Radio. I really dislike adding dependencies, but since we don't already have a JSON library there isn't much of an option, here, so we just need to be sure to minimize the burden to users, developers, and packagers.

- "to-SigMF Converters": An important tool will be something that converts a file written with the current GNU Radio File Sink into a SigMF recording. I would limit the scope of the converter work to this.

Cheers,

Ben

On 03/23/2017 07:37 PM, Kostis Triantafyllakis wrote:
> Hello,
>
> My name is Kostis Triantafyllakis, graduate student of Computer Science
> at the University of Crete.
> I'm interested in the project of SigMF functionality implementation for
> GNU Radio.
>
> I have started working on my proposal and I kindly ask for your feedback.
> Please find the draft below:
>
> https://github.com/ctriant/GSoC17-Proposal/blob/master/README.md
> https://github.com/ctriant/GSoC17-Proposal/blob/master/gsoc17_proposal.pdf
>
> Thank you in advance,
>
> Kostis
>
>
> _______________________________________________
> Discuss-gnuradio mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

_______________________________________________
Discuss-gnuradio mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio