Re: [Discuss-gnuradio] [Announcement] Ctrlport is broken -> potential candidate for removal in 3.8 -> RPC discussion

[CEL]

Hi Johannes!

let me answer in-text real quick:
On Mon, 2019-03-25 at 10:13 +0000, Johannes Demel wrote:
> Hi Marcus,
> 
> it seems like Thrift is a painful dependency. So, this is a pro
> ctrlport 
> removal. Though, what else does ctrlport removal entail?

Removal of Ctrlport and the tools dependent on it.

> Is it really only used for monitoring and control?

I'm only speaking for in-tree code, but yes.
Speaking for the whole ecosystem: I'm by now aware of 1 (one) external
project that uses CtrlPort. We're in contact.

> Just for clarification, the zeromq blocks are not affected?

Not affected.

> How will the performance monitor be preserved?

At this point, I have no clue. Suggestions welcome!

>  I really like to look at 
> the statistics to get a first idea where to look for problems and 
> analyze my system with it.

That's a fair point. But to be honest, you don't really need an RPC
framework running in a separate process to query perfcounters, do you.
A very valid model would be having a tiny perfcounter-server (e.g.
behind a req/rep zeromq socket). It wouldn't be a general RPC
framework, but it could do ctrlport's job for this very limited scope!

> 
> Also, in case ctrlport is removed, we should have a GREP to discuss
> what 
> we want and how we want ctrlport back in some other form.

YESSSS! Want to (co)author that?

Best regards,
Marcus

> 
> Cheers
> Johannes
> 
> Am 24.03.19 um 20:26 schrieb Marcus Müller:
> > Dear GNU Radio community,
> > 
> > we've got an outstanding breakage in our controlport
> > infrastructure.
> > Since fixing it seems to involve architectural changes to what
> > controlport does, and since making these changes would quite
> > possibly
> > break the semantics of using controlport anyway, we're considering
> > removing Controlport from the GNU Radio 3.8 release (this does NOT
> > apply to 3.7).
> > 
> > The negative effect of that would obviously be that we'd lose
> > controlport (which I'm not going to introduce here; if you don't
> > know
> > what it is, you haven't been using it). We would *not* lose the
> > performance counters, just the default way of querying them.
> > 
> > The most important upside would be the ability to remove the Apache
> > thrift dependency. Thrift has been the single worst thing we've
> > relied
> > on for as long as I can remember in terms of availability and
> > consistency across platforms. In fact, different distros build
> > completely different configurations of thrift, and noone seems to
> > be
> > able to build a "fully featured" thrift.
> > 
> > Another aspect to this is that albeit controlport is pretty cool in
> > theory – an adaptable RPC server within GNU Radio, with pluggable
> > RPC
> > backends – its adoption has been slow to say the least, and it
> > doesn't
> > really tie neatly into any GNU Radio applications I'd be aware of.
> > 
> > So, lest someone fixes the bug (I'll be describing it below), I'd
> > recommend we remove Controlport alltogether, and remove the thrift
> > dependency. I still stand by the very idea of controlport – having
> > RPC
> > access to what a flow graph does – but in the end, there's a
> > discussion
> > that we need to have:
> > 
> > How do we want to do RPC in a way that enables us to make GNU Radio
> > far
> > more machine-agnostic than it is? How does one not only allow for
> > gathering of statistics and calling of functions, but build an RPC
> > framework that makes heterogeneous and distributed GNU Radio really
> > feasible?
> > 
> > We've been addressing the question of what the scheduler needs to
> > become at the heterogeneous compute working group at GRCon'18. To
> > conclude, we need to get away from the "one buffer type fits all"
> > and
> > "all blocks are born equal and are just individual OS threads"
> > towards
> > domain-specific subschedulers and buffer managers. This is where
> > this
> > needs to tie in.
> > 
> > Hence: Is anyone seriously using ControlPort and needs it for 3.8?
> > Please raise a metaphorical hand on the mailing list or write a
> > private
> > one in response to this email.
> > 
> > Best regards,
> > Marcus
> > 
> > ----------------------------------
> > 
> > Bug: clang correctly has been complaining for quite a while now
> > that
> > the RPCAggregator stores a reference to a temporary object. It
> > seems
> > that in the past this just worked by chance – probably, libc never
> > really got around to reassign the memory of these temporary objects
> > and
> > all lived happily everafter. However, now on multiple platforms, we
> > just see aborts in various tests due to access to freed memory.
> > Probably memory protection just got smart enough to kick some sense
> > into this code.
> > All is fine as long as one doesn't actually use the code in
> > question.
> > 
> > 
> > _______________________________________________
> > Discuss-gnuradio mailing list
> > address@hidden
> > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> > 
> 
> _______________________________________________
> Discuss-gnuradio mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio