[Help-smalltalk] [PATCH 2/2] visualgst: Avoid live-lock on single-step debugging

[Holger Hans Peter Freyther]

Debugging "3 negated" and pressing "F7" could result in a
live-lock. While the dispatch routine of the mainloop was
ran the call-in process could wait on the "finalSemaphore"
and the "idle" process would be selected to execute as the
only process that remained runnable. This live-lock would
never be exited.

The underlying issue is that VisualGST is running in the
same image and can be impacted by the execution of the
debugged process. The only thing we can do right now is
to skip dangerous routines.

Currently the dangerous routines are related to the handling
of finalizers as the Glib/Gtk+ bindings create a lot of to
be finalized objects.

2014-05-05  Holger Hans Peter Freyther  <address@hidden>

        * Debugger/GtkDebugger.st: Introduce >>#finishDangerousContexts
        and call it from >>#debugWith:.
---
 packages/visualgst/ChangeLog               |  5 +++++
 packages/visualgst/Debugger/GtkDebugger.st | 18 ++++++++++++++++++
 2 files changed, 23 insertions(+)

diff --git a/packages/visualgst/ChangeLog b/packages/visualgst/ChangeLog
index 330b439..4451f6e 100644
--- a/packages/visualgst/ChangeLog
+++ b/packages/visualgst/ChangeLog
@@ -1,3 +1,8 @@
+2014-05-05  Holger Hans Peter Freyther  <address@hidden>
+
+       * Debugger/GtkDebugger.st: Introduce >>#finishDangerousContexts
+       and call it from >>#debugWith:.
+
 2014-02-10  Gwenael Casaccio  <address@hidden>
 
        * Source/ClassHeaderSource.st: Display instance side and class side 
pragmas.
diff --git a/packages/visualgst/Debugger/GtkDebugger.st 
b/packages/visualgst/Debugger/GtkDebugger.st
index 7c6983d..cea52c5 100644
--- a/packages/visualgst/Debugger/GtkDebugger.st
+++ b/packages/visualgst/Debugger/GtkDebugger.st
@@ -303,12 +303,30 @@ GtkBrowsingTool subclass: GtkDebugger [
         self debugWith: [ debugger finish ]
     ]
 
+    finishDangerousContexts [
+        | methods |
+        "VisualGST is running in the same image as the to be de-bugged
+        processed. If the process is suspended in a critical section
+        VisualGST might end up in a live-lock. As long as VisualGST is
+        running in the same image there is no way we can avoid this."
+
+        "Nothing is suspended."
+        debugger suspendedContext ifNil: [^self].
+
+        "Methods we need to finish or bad things will happen."
+        methods := {Object>>#removeToBeFinalized. Object>>#addToBeFinalized}.
+
+        (methods includes: debugger suspendedContext method)
+            ifTrue: [debugger finish].
+    ]
+
     debugWith: aBlock [
         <category: 'execute events'>
 
         TaskQueue uniqueInstance add: [
             self isLastContextSelected ifFalse: [ self stepToSelectedContext ].
             aBlock value.
+            self finishDangerousContexts.
             self updateContextWidget ]
     ]
 
-- 
1.9.1