[nongnu] elpa/gptel 6af54839d0 1/6: gptel: Add support for handling reasoning blocks

[ELPA Syncer]

branch: elpa/gptel
commit 6af54839d08cc74c59d786a0a2364fc96d1b0381
Author: Karthik Chikmagalur <karthikchikmagalur@gmail.com>
Commit: Karthik Chikmagalur <karthikchikmagalur@gmail.com>

    gptel: Add support for handling reasoning blocks
    
    Provide options for handling reasoning text ("<think>...</think>")
    in the response.  NOTE: This change only covers APIs that include
    the reasoning block in the main response.  APIs that provide the
    reasoning text in a separate JSON field (such as Deepseek) will be
    handled in the following commits.
    
    * gptel.el: (gptel--url-get-response, gptel--insert-response,
    gptel-include-reasoning, gptel-request): New user option
    `gptel-include-reasoning' to specify how reasoning blocks in the
    response should be handled.  They can be omitted from the
    response, included normally in the response, included but ignored
    in subsequent conversation turns, or redirected to a buffer.
    Handle thinking blocks in the process sentinel and the default
    non-streaming callback `gptel--insert-response'.
    
    gptel-request's callback is now called with a cons cell of the
    form (reasoning . text), where text is the reasoning string.
    Mention this in the documentation of gptel-request.
    
    * gptel-curl.el (gptel-curl--sentinel, gptel-curl--stream-filter,
    gptel-curl--stream-insert-response): Handle thinking blocks in the
    process filter and in the default streaming callback
    `gptel-curl--stream-insert-response'.
    
    * gptel-transient.el (transient-format-value, gptel-menu,
    gptel--infix-include-reasoning): Add a menu entry to set
    `gptel-include-reasoning'.  As with other gptel options, it can be
    set globally, for the buffer or for the next request only.
---
 README.org         |  5 ++++
 gptel-curl.el      | 41 ++++++++++++++++++++++++-
 gptel-transient.el | 43 ++++++++++++++++++++++++++-
 gptel.el           | 87 +++++++++++++++++++++++++++++++++++++++++++++++-------
 4 files changed, 164 insertions(+), 12 deletions(-)

diff --git a/README.org b/README.org
index 2e5c3a6908..f4ec0e26b1 100644
--- a/README.org
+++ b/README.org
@@ -115,6 +115,7 @@ gptel uses Curl if available, but falls back to the 
built-in url-retrieve to wor
       - [[#save-and-restore-your-chat-sessions][Save and restore your chat 
sessions]]
     - 
[[#setting-options-backend-model-request-parameters-system-prompts-and-more][Setting
 options (backend, model, request parameters, system prompts and more)]]
     - [[#include-more-context-with-requests][Include more context with 
requests]]
+    - [[#handle-reasoning-content][Handle "reasoning" content]]
     - [[#tool-use-experimental][Tool use (experimental)]]
       - [[#defining-gptel-tools][Defining gptel tools]]
       - [[#selecting-tools][Selecting tools]]
@@ -1014,6 +1015,10 @@ You can examine the active context from the menu:
 And then browse through or remove context from the context buffer:
 #+html: <img 
src="https://github.com/karthink/gptel/assets/8607532/79a5ffe8-3d63-4bf7-9bf6-0457ab61bf2a";
 align="center" alt="Image showing gptel's context buffer.">
 
+*** Handle "reasoning" content
+
+Some LLMs include in their response a "thinking" or "reasoning" block.  This 
text improves the quality of the LLM’s final output, but may not be interesting 
to you by itself.  You can decide how you would like this "reasoning" content 
to be handled by gptel by setting the user option =gptel-include-reasoning=.  
You can include it in the LLM response (the default), omit it entirely, include 
it in the buffer but ignore it on subsequent conversation turns, or redirect it 
to another buffer.  [...]
+
 *** Tool use (experimental)
 
 gptel can provide the LLM with client-side elisp "tools", or function 
specifications, along with the request.  If the LLM decides to run the tool, it 
supplies the tool call arguments, which gptel uses to run the tool in your 
Emacs session.  The result is optionally returned to the LLM to complete the 
task.
diff --git a/gptel-curl.el b/gptel-curl.el
index 424f5f8dd3..050631f64b 100644
--- a/gptel-curl.el
+++ b/gptel-curl.el
@@ -258,6 +258,18 @@ Optional RAW disables text properties and transformation."
            ;; (run-hooks 'gptel-pre-stream-hook)
            (insert response)
            (run-hooks 'gptel-post-stream-hook)))))
+    (`(reasoning . ,text)
+     (pcase (plist-get info :include-reasoning)
+       ('nil)
+       ('t (gptel-curl--stream-insert-response text info))
+       ('ignore
+        (add-text-properties
+         0 (length text) '(gptel ignore front-sticky (gptel)) text)
+        (gptel-curl--stream-insert-response text info t))
+       ((pred stringp)
+        (with-current-buffer (get-buffer-create (plist-get info :reasoning))
+          (save-excursion (goto-char (point-max))
+                          (insert text))))))
     (`(tool-call . ,tool-calls)
      (gptel--display-tool-calls tool-calls info))
     (`(tool-result . ,tool-results)
@@ -265,7 +277,8 @@ Optional RAW disables text properties and transformation."
 
 (defun gptel-curl--stream-filter (process output)
   (let* ((fsm (alist-get process gptel--request-alist))
-         (proc-info (gptel-fsm-info fsm)))
+         (proc-info (gptel-fsm-info fsm))
+         (thinking (plist-get proc-info :thinking)))
     (with-current-buffer (process-buffer process)
       ;; Insert output
       (save-excursion
@@ -296,6 +309,24 @@ Optional RAW disables text properties and transformation."
                     (response ;; (funcall (plist-get proc-info :parser) nil 
proc-info)
                      (gptel-curl--parse-stream (plist-get proc-info :backend) 
proc-info))
                     ((not (equal response ""))))
+          ;; :thinking has three states: nil before checking for <think> 
blocks,
+          ;; t when in a <think> block, 'done after or otherwise.
+          (unless (eq thinking 'done)
+            (if thinking
+                (if-let* ((idx (string-match-p "</think>" response)))
+                    (progn (funcall (or (plist-get proc-info :callback)
+                                        #'gptel-curl--stream-insert-response)
+                                    (cons 'reasoning
+                                          (string-trim-left
+                                           (substring response nil (+ idx 8))))
+                                    proc-info)
+                           (setq response (substring response (+ idx 8)))
+                           (plist-put proc-info :thinking 'done))
+                  (setq response (cons 'reasoning response)))
+              (if (string-match-p "^ *<think>" response)
+                  (progn (setq response (cons 'reasoning response))
+                         (plist-put proc-info :thinking t))
+                (plist-put proc-info :thinking 'done))))
           (funcall (or (plist-get proc-info :callback)
                        #'gptel-curl--stream-insert-response)
                    response proc-info))))))
@@ -330,6 +361,14 @@ PROCESS and _STATUS are process parameters."
         (gptel--fsm-transition fsm)     ;WAIT -> TYPE
         (when error (plist-put proc-info :error error))
         (when (or response (not (member http-status '("200" "100"))))
+          (when (string-match-p "^ *<think>\n" response)
+            (when-let* ((idx (string-search "</think>\n" response)))
+              (with-demoted-errors "gptel callback error: %S"
+                (funcall proc-callback
+                         (cons 'reasoning (substring response nil (+ idx 8)))
+                         proc-info))
+              (setq response
+                    (string-trim-left (substring response (+ idx 8))))))
           (with-demoted-errors "gptel callback error: %S"
             (funcall proc-callback response proc-info))))
       (gptel--fsm-transition fsm))      ;TYPE -> next
diff --git a/gptel-transient.el b/gptel-transient.el
index aecda6f6af..9e092dcafb 100644
--- a/gptel-transient.el
+++ b/gptel-transient.el
@@ -395,7 +395,7 @@ which see."
   (let ((display-value
          (with-slots (value display-nil display-map) obj
            (cond ((null value) display-nil)
-                 (display-map (cdr (assoc value display-map)))
+                 (display-map (or (cdr (assoc value display-map)) value))
                  (t value)))))
     (propertize
      (if (stringp display-value) display-value (prin1-to-string display-value))
@@ -603,6 +603,7 @@ Also format its value in the Transient menu."
                     (or gptel-mode gptel-track-response))))
     (gptel--infix-temperature :if (lambda () gptel-expert-commands))
     (gptel--infix-use-context)
+    (gptel--infix-include-reasoning)
     (gptel--infix-use-tools)
     (gptel--infix-track-response
      :if (lambda () (and gptel-expert-commands (not gptel-mode))))
@@ -1096,6 +1097,46 @@ Or in an extended conversation:
   :description "Add instruction"
   :transient t)
 
+;; ** Infix for reasoning block control
+
+(transient-define-infix gptel--infix-include-reasoning ()
+  "How to handle reasoning/thinking response blocks.
+
+Some LLMs include in their response a \"thinking\" section.  This
+text improves the quality of the LLM's final output, but may not
+be interesting to you by itself.
+
+You can control how gptel should handle the thinking blocks via
+this option, or by setting the variable `gptel-include-reasoning'
+via elisp, which see.
+
+Available behaviors are
+- to include thinking blocks with the response,
+- to omit them entirely,
+- to include them but ignore them in consequent conversation turns, and
+- to append them to a buffer of your choosing."
+  :description "Include reasoning"
+  :class 'gptel-lisp-variable
+  :variable 'gptel-include-reasoning
+  :format " %k %d %v"
+  :set-value #'gptel--set-with-scope
+  :display-nil "No"
+  :display-map '((nil    . "No")
+                 (ignore . "and ignore")
+                 (t      . "with response"))
+  :key "-r"
+  :prompt "Include reasoning: "
+  :reader (lambda (prompt &rest _)
+            (let* ((choices '(("no"     . nil)
+                              ("ignore" . ignore)
+                              ("yes"    . t)
+                              ("other buffer" . buffer)))
+                   (destination
+                    (completing-read prompt choices nil t)))
+              (if (equal destination "other buffer")
+                  (read-buffer "Append reasoning to buffer: ")
+                (cdr (assoc destination choices))))))
+
 ;; ** Infixes for tool use
 
 (transient-define-infix gptel--infix-use-tools ()
diff --git a/gptel.el b/gptel.el
index 52810fd9ff..f0c293315a 100644
--- a/gptel.el
+++ b/gptel.el
@@ -775,6 +775,29 @@ Currently supported options are:
           (const :tag "With system message" system)
           (const :tag "With user prompt" user)))
 
+(defcustom gptel-include-reasoning t
+  "How to handle LLM reasoning or \"thinking\" text blocks.
+
+Some LLMs include in their response a \"thinking\" section.  This
+text improves the quality of the LLM's final output, but may not
+be interesting to you by itself.
+
+Supported options are the symbols
+
+    t       - Include with the response, the default
+    nil     - Do not include
+    ignore  - Include with the response but ignore on subsequent
+              conversation turns
+
+It can also be a string naming a buffer, in which case the
+reasoning text will be inserted at the end of that buffer."
+  :group 'gptel
+  :type '(choice
+          (const :tag "Include with response" t)
+          (const :tag "Don't include" nil)
+          (const :tag "Include but ignore" ignore)
+          (string :tag "Include in buffer")))
+
 (defvar-local gptel--old-header-line nil)
 
 (defvar gptel-context--alist nil
@@ -2093,21 +2116,38 @@ RESPONSE is
 - nil if there was no response or an error.
 
 These are the only two cases you typically need to consider,
-unless you need to clean up after aborted requests, use LLM tools
-or streaming responses (see STREAM).  In these cases, RESPONSE
-can be
+unless you need to clean up after aborted requests, use LLM
+tools, handle \"reasoning\" content specially or stream
+responses (see STREAM).  In these cases, RESPONSE can be
 
 - The symbol `abort' if the request is aborted, see `gptel-abort'.
 
-- A list beginning with `tool-call'.  The cdr form is (TOOL ARGS
-  CALLBACK) ...) where TOOL is a gptel-tool struct, ARGS is a plist of
-  arguments, and CALLBACK is a function for handling the results.
+- A cons cell of the form
+
+  (tool-call . ((TOOL ARGS CB) ...))
+
+  where TOOL is a gptel-tool struct, ARGS is a plist of
+  arguments, and CB is a function for handling the results.  You
+  can call CB with the result of calling the tool to continue the
+  request.
+
+- A cons cell of the form
+
+  (tool-result . ((TOOL ARGS RESULT) ...))
+
+  where TOOL is a gptel-tool struct, ARGS is a plist of
+  arguments, and RESULT was returned from calling the tool
+  function.
+
+- A cons cell of the form
+
+  (reasoning . text)
 
-- A list beginning with `tool-result'.  The cdr form is ((TOOL ARGS
-  RESULT) ...) where TOOL is a gptel-tool struct, ARGS is a plist of
-  arguments, and RESULT was returned from calling the tool function.
+  where text is the contents of the reasoning block.  (Also see
+  STREAM if you are using streaming.)
 
-See `gptel--insert-response' for an example callback handling all cases.
+See `gptel--insert-response' for an example callback handling all
+cases.
 
 STREAM is a boolean that determines if the response should be
 streamed, as in `gptel-stream'.  If the model or the backend does
@@ -2119,6 +2159,9 @@ When streaming responses
   chunk (a string) as it is received.
 - When the HTTP request ends successfully, CALLBACK will be
   called with a RESPONSE argument of t to indicate success.
+- Similarly, CALLBACK will be called with
+  (reasoning . text-chunk) for each reasoning chunk, and
+  (reasoning . t) to indicate the end of the reasoning block.
 
 The INFO plist has (at least) the following keys:
 :data         - The request data included with the query
@@ -2254,6 +2297,8 @@ be used to rerun or continue the request at a later time."
     (when callback (plist-put info :callback callback))
     (when context (plist-put info :context context))
     (when in-place (plist-put info :in-place in-place))
+    (when gptel-include-reasoning       ;Required for next-request-only scope
+      (plist-put info :include-reasoning gptel-include-reasoning))
     (when (and gptel-use-tools gptel-tools)
       (plist-put info :tools gptel-tools))
     ;; Add info to state machine context
@@ -2419,6 +2464,19 @@ Optional RAW disables text properties and 
transformation."
              (plist-put info :tracking-marker (setq tracking-marker 
(point-marker)))
              ;; for uniformity with streaming responses
              (set-marker-insertion-type tracking-marker t)))))
+      (`(reasoning . ,text)
+       (pcase (plist-get info :include-reasoning)
+         ('t (gptel--insert-response text info))
+         ('nil)
+         ('ignore
+          (add-text-properties
+           0 (length text) '(gptel ignore front-sticky (gptel)) text)
+          (gptel--insert-response text info t))
+         ((pred stringp)
+          (with-current-buffer (get-buffer-create
+                                (plist-get info :include-reasoning))
+            (save-excursion (goto-char (point-max))
+                            (insert text))))))
       (`(tool-call . ,tool-calls)
        (gptel--display-tool-calls tool-calls info))
       (`(tool-result . ,tool-results)
@@ -2628,6 +2686,15 @@ the response is inserted into the current buffer after 
point."
                              (gptel--fsm-transition fsm) ;WAIT -> TYPE
                              (when error (plist-put info :error error))
                              (when (or response (not (member http-status 
'("200" "100"))))
+                               (when (string-match-p "^ *<think>\n" response)
+                                 (when-let* ((idx (string-search "</think>\n" 
response)))
+                                   (with-demoted-errors "gptel callback error: 
%S"
+                                     (funcall callback
+                                              (cons 'reasoning
+                                                    (substring response nil (+ 
idx 8)))
+                                              info))
+                                   (setq response (string-trim-left
+                                                   (substring response (+ idx 
8))))))
                                (with-demoted-errors "gptel callback error: %S"
                                  (funcall callback response info)))
                              (gptel--fsm-transition fsm) ;TYPE -> next