bug#34873: 27.0.50; Creating unibyte string in emacs modules

[Evgeny Zajcev]

There is no possibility to create unibyte strings from emacs modules in order to create some binary data representation

The only solution for now is to call `make_string` method and then apply `encode-coding-string' with utf-8 encoding to the result.  But on large data this became very slow

I used this patch to bypass the conversion steps and it works pretty nice

Ideally I would like to use user-ptr objects as `:data' in image specification, but solution with `make_data' module method will be suitable enough

Thanks

Here is the patch I'm currently using:

diff --git a/src/emacs-module.c b/src/emacs-module.c
index 4e2411cb29..500079c257 100644
--- a/src/emacs-module.c
+++ b/src/emacs-module.c
@@ -591,6 +591,19 @@ module_make_string (emacs_env *env, const char *str, ptrdiff_t length)
                         code_convert_string_norecord (lstr, Qutf_8, false));
 }
 
+static emacs_value
+module_make_data (emacs_env *env, const char *str, ptrdiff_t length)
+{
+  MODULE_FUNCTION_BEGIN (module_nil);
+  if (! (0 <= length && length <= STRING_BYTES_BOUND))
+    overflow_error ();
+
+  Lisp_Object lstr = make_uninit_string (length + 1);
+  memcpy (SDATA (lstr), str, length);
+  SDATA (lstr)[length] = 0;
+  return lisp_to_value (env, lstr);
+}
+
 static emacs_value
 module_make_user_ptr (emacs_env *env, emacs_finalizer_function fin, void *ptr)
 {
@@ -1081,6 +1094,7 @@ initialize_environment (emacs_env *env, struct emacs_env_private *priv)
   env->make_float = module_make_float;
   env->copy_string_contents = module_copy_string_contents;
   env->make_string = module_make_string;
+  env->make_data = module_make_data;
   env->make_user_ptr = module_make_user_ptr;
   env->get_user_ptr = module_get_user_ptr;
   env->set_user_ptr = module_set_user_ptr;

Configuration:

In GNU Emacs 27.0.50 (build 19, x86_64-pc-linux-gnu, GTK+ Version 3.22.30)
 of 2019-03-15 built on XPS
Repository revision: 020e69d992c98fd852e835c9bd707a8d137090f2
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.11906000
System Description: Ubuntu 18.04.1 LTS

Recent messages:
t
Mark set [2 times]
Auto-saving...
Mark set [2 times]
t
Saving file /home/lg/.emacs.d/*scratch-file*...
Wrote /home/lg/.emacs.d/*scratch-file*
Finding changes in /home/lg/dev/emacs/src/emacs-module.c...done
Saved text until "nv->set_user_ptr = module_set_user_ptr;
"

Configured using:
 'configure --without-makeinfo --with-xwidgets --with-modules'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS GLIB
NOTIFY INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF
XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS XWIDGETS
PDUMPER LCMS2 GMP

Important settings:
  value of $LC_MONETARY: ru_RU.UTF-8
  value of $LC_NUMERIC: ru_RU.UTF-8
  value of $LC_TIME: ru_RU.UTF-8
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: C/*

Minor modes in effect:
  icomplete-mode: t
  save-place-mode: t
  pyvenv-mode: t
  shell-dirtrack-mode: t
  dot-mode: t
  display-time-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  abbrev-mode: t

Load-path shadows:
~/github/telega.el/test hides ~/github/emacs-ffi/test
/home/lg/.emacs.d/elpa/flim-20180328.2324/md4 hides /usr/local/share/emacs/27.0.50/lisp/md4
/home/lg/.emacs.d/elpa/flim-20180328.2324/hex-util hides /usr/local/share/emacs/27.0.50/lisp/hex-util
/home/lg/.emacs.d/elpa/flim-20180328.2324/sasl-digest hides /usr/local/share/emacs/27.0.50/lisp/net/sasl-digest
/home/lg/.emacs.d/elpa/flim-20180328.2324/sasl-ntlm hides /usr/local/share/emacs/27.0.50/lisp/net/sasl-ntlm
/home/lg/.emacs.d/elpa/flim-20180328.2324/hmac-md5 hides /usr/local/share/emacs/27.0.50/lisp/net/hmac-md5
/home/lg/.emacs.d/elpa/flim-20180328.2324/sasl hides /usr/local/share/emacs/27.0.50/lisp/net/sasl
/home/lg/.emacs.d/elpa/flim-20180328.2324/ntlm hides /usr/local/share/emacs/27.0.50/lisp/net/ntlm
/home/lg/.emacs.d/elpa/flim-20180328.2324/hmac-def hides /usr/local/share/emacs/27.0.50/lisp/net/hmac-def
/home/lg/.emacs.d/elpa/flim-20180328.2324/sasl-cram hides /usr/local/share/emacs/27.0.50/lisp/net/sasl-cram

Features:
(shadow sort mail-extr emacsbug sendmail smerge-mode vc vc-dispatcher
vc-git cc-mode cc-fonts cc-guess cc-menus cc-styles cc-align ffi-wand
ffi ffi-module home desktop frameset gnus-demon nntp gnus-group
gnus-undo gnus-start gnus-cloud nnimap nnmail mail-source utf7 netrc
gnus-spec gnus-win nnoo gnus-int gnus-range message rmc puny dired
dired-loaddefs rfc822 mml mml-sec epa epg mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader gnus nnheader
gnus-util rmail rmail-loaddefs rfc2047 rfc2045 ietf-drums time-date
mail-utils mm-util mail-prsvr cal-menu calendar cal-loaddefs icomplete
saveplace cython-mode help-fns radix-tree elpy find-file-in-project ivy
delsel colir color ivy-overlay ffap windmove diff-mode elpy-shell pyvenv
esh-var esh-cmd esh-opt esh-io esh-ext esh-proc esh-arg esh-groups
eshell esh-module esh-mode esh-util elpy-profile elpy-django s
elpy-refactor python tramp-sh tramp tramp-loaddefs trampver
tramp-integration tramp-compat ucs-normalize parse-time format-spec grep
files-x etags fileloop generator xref project cus-edit cus-start
cus-load wid-edit python-mode pcase info-look which-func imenu shell
pcomplete hippie-exp flymake-proc flymake warnings thingatpt compile
cc-cmds cc-engine cc-vars cc-defs rx dot-mode server time elec-pair
google-translate google-translate-default-ui google-translate-core-ui
google-translate-core google-translate-tk url url-proxy url-privacy
url-expand url-methods url-history url-cookie url-domsuf url-util
mailcap whitespace undo-tree diff ido comint ansi-color ring avoid
edmacro kmacro browse-kill-ring advice derived cl-extra help-mode
use-package use-package-ensure use-package-delight use-package-diminish
use-package-bind-key bind-key easy-mmode use-package-core mule-util
autoinsert tex-site gh-common marshal eieio-compat cl info package
easymenu epg-config url-handlers url-parse auth-source cl-seq eieio
eieio-core cl-macs eieio-loaddefs password-cache json subr-x map
url-vars seq byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify lcms2 dynamic-setting
system-font-setting font-render-setting xwidget-internal move-toolbar
gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 457704 32112)
 (symbols 48 29656 2)
 (strings 32 92291 5700)
 (string-bytes 1 15482759)
 (vectors 16 62874)
 (vector-slots 8 1478873 216844)
 (floats 8 425 132)
 (intervals 56 879 164)
 (buffers 992 16))

--
lg


--

lg