Re: option for enabling/disabling auto-suggest feature

[Sudeepam Pandey]

On Wed, May 2, 2018 at 1:48 AM, Doug Stewart <address@hidden> wrote:

On Tue, May 1, 2018 at 3:42 PM, Sudeepam Pandey <address@hidden> wrote:

On Tue, May 1, 2018 at 9:43 PM, Sudeepam Pandey <address@hidden> wrote:

On Tue, May 1, 2018 at 3:00 AM, Rik <address@hidden> wrote:

On 04/30/2018 12:57 PM, Sudeepam Pandey wrote:

On Fri, Apr 27, 2018 at 3:13 AM, Nicholas Jankowski <address@hidden> wrote:

On Thu, Apr 26, 2018 at 4:46 PM, Sudeepam Pandey <address@hidden> wrote:

Yes, I'll make sure that the auto-suggest feature has an easy enable/disable option. Maybe something like a command/ shortcut key to disable/enable the feature.

How may depend on how you implement it. if you made it strictly a GUI popup, then a GUI switch could make sense. if it will be inline with the interpreter, then it needs to be functional for --no-gui users. a shortcut key may be too dependent on your terminal of choice. So a command similar to 'more on' 'more off' may be the simplest and most portable. As per my message in the other thread, you should review how others have done it before.

Okay, so I was looking at some implementations of suggestion feature. I examined the suggestion feature of git, mercurial, and gcc. Most thoroughly, the one of git. I have also seen how MATLAB displays suggestions (the result that is, not the code). My research is ongoing but here are some initial observations.

- The most common algorithm for finding which words to suggest seems to be edit distance.

- I haven't figured out how gcc and hg turn on/ turn off this feature yet, but here is how git does it...

...when git realizes that the user has entered a command which it does not recognize, it pops up a 'did you mean' suggestion which has been calculated based on the edit distance algorithm. By default, it simply pops out a did you mean suggestion and does nothing more...git can however be customized to execute the suggested command directly after some time delay which can be defined using the help.autocorrect <time> command. Here is an example...

I strongly suggest that we implement some command to enable/disable the feature but do not add auto-execute like git...

Yes, there must be an enable/disable feature.  I also agree that we should NOT implement an auto-execute option at this time.

Its decided then.

Although I am still doing my research, here are my initial suggestions (based on some research and the discussions so far) about what the command line suggestion feature should be like....

1) The command line suggestion feature is triggered, whenever the interpreter realizes that it has to throw a...
   - "error: '<typo>' undefined near..." error
   - "error: set: unknown <axes/line etc..> property <property name>" error

   - others? (suggestions welcome)

I don't think that looking at other softwares to see when their suggestion features are triggered would be very useful because the errors and structures of all softwares are different after-all. The essence however, for every software, probably is that the compiler/interpreter of the software doesn't understand the meaning of a particular identifier that it has encountered. Therefore, I believe, that the errors we are looking for are the kind of errors where octave interpreter realizes that it doesn't understand what a particular word that the user has written means (and hence, is most probably a typo). The two errors mentioned above probably cover most of the cases. If anyone thinks that they do not, please suggest.

This seems correct.  This should get us coverage of most of the obvious typos.

Lets start the work with these then. If someone suggests something later on, we can modify it.

2) Immediately after the aforementioned errors are thrown, a script (say didYouMean) is called with the misspelled function/property name/ keyword... etc...

This architecture is very similar to the one that Octave already uses for unimplemented functions.  See the documentation for missing_function_hook().  The function missing_function_hook() works together with the parser.  When the parser doesn't recognize an identifier, it calls whatever has been set up as the callback function in missing_function_hook().  Currently, that function is scripts/help/__unimplemented__.  Take a look as well at __unimplemented__.m to see how it operates.

Thank you. I'll take a look and ask if I have further doubts.

The Octave project coding convention is not to use CamelCase so perhaps your function should be __suggestions__ rather than didYouMean?

Of course. That was just an example.

For development, you can quickly substitute your own suggestion function by pointing the missing_function_hook to your own m-file.

Note 1: This would require adding a small piece of code to the main file that generates these error messages on the command window. Can anyone help me figure out what files these are?

It seems to me that there may need to be a missing_property_hook () function, written in C++, that would behave like missing_function_hook when a graphics property is unrecognized.

I can try to implement it after I understand how missing_function_hook () works. Can't be sure at this point though. Let me take a look at missing_function_hook () first.

Note 2: To add a on/off feature. This script should be callable from the command window, with a simple flag value (0/1) to (turn off/turn on) this feature. I am not very sure about this but I suppose that if the on/off feature is implemented in this way, then it would not require separate implementations for GUI and CLI. The flag value will be permanent and can only be changed by calling the script with the appropriate argument from the command window.

3) If the flag value is 0 then further execution of the script does not take place and nothing is printed on the octave command window. User can continue with whatever they were doing.

4) If the flag value was 1, a separate script is called, which calculates what functions/keywords to suggest based on the edit distance (to begin with), or Neural Networks (at a later stage maybe) or some other better algorithm (if we happen to find one). This algorithm should have minimum speed-accuracy trade-off. Since the algorithm will be written down in a separate script, it should be easier to modify later on as technology evolves.

Sure, this would work.  Alternatively, just set missing_component_hook to "" when you don't want to use it.

4) The suggestion(s) are returned to the previous script (didYouMean) in the form of a list (in an array that is). Let the array that stores the suggestions be named 'suggestions' The script then generates something like this on the command window...

ex: say the particular misspelled word was 'akuse'

>> akuse
>> error: 'akuse' undefined near line 1 column 1

Did you mean any of the following...

1) amuse

2) abuse

3) amused

Enter the corresponding number to execute the command...

>>1 #say user enters 1

5) The user types 1 and presses enter...didYouMean execute the command corresponding to suggestions(1).

Take a look at the function menu().  It takes a cell array of strings as input and creates a numbered list, and then returns the selected input.  It's already coded and does precisely what you want.

--Rik

Alright I'll take a look.

Thanks for your detailed reply Rik. This would give me a very good starting point.

I played with __unimplemented__.m function a little and it seems like this is exactly what we need to integrate a command line suggestion into Octave. I edited the script a little so that, whenever octave fails to find an identifier among the unimplemented functions list as well, it calls my __suggestion__.m script, and this has worked nicely to produce a "Did you mean?" suggestion in the Octave command window in both GUI and CLI.

As of now, there is no real 'suggestion code' in __suggestion__.m script and it returns a simple line only... the point was to show that it works for CLI and GUI as well.

Here are the results:-

GUI

>> cld
did you mean: clc
error: 'cld' undefined near line 1 column 1
>>

CLI

octave:1> cld
did you mean: clc
error: 'cld' undefined near line 1 column 1
octave:1>

A lot of discussions have been done by now and I am thankful to the community for the same. To me, it seems like I am ready to start with the actual coding part of the project.

If members of the community have anymore suggestions, concerns etc. please share them.

To my assigned GSoC mentors, Douglas Stewart and Nicholas Jankowski, should I do the following now, before the actual coding period begins...?

1) Share an updated timeline with both of you, based on the discussions so far.

Yes

 

2) Start a post about the discussions that went through during the community bonding period on my blog.

Yes

 

3) Share with everyone, a prototype of the 'algorithm independent' implementation that shows how this feature integrates itself with octave through my public bitbucket repository?

Not sure that this is too important yet.

 

--P Sudeepam

--

DAS

Understood Doug.

There is one last thing left for me to figure out kindly help me with the same.

Like I sad before...I edited the __unimplemented__.m script a little so that, whenever octave fails to find an identifier among the unimplemented functions list as well, it calls my __suggestion__.m script, and this has worked nicely to produce a "Did you mean?" suggestion in the Octave command window in both GUI and CLI.

There are a two things that still need to be taken care of..

1) First of all, we absolutely need a on/off switch for this feature. However, __suggestion__.m will be designed as an internal function (called when an identifier is not found in the unimplemented functions list) so users cannot directly call this function.
So to design a command which can act as a switch for this feature, we can write a m script (say flag.m) which is callable from the command window with a flag value and which would save the desired flag value (indicating on/off) in a .mat file in the "..octave/scripts/help" directory (where __suggestions__.m would reside). The __suggestions__.m script would load this value as soon as it is called and act accordingly. This way the user setting will be saved and will remain protected unless they modify it themselves.
I however, don't understand what files do I have to tweak so that the parser learns to identify this function (flag.m) as a new function of octave, callable from the command window. Also, if we went along with this, what path should I set inside the flag.m file so that the .mat file in "..octave/scripts/help" directory gets updated successfully? considering that the path BEFORE "../octave/scripts/help" would be different for each user, depending on where they have saved their copy of octave.
I understand that I may have said somethings incorrectly here, or that my thinking may be wrong at some places (for example, maybe we have to have a flag.cc file and not a flag.m file) but this approach is probably how a very good on/off feature could be made. Help and suggestions would be appreciated.

2) Another thing that bothers me is the absence of missing_property_hook() function...the approach that we have discussed does not work when we make a typo while typing a graphic/line property etc.. We can choose to add a missing_property_hook() function but this would have additional challenges. Examples are...

   a) How do we do it? What files do we tweak?

   b) If we finally make it, what does the parser prioritize when it doesn't recognize an identifier? missing_function_hook or the missing_property_hook?

...and maybe more challenges when we start thinking how?

I really would like to include properties as well but I'm not sure if they could be done right now. Can anyone help with this?

--P Sudeepam