Re: [lmi] any_member<Input>::str() slowness

[Greg Chicares]

On 2020-06-11 08:51, Vadim Zeitlin wrote:
> 
>  I've almost finished my optimizations of column auto-sizing in the census
> view: there is always more to do, but the time to do it for a census with
> 4000+ elements went down from ~2000ms to ~300ms and I think this should be
> good enough for now, especially considering that the current, wxDVC-based,
> version takes ~5500ms to do the same thing.

Thanks, that's great news.

>  However I'd like to mention one of the things which surprised me while I
> was profiling this code (as usual with profiling, a lot of things surprised
> me, but this was one of those that really stood out): any_member<>::str()
> is really, really slow. How slow exactly depends on what you measure and
> how you count, but roughly speaking this single function accounts for 30%
> of the total time spent in column auto-sizing. I understand that this is
> critical code and you are probably not terribly interested in making
> changes to it (at least my previous attempt to optimize things in roughly
> the same area in https://github.com/vadz/lmi/pull/114 never went anywhere),

At first, back in 2015, I was extremely hesitant to touch it, true,
though my 2019-05-26T22:27Z off-list email expressed much less
reluctance because of a change in external circumstances.

> but it still seems rather extraordinary that such simple operation consumes
> so much time, and I think it would be worth looking into it and trying to
> make it less inefficient.

any_member<ClassType>::str()'s implementation is just
    LMI_ASSERT(content_);
    return content_->str();
so I suppose the slowness must be here:

template<typename ClassType, typename ValueType>
std::string holder<ClassType,ValueType>::str() const
{
    LMI_ASSERT(object_);
    return value_cast<std::string>(object_->*held_);
}

in the value_cast<>() call.

This reminds me somewhat of the discussion of ledger_format() here:
  https://lists.nongnu.org/archive/html/lmi/2019-01/msg00005.html
where a technique similar to stream_cast<>() (which is presumably
the slowest part of value_cast<>()) is used.

>  Again, I won't do it for now, all my optimizations to lmi code itself (as
> opposed to those in wxGrid) are limited to census_view.cpp, but please let
> me know if you think it would be worth doing it.

Yes, I think it's worthwhile. IIRC, one of your suggestions for
any_member involved changing some standard container, e.g. from
std::map to std::unordered_map. In addition, there's the more
general issue of formatting numbers efficiently.