[MLton-user] Performance issue with -const 'Exn.keepHistory true'

[Matthew Fluet]

> We used to compile our code with MLton 20030716p1 with the option
> '-exn-history true'.
>
> It was very useful when we had to solve some minor robustness issues as it
> was not too costly (we could use it on production code) and it gave us the
> exact location of the exception and "handle" clauses.

What slowdown is considered "not too costly"?

> I know the option (now -const 'Exn.keepHistory true') has been improved in
> order to print a full call stack but the consequence is that it is now too
> costly to be used in production code when compiling using MLton 20050712.

Correspondingly, what slowdown is considered "too costly"?

> Is there a way to get the old behaviour using MLton 2005 ?

Unfortunately, no.  The infrastructure that supported the old-style 
exception history was removed.

The log for the commit that changed over to the new exception history 
style includes the following note:

   Used MLton.CallStack to improve MLton.Exn.history.  Exn.keepHistory
   true now implies -profile call.  The first time an exception is
   raised, we call MLton.CallStack.current and store the result in the
   raised exception.  Exn.history simply extracts the call stack and
   converts it to a string list.  The output of this is a vast
   improvement over what was there before.  However, there is a cost --
   raising an exception now takes time proportional to the size of the
   call stack instead of constant time.  Of course, this is only with
   Exn.keepHistory true, which will probably only be used for debugging.
   I can imagine trickier implementations that have better performance,
   only walking the stack up to the handler, and walking the rest if the
   exception is later re-raised.  A user could also roll their own call
   stack stuff (say, for Assert.assert), where the program is compiled
   with -profile call, but with Exn.keepHistory false, and only grabs the
   call stack when there is an exception that is really an error, as
   opposed to an exception used for some other control flow.  Another
   approach would be for us to provide special raise functions that {do,
   do not} grab the history.

I'll highlight three things:
  1) The cost proportional to the call stack is incurred whether or not the
     exception is handled and whether or not the exception's history is
     taken.  Hence, if one uses exceptions as a control-flow mechanism,
     then they can be fairly costly.
  2) The old exception history wasn't quite constant time.  It was time
     proportional to the (dynamic) number of exception handlers between
     the exception's raise point and the exception's handle point.  In the
     worst case, this would itself be proporational to the size of the call
     stack, but, in practice, the number of dynamic handlers is rarely very
     high.  Furthermore, if one is using exceptions as a control-flow
     mechanism, then the number of dynamic handlers is likely to be very
     small (for that raised exception).
  3) The fact that 'Exn.keepHistory true' implies '-profile call' adds some
     additional overhead to the program.  This is because it instructs the
     compiler to maintain source code position information used by the
     various kinds of profiling.  It is meant to be as lightweight as
     possible; with '-profile call', the source information does not
     translate into any runtime operations (it may with the other kinds of
     profiling).  However, the simple presence of the source code
     information in the intermediate code can affect some optimizations, so
     one won't necessarily have exactly the program.

I would suggest the following:
  1) Compile your program with  -profile call  (and without -const
     'Exn.keepHistory true').

     a) If this program doesn't have acceptable performance, then there may
        be an optimization that is missed in the presence of the profiling
        information.  We would be interested in figuring out why the
        optimization is being missed and improve the compiler to catch it.
        This could improve the performance of 'Exn.keepHistory true' so
        that you could use it on production code.

     b) If this program does have acceptable performance, then it would
        appear that your program is raising many exceptions (with large
        call stacks), but handling then without reporting the history.

        In this case, as mentioned in the note above, you could roll your
        own call stack stuff and only capture the call stack at points
        where you think you might be interested in the failure.  Of course,
        this only really helps if the robustness issues you are interested
        in correspond to raises of particular exceptions that you can
        statically find in your codebase. (i.e., you are interested in
        exceptions of the form "Assert of string", but you don't care about
        "Overflow" or "Subscript" exceptions.

        In this case, we could also consider other ways of making exception
        history more efficient.  For example, in addition to the ideas in
        the note above, we could acheive constant time raises by only
        taking a fixed number of stack frames when raising an exception.  I
        could imagine that only taking 5 frames would often give you
        sufficient context to serve debugging purposes.

Hope this helps.