tame/doc/todo.texi

@c  This document is part of the TAME manual.
@c  Copyright (C) 2017 R-T Specialty, LLC.
@c    Permission is granted to copy, distribute and/or modify this document
@c    under the terms of the GNU Free Documentation License, Version 1.3 or
@c    any later version published by the Free Software Foundation; with no
@c    Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
@c    A copy of the license is included in the section entitled ``GNU Free
@c    Documentation License''.

@node Project TODOs
@appendix Project TODOs
This project has far more TODOs than are listed here, but here are
  some higher-level goals for the future.
There are miscellaneous TODOs sprinkled throughout the code,
  but those are lower-level.


@section Linker
The linker should be one of the fastest parts of the system@mdash{
  }it should just do a topological sort,
    resolve symbols as necessary,
    and arrange code blocks.
Unfortunately,
  especially due to memory requirements of loading so many packages
    into memory (using Saxon),
  this has been a pain point.
Performance has improved drastically,
  but we can do better by gutting other parts of the system that were
    designed as kluges or now have better solutions that alleviates
    linker work.

@enumerate
  @item Limit recursion during stack processing.
        If you look at a stack trace on error,
          it can be hundreds of levels deep.
        Instead,
          use a trampoline-style approach and indicate that additional
          symbols need to be loaded,
            but don't do so immediately in a recursive manner.
        This may or may not improve the memory footprint@mdash{
          }hopefully Saxon will see that previously generated trees
            are no longer necessary and GC them.

  @item Remove concept of classifier vs. rating processes.
        This distinction was important in the past to work around
          pulling in too many dependencies,
            but there's not really much rationale for separate
              @code{classify} and @code{rate} methods now,
                and it'll get rid of a decent amount of work.
@end enumerate


@section Compiler
The compiler itself has cruft from over the years and most of the code
  exists from the design prior to object files and symbol tables.
There is plenty of room for optimization.

@enumerate
  @item Validate against an algebraic type system.
        Certain classes of bugs can be avoided entirely by stating that
          values must always be a vector of positive real numbers, for
          example.
        The compiler can determine the type of expressions by the
          symbols it references and by its child expressions.
        In situations that are not resolvable at compile-time,
          runtime checks via terminating classifications would do;
            an example being the sum of two unsigned integers being
            non-zero.
        In cases where types are simply too difficult to determine
          without significant compiler work,
            annotations can be used to declare expressions equivalent
            to certain mathematical concepts
              (e.g. ``this function is equivalent to rounding to the
                nearest integer'');
              this can also serve as an alternative to primitives in
                certain instances,
                  since native target language features could be used.
  @item Make parameters package-scoped and allow importers to
          determine what to do with them.
        If an importer provides a mapping for the parameter,
          which may be a calculation or anything else,
          then the parameter will be bound;
            otherwise it will be propagated as a parameter.
        But since we do not have any sort of loxical scoping or
          closures,
            if a parameter is bound more than once,
              an error would have to occur during linking.
        This would also replace externs,
          which was intended to be a temporary kluge when
            transitioning to a symbol-based system.
        This also has the additional benefit that parameters can
          represent inputs in shared systems for the sake of mocking
            and as an alternative to externs without having the
            downsides of a paramter---that
              it must be mapped as an input.
@end enumerate


@section Preprocessor
The preprocessor is primarily responsible for macro (template) expansion.

@enumerate
  @item Remove distinction between primitives and short-hand template
          expansion.
        Treat @emph{all} nodes as potential template expansions,
          and use the core library to mark certain templates as
          primitives.
        This will allow extending the language grammar naturally,
          and easily convert existing primitives to templates.@footnote{
            For example, @samp{lv:rate-each} is a good candidate for
              a template.
            Ideally, all XSLT-driven expansions should be converted
                into templates (the template system didn't exist back
                then).}
@end enumerate


@section Input/Return Map
@enumerate
  @item Generate normal packages and use templates rather than a
          specialized system;
            it has numerous kluges to work around inconsistencies with
              other parts of the system,
                and copies code from portions,
                particularly for preprocessing.
@end enumerate


@section Worksheet
@enumerate
  @item Implement a symbol table and treat worksheet packages as
          normal packages.
        As it stands today,
          symbols will not be retained by the linker if they are
          referenced by the worksheet because there is no symbol table;
            this puts the onus on the developer to manually place
              it in the return map,
                which is error-prone.
@end enumerate


@section Documentation
Programs are written in a literate style.

@enumerate
  @item Auto-generate LaTeX documentation using the existing @code{lvspec}
          package used to manually write specifications some time ago.
        Certain implementation details may need to be sorted out,
          like determining the section hierarchy.
@end enumerate