Mike Gerwitz
b2a996c1df
This is a rather small change for quite a bit of effort in researching what
was going wrong. It's at last seven rabbit holes deep, or maybe several
herd of yaks, depending on your choice of measure and the current conversion
rate.
The problem can be summarized fair succinctly:
`expand-sequence/expand-group` exists to prevent an expansion repass for
every single child element of the `expand-sequence`, which would be
quadratic. Basically, it restores the usual template expansion process for
that set of children.
But apparently `expand-group` was stripped on the first pass, which expanded
its children inline, which then meant that each of the children were subject
to their own individual passes, defeating the purpose of the optimization.
As is the nature of quadratic-time processes, that was not noticed until
inputs became especially large, and not only that, but were combined with
nested `expand-sequence`s.
I would say that this never worked the way that I intended it to, but I'm
not certain. I was working quite a bit with TeX at the time, so it's
possible that I modeled it after `\expandafter`. But that's not an
appropriate model for TAME.
TAMER will be removing expand-sequence entirely, since it will have enough
of an understanding of the source system to determine what requires
expansion and what requires ordering (e.g. for symbol table
iteration). I'll also be making changes to simplify the process by further
restricting what type of iteration can take place. But for the time being,
the change was necessary.
In our largest systems, this change cut off ~15m total of build time if run
serially (at `-j1`). After sorting two runtabs for comparison (e.g. `sort
-k4`), you can get the total like so:
$ paste <( sort -k4 runtab-a ) <( sort -k4 runtab-b )
| grep xmlo\$ \
| cut -f2,5,6 \
| awk '{ total += ($1 - $2) } END { print total / 1000 }'
Similarly, this Awk expression will give the time differences:
$ awk '{ print ($1 - $2)/1000, $5 }'
Further, the previous commit also introduced a `xmle-sym-cmp` tool to check
for differences between xmle symbol tables in an automated way, irrespective
of ordering (since there are many valid topological sorts). This revealed
that the change fixed a bug (likely because of the forced repass after
`expand-group` hoisting) that was causing symbol table introspection to fail
to discover symbols in certain cases, which in our case, was resulting in
the failure to generate a small number of aggregate classifications
correctly. The whole repass system is a concerning mess, but it's not worth
the effort to try to redo all of that when that work can be done in TAMER.
DEV-15069
|
||
|---|---|---|
| bin | ||
| build-aux | ||
| core | ||
| design/tpl | ||
| doc | ||
| progtest | ||
| rater | ||
| src | ||
| tamer | ||
| test | ||
| tools | ||
| .gitignore | ||
| .gitlab-ci.yml | ||
| .gitmodules | ||
| .rev-xmle | ||
| .rev-xmlo | ||
| COPYING | ||
| COPYING.FDL | ||
| HACKING | ||
| Makefile.am | ||
| README.md | ||
| RELEASES.md | ||
| VERSION.in | ||
| bootstrap | ||
| c1map.xsd | ||
| configure.ac | ||
| package-lock.json | ||
README.md
TAME
TAME is The Algebraic Metalanguage, a programming language and system of tools designed to aid in the development, understanding, and maintenance of systems performing numerous calculations on a complex graph of dependencies, conditions, and a large number of inputs.
This system was developed at Ryan Specialty Group (formerly LoVullo Associates) to handle the complexity of comparative insurance rating systems. It is a domain-specific language (DSL) that itself encourages, through the use of templates, the creation of sub-DSLs. TAME itself is at heart a calculator—processing only numerical input and output—driven by quantifiers as predicates. Calculations and quantifiers are written declaratively without concern for order of execution.
The system has powerful dependency resolution and data flow capabilities.
TAME consists of a macro processor (implementing a metalanguage), numerous compilers for various targets (JavaScript, HTML documentation and debugging environment, LaTeX, and others), linkers, and supporting tools. The input grammar is XML, and the majority of the project (including the macro processor, compilers, and linkers) is written in a combination of XSLT and Rust.
TAMER
Due to performance requirements, this project is currently being reimplemented in Rust. That project can be found in the tamer/ directory.
Documentation
Compiled documentation for the latest release is available via our GitLab mirror, which uses the same build pipeline as we do on our internal GitLab instance. Available formats are:
Getting Started
To get started, make sure Saxon version 9 or later is available and its path
set as SAXON_CP; that the path to hoxsl is set via HOXSL; and then run
the bootstrap script:
$ export SAXON_CP=/path/to/saxon9he.jar
$ export HOXSL=/path/to/hoxsl/root
$ ./boostrap
Running Test Cases
To run the test cases, invoke make check (or its alias, make test).
Testing Core Features
In order to run tests located at core/test/core/**, a supporting environment
is required. (e.g. mega rater). Inside a supporting rater, either check out a
submodule containing the core tests, or temporarily add them into the
submodule.
Build the core test suite summary page using:
$ make rater/core/test/core/suite.html
Visit the summary page in a web browser and click the Calculate Premium button. If all test cases pass, it will yield a value of $1.
Hacking
Information for TAME developers can be found in the file HACKING.
License
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.