Mike Gerwitz
e20076235e
This is an interesting one.
For some context: TAME uses `csvm` files to provide syntactic sugar for
large tables of values ("rate tables", as they're often called, since they
contain insurance rates and other data). This gets desugared into a `csv`
which in turn is compiled via `csv2xml` into a package. That package uses
the `_table-*_` templates to define a table, which is represented as a
matrix using `const/@values`.
Here's an example of a generated table in a package:
```
<t:create-table name="foo">
<t:table-rows data="
1,2,3;
4,5,6;" />
</t:create-table>
```
Some of the tables are quite large, generating tens of MiB of data in
`@data`. This in itself isn't a problem. But when Saxon parses the `@data`
attribute, it normalizes the whitespace, as mandated by the XML spec, and
removes the newlines. Therefore, when the template is expanded and the
`xmlo` file is produced, the template produced a `const/@values` with a huge
amount of data on one line.
Then, when another package imports that `xmlo` file via `<import
package="..." />`, which is done via `document()` in XSLT, Saxon takes a
long time to parse it. 60s on my machine for a ~20MiB line.
This problem does not exist for JS fragments; Saxon doesn't mind large text
nodes. So that is the approach that is taken here.
The template system doesn't have a way to output text yet, so this takes an
approach that minimizes changes as much as possible:
- `param-copy` will expand `with-param/@value` as a text node.
- `const/@values="-"` will cause TAME to use the child text node as the
value of `@values`.
- `_table-rows_` is modified to use the above two features.
The reason for using `@values="-"` is so that other parts of the compiler do
not have to be modified to recognize the new text convention, which is
otherwise awkward because newlines are text nodes. The `-` convention comes
from command line programs, which means "read from stdin", generally; this
is okay since `-` is never a valid matrix specification.
This must have been a problem for a very long time, but wasn't all that
noticeable until recent performance optimizations, since so many other things
around it were also slow.
DEV-15131
|
||
|---|---|---|
| bin | ||
| build-aux | ||
| core | ||
| design/tpl | ||
| doc | ||
| progtest | ||
| rater | ||
| src | ||
| tamer | ||
| test | ||
| tools | ||
| .gitignore | ||
| .gitlab-ci.yml | ||
| .gitmodules | ||
| .rev-xmle | ||
| .rev-xmlo | ||
| COPYING | ||
| COPYING.FDL | ||
| HACKING | ||
| Makefile.am | ||
| README.md | ||
| RELEASES.md | ||
| VERSION.in | ||
| bootstrap | ||
| c1map.xsd | ||
| configure.ac | ||
| package-lock.json | ||
README.md
TAME
TAME is The Algebraic Metalanguage, a programming language and system of tools designed to aid in the development, understanding, and maintenance of systems performing numerous calculations on a complex graph of dependencies, conditions, and a large number of inputs.
This system was developed at Ryan Specialty Group (formerly LoVullo Associates) to handle the complexity of comparative insurance rating systems. It is a domain-specific language (DSL) that itself encourages, through the use of templates, the creation of sub-DSLs. TAME itself is at heart a calculator—processing only numerical input and output—driven by quantifiers as predicates. Calculations and quantifiers are written declaratively without concern for order of execution.
The system has powerful dependency resolution and data flow capabilities.
TAME consists of a macro processor (implementing a metalanguage), numerous compilers for various targets (JavaScript, HTML documentation and debugging environment, LaTeX, and others), linkers, and supporting tools. The input grammar is XML, and the majority of the project (including the macro processor, compilers, and linkers) is written in a combination of XSLT and Rust.
TAMER
Due to performance requirements, this project is currently being reimplemented in Rust. That project can be found in the tamer/ directory.
Documentation
Compiled documentation for the latest release is available via our GitLab mirror, which uses the same build pipeline as we do on our internal GitLab instance. Available formats are:
Getting Started
To get started, make sure Saxon version 9 or later is available and its path
set as SAXON_CP; that the path to hoxsl is set via HOXSL; and then run
the bootstrap script:
$ export SAXON_CP=/path/to/saxon9he.jar
$ export HOXSL=/path/to/hoxsl/root
$ ./boostrap
Running Test Cases
To run the test cases, invoke make check (or its alias, make test).
Testing Core Features
In order to run tests located at core/test/core/**, a supporting environment
is required. (e.g. mega rater). Inside a supporting rater, either check out a
submodule containing the core tests, or temporarily add them into the
submodule.
Build the core test suite summary page using:
$ make rater/core/test/core/suite.html
Visit the summary page in a web browser and click the Calculate Premium button. If all test cases pass, it will yield a value of $1.
Hacking
Information for TAME developers can be found in the file HACKING.
License
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.