This represents the old cmatch system (which is in use today, but the
classification system has since been rewritten, though it has not yet been
merged). It was my attempt over a decade ago to reason about how this
system ought to work.
I think it's fair to say that this is absolute insanity and that the new
formulation is significantly better.
The intent originally was to try to keep developers to a reasonable name
length, but generated identifiers can easily exceed this, and we further do
not support namespacing.
This can be handled at a template level instead for enforcing naming
conventions.
This had gotten quite out of date from the actual rater.xsd, which existed
outside of this repository, that is used during our build process. That was
an unintended artifact from moving files around.
That file has been removed and symlinked to this one.
First thing to note: this belong in liza-proguic, not here. But it's here
right now, so for now I'm making the change. The relationship between TAME
and proguic is awkward and will hopefully be improved upon in the near
future.
As for this actual change: step-level fragments will be concatenated such
that the imports will appear at the step level rather than the root.
I did say it was _experimental_ guided TRO.
This waits to perform the actual argument reassignment until after
processing the expressions associated with the new arguments, since they
will otherwise be replaced when their original values are still needed.
This change simply prevents failure in such situations, (e.g. on invalidated
fields in Liza). We'll worry about proper errors and correctness, which
ought to be compile-time, in TAMER.
The MathJax CDN stopped working in April 2017. I updated it to the
recommended CDN with the last version from April 2017 to ensure it works
like it used to work before the CDN stopped.
I added the checksum to ensure the content of the script.
This problem manifested when the name of the attempted classification is the
same name as another object. For example, if we have `t:match-class
name="foo"`, and `foo` is a param instead of a class, then `@yields` will
fail, and it'd fall back to matching on the param.
This is absolutely not what we want.
The error message in this context is ugly, but it does work.
Example:
!!! Unknown match @on (/lv:package/lv:classify/match): `error: unable to
determine @yields for class `scheduled_ai' (has the class been imported?)'
is unknown for classification --vis-scheduled-ai-type
This implements TCO in the XSLT compiler by requiring a human to manually
indicate when a recursive call is in tail position. This was somewhat
urgently needed to resolve stack exhaustion on large rate tables.
TAMER will do this properly by determining itself whether a call is in tail
position. Until then, this will serve as a test for this type of feature.
Replacing the existing macros with templates will allow us to now have
to deal with macros in the new compiler.
The `indexNameType` pattern needed to change to allow for variables. I
also had to remove the prefix for the `gentle-no` option of `rate`.
Create a "yield" and add backwards compatibility for the macro of the
same name. This is one of 2 macros that need to be replaced so we do not
have to worry about them with the new compiler.
All systems should be using the provided Makefile, so this shouldn't be
invoked anymore. The new linker is still considered a proof-of-concept, but
bugs have been encountered in the old one that are not worth investing the
time into fixing.
The new linker has been used in production for nearly a couple months and is
functioning properly.
This ordering will simplify streaming processing of xmlo files in
TAMER. Specifically, we know that symbols will have been declared by the
time dependencies are added to the graph (and so we should only be creating
edges to existing nodes); and we can halt reading as soon as the closing
fragments tag is encountered, avoiding parsing the entirety of these massive
XML files.
On one particularly large program, this cuts time down from ~0.333s to
~0.300 in the POC linker.
This not only reduces file size, but also has a significant performance
benefit for the UI, which is almost entirely classifications. A run for one
of our systems was reduced from 1m30s to 11s from this change.
This was used to provide additional information on the stack for debugging
the compiled code. Since this is very rarely needed, and is only needed by
someone debugging the compiler, it can be manually enabled if desired.
This also wraps it so that it'll be stripped if it is included.
The `<t:match-class-code-lookup />` matches were not showing in the
summary pages. I loosened the selector so it is able to find the matches
when it generates the summary pages.
If an `lvm:if` is immediately followed by another 'lvm:if`, both should
be used to create the conditional. The existing code wouild only "select
the nearest condition".
The LOB being passed into the function was being ignored and instead it
was pulling it from the contract object. With Package, this caused all 3
LOB to be "COMMPKGE" rather than the correct LOB being processed at the
time.
Going forward, one cannot `map` or `pass` to "line_code" as it will be
considered a reserved word.
Co-Authored-By: Jim Grundner <james.grundner@rtspecialty.com>
This is left over from f2db9f1268, in which I
should have cleaned all of this up. One of our developers was hitting the
removed warning, which isn't necessary since the concept of a separate
"classifier" is no longer a thing after the aforementioned commit.
* rater/rater.xsd (no-extclass, no-extclass-keeps): Remove.
* src/current/rater.xsd: Likewise. (I really need to deduplicate these.)
* src/current/compiler/js.xsl (compiler:entry-rater): Remove inaccurate
comment (genclasses is used for other things).
* src/current/include/depgen.xsl (preproc:depgen-match): Remove error
checking for pulling in non-external classes (this is the error that the
developer hit that is no longer needed).
* src/current/include/preproc/eligclass.xsl (preproc:sym): Remove
`@extclass' predicate. Remove portion of comment.
* src/current/include/preproc/expand.xsl: Remove ancient footnote that
even references an old internal rater!
* src/current/include/preproc/macros.xsl (preproc:class-groupgen): Remove
external propagation.
* src/current/include/preproc/symtable.xsl (preproc:symimport): Remove
extclass checks and propagation.
(preproc:symtable)[lv:rate]: Remove external propagation.
[lv:classify]: Likewise.
* src/current/include/preproc/template.xsl (preproc:inline-apply): Remove
external sym metadata support.
These exist because TAME is nondeterministic, so all state must be passed
into it. But it's inconvenient to have users have to manually fill in
dates, so we derive them from the environment unless they are set.
* src/current/scripts/entry-form.js (fillTimeValues): New function.
(rater): Use it.
This further improves performance of the symbol table processing. The next
step will be to address how symbols are handled on a more intimate level,
since it's a huge mess atm. But I'll save that for later, after the
low-hanging fruit has been resolved.
* src/current/include/preproc/symtable.xsl (preproc:sym-discover): Use
`for-each-group' in place of `preceding-sibling'. Aggressive use of
maps for geneating the `dedup' sequence, which is a mess.
(preproc:symtable-process-symbols): Additional maps to avoid
preceding-sibling and following-sibling selectors (O(n²)=>O(n)).
Same concept as previous commits: rather than iterating over the symbol
table and scanning the tree for the matching node, iterate over the document
and look up from a symbol map: O(n²) => O(n).
This gives a respectable performance boost to compilation of certain
packages (best improving packages with many classifications or rate blocks).
* src/current/compiler/fragments.xsl (@xmlns:xs, @xmlns:map): New namespace
declarations.
(preproc:compile-fragments): Generate `preproc:fragment' nodes and match
on document rather than symbols.
[lv:package]: Generate map and tunnel it.
* src/current/compiler/js.xsl (compile)[lv:classify, lv:match]: Use
symtable-map.
(compile-class-condition)[lv:rate]: Likewise.
(compile-cmatch)[lv:rate]: Likewise.
This uses the same map strategy (and same duplicate code) as previous
commits, but this one generates a map for two separate tables.
There is more room for improvement, but this cuts down on the time a
lot. Also keep in mind that this is performed multiple times (once per
pass), so it's still worth revisiting. Performance is still very poor for
very large (many thousands of symbols) symbol tables.
The next slowest part appears to be the fragment compilation. I'm nearing
the end of the low-low-hanging fruit for maps. The /common/gl package
mentioned in previous commits that previously took over a minute to compile
now compiles in 20s as of this commit on equivalent hardware.
* src/current/include/preproc/symtable.xsl (@xmlns:map): New namespace
declaration.
(preproc:symtable-process-symbols): Create map for `cursym' and
`extresults'. Use it. Remove unused `dup'. Output message when
done (another is output slightly later on in the process).
This is the first step to improving the map. Note that this duplicates the
symbol table generation code that's used in a few other places
now---that'll be cleaned up in future commits once I have a better idea of
all the places this will be used and try to move it to a higher level.
* src/current/compiler/validate.xsl (@xmlns:xs, @xmlns:map): New namespace
definitions.
(lvv:validate)[lv:package]: Generate symbol table map. Tunnel to
templates.
[c:apply[@name], lv:classify[@as]//lv:match, lv:match[@value]]
[c:*[@name or @of], c:apply/c:arg[@name], lv:rate/lv:class]: Use it.
The existing code was not only complex (because of XSLT 1), but mostly
unnecessary. We don't need to consult remote symbol tables at all anymore.
This shaves off an additional few seconds on large packages.
* src/current/include/preproc/package.xsl (preproc:resolv-syms)[preproc:sym]:
Only consult local symbol table. Simplify max dimension calculation.
This is a first step (low-hanging-fruit kinda thing) for improving the
performance of symbol resolution, where the compiler has to figure out the
dimensions of a symbol by first resolving its dependencies,
recursively. This is approximately an O(n³) polynomial-time algorithm _per
recursive step_. Yikes.
This is traditionally where dynamic programming methods would be used, but
that's considerably more difficult in a immutable languages like XSLT, so
I'll do my best without. (Saxon does offer some support for mutability, but
I'd prefer to avoid it if possible.)
This first change improves performance 30--40%. For example, on two large
packages we have, build times drop from 55s to 35s and from 1m42s to 1m13s
respectively.
Good start, but much more to be done!
* src/current/include/preproc/package.xsl (preproc:resolv-syms)[lv:package]:
Compute maps for preproc:symtable and preproc:sym-deps at each recursive
step. Pass along via tunneling.
(preproc:resolv-syms)[preproc:sym]: Use them.
DEV-4354
This only saves 1--2s on a 30s run, but I want to move into this direction,
so it'll simplify future refactoring if I just add it. Small changes like
these will accumulate, too.
* src/current/compiler/linker.xsl (l:orig-package, l:root-symtable-map): New
variables.
(l:resov-extern): Use it.
This now uses year ranges, which I'll update annually.
This also renames "R-T Specialty" to "Ryan Specialty Group". The latter is
the parent company of the former. I was originally employed under the
former when LoVullo Associates was purchased, by I now work for the parent
company.
The previous commit made dependency lists optional for certain symbols. The
Summary Page needs to be updated to permit such a thing.
The whole Summary Page needs aggressive refactoring, though, so this doesn't
bother checking for `no-deps' to see if this is a bad thing.
* src/current/summary.xsl (typeset-final)[preproc:sym-ref]: Permit missing
symbol dependencies.
(lv:param|lv:const|lv:item): Likewise.
This is a significant performance improvement for dependency
generation (which is responsible for building the dependency graph for a
package).
The previous algorithm ran in O(n²) time: it would iterate over the given
symbol table, and for _each_ symbol, do a linear scan of the entire document
to search for the corresponding source block. This resulted in explosive
depgen time for larger packages.
This makes the algorithm run in O(n) by:
- Using an XSLT 3 map for the symbol table for O(1) lookups; and
- Iterating over the _document_ a single time rather than the symbol
table, referencing the symbol table as needed (in O(1) time).
There are other parts of the system that can benefit from these same
improvements. This is important, since we need to be able to handle many
thousands of symbols efficiently.
* src/current/compiler/linker.xsl (l:depgen-sym): Recognize smybol `no-deps'
property, permitting missing dependencies. This allows us to avoid
creating nonsense nodes just to satisfy the linker, while still allowing
the linker to perform essential checks to defend against compiler bugs.
* src/current/compiler/map.xsl (lvmc:stub-symtable): Set @no-deps on
`___head' and `___tail' symbols.
(lvmc:mapsym): Set `no-deps' as appropriate on map symbols.
(preproc:depgen)[lvm:map[@from]]: Generate `preproc:sym-dep' node, which
is now expected by the depgen process.
(preproc:depgen)[lvm:map[*]]: Likewise.
(preproc:depgen)[*[@lvmc:type='retmap']//lvmm:map[@from]]: Remove
unnecessary template.
(preproc:symtable)[lvm:map[@value]]: Pass `no-deps' to `lvmc:mapsym'.
* src/current/include/depgen.xsl (preproc:depgen)[preproc:symtable]: Create
and use XSLT 3 map in place of `preproc:symtable' tree. This allows for
constant-time lookups. Provide to templates via tunnelling. Use it in
place of exiting tree references. Process source tree rather than
iterating over symbol table.
(preproc:depgen)[lv:rate, c:sum[@generates], c:product[@generates],
lv:classify, lv:function/lv:param, lv:function, lv:typedef]: Produce
`preproc:sym-dep' nodes (which was previously done while iterating
over the symbol table).
(preproc:depgen)[preproc:sym]: Remove all such processing, since we no
longer iterate over the symbol table.
(preproc:depgen)[c:value-of]: Use symtable map.
(preproc:depgen-match): Likewise.
(preproc:depgen)[lv:union]: Modify to handle changes to lv:typedef
template.
(preproc:depgen)[text()]: Remove and replace with `node()'.
* src/current/include/preproc/package.xsl (preproc:resolv-syms): Remove
logging of symbol resolution. This has a slight performace impact since
there is a lot of output.
* src/current/include/preproc/symtable.xsl
(lv:function/lv:param, c:let/c:Values/c:value): Set `no-deps'.
* src/symtable/symbols.xsl: Add documentation of `no-deps'.
(preproc:symtable)[lv:meta]: Set `no-deps'.
These provide a more pleasent abstraction than having to reference CMP_OP_*
constants.
* core/test/core/vector/interpolate.xml: {t:when=>t:where-eq}.
* core/test/core/vector/table.xml: Likewise, but using the other variants
where appropriate given the value of `@op'.
* core/vector/interpolate.xml: Likewise.
* core/vector/table.xml (_when_, _where_): Rename former to latter and
provide deprecation warning.
(_when-lt_, _when-lte_, _when-gt_, _when-gte_): Add abstractions.
* src/current/rater.xsd: Permit template variable as tenplate name.
It's going to be like TeX before you know it... ._.
* src/current/include/preproc/package.xsl (preproc:tpl-check)
[lv:template|lv:const|lv:typedef|lv:param-copy]: Add lv:param-copy.
* src/current/include/preproc/template.xsl (preproc:apply-template)
[lv:expand-barrier, lv:skip-child-expansion]: New expansion control
structures.
This is a much more useful description if present.
* src/current/include/preproc/macros.xsl (preproc:macros)[c:value-of...]:
Default generated constant description to @label.
The term "set" is all wrong---it is actally intended to be a vector, and can
absolutely have duplicate elements (and often does).
* src/current/calc.xsd (vector): Add, recommending in place of `set'.
* src/current/compiler/js-calc.xsl (compile-calc)[c:set|c:vector]:
Add `c:vector' and provide deprecation notice for `c:set'.
* src/current/include/calc-display.xsl (c:set|c:vector): Likewise.
A better option is to pre-process all inputs, but I need a quick
fix to my stupidity. 0||""==="".
* src/current/compiler/map.xsl (lvmc:compile)[lvm:map//lvm:from[*]]: Correct oval default.
I need to revert this for now because it breaks YAML test cases. The proper
fix is a more expressive type system with dependent types that would allow
it to know the proper number of indexes to initialize relative to other
inputs. I wanted to implement this anyway to help catch iteration-related
bugs.
I'm tabling this for now, though, since I have other things that I need to
work on.
This reverts commit 4406cbe553.
I want this manual to be useful both to developers and users of TAME,
so this distinction needs to be made clear.
* doc/tame.texi (Preprocessor): chapter=>appendix.
* src/graph.texi: Top to appendix and raise subsections.
* src/symtable.texi: Top to appendix.
This is an assumption that's existed since the Summary Page was first
devised---that all vectors have at least one value. This is because the
bucket (originating from Liza) always has at least one value in its vectors.
Of course, we still have a problem in that the Summary Page initializes
everything to have a single value by default, and that's still the
case. But this will at least allow for things _outside_ the Summary Page to
provide an empty array. I'll have to address the Summary Page separately,
and that's going to be difficult, since we don't really want to change the
behavior across the board.
* src/current/compiler/js.xsl (set_defaults): Default max index to 0 if
`length' is unavailable, rather than 1.
The previous length check existed as a really bad array check (before
Array.isArray was a thing). This has been broken since Nov 2012.
The problem manifests itself when you want an empty array. We then have:
[] => [[]] => [DEFAULT_VALUE]
* src/current/compiler/map.xsl (lvmc:compile)[lvm:map//lvm:from[*]]: Use
`Array.isArray' in place of length check.
The difference is described here:
http://www.saxonica.com/html/documentation/using-xsl/embedding/
And s9api here:
http://www.saxonica.com/html/documentation/using-xsl/embedding/s9api-transformation.html
* Makefile.am (DSLC_CLASSPATH): Export for submakes.
* configure.ac (DSLC_CLASSPATH): Prefix with SAXON_CP.
* rater/rater.xsd (classNameType): Increase length 50=>75 (generated
identifiers can now exceed that, it seems).
* src/current/rater.xsd: Likewise. These files need to be combined.
* src/current/src/Makefile (CLASSPATH): Set to DSLC_CLASSPATH.
* src/current/src/com/lovullo/dslc/DslCompiler.java: Update imports.
(DslCompiler)[_DslCompiler]: New members _processor and
_xsltCompiler. Convert to s9api.
This has been broken for years. I don't object to fixing it, it's just that
I have better things to do right now and we've gotten complaints about it;
no use in keeping around something that's broken if there's no desire to fix
it. Workaround: refresh the page.
This does keep around the reset logic because it is actually used in other
places.
* src/current/include/entry-form.xsl (entry-form)[lv:package]: Remove reset
button.
* src/current/include/entry-form.js (clearTestCases): Remove broken function
call `Prior.setPriorMessage(null)'.
It wasn't until recently that I realized that the default browser font was
being used, since I have mine customized.
* src/current/summary.css (body)[font-family]: Sans-serif font stack.
* src/current/compiler/map.xsl:
(lvmc:gen-input-default): Add argument.
[dim]: New param, defaulting to `$sym/@dim'.
(lvmc:compile)[lvm:map//lvm:from[*]]: Provide appropriate dimension value
to `set_defaults'. Provide compile-time error if nesting of `from'
nodes exceeds what is appropriate for the symbol dimensions.
This fixes a number of obnoxious miscellaneous issues, summarized below.
* src/current/src/com/lovullo/dslc/DslCompiler.java (DslCompiler)[compile]:
Output termination line (DONE) on missing destination path
error. Always output exception message before termination
line (otherwise it won't output to the user). Output termination line
and remove destination file for XSD failure.
This was a bit of a nasty one. Fortunately, this was only used as a
validation, so the code that the compiler produced was still correct.
The problem was that a version of Saxon sometime between 9.5 and 9.8 added
an optimization to eliminate conditionals with no body. Consequently, the
kluge to force the variable to be evaluated was optimized away,
`lvmc:get-symbol' was never called, and no error was ever produced.
This would be best refactored, but that's not something I have time to take
up at the moment priority-wise. This should be future-proof since this
would never be a noop.
* src/current/compiler/map.xsl (lvmc:compile)[lvm:map//lvm:from[*]]: Force
evaluation of `$sym' by ensuring that the condition will not be a noop.
* src/current/c1map.xsl (lvm:c1map): Copy `@namespace' to generated
`lvmp:root'.
* src/current/c1map/render.xsl (lvmp:render)[lvmp:root]: Output
`@namespace' rather than using hardcoded string and dynamic program.
This is a long-standing bug, apparently. The location of this code makes it
difficult to test directly (that is in dire need of correcting), but
fortunately we have a number of tests in systems that use TAME that
indirectly test this.
The problem manifested when a matrix was already in the store, but then a
scalar or vector predicate was considered. Without making the branch that
was modified here, it modified store such that it would always yield a
vector.
* src/current/compiler/js.xsl (anyValue): Consider store dimension when
recursing.
This frees us from requiring a rater/ directory in the working
directory. However, it is important that we continue using it if it
exists, since there are additional things that haven't yet been moved
into the tame repo.
* bin/dslc.in: Provide path to rater/ directory.
* src/current/src/com/lovullo/dslc/DslCompiler.java: Use provided rater/ path.
* build-aux/gen-make: Do not add ".xmlo" suffix for deps with a
trailing `$'.
* src/current/pkg-dep.xsl (lvm:program|lvm:return-map): Append ".xml$" to
dep for map/@src (new dep).
This is a major step toward normalcy---removing the kluge of a build process
that was causing so many issues. Rather than echoing all operations to a
queue file before passing it off to dslc, the new build scripts in `bin/'
are used to invoke tame normally, as needed. This solves all of the current
issues with things not rebuilding when they should. And, as a bonus, tab
completion on targets works.
Sorry this took so long. There wasn't much motivation until we hired so
many people that are suffering from this.
This does a few major things, along with some miscellaneous others:
- Invoke bin/tame directly;
- Merge Makefile.2.in into Makefile.am; and
- Fix up some targets.
* build-aux/Makefile.2.in: Delete file. Mostly merged with Makefile.am.
* build-aux/Makefile.am: Add a bunch of new targets and definitions from
Makefile.2.in. Modify all that previously used .cqueue to now invoke
`$(TAME)' directly. Remove miscellaneous targets for trying to proxy
targets to Makefile.2.
(saneout, _go): Remove definitions.
(.NOTPARALLEL): Add to prevent parallel builds.
(ui/program.expanded.xml)[.version.xml]: Remove dependency for now.
(clean): Also clean generated PHP files. Follow symlinks to clean core.
This is still incomplete (does not clean all rate table stuff).
(suppliers.mk)[xmlo_cmd]: Remove. See `gen-make' and `gen-c1make'.
(lvroot)[summary-html]: New dependency.
(kill-tamed, tamed-die): New targets (former alias of latter) to kill
tamed.
* build-aux/gen-c1make: Generate `$(TAME)' invocation.
* build-aux/gen-make: Likewise. Remove `xmlo_cmd' output. Ignore recursive
`tame' symlink (this can be removed once we clean `rater/' up.
* build-aux/m4/calcdsl.m4 (TAME): Update description to reflect that it
should now be the path to `bin/tame'. Adjust `AC_CHECK_FILE' lines
accordingly.
(tame_needed_ver): Remove. We have been in the same repo as TAME itself
for quite some time. Remove associated code.
(AC_CONFIG_FILES): Remove `Makefile.2'.
* src/current/src/com/lovullo/dslc/DslCompiler.java (_DslCompiler)[compile]:
Perform validation prefore `compile' command rather than a separate
`validate' step. Remove `rm'.
[compileSrc]: Stop echoing command. This was only necessary because of
the previous Makefile klugery; now Make echoes on its own correctly.
These scripts allow the TAME compiler stack to be invoked naturally, rather
than requiring the use of a Makefile today. This will not only allow users
to more easily invoke the compiler, but will also allow us to invoke TAME
naturally from Makefile and remove the klugery that has existed for so
long.
This users a server/client architecture in order to mitigate the startup
cost of the JVM. More documentation will follow.
Note that there are a bunch of symlinks in rater/---this is a transition
step to allow the build to continue working as it did before, which relies
on a directory structure that exists outside of this repository. This will
be cleaned up in the future.
* .gitignore (bin/dslc): Add ignore for generated file.
* bin/dslc.in: New script to encapsulate Java invocation.
* bin/tame: New script (client).
* bin/tamed: New script (server).
* configure.ac (JAVA_OPTS, DSLC_CLASSPATH, AUTOGENERATED): New variables for
dslc.in. Output bin/dslc.
* rater/README.md: Note that this symlink mess is temporary.
* rater/c1map: New symlink for dslc assumptions.
* rater/c1map.xsl: Likewise.
* rater/calc.xsd: Likewise.
* rater/compile.xsl: Likewise.
* rater/compiler: Likewise.
* rater/dot.xsl: Likewise.
* rater/include: Likewise.
* rater/link.xsl: Likewise.
* rater/standalone.xsl: Likewise.
* rater/summary.xsl: Likewise.
* rater/tame: Likewise (warning: circular symlink).
* src/current/src/com/lovullo/dslc/DslCompiler.java (_DslCompiler)[compile]:
Output `DONE' lines.
This will now automatically build on recursive target `all'.
* Makefile.am (SUBDIRS): Add `src/current/src'.
* src/current/src/Makefile: (.PHONY): Add `all'.
(all): New target. Alias to `dslc'.
Most LaTeX distributions prohibit loading files from parent
directories. This gets around that for PDF builds.
* doc/src: Add symlink.
* doc/tame.texi, src/symtrable.xsl: Use it.
This was throwing a warning in non-ancient versions of Saxon. It does not
need to be there, nor should it be, nor do I know why it was put there.
* src/current/include/preproc/template.xsl (eseq:is-expandable): Remove
@override.
This has a significant performance impact: processing time is cut in about
half and memory usage is reduced by more than 50%. For example, a
package that previously took 30s and 2.1GiB of memory to link now takes
14s and less than 900MiB of memory.
I had tried to perform this optimization a couple years ago but was
thwarted (I think) by the classifier markers. The previous commit did away
with those. I'm encouraged by the gains from the low-hanging fruit.
* src/current/compiler/linker.xsl
(l:process-empty, l:stack-empty): Convert from l:pstack and
l:sym-stack (respectively) to empty preproc:sym sequences.
(l:depgen-process-sym)[preproc:sym]: Append to sequence rather than
outputting new l:sym-stack tree.
Update all annotations and uses accordingly.
This is something that I thought would be useful back in the day when TAME
was in its infancy, but it is not important. Rather than having the linker
spend time trying to figure out what symbols belong in the classifier---and
rather than keeping that complexity around---this simplifies things by
making the existing `classify' method simply perform _all_ calculations, and
then yield only the classification portion of the result.
This isn't a problem in practice because, if we only desire the use of a
classifier, then we create a "supplier" that only uses classifications and
has no other dependencies. The end result is, as far as we care, the same.
* src/current/compiler/js.xsl (compiler:entry-rater)[lv:package]: Initialize
`classes' rather than invoking classifier
(compiler:entry-classifier)[lv:package]: Invoke all calculations and
return only classes to provide equivalent behavior.
(compiler:exit-classifier): Post-process classifications from calculation
results, iterating through classmap.
(compiler:classifier-yields-map)[lv:package]: Output all classifications
that are not generated. This differs slightly from the original
implementation in that it includes all non-generated classes rather than
just classes that have a non-generated `@yields'; this distinction is
important since `compiler:exit-classifier' is now using it to produce a
classification result set that doesn't contain all the generated
stuff (since it didn't before, and shouldn't now).
* src/current/compiler/linker.xsl: Update copyright year.
(l:resolv-deps)[preproc:sym[@l:mark-inclass]]: Remove template.
(l:resolv-deps)[preproc:sym...@l:mark-inclass...]: Remove template.
(l:depgen-sym): Set type of result to `element(preproc:sym)', since
`l:mark-inclass' is no longer produced.
[inclass, needs-class-mark]: Remove variables and all instances where
they are used.
(l:dep-aug)[inclass]: Remove param. Stop producing `@inclass' attribute.
(l:link-classifier)[lv:package]: Do not process any dependencies. This
can be removed entirely in the future since it now only produces static
code, which we can perhaps combine with a different block.
(l:link-rater)[lv:package]: Remove mention of `inclass' for dependencies;
all dependencies will now be compiled into this block.
This includes a SHA256 implementation which is _not_ intended for secure
cryptographic operations; see src/js/sha256.js header for more information.
* src/current/compiler/js.xsl (compiler:static): Echo src/js/sha256.js.
[map_method_uppercase, map_method_hash]: New functions.
* src/current/link.xsl: Include dslc-base.xsl.
* src/js/sha256.js: New file.
* src/current/compiler/map.xsl
(lvmc:get-method-func, lvmc:value-ref, lvmc:transformation-wrap): New
functions, partyl extracted from below.
(lvmc:compile)[lvm:map//lvm:from]: Use `lvmc:value-ref'.
[lvm:map//lvm:from/lvm:translate]: Add `[@key]' to match.
[lvm:map//lvm:transform]: New match. Ignore node entirely.
(lvmc:concat-compile): Propagate symtable to `lvmc:compile'.
* src/current/include/dslc-base.xsl (__path-root): New param.
* src/current/src/com/lovullo/dslc/DslCompiler.java
(DslCompiler)[compile]: Resolve TAME root path.
[_transform]: Set it.
DEV-3115
This is all really confusing because this doesn't use the same import
specification as packages; maps got stuck in a partial transition. So,
let's provide some helpful errors rather than silently failing.
* src/current/compiler/map.xsl (preproc:symtable)[lvm:import]:
Error if missing `@path'. Provide more information if `@package' was
provided to help clarify.
We need to cut down on symbol imports as much as possible; the whole system
starts dragging if we are importing thousands of symbols into a single
package.
* src/current/include/preproc/symtable.xsl (preproc:symtable)[lv:rate,c:*]: Mark
as local if `@preproc:generated`.
* src/current/include/preproc/template.xsl (preproc:macros)[lv:inline-template]:
Mark generated templates as such.
* src/symtable/symbols.xsl (preproc:symtable)[lv:template]: Mark as local if
`@preproc:generated'.
* src/symtable.xsl (symtable:find-duplicates):
Update description. Find duplicates in same package.
* test/symtable.xspec: Update test case accordingly.
* src/current/compiler/js.xsl (compiler:js-number): New function to
remove leading zeroes.
(compile)[lv:const]: Use it.
* src/current/compiler/js-calc.xsl (compile-calc)[c:const]: Use it.
* src/current/compiler/js.xsl (compile-class-condtion)[lv:rate]: Do not
consider @no's in predicate generation when `@preproc:gentle-no' is set.
* src/current/include/preproc/macros.xsl (preproc:macros)[lv:rate-each]: Set
`@preproc:gentle-no' on generated `lv:rate', since the generator handles
`@no' itself.
* src/current/include/preproc/template.xsl
(preproc:gen-param-value)[lv:param-sym-value]: Suppress warning for
missing symbol and yield empty string if `@ignore-missing='true'`.
This ensures that they are compiled into the `consts' object.
* src/current/include/depgen.xsl (preproc:depgen)[lv:typedef]: Include
`lv:enum/lv:item/@name' as dependencies.
The problem with this implementation was that, any time a generator had an
associated generated @yields (which is common), it wouldn't be included in
the summary page.
We can address this in the future. It's not necessarily that it was
incorrect; it's just how the system made use of it.
* src/current/include/preproc/symtable.xsl (preproc:symtable)[lv:rate]:
Do not mark @preproc:yields-generated symbols as @preproc:generated.
Templates can expand into unexpected places, so sometimes warnings are
inappropriately issued.
* src/current/include/depgen.xsl (preproc:depgen)[lv:template]: Ignore.
[lv:template/lv:param]: Remove (now unnecessary with above).
In order for the cmatch algorithm to work properly, predicates must be
re-ordered on @dim descending.
* src/current/compiler/js.xsl (compile)[lv:classify]: Order all different
dimensions, not just scalars.
any/all blocks within classifications are extracted into their own
classifications. When they have sibling nodes, the extracted
classifications did not have their templates expanded. Ouch.
This is a situation that should never happen (I haven't yet determined
whether or not we should support this type of thing in the future), but when
it does, do something intelligent.
* src/current/include/preproc/symtable.xsl (preproc:symtable)[c:*[@generates]]:
Consider only first ancestor lv:rate.
For example, with template-generating templates, if a reference is unknown
and therefore determined to be blank, an attribute might be completely
removed while the template is being generated. (See the /when package in
core for an example.) That is not good.
* src/current/include/preproc/template
(preproc:expand-template): Add `tpl' tunneling param to
preproc:apply-template application.
(preproc:apply-template)[@*]: Retain param reference if applying template
does not define it.
This is important for template-generating templates.
* src/current/include/preproc/template.xsl (lv:if,lv:unlesS):
Retain conditional nodes verbatim when referenced param is unknown.
* src/current/include/preproc/expand.xsl
(lv:classify): Set `@preproc:yields-generated' if `@yields' was not
previously defined.
* src/current/include/preproc/macros.xsl:
(lv:rate-each): Set `@preproc:yields-generated` if `@yields' was not
previously defined.
* src/current/include/preproc/symtable.xsl:
(lv:rate, lv:classify): Set `@preproc:generated` on symbol if
`@preproc:yields-generated'.
Not time to address styling this at the moment. This code was previously
not reached until recent fixes. And I forgot that this was even
permitted (such complex calculations within matches)!
* src/current/summary.xsl (match-desc): Do not terminate on unknown
match-desc elements.
Austin Schaffer
Mike Gerwitz