Commit Graph

114 Commits (4ebe81bcb6d3445d4cd9e53d8accc1ad816c002a)

Author SHA1 Message Date
Mike Gerwitz 4ebe81bcb6 Summary page changes to aid in testing 2017-12-11 15:29:16 -05:00
Mike Gerwitz 5e5458dd3b summary: Add param text case dfn below input
The idea is to provide some guidance with how YAML test cases are supposed
to appear.

This just adds to the massive cluster that is the `entry-form.js'.

* src/current/include/entry-form.xsl (entry-form)[preproc:sym]: Add
    `entry-testcase-dfn' div.
* src/current/scripts/entry-form.js: Invoke `updateParamTestcaseDfn' in
    various event listeners.
  (populateBucket): Invoke `updateParamTestcaseDfn'.
  (updateParamTestcaseDfn, getParamTestcaseDfnElement): New functions.
* src/current/summary.css: Style `.entry-testcase-dfn'.
2017-12-11 15:25:31 -05:00
Mike Gerwitz f79deb725e summary: Add "percent*" to VOIs
* src/current/scripts/entry-form.js (updateVois): Add "percent"-prefixed
    values to VOI list.
2017-12-11 14:03:03 -05:00
Mike Gerwitz eb01129e24 summary: Filter internal classes from class overview
* src/current/scripts/entry-form.js (updateSummaryClasses): Hide
    "-"-prefixed classes from Classification Overview.
2017-12-11 14:03:03 -05:00
Mike Gerwitz d612c593b2 summary: Add param names and links aside desc in entry form
This existed in the old summary pages.  Since we'll be having certain people
reference ids, they need to be easily visible from somewhere.

* src/current/include/entry-form.xsl (preproc:sym)[entry-form]: Add param
    name and link.
* src/current/summary.css: Style it.
2017-12-11 14:03:02 -05:00
Mike Gerwitz 3970659a81 summary.css: Fix header width in FF
Long-standing bug.  Didn't often see it because FF performance was
relatively poor for the Summary Pages compared to Chromium until recently,
so I didn't often load the page in it (despite it being my primary browser).

* src/current/summary.css (legend): width:auto.
2017-12-11 11:38:23 -05:00
Mike Gerwitz 838ddb6e32 summary: Correct rendering of nested c:let expressions
If nested c:let expressions contained values of the same name, they would
all be rendered, rather than just the one intended.

* src/current/include/display.xsl (do-gen-let-list): Pass let symbol to
    `_gen-let-list-item'.
  (_gen-let-list-item): Render only values part of context let expression.
    [letsym]: New parameter.
2017-12-11 10:50:51 -05:00
Mike Gerwitz e78c3cbe98 symtable: Add @lparent to let lparam symbols
* src/current/include/preproc/symtable.xsl (preproc:symtable)[c:let]:
    Add @lparam to symbol.
2017-12-11 10:50:49 -05:00
Mike Gerwitz a38327b505 expand: Continue macro expansion after @dim conversion
This was recently introduced and was noticed because c:let/@name was not
being generated under certain circumstances.

* src/current/include/preproc/expand.xsl (preproc:expand): Continue applying
    templates after @dim expansion (rather than copying child nodes).
2017-12-11 10:42:52 -05:00
Mike Gerwitz f38e6c896b Add template barriers for metadata
This (hopefully) prevents metadata from leaking outside of template
expansions where they are not wanted.

* src/current/include/preproc/macros.xsl (preproc:macropass): Strip
    cruft (new barrier nodes) on final pass.
* src/current/include/preproc/template.xsl (preproc:expand-template):
    Enclose template expansion in preproc:tpl-barrier (which is stripped on
    the final pass).
  (preproc:apply-template)[lv:param-meta]: Document template.  "Hoist"
    node (set attribute) if not contained within lv:param-copy.
  (preproc:gen-param-value)[lv:param-inherit]: Consider hoisted nodes behind
    barrier.
2017-12-08 14:48:59 -05:00
Mike Gerwitz 75aa0fab7e Summary Page: Render template expansion points
This renders template expansions as part of the calculation breakdown, which
goes a long way to understanding what the system is doing.

This further updates the hover over the breakdown to color-code based on
depth, making it easier to observe the nesting of
expressions.  Sub-expressions that are toplevel results of template
expansions (nodes marked with templates) have a thicker border.

* src/current/summary.css: Add template id styling, breakdown colors, and
    misc. styling.
* src/current/summary.xsl (ultra-breakdown-set): Enclose label in
    span.  Output template id, if applicable.  Mark fieldset if templated.
2017-12-08 14:48:57 -05:00
Mike Gerwitz 2f8ce0f234 Mark toplevel nodes affected by template expansion
This allows for debugging expansions, which are otherwise confusing and
difficult.  This will be valuable information for the Summary Pages as
well (see future commit).

* src/current/include/preproc/template.xsl (preproc:expand-template): Mark
    toplevel nodes after expansion.
  (preproc:mark-tpl-expansion): Add templates.
2017-12-08 13:53:12 -05:00
Mike Gerwitz 65890f7156 Clean up unused JS compiler code
Cruft left around from the symbol table refactoring long ago.

* src/current/compiler/js.xsl
  (compile)[preproc:rate]: Remove template.
    [preproc:class]: Remove template.
  (compile-rates)[lv:package]: Remove template.
2017-12-08 13:51:43 -05:00
Mike Gerwitz c5e67861c9 depgen: Do not perform dependency generation within templates
This makes me slightly uncomfortable because I haven't researched why
exactly this was not a problem before.  We encountered this issue using an
inline template iterating over the symbol table yielding a `c:value-of',
which is admittedly something that we haven't done before.

Hopefully it's an isolated case.

I diffed the offending object file and the entire linked tax calculator and
they were bit-for-bit identical, so we're probably good.

* src/current/include/depgen.xsl (preproc:depgen): Do not perform dependency
    generation within templates.
2017-12-06 14:39:54 -05:00
Mike Gerwitz 2878d1099c Defer inline-templates with sym-set until symtable is available
Other code relied on expand-sequence, but this really should be implicit,
since it is deceptively useless otherwise.

* src/current/include/preproc/template.xsl (preproc:macros): Defer
    processing of inline-template with sym-set until symbol table is
    available.
2017-12-05 15:07:29 -05:00
Mike Gerwitz c207fad008 Allow ##other nodes in apply nodes
The primary motivation for this is to allow for template conditionals.

* src/current/calc.xsd (applyArgs): New group.
  (apply,recurse): Use it in sequence.
2017-12-04 13:40:12 -05:00
Mike Gerwitz 0fb1bae487 Add c:sum/@dim
This allows the result of a rate block to be a matrix; there was previously
no way for a named value to be assigned a matrix unless it was a parameter.

This is a bit of a kluge---the compiler won't discover the proper type
information and won't perform the proper safeguards.

* src/current/calc.xsd (sum)[@dim]: Add attribute.
* src/current/compiler/js-calc.xsl: Add xs namespace.
  (compile-calc): Do not perform casting when @dim > 1.
* src/current/include/preproc/expand.xsl
  (lv:rate-each): Include @dim in c:sum expansion.
* src/current/include/preproc/macros.xsl:
  (c:*/@generates): Use @dim to determine symbol dimensions.
* src/current/include/preproc/expand.xsl: Parse @dim aliases (e.g. "vector",
  "matrix").
2017-12-04 13:39:55 -05:00
Mike Gerwitz 6fa6b03dde todo: Add package-local parameter todo
* doc/todo.texi (Compiler): Add package-local parameters.
2017-11-30 13:50:56 -05:00
Mike Gerwitz ef1e6735b6 entry-form: Correct rendering of params using imported typedefs
If a param referenced a typedef that was imported from another package, the
summary page displayed an error rather than rendering the field.  This is a
problem for testing using summary pages!

Long-standing bug with a fairly easy fix.

* src/current/include/entry-form.xsl:
  (entry-form-field lv:param): Consider correct symbol (was using param
    symbol, not typedef).
  [typesrc]: Modernize syntax.
  [typedef]: Allow for non-root-child elements.  The old version of TAME
    required toplevel typedefs; that's no longer the case.
2017-11-29 13:22:29 -05:00
Mike Gerwitz e52d6c1eb0 depgen: Add error for rate block name conflicts
This can occur during template expansion if the user is not careful in
ensuring that all generated blocks will have unique names.  The error was
not useful: it failed type validation, which results in an internal error in
the compiler (from the user's perspective) and outputs no useful information
to aid in debugging.

* src/current/include/depgen.xsl
  (preproc:depgen preproc:sym): User-friendly error if more than one rate
    element.
  [rate]: Allow multiple rate elements.
2017-11-29 13:19:41 -05:00
Mike Gerwitz df4d976277 csvm2csv: Add support for date->ts conversion
* src/current/tools/csvm2csv: Modify example to include date.
  (rangeout, parseLine): Renamed from `rangeout'.  Add date->timestamp
    parsing.
2017-11-22 13:05:02 -05:00
Mike Gerwitz bc6cd539fe Remove duplicates in tools/
Also exist in src/current/tools.

 * tools/csv2xml: remove file
 * tools/csvi: remove file
 * tools/csvm2csv: remove file
 * tools/gen-make: remove file
 * tools/tdat2xml: remove file
 * tools/zipre: remove file
2017-11-22 10:15:51 -05:00
Mike Gerwitz 7e54857603 README.md: Add documentation section 2017-08-22 15:55:29 -04:00
Mike Gerwitz 481346b237 Include package eligibility classes during linking
This is important to include all terminating classifications, which
include assertions.  This is essential now that @keep support has been
removed; this essentially does the same thing, but in a more
sane/strict manner.

* src/current/compiler/linker.xsl (l:depgen)[preproc:symtable]:
  Include package-level eligibility class in initial dependency list.
2017-08-07 13:07:06 -04:00
Mike Gerwitz 450ceee2d7 todo: Add Documentation section
* doc/todo.xsl (Project TODOs/Documentation): Add section.
2017-07-26 13:56:50 -04:00
Mike Gerwitz b460c69aa5 todo: Treat all nodes as short-hand template expansions
* doc/todo.texi (Project TODOs/Preprocessor): Add section.
2017-07-26 09:50:17 -04:00
Mike Gerwitz 90c9893a9e Remove include of now-missing defnote-attr
This was removed recently.

* src/current/dot/defnode-attr.xsl: Remove include.
2017-07-21 15:26:05 -04:00
Mike Gerwitz a4b3d9e4a4 Compiler TODO for algebraic types
* doc/todo.texi (Compiler): Add section.
2017-07-19 16:22:08 -04:00
Mike Gerwitz 222a71b34c tools/strip: Accept files as arguments
Not sure how this hard-coded one ended up getting committed.

* tools/strip: Accept filenames as arguments.
2017-07-18 15:32:42 -04:00
Mike Gerwitz 08b9aecc9c doc: Some project TODOs
* doc/Makefile.am (tame_TEXINFOS): Add `todo.texi'.
* doc/tame.texi: Add reference to todo.
* doc/todo.texi: Add file.
2017-07-18 15:31:43 -04:00
Mike Gerwitz fe7849224d js-calc.xsl: Remove now-unneeded worksheet ns declaration
* src/current/compiler/js-calc.xsl: Remove `w' ns prefix from root node.
  Style root node.
2017-07-18 10:27:42 -04:00
Mike Gerwitz d1dbea91ac Remove debug collection from compiled function calls
This will hopefully provide a performance boost, and is a cheap
optimization to make.  The information it collected was pretty useless
in practice.

* src/current/compiler/js-calc.xsl (compile) [c:*]:
  Do not encase calculations with function ancestors in debug
  collection functions.
2017-07-18 10:14:22 -04:00
Mike Gerwitz 666b9d6a97 Remove broken calc debug worksheet predicates 2017-07-18 10:06:34 -04:00
Mike Gerwitz 52809c5148 src/current/compiler/js-calc.xsl: Strip xsl ns prefix
* src/current/compiler/js-calc.xsl: Strip xsl namespace prefix.
  Update copyright year.
  Set XSLT version to 2.0.
2017-07-18 10:04:27 -04:00
Mike Gerwitz aeded560f4 Include meta automatically in link list
These used to be automatically added via @keep.

* src/current/compiler/linker.xsl (l:depgen): Include meta symbols.
* src/symtable/symbols.xsl (lv:meta): @pollute instead of @keep.
2017-07-13 15:24:30 -04:00
Mike Gerwitz 32afebd631 Preprocess input map nodes
In particular, I want it to handle absolute import paths.  It does
this for the return map; I forgot to add it here.

* src/current/compiler/map.xsl (lvmc:compile) [lvm:program-map]:
  Preprocess nodes.
2017-07-13 09:42:20 -04:00
Mike Gerwitz d5867eeda8 Remove @keep from linker 2017-07-12 16:28:49 -04:00
Mike Gerwitz dbb368ab34 macros.xsl: Remove unnused param boolean class generation
A classification used to be generated for each param for convenience;
this has been deprecated by the progui package generation (when `q:'
is used).

* src/current/include/preproc/macros.xsl: Remove template.
2017-07-12 15:58:39 -04:00
Mike Gerwitz 2af0daceb3 Remove @keep support from linker
And everything else.

This is a big (important) change; it addresses one of the greatest
pains of the system.

Keeps were added during the DSL rewrite (to support symbols and such)
to work around the issue that there was no symbol-driven map; it
allowed symbols to persist disjoint from the `__yield' dependency
graph so that they could be mapped back and used by external systems.

The problem with that is that it's both messy (coupling the concept of
external dependencies with the actual code) and difficult to work
with.  It had a huge performance impact on the linker for two reasons:

  - Checking whether a package had already been seen and importing the
    keeps on first visit was expensive because of tree searching and
    manipulation; and
  - _every_ keep was imported and processed by the linker, even if it
    wouldn't end up being used by a particular program.

The later especially had huge performance impacts on the entire
system.

The entire dependency graph is now map-driven, with the exception of
the implicit `__yield' (which will eventually be moved into the map as
well and the magic `lv:yield' removed in favor of a template).

Performance-wise: our largest program ("dwelling") has many thousands
of symbols and the largest package imported the majority of them, many
of them unneeded, as the result of @keep subgraphs.  Compilation of
the largest package within that (for the UI) took about a minute and a
half and ate up ~6GiB of RAM, for what really is a trivial task of
resolving externs, some basic symbol processing, a topological sort,
and ordering code fragments.

After this change, it takes ~15s and less than 2GiB of RAM.  Still a
lot---and more improvements can be made---but much, much better.

@keep and friends was left in rater.xsd so that nothing breaks while
code is cleaned up; it'll be removed in the future.

* src/current/compiler/linker.xsl: Remove @keep support.
* src/current/dot/attr-keep.xsl: Remove now-unneeded template.
* src/current/dot/defnode.xsl: Remove @keep and related.
* src/current/include/preproc/eligclass.xsl: Remove @keep and related.
* src/current/include/preproc/expand.xsl: Remove @keep and related.
* src/current/include/preproc/macros.xsl: Remove @keep and related.
* src/current/include/preproc/symtable.xsl: Remove @keep and related.
* src/current/rater.xsd: Add TODO to remove @keep and friends.
2017-07-12 15:58:37 -04:00
Mike Gerwitz 52c1bb5ebe Properly resolve paths of symbols of imports
This solves a long-standing problem whereby relative paths are not
properly resolved, leading to incorrect symbol conflicts when relative
paths to the same package vary between two imports.

See doc/notes/path-processing for an illustration.

* src/current/include/preproc/path.xsl
  (preproc:resolve-relative-import): Add function.
  (preproc:resolv-path, preproc:resolve-path, preproc:get-path): Add
  functions to invoke existing templates.
* src/current/include/preproc/symtable.xsl (preproc:symimport): Use
  `preproc:resolve-relative-import'.
* doc/notes/path-processing: Add notes.
2017-07-12 14:32:33 -04:00
Mike Gerwitz 7472cb882c Return map symbol support
This is a backwards-incompatible change that, like the input map,
requires the use of symbols in the return map.  This will allow us to
forego the use of @keep and will have the return map be the authority
of what gets linked (all of its dependencies).

* src/current/compiler/map.xsl: Add symbol support to return-map.
2017-07-12 14:23:26 -04:00
Mike Gerwitz 5b9cfaec31 Allow ##any in return map
* src/current/map.xsd (return-map): Allow ##any.
2017-07-12 14:16:42 -04:00
Mike Gerwitz 7d9fb48e11 Strip `xsl' namespace prefix from path.xsl
This also sets the XSLT version to 2.0.

* src/current/include/preproc/path.xsl: Strip `xsl' namespace prefix.
2017-07-12 10:10:45 -04:00
Mike Gerwitz c156c10989 Provide symtable to all map compilation steps
* src/current/compiler/map.xsl: Pass symtable to all templates.
2017-07-11 08:59:17 -04:00
Mike Gerwitz 86ac6546dc linker.xsl: Strip xsl ns prefix
* src/current/compiler/linker.xsl: Strip xsl ns prefix.
2017-07-11 08:59:16 -04:00
Mike Gerwitz b5f40cb1ce Recognize meta fields in map 2017-07-07 16:44:36 -04:00
Mike Gerwitz e237ab47f2 Recognize meta fields in map
* src/current/compiler/map.xsl (lvmc:validate-ui) [lvm:program-map]:
  Recognize "meta:"-prefixed meta field ids in map.
2017-07-07 15:18:20 -04:00
Mike Gerwitz 68dd0706a8 Simplify unknown source field map detection
Simple refactoring.  Code still is terrible.

* src/current/compiler/map.xsl (lvmc:validate-ui) [lvm:program-map]:
  Factor common expressions.
2017-07-07 15:18:20 -04:00
Mike Gerwitz 652465a0a0 tools/strip: Add utility 2017-07-05 13:52:06 -04:00
Mike Gerwitz b7596515b3 map compiler: Use symbol table
This allows for the proper importing of symbols into the package
generated by the map compiler, which in turn allows for processing
their default values.
2017-07-05 13:52:04 -04:00