Commit Graph

240 Commits (96ffd5f6e5d7b26b74758640482de2d614bcc633)

Author SHA1 Message Date
Mike Gerwitz b8d9128f18 POC for full graph 2020-05-13 00:44:13 -04:00
Joseph Frazer 15f5867508 [DEV-7198] Replace `rate-each` macro with a template
Replacing the existing macros with templates will allow us to now have
to deal with macros in the new compiler.

The `indexNameType` pattern needed to change to allow for variables. I
also had to remove the prefix for the `gentle-no` option of `rate`.
2020-04-17 11:35:10 -04:00
Joseph Frazer aa2bc6eedf [DEV-7198] Create a "yield" template
Create a "yield" and add backwards compatibility for the macro of the
same name. This is one of 2 macros that need to be replaced so we do not
have to worry about them with the new compiler.
2020-04-17 07:42:09 -04:00
Joseph Frazer add610b7df [DEV-7133] Remove dependency from "lv:function/lv:param"
These dependencies do not matter and can be safely ignored. The linker
will catch these cycles in future versions so we need to remove the deps
now.
2020-03-26 08:48:43 -04:00
Mike Gerwitz bfea768f89 Copyright year 2020 update 2020-03-06 11:05:18 -05:00
Mike Gerwitz 310ddb7ea8 Replace XSLT-based linker with error
All systems should be using the provided Makefile, so this shouldn't be
invoked anymore.  The new linker is still considered a proof-of-concept, but
bugs have been encountered in the old one that are not worth investing the
time into fixing.

The new linker has been used in production for nearly a couple months and is
functioning properly.
2020-03-02 15:54:32 -05:00
Mike Gerwitz 19a6d67dc4 TAMER: Separate static xmle section 2020-02-26 10:49:01 -05:00
Mike Gerwitz 7c60b53de8 TAMER: Virtual symbol override 2020-02-26 10:49:01 -05:00
Mike Gerwitz 645908e258 TAMER: xmle output changes to support Summary Page
Co-Authored-By: Joseph Frazer <joseph.frazer@ryansg.com>
2020-02-26 10:49:00 -05:00
Mike Gerwitz 6939753ca0 TAMER: POC: Output xmle
This is a working proof-of-concept that will be finalized in future commits.
2020-02-26 10:49:00 -05:00
Mike Gerwitz 10b9caa7ad TAMER: Fail on empty fragment ids (and fix underlying problem) 2020-02-25 16:46:28 -05:00
Mike Gerwitz ff0c8bb34f Order symtable, sym-dep, fragments
This ordering will simplify streaming processing of xmlo files in
TAMER.  Specifically, we know that symbols will have been declared by the
time dependencies are added to the graph (and so we should only be creating
edges to existing nodes); and we can halt reading as soon as the closing
fragments tag is encountered, avoiding parsing the entirety of these massive
XML files.

On one particularly large program, this cuts time down from ~0.333s to
~0.300 in the POC linker.
2020-02-24 14:56:28 -05:00
Joseph Frazer f2cbc5f8ad [DEV-6947] Allow param values to remove underscores 2020-01-31 16:27:04 -05:00
Austin Schaffer 0db05c442c Pass canterm flag to raters 2020-01-29 11:14:13 -05:00
Mike Gerwitz 7a2ce00ed5 src/current/compiler/js.xsl: Remove inline defaults for anyValue
This not only reduces file size, but also has a significant performance
benefit for the UI, which is almost entirely classifications.  A run for one
of our systems was reduced from 1m30s to 11s from this change.
2020-01-22 16:31:16 -05:00
Mike Gerwitz 46d5ed286c src/current/compiler/js.xsl: Strip unused result-set (@yields alt) 2020-01-22 16:31:16 -05:00
Mike Gerwitz 661684f1e4 src/current/compiler/js.xsl: Remove last anyValue arg by default
This was used to provide additional information on the stack for debugging
the compiled code.  Since this is very rarely needed, and is only needed by
someone debugging the compiler, it can be manually enabled if desired.

This also wraps it so that it'll be stripped if it is included.
2020-01-22 16:31:16 -05:00
Mike Gerwitz b51f7fa042 src/current/compiler/js.xsl: {._CMATCH_=>[_CMATCH_]}
This was confusing Closure Compiler.
2020-01-22 16:31:16 -05:00
Mike Gerwitz 47d5dc238c src/current/compiler/js.xsl: @expose Closure Compiler annotations
This is deprecated, but neither of the recommended @export or @nocollapse
work the same way.
2020-01-22 16:31:09 -05:00
Mike Gerwitz 97806d5602 src/current/compiler/js.xsl: Remove dead arg check code
This was removed during The Great Refactoring.
It will be replaced with a better systemin TAMER.
2020-01-22 16:30:53 -05:00
Mike Gerwitz e0a78c2ed6 src/current/compiler/js-calc.xsl (compile-calc)[c:let]: Remove global assignment
The previous code was unintentionally assigning to an undefined global
variable.
2020-01-22 16:30:53 -05:00
Joseph Frazer cdacd1d93d [DEV-6595] Loosen the way we find classification matches
The `<t:match-class-code-lookup />` matches were not showing in the
summary pages. I loosened the selector so it is able to find the matches
when it generates the summary pages.
2019-12-10 08:08:52 -05:00
Joseph Frazer 0fadbe8e8a [DEV-6370] Allow recursive conditionals
If an `lvm:if` is immediately followed by another 'lvm:if`, both should
be used to create the conditional. The existing code wouild only "select
the nearest condition".
2019-11-01 10:23:19 -04:00
Joseph Frazer cbe32aff72 [DEV-6370] Pass in the $line_code rather than using it from the contract
The LOB being passed into the function was being ignored and instead it
was pulling it from the contract object. With Package, this caused all 3
LOB to be "COMMPKGE" rather than the correct LOB being processed at the
time.

Going forward, one cannot `map` or `pass` to "line_code" as it will be
considered a reserved word.

Co-Authored-By: Jim Grundner <james.grundner@rtspecialty.com>
2019-11-01 10:23:19 -04:00
Mike Gerwitz 3ef6571922 Provide friendly lv:param-typedef-lookup failure for duplicate item values
The real solution is to disallow typedefs from getting into this state to
begin with, but I don't have time for that right now.
2019-10-25 13:56:47 -04:00
Mike Gerwitz 8005268a1b lv:param-typedef-lookup: New preprocessor directive 2019-08-06 15:31:48 -04:00
Mike Gerwitz 90bedc20f8 map: Nested value support for input map
For example: meta:foo.bar.baz.

DEV-3871
2019-06-14 11:02:18 -04:00
Mike Gerwitz 13ed4cd7dc Clean up extclass remenants
This is left over from f2db9f1268, in which I
should have cleaned all of this up.  One of our developers was hitting the
removed warning, which isn't necessary since the concept of a separate
"classifier" is no longer a thing after the aforementioned commit.

* rater/rater.xsd (no-extclass, no-extclass-keeps): Remove.
* src/current/rater.xsd: Likewise.  (I really need to deduplicate these.)
* src/current/compiler/js.xsl (compiler:entry-rater): Remove inaccurate
    comment (genclasses is used for other things).
* src/current/include/depgen.xsl (preproc:depgen-match): Remove error
    checking for pulling in non-external classes (this is the error that the
    developer hit that is no longer needed).
* src/current/include/preproc/eligclass.xsl (preproc:sym): Remove
    `@extclass' predicate.  Remove portion of comment.
* src/current/include/preproc/expand.xsl: Remove ancient footnote that
    even references an old internal rater!
* src/current/include/preproc/macros.xsl (preproc:class-groupgen): Remove
    external propagation.
* src/current/include/preproc/symtable.xsl (preproc:symimport): Remove
    extclass checks and propagation.
  (preproc:symtable)[lv:rate]: Remove external propagation.
    [lv:classify]: Likewise.
* src/current/include/preproc/template.xsl (preproc:inline-apply): Remove
    external sym metadata support.
2019-05-22 12:57:35 -04:00
Mike Gerwitz 5706ab4bef summary: Fill timestamp_* param values automatically
These exist because TAME is nondeterministic, so all state must be passed
into it.  But it's inconvenient to have users have to manually fill in
dates, so we derive them from the environment unless they are set.

* src/current/scripts/entry-form.js (fillTimeValues): New function.
  (rater): Use it.
2019-04-19 11:46:29 -04:00
Mike Gerwitz 602a77443f compiler: Expose params via compiled rater function
* src/current/compiler/js.xsl (compiler:exit-rater)[lv:package]: Expose
    `params' publicly on the rater function.
2019-02-26 11:09:23 -05:00
Mike Gerwitz 5714bfb96b symtable: Substantial performance improvement in processing
This further improves performance of the symbol table processing.  The next
step will be to address how symbols are handled on a more intimate level,
since it's a huge mess atm.  But I'll save that for later, after the
low-hanging fruit has been resolved.

* src/current/include/preproc/symtable.xsl (preproc:sym-discover): Use
    `for-each-group' in place of `preceding-sibling'.  Aggressive use of
    maps for geneating the `dedup' sequence, which is a mess.
  (preproc:symtable-process-symbols): Additional maps to avoid
    preceding-sibling and following-sibling selectors (O(n²)=>O(n)).
2019-02-20 02:03:20 -05:00
Mike Gerwitz 16749a9a45 fragment: Iterate over document and use symtable map
Same concept as previous commits: rather than iterating over the symbol
table and scanning the tree for the matching node, iterate over the document
and look up from a symbol map: O(n²) => O(n).

This gives a respectable performance boost to compilation of certain
packages (best improving packages with many classifications or rate blocks).

* src/current/compiler/fragments.xsl (@xmlns:xs, @xmlns:map): New namespace
    declarations.
  (preproc:compile-fragments): Generate `preproc:fragment' nodes and match
    on document rather than symbols.
    [lv:package]: Generate map and tunnel it.
* src/current/compiler/js.xsl (compile)[lv:classify, lv:match]: Use
    symtable-map.
  (compile-class-condition)[lv:rate]: Likewise.
  (compile-cmatch)[lv:rate]: Likewise.
2019-02-20 00:26:32 -05:00
Mike Gerwitz dae1990a00 symtable: Speed up processing a bit
This uses the same map strategy (and same duplicate code) as previous
commits, but this one generates a map for two separate tables.

There is more room for improvement, but this cuts down on the time a
lot.  Also keep in mind that this is performed multiple times (once per
pass), so it's still worth revisiting.  Performance is still very poor for
very large (many thousands of symbols) symbol tables.

The next slowest part appears to be the fragment compilation.  I'm nearing
the end of the low-low-hanging fruit for maps.  The /common/gl package
mentioned in previous commits that previously took over a minute to compile
now compiles in 20s as of this commit on equivalent hardware.

* src/current/include/preproc/symtable.xsl (@xmlns:map): New namespace
    declaration.
  (preproc:symtable-process-symbols): Create map for `cursym' and
    `extresults'.  Use it.  Remove unused `dup'.  Output message when
    done (another is output slightly later on in the process).
2019-02-20 00:10:42 -05:00
Mike Gerwitz 063e68b3d0 validate: Use map for symbol table
This is the first step to improving the map.  Note that this duplicates the
symbol table generation code that's used in a few other places
now---that'll be cleaned up in future commits once I have a better idea of
all the places this will be used and try to move it to a higher level.

* src/current/compiler/validate.xsl (@xmlns:xs, @xmlns:map): New namespace
    definitions.
  (lvv:validate)[lv:package]: Generate symbol table map.  Tunnel to
    templates.
    [c:apply[@name], lv:classify[@as]//lv:match, lv:match[@value]]
    [c:*[@name or @of], c:apply/c:arg[@name], lv:rate/lv:class]: Use it.
2019-02-18 15:42:22 -05:00
Mike Gerwitz c077a71402 resolv-syms: Simplify dimension resolution
The existing code was not only complex (because of XSLT 1), but mostly
unnecessary.  We don't need to consult remote symbol tables at all anymore.

This shaves off an additional few seconds on large packages.

* src/current/include/preproc/package.xsl (preproc:resolv-syms)[preproc:sym]:
  Only consult local symbol table.  Simplify max dimension calculation.
2019-02-18 11:56:17 -05:00
Mike Gerwitz d2ab6e1149 resolv-syms: Generate maps for symtable and dependency lists
This is a first step (low-hanging-fruit kinda thing) for improving the
performance of symbol resolution, where the compiler has to figure out the
dimensions of a symbol by first resolving its dependencies,
recursively.  This is approximately an O(n³) polynomial-time algorithm _per
recursive step_.  Yikes.

This is traditionally where dynamic programming methods would be used, but
that's considerably more difficult in a immutable languages like XSLT, so
I'll do my best without.  (Saxon does offer some support for mutability, but
I'd prefer to avoid it if possible.)

This first change improves performance 30--40%.  For example, on two large
packages we have, build times drop from 55s to 35s and from 1m42s to 1m13s
respectively.

Good start, but much more to be done!

* src/current/include/preproc/package.xsl (preproc:resolv-syms)[lv:package]:
    Compute maps for preproc:symtable and preproc:sym-deps at each recursive
    step.  Pass along via tunneling.
  (preproc:resolv-syms)[preproc:sym]: Use them.

DEV-4354
2019-02-18 11:56:17 -05:00
Mike Gerwitz 09eb442c63 linker: Index root package symbol table
This only saves 1--2s on a 30s run, but I want to move into this direction,
so it'll simplify future refactoring if I just add it.  Small changes like
these will accumulate, too.

* src/current/compiler/linker.xsl (l:orig-package, l:root-symtable-map): New
    variables.
  (l:resov-extern): Use it.
2019-02-18 11:56:17 -05:00
Mike Gerwitz bd73ea0121 Strip xsl: namespace prefix from most files
Still left are files that I don't want to deal with testing right now.
2019-02-07 14:36:18 -05:00
Mike Gerwitz e022a3133d Copyright year simplification and update to Ryan Specialty Group
This now uses year ranges, which I'll update annually.

This also renames "R-T Specialty" to "Ryan Specialty Group".  The latter is
the parent company of the former.  I was originally employed under the
former when LoVullo Associates was purchased, by I now work for the parent
company.
2019-02-07 13:23:09 -05:00
Mike Gerwitz 7862eef62e summary: Accommodate now-missing dependency lists
The previous commit made dependency lists optional for certain symbols.  The
Summary Page needs to be updated to permit such a thing.

The whole Summary Page needs aggressive refactoring, though, so this doesn't
bother checking for `no-deps' to see if this is a bad thing.

* src/current/summary.xsl (typeset-final)[preproc:sym-ref]: Permit missing
    symbol dependencies.
  (lv:param|lv:const|lv:item): Likewise.
2019-02-07 13:11:31 -05:00
Mike Gerwitz b6cfdb4221 depgen: Quadratic=>linear-time algorithm
This is a significant performance improvement for dependency
generation (which is responsible for building the dependency graph for a
package).

The previous algorithm ran in O(n²) time: it would iterate over the given
symbol table, and for _each_ symbol, do a linear scan of the entire document
to search for the corresponding source block.  This resulted in explosive
depgen time for larger packages.

This makes the algorithm run in O(n) by:
  - Using an XSLT 3 map for the symbol table for O(1) lookups; and
  - Iterating over the _document_ a single time rather than the symbol
    table, referencing the symbol table as needed (in O(1) time).

There are other parts of the system that can benefit from these same
improvements.  This is important, since we need to be able to handle many
thousands of symbols efficiently.

* src/current/compiler/linker.xsl (l:depgen-sym): Recognize smybol `no-deps'
    property, permitting missing dependencies.  This allows us to avoid
    creating nonsense nodes just to satisfy the linker, while still allowing
    the linker to perform essential checks to defend against compiler bugs.
* src/current/compiler/map.xsl (lvmc:stub-symtable): Set @no-deps on
    `___head' and `___tail' symbols.
  (lvmc:mapsym): Set `no-deps' as appropriate on map symbols.
  (preproc:depgen)[lvm:map[@from]]: Generate `preproc:sym-dep' node, which
    is now expected by the depgen process.
  (preproc:depgen)[lvm:map[*]]: Likewise.
  (preproc:depgen)[*[@lvmc:type='retmap']//lvmm:map[@from]]: Remove
    unnecessary template.
  (preproc:symtable)[lvm:map[@value]]: Pass `no-deps' to `lvmc:mapsym'.
* src/current/include/depgen.xsl (preproc:depgen)[preproc:symtable]: Create
    and use XSLT 3 map in place of `preproc:symtable' tree.  This allows for
    constant-time lookups.  Provide to templates via tunnelling.  Use it in
    place of exiting tree references.  Process source tree rather than
    iterating over symbol table.
  (preproc:depgen)[lv:rate, c:sum[@generates], c:product[@generates],
    lv:classify, lv:function/lv:param, lv:function, lv:typedef]: Produce
      `preproc:sym-dep' nodes (which was previously done while iterating
      over the symbol table).
  (preproc:depgen)[preproc:sym]: Remove all such processing, since we no
    longer iterate over the symbol table.
  (preproc:depgen)[c:value-of]: Use symtable map.
  (preproc:depgen-match): Likewise.
  (preproc:depgen)[lv:union]: Modify to handle changes to lv:typedef
    template.
  (preproc:depgen)[text()]: Remove and replace with `node()'.
* src/current/include/preproc/package.xsl (preproc:resolv-syms): Remove
    logging of symbol resolution.  This has a slight performace impact since
    there is a lot of output.
* src/current/include/preproc/symtable.xsl
  (lv:function/lv:param, c:let/c:Values/c:value): Set `no-deps'.
* src/symtable/symbols.xsl: Add documentation of `no-deps'.
  (preproc:symtable)[lv:meta]: Set `no-deps'.
2019-02-07 11:39:50 -05:00
Mike Gerwitz 11109d4361 core: Add _where-*_ query predicate templates
These provide a more pleasent abstraction than having to reference CMP_OP_*
constants.

* core/test/core/vector/interpolate.xml: {t:when=>t:where-eq}.
* core/test/core/vector/table.xml: Likewise, but using the other variants
    where appropriate given the value of `@op'.
* core/vector/interpolate.xml: Likewise.
* core/vector/table.xml (_when_, _where_): Rename former to latter and
    provide deprecation warning.
  (_when-lt_, _when-lte_, _when-gt_, _when-gte_): Add abstractions.
* src/current/rater.xsd: Permit template variable as tenplate name.
2019-02-04 10:22:46 -05:00
Mike Gerwitz 9af38261b9 calc.xsd: Permit template applications within c:value-of
* src/current/calc.xsd (valueType): Permit ##other nodes.
2019-02-04 10:22:46 -05:00
Mike Gerwitz a35844f3fb depgen: Do not recurse into templates
Same logic, more efficient implementation.

* src/current/include/depgen.xsl (preproc:depgen): Stop at lv:template.
2019-02-02 23:19:40 -05:00
Mike Gerwitz 07e5dbd94b Add expand-barrier and skip-child-expansion
It's going to be like TeX before you know it... ._.

* src/current/include/preproc/package.xsl (preproc:tpl-check)
  [lv:template|lv:const|lv:typedef|lv:param-copy]: Add lv:param-copy.
* src/current/include/preproc/template.xsl (preproc:apply-template)
 [lv:expand-barrier, lv:skip-child-expansion]: New expansion control
   structures.
2019-02-01 16:01:56 -05:00
Mike Gerwitz 7ac4c1ce9d Template variable expansion on (lv:param-value|lv:param-inherit)/@name
This allows for dynamically generated metadata names.

* src/current/include/preproc/template.xsl (preproc:apply-template)
  [lv:param-meta]: Expand @name.
  [lv:param-inherit]: Expand @meta.
2019-02-01 16:01:56 -05:00
Mike Gerwitz f719d391c7 Default short-hand constant description to c:value-of/@label
This is a much more useful description if present.

* src/current/include/preproc/macros.xsl (preproc:macros)[c:value-of...]:
    Default generated constant description to @label.
2019-02-01 16:01:56 -05:00
Mike Gerwitz b725963722 Add c:vector as c:set alternative
The term "set" is all wrong---it is actally intended to be a vector, and can
absolutely have duplicate elements (and often does).

* src/current/calc.xsd (vector): Add, recommending in place of `set'.
* src/current/compiler/js-calc.xsl (compile-calc)[c:set|c:vector]:
    Add `c:vector' and provide deprecation notice for `c:set'.
* src/current/include/calc-display.xsl (c:set|c:vector): Likewise.
2019-02-01 16:01:56 -05:00
Mike Gerwitz 304faa1f07 summary: Remove rate-group processing
* src/current/summary.css (.rate-group, .rate-groups): Remove.
* src/current/summary.xsl (gen-menu): Remove rate-group processing.
  (rate-group-title): Remove.
  (lv:rate-group): Remove.
2019-02-01 16:01:56 -05:00
Mike Gerwitz 22aa59b5cf map: Properly default value in translation
A better option is to pre-process all inputs, but I need a quick
fix to my stupidity.  0||""==="".

* src/current/compiler/map.xsl (lvmc:compile)[lvm:map//lvm:from[*]]: Correct oval default.
2019-02-01 16:01:56 -05:00