This ensures that each metavariable defined within a template (a template
parameter) has, by the time that the template definition is ended, at least
one reference to each metavariable.
This has practical benefits---ensuring that you haven't forgotten to use a
metavariable; ensuring that you clean up when code is removed; and ensuring
that you didn't accidentally delete some reference that you didn't intend to
(at least in the case of _all_ references...)---but the rationale for it in
this commit is a bit different. More on that below.
This does introduce the negative effect of making it more difficult to
consume inputs without utilizing them, acting more like a relevant type
system (in terms of substructural type systems and with respect to
metavariables, at least). You can for now reference them in contexts that
would reasonably have no effect on the program or be optimized away, but in
the future it'd be nice to explicitly state "hey this isn't intended to be
used yet", especially if you're creating shells of templates, or trying to
maintain BC in a particular situation. But more on that in the future.
For now, the real reason for this change is because of how I intend for
template expansion to work: by walking the body. Rather than having to
check both the parameters of the template and then expand the body
separately, we can simply trust that each parameter is referenced. Then,
after rebinding metavariable references, we can perform the same check on
the expansion template to see if there were arguments provided that do not
correspond to parameters.
This also adds flexibility with parameters that are used
conditionally---we'll be able to have conditionally required parameters in
error reporting.
More information on this is coming, though; it'll be included in the docs of
the commit that introduces the changes.
DEV-13163
This enforces the new constraint that templates expanding into an `Expr`
context must only inline a single `Expr`.
Perhaps in the future we'll support explicit splicing, like `,@` in
Lisp. But this new restriction is intended for two purposes:
- To make templates more predictable (if you have a list of expressions
inlined then they will act differently depending on the type of
expression that they are inlined into, which means that more defensive
programming would otherwise be required); and
- To make expansion easier, since we're going to have to set aside an
expansion workspace ahead of time to ensure ordering (Petgraph can't
replace edges in-place). If we support multi-expansion, we'd have to
handle associativity in all expression contexts.
This'll become more clear in future commits.
It's nice to see all this hard work coming together now, though; it's easy
now to perform static analysis on the system, and any part of the graph
construction can throw errors with rich diagnostic information and still
recover properly. And, importantly, the system enforces its own state, and
the compiler helps us with that (the previous commits).
DEV-13163
The fixpoint tests for `meta-interp` are finally working. I could have
broken this up more, but I'm exhausted with this process, so, you get what
you get.
NIR will now recognize basic `<text>` and `<param-value>` nodes (note the
caveat for `<text>` in the comment, for now), and I finally include abstract
binding in the lowering pipeline. `xmli` output is also now able to cope
with metavariables with a single lexical association, and continues to
become more of a mess.
DEV-13163
This introduces template/param and regenerates it in the xmli output. Note
that this does not check that applications reference known params; that's a
later phase.
DEV-13163
This introduces `xmlo-experimental` for `--emit`, allowing the new parser to
be toggled selectively for individual packages. This has a few notable
benefits:
1. We'll be able to conditionally compile packages as they are
supported (TAMER will target specific packages in our system to try to
achieve certain results more quickly);
2. This cleans up the code a bit by removing awkward gated logic, allowing
natural abstractions to form; and
3. Removing the compile-time feature flag ensures that the new features
are always built and tested; there are fewer configuration combinations
to test.
DEV-13162
This is needed to then support `@desc` for shorthand desugaring; it's
required by the XSLT-based compiler (and will eventually be required by
TAMER too).
DEV-13708
This introduces `<match on="foo" />` and `<match on="foo" value="bar" />`,
which are both equality predicates. Other types of predicates are not yet
supported.
This change is a bit messy and leaves a bit to be desired. `NirToAir` is
quite messy and needs some cleanup. There's also the issue of introducing
XML-specific errors in NIR so that users know what things like "subject"
mean, but not being able to do so yet because NIR is agnostic to the source
document type; another layer of abstraction is needed.
But, my priority is first to get derivation of a particularly
expensive (generated) package in our internal systems working first.
DEV-13708
This recognizes template application within expressions. Since expressions
can occur within templates, this can occur arbitrarily deeply.
And with that, we have the core of the template system represented on the
graph. Of course, there are some glaring scoping issues to be resolved, but
those aren't unique to template application.
DEV-13708
I had hoped this would be considerably easier to implement, but there are
some confounding factors.
First of all: this accomplishes the initial task of getting nested template
applications and definitions re-output in the `xmli` file. But to do so
successfully, some assumptions had to be made.
The primary issue is that of scope. The old (XSLT-based) TAME relied on the
output JS to handle lexical scope for it at runtime in most situations. In
the case of the template system, when scoping/shadowing were needed, complex
and buggy XPaths were used to make a best effort. The equivalent here would
be a graph traversal, which is not ideal.
I had begun going down the rabbit hole of formalizing lexical scope for
TAMER with environments, but I want to get this committed and working first;
I've been holding onto this and breaking off changes for some time now.
DEV-13708
This represents a significant departure from how the XSLT-based TAME handles
the `@values@` param, but it will end up having the same effect. It builds
upon prior work, utilizing the fact that referencing a template in TAMER
will expand it.
The problem is this: allowing trees in `Meta` would add yet another
container; we have `Pkg` and `Tpl` already. This was the same problem with
template application---I didn't want to add support for binding arguments
separately, and so re-used templates themselves, reaching the generalization
I just mentioned above.
`Meta` is intended to be a lexical metasyntatic variable. That keeps its
implementation quite simple. But if we start allowing trees, that gets
rather complicated really quickly, and starts to require much more complex
AIR parser state.
But we can accomplish the same behavior by desugaring into an existing
container---a template---and placing the body within it. Then, in the
future, we'll parse `param-copy` into a simple `Air::RefIdent`, which will
expand the closed template and produce the same result as it does today in
the XSLT-based system.
This leaves open issues of closure (variable binding) in complex scenarios,
such as in templates that introduce metavariables to be utilized by the
body. That's never a practice I liked, but we'll see how things evolve.
Further, this does not yet handle nested template applications.
But this saved me a ton of work. Desugaring is much simpler.
The question is going to be how the XSLT-based compiler responds to this for
large packages with thousands of template applications. I'll have to see
if it's worth the hit at that time, or if we should inline it when
generating the `xmli` file, producing the same `@values@` as
before. But as it stands at this moment, the output is _not_ compatible
with the current compiler, as it expects `@values@` to be a tree, so a
modification would have to be made there.
DEV-13708
This applies to template application only; there's still some work to do for
template parameters in definitions (well, for deriving them in `xmli` at
least). And, as you can see, there's still a lot of TODO items here.
I ended up backtracking on tree edges to Meta, and even on cross edges to
Meta, because it complicated xmli derivation with no benefit right now;
maybe a cross edge will be re-added in the future, but I need to move on and
see where this takes me.
But, it works.
DEV-13708
This this a big change that's difficult to break up, and I don't have the
energy after it.
This introduces nullary template application, short- and long-form. Note
that a body of the short form is a `@values@` argument, so that's not
supported yet.
This continues to formalize the idea of what "template application" and
"template expansion" mean in TAMER. It makes a separate `TplApply`
unnecessary, because now application is simply a reference to a
template. Expansion and application are one and the same: when a template
expands, it'll re-bind metavariables to the parent context. So in a
template context, this amounts to application.
But applying a closed template will have nothing to bind, and so is
equivalent to expansion. And since `Meta` objects are not valid outside of
a `Tpl` context, applying a non-closed template outside of another template
will be invalid.
So we get all of this with a single primitive (getting the "value" of a
template).
The expansion is conceptually like `,@` in Lisp, where we're splicing trees.
It's a mess in some spots, but I want to get this committed before I do a
little bit of cleanup.
Mike Gerwitz