Compare commits

...

10 Commits

Author SHA1 Message Date
Mike Gerwitz c26444666f tamer: asg: Shorthand and long-form template arguments
This applies to template application only; there's still some work to do for
template parameters in definitions (well, for deriving them in `xmli` at
least).  And, as you can see, there's still a lot of TODO items here.

I ended up backtracking on tree edges to Meta, and even on cross edges to
Meta, because it complicated xmli derivation with no benefit right now;
maybe a cross edge will be re-added in the future, but I need to move on and
see where this takes me.

But, it works.

DEV-13708
2023-03-23 00:04:53 -04:00
Mike Gerwitz b2c6b7f073 tamer: asg::air::expr: Do not cache (globally) identifiers created with StoreDangling
I'm not happy with this implementation.  The linear search is undesirable,
but not too bad (and maybe wouldn't even be worth caching, if this were the
whole story), but we _also_ need to prevent duplicate identifiers.  We are
not going to want to perform a linear search of a linked list (effectively)
every time we add an identifier to check for uniqueness, so I think the
caching is going to have to be generalized very shortly anyway.

As it stands now, a duplicate identifier would cause an error at expansion
time.  That's not what we want, but it's not terrible, because you can have
that same problem in normal circumstances without local conflicts.

But this'll be used for metavariables as well, where we absolutely _do_ want
to fail at template definition time.

DEV-13708
2023-03-22 13:26:47 -04:00
Mike Gerwitz 31a39c79d3 tamer: asg::graph: *lookup{=>_global}*
Identifier lookups, as done using the graph methods today, look up from a
cache representing the global environment.

Templates must not contribute to this environment until expansion.  Further,
metavariables will not be present in this environment.  To avoid confusion
and help obviate accidental contributions to this environment, the methods
have been renamed.  This will also allow for the creation of more general
methods down the line.

DEV-13708
2023-03-22 11:28:55 -04:00
Mike Gerwitz 7a9cf6bc51 tamer: asg::air: Test formatting (token nesting)
This makes the tests quite a bit easier to understand visually.  I've been
doing this with all new tests but had to go back to some old ones, and still
have more to go back to.  Baby steps.

DEV-13708
2023-03-22 11:08:09 -04:00
Mike Gerwitz 91b787d367 tamer: nir: Desugar shorthand template params and yield AIR
I had intended for this to be a full vertical slice initially, but AIR's
parser is going to need enough work that it'll muddy this patch a bit too
much.

This keeps the desugaring simple, which is what I was hoping for.

The next step is to load it into the graph and emit regenerated longhand
sources.

I also don't like how the namespace prefix is just being ignored for
shorthand param desugaring.  This is also the case in the XSLT-based
compiler, but this violates TAMER's principle that it should parse every bit
of information; nothing should be ignored.  If something does not contribute
useful information, then it is not a useful construct and ought to be
rejected.

DEV-13708
2023-03-22 10:17:19 -04:00
Mike Gerwitz d3f8f76904 tamer: nir::NirEntity::TplParam: Optional name/value pair
This will be used for shorthand desugaring.

DEV-13708
2023-03-21 15:26:17 -04:00
Mike Gerwitz c42e693b14 tamer: Introduce desugaring operation for shorthand template application
This moves translation from NirToAir into TplShortDesugar, and changes the
output from AIR to NIR.

This is going to be much easier to reason about as a desugaring
operation (and indeed that's always how TAME has implemented it, in XSLT);
this keeps the complexity isolated.

Ideally, NirToAir wouldn't even accept tokens that it can't handle, but
that's going to take quite a bit more work and I don't have the time right
now.  Instead, we'll fail at runtime with some hopefully-useful
information.  It shouldn't actually happen in practice.

DEV-13708
2023-03-21 14:38:07 -04:00
Mike Gerwitz c91d175711 tamer: asg::air::ir: Remove TplApply
The implementation decided upon in the previous commits have made this
unnecessary, using `RefIdent` to produce `Tpl->Ident[->Tpl]` instead.

DEV-13708
2023-03-17 13:46:06 -04:00
Mike Gerwitz 8a3dec0f95 tamer: build-aux/asg-ontviz: Vary arrowhead for cross edges
This makes it more visually apparent, when looking directly at a node,
whether an edge could represent a tree edge.

Dynamic edges could be tree edges, so I left those solid; that's the more
important visual indicator that I'm interested in, and it's disambiguated by
the dashed line.

DEV-13708
2023-03-17 13:37:36 -04:00
Mike Gerwitz 8579b8bce9 tamer: asg: Dynamically determined cross edges
Previous to this commit, ontological cross edges were declared
statically.  But this doesn't fare well with the decided implementation for
template application.

The documentation details it, but we have Tpl->Ident which could mean "I
define this Ident once expanded", or it could mean "this is a reference to a
template I will be applying".  The former is a tree edge, the latter is a
cross edge, and that determination can only be made by inspecting edge data
at runtime.

It could have been resolved by introducing new Object types, but that is a
lot of work for little benefit, especially given that only (right now) the
visitor uses this information.

DEV-13708
2023-03-17 12:53:53 -04:00
36 changed files with 1503 additions and 567 deletions

View File

@ -147,7 +147,7 @@ mod base {
bench.iter(|| {
xs.iter()
.map(|sym| sut.lookup(*sym).unwrap())
.map(|sym| sut.lookup_global(*sym).unwrap())
.for_each(drop);
});
}
@ -286,7 +286,7 @@ mod base {
bench.iter(|| {
xs.iter()
.zip(xs.iter().cycle().skip(1))
.map(|(from, to)| sut.add_dep_lookup(*from, *to))
.map(|(from, to)| sut.add_dep_lookup_global(*from, *to))
.for_each(drop);
});
}
@ -303,7 +303,7 @@ mod base {
bench.iter(|| {
xs.iter()
.zip(xs.iter().cycle().skip(1))
.map(|(from, to)| sut.add_dep_lookup(*from, *to))
.map(|(from, to)| sut.add_dep_lookup_global(*from, *to))
.for_each(drop);
});
}

View File

@ -114,9 +114,23 @@ block_src && $NF ~ /\w+,$/ {
# Edge type (cross, tree)
ty = $(NF-1)
# Dashed is visually in-between solid and dotted,
# and `dyn` is either `tree` or `cross`,
# determined at runtime.
# But we may need some other representation if `dotted` is too visually
# sparse and difficult/annoying to see;
# let's see where the ASG visualization ends up first.
attrs = ""
if (ty == "cross") {
attrs="[style=dashed]"
switch (ty) {
case "tree":
attrs="[style=solid,arrowhead=normal]";
break;
case "cross":
attrs="[style=dotted,arrowhead=open]";
break;
case "dyn":
attrs="[style=dashed,arrowhead=normal]";
break;
}
gsub(/,$/, "")
@ -129,7 +143,7 @@ block_src && $NF ~ /\w+,$/ {
# the source object is right-aligned and target is left-aligned,
# so that `Src -> Target` can be easily read regardless of the width
# of the objects involved.
printf " %5s -> %-5s %14s; # %s edge\n", block_src, $NF, attrs, ty
printf " %5s -> %-5s %32s; # %s edge\n", block_src, $NF, attrs, ty
found_rels++
}

View File

@ -246,7 +246,7 @@ impl ParseState for AirAggregate {
.transition(Empty),
(Empty, AirIdent(IdentDep(sym, dep))) => {
asg.add_dep_lookup(sym, dep);
asg.add_dep_lookup_global(sym, dep);
Transition(Empty).incomplete()
}
@ -255,7 +255,7 @@ impl ParseState for AirAggregate {
}
(Empty, AirIdent(IdentRoot(sym))) => {
let obj = asg.lookup_or_missing(sym);
let obj = asg.lookup_global_or_missing(sym);
asg.add_root(obj);
Transition(Empty).incomplete()

View File

@ -417,6 +417,11 @@ mod root {
/// the sub-expression will be rooted in [`Self`],
/// but the [`Dangling`] root expression will still be rejected.
///
/// This expects identifiers to be rooted in the global environment,
/// which is the package representing the active compilation unit.
/// This may be relaxed once identifier caching is generalized;
/// at the time of writing it is too coupled to the graph root.
///
/// See [`RootStrategy`] for more information.
#[derive(Debug, PartialEq)]
pub struct ReachableOnly<O: ObjectKind>(ObjectIndex<O>)
@ -433,9 +438,9 @@ mod root {
fn defines(&self, asg: &mut Asg, id: SPair) -> ObjectIndex<Ident> {
match self {
Self(oi_root) => {
asg.lookup_or_missing(id).add_edge_from(asg, *oi_root, None)
}
Self(oi_root) => asg
.lookup_global_or_missing(id)
.add_edge_from(asg, *oi_root, None),
}
}
@ -456,6 +461,10 @@ mod root {
/// Sub-expressions can be thought of as utilizing this strategy with an
/// implicit parent [`ObjectIndex<Expr>`](ObjectIndex).
///
/// Unlike [`ReachableOnly`],
/// this does _not_ cache identifiers in the global environment.
/// See there for more information.
///
/// See [`RootStrategy`] for more information.
#[derive(Debug, PartialEq)]
pub struct StoreDangling<O: ObjectKind>(ObjectIndex<O>)
@ -470,11 +479,13 @@ mod root {
Self(oi)
}
fn defines(&self, asg: &mut Asg, id: SPair) -> ObjectIndex<Ident> {
// We are a superset of `ReachableOnly`'s behavior,
// so delegate to avoid duplication.
fn defines(&self, asg: &mut Asg, name: SPair) -> ObjectIndex<Ident> {
// This cannot simply call [`ReachableOnly`]'s `defines` because
// we cannot cache in the global environment.
// This can be realized once caching is generalized;
// see the commit that introduced this comment.
match self {
Self(oi_root) => ReachableOnly(*oi_root).defines(asg, id),
Self(oi_root) => oi_root.declare_local(asg, name),
}
}

View File

@ -45,9 +45,10 @@ pub fn collect_subexprs(
fn expr_empty_ident() {
let id = SPair("foo".into(), S2);
#[rustfmt::skip]
let toks = vec![
Air::ExprStart(ExprOp::Sum, S1),
Air::BindIdent(id),
Air::BindIdent(id),
Air::ExprEnd(S3),
];
@ -89,27 +90,30 @@ fn expr_without_pkg() {
fn close_pkg_mid_expr() {
let id = SPair("foo".into(), S4);
#[rustfmt::skip]
let toks = vec![
Air::PkgStart(S1),
Air::ExprStart(ExprOp::Sum, S2),
Air::ExprStart(ExprOp::Sum, S2),
Air::PkgEnd(S3),
// RECOVERY: Let's finish the expression first...
Air::BindIdent(id),
Air::ExprEnd(S5),
// RECOVERY: Let's finish the expression first...
Air::BindIdent(id),
Air::ExprEnd(S5),
// ...and then try to close again.
Air::PkgEnd(S6),
];
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // ExprStart
Err(ParseError::StateError(AsgError::InvalidPkgEndContext(S3))),
// RECOVERY: We should be able to close the package if we just
// finish the expression first,
// demonstrating that recovery properly maintains all state.
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
// RECOVERY: We should be able to close the package if we
// just finish the expression first,
// demonstrating that recovery properly maintains all
// state.
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
// Successful close here.
Ok(Parsed::Incomplete), // PkgEnd
],
@ -121,29 +125,31 @@ fn close_pkg_mid_expr() {
fn open_pkg_mid_expr() {
let id = SPair("foo".into(), S4);
#[rustfmt::skip]
let toks = vec![
Air::PkgStart(S1),
Air::ExprStart(ExprOp::Sum, S2),
Air::ExprStart(ExprOp::Sum, S2),
Air::PkgStart(S3),
// RECOVERY: We should still be able to complete successfully.
Air::BindIdent(id),
Air::ExprEnd(S5),
// RECOVERY: We should still be able to complete successfully.
Air::BindIdent(id),
Air::ExprEnd(S5),
// Closes the _original_ package.
Air::PkgEnd(S6),
];
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // ExprStart
Err(ParseError::StateError(AsgError::NestedPkgStart(S3, S1))),
// RECOVERY: Ignore the open and continue.
// Of course,
// this means that any identifiers would be defined in a
// different package than was likely intended,
// but at least we'll be able to keep processing.
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
// RECOVERY: Ignore the open and continue.
// Of course,
// this means that any identifiers would be defined in a
// different package than was likely intended,
// but at least we'll be able to keep processing.
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
Ok(Parsed::Incomplete), // PkgEnd
],
Sut::parse(toks.into_iter()).collect::<Vec<_>>(),
@ -155,15 +161,19 @@ fn expr_non_empty_ident_root() {
let id_a = SPair("foo".into(), S2);
let id_b = SPair("bar".into(), S2);
#[rustfmt::skip]
let toks = vec![
Air::ExprStart(ExprOp::Sum, S1),
// Identifier while still empty...
Air::BindIdent(id_a),
Air::ExprStart(ExprOp::Sum, S3),
// (note that the inner expression _does not_ have an ident binding)
Air::ExprEnd(S4),
// ...and an identifier non-empty.
Air::BindIdent(id_b),
// Identifier while still empty...
Air::BindIdent(id_a),
Air::ExprStart(ExprOp::Sum, S3),
// (note that the inner expression _does not_ have an ident
// binding)
Air::ExprEnd(S4),
// ...and an identifier non-empty.
Air::BindIdent(id_b),
Air::ExprEnd(S6),
];
@ -178,12 +188,6 @@ fn expr_non_empty_ident_root() {
// Identifiers should reference the same expression.
let expr_b = asg.expect_ident_obj::<Expr>(id_b);
assert_eq!(expr_a, expr_b);
// Ontological sanity check:
// Child expressions must not be considered cross edges since they are
// part of the same tree.
let oi_expr_a = asg.expect_ident_oi::<Expr>(id_a);
assert!(!oi_expr_a.edges(&asg).any(|rel| rel.is_cross_edge()));
}
// Binding an identifier after a child expression means that the parser is
@ -193,15 +197,17 @@ fn expr_non_empty_ident_root() {
fn expr_non_empty_bind_only_after() {
let id = SPair("foo".into(), S2);
#[rustfmt::skip]
let toks = vec![
Air::ExprStart(ExprOp::Sum, S1),
// Expression root is still dangling at this point.
Air::ExprStart(ExprOp::Sum, S2),
Air::ExprEnd(S3),
// We only bind an identifier _after_ we've created the expression,
// which should cause the still-dangling root to become
// reachable.
Air::BindIdent(id),
// Expression root is still dangling at this point.
Air::ExprStart(ExprOp::Sum, S2),
Air::ExprEnd(S3),
// We only bind an identifier _after_ we've created the expression,
// which should cause the still-dangling root to become
// reachable.
Air::BindIdent(id),
Air::ExprEnd(S5),
];
@ -231,10 +237,11 @@ fn expr_dangling_no_subexpr() {
let full_span = S1.merge(S2).unwrap();
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete),
Err(ParseError::StateError(AsgError::DanglingExpr(full_span))),
Ok(Parsed::Incomplete),
Err(ParseError::StateError(AsgError::DanglingExpr(full_span))),
// RECOVERY
Ok(Parsed::Incomplete), // PkgEnd
],
@ -244,11 +251,12 @@ fn expr_dangling_no_subexpr() {
#[test]
fn expr_dangling_with_subexpr() {
#[rustfmt::skip]
let toks = vec![
Air::ExprStart(ExprOp::Sum, S1),
// Expression root is still dangling at this point.
Air::ExprStart(ExprOp::Sum, S2),
Air::ExprEnd(S3),
// Expression root is still dangling at this point.
Air::ExprStart(ExprOp::Sum, S2),
Air::ExprEnd(S3),
// Still no ident binding,
// so root should still be dangling.
Air::ExprEnd(S4),
@ -257,12 +265,13 @@ fn expr_dangling_with_subexpr() {
let full_span = S1.merge(S4).unwrap();
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Err(ParseError::StateError(AsgError::DanglingExpr(full_span))),
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // ExprEnd
Err(ParseError::StateError(AsgError::DanglingExpr(full_span))),
// RECOVERY
Ok(Parsed::Incomplete), // PkgEnd
],
@ -274,32 +283,34 @@ fn expr_dangling_with_subexpr() {
fn expr_dangling_with_subexpr_ident() {
let id = SPair("foo".into(), S3);
#[rustfmt::skip]
let toks = vec![
Air::ExprStart(ExprOp::Sum, S1),
// Expression root is still dangling at this point.
Air::ExprStart(ExprOp::Sum, S2),
// The _inner_ expression receives an identifier,
// but that should have no impact on the dangling status of the
// root,
// especially given that subexpressions are always reachable
// anyway.
Air::BindIdent(id),
Air::ExprEnd(S4),
// But the root still has no ident binding,
// and so should still be dangling.
// Expression root is still dangling at this point.
Air::ExprStart(ExprOp::Sum, S2),
// The _inner_ expression receives an identifier,
// but that should have no impact on the dangling status of
// the root,
// especially given that subexpressions are always reachable
// anyway.
Air::BindIdent(id),
Air::ExprEnd(S4),
// But the root still has no ident binding,
// and so should still be dangling.
Air::ExprEnd(S5),
];
let full_span = S1.merge(S5).unwrap();
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Err(ParseError::StateError(AsgError::DanglingExpr(full_span))),
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIndent
Ok(Parsed::Incomplete), // ExprEnd
Err(ParseError::StateError(AsgError::DanglingExpr(full_span))),
// RECOVERY
Ok(Parsed::Incomplete), // PkgEnd
],
@ -314,11 +325,14 @@ fn expr_dangling_with_subexpr_ident() {
#[test]
fn expr_reachable_subsequent_dangling() {
let id = SPair("foo".into(), S2);
#[rustfmt::skip]
let toks = vec![
// Reachable
Air::ExprStart(ExprOp::Sum, S1),
Air::BindIdent(id),
Air::BindIdent(id),
Air::ExprEnd(S3),
// Dangling
Air::ExprStart(ExprOp::Sum, S4),
Air::ExprEnd(S5),
@ -329,13 +343,16 @@ fn expr_reachable_subsequent_dangling() {
let second_span = S4.merge(S5).unwrap();
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Err(ParseError::StateError(AsgError::DanglingExpr(second_span))),
// Reachable
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
// Dangling
Ok(Parsed::Incomplete),
Err(ParseError::StateError(AsgError::DanglingExpr(second_span))),
// RECOVERY
Ok(Parsed::Incomplete), // PkgEnd
],
@ -347,13 +364,16 @@ fn expr_reachable_subsequent_dangling() {
#[test]
fn recovery_expr_reachable_after_dangling() {
let id = SPair("foo".into(), S4);
#[rustfmt::skip]
let toks = vec![
// Dangling
Air::ExprStart(ExprOp::Sum, S1),
Air::ExprEnd(S2),
// Reachable, after error from dangling.
Air::ExprStart(ExprOp::Sum, S3),
Air::BindIdent(id),
Air::BindIdent(id),
Air::ExprEnd(S5),
];
@ -363,14 +383,16 @@ fn recovery_expr_reachable_after_dangling() {
let mut sut = parse_as_pkg_body(toks);
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete),
Err(ParseError::StateError(AsgError::DanglingExpr(err_span))),
// RECOVERY: continue at this point with the next expression.
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Err(ParseError::StateError(AsgError::DanglingExpr(err_span))),
// RECOVERY: continue at this point with the next expression.
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete), // PkgEnd
],
sut.by_ref().collect::<Vec<_>>(),
@ -396,14 +418,17 @@ fn recovery_expr_reachable_after_dangling() {
fn expr_close_unbalanced() {
let id = SPair("foo".into(), S3);
#[rustfmt::skip]
let toks = vec![
// Close before _any_ open.
Air::ExprEnd(S1),
// Should recover,
// allowing for a normal expr.
Air::ExprStart(ExprOp::Sum, S2),
Air::BindIdent(id),
Air::BindIdent(id),
Air::ExprEnd(S4),
// And now an extra close _after_ a valid expr.
Air::ExprEnd(S5),
];
@ -411,15 +436,18 @@ fn expr_close_unbalanced() {
let mut sut = parse_as_pkg_body(toks);
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Err(ParseError::StateError(AsgError::UnbalancedExpr(S1))),
// RECOVERY
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
// Another error after a successful expression.
Err(ParseError::StateError(AsgError::UnbalancedExpr(S5))),
Err(ParseError::StateError(AsgError::UnbalancedExpr(S1))),
// RECOVERY
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
// Another error after a successful expression.
Err(ParseError::StateError(AsgError::UnbalancedExpr(S5))),
// RECOVERY
Ok(Parsed::Incomplete), // PkgEnd
],
@ -440,20 +468,24 @@ fn expr_bind_to_empty() {
let id_good = SPair("good".into(), S6);
let id_noexpr_b = SPair("noexpr_b".into(), S8);
#[rustfmt::skip]
let toks = vec![
// We need to first bring ourselves out of the context of the
// package header,
// otherwise the bind will be interpreted as a bind to the
// package itself.
Air::ExprStart(ExprOp::Sum, S1),
Air::BindIdent(id_pre),
Air::BindIdent(id_pre),
Air::ExprEnd(S3),
// No open expression to bind to.
Air::BindIdent(id_noexpr_a),
// Post-recovery create an expression.
Air::ExprStart(ExprOp::Sum, S5),
Air::BindIdent(id_good),
Air::BindIdent(id_good),
Air::ExprEnd(S7),
// Once again we have nothing to bind to.
Air::BindIdent(id_noexpr_b),
];
@ -461,28 +493,32 @@ fn expr_bind_to_empty() {
let mut sut = parse_as_pkg_body(toks);
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
// Just to get out of a package header context
Ok(Parsed::Incomplete), // ExprStart (pre)
Ok(Parsed::Incomplete), // BindIdent (pre)
Ok(Parsed::Incomplete), // ExprEnd (pre)
// Now that we've encountered an expression,
// we want an error specific to expression binding,
// since it's likely that a bind token was issued too late,
// rather than trying to interpret this as being back in a
// package context and binding to the package.
Err(ParseError::StateError(AsgError::InvalidExprBindContext(
id_noexpr_a
))),
// RECOVERY
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
// Another error after a successful expression.
Err(ParseError::StateError(AsgError::InvalidExprBindContext(
id_noexpr_b
))),
// Just to get out of a package header context
Ok(Parsed::Incomplete), // ExprStart (pre)
Ok(Parsed::Incomplete), // BindIdent (pre)
Ok(Parsed::Incomplete), // ExprEnd (pre)
// Now that we've encountered an expression,
// we want an error specific to expression binding,
// since it's likely that a bind token was issued too late,
// rather than trying to interpret this as being back in a
// package context and binding to the package.
Err(ParseError::StateError(AsgError::InvalidExprBindContext(
id_noexpr_a
))),
// RECOVERY
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
// Another error after a successful expression.
Err(ParseError::StateError(AsgError::InvalidExprBindContext(
id_noexpr_b
))),
// RECOVERY
Ok(Parsed::Incomplete), // PkgEnd
],
@ -510,19 +546,23 @@ fn expr_bind_to_empty() {
fn sibling_subexprs_have_ordered_edges_to_parent() {
let id_root = SPair("root".into(), S1);
#[rustfmt::skip]
let toks = vec![
Air::ExprStart(ExprOp::Sum, S1),
// Identify the root so that it is not dangling.
Air::BindIdent(id_root),
// Sibling A
Air::ExprStart(ExprOp::Sum, S3),
Air::ExprEnd(S4),
// Sibling B
Air::ExprStart(ExprOp::Sum, S5),
Air::ExprEnd(S6),
// Sibling C
Air::ExprStart(ExprOp::Sum, S7),
Air::ExprEnd(S8),
// Identify the root so that it is not dangling.
Air::BindIdent(id_root),
// Sibling A
Air::ExprStart(ExprOp::Sum, S3),
Air::ExprEnd(S4),
// Sibling B
Air::ExprStart(ExprOp::Sum, S5),
Air::ExprEnd(S6),
// Sibling C
Air::ExprStart(ExprOp::Sum, S7),
Air::ExprEnd(S8),
Air::ExprEnd(S9),
];
@ -555,14 +595,17 @@ fn nested_subexprs_related_to_relative_parent() {
let id_root = SPair("root".into(), S1);
let id_suba = SPair("suba".into(), S2);
#[rustfmt::skip]
let toks = vec![
Air::ExprStart(ExprOp::Sum, S1), // 0
Air::BindIdent(id_root),
Air::ExprStart(ExprOp::Sum, S2), // 1
Air::BindIdent(id_suba),
Air::ExprStart(ExprOp::Sum, S3), // 2
Air::ExprEnd(S4),
Air::ExprEnd(S5),
Air::BindIdent(id_root),
Air::ExprStart(ExprOp::Sum, S2), // 1
Air::BindIdent(id_suba),
Air::ExprStart(ExprOp::Sum, S3), // 2
Air::ExprEnd(S4),
Air::ExprEnd(S5),
Air::ExprEnd(S6),
];
@ -591,30 +634,33 @@ fn expr_redefine_ident() {
let id_first = SPair("foo".into(), S2);
let id_dup = SPair("foo".into(), S3);
#[rustfmt::skip]
let toks = vec![
Air::ExprStart(ExprOp::Sum, S1),
Air::BindIdent(id_first),
Air::ExprStart(ExprOp::Sum, S3),
Air::BindIdent(id_dup),
Air::ExprEnd(S4),
Air::BindIdent(id_first),
Air::ExprStart(ExprOp::Sum, S3),
Air::BindIdent(id_dup),
Air::ExprEnd(S4),
Air::ExprEnd(S5),
];
let mut sut = parse_as_pkg_body(toks);
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent (first)
Ok(Parsed::Incomplete), // ExprStart
Err(ParseError::StateError(AsgError::IdentRedefine(
id_first,
id_dup.span(),
))),
// RECOVERY: Ignore the attempt to redefine and continue.
Ok(Parsed::Incomplete), // ExprEnd
Ok(Parsed::Incomplete), // ExprEnd
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent (first)
Ok(Parsed::Incomplete), // ExprStart
Err(ParseError::StateError(AsgError::IdentRedefine(
id_first,
id_dup.span(),
))),
// RECOVERY: Ignore the attempt to redefine and continue.
Ok(Parsed::Incomplete), // ExprEnd
Ok(Parsed::Incomplete), // ExprEnd
Ok(Parsed::Incomplete), // PkgEnd
],
sut.by_ref().collect::<Vec<_>>(),
@ -640,58 +686,64 @@ fn expr_still_dangling_on_redefine() {
let id_dup2 = SPair("foo".into(), S8);
let id_second = SPair("bar".into(), S9);
#[rustfmt::skip]
let toks = vec![
// First expr (OK)
Air::ExprStart(ExprOp::Sum, S1),
Air::BindIdent(id_first),
Air::BindIdent(id_first),
Air::ExprEnd(S3),
// Second expr should still dangle due to use of duplicate
// identifier
Air::ExprStart(ExprOp::Sum, S4),
Air::BindIdent(id_dup),
Air::BindIdent(id_dup),
Air::ExprEnd(S6),
// Third expr will error on redefine but then be successful.
// This probably won't happen in practice with TAME's original
// source language,
// but could happen at e.g. a REPL.
Air::ExprStart(ExprOp::Sum, S7),
Air::BindIdent(id_dup2), // fail
Air::BindIdent(id_second), // succeed
Air::BindIdent(id_dup2), // fail
Air::BindIdent(id_second), // succeed
Air::ExprEnd(S10),
];
let mut sut = parse_as_pkg_body(toks);
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent (first)
Ok(Parsed::Incomplete), // ExprEnd
// Beginning of second expression
Ok(Parsed::Incomplete), // ExprStart
Err(ParseError::StateError(AsgError::IdentRedefine(
id_first,
id_dup.span(),
))),
// RECOVERY: Ignore the attempt to redefine and continue.
// ...but then immediately fail _again_ because we've closed a
// dangling expression.
Err(ParseError::StateError(AsgError::DanglingExpr(
S4.merge(S6).unwrap()
))),
// RECOVERY: But we'll continue onto one final expression,
// which we will fail to define but then subsequently define
// successfully.
Ok(Parsed::Incomplete), // ExprStart
Err(ParseError::StateError(AsgError::IdentRedefine(
id_first,
id_dup2.span(),
))),
// RECOVERY: Despite the initial failure,
// we can now re-attempt to bind with a unique id.
Ok(Parsed::Incomplete), // BindIdent (second)
Ok(Parsed::Incomplete), // ExprEnd
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent (first)
Ok(Parsed::Incomplete), // ExprEnd
// Beginning of second expression
Ok(Parsed::Incomplete), // ExprStart
Err(ParseError::StateError(AsgError::IdentRedefine(
id_first,
id_dup.span(),
))),
// RECOVERY: Ignore the attempt to redefine and continue.
// ...but then immediately fail _again_ because we've closed a
// dangling expression.
Err(ParseError::StateError(AsgError::DanglingExpr(
S4.merge(S6).unwrap()
))),
// RECOVERY: But we'll continue onto one final expression,
// which we will fail to define but then subsequently define
// successfully.
Ok(Parsed::Incomplete), // ExprStart
Err(ParseError::StateError(AsgError::IdentRedefine(
id_first,
id_dup2.span(),
))),
// RECOVERY: Despite the initial failure,
// we can now re-attempt to bind with a unique id.
Ok(Parsed::Incomplete), // BindIdent (second)
Ok(Parsed::Incomplete), // ExprEnd
Ok(Parsed::Incomplete), // PkgEnd
],
sut.by_ref().collect::<Vec<_>>(),
@ -717,20 +769,23 @@ fn expr_ref_to_ident() {
let id_foo = SPair("foo".into(), S2);
let id_bar = SPair("bar".into(), S6);
#[rustfmt::skip]
let toks = vec![
Air::ExprStart(ExprOp::Sum, S1),
Air::BindIdent(id_foo),
// Reference to an as-of-yet-undefined id (okay),
// with a different span than `id_bar`.
Air::RefIdent(SPair("bar".into(), S3)),
Air::BindIdent(id_foo),
// Reference to an as-of-yet-undefined id (okay),
// with a different span than `id_bar`.
Air::RefIdent(SPair("bar".into(), S3)),
Air::ExprEnd(S4),
//
// Another expression to reference the first
// (we don't handle cyclic references until a topological sort,
// so no point in referencing ourselves;
// it'd work just fine here.)
Air::ExprStart(ExprOp::Sum, S5),
Air::BindIdent(id_bar),
Air::BindIdent(id_bar),
Air::ExprEnd(S7),
];
@ -746,12 +801,6 @@ fn expr_ref_to_ident() {
// We should have only a single reference (to `id_bar`).
assert_eq!(foo_rels.len(), 1);
// Ontological sanity check:
// references to identifiers should count as cross edges.
// This is very important to ensure that certain graph traversals work
// correctly between trees.
assert!(foo_rels.iter().all(|rel| rel.is_cross_edge()));
let oi_ident_bar =
foo_rels.pop().and_then(ExprRel::narrow::<Ident>).unwrap();
let ident_bar = oi_ident_bar.resolve(&asg);
@ -776,36 +825,42 @@ fn expr_ref_outside_of_expr_context() {
let id_pre = SPair("pre".into(), S2);
let id_foo = SPair("foo".into(), S4);
#[rustfmt::skip]
let toks = vec![
// We need to first bring ourselves out of the context of the
// package header.
Air::ExprStart(ExprOp::Sum, S1),
Air::BindIdent(id_pre),
Air::BindIdent(id_pre),
Air::ExprEnd(S3),
// This will fail since we're not in an expression context.
Air::RefIdent(id_foo),
// RECOVERY: Simply ignore the above.
Air::ExprStart(ExprOp::Sum, S1),
Air::BindIdent(id_foo),
Air::BindIdent(id_foo),
Air::ExprEnd(S3),
];
let mut sut = parse_as_pkg_body(toks);
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
// Now we're past the header and in expression parsing mode.
Err(ParseError::StateError(AsgError::InvalidExprRefContext(
id_foo
))),
// RECOVERY: Proceed as normal
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
// Now we're past the header and in expression parsing mode.
Err(ParseError::StateError(AsgError::InvalidExprRefContext(
id_foo
))),
// RECOVERY: Proceed as normal
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
Ok(Parsed::Incomplete), // PkgEnd
],
sut.by_ref().collect::<Vec<_>>(),
@ -826,15 +881,17 @@ fn idents_share_defining_pkg() {
let id_baz = SPair("baz".into(), S6);
// An expression nested within another.
#[rustfmt::skip]
let toks = vec![
Air::PkgStart(S1),
Air::ExprStart(ExprOp::Sum, S2),
Air::BindIdent(id_foo),
Air::ExprStart(ExprOp::Sum, S4),
Air::BindIdent(id_bar),
Air::RefIdent(id_baz),
Air::ExprEnd(S7),
Air::ExprEnd(S8),
Air::ExprStart(ExprOp::Sum, S2),
Air::BindIdent(id_foo),
Air::ExprStart(ExprOp::Sum, S4),
Air::BindIdent(id_bar),
Air::RefIdent(id_baz),
Air::ExprEnd(S7),
Air::ExprEnd(S8),
Air::PkgEnd(S9),
];
@ -842,22 +899,16 @@ fn idents_share_defining_pkg() {
assert!(sut.all(|x| x.is_ok()));
let asg = sut.finalize().unwrap().into_context();
let oi_foo = asg.lookup(id_foo).unwrap();
let oi_bar = asg.lookup(id_bar).unwrap();
let oi_foo = asg.lookup_global(id_foo).unwrap();
let oi_bar = asg.lookup_global(id_bar).unwrap();
assert_eq!(oi_foo.src_pkg(&asg).unwrap(), oi_bar.src_pkg(&asg).unwrap());
// Missing identifiers should not have a source package,
// since we don't know what defined it yet.
let oi_baz = asg.lookup(id_baz).unwrap();
let oi_baz = asg.lookup_global(id_baz).unwrap();
assert_eq!(None, oi_baz.src_pkg(&asg));
// Ontological sanity check:
// edges from the package to identifiers defined by it should not be
// considered cross edges.
let oi_pkg = oi_foo.src_pkg(&asg).unwrap();
assert!(oi_pkg.edges(&asg).all(|rel| !rel.is_cross_edge()));
// The package span should encompass the entire definition.
assert_eq!(
S1.merge(S9),

View File

@ -628,13 +628,47 @@ sum_ir! {
/// metasyntactic variables,
/// defined by [`AirTpl::TplMetaStart`].
///
/// If a template contains any metavariables (parameters) without
/// bound values,
/// those metavariables are said to be _free_.
/// Values may be applied to templates using [`AirTpl::TplApply`],
/// binding those values to their associated metavariables.
/// A template with no free metavariables is _closed_ and may be
/// expanded in-place.
/// Template Application
/// ====================
/// _Application_ is triggered by a [`Air::RefIdent`] to a [`Tpl`]
/// or a [`Air::TplEndRef`] and will re-bind an inner template to
/// the metavariables in the current context and expand the
/// template in place.
/// A metavariable is _free_ if it has no bound value.
/// A template is _closed_ if it has no free metavariables.
/// Application _expands_ into its context,
/// and application of a closed template is equivalent to the
/// notion of template application in NIR / TAME's source language.
///
/// Let α be the current template definition context
/// (via [`Air::TplStart`])
/// and let β be the inner template.
/// All free metavariables in β that contain default values in α
/// (via [`Air::TplMetaStart`])
/// corresponding to the same [`Ident`] will be _bound_ to
/// that value.
/// The body of the inner template β will be expanded into the
/// body of α.
/// The result is a template α which is the application of its
/// parameters to β.
///
/// Partial application is not yet supported,
/// but can be added if it is worth the effort of doing so.
/// This simplifies the semantics of this operation:
///
/// - All metavariables that are still free in β after binding
/// will assume their default values, if any; and
/// - All metavariables that are still free in β after
/// applying defaults will result in an error.
///
/// Consequently,
/// the template α will always be closed after this operation.
/// This can be thought of like a lexical thunk.
///
/// α can then be expanded in place using [`Air::TplEndRef`] if
/// it is anonymous.
/// If α is named,
/// [`Air::RefIdent`] can be used to trigger expansion.
enum AirTpl {
/// Create a new [`Tpl`] on the graph and switch to template parsing.
///
@ -657,14 +691,14 @@ sum_ir! {
/// [`AirBind::BindIdent`] before [`Self::TplMetaEnd`].
///
/// Metavariables may contain default values,
/// making their specification during [`Self::TplApply`] optional.
/// making their specification during application optional.
/// A metavariable may contain an ordered mixture of references
/// to another metavariables via [`AirBind::RefIdent`] and
/// literals via [`Self::TplLexeme`].
/// Once all metavariable references have been satisfied during
/// [`Self::TplApply`],
/// application,
/// all children will be combined into a single lexeme to
/// serve as a final identifier.
/// serve as a final identifier.
///
/// The interpretation of a metavariable depends solely on the
/// context in which it is referenced.
@ -694,43 +728,6 @@ sum_ir! {
),
},
/// Re-bind an inner template to the metavariables in the
/// current context and expand the template in place.
///
/// Let α be the current template definition context
/// (via [`Self::TplStart`])
/// and let β be the inner template.
/// All free metavariables in β that contain default values in α
/// (via [`Self::TplMetaStart`])
/// corresponding to the same [`Ident`] will be _bound_ to
/// that value.
/// The body of the inner template β will be expanded into the
/// body of α.
/// The result is a template α which is the application of its
/// parameters to β.
///
/// Partial application is not yet supported,
/// but can be added if it is worth the effort of doing so.
/// This simplifies the semantics of this operation:
///
/// - All metavariables that are still free in β after binding
/// will assume their default values, if any; and
/// - All metavariables that are still free in β after
/// applying defaults will result in an error.
///
/// Consequently,
/// the template α will always be closed after this operation.
/// This can be thought of like a lexical thunk.
///
/// α can then be expanded in place using [`Self::TplEndRef`] if
/// it is anonymous.
/// If α is named,
/// [`Air::RefIdent`] can be used to trigger expansion.
TplApply(span: Span) => {
span: span,
display: |f| write!(f, "apply param bindings to inner template"),
},
/// Complete the active [`Tpl`] and exit template parsing.
///
/// The expression stack will be restored to its prior state.

View File

@ -46,7 +46,9 @@ fn ident_decl() {
let asg = sut.finalize().unwrap().into_context();
let ident_node = asg.lookup(id).expect("identifier was not added to graph");
let ident_node = asg
.lookup_global(id)
.expect("identifier was not added to graph");
let ident = asg.get(ident_node).unwrap();
assert_eq!(
@ -85,7 +87,9 @@ fn ident_extern_decl() {
let asg = sut.finalize().unwrap().into_context();
let ident_node = asg.lookup(id).expect("identifier was not added to graph");
let ident_node = asg
.lookup_global(id)
.expect("identifier was not added to graph");
let ident = asg.get(ident_node).unwrap();
assert_eq!(
@ -122,8 +126,10 @@ fn ident_dep() {
let asg = sut.finalize().unwrap().into_context();
let ident_node = asg.lookup(id).expect("identifier was not added to graph");
let dep_node = asg.lookup(dep).expect("dep was not added to graph");
let ident_node = asg
.lookup_global(id)
.expect("identifier was not added to graph");
let dep_node = asg.lookup_global(dep).expect("dep was not added to graph");
assert!(asg.has_dep(ident_node, dep_node));
}
@ -153,7 +159,9 @@ fn ident_fragment() {
let asg = sut.finalize().unwrap().into_context();
let ident_node = asg.lookup(id).expect("identifier was not added to graph");
let ident_node = asg
.lookup_global(id)
.expect("identifier was not added to graph");
let ident = asg.get(ident_node).unwrap();
assert_eq!(
@ -189,7 +197,7 @@ fn ident_root_missing() {
let asg = sut.finalize().unwrap().into_context();
let ident_node = asg
.lookup(id)
.lookup_global(id)
.expect("identifier was not added to the graph");
let ident = asg.get(ident_node).unwrap();
@ -228,7 +236,7 @@ fn ident_root_existing() {
let asg = sut.finalize().unwrap().into_context();
let ident_node = asg
.lookup(id)
.lookup_global(id)
.expect("identifier was not added to the graph");
let ident = asg.get(ident_node).unwrap();

View File

@ -31,6 +31,7 @@ use super::{
AirExprAggregate,
};
use crate::{
asg::graph::object::Meta,
diagnose::Annotate,
diagnostic_todo,
fmt::{DisplayWrapper, TtQuote},
@ -63,12 +64,26 @@ pub enum AirTplAggregate {
/// which simplifies AIR generation.
Ready(ObjectIndex<Pkg>),
/// Toplevel template context.
///
/// Conceptually,
/// tokens that are received in this state are interpreted as directly
/// applying to the template itself,
/// or creating an object directly owned by the template.
Toplevel(
ObjectIndex<Pkg>,
TplState,
AirExprAggregateStoreDangling<Tpl>,
),
/// Defining a template metavariable.
TplMeta(
ObjectIndex<Pkg>,
TplState,
AirExprAggregateStoreDangling<Tpl>,
ObjectIndex<Meta>,
),
/// Aggregating tokens into a template.
TplExpr(
ObjectIndex<Pkg>,
@ -85,6 +100,10 @@ impl Display for AirTplAggregate {
Self::Toplevel(_, tpl, expr) | Self::TplExpr(_, tpl, expr) => {
write!(f, "building {tpl} with {expr}")
}
Self::TplMeta(_, tpl, _, _) => {
write!(f, "building {tpl} metavariable")
}
}
}
}
@ -191,7 +210,7 @@ impl ParseState for AirTplAggregate {
),
(Toplevel(oi_pkg, tpl, expr), AirBind(BindIdent(id))) => asg
.lookup_or_missing(id)
.lookup_global_or_missing(id)
.bind_definition(asg, id, tpl.oi())
.map(|oi_ident| oi_pkg.defines(asg, oi_ident))
.map(|_| ())
@ -202,20 +221,69 @@ impl ParseState for AirTplAggregate {
Transition(Toplevel(oi_pkg, tpl, expr)).incomplete()
}
(Toplevel(oi_pkg, tpl, expr), AirTpl(TplMetaStart(span))) => {
let oi_meta = asg.create(Meta::new_required(span));
Transition(TplMeta(oi_pkg, tpl, expr, oi_meta)).incomplete()
}
(
Toplevel(..),
tok @ AirTpl(TplMetaStart(..) | TplMetaEnd(..) | TplApply(..)),
TplMeta(oi_pkg, tpl, expr, oi_meta),
AirTpl(TplMetaEnd(cspan)),
) => {
oi_meta.close(asg, cspan);
Transition(Toplevel(oi_pkg, tpl, expr)).incomplete()
}
(
TplMeta(oi_pkg, tpl, expr, oi_meta),
AirTpl(TplLexeme(lexeme)),
) => Transition(TplMeta(
oi_pkg,
tpl,
expr,
oi_meta.assign_lexeme(asg, lexeme),
))
.incomplete(),
(TplMeta(oi_pkg, tpl, expr, oi_meta), AirBind(BindIdent(name))) => {
oi_meta.identify_as_tpl_param(asg, tpl.oi(), name);
Transition(TplMeta(oi_pkg, tpl, expr, oi_meta)).incomplete()
}
(TplMeta(..), tok @ AirBind(RefIdent(..))) => {
diagnostic_todo!(
vec![tok.note("for this token")],
"Toplevel meta"
vec![tok.note("this token")],
"AirBind in metavar context (param-value)"
)
}
(TplMeta(..), tok @ AirExpr(..)) => {
diagnostic_todo!(
vec![tok.note("this token")],
"AirExpr in metavar context (e.g. @values@)"
)
}
(
TplMeta(..),
tok @ AirTpl(
TplStart(..) | TplMetaStart(..) | TplEnd(..)
| TplEndRef(..),
),
) => {
diagnostic_todo!(vec![tok.note("this token")], "AirTpl variant")
}
(Toplevel(..), tok @ AirTpl(TplMetaEnd(..))) => {
diagnostic_todo!(
vec![tok.note("this token")],
"unbalanced meta"
)
}
(Toplevel(..), tok @ AirTpl(TplLexeme(..))) => {
diagnostic_todo!(
vec![tok.note("for this token")],
"err: Toplevel lexeme {tok:?} (must be within metavar)"
vec![tok.note("this token")],
"err: TplLexeme outside of metavar"
)
}
@ -275,10 +343,7 @@ impl ParseState for AirTplAggregate {
(
Ready(..) | TplExpr(..),
tok @ AirTpl(
TplMetaStart(..) | TplLexeme(..) | TplMetaEnd(..)
| TplApply(..),
),
tok @ AirTpl(TplMetaStart(..) | TplLexeme(..) | TplMetaEnd(..)),
) => {
diagnostic_todo!(
vec![tok.note("for this token")],

View File

@ -24,6 +24,7 @@ use crate::asg::{
test::{asg_from_toks, parse_as_pkg_body},
Air, AirAggregate,
},
graph::object::Meta,
Expr, ExprOp, Ident,
};
use crate::span::dummy::*;
@ -36,12 +37,13 @@ type Sut = AirAggregate;
fn tpl_defining_pkg() {
let id_tpl = SPair("_tpl_".into(), S3);
#[rustfmt::skip]
let toks = vec![
Air::PkgStart(S1),
// This also tests tpl as a transition away from the package header.
Air::TplStart(S2),
Air::BindIdent(id_tpl),
Air::TplEnd(S4),
// This also tests tpl as a transition away from the package header.
Air::TplStart(S2),
Air::BindIdent(id_tpl),
Air::TplEnd(S4),
Air::PkgEnd(S5),
];
@ -52,7 +54,7 @@ fn tpl_defining_pkg() {
let tpl = asg.expect_ident_obj::<Tpl>(id_tpl);
assert_eq!(S2.merge(S4).unwrap(), tpl.span());
let oi_id_tpl = asg.lookup(id_tpl).unwrap();
let oi_id_tpl = asg.lookup_global(id_tpl).unwrap();
assert_eq!(
S1.merge(S5),
oi_id_tpl.src_pkg(&asg).map(|pkg| pkg.resolve(&asg).span()),
@ -160,22 +162,24 @@ fn tpl_within_expr() {
fn close_tpl_without_open() {
let id_tpl = SPair("_tpl_".into(), S3);
#[rustfmt::skip]
let toks = vec![
Air::TplEnd(S1),
// RECOVERY: Try again.
Air::TplStart(S2),
Air::BindIdent(id_tpl),
Air::BindIdent(id_tpl),
Air::TplEnd(S4),
];
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Err(ParseError::StateError(AsgError::UnbalancedTpl(S1))),
// RECOVERY
Ok(Parsed::Incomplete), // TplStart
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // TplEnd
Err(ParseError::StateError(AsgError::UnbalancedTpl(S1))),
// RECOVERY
Ok(Parsed::Incomplete), // TplStart
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // TplEnd
Ok(Parsed::Incomplete), // PkgEnd
],
parse_as_pkg_body(toks).collect::<Vec<_>>(),
@ -194,10 +198,12 @@ fn tpl_with_reachable_expression() {
Air::BindIdent(id_tpl),
Air::ExprStart(ExprOp::Sum, S3),
// Must not be cached in the global env.
Air::BindIdent(id_expr_a),
Air::ExprEnd(S5),
Air::ExprStart(ExprOp::Sum, S6),
// Must not be cached in the global env.
Air::BindIdent(id_expr_b),
Air::ExprEnd(S8),
Air::TplEnd(S9),
@ -212,25 +218,41 @@ fn tpl_with_reachable_expression() {
// The inner expressions are reachable,
// but the intent is to expand them into the template's eventual
// application site.
// They have identifiers,
// but those identifiers _must not_ be cached in the global
// environment;
// such a determination will be made at expansion-time.
// Given that,
// they should be defined by the template...
assert_eq!(
vec![
asg.lookup(id_expr_b).unwrap(),
asg.lookup(id_expr_a).unwrap(),
// At the time of writing,
// this is implemented using the same `edges_filtered`,
// but the point is that we want to ensure that the
// identifiers bound to this template are only these.
oi_tpl.lookup_local_linear(&asg, id_expr_b),
oi_tpl.lookup_local_linear(&asg, id_expr_a),
],
oi_tpl.edges_filtered::<Ident>(&asg).collect::<Vec<_>>()
oi_tpl
.edges_filtered::<Ident>(&asg)
.map(Some)
.collect::<Vec<_>>()
);
// ...but not by the package containing the template.
let oi_pkg = asg.lookup(id_tpl).unwrap().src_pkg(&asg).unwrap();
let oi_pkg = asg.lookup_global(id_tpl).unwrap().src_pkg(&asg).unwrap();
assert_eq!(
vec![
// The only identifier on the package should be the template itself.
asg.lookup(id_tpl).unwrap(),
asg.lookup_global(id_tpl).unwrap(),
],
oi_pkg.edges_filtered::<Ident>(&asg).collect::<Vec<_>>()
);
// Verify the above claim that these are not cached in the global
// environment.
assert_eq!(None, asg.lookup_global(id_expr_a));
assert_eq!(None, asg.lookup_global(id_expr_b));
}
// Templates can expand into many contexts,
@ -339,16 +361,17 @@ fn unreachable_anonymous_tpl() {
let mut sut = parse_as_pkg_body(toks);
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete), // TplStart
Err(ParseError::StateError(AsgError::DanglingTpl(
S1.merge(S2).unwrap()
))),
// RECOVERY
Ok(Parsed::Incomplete), // TplStart
Ok(Parsed::Incomplete), // TplBindIdent
Ok(Parsed::Incomplete), // TplEnd
Ok(Parsed::Incomplete), // TplStart
Err(ParseError::StateError(AsgError::DanglingTpl(
S1.merge(S2).unwrap()
))),
// RECOVERY
Ok(Parsed::Incomplete), // TplStart
Ok(Parsed::Incomplete), // TplBindIdent
Ok(Parsed::Incomplete), // TplEnd
Ok(Parsed::Incomplete), // PkgEnd
],
sut.by_ref().collect::<Vec<_>>(),
@ -380,3 +403,48 @@ fn anonymous_tpl_immediate_ref() {
// TODO: More to come.
}
#[test]
fn tpl_with_param() {
let id_tpl = SPair("_tpl_".into(), S2);
let id_param1 = SPair("@param1@".into(), S4);
let pval1 = SPair("value1".into(), S5);
let id_param2 = SPair("@param2@".into(), S8);
#[rustfmt::skip]
let toks = vec![
Air::TplStart(S1),
Air::BindIdent(id_tpl),
// Metavariable with a value.
Air::TplMetaStart(S3),
Air::BindIdent(id_param1),
Air::TplLexeme(pval1),
Air::TplMetaEnd(S6),
// Required metavariable (no value).
Air::TplMetaStart(S7),
Air::BindIdent(id_param2),
Air::TplMetaEnd(S9),
Air::TplEnd(S10),
];
let asg = asg_from_toks(toks);
let oi_tpl = asg.expect_ident_oi::<Tpl>(id_tpl);
// The template should have an edge to each identifier for each
// metavariable.
let params = [id_param1, id_param2]
.iter()
.map(|id| {
oi_tpl
.lookup_local_linear(&asg, *id)
.and_then(|oi| oi.edges_filtered::<Meta>(&asg).next())
.map(ObjectIndex::cresolve(&asg))
})
.collect::<Vec<_>>();
assert_eq!(params[0], Some(&Meta::Lexeme(S3.merge(S6).unwrap(), pval1)));
assert_eq!(params[1], Some(&Meta::Required(S7.merge(S9).unwrap())));
}

View File

@ -232,11 +232,11 @@ impl Asg {
/// you must resolve the [`Ident`] object and inspect it.
///
/// See [`Ident::declare`] for more information.
pub(super) fn lookup_or_missing(
pub(super) fn lookup_global_or_missing(
&mut self,
ident: SPair,
) -> ObjectIndex<Ident> {
self.lookup(ident).unwrap_or_else(|| {
self.lookup_global(ident).unwrap_or_else(|| {
let index = self.graph.add_node(Ident::declare(ident).into());
self.index_identifier(ident.symbol(), index);
@ -247,13 +247,13 @@ impl Asg {
/// Perform a state transition on an identifier by name.
///
/// Look up `ident` or add a missing identifier if it does not yet exist
/// (see [`Self::lookup_or_missing`]).
/// (see [`Self::lookup_global_or_missing`]).
/// Then invoke `f` with the located identifier and replace the
/// identifier on the graph with the result.
///
/// This will safely restore graph state to the original identifier
/// value on transition failure.
fn with_ident_lookup<F>(
fn with_ident_lookup_global<F>(
&mut self,
name: SPair,
f: F,
@ -261,7 +261,7 @@ impl Asg {
where
F: FnOnce(Ident) -> TransitionResult<Ident>,
{
let identi = self.lookup_or_missing(name);
let identi = self.lookup_global_or_missing(name);
self.with_ident(identi, f)
}
@ -366,11 +366,13 @@ impl Asg {
) -> AsgResult<ObjectIndex<Ident>> {
let is_auto_root = kind.is_auto_root();
self.with_ident_lookup(name, |obj| obj.resolve(name.span(), kind, src))
.map(|node| {
is_auto_root.then(|| self.add_root(node));
node
})
self.with_ident_lookup_global(name, |obj| {
obj.resolve(name.span(), kind, src)
})
.map(|node| {
is_auto_root.then(|| self.add_root(node));
node
})
}
/// Declare an abstract identifier.
@ -399,7 +401,9 @@ impl Asg {
kind: IdentKind,
src: Source,
) -> AsgResult<ObjectIndex<Ident>> {
self.with_ident_lookup(name, |obj| obj.extern_(name.span(), kind, src))
self.with_ident_lookup_global(name, |obj| {
obj.extern_(name.span(), kind, src)
})
}
/// Set the fragment associated with a concrete identifier.
@ -412,7 +416,7 @@ impl Asg {
name: SPair,
text: FragmentText,
) -> AsgResult<ObjectIndex<Ident>> {
self.with_ident_lookup(name, |obj| obj.set_fragment(text))
self.with_ident_lookup_global(name, |obj| obj.set_fragment(text))
}
/// Create a new object on the graph.
@ -610,7 +614,7 @@ impl Asg {
&self,
ident: SPair,
) -> Option<ObjectIndex<O>> {
self.lookup(ident)
self.lookup_global(ident)
.and_then(|identi| {
self.graph
.neighbors_directed(identi.into(), Direction::Outgoing)
@ -636,13 +640,8 @@ impl Asg {
ident: SPair,
) -> ObjectIndex<O> {
self.get_ident_oi(ident).diagnostic_expect(
|| diagnostic_opaque_ident_desc(ident),
|| {
format!(
"opaque identifier: {} has no object binding",
TtQuote::wrap(ident),
)
},
|| diagnostic_unknown_ident_desc(ident),
|| format!("unknown identifier {}", TtQuote::wrap(ident),),
)
}
@ -715,14 +714,19 @@ impl Asg {
self.get(index)
}
/// Attempt to retrieve an identifier from the graph by name.
/// Attempt to retrieve an identifier from the graph by name utilizing
/// the global environment.
///
/// Since only identifiers carry a name,
/// this method cannot be used to retrieve all possible objects on the
/// graph---for
/// that, see [`Asg::get`].
///
/// The global environment is defined as the environment of the current
/// compilation unit,
/// which is a package.
#[inline]
pub fn lookup(&self, id: SPair) -> Option<ObjectIndex<Ident>> {
pub fn lookup_global(&self, id: SPair) -> Option<ObjectIndex<Ident>> {
let i = id.symbol().as_usize();
self.index
@ -737,7 +741,7 @@ impl Asg {
/// computed before computing the value of `ident`.
/// The [linker][crate::ld] will ensure this ordering.
///
/// See [`add_dep_lookup`][Asg::add_dep_lookup] if identifiers have to
/// See [`Self::add_dep_lookup_global`] if identifiers have to
/// be looked up by [`SymbolId`] or if they may not yet have been
/// declared.
pub fn add_dep<O: ObjectKind>(
@ -777,13 +781,13 @@ impl Asg {
/// See [`Ident::declare`] for more information.
///
/// References to both identifiers are returned in argument order.
pub fn add_dep_lookup(
pub fn add_dep_lookup_global(
&mut self,
ident: SPair,
dep: SPair,
) -> (ObjectIndex<Ident>, ObjectIndex<Ident>) {
let identi = self.lookup_or_missing(ident);
let depi = self.lookup_or_missing(dep);
let identi = self.lookup_global_or_missing(ident);
let depi = self.lookup_global_or_missing(dep);
self.graph.update_edge(
identi.into(),
@ -819,5 +823,15 @@ fn diagnostic_opaque_ident_desc(ident: SPair) -> Vec<AnnotatedSpan<'static>> {
]
}
fn diagnostic_unknown_ident_desc(ident: SPair) -> Vec<AnnotatedSpan<'static>> {
vec![
ident.internal_error("reference to an unknown identifier"),
ident.help(
"the system expects this identifier to be known, \
but it could not be found.",
),
]
}
#[cfg(test)]
mod test;

View File

@ -118,6 +118,7 @@ use crate::{
diagnose::{panic::DiagnosticPanic, Annotate, AnnotatedSpan},
diagnostic_panic,
f::Functor,
parse::util::SPair,
span::{Span, UNKNOWN_SPAN},
};
use petgraph::graph::NodeIndex;
@ -713,6 +714,79 @@ impl<O: ObjectKind> ObjectIndex<O> {
Self(index, span, _pd) => ObjectIndex::new(index, span),
}
}
/// Attempt to look up a locally bound [`Ident`] via a linear search of
/// `self`'s edges.
///
/// Performance
/// ===========
/// _This is a linear (O(1)) search of the edges of the node
/// corresponding to `self`!_
/// At the time of writing,
/// edges are stored using index references in a manner similar to a
/// linked list (petgraph).
/// And for each such edge,
/// the target object must be resolved so that its
/// [`SymbolId`](crate::sym::SymbolId) may be retrieved and compared
/// against the provided `name`.
///
/// If the number of edges is small and the objects are fairly localized
/// in memory relative to `self`,
/// then this may not be a concern.
/// However,
/// if you've arrived at this method while investigating unfavorable
/// circumstances during profiling,
/// then you should consider caching like the global environment
/// (see [`Asg::lookup_global`]).
pub fn lookup_local_linear(
&self,
asg: &Asg,
name: SPair,
) -> Option<ObjectIndex<Ident>>
where
O: ObjectRelTo<Ident>,
{
self.edges_filtered::<Ident>(asg)
.find(|oi| oi.resolve(asg).name().symbol() == name.symbol())
}
/// Declare a local identifier.
///
/// A local identifier is lexically scoped to `self`.
/// This operation is valid only for [`ObjectKind`]s that can contain
/// edges to [`Ident`]s.
///
/// TODO: This allows for duplicate local identifiers!
pub fn declare_local(
&self,
asg: &mut Asg,
name: SPair,
) -> ObjectIndex<Ident>
where
O: ObjectRelTo<Ident>,
{
asg.create(Ident::declare(name))
.add_edge_from(asg, *self, None)
}
/// Retrieve the identifier for this object,
/// if any.
///
/// If there is more than one identifier,
/// only one will be returned,
/// and the result of the operation is undefined.
/// This can be problematic if certain optimizations have been performed
/// on the graph,
/// like common subexpression elimination,
/// in which case it's best not to rely on following edges in reverse.
pub fn ident<'a>(&self, asg: &'a Asg) -> Option<&'a Ident>
where
O: ObjectRelFrom<Ident>,
{
self.incoming_edges_filtered(asg)
.map(ObjectIndex::cresolve(asg))
.next()
}
}
impl ObjectIndex<Object> {

View File

@ -246,15 +246,7 @@ impl ObjectIndex<Expr> {
/// a missing identifier will take its place to be later resolved once
/// it becomes available.
pub fn ref_expr(self, asg: &mut Asg, ident: SPair) -> Self {
let identi = asg.lookup_or_missing(ident);
let identi = asg.lookup_global_or_missing(ident);
self.add_edge_to(asg, identi, Some(ident.span()))
}
/// The [`Ident`] bound to this expression,
/// if any.
pub fn ident(self, asg: &Asg) -> Option<&Ident> {
self.incoming_edges_filtered(asg)
.map(ObjectIndex::cresolve(asg))
.next()
}
}

View File

@ -982,9 +982,7 @@ object_rel! {
tree Ident,
tree Expr,
tree Tpl,
// A metavariable is directly referenced by a template.
cross Meta,
tree Meta,
}
}
@ -1060,6 +1058,14 @@ impl ObjectIndex<Ident> {
self.edges(asg).find_map(ObjectRel::narrow) == Some(oi)
}
/// Whether this identifier is bound to an object of kind `O`.
///
/// To bind an identifier,
/// see [`Self::bind_definition`].
pub fn is_bound_to_kind<O: ObjectRelFrom<Ident>>(&self, asg: &Asg) -> bool {
self.edges_filtered::<O>(asg).next().is_some()
}
/// The source package defining this identifier,
/// if known.
pub fn src_pkg(&self, asg: &Asg) -> Option<ObjectIndex<Pkg>> {

View File

@ -26,11 +26,15 @@
use super::{
Ident, Object, ObjectIndex, ObjectRel, ObjectRelFrom, ObjectRelTy,
ObjectRelatable,
ObjectRelatable, Tpl,
};
use crate::{
asg::Asg,
diagnose::Annotate,
diagnostic_todo,
f::Functor,
fmt::{DisplayWrapper, TtQuote},
parse::{util::SPair, Token},
parse::util::SPair,
span::Span,
};
use std::fmt::Display;
@ -51,16 +55,54 @@ use std::fmt::Display;
pub enum Meta {
Required(Span),
ConcatList(Span),
Lexeme(SPair),
Lexeme(Span, SPair),
}
impl Meta {
/// Create a new metavariable without a value.
///
/// Metavariables with no value cannot be used in an expansion context.
/// Intuitively,
/// they act as required parameters.
pub fn new_required(span: Span) -> Self {
Self::Required(span)
}
pub fn span(&self) -> Span {
match self {
Self::Required(span) | Self::ConcatList(span) => *span,
Self::Lexeme(spair) => spair.span(),
Self::Required(span)
| Self::ConcatList(span)
| Self::Lexeme(span, _) => *span,
}
}
/// Assign a lexeme to a metavariable.
///
/// In a template definition context,
/// this acts as a default value for this metavariable.
/// In an application context,
/// this has the effect of binding a value to this metavariable.
pub fn assign_lexeme(self, lexeme: SPair) -> Self {
match self {
Self::Required(span) => Self::Lexeme(span, lexeme),
Self::ConcatList(_) => diagnostic_todo!(
vec![lexeme.note("while parsing this lexeme")],
"append to ConcatList",
),
Self::Lexeme(_, _) => diagnostic_todo!(
vec![lexeme.note("while parsing this lexeme")],
"Lexeme => ConcatList",
),
}
}
}
impl From<&Meta> for Span {
fn from(meta: &Meta) -> Self {
meta.span()
}
}
impl Display for Meta {
@ -72,7 +114,19 @@ impl Display for Meta {
Self::ConcatList(_) => {
write!(f, "metasyntactic concatenation list")
}
Self::Lexeme(spair) => write!(f, "lexeme {}", TtQuote::wrap(spair)),
Self::Lexeme(_, spair) => {
write!(f, "lexeme {}", TtQuote::wrap(spair))
}
}
}
}
impl Functor<Span> for Meta {
fn map(self, f: impl FnOnce(Span) -> Span) -> Self::Target {
match self {
Self::Required(span) => Self::Required(f(span)),
Self::ConcatList(span) => Self::ConcatList(f(span)),
Self::Lexeme(span, spair) => Self::Lexeme(f(span), spair),
}
}
}
@ -85,3 +139,28 @@ object_rel! {
cross Ident,
}
}
impl ObjectIndex<Meta> {
pub fn identify_as_tpl_param(
&self,
asg: &mut Asg,
oi_tpl: ObjectIndex<Tpl>,
name: SPair,
) -> ObjectIndex<Ident> {
oi_tpl
.declare_local(asg, name)
.add_edge_to(asg, *self, None)
}
pub fn assign_lexeme(self, asg: &mut Asg, lexeme: SPair) -> Self {
self.map_obj(asg, |meta| meta.assign_lexeme(lexeme))
}
pub fn close(self, asg: &mut Asg, close_span: Span) -> Self {
self.map_obj(asg, |meta| {
meta.map(|open_span| {
open_span.merge(close_span).unwrap_or(open_span)
})
})
}
}

View File

@ -80,10 +80,14 @@ macro_rules! object_rel {
}
/// The root of the graph by definition has no cross edges.
fn is_cross_edge(&self) -> bool {
fn is_cross_edge<S, T>(
&self,
#[allow(unused_variables)] // used only for `dyn` edges
rel: &$crate::asg::graph::object::rel::DynObjectRel<S, T>
) -> bool {
match self {
$(
Self::$kind(..) => object_rel!(@is_cross_edge $ety),
Self::$kind(..) => object_rel!(@is_cross_edge $ety rel),
)*
#[allow(unreachable_patterns)] // for empty Rel types
@ -126,8 +130,21 @@ macro_rules! object_rel {
)*
}};
(@is_cross_edge cross) => { true };
(@is_cross_edge tree) => { false };
// Static edge types.
(@is_cross_edge cross $_:ident) => { true };
(@is_cross_edge tree $_:ident) => { false };
// Dynamic edge type.
//
// We consider an edge to be a cross edge iff it contains a context
// span.
// If it were _not_ a cross edge,
// then the edge would represent ownership,
// and the span information on the target would be sufficient context
// on its own;
// an edge only needs supplemental span information if there is
// another context in which that object is referenced.
(@is_cross_edge dyn $rel:ident) => { $rel.ctx_span().is_some() };
}
/// A dynamic relationship (edge) from one object to another before it has
@ -248,7 +265,7 @@ impl<S> DynObjectRel<S, ObjectIndex<Object>> {
$(
ObjectRelTy::$ty => {
self.narrow_target::<$ty>().is_some_and(
|rel| rel.is_cross_edge()
|rel| rel.is_cross_edge(self)
)
},
)*
@ -509,5 +526,38 @@ pub trait ObjectRel<OA: ObjectKind + ObjectRelatable>: Sized {
/// then do _not_ assume that it is a cross edge without further
/// analysis,
/// which may require introducing more context to this method.
fn is_cross_edge(&self) -> bool;
///
/// Dynamic Cross Edges
/// ==================
/// Sometimes a cross edge cannot be determined statically due to
/// ontological compromises.
/// For example,
/// a [`Tpl`]->[`Ident`] edge could be a reference to another tree,
/// as in the case of template application,
/// or the template could be serving as a container for that
/// definition.
/// The former case is a cross edge,
/// but the ladder is not.
///
/// We could introduce a layer of indirection on the graph to
/// disambiguate,
/// but that adds complexity to the graph that many different
/// subsystems need to concern themselves with.
/// But cross edge determination is fairly isolated,
/// needed only by traversals.
///
/// Therefore,
/// this method also receives a [`DynObjectRel`] reference,
/// which contains edge information.
/// The edge can be augmented with data that helps to determine,
/// at runtime,
/// whether that particular edge is a cross edge.
///
/// The use of [`DynObjectRel`] is heavy and effectively makes this
/// method useless for callers that do not deal directly with raw
/// [`Asg`] data;
/// it may be useful to refine this further in the future to correct
/// that,
/// once all use cases are clear.
fn is_cross_edge<S, T>(&self, rel: &DynObjectRel<S, T>) -> bool;
}

View File

@ -22,10 +22,15 @@
use std::fmt::Display;
use super::{
Expr, Ident, Meta, Object, ObjectIndex, ObjectRel, ObjectRelFrom,
ObjectRelTo, ObjectRelTy, ObjectRelatable,
Expr, Ident, Object, ObjectIndex, ObjectRel, ObjectRelFrom, ObjectRelTo,
ObjectRelTy, ObjectRelatable,
};
use crate::{
asg::Asg,
f::Functor,
parse::{util::SPair, Token},
span::Span,
};
use crate::{asg::Asg, f::Functor, parse::util::SPair, span::Span};
/// Template with associated name.
#[derive(Debug, PartialEq, Eq)]
@ -61,9 +66,13 @@ object_rel! {
/// Templates may expand into nearly any context,
/// and must therefore be able to contain just about anything.
Tpl -> {
tree Ident,
// Expressions must be able to be anonymous to allow templates in
// any `Expr` context.
tree Expr,
tree Meta,
// Identifiers are used for both references and identifiers that
// will expand into an application site.
dyn Ident,
}
}
@ -86,9 +95,8 @@ impl ObjectIndex<Tpl> {
/// this application will expand the template in place,
/// re-binding metavariables to the context of `self`.
pub fn apply_named_tpl(self, asg: &mut Asg, id: SPair) -> Self {
let oi_apply = asg.lookup_or_missing(id);
// TODO: span
self.add_edge_to(asg, oi_apply, None)
let oi_apply = asg.lookup_global_or_missing(id);
self.add_edge_to(asg, oi_apply, Some(id.span()))
}
/// Directly reference this template from another object

View File

@ -142,7 +142,7 @@ fn lookup_by_symbol() -> AsgResult<()> {
},
)?;
assert_eq!(Some(node), sut.lookup(id));
assert_eq!(Some(node), sut.lookup_global(id));
Ok(())
}
@ -311,7 +311,7 @@ fn add_dep_lookup_existing() -> AsgResult<()> {
let _ = sut.declare(sym, IdentKind::Meta, Source::default())?;
let _ = sut.declare(dep, IdentKind::Meta, Source::default())?;
let (symnode, depnode) = sut.add_dep_lookup(sym, dep);
let (symnode, depnode) = sut.add_dep_lookup_global(sym, dep);
assert!(sut.has_dep(symnode, depnode));
Ok(())
@ -325,7 +325,7 @@ fn add_dep_lookup_missing() -> AsgResult<()> {
let dep = SPair("dep".into(), S2);
// both of these are missing
let (symnode, depnode) = sut.add_dep_lookup(sym, dep);
let (symnode, depnode) = sut.add_dep_lookup_global(sym, dep);
assert!(sut.has_dep(symnode, depnode));
assert_eq!(sym, sut.get_ident(symnode).unwrap().name());
@ -342,7 +342,7 @@ fn declare_return_missing_symbol() -> AsgResult<()> {
let dep = SPair("dep".into(), S2);
// both of these are missing, see add_dep_lookup_missing
let (symnode, _) = sut.add_dep_lookup(sym, dep);
let (symnode, _) = sut.add_dep_lookup_global(sym, dep);
let src = Source {
desc: Some("redeclare missing".into()),

View File

@ -110,6 +110,13 @@ use super::object::ObjectRel;
/// may modify the graph beyond recognition,
/// though they should retain ordering where it is important.
///
/// _Objects that do not have a path from the root will not be visited by
/// this traversal._
/// These objects are expected to act as additional metadata,
/// and must be queried for explicitly.
/// Such querying can be done during the traversal since this visitor holds
/// only a shared immutable reference to the [`Asg`].
///
/// For more information,
/// see [`ObjectRel::is_cross_edge`].
///

View File

@ -36,13 +36,27 @@ use Air::*;
use ObjectRelTy::*;
const SU: Span = UNKNOWN_SPAN;
fn asg_from_toks<I: IntoIterator<Item = Air>>(toks: I) -> Asg
fn tree_reconstruction_report<I: IntoIterator<Item = Air>>(
toks: I,
) -> Vec<(DynObjectRel<Span, Span>, Depth)>
where
I::IntoIter: Debug,
{
let mut parser = AirAggregate::parse(toks.into_iter());
assert!(parser.all(|x| x.is_ok()));
parser.finalize().unwrap().into_context()
let asg = &parser.finalize().unwrap().into_context();
tree_reconstruction(asg)
.map(|TreeWalkRel(rel, depth)| {
(
rel.map(|(soi, toi)| {
(soi.resolve(asg).span(), toi.resolve(asg).span())
}),
depth,
)
})
.collect()
}
// Note that this is an integration test beginning at AIR.
@ -79,13 +93,6 @@ fn traverses_ontological_tree() {
PkgEnd(S11),
];
let asg = asg_from_toks(toks);
// From the above graph,
// we're going to traverse in such a way as to reconstruct the source
// tree.
let sut = tree_reconstruction(&asg);
// We need more concise expressions for the below table of values.
let d = DynObjectRel::new;
let m = |a: Span, b: Span| a.merge(b).unwrap();
@ -107,13 +114,7 @@ fn traverses_ontological_tree() {
(d(Pkg, Ident, m(S1, S11), S9, None ), Depth(2)),
(d(Ident, Expr, S9, m(S8, S10), None ), Depth(3)),
],
sut.map(|TreeWalkRel(rel, depth)| (
rel.map(|(soi, toi)| (
soi.resolve(&asg).span(),
toi.resolve(&asg).span()
)),
depth
)).collect::<Vec<_>>(),
tree_reconstruction_report(toks),
);
}
@ -156,9 +157,6 @@ fn traverses_ontological_tree_tpl_with_sibling_at_increasing_depth() {
PkgEnd(S10),
];
let asg = asg_from_toks(toks);
let sut = tree_reconstruction(&asg);
// We need more concise expressions for the below table of values.
let d = DynObjectRel::new;
let m = |a: Span, b: Span| a.merge(b).unwrap();
@ -184,12 +182,64 @@ fn traverses_ontological_tree_tpl_with_sibling_at_increasing_depth() {
(d(Tpl, Ident, m(S2, S9), S7, None), Depth(4)), // |
(d(Ident, Expr, S7, m(S6, S8), None), Depth(5)), // <'
],
sut.map(|TreeWalkRel(rel, depth)| (
rel.map(|(soi, toi)| (
soi.resolve(&asg).span(),
toi.resolve(&asg).span()
)),
depth
)).collect::<Vec<_>>(),
tree_reconstruction_report(toks),
);
}
// The way template applications are handled on the ASG differs from NIR.
// This test ensure that the representation on the ASG is precise;
// it's far easier to catch those problems here than it is to catch them
// in an implementation utilizing these data that's failing in some
// unexpected way.
#[test]
fn traverses_ontological_tree_tpl_apply() {
let name_tpl = "_tpl-to-apply_".into();
let id_tpl = SPair(name_tpl, S3);
let ref_tpl = SPair(name_tpl, S6);
let id_param = SPair("@param@".into(), S8);
let value_param = SPair("value".into(), S9);
#[rustfmt::skip]
let toks = vec![
PkgStart(S1),
// The template that will be applied.
TplStart(S2),
BindIdent(id_tpl),
// This test is light for now,
// until we develop the ASG further.
TplEnd(S4),
// Apply the above template.
// This seems like an odd construction until we introduce
// metavariables.
TplStart(S5),
RefIdent(ref_tpl),
TplMetaStart(S7),
BindIdent(id_param),
TplLexeme(value_param),
TplMetaEnd(S10),
TplEndRef(S11), // notice the `Ref` at the end
PkgEnd(S12),
];
// We need more concise expressions for the below table of values.
let d = DynObjectRel::new;
let m = |a: Span, b: Span| a.merge(b).unwrap();
#[rustfmt::skip]
assert_eq!(
// A -|-> B | A span -|-> B span | espan | depth
vec![//-----|-------|-----------|-----------|--------|-----------------
(d(Root, Pkg, SU, m(S1, S12), None ), Depth(1)),
(d(Pkg, Ident, m(S1, S12), S3, None ), Depth(2)),
(d(Ident, Tpl, S3, m(S2, S4), None ), Depth(3)),
(d(Pkg, Tpl, m(S1, S12), m(S5, S11), None ), Depth(2)),
/*cross*/ (d(Tpl, Ident, m(S5, S11), S3, Some(S6)), Depth(3)),
(d(Tpl, Ident, m(S5, S11), S8, None ), Depth(3)),
(d(Ident, Meta, S8, m(S7, S10), None ), Depth(4)),
],
tree_reconstruction_report(toks),
);
}

View File

@ -30,8 +30,8 @@
//! or observing template expansions.
use super::object::{
DynObjectRel, Expr, Object, ObjectIndex, ObjectRelTy, OiPairObjectInner,
Pkg, Tpl,
DynObjectRel, Expr, Meta, Object, ObjectIndex, ObjectRelTy,
OiPairObjectInner, Pkg, Tpl,
};
use crate::{
asg::{
@ -39,7 +39,8 @@ use crate::{
Asg, ExprOp, Ident,
},
diagnose::{panic::DiagnosticPanic, Annotate},
diagnostic_panic, diagnostic_unreachable,
diagnostic_panic, diagnostic_todo, diagnostic_unreachable,
fmt::{DisplayWrapper, TtQuote},
parse::{prelude::*, util::SPair, Transitionable},
span::{Span, UNKNOWN_SPAN},
sym::{
@ -198,7 +199,9 @@ impl<'a> TreeContext<'a> {
self.emit_template(tpl, *oi_tpl, paired_rel.source(), depth)
}
target @ Object::Meta(..) => todo!("Object::Meta: {target:?}"),
Object::Meta((meta, oi_meta)) => {
self.emit_tpl_arg(meta, *oi_meta, depth)
}
Object::Root(..) => diagnostic_unreachable!(
vec![],
@ -312,7 +315,11 @@ impl<'a> TreeContext<'a> {
// This also gives us the opportunity to make sure that
// we're deriving something that's actually supported by the
// XSLT-based compiler.
let mut idents = oi_tpl.edges_filtered::<Ident>(self.asg);
// TODO: This doesn't handle `Missing` or invalid (non-`Tpl`)
// identifiers.
let mut idents = oi_tpl
.edges_filtered::<Ident>(self.asg)
.filter(|oi| oi.is_bound_to_kind::<Tpl>(self.asg));
let apply_tpl = idents.next().diagnostic_expect(
|| {
@ -332,7 +339,7 @@ impl<'a> TreeContext<'a> {
bad_ident
.internal_error("unexpected second identifier"),
],
"expected only one Ident for template application",
"expected only one Ident->Tpl for template application",
);
}
@ -349,6 +356,47 @@ impl<'a> TreeContext<'a> {
}
}
/// Emit a long-form template argument.
///
/// For the parent template application,
/// see [`Self::emit_template`].
fn emit_tpl_arg(
&mut self,
meta: &Meta,
oi_meta: ObjectIndex<Meta>,
depth: Depth,
) -> Option<Xirf> {
let pname = oi_meta.ident(self.asg).map(Ident::name)
.diagnostic_unwrap(|| vec![meta.internal_error(
"anonymous metavariables are not supported as template arguments"
)]);
let pval = match meta {
Meta::Required(span) => diagnostic_todo!(
vec![span.error("value expected for this param")],
"value missing for param {}",
TtQuote::wrap(pname)
),
Meta::ConcatList(span) => diagnostic_todo!(
vec![span.error("concatenation occurs here")],
"concatenation not yet supported in xmli for param {}",
TtQuote::wrap(pname)
),
Meta::Lexeme(_, value) => *value,
};
self.push(attr_value(pval));
self.push(attr_name(pname));
Some(Xirf::open(
QN_WITH_PARAM,
OpenSpan::without_name_span(meta.span()),
depth,
))
}
fn push(&mut self, tok: Xirf) {
if self.stack.is_full() {
diagnostic_panic!(
@ -408,6 +456,10 @@ fn attr_name(name: SPair) -> Xirf {
Xirf::attr(QN_NAME, name, (name.span(), name.span()))
}
fn attr_value(val: SPair) -> Xirf {
Xirf::attr(QN_VALUE, val, (val.span(), val.span()))
}
fn expr_ele(expr: &Expr, depth: Depth) -> Xirf {
use ExprOp::*;

View File

@ -56,16 +56,6 @@
//! [asg]: https://en.wikipedia.org/wiki/Abstract_semantic_graph
//! [graph]: https://en.wikipedia.org/wiki/Graph_(discrete_mathematics)
//! [scc]: https://en.wikipedia.org/wiki/Strongly_connected_component
//!
//! Missing Identifiers
//! -------------------
//! Since identifiers in TAME can be defined in any order relative to their
//! dependencies within a source file,
//! it is often the case that a dependency will have to be added to the
//! graph before it is resolved.
//! For example,
//! [`Asg::add_dep_lookup`] will add an [`Ident::Missing`] to the graph
//! if either identifier has not yet been declared.
mod error;
mod graph;

View File

@ -47,8 +47,8 @@ use tamer::{
AnnotatedSpan, Diagnostic, FsSpanResolver, Reporter, VisualReporter,
},
nir::{
InterpError, InterpolateNir, Nir, NirToAir, NirToAirError, XirfToNir,
XirfToNirError,
InterpError, InterpolateNir, Nir, NirToAir, NirToAirError,
TplShortDesugar, XirfToNir, XirfToNirError,
},
parse::{
lowerable, FinalizeError, Lower, ParseError, ParsedObject, Token,
@ -164,21 +164,23 @@ fn compile<R: Reporter>(
_,
>::lower::<_, UnrecoverableError>(src, |toks| {
Lower::<XirToXirf<64, RefinedText>, XirfToNir, _>::lower(toks, |nir| {
Lower::<XirfToNir, InterpolateNir, _>::lower(nir, |nir| {
Lower::<InterpolateNir, NirToAir, _>::lower(nir, |air| {
Lower::<NirToAir, AirAggregate, _>::lower_with_context(
air,
asg,
|end| {
end.fold(Ok(()), |x, result| match result {
Ok(_) => x,
Err(e) => {
report_err(&e, reporter, &mut ebuf)?;
x
}
})
},
)
Lower::<XirfToNir, TplShortDesugar, _>::lower(nir, |nir| {
Lower::<TplShortDesugar, InterpolateNir, _>::lower(nir, |nir| {
Lower::<InterpolateNir, NirToAir, _>::lower(nir, |air| {
Lower::<NirToAir, AirAggregate, _>::lower_with_context(
air,
asg,
|end| {
end.fold(Ok(()), |x, result| match result {
Ok(_) => x,
Err(e) => {
report_err(&e, reporter, &mut ebuf)?;
x
}
})
},
)
})
})
})
})
@ -461,6 +463,14 @@ impl<T: Token> From<ParseError<T, Infallible>> for UnrecoverableError {
}
}
impl<T: Token> From<ParseError<T, Infallible>> for RecoverableError {
fn from(_: ParseError<T, Infallible>) -> Self {
unreachable!(
"<RecoverableError as From<ParseError<T, Infallible>>>::from"
)
}
}
impl From<ParseError<UnknownToken, xir::Error>> for RecoverableError {
fn from(e: ParseError<UnknownToken, xir::Error>) -> Self {
Self::XirParseError(e)

View File

@ -97,7 +97,7 @@ where
let sym = name.into();
depgraph
.lookup(SPair(sym, UNKNOWN_SPAN))
.lookup_global(SPair(sym, UNKNOWN_SPAN))
.and_then(|id| depgraph.get(id))
.unwrap_or_else(|| {
panic!("missing internal identifier: {}", sym.lookup_str())
@ -436,7 +436,7 @@ mod test {
asg.set_fragment(sym, FragmentText::from("foo")).unwrap();
let (_, _) = asg.add_dep_lookup(sym, dep);
let (_, _) = asg.add_dep_lookup_global(sym, dep);
asg.add_root(sym_node);
@ -463,7 +463,7 @@ mod test {
let sym = SPair("sym".into(), S1);
let dep = SPair("dep".into(), S2);
asg_nonempty_no_roots.add_dep_lookup(sym, dep);
asg_nonempty_no_roots.add_dep_lookup_global(sym, dep);
assert_eq!(
sort(&asg_nonempty_no_roots, Sections::new()),
@ -505,8 +505,8 @@ mod test {
asg.set_fragment(sym, FragmentText::from("foo")).unwrap();
asg.set_fragment(dep, FragmentText::from("bar")).unwrap();
let (_, _) = asg.add_dep_lookup(sym, dep);
let (_, _) = asg.add_dep_lookup(dep, sym);
let (_, _) = asg.add_dep_lookup_global(sym, dep);
let (_, _) = asg.add_dep_lookup_global(dep, sym);
asg.add_root(sym_node);
@ -584,10 +584,10 @@ mod test {
asg.set_fragment(dep, FragmentText::from("baz")).unwrap();
asg.set_fragment(dep2, FragmentText::from("huh")).unwrap();
let (_, _) = asg.add_dep_lookup(sym, dep);
let (_, _) = asg.add_dep_lookup(dep, sym);
let (_, _) = asg.add_dep_lookup(sym2, dep2);
let (_, _) = asg.add_dep_lookup(dep2, sym2);
let (_, _) = asg.add_dep_lookup_global(sym, dep);
let (_, _) = asg.add_dep_lookup_global(dep, sym);
let (_, _) = asg.add_dep_lookup_global(sym2, dep2);
let (_, _) = asg.add_dep_lookup_global(dep2, sym2);
asg.add_root(sym_node);
@ -643,8 +643,8 @@ mod test {
asg.set_fragment(sym, FragmentText::from("foo")).unwrap();
asg.set_fragment(dep, FragmentText::from("bar")).unwrap();
let (_, _) = asg.add_dep_lookup(sym, dep);
let (_, _) = asg.add_dep_lookup(sym, dep);
let (_, _) = asg.add_dep_lookup_global(sym, dep);
let (_, _) = asg.add_dep_lookup_global(sym, dep);
asg.add_root(sym_node);
@ -703,9 +703,9 @@ mod test {
asg.set_fragment(sym2, FragmentText::from("bar")).unwrap();
asg.set_fragment(sym3, FragmentText::from("baz")).unwrap();
let (_, _) = asg.add_dep_lookup(sym1, sym2);
let (_, _) = asg.add_dep_lookup(sym2, sym3);
let (_, _) = asg.add_dep_lookup(sym3, sym1);
let (_, _) = asg.add_dep_lookup_global(sym1, sym2);
let (_, _) = asg.add_dep_lookup_global(sym2, sym3);
let (_, _) = asg.add_dep_lookup_global(sym3, sym1);
asg.add_root(sym1_node);
@ -771,9 +771,9 @@ mod test {
asg.set_fragment(sym2, FragmentText::from("bar")).unwrap();
asg.set_fragment(sym3, FragmentText::from("baz")).unwrap();
let (_, _) = asg.add_dep_lookup(sym1, sym2);
let (_, _) = asg.add_dep_lookup(sym2, sym3);
let (_, _) = asg.add_dep_lookup(sym3, sym1);
let (_, _) = asg.add_dep_lookup_global(sym1, sym2);
let (_, _) = asg.add_dep_lookup_global(sym2, sym3);
let (_, _) = asg.add_dep_lookup_global(sym3, sym1);
asg.add_root(sym1_node);
@ -838,9 +838,9 @@ mod test {
asg.set_fragment(sym2, FragmentText::from("bar")).unwrap();
asg.set_fragment(sym3, FragmentText::from("baz")).unwrap();
let (_, _) = asg.add_dep_lookup(sym1, sym2);
let (_, _) = asg.add_dep_lookup(sym2, sym3);
let (_, _) = asg.add_dep_lookup(sym3, sym1);
let (_, _) = asg.add_dep_lookup_global(sym1, sym2);
let (_, _) = asg.add_dep_lookup_global(sym2, sym3);
let (_, _) = asg.add_dep_lookup_global(sym3, sym1);
asg.add_root(sym1_node);
@ -891,8 +891,8 @@ mod test {
asg.set_fragment(sym, FragmentText::from("foo")).unwrap();
asg.set_fragment(dep, FragmentText::from("bar")).unwrap();
let (_, _) = asg.add_dep_lookup(sym, dep);
let (_, _) = asg.add_dep_lookup(dep, sym);
let (_, _) = asg.add_dep_lookup_global(sym, dep);
let (_, _) = asg.add_dep_lookup_global(dep, sym);
asg.add_root(sym_node);
@ -951,9 +951,9 @@ mod test {
asg.set_fragment(sym2, FragmentText::from("bar")).unwrap();
asg.set_fragment(sym3, FragmentText::from("baz")).unwrap();
let (_, _) = asg.add_dep_lookup(sym1, sym2);
let (_, _) = asg.add_dep_lookup(sym2, sym3);
let (_, _) = asg.add_dep_lookup(sym3, sym1);
let (_, _) = asg.add_dep_lookup_global(sym1, sym2);
let (_, _) = asg.add_dep_lookup_global(sym2, sym3);
let (_, _) = asg.add_dep_lookup_global(sym3, sym1);
asg.add_root(sym1_node);
@ -1017,8 +1017,8 @@ mod test {
asg.set_fragment(ignored, FragmentText::from("baz"))
.unwrap();
let (_, _) = asg.add_dep_lookup(sym, dep);
let (_, _) = asg.add_dep_lookup(ignored, sym);
let (_, _) = asg.add_dep_lookup_global(sym, dep);
let (_, _) = asg.add_dep_lookup_global(ignored, sym);
asg.add_root(sym_node);

View File

@ -52,6 +52,7 @@
mod air;
mod interp;
mod parse;
mod tplshort;
use crate::{
diagnose::{Annotate, Diagnostic},
@ -77,6 +78,7 @@ pub use interp::{InterpError, InterpState as InterpolateNir};
pub use parse::{
NirParseState as XirfToNir, NirParseStateError_ as XirfToNirError,
};
pub use tplshort::TplShortDesugar;
/// IR that is "near" the source code.
///
@ -201,11 +203,22 @@ pub enum NirEntity {
/// Template.
Tpl,
/// Template parameter (metavariable).
TplParam,
/// Full application and expansion of the template identified by the
/// provided name.
/// Template parameter (metavariable).
///
/// If a pair of [`SPair`]s is provided,
/// then the param is shorthand,
/// with the non-`@`-padded name as the first of the pair and the
/// value as the second.
///
/// A shorthand entity is implicitly closed and should not have a
/// matching [`Nir::Close`] token.
TplParam(Option<(SPair, SPair)>),
/// Full application and expansion a template.
///
/// If a name is provided,
/// then this template application is shorthand.
TplApply(Option<QName>),
}
@ -237,7 +250,13 @@ impl Display for NirEntity {
Any => write!(f, "disjunctive () expression"),
Tpl => write!(f, "template"),
TplParam => write!(f, "template param (metavariable)"),
TplParam(None) => write!(f, "template param (metavariable)"),
TplParam(Some((name, val))) => write!(
f,
"shorthand template param key {} with value {}",
TtQuote::wrap(name),
TtQuote::wrap(val),
),
TplApply(None) => {
write!(f, "full template application and expansion")
}
@ -252,7 +271,7 @@ impl Display for NirEntity {
impl Token for Nir {
fn ir_name() -> &'static str {
"Plain NIR"
"NIR"
}
/// Identifying span of a token.

View File

@ -27,11 +27,7 @@ use crate::{
// These are also used by the `test` module which imports `super`.
#[cfg(feature = "wip-nir-to-air")]
use crate::{
asg::ExprOp,
nir::NirEntity,
sym::{GlobalSymbolIntern, GlobalSymbolResolve},
};
use crate::{asg::ExprOp, nir::NirEntity};
use super::Nir;
@ -79,6 +75,8 @@ impl ParseState for NirToAir {
) -> TransitionResult<Self::Super> {
use NirToAir::*;
use crate::{diagnose::Annotate, diagnostic_panic};
// Single-item "queue".
if let Some(obj) = queue.take() {
return Transition(Ready).ok(obj).with_lookahead(tok);
@ -123,29 +121,48 @@ impl ParseState for NirToAir {
Transition(Ready).ok(Air::TplStart(span))
}
// Short-hand template application contains the name of the
// template _without_ the underscore padding as the local part
// of the QName.
//
// Template application will create an anonymous template,
// apply it,
// and then expand it.
(Ready, Nir::Open(NirEntity::TplApply(Some(qname)), span)) => {
// TODO: Determine whether caching these has any notable
// benefit over repeated heap allocations,
// comparing packages with very few applications and
// packages with thousands
// (we'd still have to hit the heap for the cache).
let tpl_name =
format!("_{}_", qname.local_name().lookup_str()).intern();
queue.replace(Air::RefIdent(SPair(tpl_name, span)));
Transition(Ready).ok(Air::TplStart(span))
// Short-hand template application must be handled through
// desugaring as part of the lowering pipeline,
// so that it is converted to long form before getting here.
(
Ready,
Nir::Open(
NirEntity::TplApply(Some(_)) | NirEntity::TplParam(Some(_)),
span,
),
) => {
// TODO: In the future maybe TAMER will have evolved its
// abstractions enough that there's an ROI for prohibiting
// this at the type level.
diagnostic_panic!(
vec![
span.internal_error(
"attempted shorthand template application"
),
span.help(
"TAMER must be compiled with support for \
shorthand template application by utilizing the \
nir::tplshort module in the lowering pipeline."
)
],
"shortand template application is unsupported in this \
build of TAMER"
)
}
(Ready, Nir::Close(NirEntity::TplApply(_), span)) => {
Transition(Ready).ok(Air::TplEndRef(span))
}
(Ready, Nir::Open(NirEntity::TplParam(None), span)) => {
Transition(Ready).ok(Air::TplMetaStart(span))
}
(Ready, Nir::Close(NirEntity::TplParam(_), span)) => {
Transition(Ready).ok(Air::TplMetaEnd(span))
}
(Ready, Nir::Text(lexeme)) => {
Transition(Ready).ok(Air::TplLexeme(lexeme))
}
(
Ready,
Nir::Close(
@ -168,15 +185,9 @@ impl ParseState for NirToAir {
Transition(Ready).ok(Air::RefIdent(spair))
}
(
Ready,
Nir::Todo
| Nir::TodoAttr(..)
| Nir::Desc(..)
| Nir::Text(_)
| Nir::Open(NirEntity::TplParam, _)
| Nir::Close(NirEntity::TplParam, _),
) => Transition(Ready).ok(Air::Todo(UNKNOWN_SPAN)),
(Ready, Nir::Todo | Nir::TodoAttr(..) | Nir::Desc(..)) => {
Transition(Ready).ok(Air::Todo(UNKNOWN_SPAN))
}
}
}

View File

@ -18,7 +18,7 @@
// along with this program. If not, see <http://www.gnu.org/licenses/>.
use super::*;
use crate::{convert::ExpectInto, parse::util::SPair, span::dummy::*};
use crate::{parse::util::SPair, span::dummy::*};
type Sut = NirToAir;
@ -138,45 +138,10 @@ fn tpl_with_name() {
);
}
// This is the form everyone uses.
// It applies a template much more concisely and without the underscore
// padding,
// making it look much like a language primitive
// (with the exception of the namespace prefix).
#[test]
fn short_hand_tpl_apply() {
// Shorthand converts `t:tpl-name` into `_tpl-name_`.
let qname = ("t", "tpl-name").unwrap_into();
let name = SPair("_tpl-name_".into(), S1);
let toks = vec![
Nir::Open(NirEntity::TplApply(Some(qname)), S1),
Nir::Close(NirEntity::TplApply(None), S2),
];
#[rustfmt::skip]
assert_eq!(
Ok(vec![
O(Air::TplStart(S1)),
// The span associated with the name of the template to be
// applied is the span of the entire QName from NIR.
// The reason for this is that `t:foo` is transformed into
// `_foo_`,
// and so the `t:` is a necessary part of the
// representation of the name of the template;
// `foo` is not in itself a valid template name at the
// time of writing.
O(Air::RefIdent(name)),
O(Air::TplEndRef(S2)),
]),
Sut::parse(toks.into_iter()).collect(),
);
}
// Long form takes the actual underscore-padded template name without any
// additional processing.
#[test]
fn apply_template_long_form() {
fn apply_template_long_form_nullary() {
let name = SPair("_tpl-name_".into(), S2);
#[rustfmt::skip]
@ -196,3 +161,49 @@ fn apply_template_long_form() {
Sut::parse(toks.into_iter()).collect(),
);
}
#[test]
fn apply_template_long_form_args() {
let name = SPair("_tpl-name_".into(), S2);
let p1 = SPair("@p1@".into(), S4);
let v1 = SPair("value1".into(), S5);
let p2 = SPair("@p2@".into(), S8);
let v2 = SPair("value2".into(), S9);
#[rustfmt::skip]
let toks = vec![
Nir::Open(NirEntity::TplApply(None), S1),
Nir::Ref(name),
Nir::Open(NirEntity::TplParam(None), S3),
Nir::BindIdent(p1),
Nir::Text(v1),
Nir::Close(NirEntity::TplParam(None), S6),
Nir::Open(NirEntity::TplParam(None), S7),
Nir::BindIdent(p2),
Nir::Text(v2),
Nir::Close(NirEntity::TplParam(None), S10),
Nir::Close(NirEntity::TplApply(None), S11),
];
#[rustfmt::skip]
assert_eq!(
Ok(vec![
O(Air::TplStart(S1)),
O(Air::RefIdent(name)),
O(Air::TplMetaStart(S3)),
O(Air::BindIdent(p1)),
O(Air::TplLexeme(v1)),
O(Air::TplMetaEnd(S6)),
O(Air::TplMetaStart(S7)),
O(Air::BindIdent(p2)),
O(Air::TplLexeme(v2)),
O(Air::TplMetaEnd(S10)),
O(Air::TplEndRef(S11)),
]),
Sut::parse(toks.into_iter()).collect(),
);
}

View File

@ -262,7 +262,7 @@ impl ParseState for InterpState {
Ready => match tok.symbol() {
Some(sym) if needs_interpolation(sym) => {
Transition(GenIdent(sym))
.ok(Nir::Open(NirEntity::TplParam, span))
.ok(Nir::Open(NirEntity::TplParam(None), span))
.with_lookahead(tok)
}
@ -315,7 +315,7 @@ impl ParseState for InterpState {
// symbol that we've been fed
// (the specification string).
Transition(FinishSym(s, gen_param))
.ok(Nir::Close(NirEntity::TplParam, span))
.ok(Nir::Close(NirEntity::TplParam(None), span))
.with_lookahead(tok)
};
}

View File

@ -79,7 +79,7 @@ fn expect_expanded_header(
// helpful information to a human reader.
assert_eq!(
sut.next(),
Some(Ok(Object(Nir::Open(NirEntity::TplParam, span)))),
Some(Ok(Object(Nir::Open(NirEntity::TplParam(None), span)))),
);
assert_eq!(
sut.next(),
@ -132,7 +132,7 @@ fn desugars_literal_with_ending_var() {
Object(Nir::Ref(SPair("@bar@".into(), c))),
// This is an object generated from user input, so the closing
// span has to identify what were generated from.
Object(Nir::Close(NirEntity::TplParam, a)),
Object(Nir::Close(NirEntity::TplParam(None), a)),
// Finally,
// we replace the original provided attribute
// (the interpolation specification)
@ -172,7 +172,7 @@ fn desugars_var_with_ending_literal() {
Ok(vec![
Object(Nir::Ref(SPair("@foo@".into(), b))),
Object(Nir::Text(SPair("bar".into(), c))),
Object(Nir::Close(NirEntity::TplParam, a)),
Object(Nir::Close(NirEntity::TplParam(None), a)),
Object(Nir::Ref(SPair(expect_name, a))),
]),
sut.collect(),
@ -216,7 +216,7 @@ fn desugars_many_vars_and_literals() {
// offsets.
Object(Nir::Text(SPair("baz".into(), d))),
Object(Nir::Ref(SPair("@quux@".into(), e))),
Object(Nir::Close(NirEntity::TplParam, a)),
Object(Nir::Close(NirEntity::TplParam(None), a)),
Object(Nir::Ref(SPair(expect_name, a))),
]),
sut.collect(),
@ -266,7 +266,7 @@ fn proper_multibyte_handling() {
// offsets.
Object(Nir::Text(SPair("βaζ".into(), d))),
Object(Nir::Ref(SPair("@qμuχ@".into(), e))),
Object(Nir::Close(NirEntity::TplParam, a)),
Object(Nir::Close(NirEntity::TplParam(None), a)),
Object(Nir::Ref(SPair(expect_name, a))),
]),
sut.collect(),
@ -301,7 +301,7 @@ fn desugars_adjacent_interpolated_vars() {
Object(Nir::Ref(SPair("@foo@".into(), b))),
Object(Nir::Ref(SPair("@bar@".into(), c))),
Object(Nir::Ref(SPair("@baz@".into(), d))),
Object(Nir::Close(NirEntity::TplParam, a)),
Object(Nir::Close(NirEntity::TplParam(None), a)),
Object(Nir::Ref(SPair(expect_name, a))),
]),
sut.collect(),
@ -347,7 +347,7 @@ fn error_missing_closing_interp_delim() {
// Having omitted the above token,
// we're able to proceed as if the user didn't provide it at
// all.
Ok(Object(Nir::Close(NirEntity::TplParam, a))),
Ok(Object(Nir::Close(NirEntity::TplParam(None), a))),
Ok(Object(Nir::Ref(SPair(expect_name, a)))),
],
sut.collect::<Vec<ParsedResult<Sut>>>(),
@ -395,7 +395,7 @@ fn error_nested_delim() {
// (end of the specification string)
// and ignore everything that follows rather than
// potentially interpret it in confusing ways.
Ok(Object(Nir::Close(NirEntity::TplParam, a))),
Ok(Object(Nir::Close(NirEntity::TplParam(None), a))),
Ok(Object(Nir::Ref(SPair(expect_name, a)))),
],
sut.collect::<Vec<ParsedResult<Sut>>>(),
@ -438,7 +438,7 @@ fn error_empty_interp() {
// It wouldn't have had any effect anyway,
// being empty.
Ok(Object(Nir::Text(SPair("cow".into(), d)))),
Ok(Object(Nir::Close(NirEntity::TplParam, a))),
Ok(Object(Nir::Close(NirEntity::TplParam(None), a))),
Ok(Object(Nir::Ref(SPair(expect_name, a)))),
],
sut.collect::<Vec<ParsedResult<Sut>>>(),
@ -477,7 +477,7 @@ fn error_close_before_open() {
// was supposed to be a literal or a param.
// Just bail out;
// maybe in the future we can do something better.
Ok(Object(Nir::Close(NirEntity::TplParam, a))),
Ok(Object(Nir::Close(NirEntity::TplParam(None), a))),
Ok(Object(Nir::Ref(SPair(expect_name, a)))),
],
sut.collect::<Vec<ParsedResult<Sut>>>(),

View File

@ -1674,27 +1674,47 @@ ele_parse! {
/// or even a mix of the two
/// (with statements hoisted out of expressions).
///
/// TODO: This is apparently unused by the current system,
/// in favor of a transition to [`TplApplyShort`],
/// but this is still needed to support dynamic template application
/// (templates whose names are derived from other template inputs).
/// See also [`TplApplyShort`],
/// which gets desugared into this via [`super::tplshort`].
ApplyTemplate := QN_APPLY_TEMPLATE(_, ospan) {
@ {
QN_NAME => Ref,
} => Nir::Open(NirEntity::TplApply(None), ospan.into()),
/(cspan) => Nir::Close(NirEntity::TplApply(None), cspan.into()),
// TODO: This is wrong, we just need something here for now.
AnyStmtOrExpr,
ApplyTemplateParam,
};
/// Short-hand template application.
/// Long-form template argument.
///
/// Template arguments are lexical.
///
/// See also [`TplApplyShort`],
/// which gets desugared into this via [`super::tplshort`].
ApplyTemplateParam := QN_WITH_PARAM(_, ospan) {
@ {
QN_NAME => BindIdent,
QN_VALUE => Text,
} => Nir::Open(NirEntity::TplParam(None), ospan.into()),
/(cspan) => Nir::Close(NirEntity::TplParam(None), cspan.into()),
// TODO: Need to support children, e.g. @values@
};
/// Shorthand template application.
///
/// This expands into an equivalent [`ApplyTemplate`] form where each
/// attribute is a template argument,
/// and where the body of this application is the `@values@`
/// template argument.
/// See [`ApplyTemplate`] for more information.
///
/// The name of the template omits the surrounding `_`s;
/// `t:foo` will desugar into the template name `_foo_`.
/// Params similarly omit `@` and are derived from the _local name
/// only_;
/// so `bar="baz"` will be desugared into a param `@bar@` with a
/// text value `baz`.
TplApplyShort := NS_T(qname, ospan) {
@ {} => Nir::Open(NirEntity::TplApply(Some(qname)), ospan.into()),
/(cspan) => Nir::Close(NirEntity::TplApply(None), cspan.into()),
@ -1702,7 +1722,18 @@ ele_parse! {
// Streaming attribute parsing;
// this takes precedence over any attribute parsing above
// (which is used only for emitting the opening object).
[attr](_attr) => Todo,
[attr](Attr(name, value, AttrSpan(name_span, value_span))) => {
Nir::Open(
// TODO: This simply _ignores_ the namespace prefix.
// If it's not a useful construct,
// it ought to be rejected.
NirEntity::TplParam(Some((
SPair(*name.local_name(), name_span),
SPair(value, value_span),
))),
name_span,
)
},
// Template bodies depend on context,
// so we have to just accept everything and defer to a future

View File

@ -0,0 +1,156 @@
// Shorthand template application desugaring for NIR
//
// Copyright (C) 2014-2023 Ryan Specialty, LLC.
//
// This file is part of TAME.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
//! Shorthand template application desugaring for NIR.
//!
//! A shorthand template application looks something like this,
//! in XML form:
//!
//! ```xml
//! <t:foo bar="baz">
//! <c:sum />
//! </t:foo>
//!
//! <!-- desugars into -->
//! <apply-template name="_foo_">
//! <with-param name="@bar@" value="baz" />
//! <with-param name="@values@">
//! <c:sum />
//! </with-param>
//! </apply-template>
//! ```
//!
//! The shorthand syntax makes templates look like another language
//! primitive,
//! with the exception of the namespace prefix.
//!
//! Note how desugaring also wraps template names in `'_'` and param names
//! in `'@'`.
//! These naming requirements were intended to eliminate conflicts with
//! other types of identifiers and to make it obvious when templates and
//! metavariables were being used,
//! but it works against the goal of making template applications look
//! like language primitives.
//! Shorthand form was added well after the long `apply-template` form.
//!
//! This shorthand version does not permit metavariables for template or
//! param names,
//! so the long form is still a useful language feature for more
//! sophisticated cases.
//!
//! This was originally handled in the XSLT compiler in
//! `:src/current/include/preproc/template.xsl`.
//! You may need to consult the Git history if this file is no longer
//! available or if the XSLT template was since removed.
use arrayvec::ArrayVec;
use super::{Nir, NirEntity};
use crate::{
parse::prelude::*,
sym::{GlobalSymbolIntern, GlobalSymbolResolve},
};
use std::convert::Infallible;
#[derive(Debug, PartialEq, Eq, Default)]
pub enum TplShortDesugar {
/// Waiting for shorthand template application,
/// passing tokens along in the meantime.
#[default]
Scanning,
}
impl Display for TplShortDesugar {
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
match self {
Self::Scanning => {
write!(f, "awaiting shorthand template application")
}
}
}
}
impl ParseState for TplShortDesugar {
type Token = Nir;
type Object = Nir;
type Error = Infallible;
type Context = Stack;
fn parse_token(
self,
tok: Self::Token,
stack: &mut Self::Context,
) -> TransitionResult<Self::Super> {
use TplShortDesugar::*;
if let Some(obj) = stack.pop() {
return Transition(self).ok(obj).with_lookahead(tok);
}
match (self, tok) {
// Shorthand template applications are identified by a `Some`
// QName in the `TplApply` entity.
//
// The name of the template _without_ the underscore padding is
// the local part of the QName.
(Scanning, Nir::Open(NirEntity::TplApply(Some(qname)), span)) => {
// TODO: Determine whether caching these has any notable
// benefit over repeated heap allocations,
// comparing packages with very few applications and
// packages with thousands
// (we'd still have to hit the heap for the cache).
let tpl_name =
format!("_{}_", qname.local_name().lookup_str()).intern();
stack.push(Nir::Ref(SPair(tpl_name, span)));
Transition(Scanning)
.ok(Nir::Open(NirEntity::TplApply(None), span))
}
// Shorthand template params' names do not contain the
// surrounding `@`s.
(
Scanning,
Nir::Open(NirEntity::TplParam(Some((name, val))), span),
) => {
let pname = format!("@{name}@").intern();
stack.push(Nir::Close(NirEntity::TplParam(None), span));
stack.push(Nir::Text(val));
stack.push(Nir::BindIdent(SPair(pname, name.span())));
Transition(Scanning)
.ok(Nir::Open(NirEntity::TplParam(None), span))
}
// Any tokens that we don't recognize will be passed on unchanged.
(st, tok) => Transition(st).ok(tok),
}
}
fn is_accepting(&self, stack: &Self::Context) -> bool {
matches!(self, Self::Scanning) && stack.is_empty()
}
}
type Stack = ArrayVec<Nir, 3>;
#[cfg(test)]
mod test;

View File

@ -0,0 +1,137 @@
// Tests shorthand template application desugaring for NIR
//
// Copyright (C) 2014-2023 Ryan Specialty, LLC.
//
// This file is part of TAME.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
use super::*;
use crate::{convert::ExpectInto, span::dummy::*};
use Parsed::Object as O;
type Sut = TplShortDesugar;
#[test]
fn desugars_nullary() {
// Shorthand converts `t:tpl-name` into `_tpl-name_`.
let qname = ("t", "tpl-name").unwrap_into();
let tpl = "_tpl-name_".into();
let toks = [
Nir::Open(NirEntity::TplApply(Some(qname)), S1),
Nir::Close(NirEntity::TplApply(None), S2),
];
#[rustfmt::skip]
assert_eq!(
Ok(vec![
O(Nir::Open(NirEntity::TplApply(None), S1)),
// The span associated with the name of the template to be
// applied is the span of the entire QName from NIR.
// The reason for this is that `t:foo` is transformed into
// `_foo_`,
// and so the `t:` is a necessary part of the
// representation of the name of the template;
// `foo` is not in itself a valid template name at the
// time of writing.
O(Nir::Ref(SPair(tpl, S1))),
O(Nir::Close(NirEntity::TplApply(None), S2)),
]),
Sut::parse(toks.into_iter()).collect(),
);
}
#[test]
fn desugars_unary() {
// Shorthand converts `t:tpl-name` into `_tpl-name_`.
let qname = ("t", "tpl-name").unwrap_into();
let name = SPair("_tpl-name_".into(), S1);
let aname = SPair("foo".into(), S3);
let pval = SPair("foo value".into(), S4);
// The attribute name gets padded with '@',
// much like the template does with underscores.
let pname = SPair("@foo@".into(), S3);
#[rustfmt::skip]
let toks = vec![
// <t:qname
Nir::Open(NirEntity::TplApply(Some(qname)), S1),
// foo="foo value"
Nir::Open(NirEntity::TplParam(Some((aname, pval))), S2),
// Implicit close.
// />
Nir::Close(NirEntity::TplApply(None), S6),
];
#[rustfmt::skip]
assert_eq!(
Ok(vec![
O(Nir::Open(NirEntity::TplApply(None), S1)),
O(Nir::Ref(name)),
O(Nir::Open(NirEntity::TplParam(None), S2)),
// Derived from `aname` (by padding)
O(Nir::BindIdent(pname)),
// The value is left untouched.
O(Nir::Text(pval)),
// Close is derived from open.
O(Nir::Close(NirEntity::TplParam(None), S2)),
O(Nir::Close(NirEntity::TplApply(None), S6)),
]),
Sut::parse(toks.into_iter()).collect(),
);
}
// Don't parse what we desugar into!
#[test]
fn does_not_desugar_long_form() {
let name = SPair("_tpl-name_".into(), S2);
let pname = SPair("@param".into(), S4);
let pval = SPair("value".into(), S5);
#[rustfmt::skip]
let toks = [
Nir::Open(NirEntity::TplApply(None), S1),
Nir::BindIdent(name),
Nir::Open(NirEntity::TplParam(None), S3),
Nir::BindIdent(pname),
Nir::Text(pval),
Nir::Close(NirEntity::TplParam(None), S6),
Nir::Close(NirEntity::TplApply(None), S7),
];
#[rustfmt::skip]
assert_eq!(
// We avoid #[derive(Clone)] on Nir so that we have confidence that
// we're not doing anything suspicious with tokens.
// So this is a duplicate of the above,
// mapped over `O`.
Ok(vec![
O(Nir::Open(NirEntity::TplApply(None), S1)),
O(Nir::BindIdent(name)),
O(Nir::Open(NirEntity::TplParam(None), S3)),
O(Nir::BindIdent(pname)),
O(Nir::Text(pval)),
O(Nir::Close(NirEntity::TplParam(None), S6)),
O(Nir::Close(NirEntity::TplApply(None), S7)),
]),
Sut::parse(toks.into_iter()).collect(),
);
}

View File

@ -710,6 +710,7 @@ pub mod st {
L_VIRTUAL: cid "virtual",
L_WARNING: cid "warning",
L_WHEN: cid "when",
L_WITH_PARAM: tid "with-param",
L_WORKSHEET: cid "worksheet",
L_XMLNS: cid "xmlns",
L_YIELD: cid "yield",

View File

@ -320,7 +320,7 @@ macro_rules! ele_parse {
// Special forms (`[sp](args) => expr`).
$(
[$special:ident]$(($($special_arg:ident),*))?
[$special:ident]$(($($special_arg:tt)*))?
=> $special_map:expr,
)?
@ -338,7 +338,7 @@ macro_rules! ele_parse {
@ { $($attrbody)* } => $attrmap,
/$($($close_span)?)? => ele_parse!(@!ele_close $($closemap)?),
$([$special]$(($($special_arg),*))? => $special_map,)?
$([$special]$(($($special_arg)*))? => $special_map,)?
<> {
$(
@ -422,7 +422,7 @@ macro_rules! ele_parse {
/$($close_span:ident)? => $closemap:expr,
// Streaming (as opposed to aggregate) attribute parsing.
$([attr]($attr_stream_binding:ident) => $attr_stream_map:expr,)?
$([attr]($attr_stream_binding:pat) => $attr_stream_map:expr,)?
// Nonterminal references.
<> {

View File

@ -225,6 +225,7 @@ pub mod qname {
QN_VALUES: :L_VALUES,
QN_VIRTUAL: :L_VIRTUAL,
QN_WARNING: :L_WARNING,
QN_WITH_PARAM: :L_WITH_PARAM,
QN_WORKSHEET: :L_WORKSHEET,
QN_YIELD: :L_YIELD,
QN_YIELDS: :L_YIELDS,

View File

@ -66,6 +66,19 @@
<template name="_short-hand-nullary_" />
<apply-template name="_short-hand-nullary_" />
<template name="_short-hand-unary_" />
<apply-template name="_short-hand-unary_">
<with-param name="@foo@" value="bar" />
</apply-template>
<template name="_short-hand-nary_" />
<apply-template name="_short-hand-nary_">
<with-param name="@foo@" value="bar" />
<with-param name="@bar@" value="baz" />
<with-param name="@baz@" value="quux" />
</apply-template>
</package>

View File

@ -66,8 +66,20 @@
`apply-template` form,
asserting their equivalency.
<template name="_short-hand-nullary_" />
<t:short-hand-nullary />
<template name="_short-hand-unary_" />
<t:short-hand-unary foo="bar" />
<template name="_short-hand-nary_" />
<t:short-hand-nary foo="bar" bar="baz" baz="quux" />
<!-- TODO
<t:short-hand-nullary-body>
<c:sum />
@ -77,8 +89,6 @@
<t:inner-short />
</t:short-hand-nullary-inner>
<t:short-hand foo="bar" />
<t:short-hand foo="bar">
<c:sum />
</t:short-hand>