Compare commits

...

10 Commits

Author SHA1 Message Date
Mike Gerwitz c26444666f tamer: asg: Shorthand and long-form template arguments
This applies to template application only; there's still some work to do for
template parameters in definitions (well, for deriving them in `xmli` at
least).  And, as you can see, there's still a lot of TODO items here.

I ended up backtracking on tree edges to Meta, and even on cross edges to
Meta, because it complicated xmli derivation with no benefit right now;
maybe a cross edge will be re-added in the future, but I need to move on and
see where this takes me.

But, it works.

DEV-13708
2023-03-23 00:04:53 -04:00
Mike Gerwitz b2c6b7f073 tamer: asg::air::expr: Do not cache (globally) identifiers created with StoreDangling
I'm not happy with this implementation.  The linear search is undesirable,
but not too bad (and maybe wouldn't even be worth caching, if this were the
whole story), but we _also_ need to prevent duplicate identifiers.  We are
not going to want to perform a linear search of a linked list (effectively)
every time we add an identifier to check for uniqueness, so I think the
caching is going to have to be generalized very shortly anyway.

As it stands now, a duplicate identifier would cause an error at expansion
time.  That's not what we want, but it's not terrible, because you can have
that same problem in normal circumstances without local conflicts.

But this'll be used for metavariables as well, where we absolutely _do_ want
to fail at template definition time.

DEV-13708
2023-03-22 13:26:47 -04:00
Mike Gerwitz 31a39c79d3 tamer: asg::graph: *lookup{=>_global}*
Identifier lookups, as done using the graph methods today, look up from a
cache representing the global environment.

Templates must not contribute to this environment until expansion.  Further,
metavariables will not be present in this environment.  To avoid confusion
and help obviate accidental contributions to this environment, the methods
have been renamed.  This will also allow for the creation of more general
methods down the line.

DEV-13708
2023-03-22 11:28:55 -04:00
Mike Gerwitz 7a9cf6bc51 tamer: asg::air: Test formatting (token nesting)
This makes the tests quite a bit easier to understand visually.  I've been
doing this with all new tests but had to go back to some old ones, and still
have more to go back to.  Baby steps.

DEV-13708
2023-03-22 11:08:09 -04:00
Mike Gerwitz 91b787d367 tamer: nir: Desugar shorthand template params and yield AIR
I had intended for this to be a full vertical slice initially, but AIR's
parser is going to need enough work that it'll muddy this patch a bit too
much.

This keeps the desugaring simple, which is what I was hoping for.

The next step is to load it into the graph and emit regenerated longhand
sources.

I also don't like how the namespace prefix is just being ignored for
shorthand param desugaring.  This is also the case in the XSLT-based
compiler, but this violates TAMER's principle that it should parse every bit
of information; nothing should be ignored.  If something does not contribute
useful information, then it is not a useful construct and ought to be
rejected.

DEV-13708
2023-03-22 10:17:19 -04:00
Mike Gerwitz d3f8f76904 tamer: nir::NirEntity::TplParam: Optional name/value pair
This will be used for shorthand desugaring.

DEV-13708
2023-03-21 15:26:17 -04:00
Mike Gerwitz c42e693b14 tamer: Introduce desugaring operation for shorthand template application
This moves translation from NirToAir into TplShortDesugar, and changes the
output from AIR to NIR.

This is going to be much easier to reason about as a desugaring
operation (and indeed that's always how TAME has implemented it, in XSLT);
this keeps the complexity isolated.

Ideally, NirToAir wouldn't even accept tokens that it can't handle, but
that's going to take quite a bit more work and I don't have the time right
now.  Instead, we'll fail at runtime with some hopefully-useful
information.  It shouldn't actually happen in practice.

DEV-13708
2023-03-21 14:38:07 -04:00
Mike Gerwitz c91d175711 tamer: asg::air::ir: Remove TplApply
The implementation decided upon in the previous commits have made this
unnecessary, using `RefIdent` to produce `Tpl->Ident[->Tpl]` instead.

DEV-13708
2023-03-17 13:46:06 -04:00
Mike Gerwitz 8a3dec0f95 tamer: build-aux/asg-ontviz: Vary arrowhead for cross edges
This makes it more visually apparent, when looking directly at a node,
whether an edge could represent a tree edge.

Dynamic edges could be tree edges, so I left those solid; that's the more
important visual indicator that I'm interested in, and it's disambiguated by
the dashed line.

DEV-13708
2023-03-17 13:37:36 -04:00
Mike Gerwitz 8579b8bce9 tamer: asg: Dynamically determined cross edges
Previous to this commit, ontological cross edges were declared
statically.  But this doesn't fare well with the decided implementation for
template application.

The documentation details it, but we have Tpl->Ident which could mean "I
define this Ident once expanded", or it could mean "this is a reference to a
template I will be applying".  The former is a tree edge, the latter is a
cross edge, and that determination can only be made by inspecting edge data
at runtime.

It could have been resolved by introducing new Object types, but that is a
lot of work for little benefit, especially given that only (right now) the
visitor uses this information.

DEV-13708
2023-03-17 12:53:53 -04:00
36 changed files with 1503 additions and 567 deletions

View File

@ -147,7 +147,7 @@ mod base {
bench.iter(|| {
xs.iter()
.map(|sym| sut.lookup(*sym).unwrap())
.map(|sym| sut.lookup_global(*sym).unwrap())
.for_each(drop);
});
}
@ -286,7 +286,7 @@ mod base {
bench.iter(|| {
xs.iter()
.zip(xs.iter().cycle().skip(1))
.map(|(from, to)| sut.add_dep_lookup(*from, *to))
.map(|(from, to)| sut.add_dep_lookup_global(*from, *to))
.for_each(drop);
});
}
@ -303,7 +303,7 @@ mod base {
bench.iter(|| {
xs.iter()
.zip(xs.iter().cycle().skip(1))
.map(|(from, to)| sut.add_dep_lookup(*from, *to))
.map(|(from, to)| sut.add_dep_lookup_global(*from, *to))
.for_each(drop);
});
}

View File

@ -114,9 +114,23 @@ block_src && $NF ~ /\w+,$/ {
# Edge type (cross, tree)
ty = $(NF-1)
# Dashed is visually in-between solid and dotted,
# and `dyn` is either `tree` or `cross`,
# determined at runtime.
# But we may need some other representation if `dotted` is too visually
# sparse and difficult/annoying to see;
# let's see where the ASG visualization ends up first.
attrs = ""
if (ty == "cross") {
attrs="[style=dashed]"
switch (ty) {
case "tree":
attrs="[style=solid,arrowhead=normal]";
break;
case "cross":
attrs="[style=dotted,arrowhead=open]";
break;
case "dyn":
attrs="[style=dashed,arrowhead=normal]";
break;
}
gsub(/,$/, "")
@ -129,7 +143,7 @@ block_src && $NF ~ /\w+,$/ {
# the source object is right-aligned and target is left-aligned,
# so that `Src -> Target` can be easily read regardless of the width
# of the objects involved.
printf " %5s -> %-5s %14s; # %s edge\n", block_src, $NF, attrs, ty
printf " %5s -> %-5s %32s; # %s edge\n", block_src, $NF, attrs, ty
found_rels++
}

View File

@ -246,7 +246,7 @@ impl ParseState for AirAggregate {
.transition(Empty),
(Empty, AirIdent(IdentDep(sym, dep))) => {
asg.add_dep_lookup(sym, dep);
asg.add_dep_lookup_global(sym, dep);
Transition(Empty).incomplete()
}
@ -255,7 +255,7 @@ impl ParseState for AirAggregate {
}
(Empty, AirIdent(IdentRoot(sym))) => {
let obj = asg.lookup_or_missing(sym);
let obj = asg.lookup_global_or_missing(sym);
asg.add_root(obj);
Transition(Empty).incomplete()

View File

@ -417,6 +417,11 @@ mod root {
/// the sub-expression will be rooted in [`Self`],
/// but the [`Dangling`] root expression will still be rejected.
///
/// This expects identifiers to be rooted in the global environment,
/// which is the package representing the active compilation unit.
/// This may be relaxed once identifier caching is generalized;
/// at the time of writing it is too coupled to the graph root.
///
/// See [`RootStrategy`] for more information.
#[derive(Debug, PartialEq)]
pub struct ReachableOnly<O: ObjectKind>(ObjectIndex<O>)
@ -433,9 +438,9 @@ mod root {
fn defines(&self, asg: &mut Asg, id: SPair) -> ObjectIndex<Ident> {
match self {
Self(oi_root) => {
asg.lookup_or_missing(id).add_edge_from(asg, *oi_root, None)
}
Self(oi_root) => asg
.lookup_global_or_missing(id)
.add_edge_from(asg, *oi_root, None),
}
}
@ -456,6 +461,10 @@ mod root {
/// Sub-expressions can be thought of as utilizing this strategy with an
/// implicit parent [`ObjectIndex<Expr>`](ObjectIndex).
///
/// Unlike [`ReachableOnly`],
/// this does _not_ cache identifiers in the global environment.
/// See there for more information.
///
/// See [`RootStrategy`] for more information.
#[derive(Debug, PartialEq)]
pub struct StoreDangling<O: ObjectKind>(ObjectIndex<O>)
@ -470,11 +479,13 @@ mod root {
Self(oi)
}
fn defines(&self, asg: &mut Asg, id: SPair) -> ObjectIndex<Ident> {
// We are a superset of `ReachableOnly`'s behavior,
// so delegate to avoid duplication.
fn defines(&self, asg: &mut Asg, name: SPair) -> ObjectIndex<Ident> {
// This cannot simply call [`ReachableOnly`]'s `defines` because
// we cannot cache in the global environment.
// This can be realized once caching is generalized;
// see the commit that introduced this comment.
match self {
Self(oi_root) => ReachableOnly(*oi_root).defines(asg, id),
Self(oi_root) => oi_root.declare_local(asg, name),
}
}

View File

@ -45,9 +45,10 @@ pub fn collect_subexprs(
fn expr_empty_ident() {
let id = SPair("foo".into(), S2);
#[rustfmt::skip]
let toks = vec![
Air::ExprStart(ExprOp::Sum, S1),
Air::BindIdent(id),
Air::BindIdent(id),
Air::ExprEnd(S3),
];
@ -89,27 +90,30 @@ fn expr_without_pkg() {
fn close_pkg_mid_expr() {
let id = SPair("foo".into(), S4);
#[rustfmt::skip]
let toks = vec![
Air::PkgStart(S1),
Air::ExprStart(ExprOp::Sum, S2),
Air::ExprStart(ExprOp::Sum, S2),
Air::PkgEnd(S3),
// RECOVERY: Let's finish the expression first...
Air::BindIdent(id),
Air::ExprEnd(S5),
// RECOVERY: Let's finish the expression first...
Air::BindIdent(id),
Air::ExprEnd(S5),
// ...and then try to close again.
Air::PkgEnd(S6),
];
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // ExprStart
Err(ParseError::StateError(AsgError::InvalidPkgEndContext(S3))),
// RECOVERY: We should be able to close the package if we just
// finish the expression first,
// demonstrating that recovery properly maintains all state.
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
// RECOVERY: We should be able to close the package if we
// just finish the expression first,
// demonstrating that recovery properly maintains all
// state.
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
// Successful close here.
Ok(Parsed::Incomplete), // PkgEnd
],
@ -121,29 +125,31 @@ fn close_pkg_mid_expr() {
fn open_pkg_mid_expr() {
let id = SPair("foo".into(), S4);
#[rustfmt::skip]
let toks = vec![
Air::PkgStart(S1),
Air::ExprStart(ExprOp::Sum, S2),
Air::ExprStart(ExprOp::Sum, S2),
Air::PkgStart(S3),
// RECOVERY: We should still be able to complete successfully.
Air::BindIdent(id),
Air::ExprEnd(S5),
// RECOVERY: We should still be able to complete successfully.
Air::BindIdent(id),
Air::ExprEnd(S5),
// Closes the _original_ package.
Air::PkgEnd(S6),
];
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // ExprStart
Err(ParseError::StateError(AsgError::NestedPkgStart(S3, S1))),
// RECOVERY: Ignore the open and continue.
// Of course,
// this means that any identifiers would be defined in a
// different package than was likely intended,
// but at least we'll be able to keep processing.
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
// RECOVERY: Ignore the open and continue.
// Of course,
// this means that any identifiers would be defined in a
// different package than was likely intended,
// but at least we'll be able to keep processing.
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
Ok(Parsed::Incomplete), // PkgEnd
],
Sut::parse(toks.into_iter()).collect::<Vec<_>>(),
@ -155,15 +161,19 @@ fn expr_non_empty_ident_root() {
let id_a = SPair("foo".into(), S2);
let id_b = SPair("bar".into(), S2);
#[rustfmt::skip]
let toks = vec![
Air::ExprStart(ExprOp::Sum, S1),
// Identifier while still empty...
Air::BindIdent(id_a),
Air::ExprStart(ExprOp::Sum, S3),
// (note that the inner expression _does not_ have an ident binding)
Air::ExprEnd(S4),
// ...and an identifier non-empty.
Air::BindIdent(id_b),
// Identifier while still empty...
Air::BindIdent(id_a),
Air::ExprStart(ExprOp::Sum, S3),
// (note that the inner expression _does not_ have an ident
// binding)
Air::ExprEnd(S4),
// ...and an identifier non-empty.
Air::BindIdent(id_b),
Air::ExprEnd(S6),
];
@ -178,12 +188,6 @@ fn expr_non_empty_ident_root() {
// Identifiers should reference the same expression.
let expr_b = asg.expect_ident_obj::<Expr>(id_b);
assert_eq!(expr_a, expr_b);
// Ontological sanity check:
// Child expressions must not be considered cross edges since they are
// part of the same tree.
let oi_expr_a = asg.expect_ident_oi::<Expr>(id_a);
assert!(!oi_expr_a.edges(&asg).any(|rel| rel.is_cross_edge()));
}
// Binding an identifier after a child expression means that the parser is
@ -193,15 +197,17 @@ fn expr_non_empty_ident_root() {
fn expr_non_empty_bind_only_after() {
let id = SPair("foo".into(), S2);
#[rustfmt::skip]
let toks = vec![
Air::ExprStart(ExprOp::Sum, S1),
// Expression root is still dangling at this point.
Air::ExprStart(ExprOp::Sum, S2),
Air::ExprEnd(S3),
// We only bind an identifier _after_ we've created the expression,
// which should cause the still-dangling root to become
// reachable.
Air::BindIdent(id),
// Expression root is still dangling at this point.
Air::ExprStart(ExprOp::Sum, S2),
Air::ExprEnd(S3),
// We only bind an identifier _after_ we've created the expression,
// which should cause the still-dangling root to become
// reachable.
Air::BindIdent(id),
Air::ExprEnd(S5),
];
@ -231,10 +237,11 @@ fn expr_dangling_no_subexpr() {
let full_span = S1.merge(S2).unwrap();
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete),
Err(ParseError::StateError(AsgError::DanglingExpr(full_span))),
Ok(Parsed::Incomplete),
Err(ParseError::StateError(AsgError::DanglingExpr(full_span))),
// RECOVERY
Ok(Parsed::Incomplete), // PkgEnd
],
@ -244,11 +251,12 @@ fn expr_dangling_no_subexpr() {
#[test]
fn expr_dangling_with_subexpr() {
#[rustfmt::skip]
let toks = vec![
Air::ExprStart(ExprOp::Sum, S1),
// Expression root is still dangling at this point.
Air::ExprStart(ExprOp::Sum, S2),
Air::ExprEnd(S3),
// Expression root is still dangling at this point.
Air::ExprStart(ExprOp::Sum, S2),
Air::ExprEnd(S3),
// Still no ident binding,
// so root should still be dangling.
Air::ExprEnd(S4),
@ -257,12 +265,13 @@ fn expr_dangling_with_subexpr() {
let full_span = S1.merge(S4).unwrap();
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Err(ParseError::StateError(AsgError::DanglingExpr(full_span))),
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // ExprEnd
Err(ParseError::StateError(AsgError::DanglingExpr(full_span))),
// RECOVERY
Ok(Parsed::Incomplete), // PkgEnd
],
@ -274,32 +283,34 @@ fn expr_dangling_with_subexpr() {
fn expr_dangling_with_subexpr_ident() {
let id = SPair("foo".into(), S3);
#[rustfmt::skip]
let toks = vec![
Air::ExprStart(ExprOp::Sum, S1),
// Expression root is still dangling at this point.
Air::ExprStart(ExprOp::Sum, S2),
// The _inner_ expression receives an identifier,
// but that should have no impact on the dangling status of the
// root,
// especially given that subexpressions are always reachable
// anyway.
Air::BindIdent(id),
Air::ExprEnd(S4),
// But the root still has no ident binding,
// and so should still be dangling.
// Expression root is still dangling at this point.
Air::ExprStart(ExprOp::Sum, S2),
// The _inner_ expression receives an identifier,
// but that should have no impact on the dangling status of
// the root,
// especially given that subexpressions are always reachable
// anyway.
Air::BindIdent(id),
Air::ExprEnd(S4),
// But the root still has no ident binding,
// and so should still be dangling.
Air::ExprEnd(S5),
];
let full_span = S1.merge(S5).unwrap();
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Err(ParseError::StateError(AsgError::DanglingExpr(full_span))),
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIndent
Ok(Parsed::Incomplete), // ExprEnd
Err(ParseError::StateError(AsgError::DanglingExpr(full_span))),
// RECOVERY
Ok(Parsed::Incomplete), // PkgEnd
],
@ -314,11 +325,14 @@ fn expr_dangling_with_subexpr_ident() {
#[test]
fn expr_reachable_subsequent_dangling() {
let id = SPair("foo".into(), S2);
#[rustfmt::skip]
let toks = vec![
// Reachable
Air::ExprStart(ExprOp::Sum, S1),
Air::BindIdent(id),
Air::BindIdent(id),
Air::ExprEnd(S3),
// Dangling
Air::ExprStart(ExprOp::Sum, S4),
Air::ExprEnd(S5),
@ -329,13 +343,16 @@ fn expr_reachable_subsequent_dangling() {
let second_span = S4.merge(S5).unwrap();
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Err(ParseError::StateError(AsgError::DanglingExpr(second_span))),
// Reachable
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
// Dangling
Ok(Parsed::Incomplete),
Err(ParseError::StateError(AsgError::DanglingExpr(second_span))),
// RECOVERY
Ok(Parsed::Incomplete), // PkgEnd
],
@ -347,13 +364,16 @@ fn expr_reachable_subsequent_dangling() {
#[test]
fn recovery_expr_reachable_after_dangling() {
let id = SPair("foo".into(), S4);
#[rustfmt::skip]
let toks = vec![
// Dangling
Air::ExprStart(ExprOp::Sum, S1),
Air::ExprEnd(S2),
// Reachable, after error from dangling.
Air::ExprStart(ExprOp::Sum, S3),
Air::BindIdent(id),
Air::BindIdent(id),
Air::ExprEnd(S5),
];
@ -363,14 +383,16 @@ fn recovery_expr_reachable_after_dangling() {
let mut sut = parse_as_pkg_body(toks);
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete),
Err(ParseError::StateError(AsgError::DanglingExpr(err_span))),
// RECOVERY: continue at this point with the next expression.
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Err(ParseError::StateError(AsgError::DanglingExpr(err_span))),
// RECOVERY: continue at this point with the next expression.
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete),
Ok(Parsed::Incomplete), // PkgEnd
],
sut.by_ref().collect::<Vec<_>>(),
@ -396,14 +418,17 @@ fn recovery_expr_reachable_after_dangling() {
fn expr_close_unbalanced() {
let id = SPair("foo".into(), S3);
#[rustfmt::skip]
let toks = vec![
// Close before _any_ open.
Air::ExprEnd(S1),
// Should recover,
// allowing for a normal expr.
Air::ExprStart(ExprOp::Sum, S2),
Air::BindIdent(id),
Air::BindIdent(id),
Air::ExprEnd(S4),
// And now an extra close _after_ a valid expr.
Air::ExprEnd(S5),
];
@ -411,15 +436,18 @@ fn expr_close_unbalanced() {
let mut sut = parse_as_pkg_body(toks);
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Err(ParseError::StateError(AsgError::UnbalancedExpr(S1))),
// RECOVERY
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
// Another error after a successful expression.
Err(ParseError::StateError(AsgError::UnbalancedExpr(S5))),
Err(ParseError::StateError(AsgError::UnbalancedExpr(S1))),
// RECOVERY
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
// Another error after a successful expression.
Err(ParseError::StateError(AsgError::UnbalancedExpr(S5))),
// RECOVERY
Ok(Parsed::Incomplete), // PkgEnd
],
@ -440,20 +468,24 @@ fn expr_bind_to_empty() {
let id_good = SPair("good".into(), S6);
let id_noexpr_b = SPair("noexpr_b".into(), S8);
#[rustfmt::skip]
let toks = vec![
// We need to first bring ourselves out of the context of the
// package header,
// otherwise the bind will be interpreted as a bind to the
// package itself.
Air::ExprStart(ExprOp::Sum, S1),
Air::BindIdent(id_pre),
Air::BindIdent(id_pre),
Air::ExprEnd(S3),
// No open expression to bind to.
Air::BindIdent(id_noexpr_a),
// Post-recovery create an expression.
Air::ExprStart(ExprOp::Sum, S5),
Air::BindIdent(id_good),
Air::BindIdent(id_good),
Air::ExprEnd(S7),
// Once again we have nothing to bind to.
Air::BindIdent(id_noexpr_b),
];
@ -461,28 +493,32 @@ fn expr_bind_to_empty() {
let mut sut = parse_as_pkg_body(toks);
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
// Just to get out of a package header context
Ok(Parsed::Incomplete), // ExprStart (pre)
Ok(Parsed::Incomplete), // BindIdent (pre)
Ok(Parsed::Incomplete), // ExprEnd (pre)
// Now that we've encountered an expression,
// we want an error specific to expression binding,
// since it's likely that a bind token was issued too late,
// rather than trying to interpret this as being back in a
// package context and binding to the package.
Err(ParseError::StateError(AsgError::InvalidExprBindContext(
id_noexpr_a
))),
// RECOVERY
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
// Another error after a successful expression.
Err(ParseError::StateError(AsgError::InvalidExprBindContext(
id_noexpr_b
))),
// Just to get out of a package header context
Ok(Parsed::Incomplete), // ExprStart (pre)
Ok(Parsed::Incomplete), // BindIdent (pre)
Ok(Parsed::Incomplete), // ExprEnd (pre)
// Now that we've encountered an expression,
// we want an error specific to expression binding,
// since it's likely that a bind token was issued too late,
// rather than trying to interpret this as being back in a
// package context and binding to the package.
Err(ParseError::StateError(AsgError::InvalidExprBindContext(
id_noexpr_a
))),
// RECOVERY
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
// Another error after a successful expression.
Err(ParseError::StateError(AsgError::InvalidExprBindContext(
id_noexpr_b
))),
// RECOVERY
Ok(Parsed::Incomplete), // PkgEnd
],
@ -510,19 +546,23 @@ fn expr_bind_to_empty() {
fn sibling_subexprs_have_ordered_edges_to_parent() {
let id_root = SPair("root".into(), S1);
#[rustfmt::skip]
let toks = vec![
Air::ExprStart(ExprOp::Sum, S1),
// Identify the root so that it is not dangling.
Air::BindIdent(id_root),
// Sibling A
Air::ExprStart(ExprOp::Sum, S3),
Air::ExprEnd(S4),
// Sibling B
Air::ExprStart(ExprOp::Sum, S5),
Air::ExprEnd(S6),
// Sibling C
Air::ExprStart(ExprOp::Sum, S7),
Air::ExprEnd(S8),
// Identify the root so that it is not dangling.
Air::BindIdent(id_root),
// Sibling A
Air::ExprStart(ExprOp::Sum, S3),
Air::ExprEnd(S4),
// Sibling B
Air::ExprStart(ExprOp::Sum, S5),
Air::ExprEnd(S6),
// Sibling C
Air::ExprStart(ExprOp::Sum, S7),
Air::ExprEnd(S8),
Air::ExprEnd(S9),
];
@ -555,14 +595,17 @@ fn nested_subexprs_related_to_relative_parent() {
let id_root = SPair("root".into(), S1);
let id_suba = SPair("suba".into(), S2);
#[rustfmt::skip]
let toks = vec![
Air::ExprStart(ExprOp::Sum, S1), // 0
Air::BindIdent(id_root),
Air::ExprStart(ExprOp::Sum, S2), // 1
Air::BindIdent(id_suba),
Air::ExprStart(ExprOp::Sum, S3), // 2
Air::ExprEnd(S4),
Air::ExprEnd(S5),
Air::BindIdent(id_root),
Air::ExprStart(ExprOp::Sum, S2), // 1
Air::BindIdent(id_suba),
Air::ExprStart(ExprOp::Sum, S3), // 2
Air::ExprEnd(S4),
Air::ExprEnd(S5),
Air::ExprEnd(S6),
];
@ -591,30 +634,33 @@ fn expr_redefine_ident() {
let id_first = SPair("foo".into(), S2);
let id_dup = SPair("foo".into(), S3);
#[rustfmt::skip]
let toks = vec![
Air::ExprStart(ExprOp::Sum, S1),
Air::BindIdent(id_first),
Air::ExprStart(ExprOp::Sum, S3),
Air::BindIdent(id_dup),
Air::ExprEnd(S4),
Air::BindIdent(id_first),
Air::ExprStart(ExprOp::Sum, S3),
Air::BindIdent(id_dup),
Air::ExprEnd(S4),
Air::ExprEnd(S5),
];
let mut sut = parse_as_pkg_body(toks);
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent (first)
Ok(Parsed::Incomplete), // ExprStart
Err(ParseError::StateError(AsgError::IdentRedefine(
id_first,
id_dup.span(),
))),
// RECOVERY: Ignore the attempt to redefine and continue.
Ok(Parsed::Incomplete), // ExprEnd
Ok(Parsed::Incomplete), // ExprEnd
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent (first)
Ok(Parsed::Incomplete), // ExprStart
Err(ParseError::StateError(AsgError::IdentRedefine(
id_first,
id_dup.span(),
))),
// RECOVERY: Ignore the attempt to redefine and continue.
Ok(Parsed::Incomplete), // ExprEnd
Ok(Parsed::Incomplete), // ExprEnd
Ok(Parsed::Incomplete), // PkgEnd
],
sut.by_ref().collect::<Vec<_>>(),
@ -640,58 +686,64 @@ fn expr_still_dangling_on_redefine() {
let id_dup2 = SPair("foo".into(), S8);
let id_second = SPair("bar".into(), S9);
#[rustfmt::skip]
let toks = vec![
// First expr (OK)
Air::ExprStart(ExprOp::Sum, S1),
Air::BindIdent(id_first),
Air::BindIdent(id_first),
Air::ExprEnd(S3),
// Second expr should still dangle due to use of duplicate
// identifier
Air::ExprStart(ExprOp::Sum, S4),
Air::BindIdent(id_dup),
Air::BindIdent(id_dup),
Air::ExprEnd(S6),
// Third expr will error on redefine but then be successful.
// This probably won't happen in practice with TAME's original
// source language,
// but could happen at e.g. a REPL.
Air::ExprStart(ExprOp::Sum, S7),
Air::BindIdent(id_dup2), // fail
Air::BindIdent(id_second), // succeed
Air::BindIdent(id_dup2), // fail
Air::BindIdent(id_second), // succeed
Air::ExprEnd(S10),
];
let mut sut = parse_as_pkg_body(toks);
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent (first)
Ok(Parsed::Incomplete), // ExprEnd
// Beginning of second expression
Ok(Parsed::Incomplete), // ExprStart
Err(ParseError::StateError(AsgError::IdentRedefine(
id_first,
id_dup.span(),
))),
// RECOVERY: Ignore the attempt to redefine and continue.
// ...but then immediately fail _again_ because we've closed a
// dangling expression.
Err(ParseError::StateError(AsgError::DanglingExpr(
S4.merge(S6).unwrap()
))),
// RECOVERY: But we'll continue onto one final expression,
// which we will fail to define but then subsequently define
// successfully.
Ok(Parsed::Incomplete), // ExprStart
Err(ParseError::StateError(AsgError::IdentRedefine(
id_first,
id_dup2.span(),
))),
// RECOVERY: Despite the initial failure,
// we can now re-attempt to bind with a unique id.
Ok(Parsed::Incomplete), // BindIdent (second)
Ok(Parsed::Incomplete), // ExprEnd
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent (first)
Ok(Parsed::Incomplete), // ExprEnd
// Beginning of second expression
Ok(Parsed::Incomplete), // ExprStart
Err(ParseError::StateError(AsgError::IdentRedefine(
id_first,
id_dup.span(),
))),
// RECOVERY: Ignore the attempt to redefine and continue.
// ...but then immediately fail _again_ because we've closed a
// dangling expression.
Err(ParseError::StateError(AsgError::DanglingExpr(
S4.merge(S6).unwrap()
))),
// RECOVERY: But we'll continue onto one final expression,
// which we will fail to define but then subsequently define
// successfully.
Ok(Parsed::Incomplete), // ExprStart
Err(ParseError::StateError(AsgError::IdentRedefine(
id_first,
id_dup2.span(),
))),
// RECOVERY: Despite the initial failure,
// we can now re-attempt to bind with a unique id.
Ok(Parsed::Incomplete), // BindIdent (second)
Ok(Parsed::Incomplete), // ExprEnd
Ok(Parsed::Incomplete), // PkgEnd
],
sut.by_ref().collect::<Vec<_>>(),
@ -717,20 +769,23 @@ fn expr_ref_to_ident() {
let id_foo = SPair("foo".into(), S2);
let id_bar = SPair("bar".into(), S6);
#[rustfmt::skip]
let toks = vec![
Air::ExprStart(ExprOp::Sum, S1),
Air::BindIdent(id_foo),
// Reference to an as-of-yet-undefined id (okay),
// with a different span than `id_bar`.
Air::RefIdent(SPair("bar".into(), S3)),
Air::BindIdent(id_foo),
// Reference to an as-of-yet-undefined id (okay),
// with a different span than `id_bar`.
Air::RefIdent(SPair("bar".into(), S3)),
Air::ExprEnd(S4),
//
// Another expression to reference the first
// (we don't handle cyclic references until a topological sort,
// so no point in referencing ourselves;
// it'd work just fine here.)
Air::ExprStart(ExprOp::Sum, S5),
Air::BindIdent(id_bar),
Air::BindIdent(id_bar),
Air::ExprEnd(S7),
];
@ -746,12 +801,6 @@ fn expr_ref_to_ident() {
// We should have only a single reference (to `id_bar`).
assert_eq!(foo_rels.len(), 1);
// Ontological sanity check:
// references to identifiers should count as cross edges.
// This is very important to ensure that certain graph traversals work
// correctly between trees.
assert!(foo_rels.iter().all(|rel| rel.is_cross_edge()));
let oi_ident_bar =
foo_rels.pop().and_then(ExprRel::narrow::<Ident>).unwrap();
let ident_bar = oi_ident_bar.resolve(&asg);
@ -776,36 +825,42 @@ fn expr_ref_outside_of_expr_context() {
let id_pre = SPair("pre".into(), S2);
let id_foo = SPair("foo".into(), S4);
#[rustfmt::skip]
let toks = vec![
// We need to first bring ourselves out of the context of the
// package header.
Air::ExprStart(ExprOp::Sum, S1),
Air::BindIdent(id_pre),
Air::BindIdent(id_pre),
Air::ExprEnd(S3),
// This will fail since we're not in an expression context.
Air::RefIdent(id_foo),
// RECOVERY: Simply ignore the above.
Air::ExprStart(ExprOp::Sum, S1),
Air::BindIdent(id_foo),
Air::BindIdent(id_foo),
Air::ExprEnd(S3),
];
let mut sut = parse_as_pkg_body(toks);
assert_eq!(
#[rustfmt::skip]
vec![
Ok(Parsed::Incomplete), // PkgStart
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
// Now we're past the header and in expression parsing mode.
Err(ParseError::StateError(AsgError::InvalidExprRefContext(
id_foo
))),
// RECOVERY: Proceed as normal
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
// Now we're past the header and in expression parsing mode.
Err(ParseError::StateError(AsgError::InvalidExprRefContext(
id_foo
))),
// RECOVERY: Proceed as normal
Ok(Parsed::Incomplete), // ExprStart
Ok(Parsed::Incomplete), // BindIdent
Ok(Parsed::Incomplete), // ExprEnd
Ok(Parsed::Incomplete), // PkgEnd
],
sut.by_ref().collect::<Vec<_>>(),
@ -826,15 +881,17 @@ fn idents_share_defining_pkg() {
let id_baz = SPair("baz".into(), S6);
// An expression nested within another.
#[rustfmt::skip]
let toks = vec![
Air::PkgStart(S1),
Air::ExprStart(ExprOp::Sum, S2),
Air::BindIdent(id_foo),
Air::ExprStart(ExprOp::Sum, S4),
Air::BindIdent(id_bar),
Air::RefIdent(id_baz),
Air::ExprEnd(S7),
Air::ExprEnd(S8),
Air::ExprStart(ExprOp::Sum, S2),
Air::BindIdent(id_foo),
Air::ExprStart(ExprOp::Sum, S4),
Air::BindIdent(id_bar),
Air::RefIdent(id_baz),
Air::ExprEnd(S7),
Air::ExprEnd(S8),
Air::PkgEnd(S9),
];
@ -842,22 +899,16 @@ fn idents_share_defining_pkg() {
assert!(sut.all(|x| x.is_ok()));
let asg = sut.finalize().unwrap().into_context();
let oi_foo = asg.lookup(id_foo).unwrap();
let oi_bar = asg.lookup(id_bar).unwrap();
let oi_foo = asg.lookup_global(id_foo).unwrap();
let oi_bar = asg.lookup_global(id_bar).unwrap();
assert_eq!(oi_foo.src_pkg(&asg).unwrap(), oi_bar.src_pkg(&asg).unwrap());
// Missing identifiers should not have a source package,
// since we don't know what defined it yet.
let oi_baz = asg.lookup(id_baz).unwrap();
let oi_baz = asg.lookup_global(id_baz).unwrap();
assert_eq!(None, oi_baz.src_pkg(&asg));
// Ontological sanity check:
// edges from the package to identifiers defined by it should not be
// considered cross edges.
let oi_pkg = oi_foo.src_pkg(&asg).unwrap();
assert!(oi_pkg.edges(&asg).all(|rel| !rel.is_cross_edge()));
// The package span should encompass the entire definition.
assert_eq!(
S1.merge(S9),

View File

@ -628,13 +628,47 @@ sum_ir! {
/// metasyntactic variables,
/// defined by [`AirTpl::TplMetaStart`].
///
/// If a template contains any metavariables (parameters) without
/// bound values,
/// those metavariables are said to be _free_.
/// Values may be applied to templates using [`AirTpl::TplApply`],
/// binding those values to their associated metavariables.
/// A template with no free metavariables is _closed_ and may be
/// expanded in-place.
/// Template Application
/// ====================
/// _Application_ is triggered by a [`Air::RefIdent`] to a [`Tpl`]
/// or a [`Air::TplEndRef`] and will re-bind an inner template to
/// the metavariables in the current context and expand the
/// template in place.
/// A metavariable is _free_ if it has no bound value.
/// A template is _closed_ if it has no free metavariables.
/// Application _expands_ into its context,
/// and application of a closed template is equivalent to the
/// notion of template application in NIR / TAME's source language.
///
/// Let α be the current template definition context
/// (via [`Air::TplStart`])
/// and let β be the inner template.
/// All free metavariables in β that contain default values in α
/// (via [`Air::TplMetaStart`])
/// corresponding to the same [`Ident`] will be _bound_ to
/// that value.
/// The body of the inner template β will be expanded into the
/// body of α.
/// The result is a template α which is the application of its
/// parameters to β.
///
/// Partial application is not yet supported,
/// but can be added if it is worth the effort of doing so.
/// This simplifies the semantics of this operation:
///
/// - All metavariables that are still free in β after binding
/// will assume their default values, if any; and
/// - All metavariables that are still free in β after
/// applying defaults will result in an error.
///
/// Consequently,
/// the template α will always be closed after this operation.
/// This can be thought of like a lexical thunk.
///
/// α can then be expanded in place using [`Air::TplEndRef`] if
/// it is anonymous.
/// If α is named,
/// [`Air::RefIdent`] can be used to trigger expansion.
enum AirTpl {
/// Create a new [`Tpl`] on the graph and switch to template parsing.
///
@ -657,14 +691,14 @@ sum_ir! {
/// [`AirBind::BindIdent`] before [`Self::TplMetaEnd`].
///
/// Metavariables may contain default values,
/// making their specification during [`Self::TplApply`] optional.
/// making their specification during application optional.
/// A metavariable may contain an ordered mixture of references
/// to another metavariables via [`AirBind::RefIdent`] and
/// literals via [`Self::TplLexeme`].
/// Once all metavariable references have been satisfied during
/// [`Self::TplApply`],
/// application,
/// all children will be combined into a single lexeme to
/// serve as a final identifier.
/// serve as a final identifier.
///
/// The interpretation of a metavariable depends solely on the
/// context in which it is referenced.
@ -694,43 +728,6 @@ sum_ir! {
),
},
/// Re-bind an inner template to the metavariables in the
/// current context and expand the template in place.
///
/// Let α be the current template definition context
/// (via [`Self::TplStart`])
/// and let β be the inner template.
/// All free metavariables in β that contain default values in α
/// (via [`Self::TplMetaStart`])
/// corresponding to the same [`Ident`] will be _bound_ to
/// that value.
/// The body of the inner template β will be expanded into the
/// body of α.
/// The result is a template α which is the application of its
/// parameters to β.
///
/// Partial application is not yet supported,
/// but can be added if it is worth the effort of doing so.
/// This simplifies the semantics of this operation:
///
/// - All metavariables that are still free in β after binding
/// will assume their default values, if any; and
/// - All metavariables that are still free in β after
/// applying defaults will result in an error.
///
/// Consequently,
/// the template α will always be closed after this operation.
/// This can be thought of like a lexical thunk.
///
/// α can then be expanded in place using [`Self::TplEndRef`] if
/// it is anonymous.
/// If α is named,
/// [`Air::RefIdent`] can be used to trigger expansion.
TplApply(span: Span) => {
span: span,
display: |f| write!(f, "apply param bindings to inner template"),
},
/// Complete the active [`Tpl`] and exit template parsing.
///
/// The expression stack will be restored to its prior state.

View File

@ -46,7 +46,9 @@ fn ident_decl() {
let asg = sut.finalize().unwrap().into_context();
let ident_node = asg.lookup(id).expect("identifier was not added to graph");
let ident_node = asg
.lookup_global(id)
.expect("identifier was not added to graph");
let ident = asg.get(ident_node).unwrap();
assert_eq!(
@ -85,7 +87,9 @@ fn ident_extern_decl() {
let asg = sut.finalize().unwrap().into_context();
let ident_node = asg.lookup(id).expect("identifier was not added to graph");
let ident_node = asg
.lookup_global(id)
.expect("identifier was not added to graph");
let ident = asg.get(ident_node).unwrap();
assert_eq!(
@ -122,8 +126,10 @@ fn ident_dep() {
let asg = sut.finalize().unwrap().into_context();
let ident_node = asg.lookup(id).expect("identifier was not added to graph");
let dep_node = asg.lookup(dep).expect("dep was not added to graph");
let ident_node = asg
.lookup_global(id)
.expect("identifier was not added to graph");
let dep_node = asg.lookup_global(dep).expect("dep was not added to graph");
assert!(asg.has_dep(ident_node, dep_node));
}
@ -153,7 +159,9 @@ fn ident_fragment() {
let asg = sut.finalize().unwrap().into_context();
let ident_node = asg.lookup(id).expect("identifier was not added to graph");
let ident_node = asg
.lookup_global(id)
.expect("identifier was not added to graph");
let ident = asg.get(ident_node).unwrap();
assert_eq!(
@ -189,7 +197,7 @@ fn ident_root_missing() {
let asg = sut.finalize().unwrap().into_context();
let ident_node = asg
.lookup(id)
.lookup_global(id)
.expect("identifier was not added to the graph");
let ident = asg.get(ident_node).unwrap();
@ -228,7 +236,7 @@ fn ident_root_existing() {
let asg = sut.finalize().unwrap().into_context();
let ident_node = asg
.lookup(id)
.lookup_global(id)
.expect("identifier was not added to the graph");
let ident = asg.get(ident_node).unwrap();

View File

@ -31,6 +31,7 @@ use super::{
AirExprAggregate,
};
use crate::{
asg::graph::object::Meta,
diagnose::Annotate,
diagnostic_todo,
fmt::{DisplayWrapper, TtQuote},
@ -63,12 +64,26 @@ pub enum AirTplAggregate {
/// which simplifies AIR generation.
Ready(ObjectIndex<Pkg>),
/// Toplevel template context.
///
/// Conceptually,
/// tokens that are received in this state are interpreted as directly
/// applying to the template itself,
/// or creating an object directly owned by the template.
Toplevel(
ObjectIndex<Pkg>,
TplState,
AirExprAggregateStoreDangling<Tpl>,
),
/// Defining a template metavariable.
TplMeta(
ObjectIndex<Pkg>,
TplState,
AirExprAggregateStoreDangling<Tpl>,
ObjectIndex<Meta>,
),
/// Aggregating tokens into a template.
TplExpr(
ObjectIndex<Pkg>,
@ -85,6 +100,10 @@ impl Display for AirTplAggregate {
Self::Toplevel(_, tpl, expr) | Self::TplExpr(_, tpl, expr) => {
write!(f, "building {tpl} with {expr}")
}
Self::TplMeta(_, tpl, _, _) => {
write!(f, "building {tpl} metavariable")
}
}
}
}
@ -191,7 +210,7 @@ impl ParseState for AirTplAggregate {
),
(Toplevel(oi_pkg, tpl, expr), AirBind(BindIdent(id))) => asg
.lookup_or_missing(id)
.lookup_global_or_missing(id)
.bind_definition(asg, id, tpl.oi())
.map(|oi_ident| oi_pkg.defines(asg, oi_ident))
.map(|_| ())
@ -202,20 +221,69 @@ impl ParseState for AirTplAggregate {
Transition(Toplevel(oi_pkg, tpl, expr)).incomplete()
}
(Toplevel(oi_pkg, tpl, expr), AirTpl(TplMetaStart(span))) => {
let oi_meta = asg.create(Meta::new_required(span));
Transition(TplMeta(oi_pkg, tpl, expr, oi_meta)).incomplete()
}
(
Toplevel(..),
tok @ AirTpl(TplMetaStart(..) | TplMetaEnd(..) | TplApply(..)),
TplMeta(oi_pkg, tpl, expr, oi_meta),
AirTpl(TplMetaEnd(cspan)),
) => {
oi_meta.close(asg, cspan);
Transition(Toplevel(oi_pkg, tpl, expr)).incomplete()
}
(
TplMeta(oi_pkg, tpl, expr, oi_meta),
AirTpl(TplLexeme(lexeme)),
) => Transition(TplMeta(
oi_pkg,
tpl,
expr,
oi_meta.assign_lexeme(asg, lexeme),
))
.incomplete(),
(TplMeta(oi_pkg, tpl, expr, oi_meta), AirBind(BindIdent(name))) => {
oi_meta.identify_as_tpl_param(asg, tpl.oi(), name);
Transition(TplMeta(oi_pkg, tpl, expr, oi_meta)).incomplete()
}
(TplMeta(..), tok @ AirBind(RefIdent(..))) => {
diagnostic_todo!(
vec![tok.note("for this token")],
"Toplevel meta"
vec![tok.note("this token")],
"AirBind in metavar context (param-value)"
)