employer
/
tame
1
0
Fork 0
Code Packages Releases Activity

tamer: src::asg: Scaffolding for metasyntactic variables 

Also known as metavariables or template parameters.

This is a bit of a tortured excursion, trying to figure out how I want to
best represent this.  I have a number of pages of hand-written notes that
I'd like to distill over time, but the rendered graph ontology (via
`asg-ontviz`) demonstrates the broad idea.

`AirTpl::TplApply` highlights some remaining questions.  What I had _wanted_
to do is to separate the concepts of application and expansion, and support
partial application and such.  But it's going to be too much work for now,
when it isn't needed---partial application can be worked around by simply
creating new templates and duplicating params, as we do today, although that
sucks and is a maintenance issue.  But I'd rather address that head-on in
the future.

So it's looking like Option B is going to be the approach for now, with
templates being closed (as in, no free metavariables) and expanded at the
same time.  This simplifies the parser and error conditions significantly
and makes it easier to utilize anonymous templates, since it'll still be the
active context.

My intent is to get at least the graph construction sorted out---not the
actual expansion and binding yet---enough that I can use templates to
represent parts of NIR that do not have proper graph representations or
desugaring yet, so that I can spit them back out again in the `xmli` file
and incrementally handle them.  That was an option I had considered some
months ago, but didn't want to entertain it at the time because I wasn't
sure what doing so would look like; while it was an attractive approach
since it pushes existing primitives into the template system (something I've
wanted to do for years), I didn't want to potentially tank performance or
compromise the design for it after I had spent so much effort on all of this
so far.

But my efforts have yielded a system that significantly exceeds my initial
performance expectations, with a decent abstractions, and so this seems
viable.

DEV-13708
main
Mike Gerwitz 2023-03-15 11:49:13 -04:00
parent 9e5958d89e
commit be81878dd7
10 changed files with 241 additions and 17 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
tamer/src/asg/air.rs
View File

@ -118,7 +118,6 @@ impl ParseState for AirAggregate {
) -> crate::parse::TransitionResult<Self> {
use ir::{
AirBind::*, AirIdent::*, AirPkg::*, AirSubsets::*, AirTodo::*,
AirTpl::*,
};
use AirAggregate::*;
@ -202,10 +201,6 @@ impl ParseState for AirAggregate {
todo!("templates cannot contain packages")
}
(Empty, AirTpl(TplEnd(..))) => {
todo!("Empty AirTpl::TplEnd")
}
(Empty, AirPkg(PkgEnd(span))) => {
Transition(Empty).err(AsgError::InvalidPkgEndContext(span))
}
@ -232,10 +227,9 @@ impl ParseState for AirAggregate {
}
}
(
Empty,
tok @ (AirExpr(..) | AirBind(..) | AirTpl(TplStart(_))),
) => Transition(Empty).err(AsgError::PkgExpected(tok.span())),
(Empty, tok @ (AirExpr(..) | AirBind(..) | AirTpl(..))) => {
Transition(Empty).err(AsgError::PkgExpected(tok.span()))
}
(Empty, AirIdent(IdentDecl(name, kind, src))) => {
asg.declare(name, kind, src).map(|_| ()).transition(Empty)

109
tamer/src/asg/air/ir.rs
View File

@ -621,6 +621,18 @@ sum_ir! {
}
/// Subset of [`Air`] tokens for defining [`Tpl`]s.
///
/// Templates serve as containers for objects that reference
/// metasyntactic variables,
/// defined by [`AirTpl::TplMetaStart`].
///
/// If a template contains any metavariables (parameters) without
/// bound values,
/// those metavariables are said to be _free_.
/// Values may be applied to templates using [`AirTpl::TplApply`],
/// binding those values to their associated metavariables.
/// A template with no free metavariables is _closed_ and may be
/// expanded in-place.
enum AirTpl {
/// Create a new [`Tpl`] on the graph and switch to template parsing.
///
@ -637,6 +649,101 @@ sum_ir! {
display: |f| write!(f, "open template"),
},
/// Begin a metavariable definition.
///
/// A metavariable is anonymous unless identified via
/// [`AirBind::BindIdent`] before [`Self::TplMetaEnd`].
///
/// Metavariables may contain default values,
/// making their specification during [`Self::TplApply`] optional.
/// A metavariable may contain an ordered mixture of references
/// to another metavariables via [`AirBind::RefIdent`] and
/// literals via [`Self::TplLexeme`].
/// Once all metavariable references have been satisfied during
/// [`Self::TplApply`],
/// all children will be combined into a single lexeme to
/// serve as a final identifier.
///
/// The interpretation of a metavariable depends solely on the
/// context in which it is referenced.
TplMetaStart(span: Span) => {
span: span,
display: |f| write!(
f,
"open definition of metasyntactic variable",
),
},
/// A lexeme to be interpreted in the context of a template
/// expansion.
TplLexeme(lex: SPair) => {
span: lex,
display: |f| write!(f, "lexeme {}", TtQuote::wrap(lex)),
},
/// Complete a metavariable definition.
///
/// See [`Self::TplMetaStart`] for more information.
TplMetaEnd(span: Span) => {
span: span,
display: |f| write!(
f,
"close definition of metasyntactic variable",
),
},
/// Re-bind an inner template to the metavariables in the
/// current context.
///
/// Let α be the current template definition context
/// (via [`Self::TplStart`])
/// and let β be the inner template.
/// All free metavariables in β that contain default values in α
/// (via [`Self::TplMetaStart`])
/// corresponding to the same [`Ident`] will be _bound_ to
/// that value.
///
/// The body of the inner template β will become the body of α.
///
/// TODO
/// ====
/// Pick one of these!
///
/// Option A
/// --------
/// The result of this template application is a new template α
/// whose params are the still-free params of β after having
/// applied the aforementioned operation.
/// If α contains no more free metavariables,
/// then it is _closed_ and ready for expansion.
///
/// TODO: Defaults applied or deferred?
///
/// Option B
/// --------
/// Partial application is not yet supported,
/// but can be added if it is worth the effort of doing so.
/// This simplifies the semantics of this operation:
///
/// - All metavariables that are still free in β after binding
/// will assume their default values, if any; and
/// - All metavariables that are still free in β after
/// applying defaults will result in an error.
///
/// Consequently,
/// the template α will always be closed after this operation
/// and ready for expansion.
///
/// TODO: Maybe just make this `TplApplyExpand` to do both,
/// then,
/// like TAME works today?
/// Separating them can come later if there's value in that
/// effort.
TplApply(span: Span) => {
span: span,
display: |f| write!(f, "apply param bindings to inner template"),
},
/// Close the active [`Tpl`] and exit template parsing.
///
/// The expression stack will be restored to its prior state.
@ -644,6 +751,8 @@ sum_ir! {
span: span,
display: |f| write!(f, "close template"),
},
// TODO: Separate expand or not? See TplApply above.
}
}

23
tamer/src/asg/air/tpl.rs
View File

@ -140,6 +140,17 @@ impl ParseState for AirTplAggregate {
todo!("tpl Toplevel RefIdent")
}
(
Toplevel(..),
tok @ AirTpl(TplMetaStart(..) | TplMetaEnd(..) | TplApply(..)),
) => {
todo!("Toplevel meta {tok:?}")
}
(Toplevel(..), tok @ AirTpl(TplLexeme(..))) => {
todo!("err: Toplevel lexeme {tok:?} (must be within metavar)")
}
(Toplevel(oi_pkg, oi_tpl, _expr_done, _), AirTpl(TplEnd(span))) => {
oi_tpl.close(asg, span);
Transition(Ready(oi_pkg)).incomplete()
@ -172,6 +183,18 @@ impl ParseState for AirTplAggregate {
todo!("nested template (template-generated template)")
}
(
Ready(..) | TplExpr(..),
tok @ AirTpl(
TplMetaStart(..) | TplLexeme(..) | TplMetaEnd(..)
| TplApply(..),
),
) => {
todo!(
"metasyntactic token in non-tpl-toplevel context: {tok:?}"
)
}
(st @ Ready(..), AirTpl(TplEnd(span))) => {
Transition(st).err(AsgError::UnbalancedTpl(span))
}

6
tamer/src/asg/graph/object.rs
View File

@ -132,12 +132,14 @@ mod rel;
pub mod expr;
pub mod ident;
pub mod meta;
pub mod pkg;
pub mod root;
pub mod tpl;
pub use expr::Expr;
pub use ident::Ident;
pub use meta::Meta;
pub use pkg::Pkg;
pub use rel::{
DynObjectRel, ObjectRel, ObjectRelFrom, ObjectRelTo, ObjectRelTy,
@ -314,6 +316,9 @@ object_gen! {
/// A template definition.
Tpl,
/// Metasyntactic variable (metavariable).
Meta,
}
impl Object<OnlyObjectInner> {
@ -324,6 +329,7 @@ impl Object<OnlyObjectInner> {
Self::Ident(ident) => ident.span(),
Self::Expr(expr) => expr.span(),
Self::Tpl(tpl) => tpl.span(),
Self::Meta(meta) => meta.span(),
}
}

7
tamer/src/asg/graph/object/ident.rs
View File

@ -21,7 +21,7 @@
use super::{
super::{Asg, AsgError, ObjectIndex, ObjectKind},
Expr, Object, ObjectRel, ObjectRelFrom, ObjectRelTo, ObjectRelTy,
Expr, Meta, Object, ObjectRel, ObjectRelFrom, ObjectRelTo, ObjectRelTy,
ObjectRelatable, Pkg, Tpl,
};
use crate::{
@ -978,12 +978,13 @@ object_rel! {
/// Opaque identifiers at the time of writing are used by the linker
/// which does not reason about cross edges
/// (again at the time of writing).
/// Consequently,
/// this will always return [`false`].
Ident -> {
tree Ident,
tree Expr,
tree Tpl,
// A metavariable is directly referenced by a template.
cross Meta,
}
}

87
tamer/src/asg/graph/object/meta.rs 100644
View File

@ -0,0 +1,87 @@
// Metasyntactic variables represented on the ASG
//
// Copyright (C) 2014-2023 Ryan Specialty, LLC.
//
// This file is part of TAME.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
//! Metasyntactic variables on the ASG.
//!
//! Metasyntactic variables
//! (sometimes called "metavariables" herein for short)
//! have historically been a feature of the template system.
//! The canonical metavariable is the template parameter.
use super::{
Ident, Object, ObjectIndex, ObjectRel, ObjectRelFrom, ObjectRelTy,
ObjectRelatable,
};
use crate::{
fmt::{DisplayWrapper, TtQuote},
parse::{util::SPair, Token},
span::Span,
};
use std::fmt::Display;
/// Metasyntactic variable (metavariable).
///
/// A metavariable is a lexical construct.
/// Its value is a lexeme that represents an [`Ident`],
/// whose meaning depends on the context in which the metavariable is
/// referenced.
/// Its lexeme may be composed of multiple [`Self::Lexeme`]s,
/// and may even be constructed dynamically based on the values of other
/// [`Meta`]s.
///
/// Metavariables are identified by being bound by an [`Ident`];
/// the symbol representing that identifier then acts as a metavariable.
#[derive(Debug, PartialEq, Eq)]
pub enum Meta {
Required(Span),
ConcatList(Span),
Lexeme(SPair),
}
impl Meta {
pub fn span(&self) -> Span {
match self {
Self::Required(span) | Self::ConcatList(span) => *span,
Self::Lexeme(spair) => spair.span(),
}
}
}
impl Display for Meta {
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
match self {
Self::Required(_) => {
write!(f, "metasyntactic parameter with required value")
}
Self::ConcatList(_) => {
write!(f, "metasyntactic concatenation list")
}
Self::Lexeme(spair) => write!(f, "lexeme {}", TtQuote::wrap(spair)),
}
}
}
object_rel! {
/// Templates may expand into nearly any context,
/// and must therefore be able to contain just about anything.
Meta -> {
tree Meta,
cross Ident,
}
}

5
tamer/src/asg/graph/object/rel.rs
View File

@ -21,7 +21,8 @@
//! See (parent module)[super] for more information.
use super::{
Expr, Ident, Object, ObjectIndex, ObjectKind, OiPairObjectInner, Pkg, Root,
Expr, Ident, Meta, Object, ObjectIndex, ObjectKind, OiPairObjectInner, Pkg,
Root,
};
use crate::{
asg::{graph::object::Tpl, Asg},
@ -255,7 +256,7 @@ impl<S> DynObjectRel<S, ObjectIndex<Object>> {
}
}
ty_cross_edge!(Root, Pkg, Ident, Expr, Tpl)
ty_cross_edge!(Root, Pkg, Ident, Expr, Tpl, Meta)
}
}

5
tamer/src/asg/graph/object/tpl.rs
View File

@ -22,8 +22,8 @@
use std::fmt::Display;
use super::{
Expr, Ident, Object, ObjectIndex, ObjectRel, ObjectRelFrom, ObjectRelTy,
ObjectRelatable,
Expr, Ident, Meta, Object, ObjectIndex, ObjectRel, ObjectRelFrom,
ObjectRelTy, ObjectRelatable,
};
use crate::{asg::Asg, f::Functor, span::Span};
@ -63,6 +63,7 @@ object_rel! {
Tpl -> {
tree Ident,
tree Expr,
tree Meta,
}
}

2
tamer/src/asg/graph/xmli.rs
View File

@ -198,6 +198,8 @@ impl<'a> TreeContext<'a> {
self.emit_template(tpl, paired_rel.source(), depth)
}
target @ Object::Meta(..) => todo!("Object::Meta: {target:?}"),
Object::Root(..) => diagnostic_unreachable!(
vec![],
"tree walk is not expected to emit Root",

2
tamer/src/ld/xmle/lower.rs
View File

@ -72,7 +72,7 @@ where
Object::Root(_) => (),
Object::Ident(ident) => dest.push(ident)?,
obj @ (Object::Pkg(_) | Object::Expr(_) | Object::Tpl(_)) => {
obj => {
diagnostic_unreachable!(
obj.internal_error(
"this object should not be present on the graph"