tamer: nir::interp: Ignore Text tokens

The documentation explains the intent here---existing LaTeX documentation.

The intent was to simply copy the documentation into a LaTeX document based
on the lvspec package that I had created long ago.  Of course, that's not
appropriate---we're a DSL and should provide first-class support for
documentation that will compile properly into the target format, whether it
be LaTeX, HTML, JS, or anything else.

DEV-13162
main
Mike Gerwitz 2023-04-30 14:48:33 -04:00
parent 068804b397
commit 56ab671363
2 changed files with 23 additions and 0 deletions

View File

@ -250,6 +250,13 @@ impl ParseState for InterpState {
let span = tok.span();
match self {
// TODO: For now we must maintain BC with existing documentation
// that was written as LaTeX,
// which naturally contains many curly braces.
// This can go away once we introduce a proper documentation
// language.
Ready if matches!(tok, Nir::Text(..)) => Transition(Ready).ok(tok),
// When receiving a new symbol,
// we must make a quick determination as to whether it
// requires desugaring.

View File

@ -63,6 +63,22 @@ fn does_not_desugar_tokens_without_symbols() {
);
}
// We ought to desugar text at some point,
// but we need to maintain BC with existing systems.
// The literate documentation was intended to be LaTeX back in the day,
// rather than a proper abstraction atop of it,
// and so there are many curly braces in existing code.
#[test]
fn does_not_desugar_text() {
let spair = SPair("\\footnote{foo}".into(), S1);
let toks = vec![Nir::Text(spair)];
assert_eq!(
Ok(vec![Object(Nir::Text(spair))]),
Sut::parse(toks.into_iter()).collect(),
);
}
fn expect_expanded_header(
sut: &mut Parser<InterpState, std::vec::IntoIter<Nir>>,
given_val: &str,