employer
/
tame
1
0
Fork 0
Code Packages Releases Activity
tame/tamer/src/xir/parse.rs

30 lines
1.1 KiB
Rust
Raw Normal View History

tamer: xir::parse: Attribute parser generator This is the first parser generator for the parsing framework. I've been waiting quite a while to do this because I wanted to be sure that I understood how I intended to write the attribute parsers manually. Now that I'm about to start parsing source XML files, it is necessary to have a parser generator. Typically one thinks of a parser generator as a separate program that generates code for some language, but that is not always the case---that represents a lack of expressiveness in the language itself (e.g. C). Here, I simply use Rust's macro system, which should be a concept familiar to someone coming from a language like Lisp. This also resolves where I stand on parser combinators with respect to this abstraction: they both accomplish the exact same thing (composition of smaller parsers), but this abstraction doesn't do so in the typical functional way. But the end result is the same. The parser generated by this abstraction will be optimized an inlined in the same manner as the hand-written parsers. Since they'll be tightly coupled with an element parser (which too will have a parser generator), I expect that most attribute parsers will simply be inlined; they exist as separate parsers conceptually, for the same reason that you'd use parser combinators. It's worth mentioning that this awkward reliance on dead state for a lookahead token to determine when aggregation is complete rubs me the wrong way, but resolving it would involve reintroducing the XIR AttrEnd that I had previously removed. I'll keep fighting with myself on this, but I want to get a bit further before I determine if it's worth the tradeoff of reintroducing (more complex IR but simplified parsing). DEV-7145
2022-06-13 11:17:21 -04:00
// XIR parser generators
//
// Copyright (C) 2014-2022 Ryan Specialty Group, LLC.
//
// This file is part of TAME.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
//! Parser generators for parsing of [XIRF](super::flat).
//!
//! XIRF is chosen as the input IR because it handles some initial
//! processing of the input XML to ensure well-formedness.
mod attr;
tamer: xir::parse::ele: Initial element parser generator concept This begins generating parsers that are capable of parsing elements. I need to move on, so this abstraction isn't going to go as far as it could, but let's see where it takes me. This was the work that required the recent lookahead changes, which has been detailed in previous commits. This initial support is basic, but robust. It supports parsing elements with attributes and children, but it does not yet support the equivalent of the Kleene star (`*`). Such support will likely be added by supporting parsers that are able to recurse on their own definition in tail position, which will also require supporting parsers that do not add to the stack. This generates parsers that, like all the other parsers, use enums to provide a typed stack. Stitched parsers produce a nested stack that is always bounded in size. Fortunately, expressions---which can nest deeply---do not need to maintain ancestor context on the stack, and so this should work fine; we can get away with this because XIRF ensures proper nesting for us. Statements that _do_ need to maintain such context are not nested. This also does not yet support emitting an object on closing tag, which will be necessary for NIR, which will be a streaming IR that is "near" to the source XML in structure. This will then be used to lower into AIR for the ASG, which gives structure needed for further analysis. More information to come; I just want to get this committed to serve as a mental synchronization point and clear my head, since I've been sitting on these changes for so long and have to keep stashing them as I tumble down rabbit holes covered in yak hair. DEV-7145
2022-07-13 13:55:32 -04:00
mod ele;
tamer: xir::parse: Attribute parser generator This is the first parser generator for the parsing framework. I've been waiting quite a while to do this because I wanted to be sure that I understood how I intended to write the attribute parsers manually. Now that I'm about to start parsing source XML files, it is necessary to have a parser generator. Typically one thinks of a parser generator as a separate program that generates code for some language, but that is not always the case---that represents a lack of expressiveness in the language itself (e.g. C). Here, I simply use Rust's macro system, which should be a concept familiar to someone coming from a language like Lisp. This also resolves where I stand on parser combinators with respect to this abstraction: they both accomplish the exact same thing (composition of smaller parsers), but this abstraction doesn't do so in the typical functional way. But the end result is the same. The parser generated by this abstraction will be optimized an inlined in the same manner as the hand-written parsers. Since they'll be tightly coupled with an element parser (which too will have a parser generator), I expect that most attribute parsers will simply be inlined; they exist as separate parsers conceptually, for the same reason that you'd use parser combinators. It's worth mentioning that this awkward reliance on dead state for a lookahead token to determine when aggregation is complete rubs me the wrong way, but resolving it would involve reintroducing the XIR AttrEnd that I had previously removed. I'll keep fighting with myself on this, but I want to get a bit further before I determine if it's worth the tradeoff of reintroducing (more complex IR but simplified parsing). DEV-7145
2022-06-13 11:17:21 -04:00
tamer: xir::parse: Fixes for {ele,attr}_parse! outside of module The tests had certain things in scope, but now that I'm trying to use it outside of those modules, some fixes are needed. This is admittedly a sloppy commit, with a number of miscellaneous fixes. I didn't bother separating it more because most of them are type fixes, and the `From<Attr>` stuff is going to have to change into, likely, `TryFrom<Attr>` so that parse failures can occur when attributes do not match certain patterns. DEV-7145
2022-07-20 15:36:28 -04:00
pub use attr::{parse_attrs, AttrParseError, AttrParseState};
tamer: xir::parse::ele: Transition trampoline This properly integrates the trampoline into `ele_parse!`. The implementation leaves some TODOs, most notably broken mixed text handling since we can no longer intercept those tokens before passing to the child. That is temporarily marked as incomplete; see a future commit. The introduced test `ParseState`s were to help me reason about the system intuitively as I struggled to track down some type errors in the monstrosity that is `ele_parse!`. It will fail to compile if those invariants are violated. (In the end, the problems were pretty simple to resolve, and the struggle was the type system doing its job in telling me that I needed to step back and try to reason about the problem again until it was intuitive.) This keeps around the NT states for now, which are quickly used to transition to the next NT state, like a couple of bounces on a trampoline: NT -> Dead -> Parent -> Next NT This could be optimized in the future, if it's worth doing. This also makes no attempt to implement tail calls; that would have to come after fixing mixed content and really isn't worth the added complexity now. I (desperately) need to move on, and still have a bunch of cleanup to do. I had hoped for a smaller commit, but that was too difficult to do with all the types involved. DEV-7145
2022-08-10 00:21:45 -04:00
pub use ele::{EleParseCfg, EleParseState, StateStack, StateStackContext};