liza/doc/program.texi

@c  This document is part of the Liza Data Collection Framework manual.
@c  Copyright (C) 2017 R-T Specialty, LLC.
@c
@c    Permission is granted to copy, distribute and/or modify this document
@c    under the terms of the GNU Free Documentation License, Version 1.3
@c    or any later version published by the Free Software Foundation;
@c    with no Invariant Sections, no Front-Cover Texts, and no Back-Cover
@c    Texts.  A copy of the license is included in the section entitled ``GNU
@c    Free Documentation License''.


@node Program
@chapter Program
@maintstart
@footnote{
  The @code{Program} class was one of the first prototypes created,
    and has evolved poorly with the rest of the system.
  It is the base class for all compiled programs,
    and it glues together too many other systems with a terrible
    API and little to no encapsulation.

  With that said, it is one of the least touched classes (thus its
    state); developers rarely have the need to touch @code{Program}.}
@maintend


@cindex Program
The @dfn{Program} is a declarative representation of an entire system.
It is the highest level of abstraction from a data perspective.
The user observes and interacts with a Program using the
  @ref{Program UI,,Program@tie{}UI}.

@cindex Program, XML
Programs contain a lot of metadata that is not in a convenience
  human-readable (or modifiable) format,
    some of which are redundant.
Programs are ideally compiled from a @ref{Program XML,,Program@tie{}XML}
  document.

@menu
* Program UI::
* Program XML::
* Document Metadata::  Document-level data that cannot be modified by
                       the client.
@end menu



@node Program UI
@section Program UI
@maintenance{
  The @code{Ui} class,
    in addition to @srcref{src/client/Client,Client} (@pxref{Client}),
    represent the two monoliths of the system.
  This mediates all UI-related tasks,
    and still has far too many concerns with far too many
      dependencies.
  Code is to be extracted out of this class as it is touched.
}


@cindex Program, User Interface
@cindex User Interface, Program
The @dfn{Program UI} renders a @ref{Program} as a form.

@cindex Step
@cindex Group
At the highest level,
  steps are rendered in a tab-like manner,
  above the main form content.
A step contains groups,
  which in turn contain elements such as questions.
Groups are delimited in some manner defined by their style
  (@pxref{Group Styles}).

@cindex Question
@cindex Question, Value Formatting
@cindex Bucket, Updating
Questions are rendered as form fields.
Any time the respective @ref{Bucket} field is changed,
  the form field is updated to reflect those changes,
    after having first been formatted with the appropriate validator
    (@pxref{Formatting Values}).
When a question is changed by the user,
  the value is expected to be propagated to the Bucket
  (@pxref{Bucket Assignment}).

@cindex Navigation Bar
@cindex User Interface, Navigation Bar
@cindex User Interface, Button Navigation
Navigation between steps can be done via the
  @dfn{Navigation Bar} above the step@tie{}content,
  or using ``Go@tie{}Back'' and ``Continue'' buttons at the foot of the
  step@tie{}content.

@cindex Sidebar
A @dfn{Sidebar} is rendered adjacent to the step content.
It displays the name of the Program,
  as well as configurable metadata (usually through the @samp{sidebar}
  node of the @ref{Program XML,,Program@tie{}XML}).
It also displays question help text (also configured through the XML)
  and any error messages (@pxref{Error Handling}).

@menu
* Group Styles::     Different ways of displaying groups of questions
                       to the user.
* DOM Abstraction :: Representing and efficiently manipulating the DOM.
@end menu



@node Group Styles
@subsection Group Styles
@refactor{
  Some group styles still use jQuery;
    they should be modified to use modern formatters and Liza DOM
    abstractions (see @srcrefraw{src/ui/field}
      and @srcrefraw{src/ui/styler}).}

@cindex Group, Styling
Groups support a number of @dfn{group styles} that determine how
  they are delimited from other groups (@pxref{Defining Groups});
  how the elements they contain are rendered and laid out;
  and how multiple indexes are displayed, added, and removed.
A list of available styles is detailed in @ref{t:group-styles}.

@float Table, t:group-styles
@multitable @columnfractions 0.15 0.65 0.10 0.10
  @headitem Name @tab Description @tab Multi-Index? @tab Add/Remove Index?

  @item @samp{default}
  @tab
  Groups are unstyled by default@mdash{
    }they render elements as flat fields like a traditional form.
  Only the first index of elements is rendered.
  @tab@center N
  @tab@center N

  @item @samp{collapsetable}
  @tab
  Renders element label in the leftmost column like @samp{sidetable}.
  Indexes are groups of rows delimited by headings,
    which collapse the respective group of rows when clicked.
  @tab@center Y
  @tab@center Add

  @item @samp{sidetable}
  @tab
  Renders elements as rows with label in the leftmost column rather
    than the top row.
  Each index is rendered as a column.
  @tab@center Y
  @tab@center Add

  @item @samp{stacked}
  @tab
  Groups respective indexes of elements such that one set of indexes
    appears atop of another set,
      much like separate groups are placed.
  @tab@center Y
  @tab@center N

  @item @samp{tabbedblock}
  @tab
  Each group is rendered as a block,
    with each index rendered as a tab to the right of it.
  Clicking a tab toggles the body content to the associated index.
  Elements are rendered within the box.
  @tab@center Y
  @tab@center N

  @item @samp{tabbed}
  @tab
  Like @samp{default},
    but each index has a tab at the top of the group.
  Clicking a tab toggles the body content to the associated index.
  @tab@center Y
  @tab@center Y

  @item @samp{table}
  @tab
  A vanilla table with elements as columns,
    their labels across the top row.
  Each index is rendered in its own row.
  @tab@center Y
  @tab@center Y
@end multitable
@caption{Group styles and index support}
@end float



@node DOM Abstraction
@subsection DOM Abstraction
@cindex DOM
@cindex Dojo, History
@cindex jQuery
@devnotice{jQuery is still used throughout parts of the framework and
             is a performance bottleneck@mdash{
               }it needs to be fully removed and replaced with this
                 DOM@tie{}abstraction.@footnote{
                   See @srcrefraw{src/ui/ElementStyler}.}}

@trivia{Liza was conceived long before frameworks like React existed.
          The implementation originally used Dojo because of its
            broad widget set,
              but it was later dropped because of extreme performance issues,
                especially on the browsers of the day
                  (Liza had to support Internet Explorer@tie{}6!);
                  at one point,
                    certain steps took over a minute to load for the
                    most unfortunate of users.
          jQuery was then used for various parts of the UI and for ease of
            DOM manipulation,
              because of the lack of good and universal DOM APIs back then.
          It too became a bottleneck.
          Using DOM@tie{}APIs is now easy with modern browsers.}

Liza's DOM abstraction contains a couple of components:

@cindex DOM, Context
@cindex DOM, Field
@itemize
@item @dfn{DOM Fields} represent a field on the DOM.
      Each field has a name and an index associated with the
        DOM@tie{}node.
      Nodes are cached in memory after first access and queue requests
        during lookups to prevent stampeding.
      Provides basic DOM operations,
        including styling, containing row, and  parent/sibling selecting.
      See @srcref{src/ui/field}.
@item @dfn{DOM Context} is a slice of the DOM used for restricting queries.
      It can attach and detach sections of the DOM,
        and be further split into a context hierarchy.
      The @srcrefjs{src/ui/context,DomContext} provides field querying
        (see @srcrefjs{src/ui/field,DomField}) and caching.
      See @srcrefraw{src/ui/context}.
@end itemize

@tip{It is important to always use these abstractions for any portions
      of the DOM under control of this abstraction;
        otherwise, assumptions used for caching may result in
          unintended behavior.}

Using DOM contexts,
  DOM operations can be restricted to small windows (for example,
    groups or tabs),
      further reducing the impact of DOM queries.

The @dfn{root context} is represented by
  @srcrefjs{src/ui/context,RootDomContext}@mdash{
    }sub-contexts can be obtained by invoking @jsmethod{slice} on any
      context,
        which creates a new context from a subset of the parent.
Detaching a parent will detach all child contexts.

Contexts can be manipulated in memory before being re-attached.
Detach a context from the DOM with @jsmethod{detach},
  and attach with@tie{}@jsmethod{attach}.
A context is aware of its parent and will re-attach itself to the DOM
  in the correct place.
A child context always attaches to the parent,
  and so will not be rendered until the parent attaches.

@tip{Always detach from the DOM before performing extensive manipulations;
       this prevents the need for expensive re-painting until
         manipulation is done,
           at which point the context can be re-attached.}


@menu
* Field Styling:: Styling @srcrefjs{src/ui/field,DomField}.
@end menu



@node Field Styling
@subsubsection Field Styling
@cindex Field, Styling
@helpwanted

@srcrefjs{src/ui/field,DomField} is styled using field stylers
  (see @srcrefraw{src/ui/styler}).
The two most notable stylers are
  @srcrefjs{src/ui/field/ErrorFieldStyler} and
  @srcrefjs{src/ui/field/NaFieldStyler},
    which style fields in error and hide fields that are no applicable
    respectively.


@node Program XML
@section Program XML
@helpwanted

@menu
* Defining Groups::       Grouping questions and other entities.
* Specifying Predicates:: Predicating display of entities.
@end menu


@node Defining Groups
@subsection Defining Groups

@cindex Group
A @dfn{group} organizes and relates entities.
If given a @xmlattr{title},
  a header will be displayed above the group with that title.
A group may optionally be given a unique identifier@tie{}@xmlattr{id},
  which is useful for debugging and scripting;
    any such identifier should be @code{snake_case}.

@float Figure, f:group
@example
  <group title="Foo Group">
    <question id="question_1" ... />
    <question id="question_2" ... />
  </group>
@end example
@caption{Defining a simple group with a title}
@end float

@cindex Group, Styling
Groups can be independently styled in a number of different ways to
  provide different data representations (@pxref{Group Styles}).
Style is optional@mdash{
  }the @samp{default} group displays only the first index of each
    entity.
Further styling can be done using CSS using @xmlattr{class}.

@cindex Group, Leader
@cindex Group, Indexes
All questions within a group share the same number of indexes;
  this is accomplished by monitoring the @dfn{group leader}.
By default,
  the leader is the first indexable entity in the group (question,
    answer, or display);
      this can be overridden with the @xmlattr{indexedBy} attribute.
This attribute is only practically meaningful if the chosen group
  style supports indexes (@pxref{Group Styles}).


@float Figure, f:group-leader
@example
  <group id="leader_override" indexedBy="question_2" style="tabbed">
    <question id="question_1" ... />
    <question id="question_2" ... />
  </group>
@end example
@caption{Overriding group leader using @xmlattr{indexedBy}}
@end float

@cindex Group, Locking
Some group styles allow the user to add indexes;
  set @xmlattr{locked} to@tie{}@samp{true} to suppress this feature.

@subsubsection Linking Groups
@cindex Group, Linking
Data collection for similar entities may span multiple steps or
  groups;
    for example,
      one group may allow the user to define their risk locations,
        and a future group may ask for additional information for each
        of those locations.
To have all entities within each of those groups share index length,
  they may be linked.

A @dfn{group link} is an arbitrary name given to a set of groups.
Each group that wants to be part of the same link must set
  @xmlattr{link} to the same string.
The name of the link does not matter@mdash{
  }it is @emph{not} a reference to a group@tie{}@xmlattr{id}.

@float Figure, f:group-link
@example
  <group id="location" link="locations" style="tabbed">
    <question id="address" ... />
    <question id="city" ... />
  </group>

  <group id="underwriting" link="locations" style="tabbed">
    <question id="diving_board" ... />
    <question id="rabid_dog" ... />
  </group>
@end example
@caption{Linking groups using @xmlattr{link}}
@end float

In @ref{f:group-link},
  each question in @var{location} and @var{underwriting} will have the
  same number of indexes@mdash{
    }any time a new index is added to @var{location},
      @var{underwriting} questions too will gain another index and
      vice versa.
There is no limit to the number of groups that can share the same link.

@devnote{Linked groups are implemented such that the union of all fields
       in each of the groups of a given link are assigned to each of the
       individual groups.
     When the leader of any group changes,
       a new index is initialized for each group field,
         which (in the case of linked groups) is comprised of all
         fields in the link.}



@node Specifying Predicates
@subsection Specifying Predicates

@cindex Predicate
Object predicates (@pxref{Predicate System}) are specified using the
  @xmlattr{when} attribute of certain nodes.
It must contain a string of references understood by the system
  (see domain of discourse, @ref{Predicate System}),
    all of which much match for the predicate to be true.

@float Figure, f:pred-when
@example
  <question id="describe" type="noyes"
            label="Any special notes for this location?" />

  <question id="vacant_desc" type="textarea"
            when="q:describe vacant property"
            label="Show only when a vacant property with the
                   question 'describe' non-empty and non-zero" />
@end example
@caption{Using the @xmlattr{when} attribute}
@end float

In @ref{f:pred-when} above,
  question @samp{vacant_desc} will be applicable when @emph{all} of
  the values of @samp{vacant}, @samp{property},
  and@tie{}@samp{q:describe} are true.@footnote{
    @xref{Predicate System} for what ``true'' means for a particular
      variable in the domain of discourse.}
Within the context of the @progxml,
  this concretely means that the classifications
  @samp{vacant} and@tie{}@samp{property} are true,
    and that the question @samp{describe} is answered ``yes''.
It reads as a sentence:
  ``@samp{vacant_desc}'' is applicable when we should @tie{}``describe
  a vacant property''.



@node Document Metadata
@section Document Metadata
@dfn{Document metadata} are metadata that describe certain aspects of the document;
  they are stored adjacent to the bucket in @samp{meta}@tie{}on the
  document root.@footnote{
    Terminology note: ``document'' and ``quote'' are the same thing;
      the latter is transitioning to the former for generality.}
They should be used in place of a bucket field any time
  the client has no business knowing about the data.
The @samp{meta} record is called the @dfn{Metabucket}.

@c don't use a dapi xref here; don't want to confuse the reader by
@c directing them away from this section before they continue reading
@tip{Metadata in the Metabucket should@tie{}@emph{not} be
  directly populated by external systems@mdash{
    }@dapi integration should be used instead (see below).}

Metadata can be populated using any@tie{}@dapiref@mdash{
  }return data populate the Metabucket in the same way that they
  populate the Bucket.
Definitions are stored in @code{meta.fields},
  as shown in @ref{f:meta-fields}.

@float Figure, f:meta-fields
@example
"fields":@{
  ["string(name)": @{
    "desc": "string",
    "dapi": @{
      "name": "string",
      "map": @{
        "string(dest field)": "string(source field)"
      @}
    @}
  @}
@}
@end example
@caption{Format of @code{meta.fields}.}
@end float

Further, a key-value mapping of all bucket fields that@mdash{
  }when modified,
    need to result in a metadata API@tie{}call@mdash{
  }are stored in the @code{mapis}@tie{}object;
    this is shown in @ref{f:mapis}.

@float Figure, f:mapis
@example
"mapis":@{
  ["string(field name)"]: [ "string(dapi name)", ... ]
@}
@end example
@caption{Format of @code{mapis}.}
@end float