* src/server/daemon/Daemon.js (getProgramController): Do not initialize
`no_results_url' to an empty string if not provided; instead, keep any
existing value. Otherwise, later calls with no arguments will clear
previously-set values.
DEV-3254
This further highlights that we probably should not be handling this
here. We want to ensure that we check the state of the bucket right before
we're about to do option setting, otherwise we could be in an inconsistent
state (looking at old data).
* src/client/dapi/DataApiMediator.js (_updateFieldData): Check bucket after
stack clears (along with all other operations).
Allowing the stack to clear ensures that (in practice) DelayedStagingBucket
is given a chance to do necessary processing before data are requested from
it by bucket hooks as a result of _this_ invocation, which in turn
results (in some cases) in infinite recursion.
* src/client/dapi/DataApiMediator.js (_updateFieldData): Allow stack to
clear before invoking `quote.setData'.
* test/client/dapi/DataApiMediatorTest.js: Test respectively.
DEV-3257
Without this, we have a BC break.
* src/client/Client.js (_init): Default program.dapimap to DataApiMediator
instance to an empty object to maintain BC.
* src/client/dapi/DataApiMediator.js (_populateWithMap): Update
docblock. Ignore field during expansion if it would overwrite an
existing value.
* test/client/dapi/DataApiMediatorTest.js: Update tests data to include
values for all bucket fields, not just `name'. Add test for new
condition.
DEV-3257
[*] You should not use this commit directly since this may wipe out data in
fields the user has changed. See future commit where this situation is
properly handled.
* src/client/Client (_init): Provide dapimap to DataApiMediator instance.
* src/client/dapi/DataApiMediator.js
(_dapi_map): New field.
(constructor): Accept dapimap. BC break (which is okay, since this is
still part of a topic branch). Assing to _dapi_map. Update docblock.
(monitor): Bind `dapi_manager' to first argument of handlers.
(_updateFieldData): Accept `dapi_manager' as first argument. Use
`_populateWithMap' to generate additional update data.
(_populateWithMap): New method.
(_clearFieldFailures): Accept `dapi_manager' as first argument.
* src/dapi/DataApiManager.js: Update copyright year.
(getDataExpansion): Return empty object (consistent with interface) rather
than `undefined' when field value is undefined. Use
{Error=>MissingDataError} when field data are missing. Throw instead of
emit. Fix missing comma in var declarations.
* src/dapi/MissingDataError.js: New class.
* test/client/dapi/DataApiMediatorTest.js: Update test data to test field
expansion. New test against ignoring field expansion when data are not
available. Update Sut constructors of other tests for new dapimap
parameter.
DEV-3257
This extracts existing code from Client and adds tests. The glue code is
far from ideal and highlights the amount of work needed to decouple Client
from so many parts of the system.
* src/client/Client.js (_dapiManager): New field.
(_init): Use DataApiMediator.
(_createProgram): Assign `_dapiManager' (this is not at all
ideal). Remove hooks from it: fieldLoading, updateFieldData,
clearFieldData.
* src/client/ClientDependencyFactory.js (createDataApiMediator): New alias
to DataApiMediator constructor.
* src/client/dapi/DataApiMediator.js: New class.
* test/client/dapi/DataApiMediatorTest.js: New test case.
DEV-3257
This does not go all the way, but helps improve the readability of the
algorithm a little bit and modernizes the code.
* src/field/FieldClassMatcher.js (constructor): Renamed from
`__constructor'.
(__constructor): Remove method.
(match): Extract most code into `#_reduceFieldMatches'.
(_reduceFieldMatches): New method, simplifying the algorithm slightly.
(_reduceMatch): Simplify.
* test/field/FieldClassMatcherTest.js: Update accordingly.
If the submission failed, we probably want to try again next time around.
* src/server/service/RatingServiceSubmitNotify.js
(_maybeNotify): Extract logic from `#postProcessRaterData'. Only set
notification flag in absence of dapi error.
(postProcessRaterData): Use it.
* test/server/service/RatingServiceSubmitNotifyTest.js: Update tests
accordingly.
Previously, it was just trying to serialize an object, which was useless.
* src/server/Server.js (_formatValidationFailures): New method.
(_doQuoteSave): Use it.
Requests that came in between (for example) a post request and the actual
validation of the posted data would potentially reset the flag, causing
internal assertions to fail.
* src/server/Server.js
(sendStep): Provide session to #_forwardValidate.
(_forwardValidate): Set `program.isInternal' immediately before invoking
event. Provide session to #quoteFill.
[session]: New argument.
(quoteFill): Add `session' argument. Provide session to hooks.
(loadProgram): Receive session via #quoteFill hook. Set
`program.isInternal' before submit hook.
* src/server/daemon/controller.js (doRoute): Remove `program.isInternal'
set.
This is not the ideal implementation. Instantiation of RatingService and
such should be moved out of the controller entirely, but we don't have the
time for that right now.
* src/server/db/MongoServerDao.js
(getDocumentField,setDocumentField): New methods.
* src/server/service/RatingServiceSubmitNotify.js
(postProcessRaterData): Only notify when notification flag is not set.
(_getNotifyState, _setNotified): New methods.
* test/server/service/RatingServiceSubmitNotifyTest.js: Modify accordingly.
This wasn't intended to make its way into a public repo. :)
The existing key was a long-forgotten kluge that was supposed to be
temporary, allowing internal services to create quotes without
authentication. The chances of this being practically exploited are minimal
in our environment, and it's auditable using webserver logs.
This moves the skey into a configuration file, which allows it to vary by
server and be rotated until a better solution is made available. skey is
disabled by default (empty string), and when used by us internally, the keys
are now generated using a CSPRNG rather than a brute-forcable 5-byte key
that was hard-coded.
The fact that this appears in webserver logs is a big issue as well. I
added a task to address that.
* conf/vanilla-server.json (skey): New key. Default empty.
* src/server/daemon/Daemon.js (start): Provide skey to `#getRouters'.
(getRouters): Provide skey to `#getProgramController'.
(getProgramController): Set skey on `controller'.
* src/server/daemon/controller.js (skey): New mutable export (unideal; quick
change).
(has_skey): Use it.
quicksave data wouldn't always be cleared on save, meaning it might not be
cleared until the next _empty_ quicksave from the client.
* src/server/db/MongoServerDao.js (saveQuote): Always clear quicksave.
Before this change, since `undefined' is encoded as `null' when serialized,
there was no way for the server to disambiguate between unmodified values
and a truncation point. For example:
[ undefined, undefined, null, null, null ]
The above array represents two unmodified and three removed indexes. But
this is serialzed into JSON as:
[ null, null, null, null, null ]
It isn't possible for the server to determine what the truncation point is
from that diff. The solution is to therefore truncate the array _before_
sending it to the server, providing a trailing null to indicate that a
truncation has occurred:
[ null, null, null ]
The above means that the first two indexes are unmodified, and that index 2
and later should all be truncated.
* doc/client.texi (Saving to Server): New section.
* src/client/transport/XhttpQuoteTransport.js (_truncateDiff): New method to
perform truncation.
(getBucketDataJson): Use it.
* test/client/transport/XhttpQuoteTransportTest.js: New file with respective
test case.
* src/client/Client.js (_defaultId): New property.
(__construct)[default_id]: New parameter. Set `_defaultId'.
(_getProgramId): Use `_defaultId' if not found in URL.
* src/ui/nav/HashNav.js (_initUrl): Do not require leading program id.
This technically worked fine before with our URL scheme, but that is not
guaranteed to be the case generally.
This is something that managed to slip by (but not unnoticed) for almost
exactly one year to this day (028606242a). It
can only be reproduced by changing classes that result in visibility changes
differing on the same field by index. The issue hides itself on first
load (because all fields are shown by default) and on refresh.
The problem is that, when one index shows a field but another hides it, the
hide overrode the show indexes, so only the hide took place.
* src/client/Cmatch.js (markShowHide): Make virtual. New implementation to
support concurrent show/hide.
(_handleClassMatch): Use it.
* test/client/CmatchTest.js: New test.
* npm-shrinkwrap.json: ease.js v0.2.{8=>9}.
This is to give us a fairly easy means of testing this logic for a bugfix.
This refactoring also obviates a pretty nasty bug; see docblock.
* src/client/Cmatch.js (_handleClassMatch): Extract show/hide marking.
(markShowHide): New method.
This makes the minimal number of changes necessary to ensure that all object
references remain available. It is a mess.
And despite moving all of this, Client is still a massive clusterfuck.
* src/client/Client.js (system): Add import.
(_cmatch): Now references new Cmatch class instance.
(_cmatchHidden, _classMatcher): Remove fields.
(_forceCmatchAction): Rename to `forceCmatchAction'.
(_hookClassifier, _postProcessCmatch, _cmatchVisFromUi, _handleClassMatch,
_mergeCmatchHidden, _clearCmatchFields): Extract methods. Update
references as necessary.
(getCmatchData): Remove unused method.
(_handleError): Rename to `handleError' to make accessible to
Cmatch. Update references.
* src/client/ClientDependencyFactory.js (FieldClassMatcher): Remove import.
(createFieldClassMatcher): Remove method. See `system/client'.
* src/client/Cmatch.js: New class.
* src/system/client.js (Cmatch, field): Add imports.
(cmatch): Add export.
All of the old events have been removed!
* src/client/Client.js (handleEvent): Remove remainder of old system (after
last commit, all that was left was error handling). Correct docblock.
The `set' event already existed---this merely extracts it into its own
handler.
* src/client/Client.js (handleEvent): Extract `set' handler.
* src/client/ClientDependencyFactory.js (createClientEventHandler): Add
`set'.
* src/client/event/ValueSetEventHandler.js: New class.
* test/event/ValueSetEventHandlerTest.js: Associated test case.
This matches the behavior of assertions.
This needs to be refactored into an even handler.
* src/client/Client.js (handleEvent)[set]: Use last available index.
* src/ui/ElementStyler.js (setValueByName)[change_event]: Re-add
`change_event' check, but as a warning instead of an Error. This
shouldn't be used anymore, but let's be certain _without_ breaking
things.
The intent wasn't go have this be permanant, and apparently it _is_ being
used in generated code!
* src/ui/ElementStyler.js (setValueByName): Remove error on `change_event'.
Continued eradication of jQuery for performance reasons.
* src/ui/ElementStyler.js (setValueByName): Do not use jQuery for
legacyradio styling. Throw exception on `change_event' argument set (this
should no longer be used).
This used to be a thing back when we used Dojo, but it doesn't appear to be
used anymore.
* src/ui/ElementStyler.js (setValueByName): Remove `autochange' check.
Lines starting
These animations are ugly and slow when there are many questions on a
step. I've wanted them gone for quite some time, and now we don't have a
disagreement among developers.
* src/client/ClientDependencyFactory.js (createNaFieldStyler): Remove
NaFieldStylerAnimation mixin.
* src/ui/styler/NaFieldStylerAnimation.js: Remove trait.
This is necessary to truncate on index removal; otherwise, it's not possible
to remove indexes through the editor.
* src/client/debug/BucketClientDebugTab.js (_getStagingButtons): Append null
on value change.
An error was being thrown because there are no siblings to reference when
there is only a single group. Obviously sorting is not even needed in such
a situation.
src/ui/step/GeneralStepUi.js (_sortGroups): Abort if <= 1 groups.
Was outputting NaN, because the property that was being used no longer
existed after previous changes.
* src/server/daemon/Daemon.js (_initHttpServer): Use HTTP port number
in output.
Previous to this change, we'd write the clear to the database, but
then immediately after, the quote save would restore it. Oops.
* src/server/Server.js (_monitorMetadataPromise): Clear data in
metabucket rather than directly invoking a DB call. This will be
applied when the quote is saved.
This mixes in support for non-terminating nulls. It would have been
nice to handle that in a separate commit for clarity, but the
refactoring came as a consequence of trying to provide a working
implementation.
Various inconsistencies and subtle bugs in unlikely situations have
been fixed by this, including modifying objects passed as arguments to
various methods, and inconsistent handling of diff data.
Changes are more consistently recognized. Perhaps the most noticeable
consequence is that moving between steps no longer prompts to discard
changes---previously calculated values would trigger the dirty flag on
steps even if the user didn't actually change anything. I (and
others) have wanted this fixed for many years.
This is a very dense commit that touches a core part of the
system. Hopefully the Changelog below helps.
* src/bucket/Bucket.js
(setValues): [BC-BREAK] Remove parameters `merge_index' and
`merge_null' parameters.
* src/bucket/DelayedStagingBucket.js
(setValues): [BC-BREAK] Remove `merge_index' and `merge_null
parameters. Remove distinction between `merge_index' and non-.
* src/bucket/QuoteDataBucket.js
(setValues): [BC-BREAK] Remove `merge_index' and `merge_null
parameters. Remove respective arguments from `_mergeData' call.
(_mergeData): Remove same parameters. Remove handling of
`merge_index' and `merge_null'.
(overwriteValues): Append `null' to each vector.
* src/bucket/StagingBucket.js
(_initState): Use `Object.create' instead of explicit prototype
instantiation (functionally equivalent).
(merge): Minor comment correction.
(_hasChanged): Rename to `_parseChanges'.
(_parseChanges): Rename from `_hasChanged'. Remove `merge_index'
parameter. Generate new object rather than mutation original
data (prevent dangerous and subtle bugs from side-effects). Clone
each vector rather than modifying/referencing directly (this was
previously done during merge). Remove `merge_index'
distinction. Handle non-terminating `null' values.
(setValues): [BC-BREAK] Remove `merge_index' and `merge_null'
parameters. Use new object generated by `_parseChanges'. Remove
cloning of each vector (`_parseChanges' now does that). Remove
`merge_index' distinction.
(overwriteValues): Remove argument to `setValues' call.
(getFilledDiff): [BC-BREAK] Use `_staged' rather than `_curdata'.
(commit): Remove second and third arguments of call to `setValues'
of underlying bucket.
* src/client/Client.js
(_initStepUi): Remove second argument of calls to quote `setData'.
* src/client/quote/ClientQuote.js
(setData): [BC-BREAK] Remove `merge_nulls' parameter. Remove second
and third arguments of call to staging bucket `setValues'. Add
comment indicating a long-standing problem with committing the
staging bucket contents before save has succeeded.
* src/server/request/DataProcessor.js
(processDiff): Remove `permit_null' argument of `sanitizeDiff'
call.
(sanitizeDiff): Remove `permit_null' parameter. Hard-code filter
call's `permit_null' argument to `true'.
(_determineDapiFields): Properly handle `null's (ignore) rather than
inadvertently converting them into the string "null".
* test/bucket/StagingBucketTest.js: Modify test cases
accordingly. Add tests to verify that updates and diffs operate
as expected, especially support for non-terminating `null's.
(createStubBucket): Use `QuoteDataBucket'. Ideally remove this
coupling in the future, but this is a more realistic test case for
the time being.
* test/server/request/DataProcessorTest.js: Update test to account for
hard-coded `given_null' argument.
I'm not sure if this has ever caused real problems, but this was
dangerous.
* src/server/service/RatingService.js (_performRating): Store cleaned
rating data in separate variable to ensure cleaned data are not
saved in place of the actual complete response.
This represents a portion of the refactoring that I had intended to do
until I realized that there was a simpler solution to the problem that
we were having (having proguic add stored calculated values to the
defaults object).
So ideally we'll continue extracting all quote init code out of
`Server' and into `ProgramInit' in the future.
* doc/server.texi (Liza Server): Mention `ProgramInit'.
* src/program/ProgramInit.js: Add class.
* src/server/DocumentServer.js: Use it.
* src/server/Server.js (_progInit): Add private field.
(__construct): Accept ProgramInit instance and assign to field.
(initQuote): Use promise returned by `_getDefaultBucket'.
(_getDefaultBucket): Proxy to `ProgramInit#init', which returns a
promise.
This was a bit involved because the system had to be made async all
the way up the stack. No attempt was made to clean up the mess up the
stack---no time.
* src/dapi/DataApiFactory.js
(fromType): [BC BREAK] Fix docblock. Add `api_name' param. Call
`#descLookup' and return promise.
(descLookup): Add method. Return promise resolving to provided
descriptor. Intended to be overridden by subtype.
* src/dapi/DataApiManager.js
(_dataApis): Update docblock to indicate that it now stores
promises.
(getApiData): Expect promise for `DataApiFactory#fromType' call.
* src/server/DocumentServer.js: (create): [BC BREAK] Accept
configuration. Look up dapi conf and pass to
`ServerDataApiFactory' ctor. Return promise.
* src/server/daemon/Daemon.js (_initRouters): Provide configuration.
* src/server/daemon/controller.js
(init): Accept configuration. Handle return of promise from
`_createDocumentServer'.
(_createDocumentServer): Accept configuration, providing to
`DocumentServer#create'. Because of aforementioned change to
`#create', returns promise.
* src/server/request/ServerDataApiFactory.js: Add StoreMissError
import.
(_conf): Add property.
(constructor): [BC BREAK] Accept configuration.
(descLookup): Add override. Look up configuration for provided
dapi.
This is based (very) loosely on an internal script to start the
daemon. It accepts a configuration and starts the daemon.
To accommodate the configuration, a number of miscellaneous changes
have been made.
The vanilla configuration shows the concept, but it has not yet been
fully implemented; that'll likely happen at a later date. Until then,
the existing environment-variable-based configuration will be used.
* bin/server.js: Add file.
* conf/vanilla-server.json: Example configuration added.
* src/server/daemon/Daemon.js (_httpPort): Remove field.
(_conf): Add field.
(__construct): [BC BREAK] Accept conf instead of port and log
priority. Move initialization code into `start'.
(start): [BC BREAK] Initialization code moved here. Now returns
promise for entire daemon, which will error in the event of an
error starting. Move existing code into `_startDaemon'.
(_startDaemon): Old `start' code. Invoked after `start'
initialization.
(_createDebugLog, _createAccessLog): Use configuration. Return
promise.
(_initHttpServer): Use configuration.
(_httpError): Add function to output error and exit. Extracted from
`_initHttpServer'.
* src/server/daemon/scripts.js: [BC BREAK] Append "program/" to
`LV_LEGACY_PATH' so that it can be re-used for script lookups
rather than using the cwd. This removes the need of the cwd being
the legacy src path.
This will make life much easier and less verbose, especially
considering the verbosity of promises.
* src/store/DelimitedKey.js: Add trait.
* test/store/DelimitedKeyTest.js: Add test case.
I wish I knew this option existed ~7yr ago. This should have been
done long ago, but we've had so few problems with it, that it had
never been explored.
This [version of this] mongo library is ancient and needs to be
upgraded. Until then, this'll do.
* src/server/daemon/controller.js (init): Set `auto_reconnect'.
`undefined' was inserted into the session if data could not be
retrieved.
* src/server/request/UserSession.js (_appendSessionData): Do not write
session data if original data cannot be retrieved.
This is a terrible kluge, but time doesn't permit modifying the
system. All of this also touches old code that is untested, which is
difficult to modify with confidence.
* src/server/DocumentServer.js (DocumentServer#create): Use
StagingBucket.
* src/server/Server.js: Remove logic now handled by DataProcessor.
* src/server/request/DataProcessor.js (processDiff): Wrap in
StagingBucket to filter out values that do not result in changes.
* test/server/request/DataProcessorTest.js: Update failing cases.
This is a kluge until time can be spent better factoring this
system (using Traits).
* src/bucket/StagingBucket.js (_noStagingBypass): Add field.
(forbidBypass): Add method to set field.
(setCommittedValues): Use field.
* src/dapi/DataApiFactory.js (_createDataApi): Add support for
`quote'.
* src/dapi/http/HttpDataApi.js (__construct): New `enctype' argument.
(_encodeData, _encodeKeys): Remove former, rename latter to former.
(encodeData, _urlEncode): Encode based on enctype and method.
(*): Strict mode, es6 style.
This just updates the file a bit and provides a more specific
docblock (as used in HttpDataApi).
* src/dapi/DataApi.js: strict mode and var=>const.
(request): Docblock update for params.
I'm still waiting to release GNU ease.js 0.2.9 because I still (after
months) am trying to secure a copyright waiver from my employer. The
bugs affecting this were fixed back in December!
This works around the issue of traits extending named classes, and
some internal issues with extending constructors defined by the system
for traits.
https://mikegerwitz.com/projects/easejs/commit/?id=e93f3a70f9a699f1577df6ce3ba9201b2ea97542
So, this commit is temporary; revert after 0.2.9.
* src/ui/dialog/ITableDialog.js: Add interface (temporarily).
* src/ui/dialog/ColumnLink.js: Use it.
Prefixes with "meta:".
* src/server/Server.js (initQuote): Initialze metadata to empty.
* src/server/daemon/controller.js (createQuote): Add metabucket.
* src/server/db/MongoServerDao.js (saveQuote): Add additional
documentation to docblock. Save all metadata if no bucket data
provided (which indicates a full quote save). This happens on
version change.
* src/server/rater/ProcessManager.js (_genData): Add method to merge
metadata into bucket data with "meta:" prefix.
(byId): Use merged data.
* src/server/daemon/Daemon.js (__construct): Reference new class.
* src/server/rater/ProcessManager.js: Add class (extracted from
service).
* src/server/rater/service.js: Refactored into ProcessManager.
It never was a thread; it's always been a process.
* src/server/rater/thread.js: Renamed.
* src/server/rater/process.js: Renamed from thread.js.
* src/server/rater/service.js: s/thread/process/.
A bit arbitrary, but it contains text that's coupled to the layout of
the UI, and it usually pushes the button down off of the bar.
* src/client/Client.js (_showLockedNotification): Remove end of
"unlock quote" string.
The class was renamed, but not the file. Whoops.
* src/ui/worksheet/CalcWorksheet.js: Rename from RatingWorksheet.js.
* src/ui/worksheet/RatingWorksheet.js: Remove file.
This reverts CapturedUserResponse to the state it was in prior to
being extracted into liza. The issue was supposed to have been
resolved by now, but I haven't wanted to make an ease.js release until
the company that bought the company I work for signs the copyright
waiver. Which has been slow-going, to say the least.
The problem is that it is not recognizing a named trait extending a
class as a parameterized trait (it's not seeing __mixin).
* src/server/request/CapturedUserResponse.js: Use interface.
* src/server/request/IProtUserResponse.js: Add temporary interface.
What a cluster.
This was a lot of work to work around existing, bad APIs; there is no
time to refactor at the moment; this already took much longer than
expected.
Mike Gerwitz
Mike Gerwitz