* src/server/db/MongoServerDao.js
(getDocumentField,setDocumentField): New methods.
* src/server/service/RatingServiceSubmitNotify.js
(postProcessRaterData): Only notify when notification flag is not set.
(_getNotifyState, _setNotified): New methods.
* test/server/service/RatingServiceSubmitNotifyTest.js: Modify accordingly.
This wasn't intended to make its way into a public repo. :)
The existing key was a long-forgotten kluge that was supposed to be
temporary, allowing internal services to create quotes without
authentication. The chances of this being practically exploited are minimal
in our environment, and it's auditable using webserver logs.
This moves the skey into a configuration file, which allows it to vary by
server and be rotated until a better solution is made available. skey is
disabled by default (empty string), and when used by us internally, the keys
are now generated using a CSPRNG rather than a brute-forcable 5-byte key
that was hard-coded.
The fact that this appears in webserver logs is a big issue as well. I
added a task to address that.
* conf/vanilla-server.json (skey): New key. Default empty.
* src/server/daemon/Daemon.js (start): Provide skey to `#getRouters'.
(getRouters): Provide skey to `#getProgramController'.
(getProgramController): Set skey on `controller'.
* src/server/daemon/controller.js (skey): New mutable export (unideal; quick
change).
(has_skey): Use it.
quicksave data wouldn't always be cleared on save, meaning it might not be
cleared until the next _empty_ quicksave from the client.
* src/server/db/MongoServerDao.js (saveQuote): Always clear quicksave.
Was outputting NaN, because the property that was being used no longer
existed after previous changes.
* src/server/daemon/Daemon.js (_initHttpServer): Use HTTP port number
in output.
Previous to this change, we'd write the clear to the database, but
then immediately after, the quote save would restore it. Oops.
* src/server/Server.js (_monitorMetadataPromise): Clear data in
metabucket rather than directly invoking a DB call. This will be
applied when the quote is saved.
This mixes in support for non-terminating nulls. It would have been
nice to handle that in a separate commit for clarity, but the
refactoring came as a consequence of trying to provide a working
implementation.
Various inconsistencies and subtle bugs in unlikely situations have
been fixed by this, including modifying objects passed as arguments to
various methods, and inconsistent handling of diff data.
Changes are more consistently recognized. Perhaps the most noticeable
consequence is that moving between steps no longer prompts to discard
changes---previously calculated values would trigger the dirty flag on
steps even if the user didn't actually change anything. I (and
others) have wanted this fixed for many years.
This is a very dense commit that touches a core part of the
system. Hopefully the Changelog below helps.
* src/bucket/Bucket.js
(setValues): [BC-BREAK] Remove parameters `merge_index' and
`merge_null' parameters.
* src/bucket/DelayedStagingBucket.js
(setValues): [BC-BREAK] Remove `merge_index' and `merge_null
parameters. Remove distinction between `merge_index' and non-.
* src/bucket/QuoteDataBucket.js
(setValues): [BC-BREAK] Remove `merge_index' and `merge_null
parameters. Remove respective arguments from `_mergeData' call.
(_mergeData): Remove same parameters. Remove handling of
`merge_index' and `merge_null'.
(overwriteValues): Append `null' to each vector.
* src/bucket/StagingBucket.js
(_initState): Use `Object.create' instead of explicit prototype
instantiation (functionally equivalent).
(merge): Minor comment correction.
(_hasChanged): Rename to `_parseChanges'.
(_parseChanges): Rename from `_hasChanged'. Remove `merge_index'
parameter. Generate new object rather than mutation original
data (prevent dangerous and subtle bugs from side-effects). Clone
each vector rather than modifying/referencing directly (this was
previously done during merge). Remove `merge_index'
distinction. Handle non-terminating `null' values.
(setValues): [BC-BREAK] Remove `merge_index' and `merge_null'
parameters. Use new object generated by `_parseChanges'. Remove
cloning of each vector (`_parseChanges' now does that). Remove
`merge_index' distinction.
(overwriteValues): Remove argument to `setValues' call.
(getFilledDiff): [BC-BREAK] Use `_staged' rather than `_curdata'.
(commit): Remove second and third arguments of call to `setValues'
of underlying bucket.
* src/client/Client.js
(_initStepUi): Remove second argument of calls to quote `setData'.
* src/client/quote/ClientQuote.js
(setData): [BC-BREAK] Remove `merge_nulls' parameter. Remove second
and third arguments of call to staging bucket `setValues'. Add
comment indicating a long-standing problem with committing the
staging bucket contents before save has succeeded.
* src/server/request/DataProcessor.js
(processDiff): Remove `permit_null' argument of `sanitizeDiff'
call.
(sanitizeDiff): Remove `permit_null' parameter. Hard-code filter
call's `permit_null' argument to `true'.
(_determineDapiFields): Properly handle `null's (ignore) rather than
inadvertently converting them into the string "null".
* test/bucket/StagingBucketTest.js: Modify test cases
accordingly. Add tests to verify that updates and diffs operate
as expected, especially support for non-terminating `null's.
(createStubBucket): Use `QuoteDataBucket'. Ideally remove this
coupling in the future, but this is a more realistic test case for
the time being.
* test/server/request/DataProcessorTest.js: Update test to account for
hard-coded `given_null' argument.
I'm not sure if this has ever caused real problems, but this was
dangerous.
* src/server/service/RatingService.js (_performRating): Store cleaned
rating data in separate variable to ensure cleaned data are not
saved in place of the actual complete response.
This represents a portion of the refactoring that I had intended to do
until I realized that there was a simpler solution to the problem that
we were having (having proguic add stored calculated values to the
defaults object).
So ideally we'll continue extracting all quote init code out of
`Server' and into `ProgramInit' in the future.
* doc/server.texi (Liza Server): Mention `ProgramInit'.
* src/program/ProgramInit.js: Add class.
* src/server/DocumentServer.js: Use it.
* src/server/Server.js (_progInit): Add private field.
(__construct): Accept ProgramInit instance and assign to field.
(initQuote): Use promise returned by `_getDefaultBucket'.
(_getDefaultBucket): Proxy to `ProgramInit#init', which returns a
promise.
This was a bit involved because the system had to be made async all
the way up the stack. No attempt was made to clean up the mess up the
stack---no time.
* src/dapi/DataApiFactory.js
(fromType): [BC BREAK] Fix docblock. Add `api_name' param. Call
`#descLookup' and return promise.
(descLookup): Add method. Return promise resolving to provided
descriptor. Intended to be overridden by subtype.
* src/dapi/DataApiManager.js
(_dataApis): Update docblock to indicate that it now stores
promises.
(getApiData): Expect promise for `DataApiFactory#fromType' call.
* src/server/DocumentServer.js: (create): [BC BREAK] Accept
configuration. Look up dapi conf and pass to
`ServerDataApiFactory' ctor. Return promise.
* src/server/daemon/Daemon.js (_initRouters): Provide configuration.
* src/server/daemon/controller.js
(init): Accept configuration. Handle return of promise from
`_createDocumentServer'.
(_createDocumentServer): Accept configuration, providing to
`DocumentServer#create'. Because of aforementioned change to
`#create', returns promise.
* src/server/request/ServerDataApiFactory.js: Add StoreMissError
import.
(_conf): Add property.
(constructor): [BC BREAK] Accept configuration.
(descLookup): Add override. Look up configuration for provided
dapi.
This is based (very) loosely on an internal script to start the
daemon. It accepts a configuration and starts the daemon.
To accommodate the configuration, a number of miscellaneous changes
have been made.
The vanilla configuration shows the concept, but it has not yet been
fully implemented; that'll likely happen at a later date. Until then,
the existing environment-variable-based configuration will be used.
* bin/server.js: Add file.
* conf/vanilla-server.json: Example configuration added.
* src/server/daemon/Daemon.js (_httpPort): Remove field.
(_conf): Add field.
(__construct): [BC BREAK] Accept conf instead of port and log
priority. Move initialization code into `start'.
(start): [BC BREAK] Initialization code moved here. Now returns
promise for entire daemon, which will error in the event of an
error starting. Move existing code into `_startDaemon'.
(_startDaemon): Old `start' code. Invoked after `start'
initialization.
(_createDebugLog, _createAccessLog): Use configuration. Return
promise.
(_initHttpServer): Use configuration.
(_httpError): Add function to output error and exit. Extracted from
`_initHttpServer'.
* src/server/daemon/scripts.js: [BC BREAK] Append "program/" to
`LV_LEGACY_PATH' so that it can be re-used for script lookups
rather than using the cwd. This removes the need of the cwd being
the legacy src path.
I wish I knew this option existed ~7yr ago. This should have been
done long ago, but we've had so few problems with it, that it had
never been explored.
This [version of this] mongo library is ancient and needs to be
upgraded. Until then, this'll do.
* src/server/daemon/controller.js (init): Set `auto_reconnect'.
`undefined' was inserted into the session if data could not be
retrieved.
* src/server/request/UserSession.js (_appendSessionData): Do not write
session data if original data cannot be retrieved.
This is a terrible kluge, but time doesn't permit modifying the
system. All of this also touches old code that is untested, which is
difficult to modify with confidence.
* src/server/DocumentServer.js (DocumentServer#create): Use
StagingBucket.
* src/server/Server.js: Remove logic now handled by DataProcessor.
* src/server/request/DataProcessor.js (processDiff): Wrap in
StagingBucket to filter out values that do not result in changes.
* test/server/request/DataProcessorTest.js: Update failing cases.
Prefixes with "meta:".
* src/server/Server.js (initQuote): Initialze metadata to empty.
* src/server/daemon/controller.js (createQuote): Add metabucket.
* src/server/db/MongoServerDao.js (saveQuote): Add additional
documentation to docblock. Save all metadata if no bucket data
provided (which indicates a full quote save). This happens on
version change.
* src/server/rater/ProcessManager.js (_genData): Add method to merge
metadata into bucket data with "meta:" prefix.
(byId): Use merged data.
* src/server/daemon/Daemon.js (__construct): Reference new class.
* src/server/rater/ProcessManager.js: Add class (extracted from
service).
* src/server/rater/service.js: Refactored into ProcessManager.
It never was a thread; it's always been a process.
* src/server/rater/thread.js: Renamed.
* src/server/rater/process.js: Renamed from thread.js.
* src/server/rater/service.js: s/thread/process/.
This reverts CapturedUserResponse to the state it was in prior to
being extracted into liza. The issue was supposed to have been
resolved by now, but I haven't wanted to make an ease.js release until
the company that bought the company I work for signs the copyright
waiver. Which has been slow-going, to say the least.
The problem is that it is not recognizing a named trait extending a
class as a parameterized trait (it's not seeing __mixin).
* src/server/request/CapturedUserResponse.js: Use interface.
* src/server/request/IProtUserResponse.js: Add temporary interface.
What a cluster.
This was a lot of work to work around existing, bad APIs; there is no
time to refactor at the moment; this already took much longer than
expected.
Copyright notices updated. More casual references to "LoVullo
Associates" replaced with "RT Specialty / Lovullo", which will be "RT
Specialty Buffalo" in the future. Or "RT Specialty", depending on how
this is rolled out. Or "Ryan Specialty Group". Who knows.
"R-T Specialty, LLC." is the legal name, which includes the dash. Not
to be confused with a certain television network.
I have sat on releasing a lot of this code for years because I wanted
the liza repo to be in a pristine state---tests and all---which
required a great deal of refactoring. Well, that never happened, and
time is up.
LoVullo Associates---my employer---has been purchased by another
company. This means that any agreement with LoVullo regarding
releasing free software is going to have to be re-negotiated with this
new company, and I have no idea how those negotiations will go. So,
I have no choice but to simply release everything in its current state,
or risk it being lost forever.
This represents work over the past 6--7 years, 99.9% of it written by
me. This project has been my baby for quite some time, and has been
through a number of battles with deadlines and other unfortunate
circumstances; the scars show. I also didn't really "know" JS when
starting this project. Perhaps you can help improve upon it.
There are some odds-and-ends that could be committed. And references
to insurance and LoVullo need to be removed to generalize this.
I hope that this will not be the last public commit for this project.
I'll fight the good fight and we'll see where that takes us. Maybe
it'll be easy.
Happy hacking.
Mike Gerwitz
Mike Gerwitz