Move create config functions out of the main delta-processor.ts file. Change some any types to more specific types. Define document structure better. Move prometheus logic out of processor class. Do not call logger directly, listen for events and log them. Change logger to more PSR-3 compliant style. Extract amqp connection into its own class. Create avro and mongo objects in factory.
I accidentally changed id=>after when prototyping data for the _new_
deferred rating (not yet written). There were no tests for this code, which
was originally written in July of 2012 (see private lovullo.git repo), so it
was not caught until QA.
This only replaces the callbacks at the highest level and starts to move
toward proper error handling. The private methods do not yet properly
propagate errors.
This adds ratedata to the quote collection. We use the existing saveQuote
method so that this operation is atomic. We're also continuing to save to
the bucket for now so as not to break existing code, but the intent in the
future will be to remove all but necessary data that should be exposed to
the client.
This also gets rid of the RatingServiceStub module, which is not used by
anything else. I suspect that I originally added it to be shared by traits,
but that's no longer going to be the case (and the only remaining trait is
unfortunately untested atm, and will be going away).
This was an adventure, and was also used as a peer programming exercise to
introduce TypeScript to other programmers in the office.
This class has far too many dependencies, which made this difficult. The
approach was to create .d.ts files for dependencies and wait on moving those
over for now, otherwise the task will never get done.
The RatingServicePublic trait was left as such for the time being; I was
able to work around a bug that was making it difficult to mix it into a
prototype.
There were no logic changes; this was just type refactoring.
Rather than leaving a token in a DONE state, we should either transition to
ACCEPTED or DEAD depending on whether the token was superceded.
* src/server/dapi/TokenedDataApi.ts (_replyUnlessStale): Accept token when
not superceded, otherwise kill.
[store]: New param.
(request): Use it.
* test/server/dapi/TokenedDataApiTest.ts: Update accordingly.
This changes the easejs interface for DataApi, which requires adding the
param to everything. The TS interface was created in a previous commit and
already contained this parameter. The idea is to remove the easejs
interface in the future, but traits are a barrier to that atm.
DocumentServer and controller demonstrate the mess that we have with regards
to instantiating dependencies. This needs to change---DocumentServer itself
was something that was started but never fully realized. It makes this
incredibly confusing, difficult to follow, and complicates important error
handling that ought to be taking place. It also discourages implementing
additional dependencies.
I'm not going to go through and provide a ChangeLog-style commit message for
this commit. I'm too exhausted by this crap.
Just trying to reduce some boilerplate. I kept this as a separate commit to
illustrate clearly how this type of things is done since we'll have people
learning TypeScript.
* src/types/misc.ts (NodeCallback<T,R>): New type.
* src/server/dapi/TokenedDataApi.ts: Use it.
* test/server/dapi/TokenedDataApiTest.ts: Use it.
This integrates the PersistentTokenStore into the DataAPI system via a
decorator. Unfortunately, it requires an API change and propagating data
through the system is a huge mess, which is the topic of a following
commit. The API modification was a compromise.
This modifies the interface of DataApi to include a third parameter. I am
continuing to export the old easejs interface for an incremental migration
away from it. That old interface will be modified next commit, since
it requires modifying a lot of files and will muddy up this commit.
* src/dapi/DataApi.ts: Rename from js. Add types. Add new interface.
Continue exporting old.
* src/server/dapi/TokenedDataApi.ts: New class.
* test/server/dapi/TokenedDataApiTest.ts: New test cases.
This creates an interface for TokenStore. The main motivation of this right
now is testing, since I'm punting on figuring out a mock framework right
now (due to time constraints).
* src/server/token/TokenStore.ts: Rename file.
* src/server/token/store/PersistentTokenStore.ts: Rename from TokenStore.
* test/server/token/TokenStoreTest.ts: Rename file.
* test/server/token/store/PersistentTokenStoreTest.ts: Rename from TokenStoreTest.
This is much more useful information than the last modified. For example:
- Token A is created. It becomes the last modified.
- Token B is created. It becomes the last modified.
- Token A completes. Mismatch. It becomes the last modified.
- Token B completes. Mismatch. It becomes the last modified.
So in this case, we're unable to use the flag to determine whether we should
ignore the token. But if we instead us the new flag to see what token was
last _created_, the problem is solved.
This should have been obvious the first time around.
* src/server/token/MongoTokenDao.ts (updateToken): Query
`lastState'. Return its value. Update its value.
(getToken): Query lastState. Return its value.
* src/server/token/Token.ts (Token)[last_state]: New field.
* src/server/token/TokenDao.ts (TokenQueryResult, TokenNamespaceResults):
Use type instead of interface.
(TokenStateHistory): New type.
(TokenNamespaceData)[lastState]: New optional field.
(TokenData)[prev_state]: New field.
* src/server/token/TokenStore.ts: Return previous state data for each
method.
* test/server/token/MongoTokenDaoTest.ts: Add last_state.
* test/server/token/TokenStoreTest.ts: Likewise.
The primary use case for this is currently the DataAPI, and the quote id is
only available at the highest level of the server, before dapis are
processed.
In any case, the TokenStore was already described in terms of a combination
of document id; namespace; and root field; so it makes sense for doc id to
be part of the constructor.
If a more generic TokenStore is needed in the future, we could go back to
the previous API and wrap it in another class, like a partially applied
function (e.g. `DocumentTokenStore`).
* src/server/token/TokenStore.ts: Move doc_id out of arguments and into the
constructor.
* test/server/token/TokenStoreTest.ts: Update accordingly.
Does not yet support token updates.
* src/server/token/Token.ts (TokenStateDeadable, TokenStateDoneable,
TokenStateAcceptable, Token): New types.
* src/server/token/TokenStore.ts: New class.
* test/server/token/TokenStoreTest.ts: New test case.
This causes the DAO to return the state of the document prior to the
database operation (in the case of a retrieval, the previous state is the
same as the current state). This will allow determining whether other
tokens were modified since a previous request.
* src/server/token/MongoTokenDao.ts: Stop using TokenType in favor of new
TokenState.
(updateToken): Query for and return previous state. Use findAndModify
instead of update.
(getToken): Return previous state. Minor changes to account for new
TokenQueryResult types (null=>undefined).
* src/server/token/Token.ts: Add comments for existing Token{Id,Namespace}
nominal types.
(TokenState): New string enum.
* src/server/token/TokenDao.ts: Import new TokenState.
(TokenDao)[updateToken]: Use it.
(TokenType): Remove.
(TokenQueryResult, TokenNamespaceResults, TokenNamespaceData):
null=>undefined for unavailable value. null was wrong.
(TokenStatus): Token{Type=>State}.
(TokenData)[prev_state, prev_status]: New fields.
* test/server/token/MongoTokenDaoTest.ts: Update tests accordingly.
Since some of these data are generated within TokenDao (e.g. the timestamp),
the caller cannot infer all values.
* src/server/token/MongoTokenDao.ts (updateToken): Return
Promise<{void=>TokenData}>.
* src/server/token/TokenDao.ts (TokenDao)[#updateToken]: Update interface
accordingly.
* test/server/token/MongoTokenDaoTest.ts: Update test accordingly.
Rather than replying with null, which complicates using the returned promise
efficiently, we'll respond with a unique error that allows us to distinguish
between a database failure and a missing token.
These are more traditional errors, but we're moving toward structured
logging, so I want error objects that provide more context. I'll explore
that a bit more in next commit. Unfortunately, the untypedness of Promise
rejections make for a less than desirable situation here. Async/await is
not yet an option since we're still compiling to ES5 (have to support
IE11), and TS compiles async/await into generators for environments that
don't support them, which also are not available in ES5.
* src/server/service/TokenedService.js (_getQuoteToken): Remove null check,
since this situation can no longer occur.
* src/server/token/MongoTokenDao.ts (getToken): Remove null from return type
union; reject with `UnknownTokenError' instead.
* src/server/token/TokenDao.ts: Modify interface accordingly.
* src/server/token/UnknownTokenError.ts: New class.
* test/server/token/MongoTokenDaoTest.ts: Modify tests accordingly. Add
missing test for latest token namespace missing.
TokenDao has been renamed to MongoTokenDao. While it's good for this to
have its own interface anyway, the immediate motivation was for unit tests:
I started playing with mocking with TypeScript and researching some
libraries, but I don't have time to research enough to commit to any of them
at the moment. Interfaces remove the need to mock at all.
This also stops using `export default' in favor of just importing by
name. Using defaults only saves us a few characters, and it makes for
awkward syntax in various cases (e.g. with multiple exports). But I'm still
new to TS, so who knows if I'll be flip-flopping on this decision in the
future. If we kept to our normal 1:1 file:definition convention, it
wouldn't cause problems, but based on the types I've had to define so far,
that'd cause way too much bloat and boilerplate.
* src/server/daemon/controller.js: No long import `default'. Use
`MongoTokenDao'.
* src/server/token/TokenedService.js: Stop checking type (since TS
interfaces do not result in compiler output, easejs cannot validate
against them.)
* src/server/token/MongoTokenDao.ts: Rename from `TokenDao.ts'.
* src/server/token/TokenDao.ts: Rename from `TokenQueryResult.ts'.
(TokenDao): New interface.
* src/server/token/TokenQueryResult.ts: Rename to `TokenDao.ts'.
* test/server/token/MongoTokenDaoTest.ts: Rename from `TokenDaoTest.ts'.
* src/document/Document.ts: New file.
* src/server/token/TokenDao.ts: quote=>document and use DocumentId.
* test/server/token/TokenDaoTest.ts: Likewise.
This beings an experiment with nominal typing using what the TS community
calls "branding". The lack of nominal types was one of my biggest
disappointments with TS, so this should really help to mitigate bugs
resulting from misappropriation of data.
* src/server/token/Token.ts: New file.
* src/server/token/TokenDao.ts: Use Token{Id,Namespace}.
* src/server/token/TokenQueryResult.ts: Likewise.
* src/types/misc.d.ts: Introduce NominalType and UnixTimestamp.
* test/server/token/TokenDaoTest.ts: Use nominal types.
* src/server/daemon/controller.js (getUnixTimestamp): New method. Not
ideal, but better than where it was.
(_initExportService): Pass to TokenDao constructor.
* src/server/token/TokenDao.ts (_getTimestamp): New field.
(constructor)[get_timestamp]: New param.
(updateToken): Use it.
* test/server/token/TokenDaoTest.ts: Provide stub timestamp function.
Our coding standards are to create a separate file for classes and
interfaces, but this is a bit different: this "interface" is more like a
struct, and it's used to define the return type of a method of this
class. Since it's inherently coupled, I'm keeping it in the same file.
The idea is that the caller will provide its own abstraction rather than
continuing to export this one. Note that Typescript does support
re-exporting symbols if need be.
* src/server/daemon/controller.js: Adjust import of TokenDao (TS compiles
default modules as `default').
* src/server/service/TokenedService.js: Adjust import of TokenDao.
* src/server/token/TokenDao.ts: Export TokenDao as `default'. Export
TokenData.
* test/server/token/TokenDaoTest.ts: Adjust import of TokenDao. Import
TokenData.
This makes minimal changes to TokenedService, even though there is obvious
refactoring that can be done to reduce duplication, because the class is
currently untested.
* src/server/service/TokenedService.js (_getQuoteToken, generateToken,
killToken, acceptToken, completeToken): Expect promise.
* src/server/token/TokenDao.ts (updateToken, getToken): Remove callback
param, return Promise.
* test/server/token/TokenDaoTest.ts: Use promises.
This tests the existing state of TokenDao before additional modifications
are made. This commit also further refines the types introduced in a
previous commit.
This is also the first test written in Typescript.
* package.json.in (devDependencies): Add node, chai, and mocha types.
* src/server/token/TokenDao.ts (updateToken): `data` accepts null (as it
should). Do not conditionall add data to object (it doesn't matter for
later retrieval). Note nondeterminism with date. More concise syntax
for object fields.
* src/server/token/TokenQueryResult.ts: Make all fields readonly.
(TokenStatus): Date is no longer optional (see above mention).
* src/types/mongodb.d.ts: Remove generics (erroneously added).
(Collection)[update]: Remove 3-argument declaration (see comment).
* test/server/token/TokenDaoTest.ts: New test case.
It looks like the metabucket is never initialized, so saving the quote is
right now the only thing that sets default values. That should be fixed in
the future.
This also begins adding tests for the terrible MongoServerDao, that could
use some refactoring.
* src/server/db/MongoServerDao.js: Make `meta' mutable. I had forgotten to
remove the code that mutates it (since our version of v8 right now does
not blow up for const assignments), so this is all that's needed.
* test/server/db/MongoServerDaoTest.js: New file to test this situation.
Default data was converted to an empty array if the data evaluated to
false. We only want to convert it if it is undefined so values that are
false remain false.
* src/program/ProgramInit.js (_isKnownType): Account for ancient qtype
representation (as a string).
* src/server/quote/ProgramQuoteCleaner.js (_isKnownType): Likewise.
* test/program/ProgramInitTest.js: New test case for this situation.
* test/server/quote/ProgramQuoteCleanerTest.js: Modify existing test case
for this situation.
These denote fields that are generated but do not actually have any data
associated with them. For example, select options with predicates have a
field generated so that they contribute to the group field count (so that
the group will automatically show/hide appropriately), but those should
never have values associated with them in the bucket.
This was manifesting as a nasty bug: The bucket contained a key for
generated options. When the quote is loaded, the client "empties" the
bucket. In doing so, it set the option value to the empty string, which had
the effect of rendering the dropdown useless---every value was the empty
string!
* src/program/ProgramInit.js (_isKnownType): New method.
(init): Use it and ignore fields with unknown types.
* src/server/Server.js: Add note that we shouldn't have this logic
duplicated between ProgramInit and ProgramQuoteCleaner.
* src/server/quote/ProgramQuoteCleaner.js (_fixGroup): Ignore fields with
unknown types.
(_isKnownType): New method.
* test/program/ProgramInitTest.js: Update existing tests. Add new.
* test/server/quote/ProgramQuoteCleanerTest.js: Test this case.
* src/server/quote/ProgramQuoteCleaner.js (clean): Add docblock.
Replace previous linked group cleaning with call to `_fixGroup'.
(_fixGroup): New method. Similar logic to previous linked group cleaning,
except that fields are never truncated.
(_fixLinkedGroups, _getLinkedIndexLength): Remove methods.
(_getGroupLength): New method determining group size from leader length,
which also accounts for linked groups.
* test/server/quote/ProgramQuoteCleanerTest.js: New test case.
If the submission failed, we probably want to try again next time around.
* src/server/service/RatingServiceSubmitNotify.js
(_maybeNotify): Extract logic from `#postProcessRaterData'. Only set
notification flag in absence of dapi error.
(postProcessRaterData): Use it.
* test/server/service/RatingServiceSubmitNotifyTest.js: Update tests
accordingly.
Austin Schaffer