diff --git a/COPYING.AGPL b/COPYING.AGPL new file mode 100644 index 0000000..dba13ed --- /dev/null +++ b/COPYING.AGPL @@ -0,0 +1,661 @@ + GNU AFFERO GENERAL PUBLIC LICENSE + Version 3, 19 November 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU Affero General Public License is a free, copyleft license for +software and other kinds of works, specifically designed to ensure +cooperation with the community in the case of network server software. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +our General Public Licenses are intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + Developers that use our General Public Licenses protect your rights +with two steps: (1) assert copyright on the software, and (2) offer +you this License which gives you legal permission to copy, distribute +and/or modify the software. + + A secondary benefit of defending all users' freedom is that +improvements made in alternate versions of the program, if they +receive widespread use, become available for other developers to +incorporate. Many developers of free software are heartened and +encouraged by the resulting cooperation. However, in the case of +software used on network servers, this result may fail to come about. +The GNU General Public License permits making a modified version and +letting the public access it on a server without ever releasing its +source code to the public. + + The GNU Affero General Public License is designed specifically to +ensure that, in such cases, the modified source code becomes available +to the community. It requires the operator of a network server to +provide the source code of the modified version running there to the +users of that server. Therefore, public use of a modified version, on +a publicly accessible server, gives the public access to the source +code of the modified version. + + An older license, called the Affero General Public License and +published by Affero, was designed to accomplish similar goals. This is +a different license, not a version of the Affero GPL, but Affero has +released a new version of the Affero GPL which permits relicensing under +this license. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU Affero General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Remote Network Interaction; Use with the GNU General Public License. + + Notwithstanding any other provision of this License, if you modify the +Program, your modified version must prominently offer all users +interacting with it remotely through a computer network (if your version +supports such interaction) an opportunity to receive the Corresponding +Source of your version by providing access to the Corresponding Source +from a network server at no charge, through some standard or customary +means of facilitating copying of software. This Corresponding Source +shall include the Corresponding Source for any work covered by version 3 +of the GNU General Public License that is incorporated pursuant to the +following paragraph. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the work with which it is combined will remain governed by version +3 of the GNU General Public License. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU Affero General Public License from time to time. Such new versions +will be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU Affero General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU Affero General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU Affero General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU Affero General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Affero General Public License for more details. + + You should have received a copy of the GNU Affero General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If your software can interact with users remotely through a computer +network, you should also make sure that it provides a way for users to +get its source. For example, if your program is a web application, its +interface could display a "Source" link that leads users to an archive +of the code. There are many ways you could offer source, and different +solutions will be better for different programs; see section 13 for the +specific requirements. + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU AGPL, see +. diff --git a/README.md b/README.md index c19f8d7..2ffe010 100644 --- a/README.md +++ b/README.md @@ -45,9 +45,15 @@ terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. +The liza server is licensed differently: you can redistribute it and/or +modify it under the terms of the GNU Affero General Public License as +published by the Free Software Foundation, either version 3 of the License, +or (at your option) any later version. + This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. -The full license is available in `COPYING`. +The full licenses are available in `COPYING` and `COPYING.AGPL`. + diff --git a/configure.ac b/configure.ac index 641d840..0b30840 100644 --- a/configure.ac +++ b/configure.ac @@ -18,7 +18,7 @@ # along with this program. If not, see . ## -AC_INIT([liza], [0.11.4], [dev@lovullo.com]) +AC_INIT([liza], [1.0.0], [dev@lovullo.com]) AC_CONFIG_AUX_DIR([build-aux]) AM_INIT_AUTOMAKE([foreign]) diff --git a/src/assert/Assertion.js b/src/assert/Assertion.js new file mode 100644 index 0000000..2353e48 --- /dev/null +++ b/src/assert/Assertion.js @@ -0,0 +1,387 @@ +/** + * Base Assertion class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + * + * @todo notice that this is so old that it is a prototype + */ + + +exports.create = function( name, data ) +{ + return new Assertion( name, data ); +}; + + +var Assertion = function( name, data ) +{ + if ( data === undefined ) + { + data = name; + } + + /** + * List of children that failed the assertion + * @type {Array.} + */ + this.failures = []; + + /** + * List of children that passes the assertion + * @type {Array.} + */ + this.successes = []; + + /** + * Holds data for the destination var + * @type {Object} + */ + this.dest = {}; + + /** + * Default value to return if assertions contain no values to compare + * @type {boolean} + */ + this.defaultValue = true; + + /** + * Default return value when processing data + * + * @type {boolean} + */ + this.processDefault = true; + + /** + * Whether to track failures within the failure array + * @type {boolean} + */ + this.trackFailures = true; + + /** + * Whether to automatically add successful children to the destination var + * @type {boolean} + */ + this.autoAddDest = true; + + /** + * Return default value if no data is given + * @type {boolean} + */ + this.failOnEmpty = true; + + /** + * Function to call for performing the assertion + * @type {Function} + */ + this.processCallback = null; + + /** + * Function to call to parse the expected value before the assertion + * @type {Function} + */ + this.parseExpectedCallback = null; + + this.name = ''+( name ); + + + this._init( data ); +}; + + +Assertion.prototype = +{ + /** + * Initializes the assertion with the provided data + * + * @param Object data the data passed to the constructor + * + * @return undefined + */ + _init: function( data ) + { + // we accept a shorthand; if a function is passed rather than an object, + // then use that as the assertion function + if ( data instanceof Function ) + { + data = { assert: data }; + } + + // the assertion function will be whatever they passed in, or will + // default to returning false if nothing was provided + this.processCallback = data.assert || function() + { + return false; + }; + + // If a function was provided to parse the expected value before it is + // used (allowing it to be cached between assertions), it will be used. + // Otherwise, we'll just return the value we were passed. + this.parseExpectedCallback = data.parseExpected || function( value ) + { + return value; + }; + + this.defaultValue = ( data.defaultValue === undefined ) + ? this.defaultValue + : data.defaultValue; + + this.processDefault = ( data.processDefault === undefined ) + ? this.processDefault + : data.processDefault; + }, + + + /** + * Cleans out failures and destination data from a previous run + * + * @return undefined + */ + _clean: function() + { + this.failures = []; + this.successes = []; + this.dest = {}; + }, + + + /** + * Calls the function to parse the expected value + * + * This allows the expected value to be processed and cached between + * assertions, rather than processing it each time an assertion is + * performed. This method will be called for each new expected value, if + * multiple exist for an assertion set. + * + * @param mixed value the expected value to process + * + * @return mixed result of function being called + */ + _parseExpected: function( value ) + { + return this.parseExpectedCallback.call( this, value ); + }, + + + /** + * Performs the assertion by calling the assertion callback + * + * @param mixed expected the value to compare against + * @param mixed given the value provided to be compared to the expected + * + * @return Boolean true if assertion succeeded, otherwise false + */ + _performAssertion: function( expected, given ) + { + return this.processCallback.call( { + expected: expected, + given: given + } ); + }, + + + /** + * Asserts the given data + * + * This method loops through the given data, comparing it to the associated + * expected value. The expected value is shifted off the array each time + * another value is compared, until there are no expected values left. At + * that point, the last expected value is used for the remainder of the + * assertions. + * + * @param Array expected the values to compare against + * @param Array given the values provided to be compared to the expected + * + * @return Boolean whether the assertion succeeded + */ + _processData: function( expected, given ) + { + expected = expected || []; + given = given || []; + + // make copies of the provided arrays since we'll be doing some pretty + // shady stuff to 'em + expected = expected.slice( 0 ); + + if ( given.length === undefined ) + { + given = [ given ]; + } + + given = given.slice( 0 ); + + var return_val = this.processDefault, + last = '', + len = given.length; + + for ( var i = 0; i < len; i++ ) + { + var val = given[i]; + + // ignore null values + if ( val === null ) + { + continue; + } + + // if another expected value was given, use it, otherwise use the + // last expected value + cmp = ( expected.length > 0 ) + ? last = this._parseExpected( expected.shift() ) + : last; + + if ( this._performAssertion( cmp, val ) === false ) + { + // the assertion failed + if ( this.processDefault == true ) + { + return_val = false; + } + + // if we're tracking failures, add it to the list + if ( this.trackFailures ) + { + this.failures[i] = i; + } + } + else + { + // assertion succeeded + if ( this.processDefault == false ) + { + return_val = true; + } + + if ( this.autoAddDest ) + { + this.dest[i] = val; + } + + this.successes[i] = i; + } + } + + return return_val; + }, + + + /** + * Perform an assertion on the given data + * + * @param Array expected the values to compare against + * @param Array given the values provided to be compared to the expected + * + * @return Boolean true on success, false on failure + */ + assert: function( expected, given ) + { + given = given || []; + + if ( given.length === undefined ) + { + given = [ given ]; + } + + // if we weren't given anything to compare, return the default + if ( this.failOnEmpty + && ( ( given === undefined ) || ( given.length == 0 ) ) + ) + { + return this.defaultValue; + } + + this._clean(); + + return this._processData( expected, given ); + }, + + + /** + * Perform an assertion on a single value + * + * @param mixed expected the value to compare against + * @param mixed given the value provided to be compared to the expected + * + * @return Boolean true on success, false on failure + */ + assertSingle: function( expected, given ) + { + // if we weren't given anything to compare, return the default + if ( this.failOnEmpty + && ( ( given === undefined ) || ( given.length === 0 ) ) + ) + { + return this.defaultValue; + } + + return this._performAssertion( this._parseExpected( expected ), given ); + }, + + + /** + * Returns successful children + * + * @return Array + */ + getSuccesses: function() + { + return this.successes; + }, + + + setSuccesses: function( successes ) + { + this.successes = successes; + return this; + }, + + + /** + * Returns the failed children + * + * @return Object + */ + getFailures: function() + { + return this.failures; + }, + + + setFailures: function( failures ) + { + this.failures = failures; + return this; + }, + + + /** + * Returns the destination var data + * + * @return Object + */ + getDestData: function() + { + return this.dest; + }, + + + getName: function() + { + return this.name; + } +}; + diff --git a/src/assert/BaseAssertions.js b/src/assert/BaseAssertions.js new file mode 100644 index 0000000..68a529b --- /dev/null +++ b/src/assert/BaseAssertions.js @@ -0,0 +1,278 @@ +/** + * Contains common assertions + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +// import creation method +var create = require( './Assertion' ).create; + +var explode_flip = function( value ) +{ + // 'splody 'sploy! + // ...we accept a comma-separated list + var val = value.split( ',' ); + var dest = {}; + + // flip the array (in PHP this is the equivalent of + // array_flip()) so we can do a hash lookup + var len = val.length; + for ( var i = 0; i < len; i++ ) + { + dest[ val[i] ] = i; + } + + return dest; +}; + + +/** + * Asserts that the given value is within a list of expected values + * + * @return lovullo.assert.Assertion + */ +exports.any = create( 'any', { + processDefault: false, + + parseExpected: explode_flip, + + assert: function() + { + // same as isIn + return ( this.expected[ this.given ] !== undefined ); + } +} ); + + +/** + * Asserts that the given values are between the expected values + * + * @return lovullo.assert.Assertion + */ +exports.between = create( 'between', { + parseExpected: function( value ) + { + return value.split( ',' ); + }, + + assert: function() + { + var lower = this.expected[0]; + var upper = this.expected[1]; + + return ( ( this.given < lower ) || ( this.given > upper ) ) + ? false + : true; + } +}); + + +/** + * Asserts that the given values are empty + * + * @return lovullo.assert.Assertion + */ +exports.empty = create( 'empty', function() +{ + // empty string, 0 integer or 0.00 float (and many variations), or objects + // that JS considers empty (note that [] will automatically be considered + // empty due to string cohersion when comparing with the regex) + return /^(0+\.?0*|0*\.0+)?$/.test( this.given ) || !this.given; +} ); + + + +function eqcmp( given, expected ) +{ + // if expected is a string whoose integer equivalent converted to a string + // is the same as its value, then we'll do integer comparisons + if ( ''+( +given ) === expected ) + { + given = +given; + expected = +expected; + } + else + { + // otherwise, compare them as strings + given = ''+given; + expected = ''+expected; + } + + return ( given === expected ); +} + + +/** + * Asserts that the given value is equal to the expected value + * + * @return lovullo.assert.Assertion + */ +exports.equal = create( 'equal', function() +{ + return eqcmp( this.given, this.expected ); +}); + + +/** + * Asserts that the given value is not equal to the expected value + * + * @return lovullo.assert.Assertion + */ +exports.notEqual = create( 'notEqual', function() +{ + return !eqcmp( this.given, this.expected ); +}); + + +/** + * Asserts that the given value is greater than the expected value + * + * @return lovullo.assert.Assertion + */ +exports.greaterThan = create( 'greaterThan', function() +{ + return ( +this.given > +this.expected ) + ? true + : false; +}); + +/** + * Asserts that the given value is greater or equal to the expected value + * + * @return lovullo.assert.Assertion + */ +exports.greaterOrEqualTo = create( 'greaterOrEqualTo', function() +{ + return ( +this.given >= +this.expected ) + ? true + : false; +}); + + +/** + * Always returns false + * + * @return lovullo.assert.Assertion + */ +exports.fail = create( 'fail', { + defaultValue: false, + + assert: function() + { + return false; + } +}); + + +/** + * Asserts that the given value is within a list of expected values + * + * @return lovullo.assert.Assertion + */ +exports.isIn = create( 'isIn', { + parseExpected: explode_flip, + + assert: function() + { + return ( this.expected[ this.given ] !== undefined ); + } +}); + + +/** + * Asserts that the given value is not in a list of values + * + * @return lovullo.assert.Assertion + */ +exports.isNotIn = create( 'isNotIn', { + parseExpected: explode_flip, + + assert: function() + { + return ( this.expected[ this.given ] === undefined ); + } +}); + + +/** + * Asserts that the given value is less than the expected value + * + * @return lovullo.assert.Assertion + */ +exports.lessThan = create( 'lessThan', function() +{ + return ( +this.given < +this.expected ) + ? true + : false; +}); + + +/** + * Asserts that the given value is less than or equal to the expected value + * + * @return lovullo.assert.Assertion + */ +exports.lessOrEqualTo = create( 'lessOrEqualTo', function() +{ + return ( +this.given <= +this.expected ) + ? true + : false; +}); + + +/** + * Asserts that the given values are _not_ empty + * + * @return lovullo.assert.Assertion + */ +exports.notEmpty = create( 'notEmpty', function() +{ + // just negate what empty returns + return !( exports.empty.assertSingle( + this.expected, this.given + ) ); +}); + + +/** + * Always returns true + * + * @return lovullo.assert.Assertion + */ +exports.pass = create( 'pass', function() +{ + return true; +}); + + +/** + * Asserts that the given value matches the regexp + * + * @return lovullo.assert.Assertion + */ +exports.regex = create( 'regex', { + parseExpected: function( value ) + { + return new RegExp( value, 'i' ); + }, + + assert: function() + { + return ( this.expected.test( this.given ) ); + } +}); diff --git a/src/bucket/BucketSiblingDescriptor.js b/src/bucket/BucketSiblingDescriptor.js new file mode 100644 index 0000000..2a9ccae --- /dev/null +++ b/src/bucket/BucketSiblingDescriptor.js @@ -0,0 +1,207 @@ +/** + * Describes relationships between bucket fields + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class; + + +/** + * Describe relationships between bucket fields + * + * Siblings may be grouped under a given identifier. Siblings may be a part + * of different groups. Each group can have any number of leaders which + * together may be used as a group identifier. + */ +module.exports = Class( 'BucketSiblingDescriptor', +{ + /** + * Fields and their group memberships + * @type {Object} + */ + 'private _fields': {}, + + /** + * A simple hash of group names and their associated leaders + * @type {Object} + */ + 'private _groups': {}, + + + /** + * Define a new group id with optional initial members + * + * @param {string} gid group id to define + * @param {Array.} members initial group members + * + * @return {BucketSiblingDescriptor} self + */ + 'public defineGroup': function( gid, members ) + { + // do not allow re-defining groups; use addGroupMember instead + if ( this._groups[ gid ] ) + { + throw Error( "Group '" + gid + "' already exists" ); + } + + members = members || []; + + // will store group leaders + this._groups[ gid ] = []; + + // mark each field as a member of this group + for ( var field in members ) + { + this.addGroupMember( gid, members[ field ] ); + } + + return this; + }, + + + /** + * Add a field to a defined group + * + * @param {string} gid defined group id + * @param {string} field name of field + * + * @return {BucketSiblingDescriptor} self + */ + 'public addGroupMember': function( gid, field ) + { + if ( !( this._groups[ gid ] ) ) + { + throw Error( "Group '" + gid + "' does not exist" ); + } + + if ( !( this._fields[ field ] ) ) + { + this._fields[ field ] = {}; + } + + // add group membership + this._fields[ field ][ gid ] = true; + return this; + }, + + + /** + * Retrieve names of all fields that are a member of the given group + * + * @param {string} gid group id + * + * @return {Array.} group members + */ + 'public getGroupMembers': function( gid ) + { + if ( !( this._groups[ gid ] ) ) + { + throw Error( "Group '" + gid + "' does not exist" ); + } + + var members = []; + + // construct group membership from fields + for ( var field in this._fields ) + { + if ( this._fields[ field ][ gid ] ) + { + members.push( field ); + } + } + + return members; + }, + + + /** + * Retrieve a list of all defined group ids + * + * @return {Array.} defined group ids + */ + 'public getGroupIds': function() + { + var gids = []; + + // can use Object.keys() in the future when all browsers support it...or + // write our own + for ( var group in this._groups ) + { + gids.push( group ); + } + + return gids; + }, + + + /** + * Mark fields of a defined group as leaders + * + * Each new leader must be a part of the group. Once set, the leaders cannot + * be redefined. + * + * @param {string} gid group id + * @param {Array.} new_leaders names of leaders + */ + 'public markGroupLeaders': function( gid, new_leaders ) + { + var leaders = this._groups[ gid ]; + + if ( leaders.length > 0 ) + { + throw Error( "Group '" + gid + "' leaders already set" ); + } + + var l = new_leaders.length; + for ( var i = 0; i < l; i++ ) + { + var field = new_leaders[ i ]; + if ( !( ( this._fields[ field ] || {} )[ gid ] ) ) + { + throw Error( + "Field '" + field + "' is not a member of group '" + + gid + "'" + ); + } + + leaders.push( field ); + } + + return this; + }, + + + /** + * Return the leaders of a defined group + * + * @param {string} gid group id + * + * @return {Array.} group leaders + */ + 'public getGroupLeaders': function( gid ) + { + if ( !( this._groups[ gid ] ) ) + { + throw Error( "Group '" + gid + "' is undefined" ); + } + + return Array.prototype.slice.call( this._groups[ gid ], 0 ); + } +} ); + diff --git a/src/bucket/bucket_filter.js b/src/bucket/bucket_filter.js new file mode 100644 index 0000000..a5ba6c1 --- /dev/null +++ b/src/bucket/bucket_filter.js @@ -0,0 +1,131 @@ +/** + * Filters bucket data + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +var filters = { + name: /[^a-zA-Z \'\.-]/g, + address: /[^a-zA-Z0-9 &\/\'\.,-]/g, + city: /[^a-zA-Z \'\.-]/g, + currency: /[^\-0-9\.]/g, + 'float': /[^0-9\.]/g, + 'date': /[^0-9\/\.-]/g, + dba: /[^a-zA-Z0-9 \'\.,&\(\):\/-]/g, + dollars: /[^\-0-9\.]/g, + email: /[^a-zA-Z0-9_\.@-]/g, + number: /[^0-9\.]/g, + cvv2: /[^0-9]/g, + quoteId: /[^0-9]/g, + personalId: /[^0-9]/g, + phone: /[^0-9 \(\)\.-]/g, + url: /[^a-zA-Z0-9:\/_\.\+\$\(\)\\\?&@=\';#~-]/, + year: /[^0-9]/g, + zip: /[^0-9a-zA-Z-]|[DFIOQU]/g, + radio: /[^0-9a-zA-Z_\/-]/g, + legacyradio:/[^0-9a-zA-Z_\/-]/g, + submit: /[^0-9a-zA-Z _-]/g, + select: /[^0-9a-zA-Z &\+|\.\/,\(\)'"_-]/g, + + 'default': new RegExp( '//' ), +}; + + +/** + * Filters bucket data based on the provided types + * + * If a type is not provided, the data is considered to be unwanted and is + * removed entirely. Otherwise, the filter is applied to every element in the + * array. + * + * The data is modified in place. + * + * @param Object data data to filter + * @param Object key_types filter types + * + * @return Object modified data + */ +exports.filter = function( data, key_types, ignore_types, permit_null ) +{ + permit_null = ( permit_null === undefined ) ? false : !!permit_null; + + // loop through each of the bucket values + for ( key in data ) + { + // BC between string and object representation of type data + var type_data = key_types[ key ], + type = ( type_data ) + ? type_data.type || type_data + : undefined; + + var dim = ( type_data ) + ? type_data.dim + : 1; + + var values = data[ key ]; + + // if it's not an expected bucket value, get rid of it (this prevents + // users from using us as their own personal database + if ( ( type === undefined ) || ( ignore_types[ key ] === true ) ) + { + delete data[ key ]; + continue; + } + + // attempt to get the filter, or use the default filter if one was not + // found + var filter = filters[ type ] || filters['default']; + + if ( values === undefined || values === null ) + { + values = []; + } + + // XXX: this does not handle multi-dimensional data, since we should + // _remove_ this file entirely + data[ key ] = ( dim === 1 ) + ? filterValues( values, filter, permit_null ) + : values; + } + + return data; +}; + + +function filterValues( values, filter, permit_null ) +{ + var len = values.length; + + for ( var i = 0; i < len; i++ ) + { + if ( typeof values[ i ] !== 'string' ) + { + if ( permit_null && ( values[ i ] === null ) ) + { + continue; + } + + values[ i ] = ''+( values[ i ] ); + } + + values[ i ] = values[ i ].replace( filter, '' ); + } + + return values; +} + diff --git a/src/bucket/diff/BucketDiff.js b/src/bucket/diff/BucketDiff.js new file mode 100644 index 0000000..22e7277 --- /dev/null +++ b/src/bucket/diff/BucketDiff.js @@ -0,0 +1,36 @@ +/** + * Interface for performing diffs on buckets + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +var Interface = require( 'easejs' ).Interface; + + +module.exports = Interface( 'BucketDiff', +{ + /** + * Perform a diff given a diff context + * + * @param {BucketDiffContext} context diff context + * + * @return {BucketDiffResult} result of diff + */ + 'public diff': [ 'context' ] +} ); + diff --git a/src/bucket/diff/BucketDiffContext.js b/src/bucket/diff/BucketDiffContext.js new file mode 100644 index 0000000..62da8e0 --- /dev/null +++ b/src/bucket/diff/BucketDiffContext.js @@ -0,0 +1,28 @@ +/** + * Describes a context with which diffing may take place between buckets + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +var Interface = require( 'easejs' ).Interface; + + +module.exports = Interface( 'BucketDiffContext', +{ +} ); + diff --git a/src/bucket/diff/BucketDiffResult.js b/src/bucket/diff/BucketDiffResult.js new file mode 100644 index 0000000..7458c31 --- /dev/null +++ b/src/bucket/diff/BucketDiffResult.js @@ -0,0 +1,33 @@ +/** + * Represents the result of a bucket diff + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +var Interface = require( 'easejs' ).Interface; + + +module.exports = Interface( 'BucketDiffResult', +{ + /** + * Describes what fields have changed using boolean flags; does not include + * unchanged fields + */ + 'public describeChanged': [] +} ); + diff --git a/src/bucket/diff/GroupedBucketDiffContext.js b/src/bucket/diff/GroupedBucketDiffContext.js new file mode 100644 index 0000000..bcd7ac2 --- /dev/null +++ b/src/bucket/diff/GroupedBucketDiffContext.js @@ -0,0 +1,181 @@ +/** + * Describes a context with which diffing may take place between buckets + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + Bucket = require( '../Bucket' ), + BucketDiffContext = require( './BucketDiffContext' ), + BucketSiblingDescriptor = require( '../BucketSiblingDescriptor' ); + + +/** + * Describe a context with which diffing may take place between buckets + * + * This encapsulates a standard diff context and adds an additional group + * context for performing a grouped diff on the siblings of a particular + * group. + */ +module.exports = Class( 'GroupedBucketDiffContext' ) + .implement( BucketDiffContext ) + .extend( +{ + /** + * Context to decorate + * @type {BucketDiffContext} + */ + 'private _context': null, + + /** + * Descriptor for sibling relationships + * @type {BucketSiblingDescriptor} + */ + 'private _desc': null, + + /** + * Identifier of group to diff + * @type {string} + */ + 'private _gid': '', + + /** + * Hash table of group leaders for quick reference + * @type {Object} + */ + 'private _leaders': {}, + + + /** + * Prepare a context given an existing context and a sibling descriptor + * + * Augments an existing context so that it will contain enough information + * to perform a diff on a group of fields. + * + * @param {BucketDiffContext} context existing context to augment + * @param {BucketSiblingDescriptor} desc field relationship descriptor + * @param {string} gid id of group to diff + */ + __construct: function( context, desc, gid ) + { + if ( !( Class.isA( BucketDiffContext, context ) ) ) + { + throw TypeError( "Must provide a BucketDiffContext" ); + } + else if ( !( Class.isA( BucketSiblingDescriptor, desc ) ) ) + { + throw TypeError( "Must provide a BucketSiblingDescriptor" ); + } + else if ( !gid ) + { + throw Error( "Must provide a group identifier for diffing" ); + } + + this._context = context; + this._desc = desc; + this._gid = ''+gid; + + this._validateGroup(); + this._processLeaders(); + }, + + + /** + * Validates that the provided group exists and has at least one leader + * + * @throws {Error} if group does not exist or has no leaders + */ + 'private _validateGroup': function() + { + // will throw an exception if group does not exist + if ( this._desc.getGroupLeaders( this._gid ).length === 0 ) + { + throw Error( + "Group '" + this._gid + "' must have at least one leader" + ); + } + }, + + + /** + * Process group leaders into a hash table for speedy referencing + */ + 'private _processLeaders': function() + { + var leaders = this._desc.getGroupLeaders( this._gid ), + i = leaders.length; + + while ( i-- ) + { + this._leaders[ leaders[ i ] ] = true; + } + }, + + + /** + * Invoke continuation for each field name in the intersection of the parent + * context fields with the list of fields in our group. + * + * The continuation will be passed as arguments the field name followed by + * each of the respective bucket values. + * + * @param {function( string, Array, Array )} callback continuation + * + * @return {GroupedBucketDiffContext} self + */ + 'public forEachField': function( c ) + { + var gfields = {}, + members = this._desc.getGroupMembers( this._gid ), + i = members.length; + + // hash + while ( i-- ) + { + gfields[ members[ i ] ] = true; + } + + // filter all but members of the group + this._context.forEachField( function( name ) + { + if ( !( gfields[ name ] ) ) + { + return; + } + + c.apply( null, arguments ); + } ); + }, + + + /** + * Determines if the given field is a group leader + * + * @param {string} field field name + * + * @return {boolean} true if field is a group leader, otherwise false + */ + 'public isGroupLeader': function( field ) + { + return this._leaders[ field ] || false; + }, + + + 'public proxy getFieldValues': '_context' +} ); + diff --git a/src/bucket/diff/GroupedBucketDiffResult.js b/src/bucket/diff/GroupedBucketDiffResult.js new file mode 100644 index 0000000..fd7e8b1 --- /dev/null +++ b/src/bucket/diff/GroupedBucketDiffResult.js @@ -0,0 +1,489 @@ +/** + * Represents the result of performing a grouped diff on bucket data + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + BucketDiffContext = require( './BucketDiffContext' ); + BucketDiffResult = require( './BucketDiffResult' ); + + +/** + * Result of performing a grouped diff on bucket data + * + * This class augments an existing diff result with algorithms to determine + * how a group of fields has changed; this includes recognizing when a + * group's values have changed versus when it has been moved to another + * index, added, or removed. + */ +module.exports = Class( 'GroupedBucketDiffResult' ) + .implement( BucketDiffResult ) + .extend( +{ + /** + * Result of performing the diff + * @type {BucketDiffResult} + */ + 'private _result': null, + + /** + * Context used to produce the diff, augmented with sibling descriptor + * @type {GroupedBucketDiffContext} + */ + 'private _context': null, + + /** + * Leader hash, populated at our convenience + * @type {Object} + */ + 'private _leaders': {}, + + /** + * Changed values, as provied by describeChangedValues + * @type {Object} + */ + 'private _changedValues': {}, + + + __construct: function( result, context ) + { + if ( !( Class.isA( BucketDiffResult, result ) ) ) + { + throw TypeError( "Expected a BucketDiffResult to augment" ); + } + else if ( !( Class.isA( BucketDiffContext, context ) ) ) + { + throw TypeError( "Expected BucketDiffContext" ); + } + + this._result = result; + this._context = context; + + // we'll be using this frequently + this._changedValues = result.describeChangedValues(); + }, + + + /** + * Describes what fields have changed using boolean flags; does not include + * unchanged fields + * + * These changes are subject to index tracking---that is, leaders will be + * used to determine if an index has moved, in which case it will be + * considered changed only if its values have changed relative to the move. + * For example, if two indexes are swapped but their non-leader fields + * retain the same values relative to their old positions, this does itself + * not count as a change. + * + * @return {Object} hash of arrays of boolean flags representing changes + */ + 'public describeChanged': function() + { + // the easy way to do this is to simply strip the values from the value + // changeset + var vals = this.describeChangedValues(), + ret = {}; + + for ( var field in vals ) + { + ret[ field ] = []; + + var val = vals[ field ], + i = val.length; + + // there is a change if the changeset is not undefined + while ( i-- ) + { + ret[ field ][ i ] = ( val[ i ] !== undefined ); + } + } + + return ret; + }, + + + /** + * Retrieves head and prev values for a given field + * + * It may be the case that the changeset provided by the wrapped result does + * not provide enough information; in this case, it will be looked up from + * the context. + * + * @param {string} field field name + * @param {number} mapi head index + * @param {number} mapfrom prev index (in the case of a move) + * + * @return {Array.<*>} array of [valhead, valprev] + */ + 'private _getChangedValues': function( field, mapi, mapfrom ) + { + var changed = this._changedValues[ field ], + head = changed, + valhead, + valprev; + + // these values may not be available from the standard diff if + // the index itself did not actually change; in this case, query + // for the data + if ( ( changed === undefined ) + || ( changed[ mapi ] === undefined ) + ) + { + // we know this to be true because this field/index is + // unchanged + head = this._context.getFieldValues( field )[ 0 ]; + valhead = head[ mapi ]; + valprev = ( ( mapfrom === undefined ) + ? undefined + : head[ mapfrom ] + ); + } + else + { + valhead = changed[ mapi ][ 0 ]; + valprev = ( ( mapfrom === undefined ) + ? undefined + : ( changed[ mapfrom ] || [] )[ 1 ] // may yield undefined + ); + } + + return [ valhead, valprev ]; + }, + + + /** + * Describes changes in values by listing either undefined if no change or + * an array containing, respectively, the current and previous values for + * each index + * + * @return {Object} value changes + */ + 'public describeChangedValues': function() + { + var map = this.createIndexMap(), + prevfound = [], + maxi = 0, + ret = {}; + + for ( var field in this._changedValues ) + { + var changed = this._changedValues[ field ], + mapi = map.length, + fieldret = [], + fieldn = 0; + + // process all existing indexes, keeping track of the maps that we + // encounter so that we can determine if prev indexes are accounted + // for + while ( mapi-- ) + { + var mapfrom = map[ mapi ]; + if ( mapfrom !== undefined ) + { + prevfound[ mapfrom ] = true; + } + + // if there is no change in mapping, then we can re-use the + // original changeset for this index + if ( mapfrom === mapi ) + { + fieldret[ mapi ] = changed[ mapi ]; + if ( changed[ mapi ] !== undefined ) + { + fieldn++; + } + continue; + } + + var vals = this._getChangedValues( field, mapi, mapfrom ), + valhead = vals[ 0 ], + valprev = vals[ 1 ]; + + // we must determine if there is a change relative to the + // original index + if ( valhead !== valprev ) + { + fieldret[ mapi ] = [ valhead, valprev ]; + fieldn++; + continue; + } + + // no change (set explicitly to ensure array length is correct + // and to fill any "holes" in the array that are implicitly + // undefined, which v8 cares about under certain circumstances) + fieldret[ mapi ] = undefined; + } + + // only include in the diff output if we actually have changes + if ( fieldn > 0 ) + { + ret[ field ] = fieldret; + } + } + + // if there are any prev indexes that were not in the map, then those + // indexes have been deleted; throw them onto the end of the diff + var index_cnt = this._getOriginalIndexCount(), + i = index_cnt, + addi = map.length; + + while ( i-- ) + { + // ignore handled indexes + if ( prevfound[ i ] === true ) + { + continue; + } + + var added = false; + for ( var field in this._changedValues ) + { + // may not have been initialized above + if ( !( ret[ field ] ) ) + { + ret[ field ] = []; + + // initialize each index to undefined explicitly to fill any + // holes in the prototype + var initi = index_cnt; + while ( initi-- ) + { + ret[ field ][ initi ] = undefined; + } + } + + // original value, before delete (may not be defined) + var orig = this._getChangedValues( field, i, i )[ 1 ]; + if ( orig !== undefined ) + { + ret[ field ][ addi ] = [ undefined, orig ]; + added = true; + } + } + + added && addi++; + } + + return ret; + }, + + + /** + * Determines indexes that have one or more leaders with changed values + * + * This comparison is done relative to the length of the current + * array---that is, if the original array was longer, then it will not be + * reflected here as an index change. + * + * @return {Array.} truth value for each respective index + */ + 'private _getChangedIndexes': function() + { + var ichg = []; + + for ( var field in this._changedValues ) + { + // we're only looking for leaders at the moment + if ( !( this._context.isGroupLeader( field ) ) ) + { + continue; + } + + // make our lives easier (this will not change from call to call, + // since our context cannot change) + this._leaders[ field ] = true; + + var fvals = this._changedValues[ field ], + i = fvals.length; + while ( i-- ) + { + // if this is a delete, then stop (we've reached the end of the + // new array) + if ( ( fvals[ i ] !== undefined ) + && ( fvals[ i ][ 0 ] === undefined ) + ) + { + continue; + } + + // we have a change if any previous leader for this index + // changed, or if we are not undefined (indiciating a change) + ichg[ i ] = ( ichg[ i ] || ( fvals[ i ] !== undefined ) ); + } + } + + return ichg; + }, + + + /** + * Retrieve the number of indexes in the original value + * + * @return {number} indexes found + */ + 'private _getOriginalIndexCount': function() + { + var index_count = 0; + + for ( var leader in this._leaders ) + { + if ( this._changedValues[ leader ] !== undefined ) + { + index_count = this._changedValues[ leader ].length; + break; + } + } + + return index_count; + }, + + + /** + * Attempts to locate leaders that match the original values found at + * src_index + * + * If the leader values match at a new location, then the index of that + * location is returned; if there is a match at multiple locations, then an + * error will be thrown requesting that additional leaders be used to + * disambiguate. + * + * @param {number} index current index of leader + * + * @return {number} new index if found, otherwise -1 + */ + 'private _trackDownLeaders': function( index ) + { + var found = []; + + for ( var field in this._leaders ) + { + var vals = this._changedValues[ field ], + i = vals.length; + + // check every leader for a match + var match = true; + while( i-- ) + { + // skip indexes we have already found to be non-matches + if ( found[ i ] === false ) + { + continue; + } + + // don't waste time processing our current index, which we + // already know has changed + if ( i === index ) + { + found[ i ] = false; + continue; + } + + var ivals = this._getChangedValues( field, index, i ), + valhead = ivals[ 0 ], + valprev = ivals[ 1 ]; + + // check the current value against the original value at this + // index + if ( valhead !== valprev ) + { + found[ i ] = false; + } + } + } + + var i = vals.length, + ret = -1; + while ( i-- ) + { + // non-matches are explicitly denoted as false + if ( found[ i ] === false ) + { + continue; + } + + if ( ret !== -1 ) + { + // ah, crap---ambiguous; abort! (they need to provide more + // leaders to disambiguate) + throw Error( "Ambiguous index transition; aborting analysis" ); + } + + // we found it. + ret = i; + } + + return ret; + }, + + + /** + * Generates a map that describes index reassignment + * + * Index reassignment is determined by the group leaders; if an index is + * modified and each leader is found to match identically at another index, + * then the index has been considered to be moved. If an index is added or + * cannot be found (deleted), then it maps from undefined. + * + * The returned array contains, for each respective index, the value of its + * previous index (or undefined if such a value cannot be determined). + * + * If leader values result in an ambiguous index transition, an error will + * be thrown; in this case, more leaders should be used to disambiguate. + * + * @return {Array.} index map + */ + 'public createIndexMap': function() + { + var map = []; + + // determine, based on the group leaders, which indexes have changed in + // some way (that is, the value of at least one leader at some index has + // changed) + var ichg = this._getChangedIndexes(), + i = ichg.length; + + while ( i-- ) + { + // if an index has not changed, then it maps to itself + if ( ichg[ i ] === false ) + { + map[ i ] = i; + continue; + } + + // to determine if the index has moved, we must see if every leader + // is unchanged at a different index from the original leader values + // of this index + var inew = this._trackDownLeaders( i ); + if ( inew === -1 ) + { + // the index was added or removed; it maps from nothing. + map[ i ] = undefined; + continue; + } + + // it does not matter if there were changes---we found the index + // that we map to + map[ i ] = inew; + } + + return map; + } +} ); + diff --git a/src/bucket/diff/StdBucketDiff.js b/src/bucket/diff/StdBucketDiff.js new file mode 100644 index 0000000..9272cab --- /dev/null +++ b/src/bucket/diff/StdBucketDiff.js @@ -0,0 +1,92 @@ +/** + * Performs a standard value-by-value diff between buckets + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + * + * This can be called a "dumb diff". + */ + +var Class = require( 'easejs' ).Class, + ArrayDiff = require( '../../util/ArrayDiff' ), + BucketDiff = require( './BucketDiff' ), + Context = require( './BucketDiffContext' ); + + +module.exports = Class( 'StdBucketDiff' ) + .implement( BucketDiff ) + .extend( +{ + /** + * Constructor for diff result + */ + 'private _Result': null, + + /** + * Responsible for a shallow diff of array values + * @type {ArrayDiff} + */ + 'private _arrdiff': null, + + + __construct: function( arrdiff, Result ) + { + if ( !( Class.isA( ArrayDiff, arrdiff ) ) ) + { + throw TypeError( "Expecting ArrayDiff; received " + arrdiff ); + } + + this._arrdiff = arrdiff; + this._Result = Result; + }, + + + /** + * Perform a diff given a diff context + * + * @param {BucketDiffContext} context diff context + * + * @return {BucketDiffResult} result of diff + */ + 'public diff': function( context ) + { + if ( !( Class.isA( Context, context ) ) ) + { + throw TypeError( "Expected BucketDiffContext" ); + } + + var changed = {}; + + var arrdiff = this._arrdiff; + context.forEachField( function( field, a, b ) + { + var diff = arrdiff.diff( a, b ), + i = ( a.length > b.length ) ? a.length : b.length, + changes = []; + + while ( i-- ) + { + changes[ i ] = ( diff[ i ] !== undefined ); + } + + changed[ field ] = changes; + } ); + + return this._Result( context, changed ); + } +} ); + diff --git a/src/bucket/diff/StdBucketDiffContext.js b/src/bucket/diff/StdBucketDiffContext.js new file mode 100644 index 0000000..16c017b --- /dev/null +++ b/src/bucket/diff/StdBucketDiffContext.js @@ -0,0 +1,136 @@ +/** + * Describes a context with which diffing may take place between two buckets + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + Bucket = require( '../Bucket' ), + BucketDiffContext = require( './BucketDiffContext' ); + + +/** + * Context with which diffing may take place between two buckets + * + * This does not describe the diffing algorithm---it merely desribes the + * context for performing the diff. + */ +module.exports = Class( 'StdBucketDiffContext' ) + .implement( BucketDiffContext ) + .extend( +{ + /** + * The bucket representing the current state to diff + * @type {Bucket} + */ + 'private _head': null, + + /** + * The bucket to diff against + * @type {Bucket} + */ + 'private _prev': null, + + + /** + * Initialize context with two buckets to be compared + * + * @param {Bucket} head current bucket state to diff + * @param {Bucket} prev prior bucket state to diff against + */ + __construct: function( head, prev ) + { + if ( !( Class.isA( Bucket, head ) ) ) + { + throw TypeError( "Expected head of type Bucket; given " + head ); + } + else if ( !( Class.isA( Bucket, prev ) ) ) + { + throw TypeError( "Expected prev of type Bucket; given " + prev ); + } + + this._head = head; + this._prev = prev; + }, + + + /** + * Invoke continuation for each unique field name in either bucket (the + * union of bucket field names) + * + * The continuation will be passed as arguments, respectively, the field + * name, the head value, and the prev value. + * + * @param {function( string, Array, Array )} callback continuation + * + * @return {StdBucketDiffContext} self + */ + 'public forEachField': function( callback ) + { + var _self = this, + headf = {}, + slice = Array.prototype.slice; + + // first, go through each of the fields in the head and compare the + // values in each + this._head.each( function( data, field ) + { + callback( field, + slice.call( data ), + slice.call( _self._prev.getDataByName( field ), 0 ) + ); + + // mark this field as having been recognized + headf[ field ] = true; + } ); + + // finally, report back any fields in prev that are not in head + this._prev.each( function( data, field ) + { + if ( headf[ field ] ) + { + return; + } + + callback( field, + slice.call( _self._head.getDataByName( field ), 0 ), + slice.call( data ) + ); + } ); + }, + + + /** + * Return an array [ current, prev ] of the values associated with the given + * field + * + * @type {string} field field name + * + * @return {Array.} field values for head and prev + */ + 'public getFieldValues': function( field ) + { + var slice = Array.prototype.slice; + + return [ + slice.call( this._head.getDataByName( field ), 0 ), + slice.call( this._prev.getDataByName( field ), 0 ), + ]; + } +} ); + diff --git a/src/bucket/diff/StdBucketDiffResult.js b/src/bucket/diff/StdBucketDiffResult.js new file mode 100644 index 0000000..77d536d --- /dev/null +++ b/src/bucket/diff/StdBucketDiffResult.js @@ -0,0 +1,120 @@ +/** + * Represents the result of performing a standard ("dumb") diff on bucket data + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + BucketDiffContext = require( './BucketDiffContext' ); + BucketDiffResult = require( './BucketDiffResult' ); + + +/** + * Result of performing a standard ("dumb") diff on bucket data + * + * This class handles the heavy lifting with regards to requesting + * information about a diff---the actual diff algorithm need only provide a + * list of the changes, which an instance of this class may either return + * directly or retrieve the field data from the provided context. + */ +module.exports = Class( 'StdBucketDiffResult' ) + .implement( BucketDiffResult ) + .extend( +{ + /** + * Context used to produce the diff + * @type {BucketDiffContext} + */ + 'private _context': null, + + /** + * List of flags describing field changes + * @type {Object} + */ + 'private _changes': null, + + + __construct: function( context, changes ) + { + if ( !( Class.isA( BucketDiffContext, context ) ) ) + { + throw TypeError( + "Expected BucketDiffContext; received " + context + ); + } + + this._context = context; + this._changes = changes; + }, + + + /** + * Describes what fields have changed using boolean flags; does not include + * unchanged fields + * + * As an example: + * { field: [ false, true, false ] } + * + * @return {Object} hash of arrays of boolean flags representing changes + */ + 'public describeChanged': function() + { + return this._changes; + }, + + + /** + * Describes changes in values by listing either undefined if no change or + * an array containing, respectively, the current and previous values for + * each index + * + * As an example: + * { field: [ undefined, [ 'current', 'prev' ], undefined ] } + * + * @return {Object} value changes + */ + 'public describeChangedValues': function() + { + var ret = {}; + + for ( var field in this._changes ) + { + var change = this._changes[ field ], + i = change.length, + values = this._context.getFieldValues( field ); + + ret[ field ] = []; + while ( i-- ) + { + if ( change[ i ] !== true ) + { + ret[ field ][ i ] = undefined; + continue; + } + + // return both the current and previous values respectively + ret[ field ][ i ] = [ + values[ 0 ][ i ], + values[ 1 ][ i ], + ]; + } + } + + return ret; + } +} ); diff --git a/src/calc/Calc.js b/src/calc/Calc.js new file mode 100644 index 0000000..ba1fddd --- /dev/null +++ b/src/calc/Calc.js @@ -0,0 +1,654 @@ +/** + * Contains calculation methods + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +function _each( data, value, callback ) +{ + var data_len = data.length, + result = [], + cur_val = null; + + for ( var i = 0; i < data_len; i++ ) + { + cur_val = ( value[ i ] !== undefined ) ? value[ i ] : cur_val; + result.push( callback( data[ i ], cur_val, i ) ); + } + + return result; +} + +exports.append = function( data, value ) +{ + return _each( data, value, function( arr1, arr2 ) + { + if( !( arr1 instanceof Array ) ) + { + arr1 = [ arr1 ]; + } + + return arr1.concat( arr2 ); + }); +}; + +exports.join = function( data, value ) +{ + return _each( data, value, function( arr, delimiter ) + { + return arr.join( delimiter ); + }); +}; + +exports[ 'if' ] = function( data, value ) +{ + var result = []; + + for ( var i = 0; i < data.length; i++ ) + { + result.push( ( ( data[ i ] === "1" ) ? value[ i ] : '' ) ); + } + + return result; +}; + +exports.sum = function( data ) +{ + var data_len = data.length; + + // calculate and return the sum + for ( var i = 0, sum = 0; i < data_len; sum += +data[i++] ); + return [ sum ]; +}; + + +/** + * Return an object containing a range of elements + * + * @param data The first value of the sequence of numbers + * @param value The ending value of the sequence + * + * @example range( 1, 3 ) will yield [ "1", "2", "3" ] + * range( 5, 2 ) will yield [ "5", "4", "3", "2" ] + */ +exports.range = function( data, value ) +{ + var result = [], + range_from = data[ 0 ], + range_to = value[ 0 ], + low = Math.min( range_from, range_to ), + high = Math.max( range_from, range_to ), + index = 0; + + for ( var i = low; i < high; i++ ) + { + result[ index ] = ( ''+i ); + index++; + } + + return ( range_from < range_to ) + ? result + : result.reverse(); +}; + + +exports.length = function( data ) +{ + var result = [], + item = '', + i = 0; + + while ( true ) + { + item = data[ i++ ]; + if ( ( item === null ) || ( item === undefined ) ) + { + break; + } + + result.push( item.length ); + } + + return result; +}; + + +/** + * Return the number of elements in the set that optionally match the given + * value + */ +exports.count = function( data, value ) +{ + var len = data.length; + + if ( !( value ) || value.length === 0 ) + { + return [ ''+( len ) ]; + } + + var count = 0; + for ( var i = 0; i < len; i++ ) + { + // intentional lazy cmp + if ( data[ i ] == value[ 0 ] ) + { + count++; + } + } + + return [ ''+( count ) ]; +}; + + +exports.countNonEmpty = function( data, value ) +{ + var count = 0; + for ( var i in data ) + { + if ( data[ i ] !== '' ) + { + count++; + } + } + + return [ ''+count ]; +}; + + +exports.divide = function( data, value ) +{ + return _each( data, value, function( val1, val2 ) + { + return ( +val1 / +val2 ); + } ); +}; + + +exports.multiply = function( data, value ) +{ + return _each( data, value, function( val1, val2 ) + { + return ( +val1 * +val2 ); + } ); +}; + + +exports.add = function( data, value ) +{ + return _each( data, value, function( val1, val2 ) + { + return ( +val1 + +val2 ); + } ); +}; + + +exports.subtract = function( data, value ) +{ + return _each( data, value, function( val1, val2 ) + { + return ( +val1 - +val2 ); + } ); +}; + + +exports.date = function() +{ + var now = new Date(); + + // return in the YYYY-MM-DD format, since that's what our fields are + // formatted as + return [ + now.getFullYear() + '-' + + ( now.getMonth() + 1 ) + '-' + + now.getDate() + ]; +}; + + +exports.userAgent = function() +{ + return ( typeof window !== 'undefined' ) + ? [ window.navigator.userAgent ] + : [ '' ]; +}; + + +exports.relativeDate = function( data, value ) +{ + return _each( data, value, function( curdate, format ) + { + var type = format.substr( -1 ), + tval = format.substr( 0, format.length - 1 ), + ms = 0; + + var now = ( curdate ) ? new Date( curdate ) : new Date(), + now_year = now.getUTCFullYear(), + now_month = now.getUTCMonth() + 1, + now_day = now.getUTCDate(), + date_new = null; + + switch ( type ) + { + // years + case 'y': + date_new = new Date( + ( now_year + +tval ) + '/' + now_month + '/' + now_day + ); + break; + + // months + case 'm': + date_new = new Date( + now_year + '/' + ( now_month + +tval ) + '/' + now_day + ); + break; + + // days + case 'd': + date_new = new Date( + now_year + '/' + now_month + '/' + ( now_day + +tval ) + ); + break; + + // seconds + case 's': + date_new = new Date( now.getTime() + ( tval * 1000 ) ); + break; + + default: + return ''; + } + + // return in the YYYY-MM-DD format, since that's what our fields are + // formatted as + return date_new.getFullYear() + '-' + + ( date_new.getMonth() + 1 ) + '-' + + date_new.getDate(); + } ); +}; + + +exports.copy = function( data ) +{ + return data; +}; + + +exports.month = function( data ) +{ + // if no reference was provided, return the current month + if ( data.length === 0 ) + { + return [ new Date().getMonth() + 1 ]; + } + + // otherwise, get the month from each of the provided dates + var len = data.length, + result = []; + + for ( var i = 0; i < len; i++ ) + { + result.push( + new Date( + Date.parse( data[i].replace( /-/g, '/' ) ) + ).getMonth() + 1 + ); + } + + return result; +}; + + +exports.year = function( data ) +{ + // if no reference was provided, return the current year + if ( data.length === 0 ) + { + return [ new Date().getFullYear() ]; + } + + // otherwise, get the year from each of the provided dates + var len = data.length, + result = []; + + for ( var i = 0; i < len; i++ ) + { + if ( data[ i ] === '' ) + { + result.push( new Date().getFullYear() ); + continue; + } + + result.push( + new Date( + Date.parse( data[i].replace( /-/g, '/' ) ) + ).getFullYear() + ); + } + + return result; +}; + + +// tests if a value(s) exists in data +exports.isIn = function( data, value ) +{ + var all_values_found = false; + + for ( var v_key = 0; v_key < value.length; v_key++ ) + { + var value_found = false; + + for ( var d_key = 0; d_key < data.length; d_key++ ) + { + // check to see if this value has a match + if ( +value[ v_key ] === +data[ d_key ] ) + { + value_found = true; + } + } + + // all values sent in must have a match for isIn to be true + all_values_found = ( value_found ) + ? true + : all_values_found; + } + + return ( all_values_found === true ) + ? [ '1' ] + : [ '0' ]; +}; + + +exports.identical = function( data, value ) +{ + // true if all values in data are the same + var len = data.length, + cmp_val = data[ 0 ]; + + for ( var i = 0; i < len; i++ ) + { + // null indicates that the location is marked for deletion + if ( data[ i ] !== cmp_val && data[ i ] !== null ) + { + // something doesn't match + return [ '0' ]; + } + } + + return [ '1' ]; +}; + + +exports.dateDiff = function( data, value ) +{ + var data_len = data.length, + result = [], + cmp_ts = 0; + + for ( var i = 0; i < data_len; i++ ) + { + // use the last available value for comparison + if ( value[i] ) + { + // convert it to a timestamp + cmp_ts = ( Date.parse( value[i].replace( /-/g, '/' ) ) / 1000 ); + } + + // convert data to timestamp + var data_ts = ( Date.parse( data[i].replace( /-/g, '/' ) ) / 1000 ); + + // if the given date is higher than the comparison value, then a + // positive number will result, otherwise a negative + var diff = data_ts - cmp_ts; + result.push( isNaN( diff ) ? 0 : diff ); + } + + return result; +}; + + +exports.split = function( data, value ) +{ + return _each( data, value, function( item, delim ) + { + // split using the provided delimiter + return ( ( ''+( item ) ).split( delim ) ); + }); +}; + + +exports.keyValue = function( data, value ) +{ + return _each( data, value, function( item, index ) + { + if ( !( item instanceof Array ) ) + { + return null; + } + + // return requested index + return item[ index ]; + }); +}; + + +exports.match = function( data, value ) +{ + return _each( data, value, function( item, regex ) + { + return ( ( ''+item ).match( new RegExp( regex ) ) ); + }); +}; + + +/** + * Return min numeric value in data set + */ +exports.min = function( data ) +{ + var min_val = +data[ 0 ], + i = data.length; + + while ( i-- ) + { + var cur_val = +data[ i ]; + min_val = ( min_val < cur_val ) ? min_val : cur_val; + } + + return [ ''+( min_val ) ]; +}; + + +/** + * Finds min numeric value in data set and + * returns its position + */ +exports.minPos = function( data ) +{ + var min_val = +data[ 0 ], + min_pos = 0; + + for ( var i = 0; i < data.length; i++ ) + { + var cur_val = +data[ i ]; + if ( cur_val < min_val ) + { + min_val = cur_val; + min_pos = i; + } + } + + return [ ''+min_pos ]; +}; + + +/** + * Finds max numeric value in data set and + * returns its position + */ +exports.maxPos = function( data ) +{ + var max_val = +data[ 0 ], + max_pos = 0; + + for ( var i = 0; i < data.length; i++ ) + { + var cur_val = +data[ i ]; + if ( cur_val > max_val ) + { + max_val = cur_val; + max_pos = i; + } + } + + return [ ''+max_pos ]; +}; + + +/** + * Return max numeric value in data set + */ +exports.max = function( data ) +{ + var max_val = +data[ 0 ], + i = data.length, + result = []; + + while ( i-- ) + { + var cur_val = +data[ i ]; + max_val = ( max_val > cur_val ) ? max_val : cur_val; + } + + return [ ''+( max_val ) ]; +}; + + +/** + * Return array of each unique element in the data set + */ +exports.uniq = function( data ) +{ + var found = {}, + uniq = [], + i = data.length; + + while ( i-- ) + { + var val = data[ i ]; + + if ( found[ val ] ) + { + continue; + } + + found[ val ] = true; + uniq.push( val ); + } + + return uniq; +}; + + +/** + * Join array of elements with a delimiter + * + * @param data The array of values to join + * @param value The delimiter + * + * @example implode( [ "foo", "bar" ], "," ) will yield [ "foo,bar" ] + */ +exports.implode = function( data, value ) +{ + return [ ( data.join( value || '' ) ) ]; +}; + + +/** + * Given a regular expression, returns '1' for positive test and '0' for + * negative. + */ +exports.test = function( data, value ) +{ + return _each( data, value, function( item, regex ) + { + // return '1' for true, '0' for false + return ''+( +( ( new RegExp( regex ) ).test( item ) ) ); + } ); +}; + + +/** + * Opposite of test + */ +exports.testNot = function( data, value ) +{ + return _each( data, value, function( item, regex ) + { + // return '1' for true, '0' for false + return ''+( +( !( ( new RegExp( regex ) ).test( item ) ) ) ); + } ); +}; + + +/** + * N-based index + */ +exports.position = function( data, value ) +{ + return _each( data, value, function( item, offset, i ) + { + return +i + ( +offset || 0 ); + } ); +}; + + +/** + * Given a set of indexes, return array of values + */ +exports.value = function( data, indexes ) +{ + var len = indexes.length, + values = [], + key = 0; + + for ( var i = 0; i < len; i++ ) + { + key = indexes[ i ]; + + if ( data[ key ] !== null ) + { + // found a value + values.push( data[ key ] ); + } + else + { + values.push( null ); + } + } + + return values; +}; + + +exports[ 'void' ] = function() +{ + return []; +}; + diff --git a/src/client/Client.js b/src/client/Client.js index 9338860..70e3525 100644 --- a/src/client/Client.js +++ b/src/client/Client.js @@ -22,7 +22,7 @@ const Class = require( 'easejs' ).Class; const EventEmitter = require( 'events' ).EventEmitter; const DomFieldNotFoundError = require( '../ui/field/DomFieldNotFoundError' ); -const UnknownEventError = require( '../event/UnknownEventError' ); +const UnknownEventError = require( './event/UnknownEventError' ); /** @@ -117,7 +117,7 @@ module.exports = Class( 'Client' ) /** * Current quote - * @type {QuoteClient} + * @type {ClientQuote} */ 'private _quote': null, @@ -2708,7 +2708,7 @@ module.exports = Class( 'Client' ) /** * Returns the current quote * - * @return {QuoteClient} + * @return {ClientQuote} */ getQuote: function() { diff --git a/src/client/ClientDataProxy.js b/src/client/ClientDataProxy.js new file mode 100644 index 0000000..ac64dbf --- /dev/null +++ b/src/client/ClientDataProxy.js @@ -0,0 +1,114 @@ +/** + * Contains ClientDataProxy class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + * + * @todo This has hard-coded references to the ``quote server'' and LoVullo + */ + +var Class = require( 'easejs' ).Class, + JqueryHttpDataProxy = require( './proxy/JqueryHttpDataProxy' ); + + +/** + * Handles additional processing before passing HTTP request off to the + * requester + */ +module.exports = Class( 'ClientDataProxy' ) + .extend( JqueryHttpDataProxy, +{ + /** + * Log error (if any) and process data before returning the parent's GET + * request + * + * @param {string} url request URL + * @param {function( Object, * }) callback request callback (success or + * failure) + * + * @return {ClientDataProxy} self + */ + 'protected override getData': function( url, callback ) + { + var _self = this; + + this.__super( url, function( data, err ) + { + if ( err ) + { + console.error( 'XHR GET Error: (%s) %s', url, err ); + } + + callback( _self._processData( data ), err ); + } ); + + return this; + }, + + + /** + * Log error (if any) and process data before returning the parent's POST + * request + * + * @param {string} url request URL + * @param {Object} data data to post to server + * @param {function( Object, * }) callback request callback (success or + * failure) + * + * @return {ClientDataProxy} self + */ + 'protected override postData': function( url, data, callback ) + { + var _self = this; + + this.__super( url, data, function( data, err ) + { + if ( err ) + { + console.error( 'XHR POST Error: (%s) %s', url, err ); + } + + callback( _self._processData( data ), err ); + } ); + }, + + + /** + * Basic data post-processing + * + * If the response is empty/doesn't make sense, this will generate a + * generic error. + * + * @param {Object} data response data + * + * @return processed data + */ + 'private _processData': function( data ) + { + // if the data is null, then we didn't actually get a response from the + // server - rather, the response was empty (that's bad!) + data = data || { + hasError: true, + content: 'There was a problem communicating with the quote ' + + 'server. If you continue to receive this message, please ' + + 'contact LoVullo Associates for assistance.' + }; + + return data; + } +} ); + diff --git a/src/client/ClientDependencyFactory.js b/src/client/ClientDependencyFactory.js index 178cdc5..c6958da 100644 --- a/src/client/ClientDependencyFactory.js +++ b/src/client/ClientDependencyFactory.js @@ -46,47 +46,45 @@ var Step = require( '../step/Step' ), // TODO: delete me AccordionGroupUi = require( 'program/ui/AccordionGroupUi' ), - Ui = require( 'program/ui/Ui' ), - UiStyler = require( 'program/ui/UiStyler' ), - UiNotifyBar = require( 'program/ui/UiNotifyBar' ), - UiNavBar = require( 'program/ui/UiNavBar' ), - UiDialog = require( 'program/ui/dialog/UiDialog' ), - JqueryDialog = require( 'program/ui/dialog/JqueryDialog' ), + Ui = require( '../ui/Ui' ), + UiStyler = require( '../ui/UiStyler' ), + UiNotifyBar = require( '../ui/UiNotifyBar' ), + UiNavBar = require( '../ui/nav/UiNavBar' ), + UiDialog = require( '../ui/dialog/UiDialog' ), + JqueryDialog = require( '../ui/dialog/JqueryDialog' ), - BaseQuote = require( 'program/quote/BaseQuote' ), - QuoteClient = require( 'program/QuoteClient' ), + BaseQuote = require( '../quote/BaseQuote' ), + ClientQuote = require( './quote/ClientQuote' ), QuoteDataBucket = require( '../bucket/QuoteDataBucket' ), StagingBucket = require( '../bucket/StagingBucket' ), StagingBucketAutoDiscard = require( '../bucket/StagingBucketAutoDiscard' ), DelayedStagingBucket = require( '../bucket/DelayedStagingBucket' ), + standardValidator = require( '../validate/standardBucketValidator' ), DataValidator = require( '../validate/DataValidator' ), ValidStateMonitor = require( '../validate/ValidStateMonitor' ), - standardValidator = require( 'program/standardBucketValidator' ), Failure = require( '../validate/Failure' ), Field = require( '../field/BucketField' ), - XhttpQuoteTransport = require( 'program/transport/XhttpQuoteTransport' ), - JqueryHttpDataProxy = require( 'program/proxy/JqueryHttpDataProxy' ), + XhttpQuoteTransport = require( './transport/XhttpQuoteTransport' ), + JqueryHttpDataProxy = require( './proxy/JqueryHttpDataProxy' ), XhttpQuoteStagingTransport = - require( 'program/transport/XhttpQuoteStagingTransport' ), + require( './transport/XhttpQuoteStagingTransport' ), - Nav = require( 'program/Nav' ), - HashNav = require( 'program/ui/HashNav' ), + Nav = require( './nav//Nav' ), + HashNav = require( '../ui/nav/HashNav' ), - ClientDataProxy = require( 'program/ClientDataProxy' ), + ClientDataProxy = require( './ClientDataProxy' ), ElementStyler = require( '../ui/ElementStyler' ), - FormErrorBox = require( 'program/ui/FormErrorBox' ), - NavStyler = require( 'program/ui/NavStyler' ), - Sidebar = require( 'program/ui/Sidebar' ), + FormErrorBox = require( '../ui/sidebar/FormErrorBox' ), + NavStyler = require( '../ui/nav/NavStyler' ), + Sidebar = require( '../ui/sidebar/Sidebar' ), - FieldClassMatcher = require( 'program/FieldClassMatcher' ), - DataApiFactory = require( 'program/api/DataApiFactory' ), + FieldClassMatcher = require( '../field/FieldClassMatcher' ), + DataApiFactory = require( '../dapi/DataApiFactory' ), DataApiManager = require( '../dapi/DataApiManager' ), - RatingWorksheet = require( 'program/ui/RatingWorksheet' ), - RootDomContext = require( '../ui/context/RootDomContext' ), DomFieldFactory = require( '../ui/field/DomFieldFactory' ), StepErrorStyler = require( '../ui/styler/StepErrorStyler' ), @@ -96,14 +94,14 @@ var Step = require( '../step/Step' ), NaFieldStyler = require( '../ui/styler/NaFieldStyler' ), NaFieldStylerAnimation = require( '../ui/styler/NaFieldStylerAnimation' ), - DelegateEventHandler = require( '../event/DelegateEventHandler' ), + DelegateEventHandler = require( './event/DelegateEventHandler' ), diffStore = require( 'liza/system/client' ).data.diffStore, Class = require( 'easejs' ).Class; -var liza_event = require( '../event' ); +var liza_event = require( './event' ); function requireh( name ) @@ -187,7 +185,7 @@ module.exports = Class( 'ClientDependencyFactory', { var _self = this; - return QuoteClient( + return ClientQuote( BaseQuote( quote_id, this.createDataBucket() ), data, function( bucket ) diff --git a/src/client/bucket/SimpleBucketListener.js b/src/client/bucket/SimpleBucketListener.js new file mode 100644 index 0000000..1231560 --- /dev/null +++ b/src/client/bucket/SimpleBucketListener.js @@ -0,0 +1,305 @@ +/** + * Contains SimpleBucketListener class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + Client = require( '../Client' ); + + +/** + * Provides a simple API for listening for bucket changes on a given Client + */ +module.exports = Class( 'SimpleBucketListener', +{ + /** + * Contains callbacks to trigger on update event + * @type {Object.>)>} + */ + 'private _updateEvents': {}, + + /** + * Contains callbacks to trigger on updateEach event + * @type {Object.)>} + */ + 'private _updateEachEvents': {}, + + /** + * Quick-n-easy reference to the current step id in context of the given + * Client instance + * + * This is updated automatically by monitoring the Client's quoteChange + * event; a fast alternative to querying the quote for each event. + * + * @type {number} + */ + 'private _curStepId': 0, + + + /** + * Initializes listener with the provided Client + * + * @param {Client} client client to monitor + */ + __construct: function( client ) + { + if ( !( Class.isA( Client, client ) ) ) + { + throw TypeError( 'Expected Client object; received ' + client ); + } + + this._hookQuoteChange( client ); + + // in the event that we have a quote available, hook it immediately + var quote; + if ( quote = client.getQuote() ) + { + this._hookQuote( quote ); + } + }, + + + /** + * Hook client's quote change event to properly hook the new bucket on quote + * change + * + * @param {Client} client client to hook + * + * @return {undefined} + */ + 'private _hookQuoteChange': function( client ) + { + var _self = this; + + client.on( 'quoteChange', function() + { + var quote_client = client.getQuote(); + quote_client && _self._hookQuote( quote_client ); + } ); + }, + + + /** + * Hook quote data events + * + * @param {ClientQuote} quote_client quote client to hook + * + * @return {undefined} + */ + 'private _hookQuote': function( quote_client ) + { + var _self = this; + + this._quote = quote_client; + + quote_client + .on( 'dataUpdate', function( data ) + { + _self._handleUpdate( data ); + } ) + .on( 'stepChange', function( step_id ) + { + _self._curStepId = step_id; + } ); + }, + + + /** + * Handle a data update event from the quote client + * + * @param {Object} data data diff + * + * @return {undefined} + */ + 'private _handleUpdate': function( data ) + { + var hook; + + // we're not going to perform a hasOwnProperty() check because we are to + // assume that the data provided does not have any other enumerable + // members + for ( id in data ) + { + this._tryUpdateHook( id, data ); + this._tryUpdateEachHook( id, data ); + } + }, + + + /** + * Calls any update hooks for the given id + * + * @param {string} id hook id + * @param {Object.>} data update diff + * + * @return {undefined} + */ + 'private _tryUpdateHook': function( id, data ) + { + var hooks; + + if ( hooks = this._updateEvents[ id ] ) + { + this._callHooks( hooks, id, [ data[ id ] ] ); + } + }, + + + /** + * Calls any updateEach hooks for the given id + * + * @param {string} id hook id + * @param {Object.>} data update diff + * + * @return {undefined} + */ + 'private _tryUpdateEachHook': function( id, data ) + { + var hooks; + + if ( ( hooks = this._updateEachEvents[ id ] ) === undefined ) + { + return; + } + + var id_data = data[ id ], + i = id_data.length, + val = null; + + while ( i-- ) + { + val = id_data[ i ]; + + // ignore unchanged indexes or deleted + if ( ( val === undefined ) || ( val === null ) ) + { + continue; + } + + this._callHooks( hooks, id, [ i, val ] ); + } + }, + + + /** + * Calls a set of hooks, so long as the options are valid + * + * @param {Array.} hook hook to call + * @param {string} id id of hook (key) + * @param {Array} args arguments to apply + * + * @return {undefined} + */ + 'private _callHooks': function( hooks, id, args ) + { + var i = hooks.length; + + while ( i-- ) + { + var hook = hooks[ i ]; + + if ( hook.step && ( hook.step !== this._curStepId ) ) + { + continue; + } + + hook.apply( { id: id }, args ); + } + }, + + + /** + * Attach the given callback(s) + * + * @param {Object} dest callback store + * @param {string|Array.} id id or array of ids to monitor for + * @param {Function} callback callback to store + * + * @return {undefined} + */ + 'private _attachCallbacks': function( dest, id, opts, callback ) + { + // options are optional + if ( callback === undefined ) + { + callback = opts; + } + + if ( typeof callback !== 'function' ) + { + throw TypeError( 'Expected callback; given ' + callback ); + } + + if ( opts.step ) + { + callback.step = opts.step; + } + + if ( typeof id === 'string' ) + { + ( dest[ id ] = dest[ id ] || [] ).push( callback ); + } + else + { + var i = id.length; + while ( i-- ) + { + ( dest[ id[ i ] ] = dest[ id[ i ] ] || [] ).push( callback ); + } + } + }, + + + /** + * Trigger callback when the given id(s) is/are updated + * + * @param {string|Array.} id id or array of ids to monitor + * + * @param {{step}|function(Object.>)} opts options or callback + * + * @param {function(Object.>)=} callback to call on id change, + * if opts provided + * + * @return {SimpleBucketListener} self + */ + 'public onUpdate': function( id, opts, callback ) + { + this._attachCallbacks( this._updateEvents, id, opts, callback ); + return this; + }, + + + /** + * Trigger callback when the given id(s) is/are updated + * + * @param {string|Array.} id id or array of ids to monitor + * + * @param {{step}|function(Array.)} opts options or callback + * @param {function(Array.)=} callback to call on id change, + * if opts provided + * + * @return {SimpleBucketListener} self + */ + 'public onUpdateEach': function( id, opts, callback ) + { + this._attachCallbacks( this._updateEachEvents, id, opts, callback ); + return this; + } +} ); + diff --git a/src/client/debug/CalcClientDebugTab.js b/src/client/debug/CalcClientDebugTab.js index db90aa0..b1c4bee 100644 --- a/src/client/debug/CalcClientDebugTab.js +++ b/src/client/debug/CalcClientDebugTab.js @@ -19,12 +19,9 @@ * along with this program. If not, see . */ -var Class = require( 'easejs' ).Class, - +var Class = require( 'easejs' ).Class, ClientDebugTab = require( './ClientDebugTab' ), - - calc = require( 'program/Calc' ) -; + calc = require( '../../calc/Calc' ); /** diff --git a/src/client/event/ClientEventHandler.js b/src/client/event/ClientEventHandler.js new file mode 100644 index 0000000..80bff39 --- /dev/null +++ b/src/client/event/ClientEventHandler.js @@ -0,0 +1,46 @@ +/** + * ClientEventHandler interface + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +var Interface = require( 'easejs' ).Interface, + EventHandler = require( './EventHandler' ); + + +module.exports = Interface + .extend( EventHandler, +{ + /** + * Handle an event of the given type + * + * An exception should be thrown if the event cannot be handled. + * + * The handler should always return itself; if a return value is needed to + * the caller, then a callback should be provided as an argument to the + * handler. + * + * @param {string} type event id + * + * @param {function(*,Object)} continuation (error, data) + * + * @return {EventHandler} self + */ + 'public handle': [ 'type', 'callback' /*, ... */ ] +} ); + diff --git a/src/event/Cvv2DialogEventHandler.js b/src/client/event/Cvv2DialogEventHandler.js similarity index 100% rename from src/event/Cvv2DialogEventHandler.js rename to src/client/event/Cvv2DialogEventHandler.js diff --git a/src/event/DelegateEventHandler.js b/src/client/event/DelegateEventHandler.js similarity index 100% rename from src/event/DelegateEventHandler.js rename to src/client/event/DelegateEventHandler.js diff --git a/src/event/EventHandler.js b/src/client/event/EventHandler.js similarity index 100% rename from src/event/EventHandler.js rename to src/client/event/EventHandler.js diff --git a/src/event/FieldVisibilityEventHandler.js b/src/client/event/FieldVisibilityEventHandler.js similarity index 100% rename from src/event/FieldVisibilityEventHandler.js rename to src/client/event/FieldVisibilityEventHandler.js diff --git a/src/event/IndvRateEventHandler.js b/src/client/event/IndvRateEventHandler.js similarity index 100% rename from src/event/IndvRateEventHandler.js rename to src/client/event/IndvRateEventHandler.js diff --git a/src/event/KickbackEventHandler.js b/src/client/event/KickbackEventHandler.js similarity index 100% rename from src/event/KickbackEventHandler.js rename to src/client/event/KickbackEventHandler.js diff --git a/src/event/RateEventHandler.js b/src/client/event/RateEventHandler.js similarity index 100% rename from src/event/RateEventHandler.js rename to src/client/event/RateEventHandler.js diff --git a/src/event/StatusEventHandler.js b/src/client/event/StatusEventHandler.js similarity index 100% rename from src/event/StatusEventHandler.js rename to src/client/event/StatusEventHandler.js diff --git a/src/event/UnknownEventError.js b/src/client/event/UnknownEventError.js similarity index 94% rename from src/event/UnknownEventError.js rename to src/client/event/UnknownEventError.js index 31e5537..30bbe0d 100644 --- a/src/event/UnknownEventError.js +++ b/src/client/event/UnknownEventError.js @@ -22,7 +22,7 @@ const Class = require( 'easejs' ).Class; -module.exports = Class( 'UnknownEventHandler' ) +module.exports = Class( 'UnknownEventError' ) .extend( TypeError, { } ); diff --git a/src/client/nav/Nav.js b/src/client/nav/Nav.js new file mode 100644 index 0000000..a81314a --- /dev/null +++ b/src/client/nav/Nav.js @@ -0,0 +1,517 @@ +/** + * Contains program Nav class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + EventEmitter = require( 'events' ).EventEmitter; + + +/** + * Handles navigation logic + * + * The step_builder function should accept two arguments: the step id and a + * callback to be executed once the step has reached its ready state. + */ +module.exports = Class( 'Nav' ) + .extend( EventEmitter, +{ + /** + * When user attempts to navigate away from the page + * @type {string} + */ + 'const EVENT_UNLOAD': 'unload', + + /** + * Attempt to navigate to another step (allows preventing navigation) + * @type {string} + */ + 'const EVENT_STEP_PRE_CHANGE': 'preStepChange', + + /** + * Navigation to another step has completed + * @type {string} + */ + 'const EVENT_STEP_CHANGE': 'stepChange', + + /** + * Quote id has been changed + * @type {string} + */ + 'const EVENT_QUOTE_ID_CHANGE': 'quoteIdChange', + + + /** + * Quote id to use in navigation + * + * -1 by default to ensure that the default quote id of 0 will trigger an id + * change + * + * @type {number} + */ + 'private _quoteId': -1, + + /** + * Stores the last step id to determine if the step has changed + * @type {number} + */ + 'private _lastStepId': 0, + + /** + * Id of the max visited step + * @type {number} + */ + 'private _topVisitedStepId': 0, + + /** + * Number of steps + * @type {number} + */ + 'private _stepCount': 0, + + /** + * Contains information regarding the steps + * @type {Object.>} + */ + 'private _stepData': {}, + + /** + * The first step id as far as navigation is concerned + * @type {number} + */ + 'private _firstStepId': 1, + + /** + * Minimum step id permitted to visit, unless 0 + * @type {number} + */ + 'private _minStepId': 0, + + + /** + * Initializes navigation + * + * This will initialize the navigation bar hooks to permit navigation and + * hook to the address, allowing back/forward buttons to work properly, + * bookmarks, and properly navigate if the hash in the URL is modified. + * + * The URL hash will then be altered to reflect the default, if one was not + * already provided. + * + * @return Nav self to allow for method chaining + */ + 'public __construct': function( step_data ) + { + this._initUnloadHook(); + + this._stepData = step_data; + this.setStepCount( step_data.length - 1 ); + + return this; + }, + + + 'public setFirstStepId': function( id ) + { + this._firstStepId = +id; + }, + + + /** + * Binds to the 'beforeunload' event + * + * This allows a message to be displayed to the user when they attempt to + * navigate away from the page. + * + * @return void + */ + _initUnloadHook: function() + { + var _self = this, + _event = this.__self.$('EVENT_UNLOAD'); + + $( window ).bind( 'beforeunload.program', function( e ) + { + var event = { returnValue: undefined }; + _self.emit( _event, event ); + + // IE and Firefox + if ( e ) + { + e.returnValue = event.returnValue; + } + + // Safari + return event.returnValue; + }); + }, + + + /** + * Sets the number of available steps + * + * @param {number} count step count + * + * @return {Nav} self + */ + setStepCount: function( count ) + { + this._stepCount = +count; + return this; + }, + + + /** + * Navigates to a step via the step id + * + * Step navigation is handled by the address hook. Therefore, we simply + * modify the hash tag. + * + * @param {number} step_id id of step to navigate to + * @param {boolean} force do not check if next step is valid + * + * @return Nav self to allow for method chaining + */ + navigateToStep: function( step_id, force, cur_check ) + { + step_id = +step_id; + force = !!force; + cur_check = ( cur_check === undefined ) ? true : !!cur_check; + + var nav = this; + var event = { + stepId: step_id, + currentStepId: this._lastStepId, + + abort: false, + resume: function( revalidate, callback ) + { + // don't let 'em jump ahead + if ( ( force !== true ) + && ( nav.isValidNextStep( step_id ) !== true ) + ) + { + return; + } + + revalidate = !!revalidate; + + // if a callback was provided, then provide a new resume + // continuation that will queue up each callback, ensuring they + // are all called once we successfully finish + var orig_resume = event.resume; + event.resume = ( callback ) + ? function( revalidate, callback_new ) + { + orig_resume( revalidate, function() + { + callback_new && callback_new(); + callback(); + } ); + } + : orig_resume; + + // nothing to do if we're already on the step + if ( cur_check && ( nav._lastStepId == step_id ) ) + { + return nav; + } + + // call the pre-change hooks + event.abort = false; + event.force = force; + + if ( revalidate ) + { + nav.emit( nav.__self.$('EVENT_STEP_PRE_CHANGE'), event ); + + if ( event.abort === true ) + { + return; + } + } + + nav._lastStepId = step_id; + if ( step_id > nav._topVisitedStepId ) + { + nav._topVisitedStepId = step_id; + } + + // the step has changed + nav.emit( nav.__self.$('EVENT_STEP_CHANGE'), step_id ); + + callback && callback(); + } + }; + + event.resume( true ); + return this; + }, + + + /** + * Returns the highest step the user has gotten to + * + * @return Integer top step id + */ + getTopVisitedStepId: function() + { + return this._topVisitedStepId; + }, + + + /** + * Attempts to return the id of the next available step + * + * @return Integer id of the next step if available, otherwise id of the + * current step + */ + getNextStepId: function() + { + return ( this.hasNextStep() ) + ? this._lastStepId + 1 + : this._lastStepId; + }, + + + /** + * Attempts to get the id of the next available previous step + * + * @return Integer id of previous step if available, otherwise id of the + * current step + */ + getPrevStepId: function() + { + return ( this.hasPrevStep() ) + ? this._lastStepId - 1 + : this._lastStepId; + }, + + + /** + * Returns whether a step has been visited + * + * @param Integer step_id id of step to check + * + * @return Boolean true if step has been previous visited, otherwise false + */ + isStepVisited: function( step_id ) + { + // if we add hidden steps, this logic will need to change to keep track + // of specific steps + return ( step_id <= this._topVisitedStepId ) + ? true + : false; + }, + + + /** + * Returns whether a next step is available to navigate to + * + * @return Boolean true if a next step is available, otherwise false + */ + hasNextStep: function() + { + return true; + }, + + + /** + * Returns whether a previous step is available to navigate to + * + * @return Boolean true if a previous step is available, otherwise false + */ + hasPrevStep: function() + { + // this logic is bound to change in the future + return ( +this._lastStepId <= +this._firstStepId ) + ? false + : true; + }, + + + /** + * Attempts to navigate to the next available step + * + * @return Nav self to allow for method chaining + */ + navigateToNextStep: function() + { + // we don't know what to do if we don't know what step we're on! + if ( this._lastStepId == 0 ) + { + return; + } + + var step_id = this.getNextStepId(); + this.navigateToStep( step_id ); + + return this; + }, + + + /** + * Attempts to navigate to the previous available step + * + * @return Nav self to allow for method chaining + */ + navigateToPrevStep: function() + { + if ( this.hasPrevStep() === false ) + { + return; + } + + + var step_id = this.getPrevStepId(); + this.navigateToStep( step_id ); + + return this; + }, + + + setTopVisitedStepId: function( step_id ) + { + this._topVisitedStepId = +step_id; + }, + + + /** + * Sets the quote id to be used for navigation + * + * @param Integer id quote id + * + * @return Nav self to allow for method chaining + */ + setQuoteId: function( id, clear_step ) + { + clear_step = ( clear_step === undefined ) ? false : !!clear_step; + + // if the quote id is the same, don't do anything + if ( ( id == this._quoteId ) || ( id === undefined ) ) + { + return this; + } + + this._quoteId = id; + + // raise the event + this.emit( this.__self.$('EVENT_QUOTE_ID_CHANGE'), + this._quoteId, clear_step + ); + + return this; + }, + + + /** + * Returns the quote id + * + * @return Integer quote id + */ + getQuoteId: function() + { + return this._quoteId; + }, + + + /** + * Returns the id of the current step + * + * @return Integer id of the current step + */ + getCurrentStepId: function() + { + return this._lastStepId; + }, + + + isValidNextStep: function( step_id ) + { + return ( step_id > ( this.getTopVisitedStepId() + 1 ) ) + ? false + : true; + }, + + + /** + * Returns whether or not the given step is the last step + * + * @param {number} step_id + * + * @return {boolean} true if last step, otherwise false + */ + isLastStep: function( step_id ) + { + return ( step_id === this._stepCount ) + ? true + : false; + }, + + + /** + * Returns whether or not the given step is the quote review + * step. + * + * @param {number} step_id + * + * @return {boolean} true if quote review step, otherwise false + */ + isQuoteReviewStep: function( step_id ) + { + return ( this._stepData[ step_id ].type === 'review' ) + ? true + : false; + }, + + + /** + * Returns whether or not the given step is the manage quote + * step. + * + * @param {number} step_id + * + * @return {boolean} true if manage quote step, otherwise false + */ + isManageQuoteStep: function( step_id ) + { + return ( this._stepData[ step_id ].type === 'manage' ) + ? true + : false; + }, + + + 'public getFirstStepId': function() + { + return this._firstStepId; + }, + + + 'public setMinStepId': function( id ) + { + this._minStepId = +id || 0; + return this; + }, + + 'public getMinStepId': function() + { + return this._minStepId; + } +} ); + diff --git a/src/client/proxy/HttpDataProxy.js b/src/client/proxy/HttpDataProxy.js new file mode 100644 index 0000000..ea48370 --- /dev/null +++ b/src/client/proxy/HttpDataProxy.js @@ -0,0 +1,161 @@ +/** + * HttpDataProxy interface + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + * + * @todo This is a deprecated system; remove. + */ + +var AbstractClass = require( 'easejs' ).AbstractClass, + EventEmitter = require( 'events' ).EventEmitter; + + +/** + * Abstract class providing the foundation for a data proxy + * + * Data proxies simply abstract the means to communicate with another data + * source, such as a remote web server. + */ +module.exports = AbstractClass( 'HttpDataProxy' ) + .extend( EventEmitter, +{ + /** + * Triggered when data is received + * @type {string} + */ + 'const EVENT_RECEIVED': 'received', + + + /** + * Retrieves data using a HTTP GET request + * + * @param {string} url URL to request + * @param {function( Object, * )} callback function to call when complete + * + * @return {HttpDataProxy} self + */ + 'public get': function( url, callback ) + { + var _self = this; + + this.getData( url, function( data, error ) + { + _self._dataResponse( data, error, callback ); + }); + + return this; + }, + + + /** + * Retrieves data using a HTTP POST request + * + * @param {string} url URL to request + * @param {Object} postdata data to post + * @param {function( Object, * )} callback function to call when complete + * + * @return {HttpDataProxy} self + */ + 'public post': function( url, postdata, callback ) + { + var _self = this; + + this.postData( url, postdata, function( data, error ) + { + _self._dataResponse( data, error, callback ); + }); + + return this; + }, + + + /** + * Called when a response is received from the server + * + * The callback is not called if the process is aborted. + * + * @param {Object} data data received from server + * @param {Object} error error data, otherwise null + * + * @param {function( Object, * )} callback function to call to return + * received data + * + * @return void + */ + 'protected _dataResponse': function( data, error, callback ) + { + var abort = false, + event = { + abort: function() + { + abort = true; + } + }; + + this.emit( this.__self.$('EVENT_RECEIVED'), data, event ); + + // if aborted, we don't want to call the callback + if ( abort ) + { + return; + } + + if ( callback instanceof Function ) + { + callback( data, error ); + } + }, + + + /** + * Permits subtype to retrieve data + * + * Subtypes should override this method to implement a method for retrieving + * data + * + * @param {string} url URL to request + * @param {function( Object, * )} callback function to call when complete + * + * @return {undefined} + */ + 'abstract protected getData': [ 'url', 'callback' ], + + + /** + * Permits subtype to retrieve data + * + * Subtypes should override this method to implement a method for retrieving + * data + * + * @param {string} url URL to request + * @param {Object} data data to post + * @param {function( Object, * )} callback function to call when complete + * + * @return {undefined} + */ + 'abstract protected postData': [ 'url', 'data', 'callback' ], + + + /** + * Aborts all current requests + * + * @return {HttpDataProxy} self + */ + 'abstract public abortAll': [] +} ); + diff --git a/src/client/proxy/JqueryHttpDataProxy.js b/src/client/proxy/JqueryHttpDataProxy.js new file mode 100644 index 0000000..e596ba2 --- /dev/null +++ b/src/client/proxy/JqueryHttpDataProxy.js @@ -0,0 +1,148 @@ +/** + * Contains JqueryHttpDataProxy + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + * + * @todo This is a deprecated system; remove. + */ + +var Class = require( 'easejs' ).Class, + HttpDataProxy = require( './HttpDataProxy' ); + + +module.exports = Class( 'JqueryHttpDataProxy' ) + .extend( HttpDataProxy, +{ + /** + * jQuery object to use + * @type {jQuery} + */ + 'private _jquery': null, + + /** + * Pool of outstanding requests + * @type {Array.} + */ + 'private _requestPool': [], + + + /** + * Initializes data proxy with the given jQuery object + * + * The jQuery object is injected to both decouple the two and to make it + * stubbable for tests + * + * @param {jQuery} jquery jQuery object + * + * @return {undefined} + */ + 'public __construct': function( jquery ) + { + if ( !( jquery ) ) + { + throw Error( 'No jQuery instance provided' ); + } + + this._jquery = jquery; + }, + + + 'private _doXhr': function( url, callback, type, data ) + { + callback = callback || function() {}; + + var _self = this, + pool_pos = -1, + done = false, + xhr = this._jquery.ajax( { + type: type, + data: data, + url: url, + dataType: 'json', + + success: function( data ) + { + _self._removeFromPool( pool_pos ); + callback( data || {}, null ); + + done = true; + }, + + error: function( xhr, text_status ) + { + _self._removeFromPool( pool_pos ); + callback( null, text_status ); + + done = true; + } + } ); + + // add request to the pool, only for async requests + if ( !done ) + { + pool_pos = ( this._requestPool.push( xhr ) - 1 ); + } + }, + + + 'virtual protected getData': function( url, callback ) + { + this._doXhr( url, callback, 'GET' ); + }, + + + 'virtual protected postData': function( url, data, callback ) + { + this._doXhr( url, callback, 'POST', data ); + }, + + + 'private _removeFromPool': function( pos ) + { + // if the position is -1, then the pool wasn't yet created + if ( pos === -1 ) + { + return; + } + + delete this._requestPool[ pos ]; + }, + + + 'abortAll': function() + { + var i = this._requestPool.length; + + while ( i-- ) + { + var xhr = this._requestPool[ i ]; + + // will be undefined if it finished + if ( xhr === undefined ) + { + continue; + } + + xhr.abort(); + } + + // remove 'em all from memory and start with a fresh array + this.requestPool = []; + } +}); + diff --git a/src/client/quote/ClientQuote.js b/src/client/quote/ClientQuote.js new file mode 100644 index 0000000..75d7ef2 --- /dev/null +++ b/src/client/quote/ClientQuote.js @@ -0,0 +1,720 @@ +/** + * Client-side quote + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + Quote = require( '../../quote/Quote' ), + EventEmitter = require( 'events' ).EventEmitter; + + +/** + * Client interface to the Quote with additional functionality useful to the + * client, such as staging changes and initiating saving + */ +module.exports = Class( 'ClientQuote' ) + .implement( Quote ) + .extend( EventEmitter, +{ + /** + * Emitted when data is committed + * @type {string} + */ + 'const EVENT_DATA_COMMIT': 'dataCommit', + + /** + * Emitted before data is updated + * + * This permits altering the values before they are added to the bucket and + * is useful for, say, validations. + * + * @type {string} + */ + 'const EVENT_PRE_DATA_UPDATE': 'preDataUpdate', + + /** + * Emitted when data is updated (but not committed) + * @type {string} + */ + 'const EVENT_DATA_UPDATE': 'dataUpdate', + + /** + * Raised when current step id changes + * @type {string} + */ + 'const EVENT_STEP_CHANGE': 'stepChange', + + /** + * Emitted when a new data classification has taken place + * + * Does not necessarily imply that any classifications have changed. + * + * @type {string} + */ + 'const EVENT_CLASSIFY': 'classify', + + + /** + * Quote to operate on + * @type {Quote} + */ + 'private _quote': null, + + /** + * Staged changes to write to quote + * @type {StagingBucket} + */ + 'private _staging': null, + + /** + * Forcefully consider the quote unlocked, ignoring criteria + * @type {boolean} + */ + 'private _forceUnlock': false, + + /** + * Results of last classification + * @type {Object} + */ + 'private _lastClassify': null, + + 'private _classifier': null, + + 'private _classKnown': {}, + + + /** + * Initializes component with the given quote + * + * @param {Quote} quote quote to augment + * @param {Object} data data to initialize quote with (from server) + * + * @return {undefined} + */ + 'public __construct': function( quote, data, staging_callback ) + { + var _self = this; + + this._quote = this.initQuote( quote, data ); + + // create the staging bucket + quote.visitData( function( bucket ) + { + _self._staging = staging_callback( + bucket + ); + + _self.hookBuckets( _self._staging, bucket ); + } ); + + }, + + + 'virtual protected initQuote': function( quote, data ) + { + var _self = this; + + return quote + .setData( data.data || {} ) + .setCurrentStepId( data.currentStepId || 0 ) + .setTopVisitedStepId( data.topVisitedStepId || 0 ) + .setAgentId( data.agentId || 0 ) + .setImported( data.imported || false ) + .setBound( data.bound || false ) + .needsImport( data.needsImport || false ) + .setExplicitLock( + ( data.explicitLock || '' ), + ( data.explicitLockStepId || 0 ) + ) + .on( 'stepChange', function( step_id ) + { + _self.emit( _self.__self.$('EVENT_STEP_CHANGE'), step_id ); + } ); + }, + + + 'virtual protected hookBuckets': function( staging, bucket ) + { + var _self = this; + + // forward update events + bucket.on( 'update', function( data ) + { + _self.emit( _self.__self.$('EVENT_DATA_COMMIT'), data ); + } ); + + // forward staging update events + staging.on( 'preStagingUpdate', function( data ) + { + _self.emit( _self.__self.$('EVENT_PRE_DATA_UPDATE'), data ); + } ); + + staging.on( 'stagingUpdate', function( data ) + { + _self.emit( _self.__self.$('EVENT_DATA_UPDATE'), data ); + + // perform classification + _self._classify( staging, data ); + } ); + }, + + + /** + * Set the classifier to be used for data classification + * + * The classifier should return an object containing all classifications and + * a single boolean value per classification. + * + * @param {function(Object)} classifier classifier function + * + * @return {ClientQuote} self + */ + 'public setClassifier': function( known_fields, classifier ) + { + if ( !( typeof classifier === 'function' ) ) + { + throw TypeError( 'Classifier must be a function' ); + } + + this._classifier = classifier; + this._classKnown = known_fields; + + return this; + }, + + + 'public forceClassify': function() + { + this._classify( this._staging, null ); + return this; + }, + + + /** + * Emit data classifications + * + * @param {Bucket} staging staging bucket + * + * @return {undefined} + */ + 'private _classify': function( staging, data ) + { + if ( !( this._classifier ) ) + { + return; + } + + // ignore fields that do not affect the classifier (if the given data is + // null, that signifies that we should perform the classification + // regardless) + if ( data ) + { + var found = false; + for ( var name in data ) + { + if ( this._classKnown[ name ] ) + { + found = true; + break; + } + } + + if ( !( found ) ) + { + return; + } + } + + this.emit( + this.__self.$('EVENT_CLASSIFY'), + this._lastClassify = this._classifier( staging.getData() ) + ); + }, + + + /** + * Hooks classify event and immediately triggers the continuation with the + * last classification data, if any + * + * Useful if the classification event has already been kicked off, but the + * data is needed (prevents race conditions). + * + * We provide this method rather than a getter to enforce encapsulation (we + * won't consider it violated here since event handlers normally have access + * to that data anyway)---this signifies intent. + * + * @param {function(Object)} c continuation + */ + 'public onClassifyAndNow': function( c ) + { + this.on( this.__self.$('EVENT_CLASSIFY'), c ); + + // we may have been called too early, in which case we have nothing to + // provide + if ( this._lastClassify !== null ) + { + c( this._lastClassify ); + } + + return this; + }, + + + /** + * Return the quote id + * + * @return {number} quote id + */ + 'public proxy getId': '_quote', + + + /** + * Stages the given data + * + * Data is not written directly to the quote. It must be committed. + * + * @param {Object.>} data data to set + * + * @return {ClientQuote} self + */ + 'public setData': function( data, merge_nulls ) + { + merge_nulls = !!merge_nulls; + + this._staging.setValues( data, true, merge_nulls ); + return this; + }, + + + 'public setDataByName': function( name, values ) + { + var data = {}; + data[ name ] = values; + + this.setData( data ); + }, + + + /** + * Stages the given data for a certain index + * + * This is an alternative to manually generating an array with a single + * value for the given index using setData(). + * + * @param {number} index index to set + * @param {Object.} data data to set for index of each id + * + * @return {ClientQuote} self + */ + 'public setDataByIndex': function( index, data ) + { + var diff = {}; + + // generate diff object + for ( var name in data ) + { + var item = []; + item[ index ] = data[ name ]; + + diff[ name ] = item; + } + + return this.setData( diff ); + }, + + + /** + * Overwrites data, preventing merges + * + * @param {Object} data + * + * @return {ClientQuote} self + */ + 'public overwriteData': function( data ) + { + this._staging.overwriteValues( data ); + return this; + }, + + + /** + * Set quote data without considering it to be a modification + * + * Set underlying quote data DATA without triggering a client-side + * modifications. The intended us of this is to refresh quote + * data from the server, which has already been saved. + * + * DATA will completely overwrite existing values; it will not + * merge indexes like normal updates. + * + * @param {Object} data data to overwrite + * + * @return {ClientQuote} self + */ + 'public refreshData': function( data ) + { + this._staging.setCommittedValues( data, false ); + return this; + }, + + + /** + * Returns data from the quote + * + * @param {string} name name of data to retrieve + * + * @return {Array} quote data + */ + 'public proxy getDataByName': '_staging', + + + /** + * Invoke callback for each value associated with the quote + * + * @param {function(Array.,string)} callback function to call + * + * @return {ClientQuote} self + */ + 'public eachValue': function( callback ) + { + this._staging.each( callback ); + return this; + }, + + + /** + * Invoke callback for each value associated with the quote whose name + * matches the given pattern + * + * @param {Regexp} regex pattern to match name + * @param {function(Array.,string)} callback function to call + * + * @return {ClientQuote} self + */ + 'public eachValueMatch': function( regex, callback ) + { + this.eachValue( function( data, name ) + { + if ( regex.test( name ) ) + { + callback( data, name ); + } + } ); + + return this; + }, + + + /** + * Commits changes to quote and attempts to save + * + * @return {ClientQuote} self + */ + 'public save': function( transport, callback ) + { + var _self = this, + old_store = {}; + + this._doSave( transport, function( data ) + { + // re-populate the previously staged values on error; otherwise, + // they would not be saved the next time around! + if ( data.hasError ) + { + _self._staging.setValues( old_store.old, true, false ); + } + + callback.apply( null, arguments ); + } ); + + // commit staged quote data to the data bucket (important: do this + // *after* save); will make the staged values available as old_store.old + this._staging.commit( old_store ); + + return this; + }, + + + 'public saveStaging': function( transport, callback ) + { + this._doSave( transport, callback ); + return this; + }, + + + 'private _doSave': function( transport, callback ) + { + // if no transport was given, then don't save to the server + if ( transport === undefined ) + { + return this; + } + + var _self = this; + + // send quote data to server + transport.send( this, function( err, data ) + { + // if bucket data is returned, then apply it + if ( data && data.content && !data.hasError ) + { + // the server has likely already applied these changes, so do + // not allow them to be discarded + _self._staging.setCommittedValues( data.content, true, false ); + } + + if ( err ) + { + data = data || {}; + data.hasError = true; + data.content = err.message || err; + } + + if ( typeof callback === 'function' ) + { + callback( data ); + } + } ); + }, + + + 'public isDirty': function() + { + return this._staging.isDirty(); + }, + + + 'public clientSideUnlock': function() + { + this._forceUnlock = true; + }, + + + 'public clientSideRelock': function() + { + this._forceUnlock = false; + }, + + + /** + * Visits staging data + * + * @param {function( Bucket )} visitor + * + * @return void + */ + 'public visitData': function( visitor ) + { + visitor( this._staging ); + }, + + + /** + * Returns the program id associated with the quote + * + * @return {string} program id + */ + 'public proxy getProgramId': '_quote', + + + /** + * Sets the program id associated with the quote + * + * @return {string} program id + */ + 'public proxy setProgram': '_quote', + + + /** + * Returns the quote start date + * + * @return {number} quote start date + */ + 'public proxy getStartDate': '_quote', + + + /** + * Returns the id of the agent that owns the quote + * + * @return {number} agent id + */ + 'public proxy getAgentId': '_quote', + + + /** + * Returns whether the quote has been imported + * + * @return {boolean} true if imported, otherwise false + */ + 'public proxy isImported': '_quote', + + + /** + * Returns whether the quote has been bound + * + * @return {boolean} true if bound, otherwise false + */ + 'public proxy isBound': '_quote', + + + /** + * Returns the id of the current step + * + * @return {number} id of current step + */ + 'public proxy getCurrentStepId': '_quote', + + + /** + * Sets the current step id + * + * @param {number} step_id id of the step to set + * + * @return {ClientQuote} self + */ + 'public proxy setCurrentStepId': '_quote', + + + /** + * Returns the id of the highest step the quote has reached + * + * @return {number} top visited step id + */ + 'public proxy getTopVisitedStepId': '_quote', + + + /** + * Sets the top visited step id + * + * If the provided step id is less than the current step, then the current + * step id is used instead. + * + * @return {ClientQuote} self + */ + 'public proxy setTopVisitedStepId': '_quote', + + + /** + * Returns whether the quote is locked from modifications + * + * @return {boolean} true if locked, otherwise false + */ + 'public isLocked': function() + { + if ( this._forceUnlock ) + { + return false; + } + + return this._quote.isLocked(); + }, + + + /** + * Returns whether the given step has been visited + * + * @param {number} id step id + * + * @return {boolean} whether step has been visited + */ + 'public proxy hasVisitedStep': '_quote', + + + /** + * Sets a quote's imported status + * + * @param {boolean} imported + * + * @return {ClientQuote} self + */ + 'public proxy setImported': '_quote', + + + /** + * Set quicksave data + * + * @param {Object} data quicksave data, diff format + * + * @return {Quote} self + */ + 'public proxy setQuickSaveData': '_quote', + + + /** + * Retrieve quicksave data + * + * @return {Object} quicksave data + */ + 'public proxy getQuickSaveData': '_quote', + + + /** + * Sets an explicit lock, providing a reason for doing so + * + * @param {string} reason lock reason + * + * @return {Quote} self + */ + 'public proxy setExplicitLock': '_quote', + + + /** + * Clears an explicit lock + * + * @return {Quote} self + */ + 'public proxy clearExplicitLock': '_quote', + + + /** + * Retrieves the reason for an explicit lock + * + * @return {string} lock reason + */ + 'public getExplicitLockReason': function() + { + return ( this._forceUnlock ) + ? '' + : this._quote.getExplicitLockReason(); + }, + + /** + * Returns the explicit lock step, if applicable + * + * @return {number} lock step, otherwise 0 + */ + 'public getExplicitLockStep': function() + { + return ( this._forceUnlock ) + ? 0 + : this._quote.getExplicitLockStep(); + }, + + + 'public needsImport': function( set ) + { + if ( set !== undefined ) + { + return this._quote.needsImport( set ); + } + + return !this.isLocked() && this._quote.needsImport(); + } +} ); + diff --git a/src/client/service/ExportClient.js b/src/client/service/ExportClient.js new file mode 100644 index 0000000..361c2c6 --- /dev/null +++ b/src/client/service/ExportClient.js @@ -0,0 +1,261 @@ +/** + * Export client + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + HttpDataApi = require( '../../dapi/http/HttpDataApi' ), + XhrHttpImpl = require( '../../dapi/http/XhrHttpImpl' ), + AutoRetry = require( '../../dapi/AutoRetry' ), + JsonResponse = require( '../../dapi/format/JsonResponse' ); + + +/** + * Facade handling export request + * + * TODO: If this is to be a facade, then this needs to have most of its + * logic extracted into another class. Time constraints. + */ +module.exports = Class( 'ExportClient', +{ + /** + * Export service path + * @var {string} + */ + 'private _service': '', + + + /** + * Initialize with service path + * + * The service path SERVICE is relative to the base URI of the quote. + * + * @param {string} service path to service + */ + __construct: function( service ) + { + this._service = ''+service; + }, + + + /** + * Initiate and await result of export of quote data into external + * system + * + * The export will first attempt to initiate the remote process; if this + * fails due to an active process for the same quote, it will continue + * to retry. Once initiated, the server is polled for the result data + * and accepted once available, completing the request. + * + * If there is any error during initiation or polling, the process is + * aborted and an error passed to CALLBACK. + * + * @param {Quote} quote quote to export + * @param {Object.} params additional parameters to pass + * to remote import script + * + * @param {function(?Error,Object)} callback result of export + * + * @return {ExportClient} self + */ + 'public export': function( quote, params, callback ) + { + var _self = this; + + this._startExport( quote, params, function( e, response ) + { + // bail out on export request failure + if ( e !== null ) + { + callback( e, response ); + } + + var token_id = response.data.tokenId; + if ( !token_id ) + { + callback( + Error( "No token id provided for export request" ), + response + ); + } + + // monitor the token and accept the data once available + _self._startAccept( quote, token_id, function( e, response ) + { + if ( e !== null ) + { + callback( e, response ); + } + + // parse the return data; we're done + try + { + callback( null, JSON.parse( response.data.tokenData ) ); + } + catch ( parse_err ) + { + callback( parse_err, response.data ); + } + } ); + } ); + + return this; + }, + + + /** + * Create an API suitable for the export request + * + * TODO: This is the facade part; all other logic should be stripped + * from this class. + * + * @param {string} uri base URI for export request + * @param {function(?Error,Object)} pred retry predicate + * + * @return {undefined} + */ + 'private _createExportApi': function( uri, pred ) + { + // we will give it a good two minutes (test site can be slow) + var delay_s = 2, + tries = ( 120 / delay ); + + var delay = function( remain, callback ) + { + setTimeout( callback, 2e3 ); + }; + + return HttpDataApi + .use( JsonResponse ) + .use( AutoRetry( pred, tries, delay ) ) + ( + uri, + 'GET', + XhrHttpImpl( XMLHttpRequest ) + ); + }, + + + /** + * Initiate export + * + * @param {Quote} quote quote to export + * @param {Object.} params additional parameters to pass + * to remote import script + * + * @param {function(?Error,Object)} callback result of export + * + * @return {undefined} + */ + 'private _startExport': function( quote, params, callback ) + { + this._startRequest( quote, params, null, 'ACTIVE', callback ); + }, + + + /** + * Await export completion and accept response + * + * @param {Quote} quote quote to export + * @param {string} token token to monitor + * + * @param {function(?Error,Object)} callback result of export + * + * @return {undefined} + */ + 'private _startAccept': function( quote, token, callback ) + { + var action = 'accept/' + token; + + this._startRequest( quote, null, action, 'DONE', callback ); + }, + + + /** + * Initiate auto-retrying request + * + * @param {Quote} quote quote to export + * @param {?Object.} params additional parameters to pass + * to remote import script + * @param {?string} action service action + * @param {string} status token status to await + * + * @param {function(?Error,Object)} callback result of export + * + * @return {undefined} + */ + 'private _startRequest': function( quote, params, action, status, callback ) + { + var _self = this; + + var c1_export = this._createExportApi( + this._genRequestUri( quote, action ), + + function( e, output ) + { + // if this is a legitimate failure, then we should _not_ + // retry: something went wrong, and we do not want to + // continue to cause problems + if ( e !== null ) + { + return _self._shouldTryAgain( e.status, output.error ); + } + + // continue requesting until we produce an active import + return ( output.data.status !== status ); + } + ); + + c1_export.request( params, callback ); + }, + + + /** + * Generate URI for export request + * + * @param {Quote} quote quote to export + * @param {string=} action service action + * + * @return {string} export request URI + */ + 'private _genRequestUri': function( quote, action ) + { + // XXX: we should not make these assumptions; abstract! + return '/quote/' + + quote.getProgramId() + + '/' + quote.getId() + + '/' + this._service + + ( ( action ) ? '/' + action : '' ); + }, + + + /** + * Determine whether the response indicates that the request should be + * re-tried + * + * @param {number} status HTTP status code + * @param {string} error error response from server + */ + 'private _shouldTryAgain': function( status, error ) + { + return ( status === 503 ) + && error === 'EAGAIN'; + } +} ); + diff --git a/src/client/transport/QuoteTransport.js b/src/client/transport/QuoteTransport.js new file mode 100644 index 0000000..38c14b7 --- /dev/null +++ b/src/client/transport/QuoteTransport.js @@ -0,0 +1,37 @@ +/** + * QuoteTransport interface + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +var Interface = require( 'easejs' ).Interface; + + +/** + * Interface used for types that wish to transfer quote data + * + * Implementors must accept a quote for transfer. The interface does not + * specify what data should be transfered, but simply that the quote needs + * to in some way to be transfered from one point to another. The + * implementation is left to the transporter. + */ +module.exports = Interface( 'QuoteTransport', +{ + 'public send': [ 'quote' ] +} ); + diff --git a/src/client/transport/XhttpQuoteStagingTransport.js b/src/client/transport/XhttpQuoteStagingTransport.js new file mode 100644 index 0000000..9ede976 --- /dev/null +++ b/src/client/transport/XhttpQuoteStagingTransport.js @@ -0,0 +1,35 @@ +/** + * XhttpQuoteStagingTransport class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + XhttpQuoteTransport = require( './XhttpQuoteTransport' ); + + +module.exports = Class( 'XhttpQuoteStagingTransport' ) + .extend( XhttpQuoteTransport, +{ + 'override protected getBucketDataJson': function( bucket ) + { + // return the staged changes + return JSON.stringify( bucket.getDiff() ); + } +} ); + diff --git a/src/client/transport/XhttpQuoteTransport.js b/src/client/transport/XhttpQuoteTransport.js new file mode 100644 index 0000000..f0c6c3f --- /dev/null +++ b/src/client/transport/XhttpQuoteTransport.js @@ -0,0 +1,114 @@ +/** + * Contains JqueryXhttpQuoteTransport class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + QuoteTransport = require( './QuoteTransport'); + + +/** + * Transfers quote data via an XHTTP request using jQuery + */ +module.exports = Class( 'XhttpQuoteTransport' ) + .implement( QuoteTransport ) + .extend( +{ + /** + * HttpProxy used to send data + * @type {HttpProxy} + */ + 'private _proxy': null, + + /** + * URL to post quote data to + * @type {string} + */ + 'private _url': '', + + + /** + * Constructs a new quote transport with the destination URL and proxy + * + * @param {string} url destination URL + * @param {HttpDataProxy} proxy proxy to use for transfer + * + * @return {undefined} + */ + 'public __construct': function( url, proxy ) + { + this._url = ''+( url ); + this._proxy = proxy; + }, + + + /** + * Transfers quote data to the remote server + * + * The callback function is called even if an error occurs. Be sure to check + * the error argument for problems before assuming that everything went + * well. + * + * @param {ClientQuote} quote quote to transfer + * @param {function( * )} callback function to call when complete + * + * @return void + */ + 'public send': function( quote, callback ) + { + var _self = this; + + quote.visitData( function( bucket ) + { + // get the data from the bucket + var data = _self.getBucketDataJson( bucket ); + + // post the data + _self._proxy.post( _self._url, { data: data }, + function( data, error ) + { + if ( typeof callback === 'function' ) + { + callback.call( this, error || null, data ); + } + } + ); + } ); + }, + + + /** + * Retrieve bucket data in JSON format + * + * Allows subtypes to override what data is retrieved from the bucket + * + * @param {Bucket} bucket bucket from which to retrieve data + * + * @return {XhttpQuoteTransport} self + */ + 'virtual protected getBucketDataJson': function( bucket ) + { + // get a "filled" diff containing the merged values of only the fields + // that have changed + var data = bucket.getFilledDiff(); + + return JSON.stringify( data ); + } +} ); + diff --git a/src/dapi/DataApiFactory.js b/src/dapi/DataApiFactory.js new file mode 100644 index 0000000..b449374 --- /dev/null +++ b/src/dapi/DataApiFactory.js @@ -0,0 +1,88 @@ +/** + * Instantiate appropriate DataApi + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + + HttpDataApi = require( './http/HttpDataApi' ), + XhrHttpImpl = require( './http/XhrHttpImpl' ), + JsonResponse = require( './format/JsonResponse' ), + RestrictedDataApi = require( './RestrictedDataApi' ), + StaticAdditionDataApi = require( './StaticAdditionDataApi' ), + BucketDataApi = require( './BucketDataApi' ); + + +/** + * Instantiates the appropriate DataApi object for the givne service type + */ +module.exports = Class( 'DataApiFactory', +{ + /** + * Return a DataApi instance for the requested service type + * + * The source and method have type-specific meaning; that is, "source" may + * be a URL and "method" may be get/post for a RESTful service. + * + * @param {string} type service type (e.g. "rest") + * @param {Object} desc API description + * + * @return {DataApi} appropriate DataApi instance + */ + 'public fromType': function( type, desc, bucket ) + { + var api = null, + source = ( desc.source || '' ), + method = ( desc.method || '' ), + + static_data = ( desc['static'] || [] ), + nonempty = !!desc.static_nonempty, + multiple = !!desc.static_multiple; + + switch ( type ) + { + case 'rest': + api = HttpDataApi.use( JsonResponse )( + source, + method.toUpperCase(), + XhrHttpImpl( XMLHttpRequest ) + ); + break; + + case 'local': + // currently, only local bucket data sources are supported + if ( source !== 'bucket' ) + { + throw Error( "Unknown local data API source: " + source ); + } + + api = BucketDataApi( bucket, desc.retvals ); + break; + + default: + throw Error( 'Unknown data API type: ' + type ); + }; + + return RestrictedDataApi( + StaticAdditionDataApi( api, nonempty, multiple, static_data ), + desc + ); + } +} ); + diff --git a/src/dapi/GetRestDataApiStrategy.js b/src/dapi/GetRestDataApiStrategy.js new file mode 100644 index 0000000..8ddc0c7 --- /dev/null +++ b/src/dapi/GetRestDataApiStrategy.js @@ -0,0 +1,110 @@ +/** + * Concrete PostRestDataApiStrategy class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + RestDataApiStrategy = require( './RestDataApiStrategy' ), + + // XXX: decouple (this couples us to the client)! + HttpDataProxy = require( '../client/proxy/HttpDataProxy' ); + + +/** + * Requests data from a RESTful service, GETing arguments + */ +module.exports = Class( 'GetRestDataApiStrategy' ) + .implement( RestDataApiStrategy ) + .extend( +{ + /** + * Data proxy used to communicate with the service + * @type {HttpDataProxy} + */ + 'private _proxy': null, + + + /** + * Initialize strategy + * + * The strategy is independent of any URL; that is, we should not pass the + * URL here, as the strategy can be re-used for *any* RESTful service. + * + * @param {HttpDataProxy} data_proxy proxy to handle all requests + */ + __construct: function( data_proxy ) + { + if ( !( Class.isA( HttpDataProxy, data_proxy ) ) ) + { + throw Error( + 'Expected HttpDataProxy; given ' + data_proxy + ); + } + + this._proxy = data_proxy; + }, + + + /** + * Request data from the service + * + * @param {string} url service URL + * @param {Object} data request params + * @param {function(Object)} callback server response callback + * + * @return {RestDataApi} self + */ + 'public requestData': function( url, data, callback ) + { + // generate the params for the URL + var params = this._genUrlParams( data ); + + // only delimit the params if they exist + var geturl = url + ( ( params ) ? '?' + params : '' ); + + // make the request + this._proxy.get( geturl, function( retdata ) + { + callback( retdata ); + } ); + + return this; + }, + + + /** + * Generate URL params for a GET request + * + * @param {Object} data data to parameterize + * + * @return {string} URL with params and arguments + */ + 'private _genUrlParams': function( data ) + { + var params = ''; + for ( var field in data ) + { + params += ( ( params ) ? '&' : '' ) + + field + '=' + data[ field ]; + } + + return encodeURI( params ); + } +} ); + diff --git a/src/dapi/PostRestDataApiStrategy.js b/src/dapi/PostRestDataApiStrategy.js new file mode 100644 index 0000000..9123896 --- /dev/null +++ b/src/dapi/PostRestDataApiStrategy.js @@ -0,0 +1,83 @@ +/** + * Concrete PostRestDataApiStrategy class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + RestDataApiStrategy = require( './RestDataApiStrategy' ), + + // XXX: decouple (this couples us to the client)! + HttpDataProxy = require( '../client/proxy/HttpDataProxy' ); + + +/** + * Requests data from a RESTful service, POSTing arguments + */ +module.exports = Class( 'PostRestDataApiStrategy' ) + .implement( RestDataApiStrategy ) + .extend( +{ + /** + * Data proxy used to communicate with the service + * @type {HttpDataProxy} + */ + 'private _proxy': null, + + + /** + * Initialize strategy + * + * The strategy is independent of any URL; that is, we should not pass the + * URL here, as the strategy can be re-used for *any* RESTful service. + * + * @param {HttpDataProxy} data_proxy proxy to handle all requests + */ + __construct: function( data_proxy ) + { + if ( !( Class.isA( HttpDataProxy, data_proxy ) ) ) + { + throw Error( + 'Expected HttpDataProxy; given ' + data_proxy + ); + } + + this._proxy = data_proxy; + }, + + + /** + * Request data from the service + * + * @param {string} url service URL + * @param {Object} data request params + * @param {function(Object)} callback server response callback + * + * @return {RestDataApi} self + */ + 'public requestData': function( url, data, callback ) + { + this._proxy.post( url, data, function( retdata ) + { + callback( retdata ); + } ); + + return this; + } +} ); + diff --git a/src/dapi/RestDataApi.js b/src/dapi/RestDataApi.js new file mode 100644 index 0000000..10348d9 --- /dev/null +++ b/src/dapi/RestDataApi.js @@ -0,0 +1,89 @@ +/** + * Contains concrete RestDataApi class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + DataApi = require( 'liza/dapi/DataApi' ), + + RestDataApiStrategy = require( './RestDataApiStrategy' ); + + +/** + * Retrieve data over a RESTful API + */ +module.exports = Class( 'RestDataApi' ) + .implement( DataApi ) + .extend( +{ + /** + * URL of RESTful service + * @type {string} + */ + 'private _url': '', + + /** + * Strategy used to request data from the service + * @type {RestDataApiStrategy} + */ + 'private _strategy': null, + + + /** + * Initialize data API + * + * @param {string} url service URL + * @param {RestDataApiStrategy} strategy request strategy + */ + __construct: function( url, strategy ) + { + if ( !( Class.isA( RestDataApiStrategy, strategy ) ) ) + { + throw Error( + 'Expected RestDataApiStrategy; given ' + strategy + ); + } + + this._url = ''+( url ); + this._strategy = strategy; + }, + + + /** + * Request data from the service + * + * @param {Object} data request params + * @param {function(Object)} callback server response callback + * + * @return {RestDataApi} self + */ + 'public request': function( data, callback ) + { + var _self = this.__inst; + + this._strategy.requestData( this._url, data, function( data ) + { + // return the data, but bind 'this' to ourself + callback.call( _self, data ); + } ); + + return this; + } +} ); + diff --git a/src/dapi/RestDataApiStrategy.js b/src/dapi/RestDataApiStrategy.js new file mode 100644 index 0000000..62db2a5 --- /dev/null +++ b/src/dapi/RestDataApiStrategy.js @@ -0,0 +1,42 @@ +/** + * RestDataApiStrategy interface + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +var Interface = require( 'easejs' ).Interface; + + +/** + * Represents a strategy used to request data from a RESTful service + * + * Such may be used to implement GET, POST, PUT, DELETE, etc. + */ +module.exports = Interface( 'RestDataApiStrategy', +{ + /** + * Request data from the service + * + * @param {Object} data request params + * @param {function(Object)} callback server response callback + * + * @return {RestDataApi} self + */ + 'public requestData': [ 'url', 'data', 'callback' ] +} ); + diff --git a/src/field/FieldClassMatcher.js b/src/field/FieldClassMatcher.js new file mode 100644 index 0000000..60bb1a7 --- /dev/null +++ b/src/field/FieldClassMatcher.js @@ -0,0 +1,190 @@ +/** + * Contains FieldClassMatcher class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class; + + +/** + * Generates match sets for field based on their classifications and a given + * classification set + */ +module.exports = Class( 'FieldClassMatcher', +{ + /** + * Fields and their accepted classes + * @type {Object.>} + */ + 'private _fields': {}, + + + /** + * Initialize matcher with a list of fields and their classifications + * + * @param {Object.>} fields field names and their classes + */ + __construct: function( fields ) + { + this._fields = fields; + }, + + + /** + * Generate classification match array for each field + * + * Any index for any field will be considered to be a match if the index + * is classified as each of the field's required classifications. + * + * @param {Object.} classes classifications + * + * @param {function(Object.)} callback with cmatch data + * + * @return {FieldClassMatcher} self + */ + 'public match': function( classes, callback ) + { + var cmatch = {}; + cmatch.__classes = classes; + + for ( var field in this._fields ) + { + var cur = this._fields[ field ], + vis = [], + all = true, + hasall = false; + + if ( cur.length === 0 ) + { + continue; + } + + // determine if we have a match based on the given classifications + for ( var c in cur ) + { + // if the indexes property is a scalar, then it applies to all + // indexes + var data = ( classes[ cur[ c ] ] || {} ), + thisall = ( typeof data.indexes !== 'object' ), + alltrue = ( !thisall || data.indexes === 1 ); + + // if no indexes apply for the given classification (it may be a + // pure boolean), then this variable will be true if any only if + // all of them are true. Note that we only want to take the + // value of thisall if we're on our first index, as if hasall is + // empty thereafter, then all of them certainly aren't true! + hasall = ( hasall || ( thisall && +c === 0 ) ); + + // this will ensure that, if we've already determined some sort + // of visibility, that encountering a scalar will still manage + // to affect previous results even if it is the last + // classification that we are checking + var indexes = ( thisall ) ? vis : data.indexes; + + for ( var i in indexes ) + { + // default to visible; note that, if we've encountered any + // "all index" situations (scalars), then we must only be + // true if the scalar value was true + vis[ i ] = ( + ( !hasall || all ) + && ( ( vis[ i ] === undefined ) + ? 1 + : vis[ i ] + ) + && this._reduceMatch( + ( thisall ) ? data.indexes : data.indexes[ i ] + ) + ); + + // all are true unless one is false (duh?) + alltrue = !!( alltrue && vis[ i ] ); + } + + all = ( all && alltrue ); + } + + // default 'any' to 'all'; this will have the effect of saying "yes + // there are matches, but we don't care what" if there are no + // indexes associated with the match, implying that all indexes + // should match + var any = all; + for ( var i = 0, len = vis.length; i < len; i++ ) + { + if ( vis[ i ] ) + { + any = true; + break; + } + } + + // store the classification match data for assertions, etc + cmatch[ field ] = { + all: all, + any: any, + indexes: vis + }; + } + + // currently not asynchronous, but leaves open the possibility + callback( cmatch ); + + return this; + }, + + + /** + * Reduces the given scalar or vector + * + * If a scalar is provided, then it is immediately returned. Otherwise, the + * vector is reduced using a logical or and the integer representation of + * the boolean value returned. + * + * This is useful for classification matrices since each match will be a + * vector and we wish to consider any match within that vector to be a + * positive match. + * + * @param {*} result result + * + * @return {number} 0 if false otherwise 1 for true + */ + 'private _reduceMatch': function( result ) + { + if ( ( result === undefined ) + || ( result === null ) + || ( result.length === undefined ) + ) + { + return result; + } + + var ret = false, + i = result.length; + + // reduce with logical or + while( i-- ) + { + // recurse just in case we have another array of values + ret = ret || this._reduceMatch( result[ i ] ); + } + + return +ret; + } +} ); + diff --git a/src/program/Program.js b/src/program/Program.js new file mode 100644 index 0000000..8a8f787 --- /dev/null +++ b/src/program/Program.js @@ -0,0 +1,611 @@ +/** + * Contains Program base class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + * + * @todo This is one of the system's oldest relics; evolve! + */ + +var AbstractClass = require( 'easejs' ).AbstractClass + EventEmitter = require( 'events' ).EventEmitter, + + // XXX coupling + Failure = require( '../validate/Failure' ), + BucketField = require( '../field/BucketField' ); + + +exports.Program = AbstractClass( 'Program' ) + .extend( EventEmitter, +{ + /** + * Program id + * @type {string} + */ + id: 'undefined', + + /** + * Program title + * @type {string} + */ + title: 'LoVullo Rater', + + + eventData: [], + + + /** + * Stores program metadata + * @type {Object} + */ + meta: {}, + + + /** + * Array of step titles + * @type {Array.} + */ + steps: [], + + /** + * Contains sidebar data + * @type {Object} + */ + sidebar: { overview: {}, static_content: {} }, + + /** + * Questions that should only be visible internally + * @type {Array.} + */ + internal: [], + + /** + * Default values for questions + * @type {Object.} + */ + defaults: {}, + + /** + * Fields contained within groups + * @type {Object.>} + */ + groupFields: {}, + + /** + * API descriptions + * @type {Object} + */ + 'protected apis': {}, + + 'public secureFields': [], + + 'private _assertDepth': 0, + + 'protected classifier': '', + + 'private _classify': null, + + 'private _classifyKnownFields': {}, + + /** + * Id of the first valid step + * + * Useful if early steps are used for, say, management purposes. + * + * @type {number} + */ + 'protected firstStepId': 1, + + /** + * Data API manager + * @type {DataApiManager} + */ + 'protected dapiManager': null, + + + /** + * Initialize program + * + * @param {DataApiManager} dapi_manager + */ + __construct: function( dapi_manager ) + { + if ( dapi_manager ) + { + this.dapiManager = dapi_manager; + this.dapiManager.setApis( this.apis ); + } + + try + { + this._classify = ( this.classifier ) + ? require( this.classifier ) + : function() { return {}; }; + } + catch ( e ) + { + throw Error( + "Failed to load global classifier: " + + e.message + ); + } + + this._initClasses(); + }, + + + 'private _initClasses': function() + { + var fieldc = ( this._classify.fieldClasses || {} ); + for ( var field in fieldc ) + { + // UI is considered to completely override (may change) + this.whens[ field ] = this.whens[ field ] || [ fieldc[ field ] ]; + } + + this._initKnownClassFields(); + }, + + + 'private _initKnownClassFields': function() + { + var known = this._classifyKnownFields; + + // maintain BC for the time being (old and new API, respectively) + var cfields = this._classify.knownFields + || this._classify.rater.knownFields; + + // from global classifier + for ( var f in cfields ) + { + known[ f ] = true; + } + + // from whens that reference questions in the UI directly + for ( var f in ( this.qwhens || {} ) ) + { + known[ f ] = true; + } + }, + + + /** + * Returns the program id + * + * @return String program id + */ + getId: function() + { + return this.id; + }, + + + 'abstract public initQuote': [ 'bucket', 'store_only' ], + + + submit: function( step_id, bucket, cmatch, trigger_callback ) + { + // if there are any pending data api calls, do not + // bother validating the rest of the step data + var pending_request = this._getPendingApiCall( step_id ); + + if ( pending_request !== null ) + { + return pending_request; + } + + trigger_callback = trigger_callback || function() {}; + + var callback = this.eventData[ step_id ].submit; + + // if there was no callback for this step, then we haven't anything + // to do + if ( callback === undefined ) + { + return true; + } + + return callback.call( this, bucket, {}, cmatch, trigger_callback ); + }, + + + postSubmit: function( step_id, bucket, trigger_callback ) + { + trigger_callback = trigger_callback || function() {}; + + // make sure that the step they're trying to load actually exists + if ( this.eventData[ step_id ] === undefined ) + { + return false; + } + + var callback = this.eventData[ step_id ].postSubmit; + if ( callback === undefined ) + { + return false; + } + + callback.call( this, trigger_callback, bucket ); + return true; + }, + + + /** + * Trigger processing of `forward' event + * + * @param {number} step_id step id to trigger event on + * @param {function()} trigger_callback callback for event triggers + * + * @return {Object} failures + */ + forward: function( step_id, bucket, cmatch, trigger_callback ) + { + trigger_callback = trigger_callback || function() {}; + + var callback = ( this.eventData[ step_id ] || {} ).forward; + if ( callback === undefined ) + { + return null; + } + + return callback.call( this, bucket, {}, cmatch, trigger_callback ); + }, + + + 'public change': function( step_id, name, bucket, diff, cmatch, trigger_callback ) + { + var change = ( this.eventData[ step_id ] || {} ).change; + + if ( !change || !( change[ name ] ) ) + { + return null; + } + + return change[ name ].call( this, bucket, diff, cmatch, trigger_callback ); + }, + + + 'public dapi': function( step_id, name, bucket, diff, cmatch, trigger_callback ) + { + var dapi = ( this.eventData[ step_id ] || {} ).dapi; + + if ( !dapi || !( dapi[ name ] ) ) + { + return null; + } + + return dapi[ name ].call( this, bucket, diff, cmatch, trigger_callback ); + }, + + + 'public addFailure': function( dest, name, indexes, message, cause_names ) + { + var to = dest[ name ] = dest[ name ] || []; + + for ( var i in indexes ) + { + var index = indexes[ i ], + field = BucketField( name, index ), + causes = []; + + for ( var cause_i in cause_names ) + { + causes.push( BucketField( + cause_names[ cause_i ], + index + ) ); + } + + to[ index ] = Failure( field, message, causes ); + } + }, + + + 'public action': function( + step_id, type, ref, index, bucket, trigger_callback + ) + { + var action = ( this.eventData[ step_id ] || {} ).action; + + // the double-negative-or prevents the latter from being executed + // (yielding an error) if the former fails + if ( !action || !action[ ref ] ) + { + return this; + } + + // attempt to locate this type of action for the given ref + var action = action[ ref ][ type ]; + if ( action === undefined ) + { + return this; + } + + // found it! + action.call( this, trigger_callback, bucket, index ); + return this; + }, + + + /** + * Determine if any Api Calls are still pending + * + * @param {integer} step id to get pending api calls for + * + * @return {object|null} null if none are pending otherwise message + */ + 'private _getPendingApiCall': function( step_id ) + { + if ( !this.dapiManager ) + { + return null; + } + + var changes = this.eventData[ step_id ].change, + pending = this.dapiManager.getPendingApiCalls(); + + for ( var id in pending ) + { + if ( pending[ id ] === undefined ) + { + continue; + } + + if ( pending[ id ].uid !== undefined ) + { + // no reason to check any further, return first pending + // api call + var ret_val = {}, + failed = [], + name = pending[ id ].name, + index = pending[ id ].index; + + // we only care if this data api request is associated + // to this step + if ( changes[ name ] !== undefined ) + { + this.addFailure( + failed, + name, + [ index ], + 'Question is still loading; please wait...', + [ name ] + ); + + ret_val[ name ] = failed[ name ]; + return ( ret_val ); + } + } + } + + // no pending requests + return null; + }, + + + eachChangeById: function( step_id, callback, trigger_callback ) + { + trigger_callback = trigger_callback || function() {}; + + var change = this.eventData[ step_id ].change; + + // if there's no change events, we don't need to do anything + if ( change === undefined ) + { + return this; + } + + // call the callback for each element that has a change function + var program = this; + for ( name in change ) + { + // use a closure to ensure that the variable we pass in will not be + // changed (remember, we're in a loop) + ( function( change_callback ) + { + // call the callback, passing in a callback of our own to be + // called when the change event is triggered + callback.call( program, name, function( bucket, diff, cmatch ) + { + // run the assertions and return the result + return change_callback.call( + program, bucket, diff, cmatch, trigger_callback + ); + }, +step_id ); + } )( change[name] ); + } + + return this; + }, + + + beforeLoadStep: function( step_id, bucket, trigger_callback ) + { + trigger_callback = trigger_callback || function() {}; + + // make sure that the step they're trying to load actually exists + if ( this.eventData[ step_id ] === undefined ) + { + return false; + } + + var callback = this.eventData[ step_id ].beforeLoad; + if ( callback === undefined ) + { + return false; + } + + callback.call( this, trigger_callback, bucket ); + return true; + }, + + + visitStep: function( step_id, bucket ) + { + // make sure that the step they're trying to load actually exists + if ( this.eventData[ step_id ] === undefined ) + { + return false; + } + + var callback = this.eventData[ step_id ].visit; + if ( callback === undefined ) + { + return false; + } + + callback.call( this, function() {}, bucket ); + return true; + }, + + + doAssertion: function( + assertion, qid, expected, given, success, failure, record + ) + { + var thisresult = false, + result = false; + + this._assertDepth++; + + if ( assertion.assert( expected, given ) ) + { + thisresult = true; + result = ( success ) + ? success.call( this ) + : true; + } + else + { + thisresult = false; + result = ( failure ) + ? failure.call( this ) + : false; + } + + this._assertDepth--; + + this.emit( 'assert', + assertion, qid, expected, given, thisresult, result, record, + this._assertDepth + ); + + return result; + }, + + + /** + * Classify the given bucket data + * + * @param {Object} data bucket data to classify + * + * @return {Object.>} classification results + */ + 'public classify': function( data ) + { + // maintain BC for the time being (new and old respectively); can be + // cleaned up to be less verbose once we remove compatibility + var result = ( this._classify.rater ) + ? this._classify.rater.classify.fromMap( data, false ) + : this._classify( data ); + + // add qwhens (TODO: let's not do this every single time; use a diff) + this.qwhens = this.qwhens || {}; + for ( var f in this.qwhens ) + { + var values = data[ f ], + match = [], + is = true, + expect = this.qwhens[ f ]; + + for ( var i in values ) + { + match[ i ] = +( + !!( ( values[ i ] !== '0' ) && !!values[ i ] ) + === expect + ); + + is = is && !!match[ i ]; + } + + result[ 'q:' + f ] = { + indexes: match, + is: is + }; + } + + return result; + }, + + + /** + * Checks the given indexes against classification matches + * + * If the cmatch array indicates that the given index does not match its + * required classification, then the index will be removed. This has the + * effect of ignoring indexes for fields that do not match their required + * classifications. + * + * The index array should be an array of index numbers. The values of the + * index array will be used to check the associated index of the cmatch + * array for a boolean value. + * + * @param {Array.} cmatch match array + * @param {Array.} indexes indexes to check + * + * @return {Array.} cmatch-filtered index array + */ + 'protected cmatchCheck': function( cmatch, indexes ) + { + // if there's no cmatch data for this field, or if the cmatch data + // exists but is empty (indicating a true match for any number of + // indexes) then simply return what we were given + if ( !cmatch || ( cmatch.length === 0 ) ) + { + return indexes; + } + + var ret = [], + len = indexes.length; + + // return the indexes of only the available cmatch indexes + for ( var i = 0; i < len; i++ ) + { + var index = indexes[ i ]; + + if ( cmatch[ index ] ) + { + ret.push( index ); + } + } + + return ret; + }, + + + 'public getClassifierKnownFields': function() + { + return this._classifyKnownFields; + }, + + + 'public getFirstStepId': function() + { + return this.firstStepId; + } +} ); + diff --git a/src/quote/BaseQuote.js b/src/quote/BaseQuote.js new file mode 100644 index 0000000..5b1902a --- /dev/null +++ b/src/quote/BaseQuote.js @@ -0,0 +1,663 @@ +/** + * Contains program Quote class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + * + * @todo Use ``document'' terminology in place of ``quote'' + */ + +var Class = require( 'easejs' ).Class, + Quote = require( './Quote' ), + Program = require( '../program/Program' ).Program, + EventEmitter = require( 'events' ).EventEmitter; + + +/** + * Creates a new quote + * + * TODO: This also has a bit of server-side logic; extend/decorate the class + * for use server-side and remove all the server-only logic + */ +module.exports = Class( 'BaseQuote' ) + .implement( Quote ) + .extend( EventEmitter, +{ + /** + * Raised when current step id changes + * @type {string} + */ + 'const EVENT_STEP_CHANGE': 'stepChange', + + + /** + * Quote id + * @type {number} + */ + 'private _id': 0, + + /** + * Data bucket + * @type {QuoteDataBucket} + */ + 'private _bucket': null, + + /** + * Id of the current step + * @type {number} + */ + 'private _currentStepId': 1, + + /** + * Id of the highest step that the user has reached + * @type {number} + */ + 'private _topVisitedStepId': 1, + + /** + * Program to which the quote belongs + * @type {Program} + */ + 'private _program': null, + + /** + * Date (UNIX timestamp) that the quote was started + * @type {number} + */ + 'private _startDate': 0, + + /** + * Id of agent that owns the quote + * @type {number} + */ + 'private _agentId': 0, + + /** + * Agency name + * @type {string} + */ + 'private _agentName': '', + + /** + * Whether the quote has been imported + * @type {boolean} + */ + 'private _imported': false, + + /** + * Whether the quote has been bound + * @type {boolean} + */ + 'private _bound': false, + + /** + * Quote-wide error (should invalidate quote until cleared) + * @type {string} + */ + 'private _error': '', + + /** + * Quick save data (diff format) + * @type {Object} + */ + 'private _quickSaveData': {}, + + /** + * Allows quote to be locked for any reason + * @type {string} + */ + 'private _explicitLock': '', + + /** + * Optional step associated with explicit lock + * @type {number} + */ + 'private _explicitLockStep': 0, + + + /** + * Initializes quote with the given id and bucket + * + * @return {undefined} + */ + 'public __construct': function( id, bucket ) + { + this._id = id; + this._bucket = bucket; + }, + + + /** + * Returns the quote id + * + * The quote id is immutable. A different quote id would represent a + * different quote, therefore a new object should be created with the + * desired quote id. + * + * @return {number} quote id + */ + 'public getId': function() + { + return this._id; + }, + + + /** + * Returns the bucket used to store the quote form data + * + * @return {QuoteDataBucket} + */ + 'public getBucket': function() + { + return this._bucket; + }, + + + /** + * Sets the program id to associate with the quote + * + * @param {string} id program id + * + * @return {Quote} self + */ + 'public setProgram': function( program ) + { + if ( !Class.isA( Program, program ) ) + { + throw Error( 'Program expected; given ' + program ); + } + + this._program = program; + return this; + }, + + + /** + * Returns the program id associated with the quote + * + * @return {string} program id + */ + 'public getProgramId': function() + { + return ( this._program !== null ) + ? this._program.getId() + : ''; + }, + + + /** + * Retrieve Program associated with quote + * + * @return {Program} quote program + */ + 'public getProgram': function() + { + return this._program; + }, + + + /** + * Sets the quote start date + * + * @param {number} time start date as a UNIX timestamp + * + * @return {Quote} self + */ + 'public setStartDate': function( time ) + { + this._startDate = +( time ); + return this; + }, + + + /** + * Returns the quote start date + * + * @return {number} quote start date + */ + 'public getStartDate': function() + { + return this._startDate; + }, + + + /** + * Sets id of agent that owns the quote + * + * @param {number} id agent id + * + * @return {Quote} self + */ + 'public setAgentId': function( id ) + { + this._agentId = +( id ); + return this; + }, + + + /** + * Returns the id of the agent that owns the quote + * + * @return {number} agent id + */ + 'public getAgentId': function() + { + return this._agentId; + }, + + + /** + * Sets name of agent that owns the quote + * + * @param {string} name agent name + * + * @return {Quote} self + */ + 'public setAgentName': function( name ) + { + this._agentName = ''+( name ); + return this; + }, + + + /** + * Returns the name of the agent that owns the quote + * + * @return {string} agent name + */ + 'public getAgentName': function() + { + return this._agentName; + }, + + + /** + * Sets quote imported status + * + * Represents whether the quote has been imported into our agency management + * system. + * + * @param {boolean} value true if imported, otherwise false + * + * @return {Quote} self + */ + 'public setImported': function( value ) + { + this._imported = !!value; + this._needsImport = false; + + return this; + }, + + + /** + * Returns whether the quote has been imported + * + * @return {boolean} true if imported, otherwise false + */ + 'public isImported': function() + { + return this._imported; + }, + + + /** + * Sets quote bound status + * + * Represents whether the quote has been bound + * + * @param {boolean} value true if bound, otherwise false + * + * @return {Quote} self + */ + 'public setBound': function( value ) + { + this._bound = !!value; + + return this; + }, + + + /** + * Returns whether the quote has been bound + * + * @return {boolean} true if bound, otherwise false + */ + 'public isBound': function() + { + return this._bound; + }, + + + /** + * Returns the id of the current step + * + * @return {number} id of current step + */ + 'public getCurrentStepId': function() + { + return this._currentStepId; + }, + + + /** + * Sets the top visited step id + * + * If the provided step id is less than the current step, then the current + * step id is used instead. + * + * @return {Quote} self + */ + 'public setTopVisitedStepId': function( step_id ) + { + step_id = +step_id; + + this._topVisitedStepId = ( step_id < this._currentStepId ) + ? this._currentStepId + : step_id; + + return this; + }, + + + /** + * Returns the id of the highest step the quote has reached + * + * @return {number} top visited step id + */ + 'public getTopVisitedStepId': function() + { + return this._topVisitedStepId; + }, + + + /** + * Sets the current step id + * + * @param {number} step_id id of the step to set + * + * @return {Quote} self + */ + 'public setCurrentStepId': function( step_id ) + { + step_id = +step_id; + + this._currentStepId = step_id; + + // if this step is higher than the highest step this quote has reached, + // then update it + if ( step_id > this._topVisitedStepId ) + { + this._topVisitedStepId = step_id; + } + + this.emit( this.__self.$('EVENT_STEP_CHANGE'), this._currentStepId ); + return this; + }, + + + 'public getTopSavedStepId': function() + { + return this._topSavedStepId; + }, + + + 'public setTopSavedStepId': function( id ) + { + this._topSavedStepId = +id; + return this; + }, + + + /** + * Returns whether the step has been previously visited + * + * @param {number} step_id id of the step to check + * + * @return {boolean} true if visited, otherwise false + */ + 'public hasVisitedStep': function( step_id ) + { + if ( step_id <= 0 ) + { + return false; + } + + return ( step_id <= this.getTopVisitedStepId() ) ? true : false; + }, + + + /** + * Sets quote data + * + * The data will be merged, not overwritten. + * + * @param {Object.} data data to set on the quote + * + * @return {Quote} self + */ + 'public setData': function( data ) + { + this._bucket.setValues( data ); + return this; + }, + + + /** + * Returns whether the quote should be locked from modifications + * + * If an explicit lock is set, then we shall only consider the quote to be + * in a locked state if there is no explicit step restriction on the lock + * (since otherwise the user may access the unlocked steps). + * + * If a quote is imported, it will not be considered locked if there is an + * explicit step restriction set; this permits users to modify imported + * quotes, if such an ability should be granted. + * + * If the quote is bound, then it is locked, full stop. + * + * @return {boolean} true if locked, otherwise false + */ + 'public isLocked': function() + { + var exlock = ( this._explicitLock !== '' ), + slock = ( this._explicitLockStep !== 0 ), + ilock = ( ( this._imported && !slock ) || this._bound ); + + // we are locked if we (a) have the import/bind lock or (b) have an + // exclusive lock without a step constraint + return ilock || ( exlock && !slock ); + }, + + + /** + * Returns quote data + * + * @param {string} name name of data to retrieve + * + * @return {Array} quote data + */ + 'public getDataByName': function( name ) + { + return this._bucket.getDataByName( name ); + }, + + + /** + * Calls visitor callback with the data bucket + * + * todo: this pretty much breaks encapsulation, so ultimately we won't want + * to send in the actual bucket + * + * @param {function( QuoteDataBucket )} callback visitor + * + * @return {Quote} self + */ + 'public visitData': function( callback ) + { + callback.call( this, this._bucket ); + return this; + }, + + + /** + * Sets a quote-wide error + * + * This error should invalidate the entire quote until it is cleared. + * + * @param {string} error error string + * + * @return {Quote} self + */ + 'public setError': function( error ) + { + this._error = ''+( error ); + return this; + }, + + + /** + * Retrieve quote-wide error + * + * @return {string} quote-wide error, or empty string + */ + 'public getError': function() + { + return this._error; + }, + + + /** + * Determine whether or not a quote-wide error exists + * + * Use getError() to retrieve the actual error string. + * + * @return {boolean} true if error exists, otherwise false + */ + 'public hasError': function() + { + return ( this._error !== '' ); + }, + + + /** + * Set quicksave data + * + * @param {Object} data quicksave data, diff format + * + * @return {Quote} self + */ + 'public setQuickSaveData': function( data ) + { + this._quickSaveData = data; + return this; + }, + + + /** + * Retrieve quicksave data + * + * @return {Object} quicksave data + */ + 'public getQuickSaveData': function() + { + return this._quickSaveData; + }, + + + /** + * Sets an explicit lock, providing a reason for doing so + * + * @param {string} reason lock reason + * @param {number} step step that user may not navigate prior + * + * @return {Quote} self + */ + 'public setExplicitLock': function( reason, step ) + { + step = +step || 0; + + this._explicitLock = ''+( reason ); + this._explicitLockStep = step; + + return this; + }, + + + /** + * Clears an explicit lock + * + * @return {Quote} self + */ + 'public clearExplicitLock': function() + { + this._explicitLock = ''; + this._explicitLockStep = 0; + + return this; + }, + + + /** + * Retrieves the reason for an explicit lock + * + * @return {string} lock reason + */ + 'public getExplicitLockReason': function() + { + return ( this.isBound() ) + ? 'Quote has been bound' + : this._explicitLock; + }, + + + /** + * Returns the maximum step to which the explicit lock applies + * + * If no step restriction is set, then 0 will be returned. + * + * @return {number} locked max step or 0 if not applicable + */ + 'public getExplicitLockStep': function() + { + return ( this.isBound() ) + ? 0 + : this._explicitLockStep; + }, + + + /** + * Determine whether quote needs to be imported + * + * Bound quotes will never need importing. + * + * @param {boolean=} set flag value + */ + 'public needsImport': function( set ) + { + if ( set !== undefined ) + { + this._importDirty = !!set; + return this; + } + + return ( this.isBound() ) + ? false + : this._importDirty; + } +} ); + diff --git a/src/quote/Quote.js b/src/quote/Quote.js new file mode 100644 index 0000000..9680aca --- /dev/null +++ b/src/quote/Quote.js @@ -0,0 +1,31 @@ +/** + * Generic interface to represent quotes + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + * + * @todo Use ``document'' terminology in place of ``quote'' + */ + +var Interface = require( 'easejs' ).Interface; + + +module.exports = Interface( 'Quote', +{ + /** TODO **/ +} ); + diff --git a/src/quote/README b/src/quote/README new file mode 100644 index 0000000..95d1e14 --- /dev/null +++ b/src/quote/README @@ -0,0 +1,7 @@ +Quote => Document +================= + +The original terminology for a document containing data is ``quote'', which +represents what Liza was designed for. To generalize this, it should be +changed to ``document'', as in a database document. + diff --git a/src/server/Server.js b/src/server/Server.js new file mode 100644 index 0000000..e768378 --- /dev/null +++ b/src/server/Server.js @@ -0,0 +1,1800 @@ +/* + * Contains program Server class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * @todo like Client and Ui, this mammoth did not evolve well and has too + * many responsibilities; refactor + */ + +const { Class } = require( 'easejs' ); +const { EventEmitter } = require( 'events' ); + +const fs = require( 'fs' ); +const util = require( 'util' ); + +const { + bucket: { + bucket_filter, + QuoteDataBucket, + BucketSiblingDescriptor, + + diff: { + StdBucketDiffContext, + GroupedBucketDiffContext, + StdBucketDiffResult, + GroupedBucketDiffResult, + StdBucketDiff, + }, + }, + + field: { + FieldClassMatcher, + }, + + server: { + encsvc: { + QuoteDataBucketCipher, + }, + }, + + util: { + ShallowArrayDiff, + }, +} = require( '..' ); + + +module.exports = Class( 'Server' ) + .extend( EventEmitter, +{ + 'private response': null, + + /** + * Dao + * @type {MongoServerDao} + */ + 'private dao': null, + + /** + * Stores references to program objects + * @type {Object} + */ + 'private programs': {}, + + 'private quoteFillHooks': [], + + /** + * Logger to use + * @type {PriorityLog} + */ + 'private logger': null, + + /** + * Default bucket data for various programs + * @type {Object} + */ + 'private _defaultBuckets': {}, + + /** + * Encryption service client + * @type {EncryptionService} + */ + 'private _encService': null, + + /** + * Holds bucket ciphers for each program + * @type {Object.} + */ + 'private _bucketCiphers': {}, + + /** + * Step and program cache + * + * @type {Store} + */ + 'private _cache': null, + + + 'public __construct': function( response, dao, logger, encsvc ) + { + this.response = response; + this.dao = dao; + this.logger = logger; + this._encService = encsvc; + }, + + + 'public init': function( cache, rater ) + { + this._cache = cache; + + this._initDb(); + this.reload( rater ); + }, + + + 'public reload': function( rater ) + { + var _self = this; + + rater.init( + // log + function( msg ) + { + _self.logger.log( _self.logger.PRIORITY_IMPORTANT, msg ); + }, + + // error + function( msg, stack ) + { + stack = stack || ''; + + _self.logger.log( _self.logger.PRIORITY_ERROR, + "%s\n-!%s", + msg, + stack.replace( /\n/g, '\n-!' ) + ); + } + ); + }, + + + /** + * Initializes the database (attempt DAO connection) + * + * @return undefined + */ + 'private _initDb': function() + { + var server = this; + + // error listeners + this.dao.on( 'connectError', function( err ) + { + server.logger.log( server.logger.PRIORITY_ERROR, + "Database connection failure: (%s) %s", + err.errno || '-', + err.message || '' + ); + + // attempt to reconnect every 5 seconds + setTimeout( function() + { + server.logger.log( server.logger.PRIORITY_DB, + "[Server] Attempting to reconnect to database..." + ); + server.dao.connect(); + }, 5000 ); + }).on( 'saveQuoteError', function( err, quote ) + { + server.logger.log( server.logger.PRIORITY_ERROR, + "Failed to save quote %d: %s", + quote.getId(), + err.message || '' + ); + }).on( 'seqError', function( err ) + { + server.logger.log( server.logger.PRIORITY_ERROR, + "Sequence error: %s", + err + ); + }).on( 'seqInit', function( seq ) + { + server.logger.log( server.logger.PRIORITY_DB, + "Initialized default sequence: %s", + seq + ); + }).on( 'ready', function() + { + server.logger.log( server.logger.PRIORITY_DB, + "[Server] Connected to database; DAO ready" + ); + }); + + server.logger.log( server.logger.PRIORITY_DB, + "[Server] Connecting to database..." + ); + + this.dao.init(); + }, + + + sendResponse: function( request, quote, data, action ) + { + request.end( this.response.from( quote, data, action ) ); + return this; + }, + + + sendError: function( request, message, action, btn_caption ) + { + request.end( this.response.error( message, action, btn_caption ) ); + return this; + }, + + + /** + * Initializes a quote with any existing quote data + * + * @return Server self to allow for method chaining + */ + initQuote: function( quote, program, request, callback, error_callback ) + { + var server = this, + quote_id = quote.getId(), + session = request.getSession(), + agent_id = session.agentId(), + agent_name = session.agentName(); + + // get the data for this quote + this.dao.pullQuote( quote_id, function( quote_data ) + { + var new_quote = false; + if ( !( quote_data ) ) + { + quote_data = {}; + new_quote = true; + + // ensure it's a valid quote id + server.dao.getMinQuoteId( function( min_id ) + { + // don't allow before the min quote id + if ( quote_id < min_id ) + { + error_callback.call( server ); + return; + } + + server.dao.getMaxQuoteId( function( max_id ) + { + if ( quote_id > max_id ) + { + // we has a problem + error_callback.call( server ); + return; + } + + // we're good + init_finish( program ); + }); + }); + } + else + { + // quote is not new; just continue + server.getProgram( quote_data.programId ) + .then( function( quote_program ) + { + init_finish( quote_program ); + } ); + } + + function init_finish( quote_program ) + { + // fill in the quote data (with reasonable defaults if the quote + // does not yet exist); IMPORTANT: do not set pver to the + // current version here; the quote will be repaired if it is not + // set + quote + .setData( + server._getDefaultBucket( quote_program, quote_data ) + ) + .setQuickSaveData( quote_data.quicksave || {} ) + .setAgentId( quote_data.agentId || agent_id ) + .setAgentName( quote_data.agentName || agent_name ) + .setStartDate( + quote_data.getStartDate + || Math.round( new Date().getTime() / 1000 ) + ) + .setImported( quote_data.importedInd || false ) + .setBound( quote_data.boundInd || false ) + .needsImport( quote_data.importDirty || false ) + .setCurrentStepId( + quote_data.currentStepId + || quote_program.getFirstStepId() + ) + .setTopVisitedStepId( + quote_data.topVisitedStepId + || quote_program.getFirstStepId() + ) + // it is important that we set this to top visited to + // ensure that (a) they cannot init a quote and skip the + // first step and (b) that older quotes without this field + // are properly initialized + .setTopSavedStepId( + quote_data.topSavedStepId + || ( quote.getTopVisitedStepId() - 1 ) + ) + .setProgram( quote_program ) + .setProgramVersion( quote_data.pver || '' ) + .setExplicitLock( + ( quote_data.explicitLock || '' ), + ( quote_data.explicitLockStepId || 0 ) + ) + .setError( quote_data.error || '' ) + .setCreditScoreRef( quote_data.creditScoreRef || 0 ) + .setLastPremiumDate( quote_data.lastPremDate || 0 ) + .setRatedDate( quote_data.initialRatedDate || 0 ) + .on( 'stepChange', function( step_id ) + { + // save the quote state (we don't care if it succeeds or + // fails because (a) failures will be automatically + // logged and (b) we may not be dealing with a request, + // so we may not be able to send a response to the + // client) + server.dao.saveQuoteState( quote ); + }); + + // if no data was returned, then the quote doesn't exist in the + // database + if ( new_quote ) + { + // initialize it + server.dao.saveQuote( quote, null, null, { + agentId: agent_id, + agentName: agent_name, + agentEntityId: session.agentEntityId(), + startDate: quote.getStartDate(), + programId: quote.getProgramId(), + initialRatedDate: 0, + importedInd: ( quote.isImported() ) ? 1 : 0, + boundInd: ( quote.isBound() ) ? 1 : 0, + importDirty: 0, + syncInd: 0, + boundInd: 0, + notifyInd: 0, + syncDate: 0, + lastPremDate: 0, + internal: ( session.isInternal() ) ? 1: 0, + pver: program.version, + + explicitLock: quote.getExplicitLockReason(), + explicitLockStepId: quote.getExplicitLockStep(), + } ); + } + + callback.call( server ); + } + }); + + return this; + }, + + + 'private _checkQuotePver': function( quote, program, callback ) + { + // note that if program.version is not set, then something is likely + // wrong with the build that generates it: always clean in this case to + // be safe + if ( program.version + && ( quote.getProgramVersion() === program.version ) + ) + { + callback( false, false ); + return; + } + + var _self = this; + + this.logger.log( this.logger.PRIORITY_INFO, + 'Quote %s program version change (%s -> %s); will be scanned.', + quote.getId(), + quote.getProgramVersion(), + program.version + ); + + // TODO: thread + // service any other requests first, and then proceed to cleaning + process.nextTick( function() + { + var nwait = 0, + msg = [], + handled = false; + + // by default, clear is undefined; event handlers should call the + // appropriate function to state whether the quote has been properly + // upgraded; if no handlers indicate success, or if any indiciate + // failure, then disallow servicing the quote + var clear = undefined, + event = { + good: function() + { + // if undefined, then we're good, otherwise keep the + // existing value (we cannot override bad) + clear = ( clear === undefined ) ? true : clear; + }, + + bad: function( s ) + { + // bad trumps all + clear = false; + msg.push( s ); + }, + + wait: function() + { + nwait++; + return c; + } + }; + + // trigger the event and let someone (hopefully) take care of this + try + { + _self.emit( 'quotePverUpdate', quote, program, event ); + } + catch ( e ) + { + // ruh roh... + event.bad( e.message ); + nwait = 0; + + // this is an unhandled exception, as far as we're concerned; + // re-throw so that we have a stack trace, but do so after we're + // done processing + process.nextTick( function() + { + throw e; + } ); + } + + function c() + { + // do nothing until we're done waiting + if ( --nwait > 0 ) + { + return; + } + + if ( clear === true ) + { + // clear for version update + quote.setProgramVersion( program.version ); + } + else + { + // default message + if ( msg.length === 0 ) + { + msg.push( ''+clear ); + } + + // see comments for clear var above + _self.logger.log( _self.logger.PRIORITY_ERROR, + 'Quote %s scan failed (' + msg.join( '; ' ) + ')', + quote.getId() + ); + } + + if ( !handled ) + { + handled = true; + callback( !clear, true ); + } + } + + // if nothing has requested that we wait, then continue immediately + if ( !handled && ( nwait === 0 ) ) + { + handled = true; + c(); + } + } ); + }, + + + /** + * Generates default bucket data for the given program + * + * @return {Object} default bucket data + */ + 'private _getDefaultBucket': function( program, quote_data ) + { + var defaults = program.defaults, + bucket = quote_data.data || {}, + pre = this._defaultBuckets[ program.getId() ]; + + // we only want to merge in the defaults if this is the first visit to + // the quote + if ( quote_data.currentStepId > 0 ) + { + // todo: uncomment later; for now we want older quotes to still work + //return bucket; + } + + // if we already generated the default bucket data and have no + // quote-specific data, return it + if ( pre && ( quote_data.data === undefined ) ) + { + return pre; + } + + // generate + for ( item in program.defaults ) + { + if ( bucket[ item ] === undefined ) + { + bucket[ item ] = [ defaults[ item ] ]; + } + } + + // set as default bucket only if we didn't merge + if ( quote_data.data === undefined ) + { + this._defaultBuckets[ program.getId() ] = bucket; + } + + return bucket; + }, + + + /** + * Sends a new quote initialization request to the client + * + * @param {HttpServerRequest} request + * @param {Function} quote_new function to create new quote + * + * @return {Server} self + */ + sendNewQuote: function( request, quote_new ) + { + var server = this, + session = request.getSession(); + + function donew( quote_id ) + { + var quote = quote_new( quote_id ); + server.sendResponse( request, quote, { valid: false } ); + } + + // should we override the quote id? + var rqn; + if ( ( rqn = session.getReturnQuoteNumber() ) > 0 ) + { + donew( rqn ); + + // we don't need to wait for this to finish, since the next request + // won't be for a new quote + session.clearReturnQuoteNumber(); + } + else + { + // get the next available quote id + this.dao.getNextQuoteId( donew ); + } + + return this; + }, + + + sendInit: function( request, quote, program, quote_new, prev ) + { + var _self = this, + args = arguments; + + this._checkQuotePver( quote, program, function( err, mod ) + { + if ( err ) + { + // return as fatal + _self.sendError( request, "Quote sanitization failed" ); + return; + } + + // save the quote updates (but only if it was modified) + if ( mod ) + { + _self.dao.saveQuote( quote, + function() + { + _self._processInit.apply( _self, args ); + }, + function() + { + _self.sendError( request, + "Quote sanitization failed to commit" + ); + } + ); + } + else + { + _self._processInit.apply( _self, args ); + } + } ); + + return this; + }, + + + /** + * Sends /init response + * + * @param UserRequest request + * @param Quote quote + * @param {Program} program + * @param {Function} quote_new function to create new quote + * + * @return Server self to allow for method chaining + * + * @todo generate quote # rather than prompting + */ + _processInit: function( request, quote, program, quote_new, prev ) + { + var actions = null, + valid = true, + program_id = program.getId(), + session = request.getSession(), + internal = session.isInternal(); + + // if no quote id was given, simply prompt for one for now + if ( quote.getId() == 0 ) + { + this.sendNewQuote( request, quote_new ); + return this; + } + else if ( quote.getProgramId() !== program_id ) + { + // invalid program; change the program id + actions = [ { + action: 'setProgram', + id: quote.getProgramId(), + quoteId: quote.getId(), + } ]; + + valid = false; + } + else if ( quote.hasError() ) + { + this.sendError( request, quote.getError() ); + return this; + } + // ensure that the agent id matches the quote's agent (unless internal) + else if ( ( internal === false ) + && ( request.getSession().agentId() != quote.getAgentId() ) + ) + { + // todo: generate a new quote # + actions = [ { action: 'quotePrompt' } ]; + valid = false; + } + + // don't return any quote data if invalid - we don't want people spying + // on the data! + if ( valid === false ) + { + this.sendResponse( request, quote, { + valid: false, + }, actions ); + + return this; + } + + var data = quote.getBucket().getData() || {}; + + // if we're not internal, filter out the internal questions from the + // data array to ensure that they can't spy on our internal data + if ( request.getSession().isInternal() === false ) + { + for ( id in program.internal ) + { + delete data[ id ]; + } + } + + var bucket = quote.getBucket(), + lock = quote.getExplicitLockReason(), + lock_step = quote.getExplicitLockStep(), + _self = this; + + if ( valid && !lock && prev ) + { + actions = [ { + action: 'warning', + message: ( + 'Somebody else is currently viewing this quote; it ' + + 'has been locked and will be read-only until the ' + + 'other person is finished. Please try again later.' + + + ( ( internal ) + ? '

N.B.: You are an internal user, so you ' + + 'may unlock the quote above, but be warned that ' + + 'concurrent writes may have negative affects on ' + + 'the integrity of the quote.' + + '

' + + 'Currently viewing: ' + prev + : '' + ) + ) + } ]; + + lock = 'concurrent-access'; + } + + // decrypt bucket contents, if necessary, and return + this._getBucketCipher( program ).decrypt( bucket, function() + { + _self.sendResponse( request, quote, { + valid: valid, + data: bucket.getData() || {}, + + currentStepId: quote.getCurrentStepId(), + topVisitedStepId: quote.getTopVisitedStepId(), + imported: quote.isImported(), + bound: quote.isBound(), + needsImport: quote.needsImport(), + explicitLock: lock, + explicitLockStepId: lock_step, + agentId: quote.getAgentId(), + agentName: quote.getAgentName(), + + quicksave: quote.getQuickSaveData(), + + // set to undefined if not internal so it's not included in the + // JSON response + internal: ( ( request.getSession().isInternal() === true ) + ? true + : undefined + ), + }, actions ); + } ); + + return this; + }, + + + /** + * Sends a step to the client + * + * @param UserRequest request + * @param Quote quote + * @param Integer program + * @param Integer step_id id of the step + * + * @return void + */ + sendStep: function( request, quote, program, step_id, session ) + { + var cur_id = quote.getCurrentStepId(), + saved_id = quote.getTopSavedStepId(), + program_id = program.id; + + if ( program.steps[ step_id ] === undefined ) + { + this.sendError( request, + "Invalid step request; step " + step_id + " does not exist.", + [ { action: 'gostep', id: cur_id } ] + ); + + return; + } + + var type = program.steps[ step_id ].type; + + // is this a management step? if so, we must be internal + if ( ( type === 'manage' ) && ( !session.isInternal() ) ) + { + // we're not internal, so let's send them back to the first step + this.sendResponse( request, quote, {}, [ + { action: 'gostep', id: program.getFirstStepId() } + ] ); + } + + // are they permitted to navigate to this step? + if ( step_id > ( quote.getTopVisitedStepId() + 1 ) ) + { + // knock them back to the next step they're able to save + var tostep_id = ( quote.getTopSavedStepId() + 1 ); + + this.logger.log( this.logger.PRIORITY_ERROR, + "Quote " + quote.getId() + " has not yet reached step " + + step_id + "; forcing to step " + tostep_id + ); + + this.sendError( request, + "Failed to navigate to step: you have not yet reached " + + "the requested step.", + [ { action: 'gostep', id: tostep_id } ] + ); + + return; + } + + // perform forward-validations *on the current step* to ensure that they + // cannot leave the quote and then return, requesting a future step (if + // permitted), thereby evading client-side forward-validations + if ( step_id > cur_id ) + { + if ( this._forwardValidate( quote, program, cur_id ) === false ) + { + this.sendError( request, + "The previous step contains errors; please correct them " + + "before continuing.", + [ { action: 'gostep', id: cur_id } ] + ); + return; + } + } + + var server = this; + + this._cache.get( 'step_html' ) + .then( prog => prog.get( program_id ) ) + .then( shtml => shtml.get( step_id ) ) + .then( data => + { + // send the step HTML to the client + server.sendResponse( request, quote, { + html: data, + } ); + } ) + .catch( err => + { + server.logger.log( server.logger.PRIORITY_ERROR, + "Failed to load program '%s' step %d: %s", + program_id, + step_id, + err.message + ); + + server.sendError( request, + 'The step you requested could not be loaded.' + ); + + throw err; + } ); + + return this; + }, + + + /** + * Step HTML cache miss function + * + * Load step HTML from disk. This is intended to be used as a + * miss function. + * + * TODO: Extract method + * + * @param {string} program_id program containing step + * @param {number} step_id step to load + * + * @return {Promise} + */ + 'public loadStepHtml': function( program_id, step_id ) + { + var step_filename = + process.env.LV_ROOT_PATH + '/src/_gen/views/scripts/quote/' + + program_id + '/steps/' + step_id + '.phtml'; + + return new Promise( function( resolve, reject ) + { + fs.readFile( step_filename, 'utf-8', function( err, data ) + { + // we had a problem with the step + if ( err !== null ) + { + reject( err ); + return; + } + + resolve( data ); + }); + } ); + }, + + + /** + * Perform forward-validations for a given quote and step + * + * This check is necessary to ensure that the client-side events are not + * bypassed, which is realatively simple to do. For example, one could leave + * the quote and return at a future step (so long as the operation is + * otherwise permitted), preventing the `forward' event from triggering on + * the client (as it is a relative event). + * + * @param {Quote} quote quote to forward-validate + * @param {Program} program program to validate against + * @param {number} step_id id of current step (before navigation) + * + * @return {boolean} validation success/failure + */ + 'private _forwardValidate': function( quote, program, step_id ) + { + var success = false, + _self = this; + + // TODO: we need cmatch data to pass to `forward' + return true; + + quote.visitData( function( bucket ) + { + try + { + // forward event returns an object containing failures + success = ( program.forward( step_id, bucket, {} ) === null ); + } + catch ( e ) + { + // this should never happen, but in case it does, we need to + // make sure the user isn't left hanging with no response from + // the server; return gracefully after logging the error + _self.logger_log( + _self.logger.PRIORITY_ERROR, + 'Forward-validation error (%s): WEB#%s, step %s', + program.id, + quote.getId(), + step_id + ); + } + } ); + + // N.B.: defaults to false above + return success; + }, + + + visitStep: function( step_id, request, quote ) + { + // update the quote step, if valid + if ( step_id <= ( quote.getTopVisitedStepId() + 1 ) ) + { + quote.setCurrentStepId( step_id ); + } + + this.sendResponse( request, quote, {} ); + return this; + }, + + + sendProgramJs: function( request, program_id ) + { + var server = this; + + this._cache.get( 'program_js' ) + .then( progjs => progjs.get( program_id ) ) + .then( data => + { + // send the JS to the client + request.setContentType( 'text/javascript' ).end( data ); + } ) + .catch( err => + { + server.logger.log( server.logger.PRIORITY_ERROR, + "Failed to load program '%s' JS: %s", + program_id, + err + ); + + server.sendError( request, + 'Unable to retrieve program data' + ); + + throw err; + } ); + + return this; + }, + + + /** + * Program JS cache miss function + * + * Loads program JS from disk. This is intended to be used as a + * miss function. + * + * TODO: Extract method + * + * @param {string} program_id program to load + * + * @return {Promise} + */ + 'public loadProgramFiles': function( program_id ) + { + var root_path = process.env.LV_ROOT_PATH + '/src/_gen/scripts/program/' + program_id, + js_filename = root_path + '/Program.js', + inc_filename = root_path + '/include.js', + retjs = ''; + + return new Promise( function( resolve, reject ) + { + // read both files + fs.readFile( js_filename, 'utf8', function( err, data ) + { + if ( err !== null ) + { + reject( err ); + return; + } + + // wrap in closure + data = "(function(require,module){" + + "var exports=module.exports={};" + + data + + "\n})(require,modules['program/" + program_id + + "/Program']={});" + + retjs = data; + + // read include file + fs.readFile( inc_filename, 'utf8', function( err, data ) + { + if ( err === null ) + { + retjs += data; + } + + // we have all of our data; return the result + resolve( retjs ); + } ); + } ); + } ); + }, + + + /** + * Handles a quote data post + * + * This function is called when an HTTP POST is made to save quote data. + * + * @param Integer step_id id of the step + * @param UserRequest request request object + * @param Quote quote instance of quote to operate on + * @param Program program program associated with the quote + * + * @param {UserSession} session user session + * + * @return undefined + */ + handlePost: function( step_id, request, quote, program, session ) + { + var server = this; + + // do not allow quote modification if locked unless logged in as an + // internal user (FS#5772) and the program is unlockable + if ( ( + quote.isLocked() + || ( step_id < quote.getExplicitLockStep() ) + ) + && !( session.isInternal() && program.unlockable ) + ) + { + server.logger.log( server.logger.PRIORITY_INFO, + "Cannot save imported quote: %s", + quote.getId() + ); + + server.sendError( request, + "This quote has been locked and can no longer be modified." + ); + + return this; + } + + // are they getting too far ahead of themselves? + if ( step_id > ( quote.getTopSavedStepId() + 1 ) ) + { + // knock back to next step that they're able to save + var tostep_id = ( quote.getTopSavedStepId() + 1 ); + + this.logger.log( this.logger.PRIORITY_ERROR, + "Quote " + quote.getId() + " cannot yet save step " + + step_id + "; forcing to step " + tostep_id + ); + + this.sendError( request, + "Unable to save step: you have not yet reached " + + "the requested step.", + [ { action: 'gostep', id: tostep_id, title: 'Go Back' } ] + ); + + // prohibit save + return this; + } + + + request.getPostData( function( post_data ) + { + // fill the quote with the posted data + if ( post_data.data ) + { + try + { + var filtered = server._sanitizeBucketData( + post_data.data, request, program + ); + + quote.setData( filtered ); + + // calculated values (store only) + program.initQuote( quote.getBucket(), true ); + } + catch ( err ) + { + server.logger.log( server.logger.PRIORITY_ERROR, + "Invalid POST data string (%s): %s", + err, + post_data.data + ); + + server.sendError( request, + 'There was an error saving your data. Please ' + + 'try again.', + [ { action: 'gostep', id: step_id, title: 'Go Back' } ] + ); + + return this; + } + } + + // save the quote + server._doQuoteSave( step_id, request, quote, program ); + }); + + return this; + }, + + + /** + * Sanitize the given bucket data + * + * Ensures that we are storing only "correct" data within our database. This + * also strips any unknown bucket values, preventing users from using us as + * their own personal database. + */ + 'private _sanitizeBucketData': function( + bucket_data, request, program, permit_null + ) + { + var data = JSON.parse( bucket_data ), + types = program.meta.qtypes, + ignore = {}; + + // if we're not internal, filter out the internal questions + // (so they can't post to them) + if ( request.getSession().isInternal() === false ) + { + for ( id in program.internal ) + { + ignore[ id ] = true; + } + } + + // return the filtered data + bucket_filter.filter( data, types, ignore, permit_null ); + }, + + + 'private _doQuoteSave': function( step_id, request, quote, program, c ) + { + var server = this; + + // whenever they save, we want to make sure we invalidate the premium, + // unless this is a rating step + if ( ( program.rateSteps || [] )[ step_id ] !== true ) + { + quote.setLastPremiumDate( 0 ); + } + + server.quoteFill( quote, step_id, + // success + function() + { + // encrypt bucket + var bucket = quote.getBucket(); + server._getBucketCipher( program ).encrypt( bucket, function() + { + // as a precaution to prevent navigation burps, update the + // step if it's greater than the previous + if ( step_id > quote.getTopVisitedStepId() ) + { + quote.setCurrentStepId( step_id ); + } + if ( step_id > quote.getTopSavedStepId() ) + { + // only updated by saveQuoteState + quote.setTopSavedStepId( step_id ); + server.dao.saveQuoteState( quote ); + } + + server.dao.saveQuote( quote, + // quote was saved successfully + function() + { + server._postSubmit( + request, quote, step_id, program, + request.getSession().isInternal() + ); + + c && c( true ); + }, + // failed to save the quote + function() + { + // todo: option to allow them to try again + server.sendError( request, + 'There was a problem saving your quote. ' + + 'The previous step was not saved!' + ); + + c && c( false ); + } + ); + } ); + }, + // failure + function( failures ) + { + // todo: detailed logging (this shouldn't happen) + server.logger.log( server.logger.PRIORITY_ERROR, + "Server-side quote data validation failure for " + + "quote #%s, program %s, step %d:\n%s", + quote.getId(), + program.id, + step_id, + util.inspect( failures ) + ); + + server.sendError( request, + 'There was a problem with the data you entered. ' + + 'Please click "Go Back" below to go back to the ' + + 'previous step and correct the errors.', + [ + { action: 'gostep', id: step_id }, + { action: 'invalidate', errors: failures }, + ], + 'Go Back' + ); + + c && c( false ); + } + ); + }, + + + 'private _postSubmit': function( request, quote, step_id, program, internal ) + { + var server = this, + actions = [], + bucket = null; + + // XXX + quote.visitData( function( b ) + { + bucket = b; + } ); + + var result = program.postSubmit( + step_id, + bucket, + function( event, question_id, value ) + { + switch ( event ) + { + // kick back to the given step, if they're already past it + case 'kickBack': + var step_id = +value; + + // clear any fields scheduled to be cleared on kickback + var retdata = server._kbclear( program, quote ); + + if ( quote.getTopVisitedStepId() > step_id ) + { + quote.setTopVisitedStepId( step_id ); + + // knock them back to the step if they're currently + // further + if ( quote.getCurrentStepId() > step_id ) + { + quote.setCurrentStepId( step_id ); + actions.push( { + action: 'gostep', + id: step_id, + } ); + + } + } + + server.dao.mergeBucket( quote, retdata, function() + { + server.dao.saveQuoteState( quote, function() + { + // if we're not internal, strip any potential + // internal data from the response + // XXX: maybe we should do this in + // sendResponse() to ensure consistency + if ( internal === false ) + { + for ( id in program.internal ) + { + delete retdata[ id ]; + } + } + + // don't send the response until the state + // is saved; we don't want a race condition + // if they're speeding through steps! + finish( retdata ); + } ); + } ); + + break; + + default: + server.logger.log( server.logger.PRIORITY_ERROR, + "Unknown postSubmit event: %s", + event + ); + + finish(); + return; + } + + function finish( data ) + { + data = data || {}; + server.sendResponse( request, quote, data, actions ); + } + } + ); + + // if there's no events, then just respond with a generic OK + if ( result === false ) + { + server.sendResponse( request, quote ); + } + }, + + + 'private _kbclear': function( program, quote ) + { + var set = {}; + + for ( var field in program.kbclear ) + { + var data = quote.getDataByName( field ), + val = ( program.defaults[ field ] || '' ); + + for ( var i in data ) + { + data[ i ] = val; + } + + set[ field ] = data; + } + + quote.setData( set ); + + // return the fields that have changed + return set; + }, + + + quoteFill: function( data, step_id, success, failure ) + { + if ( data instanceof Function ) + { + this.quoteFillHooks.push( data ); + return this; + } + + var abort = false, + failures = {}; + + var event = { + abort: function( failure_data ) + { + failures = failure_data; + abort = true; + }, + }; + + var len = this.quoteFillHooks.length; + for ( var i = 0; i < len; i++ ) + { + this.quoteFillHooks[i].call( event, data, step_id ); + + // if we aborted, there's no need to continue + if ( abort ) + { + break; + } + } + + // only call the callback if we did not abort + if ( abort ) + { + failure.call( this, failures ); + } + else + { + success.call( this ); + } + + return this; + }, + + + /** + * Lazily loads and returns the requested Program object + * + * @param String program_id id of the program to retrieve + * + * @return Server self to allow for method chaining + */ + 'public getProgram': function( program_id ) + { + var _self = this; + + return this._cache.get( 'program' ) + .then( pcache => pcache.get( program_id ) ) + .catch( function( e ) + { + // looks like it doesn't exist + _self.logger.log( _self.logger.PRIORITY_ERROR, + "Program class '%s' could not be loaded: %s", + program_id, + e.message + ); + + throw e; + } ); + }, + + + /** + * Program object cache miss function + * + * Instantiates program. This is intended to be used as a miss + * function. + * + * TODO: Extract method + * + * @param {string} program_id program to instantiate + * + * @return {Promise} + */ + 'public loadProgram': function( program_id ) + { + var server = this; + + return new Promise( function( resolve, reject ) + { + try + { + const program_path = 'program/' + program_id + '/Program'; + + // node caches modules; make sure it's cleared + delete require.cache[ + require.resolve( program_path ) + ]; + + // attempt to load the program class + const program_module = require( program_path ); + const program = program_module(); + + // hook ourselves + server.quoteFill( function( quote, step_id ) + { + var _self = this; + + // only perform quote validation if the quote is + // using this program + if ( quote.getProgramId() !== program_id ) + { + return; + } + + // todo: unnecessary dependency + var bucket_quote = quote.getBucket().getData(), + bucket_tmp = QuoteDataBucket(), + data_tmp = {}; + + // this actually takes only 1ms, even with a reasonably + // sized bucket (tested with snowmobile) - both the copy and + // setValues() + for ( item in bucket_quote ) + { + if ( !Array.isArray( bucket_quote[ item ] ) ) + { + // this is a problem (FS#5849) + bucket_quote[ item ] = []; + + server.logger.log( server.logger.PRIORITY_ERROR, + "Bucket item '%s' not an array for " + + "quote id %s in program %s; set to empty", + + item, quote.getId(), program_id + ); + } + + data_tmp[ item ] = bucket_quote[ item ].slice( 0 ); + } + + bucket_tmp.setValues( data_tmp ); + + // Run all initialization stuff (e.g. calculated + // values) on the bucket to prepare for + // assertions. It's important to note that we + // duplicate the bucket to ensure that none of the + // calculated values are saved (the ones we want + // to save are already in there). + program.initQuote( bucket_tmp ); + + var classdata = program.classify( bucket_tmp.getData() ); + + // XXX + FieldClassMatcher( program.whens ) + .match( classdata, function( cmatch ) + { + var failures = program.submit( step_id, + bucket_tmp, + cmatch + ); + + // if there's any failures, abort the operation + if ( failures !== null ) + { + server.logger.log( server.logger.PRIORITY_ERROR, + "Server-side validation failure" + ); + _self.abort( failures ); + } + } ); + } ); + + resolve( program ); + } + catch ( e ) + { + reject( e ); + } + } ); + }, + + + 'private _getBucketCipher': function( program ) + { + var _self = this; + + return this._bucketCiphers[ program.id ] || ( function() + { + // create a new bucket cipher + var c = _self._bucketCiphers[ program.id ] = QuoteDataBucketCipher( + _self._encService, + program.secureFields.slice(0) || [] + ); + + c.on( 'encrecover', function( field, length ) + { + _self.logger.log( _self.logger.PRIORITY_ERROR, + "Invalid encrypted field data (%s of length %d); cleared.", + field, + length + ); + } ); + + return c; + } )(); + }, + + + /** + * Handle quick save request + * + * @param {UserRequest} request user request + * @param {Quote} quote quote to save + * @param {Program} program quote program + * + * @return {Server} self + */ + 'public handleQuickSave': function( request, quote, program ) + { + var _self = this; + + // do not allow quote modification if locked unless logged in as an + // internal user (FS#5772) and the program is unlockable + if ( quote.isImported() ) + { + //return this; + } + + request.getPostData( function( post_data ) + { + // sanitize, permitting nulls (since the diff will have them) + try + { + var filtered = _self._sanitizeBucketData( + post_data.data, request, program, true + ); + } + catch ( e ) + { + _self.logger.log( server.logger.PRIORITY_ERROR, + "Invalid quicksave data string (%s): %s", + e.message, + post_data.data + ); + + return; + } + + var secure = program.secureFields, + i = secure.length; + + // strip out secure fields (we can encrypt them later; this is just + // a quick solution to prevent sensitive data in plain text) + while ( i-- ) + { + delete filtered[ secure[ i ] ]; + } + + // attempt to save the diff + _self.dao.quickSaveQuote( quote, filtered, function( err ) + { + if ( !err ) + { + return; + } + + _self.logger.log( server.logger.PRIORITY_DB, + "[Quick Save] Quick Save Failed: " + err + ); + } ); + } ); + + // just send the response immediately, as they do not need feedback if + // the quick-save fails (it is for debugging/backup in the event of a + // problem, so we need only concern ourselves if there is an issue) + this.sendEmptyReply( request, quote ); + + return this; + }, + + + 'public createRevision': function( request, quote ) + { + var _self = this; + + this.dao.createRevision( quote, function( err ) + { + if ( err ) + { + _self.logger.log( _self.logger.PRIORITY_DB, + "[mkrev] failed to create revision: " + err + ); + + _self.sendError( request, 'Failed to create revision.' ); + return; + } + + _self.logger.log( _self.logger.PRIORITY_INFO, + "[mkrev] created new revision for quote %s", + quote.getId() + ); + + _self.sendEmptyReply( request, quote ); + } ); + }, + + + // TODO: currently only diffs against current revision (that is, the live + // bucket) + 'public diffRevisionGroup': function( request, program, quote, gid, revid ) + { + var _self = this, + progid = quote.getProgramId(); + + // this really should not happen...unless we delete a program, I suppose + if ( program === undefined ) + { + this.sendError( request, + "Quote program id '" + progid + "' unknown" + ); + + return; + } + + // get all fields linked to this group---not just the exclusive fields + var gfields = program.groupFields[ gid ]; + if ( gfields === undefined ) + { + this.sendError( request, + "Unknown group '" + gid + "' for program '" + progid + "'" + ); + + return; + } + + // do we have leaders? + var lead_data = request.getGetData().leaders; + if ( !lead_data ) + { + this.sendError( request, + "No leaders provided for group '" + gid + "'; available " + + "fields are: " + gfields.join( ', ' ) + ); + + return; + } + + var leaders = lead_data.split( ',' ); + this.dao.getRevision( quote, revid, function( revdata ) + { + if ( !revdata ) + { + _self.sendError( request, + "Revision " + revid + " not found for quote " + + quote.getId() + ); + return; + } + + var revbucket = QuoteDataBucket().setValues( revdata.data ); + + // XXX: tightly coupled; temporary impl + try + { + var desc = BucketSiblingDescriptor() + .defineGroup( gid, gfields ) + .markGroupLeaders( gid, leaders ); + + var diff = StdBucketDiff( + ShallowArrayDiff(), + function( context, changes ) + { + return GroupedBucketDiffResult( + StdBucketDiffResult( context, changes ), + context + ); + } + ) + .diff( + GroupedBucketDiffContext( + StdBucketDiffContext( + quote.getBucket(), + revbucket + ), + desc, + gid + ) + ); + } + catch ( e ) + { + _self.sendError( request, + "An error occurred during processing: " + + e.message + ); + throw e; + } + + _self.sendResponse( request, quote, { + map: diff.createIndexMap(), + diff: diff.describeChangedValues(), + } ); + } ); + }, + + + 'public sendEmptyReply': function( request, quote ) + { + this.sendResponse( request, quote, {} ); + } +} ); + diff --git a/src/server/cache/ResilientMemcache.js b/src/server/cache/ResilientMemcache.js new file mode 100644 index 0000000..939f1c6 --- /dev/null +++ b/src/server/cache/ResilientMemcache.js @@ -0,0 +1,447 @@ +/** + * ResilientMemcache class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + EventEmitter = require( 'events' ).EventEmitter; + + +/** + * Wraps the crappy memcache client implementation that we're using and + * automatically queues requests and reconnects on connection failure. + * + * Note that this only implements the methods that we actually use---in + * particular, connect(), get() and set(). + */ +module.exports = Class( 'ResilientMemcache' ) + .extend( EventEmitter, +{ + /** + * Memcache client + * @type {MemcacheClient} + */ + 'private _memcache': null, + + /** + * Connection failure count (cleared on successful connection) + * @type {number} + */ + 'private _fail_count': 0, + + /** + * Number of failures before request queue is purged + * @type {number} + */ + 'private _fail_limit': 5, + + /** + * Queue of pending requests (in case of a connection failure) + * @type {Object} + */ + 'private _queue': [], + + /** + * Connection state + * @type {boolean} + */ + 'private _connected': false, + + + /** + * Initialize decorator with existing memcache client instance + * + * The client is assumed to be disconnected from the server. + * + * @param {MemcacheClient} memcache memcache client + * @param {number} fail_limit number of failed connection attempts + * before request queue is purged + */ + __construct: function( memcache, fail_limit ) + { + this._memcache = memcache; + this._failHook(); + + this._connected = false; + + // keep default unless set + if ( fail_limit ) + { + this._fail_limit = +fail_limit; + } + }, + + + /** + * Attempts to connect to memcached + * + * The continuation is passed null if the connection is successful; + * otherwise, it will be passed an exception. + * + * If retry is ommitted or true, then a connection will be repeatedly + * attempted until a successful connection is made. There is currently no + * way to cancel this retry, as we have no use for such a feature at + * present. Note that, regardless of this setting, a reconnect will *always* + * be retried if an established connection is broken. + * + * @param {function(*)} callback continuation to invoke on connect/error + * @param {boolean} retry retry on error (default true) + * + * @return {ResilientMemcache} self + */ + 'public connect': function( callback, retry ) + { + var _self = this, + memc = this._memcache; + + callback = callback || function() {}; + retry = ( retry === undefined ) ? true : false; + + function cok() + { + memc.removeListener( 'error', cfail ); + callback( null ); + + _self._connectSuccess(); + }; + + function cfail( e ) + { + memc.removeListener( 'connect', cok ); + _self._connectFailure( e ); + + // re-try the connection attempt unless it has been requested that + // we do not + retry && _self._reconn( true ); + + callback( e ); + }; + + this.emit( 'preConnect' ); + + try + { + // for good measure, otherwise the connect event will not be kicked + // off + this._connected = false; + memc.close(); + + memc + .once( 'connect', cok ) + .once( 'error', cfail ); + + memc.connect(); + } + catch ( e ) + { + memc.removeListener( 'error', cfail ); + cfail( e ); + } + + return this; + }, + + + /** + * Retrieve data; enqueue request if disconnected + * + * On success, the continuation will be passed the value associated with the + * given key; otherwise, it will be passed null to indicate a failure. + * + * @param {string} key lookup key + * @param {function(string)} callback success/failure continuation + * + * @return {ResilientMemcache} self + */ + 'public get': function( key, callback ) + { + this._do( 'get', 1, arguments ); + return this; + }, + + + /** + * Set the value of a key; enqueue request if disconnected + * + * On failure, the continuation will be passed null; otherwise, its value is + * undefined. + * + * @param {string} key key to set + * @param {string} value key value + * @param {function(*)} callback success/failure continuation + * @param {number} lifetime key lifetime (see memcache docs) + * @param {number} flags see memcache docs + * + * @return {ResilientMemcache} self + */ + 'public set': function( key, value, callback, lifetime, flags ) + { + this._do( 'set', 2, arguments ); + return this; + }, + + + /** + * Perform client action, enqueing if disconnected + * + * If connected, the request will be performed immediately. + * + * If an enqueued request is re-attempted and would be enqueued a second + * time, it will immediately fail and invoke the original caller's + * continuation with the value null. + * + * @param {string} method memcache client method + * @param {number} cidx caller continuation argument index + * @param {arguments} args caller arguments to re-attmempt on method + * + * @return {undefined} + */ + 'private _do': function( method, cidx, args ) + { + var callback = args[ cidx ]; + + if ( this._connected === false ) + { + var _self = this; + this._enqueue( callback, function() + { + // we do not want to do this a second time. + args[ cidx ] = function() + { + // if this gets called, then that means that we've been + // enqueued a second time, implying that we have + // disconnected yet again...give up to prevent a perpetual + // song and dance that would make anyone's ears and eyes + // bleed + callback( null ); + }; + + // re-try with our new callback + _self[ method ].apply( _self, args ); + } ); + + return; + } + + this._memcache[ method ].apply( this._memcache, args ); + }, + + + /** + * Attempt reconnection + * + * Will continuously recurse with a delay until a connection is established. + * The delay is not currently configurable. + * + * This method will, by default, do nothing if disconnect (this is to ensure + * that reconnections are not spammed); in order to attempt to reconnect + * while already disconnected, use the force parameter. + * + * @param {boolean} force take action + * + * @return {undefined} + */ + 'private _reconn': function( force ) + { + var _self = this; + + // if we're not connected, then ignore this; we're already taking + // care of the issue + if ( !force && ( _self._connected === false ) ) + { + return; + } + + // attempt to reconnect + _self.connect( function( err ) + { + if ( err === null ) + { + // we're good + return; + } + + // this is no good; try again shortly + setTimeout( function() + { + _self._reconn( true ); + }, 1000 ); + }, false ); + }, + + + /** + * Hook memcache client to report errors and attempt reconnects + * + * On connection close or timeout, a connection will automatically be + * reattempted. In the case of an error, it will be bubbled up to be + * reported via an event, but will not constitute a connection failure. + * + * @return {undefined} + */ + 'private _failHook': function() + { + var _self = this, + reconn = this._reconn.bind( this ); + + this._memcache + .on( 'close', reconn ) + .on( 'timeout', reconn ) + + .on( 'error', function( e ) + { + // if we're not yet connected, then all errors will be + // considered connection errors, which we will handle separately + if ( _self._connected ) + { + _self.emit( 'error', e ); + } + } ); + }, + + + /** + * Enqueue a request to be re-attempted once a memcached connection is + * re-established + * + * The orginal caller's callback continuation is only useful for aborting + * requests; the retry continuation is used to re-attempt the original + * request. + * + * @param {function(*)} callback original caller continuation + * @param {function()} retry retry continuation + * + * @return {undefined} + */ + 'private _enqueue': function( callback, retry ) + { + this._queue.push( [ callback, retry ] ); + }, + + + /** + * Purges the request queue, aborting all requests + * + * Immediately invokes the original request's continuation with the value + * null to indicate a failure. This should be called periodically to ensure + * that requests do not stall for too long. + * + * If the queue has entries, then this will result in the queuedPurged event + * being raised with the number of requests that were purged. + * + * This is the worst-case scenerio. + * + * @return {undefined} + */ + 'private _purgeQueue': function() + { + var cur, + n = this._queue.length; + + // do not purge a queue with nothing in it + if ( n === 0 ) + { + return; + } + + // oh nooooo + while ( cur = this._queue.shift() ) + { + // invoke the continuation with a null to indicate a failure + cur[ 0 ]( null ); + } + + // in case anyone cares that we just told everyone to fuck off + this.emit( 'queuePurged', n ); + }, + + + /** + * Re-attemped all enqueued requests + * + * This should be invoked when a connection is re-established to memcache. + * The end result is that the original request receives the data they + * requested (assuming that nothing else goes wrong) with nothing more than + * an additional delay. + * + * @return {undefined} + */ + 'private _processQueue': function() + { + var cur; + + // yay...daises and butterflies... + while ( cur = this._queue.shift() ) + { + // give it another try + cur[ 1 ](); + } + }, + + + /** + * Invoked when a connection is successfully established with memcached + * + * This will raise the connect event, clear the connection failure count and + * begin processing the request queue. + * + * @return {undefined} + */ + 'private _connectSuccess': function() + { + this._fail_count = 0; + this._connected = true; + + this.emit( 'connect' ); + + // empty the queue + this._processQueue(); + }, + + + /** + * Invoked when a memcached connection attempt fails + * + * This will raise the connectionError event and increment the failure + * count; if this count reaches the failure limit, then the count will be + * reset and the request queue purged, preventing requests from lingering + * for too long. + * + * Note that this consequently implies a *maximum* time of delay * limit in + * the queue; a request could potentially be purged from the queue + * immediately after it is made. No attempt is made to ensure queue time, + * since a failure count implies that we are having difficulty reconnecting. + * + * @return {undefined} + */ + 'private _connectFailure': function( e ) + { + this._connected = false; + + this.emit( 'connectError', e ); + + // if we have reached our reconnect attempt limit, then purge the queue + // to ensure that requests are not stalling for too long + if ( ++this._fail_count === this._fail_limit ) + { + this._purgeQueue(); + this._fail_count = 0; + } + } +} ); diff --git a/src/server/daemon/Daemon.js b/src/server/daemon/Daemon.js new file mode 100644 index 0000000..89e6e53 --- /dev/null +++ b/src/server/daemon/Daemon.js @@ -0,0 +1,538 @@ +/** + * Daemon class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var AbstractClass = require( 'easejs' ).AbstractClass, + + liza = require( '../..' ), + sys = require( 'util' ), + sprintf = require( 'php' ).sprintf; + + +/** + * Facade handling core logic for the daemon + * + * TODO: Factor out unrelated logic + */ +module.exports = AbstractClass( 'Daemon', +{ + /** + * Quote server port + * @type {number} + */ + 'private _httpPort': 0, + + /** + * Server to accept HTTP requests + * @type {HttpServer} + */ + 'private _httpServer': null, + + /** + * Path to access log + * @var {string} + */ + 'private _accessLogPath': '', + + /** + * Path to debug log + * @var {string} + */ + 'private _debugLogPath': '', + + /** + * Access logger + * @type {AccessLog} + */ + 'private _accessLog': null, + + /** + * Debug logger + * @type {DebugLog} + */ + 'private _debugLog': null, + + /** + * Encryption service + * @type {EncryptionService} + */ + 'private _encService': null, + + /** + * Memcache client + * @type {Object} + */ + 'private _memcache': null, + + /** + * Routers to use to handle user requests, ordered from most likely to be + * used to least for performance reasons + * + * @type {Array.} + */ + 'private _routers': null, + + /** + * Rating service + * @type {Object} + */ + 'private _rater': null, + + + 'public __construct': function( http_port, log_priority ) + { + this._httpPort = http_port; + + this._rater = liza.server.rater.service; + this._httpServer = this.getHttpServer(); + this._accessLog = this._createAccessLog(); + this._debugLog = this._createDebugLog( log_priority ); + this._encService = this.getEncryptionService(); + this._memcache = this.getMemcacheClient(); + this._routers = this.getRouters(); + }, + + + /** + * Starts initializing the daemon + * + * @return {undefined} + */ + 'public start': function() + { + var _self = this; + + this._debugLog.log( this._debugLog.PRIORITY_IMPORTANT, + "Access log path: %s", this._accessLogPath + ); + + this._debugLog.log( this._debugLog.PRIORITY_IMPORTANT, + "Debug log path: %s", this._debugLogPath + ); + + this._initSignalHandlers(); + this._testEncryptionService( function() + { + _self._memcacheConnect(); + _self._initMemoryLogger(); + + _self._initRouters(); + _self._initHttpServer( function() + { + _self._initUncaughtExceptionHandler(); + + // ready to roll + _self._debugLog.log( _self._debugLog.PRIORITY_INFO, + "Daemon initialization complete." + ); + } ); + } ); + }, + + + 'protected getDebugLog': function() + { + return this._debugLog; + }, + + + 'protected getHttpServer': function() + { + return require( './http_server' ); + }, + + + 'protected getAccessLog': function() + { + return liza.server.log.AccessLog; + }, + + + 'protected getPriorityLog': function() + { + return liza.server.log.PriorityLog; + }, + + + 'protected getProgramController': function() + { + var controller = require( './controller' ); + controller.rater = this._rater; + + return controller; + }, + + + 'protected getScriptsController': function() + { + return require( './scripts' ); + }, + + + 'protected getClientErrorController': function() + { + return require( './clienterr' ); + }, + + + 'protected getUserRequest': function() + { + return liza.server.request.UserRequest; + }, + + + 'protected getUserSession': function() + { + return liza.server.request.UserSession; + }, + + + 'protected getMemcacheClient': function() + { + var MemcacheClient = require( 'memcache/lib/memcache' ).Client, + ResilientMemcache = liza.server.cache.ResilientMemcache, + + memc = ResilientMemcache( + new MemcacheClient( + process.env.MEMCACHE_PORT || 11211, + process.env.MEMCACHE_HOST || 'localhost' + ) + ); + + var _self = this; + + memc + .on( 'preConnect', function() + { + _self._debugLog.log( _self._debugLog.PRIORITY_IMPORTANT, + 'Connecting to memcache server...' + ); + } ) + .on( 'connect', function() + { + _self._debugLog.log( _self._debugLog.PRIORITY_IMPORTANT, + 'Connected to memcache server.' + ); + } ) + .on( 'connectError', function( e ) + { + _self._debugLog.log( _self._debugLog.PRIORITY_ERROR, + 'Failed to connect to memcached: %s', + e.message + ); + } ) + .on( 'queuePurged', function( n ) + { + _self._debugLog.log( _self._debugLog.PRIORITY_ERROR, + 'Memcache request queue (size %d) purged!', + n + ); + } ) + .on( 'error', function( e ) + { + _self._debugLog.log( _self._debugLog.PRIORITY_ERROR, + 'Memcache error: %s', + e.message + ); + } ); + + return memc; + }, + + + 'abstract protected getEncryptionService': [], + + + 'protected getRouters': function() + { + return [ + this.getProgramController(), + this.getScriptsController(), + this.getClientErrorController(), + ]; + }, + + + /** + * Perform a graceful shutdown + * + * @param {string} signal the signal that caused the shutdown + * + * @return {undefined} + */ + 'protected shutdown': function( signal ) + { + this._debugLog.log( this._debugLog.PRIORITY_IMPORTANT, + "Received %s. Beginning graceful shutdown...", + signal + ); + + this._debugLog.log( this._debugLog.PRIOIRTY_IMPORTANT, + "Closing HTTP server..." + ); + this._httpServer.close(); + + this._debugLog.log( this._debugLog.PRIORITY_IMPORTANT, + "Shutdown complete. Exiting..." + ); + + process.exit(); + }, + + + 'private _createAccessLog': function() + { + this._accessLogPath = + ( process.env.LOG_PATH_ACCESS || '/var/log/node/access.log' ); + + return this.getAccessLog()( this._accessLogPath ); + }, + + + 'private _createDebugLog': function( log_priority ) + { + this._debugLogPath = + ( process.env.LOG_PATH_DEBUG || '/var/log/node/debug.log' ); + + return this.getPriorityLog()( + this._debugLogPath, + ( process.env.LOG_PRIORITY || log_priority ) + ); + }, + + + /** + * Catches and logs uncaught exceptions to prevent early termination + * + * @return {undefined} + */ + 'private _initUncaughtExceptionHandler': function() + { + var _self = this; + + // chances are, we don't want to crash; we are, after all, a webserver + process.on( 'uncaughtException', function( err ) + { + _self._debugLog.log( _self._debugLog.PRIORITY_ERROR, + "Uncaught exception: %s\n%s", + err, + ( err.stack ) ? err.stack : '(No stack trace)' + ); + + // should we terminate on uncaught exceptions? + if ( process.env.NODEJS_UCE_TERM ) + { + _self._debugLog.log( _self._debugLog.PRIORITY_ERROR, + "NODEJS_UCE_TERM set; terminating..." + ); + + // SIGINT + process.kill( process.pid ); + } + }); + + // notify the user of the UCE_TERM flag is set + if ( process.env.NODEJS_UCE_TERM ) + { + this._debugLog.log( 1, + "NODEJS_UCE_TERM set; " + + "will terminate on uncaught exceptions." + ); + } + }, + + + 'private _initSignalHandlers': function() + { + var _self = this; + + // graceful shutdown on SIGINT and SIGTERM (cannot catch SIGKILL) + try + { + process + .on( 'SIGHUP', function() + { + _self._debugLog.log( _self._debugLog.PRIORITY_IMPORTANT, + "SIGHUP received; requesting reload" + ); + + _self.getProgramController().reload(); + } ) + .on( 'SIGINT', function() + { + _self.shutdown( 'SIGINT' ); + } ) + .on( 'SIGTERM', function() + { + _self.shutdown( 'SIGTERM' ); + } ); + } + catch ( e ) + { + console.log( "note: signal handling unsupported on this OS" ); + } + }, + + + 'private _testEncryptionService': function( callback ) + { + var enc_test = 'test string', + _self = this; + + this._debugLog.log( this._debugLog.PRIORITY_INFO, + "Performing encryption service sanity check..." + ); + + // encryption sanity check to ensure we won't end up working with data + // that will only become corrupt + this._encService.encrypt( enc_test, function( data ) + { + _self._encService.decrypt( data, function( data ) + { + if ( enc_test !== data.toString( 'ascii' ) ) + { + _self._debugLog.log( _self._debugLog.PRIORITY_ERROR, + "Encryption service is incompetant. Aborting." + ); + process.exit( 1 ); + } + + _self._debugLog.log( _self._debugLog.PRIORITY_INFO, + "Encryption service sanity check passed." + ); + + callback(); + } ); + } ); + }, + + + /** + * Attempts to make connection to memcache server + * + * @param memcache.Client memcache client to connect to server + * + * @return undefined + */ + 'private _memcacheConnect': function() + { + try + { + this._memcache.connect(); + } + catch( err ) + { + this._debugLog.log( this._debugLog.PRIORITY_ERROR, + "Failed to connected to memcached server: %s", + err + ); + } + }, + + + 'private _initMemoryLogger': function() + { + var _self = this; + + // log memory usage (every 15 min) + setInterval( function() + { + _self._debugLog.log( _self._debugLog.PRIORITY_IMPORTANT, + 'Memory usage: %s MB (rss), %s/%s MB (V8 heap)', + ( process.memoryUsage().rss / 1024 / 1024 ).toFixed( 2 ), + ( process.memoryUsage().heapUsed / 1024 / 1024 ).toFixed( 2 ), + ( process.memoryUsage().heapTotal / 1024 / 1024 ).toFixed( 2 ) + ); + }, 900000 ); + }, + + + 'private _initRouters': function() + { + var _self = this; + + // initialize each router + this._routers.forEach( function( router ) + { + if ( router.init instanceof Function ) + { + router.init( _self._debugLog, _self._encService ); + } + }); + }, + + + 'private _initHttpServer': function( callback ) + { + var _self = this; + + /** + * Builds UserRequest from the provided request and response objects + * + * @param {HttpServerRequest} request + * @param {HttpServerResponse} response + * + * @return {UserRequest} instance to represent current request + */ + function request_builder( request, response ) + { + return _self.getUserRequest()( + request, + response, + function( sess_id ) + { + // build a new user session from the given session id + return _self.getUserSession()( sess_id, _self._memcache ); + } + ); + } + + // create the HTTP server and listen for connections + try + { + this._httpServer = this.getHttpServer().create( + this._routers, + request_builder, + this._accessLog, + this._debugLog + ); + + this._httpServer.listen( this._httpPort, function() + { + _self._debugLog.log( + 1, "Server running on port %d", _self._httpPort + ); + + callback(); + } ); + } + catch( err ) + { + this._debugLog.log( this._debugLog.PRIORITY_ERROR, + "Unable to start HTTP server: %s", + err + ); + + // exit with an error + process.exit( 1 ); + } + }, +} ); + diff --git a/src/server/daemon/DevDaemon.js b/src/server/daemon/DevDaemon.js new file mode 100644 index 0000000..26766f8 --- /dev/null +++ b/src/server/daemon/DevDaemon.js @@ -0,0 +1,50 @@ +/** + * Contains DevDaemon class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + Daemon = require( './Daemon' ); + + +/** + * Daemon to use for local development + * + * This daemon does not use the encryption service. + */ +module.exports = Class( 'DevDaemon' ).extend( Daemon, +{ + /** + * Returns dummy encryption service + * + * For development purposes, running the encryption service is unnecessary. + * Instead, use a dummy service that simply returns what it was given. + * + * @return {EncryptionService} + */ + 'protected getEncryptionService': function() + { + var log = this.getDebugLog(); + log.log( log.PRIORITY_INFO, "Using dummy (echo) encryption service" ); + + return require( '../encsvc/EchoEncryptionServiceFactory' )() + .create(); + }, +} ); + diff --git a/src/server/daemon/clienterr.js b/src/server/daemon/clienterr.js new file mode 100644 index 0000000..dbc847f --- /dev/null +++ b/src/server/daemon/clienterr.js @@ -0,0 +1,52 @@ +/** + * Logs client-side errors + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +exports.route = function( request, log ) +{ + if ( !( request.getSession().isLoggedIn() ) ) + { + return Promise.resolve( false ); + } + + if ( !( request.getUri().match( /^clienterr/ ) ) ) + { + return Promise.resolve( false ); + } + + request.getPostData( function( data ) + { + log.log( log.PRIORITY_ERROR, + "[Client-side error] %s \"%s\" %s:%s \"%s\" \"%s\" :: %s", + request.getSession().agentId(), + data.file || '', + data.line || '-', + data.column || '-', + data.message || '', + request.getRequest().headers['user-agent'] || '', + data.stack && JSON.stringify( data.stack ) || '' + ); + } ); + + // we handled the request + request.end(); + return Promise.resolve( true ); +} + diff --git a/src/server/daemon/controller.js b/src/server/daemon/controller.js new file mode 100644 index 0000000..9eb9d75 --- /dev/null +++ b/src/server/daemon/controller.js @@ -0,0 +1,747 @@ +/** + * Route controller + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * @todo this is a mess of routing and glue code + */ + +const { + Db: MongoDb, + Server: MongoServer, + Connection: MongoConnection, +} = require( 'mongodb/lib/mongodb' ); + +const regex_base = /^\/quote\/([a-z0-9-]+)\/?(?:\/(\d+)\/?(?:\/(.*))?|\/(program.js))?$/; +const regex_step = /^step\/(\d+)\/?(?:\/(post|visit))?$/; + +const http = require( 'http' ); +const crypto = require( 'crypto' ); + +var server = null; +var server_cache = null; +var rating_service = null; + +const { + bucket: { + QuoteDataBucket, + }, + + server: { + Server, + + db: { + MongoServerDao, + }, + + lock: { + Semaphore, + }, + + quote: { + ServerSideQuote: Quote, + ProgramQuoteCleaner, + }, + + service: { + export: { + ExportService, + }, + + RatingService, + TokenedService, + TokenDao, + }, + + request: { + CapturedUserResponse, + JsonServerResponse, + SessionSpoofHttpClient, + UserResponse, + }, + }, + + store, +} = require( '../..' ); + + +// read and write locks, as separate semaphores +var rlock = Semaphore(), + wlock = Semaphore(); + +// concurrent session flag +var sflag = {}; + + +// TODO: kluge to get liza somewhat decoupled from lovullo (rating module) +exports.rater = {}; + + +exports.init = function( logger, enc_service ) +{ + var db = new MongoDb( + 'program', + new MongoServer( + process.env.MONGODB_HOST || '127.0.0.1', + +process.env.MONGODB_PORT || MongoConnection.DEFAULT_PORT, + {} + ), + { native_parser: false, safe: false } + ); + + var dao = MongoServerDao( db ); + + server = Server( + new JsonServerResponse.create(), + dao, + logger, + enc_service + ); + + server_cache = _createCache( server ); + server.init( server_cache, exports.rater ); + + rating_service = RatingService( logger, dao, server, exports.rater ); + + // TODO: exports.init needs to support callbacks; this will work, but + // only because it's unlikely that we'll get a request within + // milliseconds of coming online + _initExportService( db, function( service ) + { + c1_export_service = service; + } ); + + server.on( 'quotePverUpdate', function( quote, program, event ) + { + // let them know that we're going to be a moment + var c = event.wait(); + + getCleaner( program ).clean( quote, function( err ) + { + // report on our success/failure + if ( err ) + { + event.bad( err ); + } + else + { + event.good(); + } + + // we're done + c(); + } ); + } ); +} + + +function _initExportService( db, callback ) +{ + db.collection( 'quotes', function( err, collection ) + { + if ( collection === null ) + { + return; + } + + var spoof_host = ( + ''+( + process.env.C1_EXPORT_HOST + || process.env.LV_RATE_DOMAIN + || process.env.LV_RATE_HOST + ).trim() + ); + + var spoof = SessionSpoofHttpClient( http, spoof_host ); + + callback( + ExportService + .use( TokenedService( + 'c1import', + TokenDao( collection ), + function tokgen() + { + var shasum = crypto.createHash( 'sha1' ); + shasum.update( ''+Math.random() ); + + return shasum.digest( 'hex' ); + }, + function newcapturedResponse( request, callback ) + { + return UserResponse + .use( CapturedUserResponse( callback ) ) + ( request ); + } + ) ) + ( spoof ) + ); + + } ); +} + + +/** + * Create server cache + * + * TODO: This needs to be moved elsewhere; it is a stepping-stone + * kluge. + * + * @param {Server} server server containing miss methods + * + * @return {Store} cache + */ +function _createCache( server ) +{ + const progjs_cache = store.MemoryStore.use( + store.MissLookup( server.loadProgramFiles.bind( server ) ) + )(); + + const step_prog_cache = store.MemoryStore.use( + store.MissLookup( program_id => Promise.resolve( + store.MemoryStore.use( + store.MissLookup( + server.loadStepHtml.bind( server, program_id ) + ) + )() + ) ) + )(); + + const prog_cache = store.MemoryStore.use( + store.MissLookup( server.loadProgram.bind( server ) ) + )(); + + const cache = store.MemoryStore.use( store.Cascading )(); + cache.add( 'program_js', progjs_cache ); + cache.add( 'step_html', step_prog_cache ); + cache.add( 'program', prog_cache ); + + return cache; +} + + +exports.reload = function() +{ + // will cause all steps, programs, etc to be reloaded on demand + server_cache.clear(); + + server.reload( exports.rater ); +}; + + +exports.route = function( request ) +{ + var data; + + if ( !( data = request.getUri().match( regex_base ) ) ) + { + // we don't handle this URI + return Promise.resolve( false ); + } + + // we don't want to cache the responses, as most of them change with each + // request + request.noCache(); + + var program_id = data[1]; + + return server.getProgram( program_id ) + .then( function( program ) + { + return new Promise( function( resolve, reject ) + { + doRoute( program, request, data, resolve, reject ); + } ); + } ); +} + + +function doRoute( program, request, data, resolve, reject ) +{ + // store our data in more sensible vars + var program_id = data[1], + quote_id = +data[2] || 0, + cmd = data[3] || data[4] || '', + session = request.getSession(); + + // if we were unable to load the program class, that's a problem + if ( program === null ) + { + server.sendError( request, + 'Internal error. Please contact LoVullo Associates for ' + + 'support.' + + '

Your information has not been saved!' + ); + + resolve( true ); + } + + var skey = has_skey( request ); + + // is the user currently logged in? + if ( ( request.getSession().isLoggedIn() === false ) + && !skey + ) + { + // todo: this is temporary so we don't break our current setup; remove + // this check once we can error out before we even get to this point + // (PHP current handles the initial page load) + if ( cmd !== 'program.js' ) + { + session.setRedirect( '/quote/' + program_id + '/', function() + { + session.setReturnQuoteNumber( quote_id, function() + { + // peoples are trying to steal our secrets!?!!? + server.sendError( + request, + 'Please click here to log in.' + ); + } ); + } ); + + resolve( true ); + } + } + + // if the session key was provided, mark us as internal + if ( skey ) + { + request.getSession().setAgentId( '900000' ); + } + + // if we're internal, let the program know for the sake of assertions + program.isInternal = request.getSession().isInternal(); + + // we'll be serving all our responses as plain text + request.setContentType( 'text/plain' ); + + if ( data = cmd.match( regex_step ) ) + { + var step_id = data[1]; + var step_action = ( data[2] !== undefined ) ? data[2] : ''; + + switch ( step_action ) + { + case 'post': + acquireWriteLock( quote_id, request, function() + { + handleRequest( function( quote ) + { + server.handlePost( + step_id, request, quote, program, session + ); + } ); + } ); + break; + + case 'visit': + acquireRwLock( quote_id, request, function() + { + handleRequest( function( quote ) + { + server.visitStep( step_id, request, quote ); + } ); + } ); + break; + + default: + // send the requested step to the client + acquireReadLock( quote_id, request, function() + { + handleRequest( function( quote ) + { + server.sendStep( + request, quote, program, step_id, session + ); + } ); + } ); + break; + } + } + else if ( cmd == 'init' ) + { + acquireWriteLock( quote_id, request, function() + { + handleRequest( function( quote ) + { + server.sendInit( + request, + quote, + program, + + // for invalid quote requests + createQuoteQuick, + + // concurrent access? + getConcurrentSessionUser( quote_id, session ) + ); + } ); + } ); + } + else if ( cmd === 'mkrev' ) + { + // the database operation for this is atomic and disjoint from + // anything else we're doing, so no need to acquire any sort of + // lock + handleRequest( function( quote ) + { + server.createRevision( request, quote ); + } ); + } + // TODO: diff against other revisions as well + else if ( data = cmd.match( /^revdiffgrp\/(.*?)\/(\d+)$/ ) ) + { + // similar to above; no locking needed + handleRequest( function( quote ) + { + var gid = data[ 1 ] || '', + revid = +data[ 2 ] || 0; + + server.diffRevisionGroup( request, program, quote, gid, revid ); + } ); + } + else if ( cmd == 'program.js' ) + { + // no quote involved; just send the JS + server.sendProgramJs( request, program_id ); + resolve( true ); + } + else if ( /^rate\b/.test( cmd ) ) + { + // the client may have optionally requested the rate for a specific + // alias + var ratedata = cmd.match( /^rate(?:\/([a-z]+))?/ ), + alias = ratedata[ 1 ]; + + // request manual lock freeing; allows us to free the lock when we + // want to (since we'll be saving data to the DB async, after the + // response is already returned) + acquireWriteLock( quote_id, request, function( free ) + { + // if we're performing deferred rating, it must be async; + // immediately free the locks and trust that the deferred process + // knows what it is doing and can properly handle such concurrency + alias && free(); + + handleRequest( function( quote ) + { + var response = UserResponse( request ); + + rating_service.request( request, response, quote, alias, function() + { + // we're done; free the lock + free(); + } ); + } ); + }, true ); + } + else if ( /^worksheet\//.test( cmd ) ) + { + var wdata = cmd.match( /^worksheet\/(.+)\/([0-9]+)/ ), + supplier = wdata[ 1 ], + index = +wdata[ 2 ]; + + handleRequest( function( quote ) + { + rating_service.serveWorksheet( request, quote, supplier, index ); + } ); + } + else if ( /^export\//.test( cmd ) ) + { + var import_data = cmd.match( /^export\/(.+?)(?:\/(.+))?$/ ), + type = import_data[ 1 ], + subcmd = import_data[ 2 ]; + + + // TODO: extract body + handleRequest( function( quote ) + { + // TODO: support type + c1_export_service.request( + request, + UserResponse( request ), + quote, + subcmd + ); + } ); + } + else if ( cmd === 'quicksave' ) + { + // attempt to acquire the write lock, aborting immediately if we + // cannot (instead of queueing) + acquireWriteLockImmediate( quote_id, request, function( free ) + { + handleRequest( function( quote ) + { + // if we could not immediately obtain the lock, then something + // is currently saving, meaning that the quicksave data is + // probably irrelevant (since it would just be wiped out anyway + // if we had issued this request moments earlier); abort + if ( !free ) + { + server.sendEmptyReply( request, quote ); + return; + } + + server.handleQuickSave( request, quote, program ); + } ); + } ); + + // keep the session alive + touchSession( quote_id, session ); + } + else if ( /^log\//.test( cmd ) ) + { + // the "log" URI currently does absolutely nothing; ideally, we'd be + // able to post to this and log somewhere useful, but for now it + // just appears in the logs + handleRequest( function( quote ) + { + server.sendEmptyReply( request, quote ); + } ); + } + else + { + resolve( false ); + } + + // create a quote to represent this request + function handleRequest( operation ) + { + createQuote( quote_id, program, request, operation, function( fatal ) + { + // if fatal, notify the user and bail out + if ( fatal ) + { + // an error occurred; quote invalid + server.sendError( request, + 'There was a problem loading this quote; please contact ' + + 'LoVullo Associates for support.' + ); + + return; + } + + // otherwise, the given quote is invalid, but we can provide a new + // one + server.sendNewQuote( request, createQuoteQuick ); + } ); + } + + // we handled the request; don't do any additional routing + resolve( true ); +} + + +/** + * Creates a new quote instance with the given quote id + * + * @param Integer quote_id id of the quote + * @param Program program program that the quote will be a part of + * @param Function( quote ) callback function to call when quote is ready + * + * @return undefined + */ +function createQuote( quote_id, program, request, callback, error_callback ) +{ + // if an invalid callback was given, log it to the console...that's a + // problem, since the quote won't even be returned! + callback = callback || function() + { + server.logger.log( log.PRIORITY_ERROR, + "Invalid createQuote() callback" + ); + } + + var bucket = QuoteDataBucket(), + quote = Quote( quote_id, bucket ); + + var controller = this; + return server.initQuote( quote, program, request, + function() + { + callback.call( controller, quote ); + }, + function() + { + error_callback.apply( controller, arguments ); + } + ); +} + + +function createQuoteQuick( id ) +{ + return Quote( + id, + QuoteDataBucket() + ); +} + + +function has_skey( user_request ) +{ + // a basic authentication token that allows our systems to bypass + // authentication...this isn't really secure, but it doesn't need to be, + // because for our uses, they really cannot do any damage + return ( user_request.getGetData().skey === 'fd29d02ac1' ) +} + + +function getCleaner( program ) +{ + return ProgramQuoteCleaner( program ); +} + + +/** + * Acquire a semaphore for a quote id + * + * Note that, since this controller is single-threaded, we do not have to worry + * about race conditions with regards to acquiring the lock. + */ +function acquireLock( type, id, request, c, manual ) +{ + type.acquire( id, function( free ) + { + // automatically release the lock once the request completes (this is + // also safer, as it is hopefully immune to exceptions before lock + // release and will still work with the timeout system) + if ( !manual ) + { + request.once( 'end', function() + { + free(); + } ); + } + + // we're good! + c( free ); + } ); + + // keep the quote session alive + touchSession( id, request.getSession() ); +} + +/** + * ALWAYS USE THIS FUNCTION WHEN TRYING TO ACQUIRE BOTH A READ AND A WRITE LOCK! + * Otherwise, the possibility for a deadlock is introduced if something else is + * attempting to acquire both locks in the opposite order! + */ +function acquireRwLock( id, request, c, manual ) +{ + acquireWriteLock( id, request, function() + { + acquireReadLock( id, request, c ); + }, manual ); +} + +function acquireWriteLock( id, request, c, manual ) +{ + acquireLock( wlock, id, request, c, manual ); +} + +function acquireReadLock( id, request, c, manual ) +{ + acquireLock( rlock, id, request, c, manual ); +} + +function acquireWriteLockImmediate( id, request, c, manual ) +{ + if ( wlock.isLocked( id ) ) + { + // we could not obtain the lock + c( null ); + return; + } + + // lock is free; acquire it + acquireWriteLock.apply( null, arguments ); +} + + +function touchSession( id, session ) +{ + var cur = sflag[ id ]; + + // do not allow touching the session if we're not the owner + if ( cur && ( cur.agentName !== session.agentName() ) ) + { + return; + } + + sflag[ id ] = { + agentName: session.agentName(), + time: ( new Date() ).getTime(), + }; +} + +function getConcurrentSessionUser( id, session ) +{ + var flag = sflag[ id ]; + + // we have a flag; if we're the same user that created it (we check on name + // because internally we all have the same agent id), then do not consider + // this a concurrent access attempt + if ( !flag || ( flag.agentName === session.agentName() ) ) + { + return ''; + } + + return ( flag.agentName ); +} + + + +// in the unfortunate situation where a write lock hangs, for whatever reason, +// this will ensure that it is eventually freed; note that, since this shouldn't +// ever happen, this interval is 30s, meaning that a given lock may exist for +// just under 60s +var __wlock_stale_interval = 30e3; +setInterval( function __wlock_stale_free() +{ + // TODO: log properly and possibly kill the request + // TODO: if the same quote repeatedly has stale locks, perhaps the + // quote data is bad and should be locked + wlock.freeStale( __wlock_stale_interval, function( id ) + { + console.log( 'Freeing stale write lock: ' + id ); + } ); + + rlock.freeStale( __wlock_stale_interval, function( id ) + { + console.log( 'Freeing stale read lock: ' + id ); + } ); +}, __wlock_stale_interval ); + + +// set this to ~10s after the quicksave interval +var __sclear_interval = 70e3; +setInterval( function __sclear_timeout() +{ + var now = ( new Date() ).getTime(); + + for ( var id in sflag ) + { + // clear all session flags that have timed out + if ( ( now - sflag[ id ].time ) > __sclear_interval ) + { + delete sflag[ id ]; + } + } +}, __sclear_interval ); diff --git a/src/server/daemon/http_server.js b/src/server/daemon/http_server.js new file mode 100644 index 0000000..77447ee --- /dev/null +++ b/src/server/daemon/http_server.js @@ -0,0 +1,121 @@ +/** + * Liza HTTP server + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * Also called the "quote server". + * + * This is ancient---it's an evolution of the first prototype written for + * liza's server, and has barely evolved since then. + */ + +var http = require( 'http' ); + + +exports.create = function( routers, request_builder, access_log, debug_log ) +{ + return log_server( http.createServer( function( request, response ) + { + // easy request/response management + var user_request = request_builder( request, response ); + + // log this request to the access log + access_log.attach( user_request ); + + // process the request when it's ready (all data is available) + user_request.on( 'ready', function() + { + var routed = false; + Promise.all( + routers.map( function( router ) + { + return router.route( user_request, debug_log ); + } ) + ).then( function( routed ) + { + var was_routed = routed.some( function( handled ) + { + return handled === true; + } ); + + // display a 404 if we weren't able to route the request + if ( !was_routed ) + { + return_404( user_request ); + } + } ); + }); + }), debug_log ); +}; + + +function return_404( response ) +{ + response.setResponseCode( 404 ); + response.end( '404 Not found.' ); +} + + +/** + * Enables logging on the server + * + * @param {HttpServer} server server on which to enable logging + * @param {PriorityLogger} debug_log logger to use + * + * @return {HttpServer} + */ +function log_server( server, debug_log ) +{ + server + .on( 'connection', function( stream ) + { + /** this is useless until not behind a proxy, since the IP address + * is always the same + debug_log.log( debug_log.PRIORITY_SOCKET, + 'HTTP connection received from %s', + stream.remoteAddress + ); + */ + + // log errors on the connection + stream.on( 'error', function( exception ) + { + debug_log.log( debug_log.PRIORITY_SOCKET, + 'HTTP server connection error on %s: %s', + stream.remoteAddress, + exception + ); + }); + }) + .on( 'close', function( errno ) + { + debug_log.log( debug_log.PRIORITY_SOCKET, + "HTTP server connection closed." + ); + }) + .on( 'clientError', function( exception ) + { + debug_log.log( debug_log.PRIORITY_SOCKET, + 'HTTP client connection error: %s', + exception + ); + }); + + return server; +} + diff --git a/src/server/daemon/scripts.js b/src/server/daemon/scripts.js new file mode 100644 index 0000000..0cb5311 --- /dev/null +++ b/src/server/daemon/scripts.js @@ -0,0 +1,184 @@ +/** + * Script provider + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * This guy has an interesting history. It allows for local development + * loading each source file individually; a separate build process exists + * (within LoVullo) to build a file for distribution. This is before + * browserify existed---the Node.js community was still very young. How far + * we've come. + * + * To get code working on the client in CommonJS format, this dynamically + * wraps the scripts. To maintain line numbering for errors, it does not + * put a newline at the beginning. + * + * @todo decouple from lovullo and its paths + */ + +var fs = require( 'fs' ); + + +/** + * Grabs the requested script out of the regex + * + * @var {RegExp} + * @const + */ +var script_regex = /scripts\/([a-zA-Z0-9\/\._-]+.js)/; + +/** + * Script paths to attempt to locate file in, in order of precedence + * @var {array.} + */ +var script_paths = [ + ( process.env.LV_ROOT_PATH || '.' ) + '/src/_gen/scripts/', + ( process.env.LV_ROOT_PATH || '.' ) + '/src/www/scripts/program/', +]; + +var script_prefix = { + liza: __dirname + '/../../', + assert: __dirname + '/../../assert/', + program: ( process.env.LV_LEGACY_PATH + '/' ) || '', +}; + +/** + * Cache scripts in memory to avoid constant directory lookups and disk I/O + * @var {Object} + */ +var script_cache = {}; + +/** + * Whether to cache scripts in memory + * @var {boolean} + */ +var cache = false; + + +/** + * Scripts router + * + * @param {UserRequest} request request to route + * + * @return {boolean} true if request was handled, otherwise false + */ +exports.route = function( request, log ) +{ + var data; + if ( !( data = request.getUri().match( script_regex ) ) ) + { + // request could not be routed + return Promise.resolve( false ); + } + + // grab the filename, stripping off version tags + var file = data[1].replace( /\.[0-9]+\.js/, '.js' ); + + // is this already in memory? + var cache_data = script_cache[ file ]; + if ( cache_data !== undefined ) + { + // serve from memory + request.setContentType( 'text/javascript' ).end( cache_data ); + return Promise.resolve( true ); + } + + var parts = file.match( /^(?:(.+?)\/)?(.*)$/ ), + prefix = parts[ 1 ], + suffix = parts[ 2 ]; + + var chk_paths = script_paths.slice(); + chk_paths.unshift( script_prefix[ prefix ] || './' ); + + // check each of the paths for the script that was requested + ( function check_path( paths ) + { + var cur_path = paths.shift(); + if ( cur_path === undefined ) + { + // no more dirs to check; not found + request.setResponseCode( 404 ).end(); + + return; + } + + // check to see if the file exists within the path + var filename = ( cur_path + suffix ); + console.log( filename ); + fs.exists( filename, function( exists ) + { + if ( !exists ) + { + // next! + check_path( paths ); + return; + } + + // serve the file! + fs.readFile( filename, function( err, data ) + { + // an error occurred + if ( err ) + { + log.log( log.PRIORITY_ERROR, + "Failed to serve file (%s): %s", + filename, err + ); + + request.setResponseCode( 500 ).end(); + } + + // cache the data in memory so we don't have to do this again + if ( cache ) + { + script_cache[ file ] = data; + } + + // send the data wrapped to support the CommonJS format + request.setContentType( 'text/javascript' ); + + // get the requested module name and secure it + var module = request.getGetData().module; + + if ( module ) + { + // secure module name from any attacks + module = module.replace( /[^a-zA-Z0-9\/]/, '' ); + + // serve as a CommonJS module (which won't work client-side + // by default) + request.end( + "(function(module,require){" + + "var exports=module.exports={};" + + data + + "\n})(modules['" + module + "']={},mkrequire('" + module + "'));" + ); + } + else + { + // do not serve a CommonJS module + request.end( data ); + } + }); + }); + })( chk_paths ); + + // request was handled + return Promise.resolve( true ); +} + diff --git a/src/server/db/MongoServerDao.js b/src/server/db/MongoServerDao.js new file mode 100644 index 0000000..c08243e --- /dev/null +++ b/src/server/db/MongoServerDao.js @@ -0,0 +1,749 @@ +/** + * Mongo DB DAO for program server + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + EventEmitter = require( 'events' ).EventEmitter, + ServerDao = require( './ServerDao' ).ServerDao; + + +/** + * Uses MongoDB as a data store + */ +module.exports = Class( 'MongoServerDao' ) + .implement( ServerDao ) + .extend( EventEmitter, +{ + /** + * Collection used to store quotes + * @type String + */ + 'const COLLECTION': 'quotes', + + /** + * Sequence (auto-increment) collection + * @type {string} + */ + 'const COLLECTION_SEQ': 'seq', + + /** + * Sequence key for quote ids + * + * @type {string} + * @const + */ + 'const SEQ_QUOTE_ID': 'quoteId', + + /** + * Sequence quoteId default + * + * @type {number} + * @const + */ + 'const SEQ_QUOTE_ID_DEFAULT': 200000, + + + /** + * Database instance + * @type Mongo.Db + */ + 'private _db': null, + + /** + * Whether the DAO is initialized and ready to be used + * @type Boolean + */ + 'private _ready': false, + + /** + * Collection to save data to + * @type null|Collection + */ + 'private _collection': null, + + /** + * Collection to read sequences (auto-increments) from + * @type {null|Collection} + */ + 'private _seqCollection': null, + + + /** + * Initializes DAO + * + * @param {Mongo.Db} db mongo database connection + * + * @return undefined + */ + 'public __construct': function( db ) + { + this._db = db; + }, + + + /** + * Initializes error events and attempts to connect to the database + * + * connectError event will be emitted on failure. + * + * @param Function callback function to call when connection is complete + * (will not be called if connection fails) + * + * @return MongoServerDao self to allow for method chaining + */ + 'public init': function( callback ) + { + var dao = this; + + // map db error event (on connection error) to our connectError event + this._db.on( 'error', function( err ) + { + dao._ready = false; + dao._collection = null; + + dao.emit( 'connectError', err ); + }); + + this.connect( callback ); + return this; + }, + + + /** + * Attempts to connect to the database + * + * connectError event will be emitted on failure. + * + * @param Function callback function to call when connection is complete + * (will not be called if connection fails) + * + * @return MongoServerDao self to allow for method chaining + */ + 'public connect': function( callback ) + { + var dao = this; + + // attempt to connect to the database + this._db.open( function( err, db ) + { + // if there was an error, don't bother with anything else + if ( err ) + { + // in some circumstances, it may just be telling us that we're + // already connected (even though the connection may have been + // broken) + if ( err.errno !== undefined ) + { + dao.emit( 'connectError', err ); + return; + } + } + + var ready_count = 0; + var check_ready = function() + { + if ( ++ready_count < 2 ) + { + return; + } + + // we're ready to roll! + dao._ready = true; + dao.emit( 'ready' ); + + // connection was successful; call the callback + if ( callback instanceof Function ) + { + callback.call( dao ); + } + } + + // quotes collection + db.collection( dao.__self.$('COLLECTION'), function( err, collection ) + { + // for some reason this gets called more than once + if ( collection == null ) + { + return; + } + + // initialize indexes + collection.createIndex( + [ ['id', 1] ], + true, + function( err, index ) + { + // mark the DAO as ready to be used + dao._collection = collection; + check_ready(); + } + ); + }); + + // seq collection + db.collection( dao.__self.$('COLLECTION_SEQ'), function( err, collection ) + { + if ( err ) + { + dao.emit( 'seqError', err ); + return; + } + + if ( collection == null ) + { + return; + } + + dao._seqCollection = collection; + + // has the sequence we'll be referencing been initialized? + collection.find( + { _id: dao.__self.$('SEQ_QUOTE_ID') }, + { limit: 1 }, + function( err, cursor ) + { + if ( err ) + { + dao.initQuoteIdSeq( check_ready ) + return; + } + + cursor.toArray( function( err, data ) + { + if ( data.length == 0 ) + { + dao.initQuoteIdSeq( check_ready ); + return; + } + + check_ready(); + }); + } + ); + }); + }); + + return this; + }, + + + 'public initQuoteIdSeq': function( callback ) + { + var dao = this; + + this._seqCollection.insert( + { + _id: this.__self.$('SEQ_QUOTE_ID'), + val: this.__self.$('SEQ_QUOTE_ID_DEFAULT'), + }, + function( err, docs ) + { + if ( err ) + { + dao.emit( 'seqError', err ); + return; + } + + dao.emit( 'seqInit', this.__self.$('SEQ_QUOTE_ID') ); + callback.call( this ); + } + ); + }, + + + /** + * Saves a quote to the database + * + * @param Quote quote the quote to save + * @param Function success_callback function to call on success + * @param Function failure_callback function to call if save fails + * @param Object save_data quote data to save (optional) + * + * @return MongoServerDao self to allow for method chaining + */ + 'public saveQuote': function( + quote, success_callback, failure_callback, save_data + ) + { + var dao = this; + + // if we're not ready, then we can't save the quote! + if ( this._ready === false ) + { + this.emit( 'saveQuoteError', + { message: 'Database server not ready' }, + Error( 'Database not ready' ), + quote + ); + + failure_callback.call( this, quote ); + return; + } + + if ( save_data === undefined ) + { + save_data = { + data: quote.getBucket().getData(), + }; + } + else if ( save_data.data !== undefined ) + { + // when we update the quote data, clear quick save data (this data + // should take precedence) + save_data.quicksave = {}; + } + + var id = quote.getId(); + + // some data should always be saved because the quote will be created if + // it does not yet exist + save_data.id = id; + save_data.pver = quote.getProgramVersion(); + save_data.importDirty = 1; + save_data.lastPremDate = quote.getLastPremiumDate(); + save_data.initialRatedDate = quote.getRatedDate(); + save_data.explicitLock = quote.getExplicitLockReason(); + save_data.explicitLockStepId = quote.getExplicitLockStep(); + save_data.importedInd = +quote.isImported(); + save_data.boundInd = +quote.isBound(); + save_data.lastUpdate = Math.round( + ( new Date() ).getTime() / 1000 + ); + + // save the stack so we can track this call via the oplog + save_data._stack = ( new Error() ).stack; + + // update the quote data if it already exists (same id), otherwise + // insert it + this._collection.update( { id: id }, + { '$set': save_data }, + + // create record if it does not yet exist + { upsert: true }, + + // on complete + function( err, docs ) + { + // if an error occurred, then we cannot continue + if ( err ) + { + dao.emit( 'saveQuoteError', err, quote ); + + // let the caller handle the error + if ( failure_callback instanceof Function ) + { + failure_callback.call( dao, quote ); + } + + return; + } + + // successful + if ( success_callback instanceof Function ) + { + success_callback.call( dao, quote ); + } + } + ); + + return this; + }, + + + /** + * Merges quote data with the existing (rather than overwriting) + * + * @param {Quote} quote quote to save + * @param {Object} data quote data + * @param {Function} scallback successful callback + * @param {Function} fcallback failure callback + * + * @return {MongoServerDao} self + */ + 'public mergeData': function( quote, data, scallback, fcallback ) + { + // we do not want to alter the original data; use it as a prototype + var update = data; + + // save the stack so we can track this call via the oplog + var _self = this; + this._collection.update( { id: quote.getId() }, + { '$set': update }, + {}, + + function( err, docs ) + { + if ( err ) + { + _self.emit( 'saveQuoteError', err, quote ); + + if ( typeof fcallback === 'function' ) + { + fcallback( quote ); + } + + return; + } + + if ( typeof scallback === 'function' ) + { + scallback( quote ); + } + } + ); + + return this; + }, + + + /** + * Merges bucket data with the existing bucket (rather than overwriting the + * entire bucket) + * + * @param {Quote} quote quote to save + * @param {Object} data bucket data + * @param {Function} scallback successful callback + * @param {Function} fcallback failure callback + * + * @return {MongoServerDao} self + */ + 'public mergeBucket': function( quote, data, scallback, fcallback ) + { + var update = {}; + + for ( var field in data ) + { + if ( !field ) + { + continue; + } + + update[ 'data.' + field ] = data[ field ]; + } + + return this.mergeData( quote, update, scallback, fcallback ); + }, + + + /** + * Perform a "quick save" + * + * A quick save simply saves the given diff to the database for recovery + * purposes + * + * @param {Quote} quote quote being saved + * @param {Object} diff staged changes + * + * @param {function(*)} callback callback to call when complete + * + * @return {MongoServerDao} self + */ + 'public quickSaveQuote': function( quote, diff, callback ) + { + // unlikely, but possible for a request to come in before we're ready + // since this system is asynchronous + if ( this._ready === false ) + { + callback( Error( 'Database server not ready' ) ); + return; + } + + var id = quote.getId(); + + this._collection.update( + { id: id }, + { $set: { quicksave: diff } }, + + // on complete + function( err, docs ) + { + callback( err ); + } + ); + + return this; + }, + + + /** + * Saves the quote state to the database + * + * The quote state includes the current step, the top visited step and the + * explicit lock message. + * + * @param Quote quote the quote to save + * @param Function success_callback function to call on success + * @param Function failure_callback function to call if save fails + * + * @return MongoServerDao self + */ + 'public saveQuoteState': function( + quote, success_callback, failure_callback + ) + { + var update = { + currentStepId: quote.getCurrentStepId(), + topVisitedStepId: quote.getTopVisitedStepId(), + topSavedStepId: quote.getTopSavedStepId(), + }; + + return this.mergeData( + quote, update, success_callback, failure_callback + ); + }, + + + 'public saveQuoteClasses': function( quote, classes, success, failure ) + { + return this.mergeData( + quote, + { classData: classes }, + success, + failure + ); + }, + + + /** + * Saves the quote lock state to the database + * + * @param Quote quote the quote to save + * @param Function success_callback function to call on success + * @param Function failure_callback function to call if save fails + * + * @return MongoServerDao self + */ + 'public saveQuoteLockState': function( + quote, success_callback, failure_callback + ) + { + // lock state is saved by default + return this.saveQuote( quote, success_callback, failure_callback, {} ); + }, + + + /** + * Pulls quote data from the database + * + * @param Integer quote_id id of quote + * @param Function( data ) callback function to call when data is available + * + * @return MongoServerDao self to allow for method chaining + */ + 'public pullQuote': function( quote_id, callback ) + { + var dao = this; + + // XXX: TODO: Do not read whole of record into memory; filter out + // revisions! + this._collection.find( { id: quote_id }, { limit: 1 }, + function( err, cursor ) + { + cursor.toArray( function( err, data ) + { + // was the quote found? + if ( data.length == 0 ) + { + callback.call( dao, null ); + return; + } + + // return the quote data + callback.call( dao, data[ 0 ] ); + }); + } + ); + + return this; + }, + + + 'public getMinQuoteId': function( callback ) + { + // just in case it's asynchronous later on + callback.call( this, this.__self.$('SEQ_QUOTE_ID_DEFAULT') ); + return this; + }, + + + 'public getMaxQuoteId': function( callback ) + { + var dao = this; + + this._seqCollection.find( + { _id: this.__self.$('SEQ_QUOTE_ID') }, + { limit: 1 }, + function( err, cursor ) + { + cursor.toArray( function( err, data ) + { + if ( data.length == 0 ) + { + callback.call( dao, 0 ); + return; + } + + // return the max quote id + callback.call( dao, data[ 0 ].val ); + }); + } + ); + }, + + + 'public getNextQuoteId': function( callback ) + { + var dao = this; + + this._seqCollection.findAndModify( + { _id: this.__self.$('SEQ_QUOTE_ID') }, + [ [ 'val', 'descending' ] ], + { $inc: { val: 1 } }, + { 'new': true }, + + function( err, doc ) + { + if ( err ) + { + dao.emit( 'seqError', err ); + + callback.call( dao, 0 ); + return; + } + + // return the new id + callback.call( dao, doc.val ); + } + ); + + return this; + }, + + + /** + * Create a new revision with the provided quote data + * + * The revision will contain the whole the quote. If space is a concern, we + * can (in the future) calculate a delta instead (Mike recommends the Git + * model of storing the deltas in previous revisions and the whole of the + * bucket in the most recently created revision). + */ + 'public createRevision': function( quote, callback ) + { + var _self = this, + qid = quote.getId(), + data = quote.getBucket().getData(); + + this._collection.update( { id: qid }, + { '$push': { revisions: { data: data } } }, + + // create record if it does not yet exist + { upsert: true }, + + // on complete + function( err ) + { + if ( err ) + { + _self.emit( 'mkrevError', err ); + } + + callback( err ); + return; + } + ); + }, + + + 'public getRevision': function( quote, revid, callback ) + { + revid = +revid; + + // XXX: TODO: Filter out all but the revision we want + this._collection.find( + { id: quote.getId() }, + { limit: 1 }, + function( err, cursor ) + { + cursor.toArray( function( err, data ) + { + // was the quote found? + if ( ( data.length === 0 ) + || ( data[ 0 ].revisions.length < ( revid + 1 ) ) + ) + { + callback( null ); + return; + } + + // return the quote data + callback( data[ 0 ].revisions[ revid ] ); + }); + } + ); + }, + + + 'public setWorksheets': function( qid, data, callback ) + { + this._collection.update( { id: qid }, + { '$set': { worksheets: { data: data } } }, + + // create record if it does not yet exist + { upsert: true }, + + // on complete + function( err ) + { + callback( err ); + return; + } + ); + }, + + + 'public getWorksheet': function( qid, supplier, index, callback ) + { + this._collection.find( + { id: qid }, + { limit: 1 }, + function( err, cursor ) + { + cursor.toArray( function( err, data ) + { + // was the quote found? + if ( ( data.length === 0 ) + || ( !data[ 0 ].worksheets ) + || ( !data[ 0 ].worksheets.data ) + || ( !data[ 0 ].worksheets.data[ supplier ] ) + ) + { + callback( null ); + return; + } + + // return the quote data + callback( data[ 0 ].worksheets.data[ supplier ][ index ] ); + }); + } + ); + }, +} ); + diff --git a/src/server/db/ServerDao.js b/src/server/db/ServerDao.js new file mode 100644 index 0000000..660a68b --- /dev/null +++ b/src/server/db/ServerDao.js @@ -0,0 +1,60 @@ +/** + * Contains ServerDao interface + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Interface = require( 'easejs' ).Interface; + + +/** + * Represents server DAO + * + * @todo: terminology is tied very tightly with mongo; fix that + */ +exports.ServerDao = Interface.extend( +{ + 'public init': [ 'callback' ], + + 'public connect': [ 'callback' ], + + 'public initQuoteIdSeq': [ 'callback' ], + + 'public saveQuote': [ + 'quote', 'success_callback', 'failure_callback', 'save_data' + ], + + 'public saveQuoteState': [ + 'quote', 'succes_callback', 'failure_callback' + ], + + 'public pullQuote': [ 'quote_id', 'callback' ], + + 'public getMinQuoteId': [ 'callback' ], + + 'public getMaxQuoteId': [ 'callback' ], + + 'public getNextQuoteId': [ 'callback' ], + + + /** + * Create a new revision with the provided quote data + */ + 'public createRevision': [ 'quote', 'callback' ], +} ); + diff --git a/src/server/encsvc/EchoEncryptionServiceFactory.js b/src/server/encsvc/EchoEncryptionServiceFactory.js new file mode 100644 index 0000000..994113e --- /dev/null +++ b/src/server/encsvc/EchoEncryptionServiceFactory.js @@ -0,0 +1,46 @@ +/** + * Contains EchoEncryptionServiceFactory + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + + Service = require( './EncryptionService' ), + Transfer = require( './EchoEncryptionServiceTransfer' ); + + +/** + * Factory for creating a dummy encryption service via the echo transfer + * + * This is useful when the encryption service is unnecessary or unavailable, + * such as local development + */ +module.exports = Class( 'EchoEncryptionServiceFactory', +{ + /** + * Creates a new encryption service + * + * @return {EncryptionService} + */ + 'public create': function() + { + return Service( Transfer() ); + }, +} ); + diff --git a/src/server/encsvc/EchoEncryptionServiceTransfer.js b/src/server/encsvc/EchoEncryptionServiceTransfer.js new file mode 100644 index 0000000..1837cbc --- /dev/null +++ b/src/server/encsvc/EchoEncryptionServiceTransfer.js @@ -0,0 +1,71 @@ +/** + * Contains EchoEncryptionServiceTransfer class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class; + + /** + * Interface + * @type {EncryptionServiceTransfer} + */ + EncryptionServiceTransfer = require( './EncryptionServiceTransfer' ); + + +/** + * Simply echoes back the data to be encrypted. Does not do any actual + * encryption. Useful fallback when service is unavailable or is not intended to + * be available (e.g. local development). + */ +module.exports = Class( 'EchoEncryptionServiceTransfer' ) + .implement( EncryptionServiceTransfer ) + .extend( +{ + /** + * Echo back the provided data + * + * @param {string} data data to encrypt + * @param {function( Buffer )} callback function to call with encrypted data + * + * @return undefined + */ + 'public encrypt': function( data, callback ) + { + // simply echo the data back as a buffer + callback( new Buffer( data, 'binary' ) ); + }, + + + /** + * Echo back the provided data + * + * This operation is asynchronous. + * + * @param {Buffer} data data to decrypt + * @param {function( string )} callback function to call with decrypted data + * + * @return undefined + */ + 'public decrypt': function( data, callback ) + { + // simply echo the data back as a buffer + callback( new Buffer( data, 'binary' ) ); + }, +} ); + diff --git a/src/server/encsvc/EncryptionService.js b/src/server/encsvc/EncryptionService.js new file mode 100644 index 0000000..898ce4b --- /dev/null +++ b/src/server/encsvc/EncryptionService.js @@ -0,0 +1,186 @@ +/** + * Contains EncryptionService class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class; + + +/** + * Simple client to encrypt/decrypt data through use of a service + */ +module.exports = Class( 'EncryptionService', +{ + /** + * Identifies a block of data as encrypted through this service + * @type {string} + */ + 'private const ENC_HEADER': "\x06\x03\x05", + + + /** + * Object to do the actual data transfer to/from the service + * @type {EncryptionServiceTransfer} + */ + 'private _transfer': null, + + + /** + * Initializes service client + * + * @param {EncryptionServiceTransfer} transfer transfer object + * + * @return {undefined} + */ + 'public __construct': function( transfer ) + { + this._transfer = transfer; + }, + + + /** + * Encrypts the given data and returns the result + * + * This operation is asynchronous. + * + * @param {Buffer} data data to encrypt + * @param {function(Buffer)} callback function to call with resulting data + * + * @return {undefined} + */ + 'public encrypt': function( data, callback ) + { + var _self = this; + + this._transfer.encrypt( data, function( data ) + { + // return the encrypted data, complete with header + callback( _self._addHeader( data ) ); + } ); + + return this; + }, + + + /** + * Decrypts the given data and returns the result + * + * This operation is asynchronous. + * + * @param {Buffer} data data to decrypt + * @param {function(Buffer)} callback function to call with resulting data + * + * @return {undefined} + */ + 'public decrypt': function( data, callback ) + { + // if the first bytes are not the encryption header, then it may not be + // safe to decrypt + if ( this.isEncrypted( data ) === false ) + { + throw TypeError( "Missing encryption header" ); + } + + this._transfer.decrypt( this._stripHeader( data ), callback ); + return this; + }, + + + /** + * Returns whether the provided data is encrypted + * + * This operates by checking for the encryption header. If it is present, it + * is considered to be encrypted. + * + * @param {Buffer} data data to check + * + * @return {boolean} true if encrypted, otherwise false + */ + 'public isEncrypted': function( data ) + { + if ( !( data instanceof Buffer ) ) + { + return false; + } + + // if it's too small, it can't possibly include a header + if ( data.length < 3 ) + { + return false; + } + + var head_bytes = data.slice( 0, this.__self.$('ENC_HEADER').length ) + .toString( 'ascii' ); + + return ( head_bytes === this.__self.$('ENC_HEADER') ) + ? true + : false; + }, + + + /** + * Prepends encryption header to data + * + * This operation is synchronous, but memcpy() is fairly quick. If we + * experience many problems, we'll make it async. + * + * @param {Buffer} data + * + * @return {Buffer} data, with header + */ + 'private _addHeader': function( data ) + { + var header_len = this.__self.$('ENC_HEADER').length; + + // create a new buffer to store the encrypted data along with the header + var buf = new Buffer( data.length + header_len ); + + // write the header and copy in the encrypted data + buf.write( this.__self.$('ENC_HEADER') ); + data.copy( buf, header_len ); + + return buf; + }, + + + /** + * Strips encryption header from data + * + * @param {Buffer} data + * + * @return {Buffer} data, without header + */ + 'private _stripHeader': function( data ) + { + var header_len = this.__self.$('ENC_HEADER').length; + + // create a new buffer and copy all data except for the header + var buf = new Buffer( data.length - header_len ); + data.copy( buf, 0, header_len ); + + return buf; + }, + + + 'public getHeader': function() + { + return this.__self.$('ENC_HEADER'); + } +} ); + diff --git a/src/server/encsvc/EncryptionServiceTransfer.js b/src/server/encsvc/EncryptionServiceTransfer.js new file mode 100644 index 0000000..2865173 --- /dev/null +++ b/src/server/encsvc/EncryptionServiceTransfer.js @@ -0,0 +1,55 @@ +/** + * Contains EncryptionServiceTransfer class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Interface = require( 'easejs' ).Interface; + + +/** + * Facilitates data transfer between encryption service and client + */ +module.exports = Interface( 'EncryptionServiceTransfer', +{ + /** + * Encrypt the provided data + * + * This operation is asynchronous. + * + * @param {string} data data to encrypt + * @param {function( Buffer )} callback function to call with encrypted data + * + * @return undefined + */ + 'public encrypt': [ 'data', 'callback' ], + + + /** + * Decrypts the provided data + * + * This operation is asynchronous. + * + * @param {Buffer} data data to decrypt + * @param {function( string )} callback function to call with decrypted data + * + * @return undefined + */ + 'public decrypt': [ 'data', 'callback' ], +} ); + diff --git a/src/server/encsvc/HttpEncryptionServiceTransfer.js b/src/server/encsvc/HttpEncryptionServiceTransfer.js new file mode 100644 index 0000000..7adf730 --- /dev/null +++ b/src/server/encsvc/HttpEncryptionServiceTransfer.js @@ -0,0 +1,177 @@ +/** + * Contains HttpEncryptionServiceTransfer class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + http = require( 'http' ), + + /** + * Interface + * @type {EncryptionServiceTransfer} + */ + EncryptionServiceTransfer = require( './EncryptionServiceTransfer' ); + + +/** + * Responsible for performing encrypt/decryption through use of a RESTful + * service + */ +module.exports = Class( 'HttpEncryptionServiceTransfer' ) + .implement( EncryptionServiceTransfer ) + .extend( +{ + /** + * Path to provide for encryption request + * @type {string} + */ + 'private const PATH_ENC': '/enc', + + /** + * Path to provide for decryption request + * @type {string} + */ + 'private const PATH_DEC': '/dec', + + + /** + * Holds URI of encryption REST service + * @type {string} + */ + 'private _host': '', + + /** + * Holds port to connect to + * @type {number} + */ + 'private _port': 0, + + + /** + * Initializes client + * + * @param {string} host service host + * @param {number} port service port + * + * @return undefined + */ + 'public __construct': function( host, port ) + { + this._host = ''+host; + this._port = +port; + }, + + + /** + * Connect to the remote service + * + * The URI and PORT and sent in as arguments beacuse they are encapsulated + * from subtypes. + * + * @param {string} host service URI + * @param {number} port service port + * + * @return {http.ClientRequest} + */ + 'virtual protected connect': function( host, port, path ) + { + var options = { + host: host, + port: port, + path: path, + method: 'POST', + }; + + return http.request( options ); + }, + + + /** + * Encrypt the provided data + * + * This operation is asynchronous. + * + * @param {string} data data to encrypt + * @param {function( Buffer )} callback function to call with encrypted data + * + * @return undefined + */ + 'public encrypt': function( data, callback ) + { + this._send( + this.connect( this._host, this._port, this.__self.$('PATH_ENC') ), + data, + callback + ); + }, + + + /** + * Decrypts the provided data + * + * This operation is asynchronous. + * + * @param {Buffer} data data to decrypt + * @param {function( string )} callback function to call with decrypted data + * + * @return undefined + */ + 'public decrypt': function( data, callback ) + { + this._send( + this.connect( this._host, this._port, this.__self.$('PATH_DEC') ), + data, + callback + ); + }, + + + /** + * Sends a request to the server + * + * @param {Object} client HTTP client + * @param {Buffer} data data to send (enc/dec) + * @param {function( string )} callback function to call with resulting data + * + * @return {undefined} + */ + 'private _send': function( client, data, callback ) + { + client.on( 'response', function( response ) + { + var response_data = ''; + + response.setEncoding( 'binary' ); + + response + .on( 'data', function( chunk ) + { + response_data += chunk; + }) + .on( 'end', function() + { + callback( new Buffer( response_data, 'binary' ) ); + }); + }); + + client.write( data, 'binary' ); + client.end(); + } +} ); + diff --git a/src/server/encsvc/QuoteDataBucketCipher.js b/src/server/encsvc/QuoteDataBucketCipher.js new file mode 100644 index 0000000..10a54c4 --- /dev/null +++ b/src/server/encsvc/QuoteDataBucketCipher.js @@ -0,0 +1,276 @@ +/** + * Handles encryption/decryption of buckets + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + EventEmitter = require( 'events' ).EventEmitter; + + +/** + * Encrypted/decrypts bucket contents + */ +module.exports = Class( 'QuoteDataBucketCipher' ) + .extend( EventEmitter, +{ + /** + * Service used to encrypt data + * @type {EncryptionService} + */ + 'private _encService': null, + + /** + * Fields to encrypt + * @type {Object} + */ + 'private _fields': null, + + + /** + * Initializes with the encryption service to be used and the fields that + * should be encrypted + * + * Only the fields specified will ever be encrypted in a bucket. All other + * fields will be left untouched. + * + * @param {EncryptionService} env_svc encryption service + * @param {Array.} fields names of fields to encrypt + * + * @return {undefined} + */ + 'public __construct': function( enc_svc, fields ) + { + this._encService = enc_svc; + this._fields = fields; + }, + + + /** + * Encrypts the bucket data using the predefined encryption service and + * fields + * + * @param {Bucket} bucket bucket to encrypt + * @param {function()} callback function to call when operation is complete + * + * @return {undefined} + */ + 'public encrypt': function( bucket, callback ) + { + if ( this._fields.length === 0 ) + { + callback(); + return this; + } + + this._encryptFields( + bucket.getData(), + ( this._fields.length - 1 ), + callback + ); + + return this; + }, + + + /** + * Decrypts the bucket data using the predefined encryption service and + * fields + * + * @param {Bucket} bucket bucket to decrypt + * @param {function()} callback function to call when operation is complete + * + * @return {undefined} + */ + 'public decrypt': function( bucket, callback ) + { + if ( this._fields.length === 0 ) + { + callback(); + return this; + } + + this._decryptFields( + bucket.getData(), + ( this._fields.length - 1 ), + callback + ); + + return this; + }, + + + /** + * Recursively encrypts the requested bucket fields + * + * @param {Object} data bucket data + * @param {number} i field index + * @param {function()} callback function to call when all fields are + * encrypted + * + * @return {undefined} + */ + 'private _encryptFields': function( data, i, callback ) + { + var _self = this, + field = this._fields[ i ]; + + try + { + var first = new Buffer( ( data[ field ] || [''] )[ 0 ], 'base64' ); + } + catch ( e ) + { + // data must be invalid + data[ field ] = []; + + c(); + return; + } + + function c() + { + if ( i > 0 ) + { + _self._encryptFields( data, ( i - 1 ), callback ); + } + else + { + callback(); + } + } + + // sanity check (do we have more than 5kB of data?) + if ( first.length > ( 1024 * 5 ) ) + { + // we should never have data this large in the bucket; notify hooks + // and recover + this.emit( 'encrecover', field, first.length ); + for ( var i in data ) + { + data[ field ][ i ] = ''; + } + + c(); + } + + // if the data is already encrypted, then we do not want to continue to + // encrypt it (there was a nasty bug with this...imagine how large the + // data got...) + if ( this._encService.isEncrypted( first ) === true ) + { + c(); + return; + } + + // JSON-encode the data before encrypting to ensure that we're + // encrypting the actual data, not "[object Array]" or something + var json_data = JSON.stringify( data[ field ] ); + + // if the data is undefined, just return + if ( json_data === undefined ) + { + c(); + return; + } + + // encrypt the field and store as a single-element array + this._encService.encrypt( json_data, function( enc_data ) + { + data[ field ] = [ enc_data.toString( 'base64' ) ]; + + // recursively encrypt until we read 0 + c(); + } ); + }, + + + /** + * Recursively decrypts the requested bucket fields + * + * If a certain field is not encrypted, it is ignored. This way, if older + * data isn't encrypted when newer data is, we'll skill be compatible. + * + * @param {Object} data bucket data + * @param {number} i field index + * @param {function()} callback function to call when all fields are + * decrypted + * + * @return {undefined} + */ + 'private _decryptFields': function( data, i, callback ) + { + var _self = this, + field = this._fields[ i ], + + next = function() + { + // recursively encrypt until we read 0 + if ( i > 0 ) + { + _self._decryptFields( data, ( i - 1 ), callback ); + } + else + { + callback(); + } + } + ; + + try + { + var enc_data = ( data[ field ] ) + ? new Buffer( data[ field ][ 0 ], 'base64' ) + : {}; + } + catch ( e ) + { + // data is invalid; clear the field + data[ field ] = []; + + next(); + return; + } + + // if the data is not encrypted, then skip it + if ( this._encService.isEncrypted( enc_data ) === false ) + { + next(); + return; + } + + // encrypt the field and store as a single-element array + this._encService.decrypt( enc_data, function( dec_data ) + { + try + { + // the data was converted to JSON before encryption; convert it + // back + data[ field ] = JSON.parse( dec_data.toString() ); + } + catch ( e ) + { + // if invalid JSON, default to empty + data[ field ] = [""]; + } + + next(); + } ); + } +} ); + diff --git a/src/server/encsvc/RestEncryptionServiceFactory.js b/src/server/encsvc/RestEncryptionServiceFactory.js new file mode 100644 index 0000000..a954e0c --- /dev/null +++ b/src/server/encsvc/RestEncryptionServiceFactory.js @@ -0,0 +1,46 @@ +/** + * Contains RestEncryptionServiceFactory + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + + Service = require( './EncryptionService' ), + Transfer = require( './HttpEncryptionServiceTransfer' ); + + +/** + * Factory for creating a client to a RESTful encryption service + */ +module.exports = Class( 'RestEncryptionServiceFactory', +{ + /** + * Creates a new encryption service + * + * @param {string} host service host + * @param {number} port service port + * + * @return {EncryptionService} + */ + 'public create': function( host, port ) + { + return Service( Transfer( host, port ) ); + }, +} ); + diff --git a/src/server/lock/Semaphore.js b/src/server/lock/Semaphore.js new file mode 100644 index 0000000..216a53f --- /dev/null +++ b/src/server/lock/Semaphore.js @@ -0,0 +1,191 @@ +/** + * Semaphore class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * @todo this is actually a mutex; rename? + */ + +var Class = require( 'easejs' ).Class; + + +/** + * Provides the ability to acquire locks and automatically queues acquisition + * requests to be executed when the lock becomes available. + * + * N.B. This implementation assumes a single-threaded implementation; it must be + * modified to avoid lock race conditions should a multi-threaded server be + * used. + */ +module.exports = Class( 'Semaphore', +{ + /** + * Timestamp representing the date of the current lock, if any, per id + * @type {Object} + */ + 'private _lock': {}, + + /** + * Queue of acquisition requsts waiting for a particular lock, per id + * @type {Object} + */ + 'private _queue': {}, + + + /** + * Attempts to acquire the lock + * + * If the lock is available, immediately invokes the provided continuation. + * + * If the lock is not available, queues the continuation to be invoked as + * soon as the lock is freed. + * + * Once the lock is successfully acquired, the provided continuation is + * invoked with a single argument to another continuation that may be + * invoked in order to free the lock. + * + * @param {string|number} id lock id + * + * @param {function(function())} c continuation to invoke when lock is + * successfully acquired + * + * @return {Semaphore} self + */ + 'public acquire': function( id, c ) + { + var _self = this; + + this._queue[ id ] = this._queue[ id ] || []; + + // if the lock is already acquired, then we must wait for it to be + // free'd; block and add to callback queue + if ( this._lock[ id ] ) + { + this._queue[ id ].push( function() + { + // try again (should succeed) + _self.acquire( id, c ); + } ); + + return; + } + + this._lock[ id ] = ( new Date() ).getTime(); + + // return a continuation that will free the lock (this is an alternative + // to providing a free() method, which prevents impatient requests from + // freeing the lock themselves) + c( function() + { + _self._free( id, _self._lock[ id ] ); + } ); + }, + + + /** + * Attempts to free a lock + * + * If ts is provided, will check against the timestamp of the current lock + * for the given id. If they do not match, then the lock will not be freed; + * this prevents malicious or accidental frees by acquiring a lock and + * storing the free continuation in memory to be invoked repeatedly to + * forcefully obtain a lock. + * + * After the lock is freed, this method will immediately return. On the next + * tick, the acquisition queue will then be processed. + * + * @param {number|string} id lock id + * @param {number} ts timestamp to validate against + * + * @return {undefined} + */ + 'private _free': function( id, ts ) + { + // prevent previous lock acquisition requests from freeing future ones + if ( ( ts !== undefined ) && ( ts !== this._lock[ id ] ) ) + { + return; + } + + // yes, this is known to slow down v8 a little, but we need to free the + // memory + delete this._lock[ id ]; + + // let the current request finish before we begin processing the next + // lock request + var queue = this._queue; + process.nextTick( function() + { + // are there any queued acquisition requests? (perform this check + // after the tick to ensure that the array is not modified between + // our check and shift) + if ( queue[ id ] && queue[ id ].length ) + { + // no, this is not a typo. + queue[ id ].shift()(); + return; + } + + delete queue[ id ]; + } ); + }, + + + /** + * Frees locks that have been acquired for longer than the given amount of + * time + * + * Note that this operation is potentially unsafe; only with sufficient time + * intervals so as to determine that a request is unlikely to ever free the + * lock, thus avoiding a deadlock. + * + * This operation is synchronous. + * + * @param {number} maxtime maximum number of time in milliseconds + * + * @param {function(string|number)} c continuation to invoke with id of + * freed lock(s), if any + * + * @return {Semaphore} self + */ + 'public freeStale': function( maxtime, c ) + { + var now = ( new Date() ).getTime(); + + for ( var id in this._lock ) + { + if ( ( now - this._lock[ id ] ) > maxtime ) + { + // notify caller + c && c( id ); + + // free the lock + this._free( id ); + } + } + + return this; + }, + + + 'public isLocked': function( id ) + { + return ( this._lock[ id ] > 0 ); + } +} ); + diff --git a/src/server/log/AccessLog.js b/src/server/log/AccessLog.js new file mode 100644 index 0000000..f3b0948 --- /dev/null +++ b/src/server/log/AccessLog.js @@ -0,0 +1,79 @@ +/** + * HTTP access log + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + UserRequest = require( '../request/UserRequest' ); + + +/** + * Logs HTTP access in Apache's combined log format + */ +module.exports = Class( 'AccessLog' ) + .extend( require( './Log' ), +{ + /** + * Monitors a user request for the end of the connection and adds an entry + * to the access log + * + * @param UserRequest user_request + * + * @return AccessLog self + */ + 'public attach': function( user_request ) + { + if ( !( Class.isA( UserRequest, user_request ) ) ) + { + throw new TypeError( + 'UserRequest expected, ' + ( user_request.toString() ) + + ' given' + ); + } + + var self = this, + request = user_request.getRequest(); + + // log when the request is complete + user_request.on( 'end', function() + { + // determine the remote address (in case we're behind a proxy, look + // at X-Forwarded-For) + var remote_addr = user_request.getRemoteAddr(), + username = user_request.getSession().agentId() || '-'; + + // access log (apache combined log format) + self.write( '%s %s - - [%s] "%s %s HTTP/%s" %d %d "%s" "%s"', + remote_addr, + username, + new Date(), + request.method, + request.url, + request.httpVersion, + user_request.getResponseCode(), + user_request.getResponseLength(), + request.headers['referer'] || '-', + request.headers['user-agent'] || '-' + ); + }); + + return this; + } +} ); + diff --git a/src/server/log/Log.js b/src/server/log/Log.js new file mode 100644 index 0000000..8d29b98 --- /dev/null +++ b/src/server/log/Log.js @@ -0,0 +1,125 @@ +/** + * Generic logger + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + + +var fs = require( 'fs' ), + sprintf = require( 'php' ).sprintf, + Class = require( 'easejs' ).Class; + + +/** + * Base logging class + */ +module.exports = Class( 'Log', +{ + /** + * File descriptor (if any) + * @var String|null + */ + 'private _fd': null, + + /** + * Name of file that is being logged to (if any) + * @var String + */ + 'private _filename': '', + + /** + * Log to standard out + * @var Boolean + */ + stdout: true, + + /** + * Log to standard error + * @var Boolean + */ + stderr: false, + + + set filename( val ) + { + if ( val === null ) + { + this._file = null; + return; + } + + // must be a string if not null + var fn = ''+( val ); + + this._filename = fn; + this._fdWaiting = true; + + // open synchronously (logs are important, yo! - which is why we also + // aren't going to enclose this in a try/catch block) + this._fd = fs.openSync( fn, 'a' ); + }, + + get filename() + { + return this._filename; + }, + + + /** + * Initializes access log + * + * @param String filename file to log output to + * + * @return undefined + */ + 'virtual public __construct': function( filename ) + { + if ( filename ) + { + this.filename = ''+( filename ); + } + }, + + + /** + * Writes to log in sprintf-style manner + * + * @return Log self + */ + 'public write': function() + { + // convert arguments to an array + var args = Array.prototype.slice.call( arguments ); + + // log to standard out? + if ( this.stdout === true ) + { + console.log.apply( this, args ); + } + + // log to file? + if ( this._fd !== null ) + { + var buffer = new Buffer( sprintf.apply( this, args ) + "\n" ); + fs.write( this._fd, buffer, 0, buffer.length, null ); + } + + return this; + }, +} ); + diff --git a/src/server/log/PriorityLog.js b/src/server/log/PriorityLog.js new file mode 100644 index 0000000..7bc47df --- /dev/null +++ b/src/server/log/PriorityLog.js @@ -0,0 +1,116 @@ +/** + * Priority log + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class; + + +/** + * Logs messages only if they meet the requested priority (allowing for varying + * levels of verbosity) + */ +module.exports = Class( 'PriorityLog' ) + .extend( require( './Log' ), +{ + 'PRIORITY_ERROR': 0, + 'PRIORITY_IMPORTANT': 1, + 'PRIORITY_DB': 2, + 'PRIORITY_INFO': 3, + 'PRIORITY_SOCKET': 5, + + /** + * Highest priority to log + * @var Integer + */ + 'private _priority': 10, + + + /** + * Initialize logger with filename and priorities to log + * + * @param String filename file to log to + * @param Integer priority highest priority to log + * + * @return undefined + */ + 'override public __construct': function( filename, priority ) + { + this.priority = +priority || this.priority; + + // call the parent constructor + this.__super( filename ); + }, + + + /** + * Write to the log at the given priority + * + * If the priority is less than or equal to the set priority for this + * object, it will be logged. Otherwise, the message will be ignored. + * + * The first argument should be the priority. The remaining arguments should + * be provided in a sprintf()-style fashion + * + * @return void + */ + 'public log': function() + { + var args = Array.prototype.slice.call( arguments ), + priority = +args.shift(); + + // don't log if the provided priority is outside the scope that was + // requested + if ( priority > this._priority ) + { + return this; + } + + // if this was an error, prefix it with the error char (easy grepping) + var status_char = ' '; + switch ( priority ) + { + case this.PRIORITY_IMPORTANT: + status_char = '*'; + break; + + case this.PRIORITY_ERROR: + status_char = '!'; + break; + + case this.PRIORITY_DB: + status_char = '%'; + break; + + case this.PRIORITY_SOCKET: + status_char = '>'; + break; + } + + // timestamp + args[0] = status_char + '[' + ( new Date() ).toString() + '] ' + + args[0]; + + // forward to write method + this.write.apply( this, args ); + + return this; + }, +} ); + diff --git a/src/server/quote/ProgramQuoteCleaner.js b/src/server/quote/ProgramQuoteCleaner.js new file mode 100644 index 0000000..1b3bd98 --- /dev/null +++ b/src/server/quote/ProgramQuoteCleaner.js @@ -0,0 +1,154 @@ +/** + * Contains ProgramQuoteCleaner + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class; + + +module.exports = Class( 'ProgramQuoteCleaner', +{ + /** + * Program associated with the quote + * @type {Program} + */ + 'private _program': null, + + + __construct: function( program ) + { + this._program = program; + }, + + + 'public clean': function( quote, callback ) + { + // consider it an error to attempt cleaning a quote with the incorrect + // program, which would surely corrupt it [even further] + if ( quote.getProgramId() !== this._program.getId() ) + { + callback( null ); + return; + + // TODO: once we move the program redirect before this check + // callback( Error( 'Program mismatch' ) ); + } + + // fix any problems with linked groups + this._fixLinkedGroups( quote, function( err ) + { + // done + callback( err ); + } ); + }, + + + 'private _fixLinkedGroups': function( quote, callback ) + { + var links = this._program.links, + update = {}; + + for ( var link in links ) + { + var len = this._getLinkedIndexLength( link, quote ), + cur = links[ link ]; + + // for each field less than the given length, correct it by adding + // the necessary number of indexes and filling them with their + // default values + for ( var i in cur ) + { + var field = cur[ i ]; + + if ( !field ) + { + continue; + } + + var data = quote.getDataByName( field ), + flen = data.length; + + //varnity check + if ( !( Array.isArray( data ) ) ) + { + data = []; + flen = 0; + } + + // if the length matches, continue + if ( flen === len ) + { + continue; + } + else if ( flen > len ) + { + // length is greater; cut it off + data = data.slice( 0, len ); + } + + var d = this._program.defaults[ field ] || ''; + for ( var j = flen; j < len; j++ ) + { + data[ j ] = d; + } + + update[ field ] = data; + } + } + + // perform quote update a single time once we have decided what needs to + // be done + quote.setData( update ); + + // we're not async, but we'll keep with the callback to simplify such a + // possibility in the future + callback( null ); + }, + + + 'private _getLinkedIndexLength': function( link, quote ) + { + var fields = this._program.links[ link ], + chklen = 20, + len = 0; + + // loop through the first N fields, take the largest index length and + // consider that to be the length of the group + for ( var i = 0; i < chklen; i++ ) + { + var field = fields[ i ]; + if ( !field ) + { + break; + } + + var data = quote.getDataByName( field ); + if ( !( Array.isArray( data ) ) ) + { + continue; + } + + // increaes the length if a larger field was found + len = ( len > data.length ) ? len : data.length; + } + + return len; + } +} ); + diff --git a/src/server/quote/ServerSideQuote.js b/src/server/quote/ServerSideQuote.js new file mode 100644 index 0000000..908e6a5 --- /dev/null +++ b/src/server/quote/ServerSideQuote.js @@ -0,0 +1,136 @@ +/** + * Augments a quote with additional data for use by the quote server + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + Quote = require( '../../quote/Quote' ), + BaseQuote = require( '../../quote/BaseQuote' ); + + +module.exports = Class( 'ServerSideQuote' ) + .implement( Quote ) + .extend( BaseQuote, +{ + /** + * Program version + * @type {string} + */ + 'private _pver': '', + + /** + * Credit score reference number + * @type {number} + */ + 'private _creditScoreRef': 0, + + /** + * Unix timestamp containing date of last premium calculation + * @type {number} + */ + 'private _lastPremDate': 0, + + /** + * Unix timestamp containing date of first premium calculation + * @type {number} + */ + 'private _rated_date': 0, + + + 'public setProgramVersion': function( version ) + { + this._pver = ''+( version ); + return this; + }, + + + 'public getProgramVersion': function() + { + return this._pver; + }, + + + 'public setCreditScoreRef': function( ref ) + { + this._creditScoreRef = +ref; + return this; + }, + + + 'public getCreditScoreRef': function() + { + return this._creditScoreRef; + }, + + + /** + * Set the date that the premium was calculated as a Unix timestamp + * + * @param {number} timestamp Unix timestamp representing premium date + * + * @return {Quote} self + */ + 'public setLastPremiumDate': function( timestamp ) + { + this._lastPremDate = ( timestamp || 0 ); + return this; + }, + + + /** + * Set the timestamp of the first time quote was rated + * + * @param {number} timestamp Unix timestamp representing first rated date + * + * @return {Quote} self + */ + 'public setRatedDate': function( timestamp ) + { + // do not overwrite date if it exists + if ( this._rated_date === 0 ) + { + this._rated_date = ( +timestamp || 0 ); + } + + return this; + }, + + + /** + * Retrieve the last time the premium was calculated + * + * @return {number} last calculated time or 0 + */ + 'public getLastPremiumDate': function() + { + return ( this._lastPremDate || 0 ); + }, + + + /** + * If the quote has been rated + * + * @return {boolean} has been rated + */ + 'public getRatedDate': function() + { + return this._rated_date; + } +} ); + diff --git a/src/server/rater/DslRater.js b/src/server/rater/DslRater.js new file mode 100644 index 0000000..abb1cfb --- /dev/null +++ b/src/server/rater/DslRater.js @@ -0,0 +1,229 @@ +/** + * Handles rating with local, JS-compiled TAME-written raters + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + Rater = require( './Rater' ), + EventEmitter = require( 'events' ).EventEmitter, + + DslRaterContext = require( './DslRaterContext' ); + + +module.exports = Class( 'DslRater' ) + .extend( EventEmitter, +{ + /** + * List of raters + * @type {Array.>} + */ + 'private _raters': [], + + /** + * ResultSet constructor (used in place of a factory) + * @type {Function} + */ + 'private _ResultSet': null, + + + __construct: function( raters, ResultSet ) + { + this._raters = raters; + this._ResultSet = ResultSet; + }, + + + 'public rate': function( context ) + { + if ( this._raters.length === 0 ) + { + // TODO: this is a BS answer + callback( + Error( 'No dwelling raters are available at this time.' ), + null + ); + return this; + } + + if ( !( Class.isA( DslRaterContext, context ) ) ) + { + throw TypeError( "Invalid DslRaterContext provided" ); + } + + this._doRate( this._raters.slice( 0 ), context ); + }, + + + 'private _queueNext': function() + { + var _self = this, + args = arguments; + + process.nextTick( function() + { + _self._doRate.apply( _self, args ); + } ); + }, + + + /** + * Process next available supplier until complete + */ + 'private _doRate': function( queue, context ) + { + var _self = this, + args = arguments; + + var rater = queue.pop(); + if ( rater === undefined ) + { + this._complete( context ); + return; + } + + // null means that the rater failed to load + if ( rater === null ) + { + // TODO: log error + this._queueNext.apply( this, args ); + return; + } + + var name = rater.supplier, + meta = rater.rater.meta, + set = this._ResultSet( name ); + + // give context the chance to augment the data and determine how many + // times we should rate with a given supplier; this continuation will be + // called for each time that the context wishes to perform rating + var data = context.rate( name, meta, function( data, rcontext ) + { + try + { + var single = rater( data ); + + // ensures that any previous eligibility errors are cleared out + single.ineligible = ''; + single.submit = ''; + } + catch ( e ) + { + // ineligible. + single = { + ineligible: e.message, + submit: '', + premium: 0.00, + }; + } + + // give the context to process the result of the rating before we + // perform any of our own processing (that way they can trigger + // submits, etc) + single = context.processResult( single, meta, rcontext ); + + _self._processSubmits( rater, single, context, data, rcontext ) + ._flagEligibility( single ) + ._cleanResult( single ); + + // purposely omitted third argument + set.addResult( single, rcontext ); + }, complete ); + + // to be called by context when rating is complete for this rater + function complete() + { + context.addResultSet( name, set ); + _self._queueNext.apply( _self, args ); + } + }, + + + 'private _processSubmits': function( + rater, single, context, data, rcontext + ) + { + // ineligible results cannot submit, as they did not complete rating + if ( single.ineligible ) + { + return this; + } + + // submission processing may be disabled (e.g. via a runtime flag) + if ( !( context.canSubmit( single, data, rcontext ) ) ) + { + return this; + } + + var c = single.__classes; + if ( !( c.submit ) ) + { + return this; + } + + var submits = []; + for ( cname in c ) + { + // Process submit classifications if they are *true*. Classes + // that are suffixed with a dash represent a generated rule that + // should _not_ be displayed to the user + if ( /^submit-.*[^-]$/.test( cname ) && c[ cname ] ) + { + submits.push( this._getCdesc( cname, rater ) ); + } + } + + single.submit = submits.join( '; ' ); + return this; + }, + + + 'private _flagEligibility': function( single ) + { + // from a broker's perspective + single._unavailable = ( single.submit || single.ineligible ) + ? '1' + : '0'; + + return this; + }, + + + 'private _cleanResult': function( result ) + { + return this; + }, + + + 'private _getCdesc': function( cname, rater, default_value ) + { + default_value = ( default_value === undefined ) + ? cname + : default_value; + + return rater.rater.classify.desc[ cname ] || default_value; + }, + + + 'private _complete': function( context ) + { + // let the context know that we're finished rating + context.complete(); + }, +} ); + diff --git a/src/server/rater/DslRaterContext.js b/src/server/rater/DslRaterContext.js new file mode 100644 index 0000000..ffd923f --- /dev/null +++ b/src/server/rater/DslRaterContext.js @@ -0,0 +1,343 @@ +/** + * Provides context-specific data to the DslRater + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + Rater = require( './Rater' ), + EventEmitter = require( 'events' ).EventEmitter, + Quote = require( '../../quote/Quote' ); + + +module.exports = Class( 'DslRaterContext' ) + .extend( EventEmitter, +{ + /** + * Hash of classes that will result in a global submit + * @type {Object} + */ + 'private _globalSubmits': {}, + + /** + * Whether a particular global submit has been triggered + * @type {Object} + */ + 'private _hasGsubmit': {}, + + /** + * Rater corestrictions + * @type {Object} + */ + 'private _restrict': {}, + + /** + * Quote data with which to rate + * @type {Object} + */ + 'private _data': null, + + /** + * Result sets + * @type {Object} + */ + 'private _results': [], + + /** + * Number of available results + * @type {number} + */ + 'private _availCount': 0, + + /** + * Total number of results + * @type {number} + */ + 'private _totalCount': 0, + + + __construct: function( data ) + { + this._data = data; + this.init(); + }, + + + /** XXX: return to protected (see commit message) **/ + 'public getSourceData': function() + { + return this._data; + }, + + + 'virtual protected init': function() + { + // may be implemented by subtypes + }, + + + 'virtual public rate': function( name, meta, rate, complete ) + { + rate( this._data ); + }, + + + 'virtual public processResult': function( result, meta, context ) + { + return result; + }, + + + /** + * Add a completed ResultSet + * + * @param {string} name supplier/rater name + * @param {ResultSet} set completed rating result set + * + * @return {DslRaterContext} self + */ + 'public addResultSet': function( name, set ) + { + this._totalCount += set.getResultCount(); + this._availCount += set.getAvailableCount(); + + this._checkGlobalSubmits( set ); + + this._results.push( set ); + return this; + }, + + + /** + * Checks each result in a set to determine if a global submit is to be + * triggered + * + * @param {ResultSet} set result set to scan + * + * @return {undefined} + */ + 'private _checkGlobalSubmits': function( set ) + { + var _self = this; + set.forEachResult( function( result, rcontext ) + { + if ( !result.__classes ) + { + return; + } + + for ( var cname in _self._globalSubmits ) + { + if ( result.__classes[ cname ] ) + { + _self._hasGsubmit[ cname ] = true; + } + } + } ); + }, + + + 'public setGlobalSubmits': function( submits ) + { + var i = submits.length; + while ( i-- ) + { + this._globalSubmits[ submits[ i ] ] = true; + } + + return this; + }, + + + 'public restrictSupplier': function( id, restricts ) + { + this._restrict[ id ] = restricts; + return this; + }, + + + 'virtual public canSubmit': function( result, rcontext ) + { + return true; + }, + + + 'public complete': function() + { + // allow context some time to manipulate the results mercilessly + this._availCount = this.processCompleted( + this._results, this._availCount + ); + + this._processGlobalSubmits(); + this._emitResults(); + }, + + + /** + * Process result sets after rating is complete + * + * This is called before post-processing, which flattens into the final + * result; therefore, the ResultSet objects themselves can still be + * manipulated. + * + * If the availablity count is unchanged, COUNT should be returned. + * + * @param {Array.} results result sets + * @param {number} count availaabilty count + * + * @return {number} availability count after processing + */ + 'virtual protected processCompleted': function( results, count ) + { + return this._processCorestrictions( results, count ); + }, + + + /** + * FIXME: I need to be cleaned up + */ + 'private _processCorestrictions': function( results, count ) + { + var _self = this; + + if ( this._restrict.length === 0 ) + { + return {}; + } + + // index results by id + var id_results = results.reduce( function( byid, result_set ) + { + byid[ result_set.getId() ] = result_set; + return byid; + }, {} ); + + var rnames = Object.keys( this._restrict ); + + return rnames.reduce( function( count, name ) + { + var result_set = id_results[ name ]; + + if ( !( result_set && result_set.getAvailableCount() > 0 ) ) + { + return count; + } + + // array of suppliers that we cannot be displayed with + var chk = _self._restrict[ name ]; + for ( var chk_i in chk ) + { + var chkname = chk[ chk_i ]; + + var chk_results; + + if ( ( chk_results = id_results[ chk ] ) + && ( chk_results.getAvailableCount() > 0 ) + ) + { + var n = chk_results.getResultCount(); + + chk_results.forEachResult( function( result ) + { + if ( +result._unavailable ) + { + return; + } + + result._unavailable = '1'; + result.ineligible = 'Cannot display with ' + + 'carrier ' + name; + + count--; + } ); + + return count; + } + } + + return count; + }, count ); + }, + + + /** + * Apply global submits to result sets + * + * If global submits have been found, they will be applied to each result + * individually, resulting in submits across the board. + * + * @return {undefined} + */ + 'private _processGlobalSubmits': function() + { + for ( var cname in this._hasGsubmit ) + { + this._results.forEach( function( set ) + { + set.forEachResult( function( result ) + { + // TODO: class desc + result.submit += ( ( result.submit ) ? '; ' : '' ) + + cname; + + result._unavailable = '1'; + this._availCount--; + } ); + } ); + } + }, + + + 'private _emitResults': function() + { + this.emit( 'complete', + this.postProcessResults( this._results ) + ); + }, + + + 'virtual protected postProcessResults': function( results ) + { + var ret = {}; + + results.forEach( function( set ) + { + var id = set.getId(), + merged = set.getMergedResults(); + + // prefix each field with the result set id (TODO: this should + // really be part of a decorator, not this class) + for ( var field in merged ) + { + if ( !field ) + { + continue; + } + + ret[ id + '_' + field ] = merged[ field ]; + } + + } ); + + ret.__prem_avail_count = [ this._availCount ]; + + return ret; + } +} ); + diff --git a/src/server/rater/HttpRater.js b/src/server/rater/HttpRater.js new file mode 100644 index 0000000..526cdbc --- /dev/null +++ b/src/server/rater/HttpRater.js @@ -0,0 +1,269 @@ +/** + * Remote rater over HTTP + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * @todo it's used with a PHP endpoint, but doesn't have to be called + * "HttpRater" + */ + +var Class = require( 'easejs' ).Class, + Rater = require( './Rater' ), + + querystring = require( 'querystring' ) +; + + +/** + * Rates using one of the PHP raters + */ +module.exports = Class( 'HttpRater' ) + .implement( Rater ) + .extend( +{ + /** + * Used to communicate with PHP web server + * @type {http} + */ + 'private _client': null, + + /** + * Currently logged in session (performing rating) + * @type {UserSession} + */ + 'private _session': null, + + /** + * Alias of rater to use if only a single rate is desired + * @type {string} + */ + 'private _only': '', + + /** + * Number of seconds before rating will abort with a timeout error + * + * This is intended to provide more useful information, rather than waiting + * for a socket timeout. + * + * @var number + */ + 'private _timeout': 100, + + + /** + * Initialize rater with the id of the program + * + * @param {http} http_client Node.js http client + * @param {UserSession} session current session (performing rating) + * @param {string} host hostname, if different from host addr + * @param {string} remote_path endpoint path on server + */ + __construct: function( http_client, session, host, remote_path ) + { + this._client = http_client; + this._session = session; + this._host = ( host ) ? ''+host : this._client.host; + this._path = ''+remote_path; + }, + + + /** + * Indicate that only the given alias should be used when rating + * + * @param {string} alias alias to rate with + * + * @return {HttpRater} self + */ + 'public only': function( alias ) + { + if ( !alias ) + { + return this; + } + + this._only = ''+( alias ); + return this; + }, + + + /** + * Rate the provided quote + * + * The agent id is appended to the arguments for each request. This is + * necessary, since we are making the request internally, not the broker. As + * such, the session data will not be populated within PHP as it would + * normally. We may wish for a better solution in the future (e.g. passing + * the actual session id so we can simply load all of the data from within + * PHP). + * + * Please note: args will be modified to append agent_id. It is not removed, + * so be careful when passing objects to this method. + * + * @param {Quote} quote quote to rate + * @param {function(err,data)} callback to call when complete + * + * @return {HttpRater} self + */ + 'public rate': function( quote, args, callback ) + { + var _self = this, + + // data to include in GET request (errdetail will allow us to see + // the actual error message) + data = querystring.stringify( { + quoteId: quote.getId(), + errdetail: 1, + } ), + + path = ( this._path + '?' + data ) + ; + + // append agent_id of current session and agent id associated with the + // quote + args.agent_id = this._session.agentId(); + args.quote_agent_id = quote.getAgentId(); + + // we may wish to only retrieve the rates for a single alias + if ( this._only ) + { + args.only = this._only; + args.noautodefer = true; + } + + // return an error if we do not receive a response within a + // certain period of time + var http_resp = null, + req = null, + timeout = setTimeout( function() + { + // if we have a response, close the connection + http_resp && http_resp.end(); + req && req.end(); + + callback( Error( 'Rating timeout' ), null ); + }, ( _self._timeout * 1000 ) ); + + function _clientErr( err ) + { + clearTimeout( timeout ); + callback( err, null ); + }; + + function _cleanup() + { + clearTimeout( timeout ); + _self._client.removeListener( 'error', _clientErr ); + } + + this._client.once( 'error', _clientErr ); + + req = this._client.request( 'POST', path, { host: this._host } ) + .on( 'response', function( response ) + { + var data = ''; + + http_resp = response; + + response + .on( 'data', function( chunk ) + { + data += chunk.toString( 'utf8' ); + } ) + .on( 'end', function() + { + _cleanup(); + _self._parseResponse( data, callback ); + } ) + .on( 'error', function( err ) + { + _cleanup(); + callback( err, null ); + } ) + } ) + .on( 'error', function( err ) + { + _cleanup(); + callback( err, null ); + } ) + .end( JSON.stringify( args ) ); + }, + + + /** + * Parses response from webserver + * + * The callback function will be called with any error (or null) as the + * first argument and the response data (or null if error) as the second. + * This is a common convention for callbacks since thorwing exceptions does + * not work well with asynchronous calls (since async only executed after + * stack has been cleared). + * + * @param {string} data_str response string from webserver + * @param {function(err, data)} callback function to call with response + * + * @return {undefined} + */ + 'private _parseResponse': function( data_str, callback ) + { + var error = null, + retdata = null; + + try + { + data = JSON.parse( data_str ); + + // if the server responded with an error, return it + if ( data.hasError !== false ) + { + error = Error( + 'Server responded with error: ' + + ( data.content || '(empty)' ) + ); + } + else + { + // return the content of the response rather than the entire + // response (we treat the rest a bit like one would treat a + // header) + retdata = data.content; + } + } + catch ( e ) + { + error = TypeError( 'Invalid JSON provided: ' + data_str ); + } + + callback( error, retdata ); + }, + + + /** + * Sets number of seconds before rating attempt will time out and trigger a + * rating error + * + * @param {number} seconds timeout in seconds + * + * @return {HttpRater} self + */ + 'public setTimeout': function( seconds ) + { + this._timeout = +seconds; + return this; + }, +} ); + diff --git a/src/server/rater/Rater.js b/src/server/rater/Rater.js new file mode 100644 index 0000000..f10953b --- /dev/null +++ b/src/server/rater/Rater.js @@ -0,0 +1,40 @@ +/** + * Contains Rater interface + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Interface = require( 'easejs' ).Interface; + + +/** + * Represents a rater that will generate a quote from a given set of values + */ +module.exports = Interface( 'Rater', +{ + /** + * Asynchronously performs rating using the data from the given bucket + * + * @param {Quote} quote to rate + * @param {function()} callback function to call when complete + * + * @return {Rater} self + */ + 'public rate': [ 'quote', 'args', 'callback' ], +} ); + diff --git a/src/server/rater/ResultSet.js b/src/server/rater/ResultSet.js new file mode 100644 index 0000000..17d8f25 --- /dev/null +++ b/src/server/rater/ResultSet.js @@ -0,0 +1,182 @@ +/** + * Holds rating results and metadata + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class; + + +module.exports = Class( 'ResultSet', +{ + /** + * Unique identifier for result set + * @type {string} + */ + 'private _id': '', + + /** + * Individual results as separate objects + * @type {Array.} + */ + 'private _results': [], + + /** + * Metadata associated with individual results + * @type {Array.} + */ + 'private _meta': [], + + /** + * Results as a combined object + * @type {Object} + */ + 'private _mergedResults': {}, + + /** + * Current result index + * @type {number} + */ + 'private _resulti': 0, + + /** + * Number of available results + * @type {number} + */ + 'private _availCount': 0, + + + __construct: function( id ) + { + this._id = ''+id; + }, + + + 'public addResult': function( result, meta, index ) + { + if ( index === undefined ) + { + index = this._resulti++; + } + else if ( index > this._resulti ) + { + index = +index; + this._resulti = index; + } + + if ( result._unavailable === '0' ) + { + this._availCount++; + } + + this._results[ index ] = result; + this._meta[ index ] = meta; + }, + + + 'public getAvailableCount': function() + { + return this._availCount; + }, + + + 'public getResultCount': function() + { + return this._results.length; + }, + + + 'public getId': function() + { + return this._id; + }, + + + /** + * Merge each result into a combined object + * + * Each field will contain an array with a value from each result. If a + * result is missing a field, then the value for that particular index will + * be defaulted to the empty string. + * + * @return {Object} merged results + */ + 'public getMergedResults': function() + { + var set = {}, + fields = {}; + + // merge all fields into a single object + this._results.forEach( function( result, i ) + { + for ( var field in result ) + { + ( set[ field ] = set[ field ] || [] )[ i ] = result[ field ]; + + fields[ field ] |= 0; + fields[ field ]++; + } + } ); + + // fill in gaps in the array (if a field is not defined in a result) + for ( var field in fields ) + { + var chk = set[ field ]; + + // if the destination array is larger than the number of items it + // contains, then we have holes to fill + if ( fields[ field ] === chk.length ) + { + continue; + } + + var i = chk.length; + while ( i-- ) + { + // default to an empty string + chk[ i ] = chk[ i ] || ''; + } + } + + return this._processSet( set ); + }, + + + 'public forEachResult': function( c ) + { + var i = this._results.length; + while ( i-- ) + { + c( this._results[ i ], this._meta[ i ], i ); + } + }, + + + 'private _processSet': function( set ) + { + // combined availability + set._unavailable_all = [ + ( this._availCount === 0 ) + ? '1' + : '0' + ]; + + return set; + } +} ); + diff --git a/src/server/rater/service.js b/src/server/rater/service.js new file mode 100644 index 0000000..db35aaf --- /dev/null +++ b/src/server/rater/service.js @@ -0,0 +1,261 @@ +/** + * Rating service + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * @todo decouple insurance terminology + */ + +var child_process = require( 'child_process' ), + child = null; + +// POSIX signal numbers +const _signum = { + SIGHUP: 1, + SIGINT: 2, + SIGQUIT: 3, + SIGILL: 4, + SIGTRAP: 5, + SIGABRT: 6, + SIGIOT: 6, + SIGBUS: 7, + SIGFPE: 8, + SIGKILL: 9, + SIGUSR1: 10, + SIGSEGV: 11, + SIGUSR2: 12, + SIGPIPE: 13, + SIGALRM: 14, + SIGTERM: 15, + SIGSTKFLT: 16, + SIGCHLD: 17, + SIGCONT: 18, + SIGSTOP: 19, + SIGTSTP: 20, + SIGTTIN: 21, + SIGTTOU: 22, + SIGURG: 23, + SIGXCPU: 24, + SIGXFSZ: 25, + SIGVTALARM: 26, + SIGPROF: 27, + SIGWINCH: 28, + SIGIO: 29, + SIGPOLL: 29, + SIGPWR: 30, + SIGSYS: 31, +}; + + +exports.init = function( logc, errc ) +{ + if ( child !== null ) + { + // end the child + child.kill( 'SIGHUP' ); + return; + } + + child = child_process.fork( + __dirname + '/thread.js', + [], + + // pass all our arguments to the child, incrementing the debug + // port if --debug was provided + { + env: process.env, + execArgv: process.execArgv.map( arg => + { + const debug = arg.match( /^--debug(?:=(\d+))?$/ ); + + if ( debug === null ) + { + return arg; + } + + // our debug port will be our parent's plus one + const parent_port = +debug[ 1 ] || 5858, + new_port = parent_port + 1; + + return '--debug=' + new_port; + } ), + } + ); + + child.on( 'message', function( msg ) + { + var cmd = msg.cmd; + + switch ( cmd ) + { + case 'log': + logc( msg.msg ); + break; + + case 'error': + errc( msg.msg, msg.stack ); + break; + + case 'rate-reply': + rateReply( msg, errc ); + break; + + default: + errc( "Unknown message from rater thread: " + msg.cmd ); + } + } ); + + child.on( 'exit', function( excode, sig ) + { + child = null; + + // c'mon node...use the POSIX exit status + if ( ( excode == null ) && sig ) + { + excode = getsignum( sig ); + } + + if ( excode !== 0 ) + { + errc( + "Rater thread exited with error code " + excode + + ( ( sig ) ? " (" + sig + ")" : '' ) + ); + + purgeRateRequests( "Rater thread died unexpectedly" ); + } + else + { + logc( "Rater thread exited gracefully." ); + } + + // purge anything remaining in the queue (hopefully nothing; this is + // purely a catch-all in case a request somehow snuck in due to a bug) + purgeRateRequests( "Rater thread is being restarted." ); + + // start a new thread + logc( "Restarting rater thread..." ); + exports.init( logc, errc ); + } ); +}; + + +var _requests = {}; + +/** + * Returns the rater associated with the given id + * + * @param {string} id rater id + * + * @return {Object|null} requested rater or null if it does not exist + */ +exports.byId = function( id ) +{ + // temporary until refactoring is complete + return { rate: function( quote, session, indv, success, failure ) + { + var rqid = createRqid( quote.getId(), indv ); + _requests[ rqid ] = [ success, failure ]; + + child.send( { + cmd: 'rate', + supplier: id, + indv: indv, + rqid: rqid, + + quote: { + id: quote.getId(), + agentId: quote.getAgentId(), + agentName: quote.getAgentName(), + data: quote.getBucket().getData(), + + creditScoreRef: quote.getCreditScoreRef(), + }, + + agentId: session.agentId(), + internal: session.isInternal(), + } ); + } }; +} + + +function rateReply( msg, errc ) +{ + var rqid = msg.rqid, + rq = _requests[ rqid ]; + + // does this request exits? + if ( !rq ) + { + // uh...beaver? + errc( "Reply to unknown rqid: " + rqid ); + return; + } + + // remove the rqid from the pending request list; we now hold the only + // reference + delete _requests[ rqid ]; + + var success = rq[ 0 ], + failure = rq[ 1 ]; + + // if we did not rate succesfully, abort + if ( msg.status !== 'ok' ) + { + failure( msg.msg ); + return; + } + + if ( !msg.data ) + { + failure( "Rater indicated success, but no data was returned" ); + return; + } + + // that's right; who da man (or wo-man)? + success( msg.data, ( msg.actions || [] ) ); +} + + +function purgeRateRequests( msg ) +{ + // this is never a good thing + for ( var rqid in _requests ) + { + // invoke failure function + _requests[ rqid ][ 1 ]( msg ); + } + + // clear out request references + _requests = {}; +} + + +function getsignum( sig ) +{ + return 128 + ( _signum[ sig ] || 0 ); +} + + +function createRqid( qid, indv ) +{ + return qid + '_' + + ( indv ? indv + '_' : '' ) + + ( new Date() ).getTime(); +} + diff --git a/src/server/rater/thread.js b/src/server/rater/thread.js new file mode 100644 index 0000000..b28255b --- /dev/null +++ b/src/server/rater/thread.js @@ -0,0 +1,225 @@ +/** + * Rating thread + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * This thread permits both asynchronous rating (in the case of JS raters) as + * well as hassle-free runtime reloading of raters withour risk of memory leaks + * that may be caused by deleting entries from node's module cache. + */ + +var map = {}, + fs = require( 'fs' ); + +var rater_path = process.env.RATER_PROGRAM_PATH; + + +function forEachProgram( c ) +{ + fs.readdirSync( rater_path ) + .forEach( function( filename ) + { + // ignore non-js files + if ( !( filename.match( /\.js$/ ) ) ) + { + return; + } + + c( filename, ( rater_path + '/' + filename ) ); + } ); +} + +// sent by parent to indicate that we should exit (we should never receive this +// signal from any sort of shell or anything) +try +{ + process.on( 'SIGHUP', function() + { + // we're good + process.exit( 0 ); + } ); +} +catch ( e ) +{ + sendLog( + "WARNING: OS does not support signal handlers; thread will always " + + "appear to exit in error if reload is requested" + ); +} + + +process.on( 'message', function( msg ) +{ + switch ( msg.cmd ) + { + case 'rate': + doRate( msg ); + break; + + default: + logError( "Unknown command from parent: " + msg.cmd ); + } +} ); + + +process.on( 'uncaughtException', function( e ) +{ + sendError( e ); +} ); + + +// load raters +forEachProgram( function( filename, path ) +{ + var rater = null; + + try + { + rater = require( path ); + sendLog( 'Loaded rater: ' + filename ); + } + catch ( e ) + { + sendError( e ); + } + + var name = filename.replace( /\.js$/, '' ); + map[ name ] = rater; +} ); + + +sendLog( 'Rating process ready (PID ' + process.pid + ').' ); + + +function doRate( data ) +{ + var quote = data.quote, + rqid = data.rqid, + rater = map[ data.supplier ]; + + if ( !rater ) + { + process.send( { + cmd: 'rate-reply', + rqid: rqid, + + status: 'error', + msg: "Unknown supplier: " + data.supplier, + } ); + + return; + } + + // TODO: temporary, until refactoring is complete + var imported_flag = false; + var rate_quote = { + getId: function() + { + return quote.id; + }, + + getAgentId: function() + { + return quote.agentId; + }, + + getAgentName: function() + { + return quote.agentName; + }, + + getCreditScoreRef: function() + { + return quote.creditScoreRef; + }, + + getBucket: function() + { + return { + getData: function() + { + return quote.data; + } + }; + }, + + setImported: function( val ) + { + val = ( val === undefined ) ? true : !!val; + imported_flag = val; + }, + }; + + var rate_session = { + agentId: function() + { + return data.agentId; + }, + + isInternal: function() + { + return data.internal; + }, + }; + + // perform rating + rater.rate( rate_quote, rate_session, data.indv, + function( rdata, actions ) + { + process.send( { + cmd: 'rate-reply', + rqid: rqid, + + status: 'ok', + data: rdata, + actions: actions, + + // not used by all raters + imported: imported_flag, + } ); + }, + + function( msg ) + { + process.send( { + cmd: 'rate-reply', + rqid: rqid, + + status: 'error', + msg: msg, + } ); + } + ); +} + + +function sendLog( msg ) +{ + process.send( { cmd: 'log', msg: msg } ); +} + + +function sendError( e ) +{ + process.send( { + cmd: 'error', + msg: e.message, + stack: e.stack + } ); +} + diff --git a/src/server/request/CapturedUserResponse.js b/src/server/request/CapturedUserResponse.js new file mode 100644 index 0000000..bc061e7 --- /dev/null +++ b/src/server/request/CapturedUserResponse.js @@ -0,0 +1,50 @@ +/** + * Invoke callback on response + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Trait = require( 'easejs' ).Trait, + UserResponse = require( './UserResponse' ); + +module.exports = Trait( 'CapturedUserResponse' ) + .extend( UserResponse, +{ + 'private _callback': null, + + + __mixin: function( callback ) + { + if ( typeof callback !== 'function' ) + { + throw TypeError( "Callback expected; given " + callback ); + } + + this._callback = callback; + }, + + + /** TODO: Make public once IProtUserResponse is removed **/ + 'virtual abstract override public endRequest': function( + code, error, data + ) + { + this._callback( code, error, data ); + } +} ); + diff --git a/src/server/request/JsonServerResponse.js b/src/server/request/JsonServerResponse.js new file mode 100644 index 0000000..b2ce2f8 --- /dev/null +++ b/src/server/request/JsonServerResponse.js @@ -0,0 +1,62 @@ +/** + * Contains program JsonServerResponse class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class; + + +exports.create = function () +{ + return new JsonServerResponse(); +}; + + +var JsonServerResponse = Class.extend( +{ + from: function( quote, content, actions ) + { + return JSON.stringify({ + quoteId: + quote.getId(), + content: + content, + hasError: + false, + actions: + actions, + }); + }, + + + error: function( message, actions, btn_caption ) + { + return JSON.stringify({ + hasError: + true, + content: + message, + actions: + actions, + btnCaption: + btn_caption, + }); + }, +} ); + diff --git a/src/server/request/SessionSpoofHttpClient.js b/src/server/request/SessionSpoofHttpClient.js new file mode 100644 index 0000000..f6609b6 --- /dev/null +++ b/src/server/request/SessionSpoofHttpClient.js @@ -0,0 +1,90 @@ +/** + * Spoof HTTP request metadata + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class; + + +/** + * Spoofs an HTTP request with a user session + * + * This attempts to make a request to an HTTP server using the user agent, + * IP address, and session cookie of the original user request. + */ +module.exports = Class( 'SessionSpoofHttpClient', +{ + /** + * HTTP object with #request + * @type {Object} + */ + 'private _http': null, + + /** + * HTTP server hostname + * @type {string} + */ + 'private _host': '', + + + /** + * Initialize spoof client + * + * HTTP must be an object with a `request' method. + * + * @param {Object} htto HTTP module with #request + * @param {string} hostname remote host name + */ + __construct: function( http, hostname ) + { + this._http = http; + this._host = ''+hostname; + }, + + + /** + * Initialized HTTP requested with spoofed session + * + * @param {UserRequest} user_request original user request + * @param {string} path requested path on server + */ + 'request': function( user_request, path ) + { + path = ''+path; + + var sessid = user_request.getSessionId(), + sessname = user_request.getSessionIdName(); + + if ( sessid === null ) + { + throw Error( "Session id unavailable" ); + } + + return this._http.request( { + hostname: this._host, + path: path, + headers: { + 'User-Agent': user_request.getUserAgent(), + 'X-Forwarded-For': user_request.getRemoteAddr(), + 'Cookie': sessname + '=' + sessid, + } + } ); + } +} ); + diff --git a/src/server/request/UserRequest.js b/src/server/request/UserRequest.js new file mode 100644 index 0000000..17fcdf2 --- /dev/null +++ b/src/server/request/UserRequest.js @@ -0,0 +1,571 @@ +/** + * UserRequest class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class; + + +/** + * Encapsulates user request and response data in an easy-to-use class + * + * This class doesn't really add any new functionality. It just makes working + * with requests and responses a bit easier. + */ +module.exports = Class.extend( require( 'events' ).EventEmitter, +{ + /** + * Request timeout in seconds + * + * In the future, we can make this configurable. Currently, it's + * unnecessary. Feel free to add such a feature if you have need for it. + * + * @type {number} + */ + 'const TIMEOUT': 120, + + /** + * Requested URI + * @var String + */ + uri: '', + + /** + * Query (GET vars) + * @var {Object} + */ + query: {}, + + /** + * Request object + * @var http.ServerRequest + */ + request: null, + + /** + * Response object + * @var http.ServerResponse + */ + response: null, + + /** + * Contains post data, when available + * @var undefined|Object + */ + postData: undefined, + + /** + * Functions to call when post data is available + * @var Array + */ + postDataCallbacks: [], + + /** + * HTTP status code to respond with (default 200 OK) + * @var Integer + */ + responseCode: 200, + + /** + * HTTP MIME Content type (text/html by default) + * @var String + */ + contentType: 'text/html; charset=utf-8', + + /** + * Headers to send + * @var {Object} + */ + headers: {}, + + /** + * Whether the headers have been sent + * @var Boolean + */ + headersSent: false, + + /** + * Length of response + * + * @var Integer + */ + responseLength: 0, + + /** + * User's session + * @var UserSession + */ + session: null, + + /** + * Timer that will cause a timeout after TIMEOUT seconds + * @type {!number} + */ + 'private _timeout': null, + + + /** + * String representation of object + * + * @return String + */ + toString: function() + { + return '[object UserRequest]'; + }, + + + /** + * Constructor + * + * @return undefined + */ + __construct: function( request, response, session_builder ) + { + var request_data = require( 'url' ).parse( request.url, true ); + + this.uri = request_data.pathname; + this.query = request_data.query || {}; + this.request = request; + this.response = response; + + this._initRequestPost(); + + // "session key" used internally for certain scripts + var skey = this.query.skey; + + // initialize session + var _self = this; + this.session = session_builder( this.getCookies().PHPSESSID ) + .on( 'ready', function( data ) + { + // if no data is available, then the session could not be + // initialized; abort the request :( (this should be ignored if + // a session key is provided, since in that case we do not care + // about a session) + if ( ( data === null ) && !skey ) + { + _self.setResponseCode( 500 ); + _self.end( 'Session initialization failure.' ); + + return; + } + + // now we're ready to roll + _self.emit( 'ready' ); + } ); + + // set timeout in the event we fail to respond due to some bug/uncaught + // exception/etc + this._timeout = setTimeout( + function() + { + _self.setResponseCode( 408 ); + _self.end( 'Request timed out.' ); + }, + ( this.__self.$( 'TIMEOUT' ) * 1000 ) + ); + }, + + + /** + * Watches for post data and returns the data to any waiting callbacks + * + * @return undefined + */ + _initRequestPost: function() + { + var querystring = require( 'querystring' ), + post_raw = '', + request = this; + + this.request + .addListener( 'data', function( data ) + { + post_raw += data; + }) + .addListener( 'end', function() + { + request.postData = querystring.parse( post_raw ); + + // call any callbacks that are waiting for the data + var func = null; + while ( func = request.postDataCallbacks.pop() ) + { + func.call( request, request.postData ); + } + }); + }, + + + /** + * Performs general initialization tasks (template method) + * + * @return undefined + */ + _init: function( session_builder ) + { + var request = this; + + this._initRequestPost(); + + }, + + + /** + * Returns the requested URI + * + * @return String requested URI + */ + getUri: function() + { + return this.uri; + }, + + + /** + * Returns query (GET) data + * + * @return Object GET data + */ + getGetData: function() + { + return this.query; + }, + + + /** + * Requests the post data + * + * This is asynchronous. If the data is already available, the callback will + * be called immediately. If the data is not yet available, it will be + * called as soon as it becomes available. + * + * @param Function( data ) callback function to call when data is available + * + * @return UserRequest self to allow for method chaining + */ + getPostData: function( callback ) + { + // if we already have the post data, give it to them immediately + if ( this.postData !== undefined ) + { + callback.call( this, this.postData ); + return this; + } + + // otherwise, we need to call the callback when the data is available + this.postDataCallbacks.push( callback ); + return this; + }, + + + /** + * Sets the HTTP status code to respond with + * + * @param Integer code HTTP status code + * + * @return UserRequest self + */ + setResponseCode: function( code ) + { + if ( this.headersSent === true ) + { + console.error( 'Headers already sent; response code not set' ); + return this; + } + + this.responseCode = +code; + return this; + }, + + + /** + * Returns the HTTP status code sent to the client + * + * @return Integer HTTP status code + */ + getResponseCode: function() + { + return this.responseCode; + }, + + + /** + * Sets the content type + * + * @param String type content type + * + * @return UserRequest self + */ + setContentType: function( type ) + { + this.contentType = ''+type; + return this; + }, + + + /** + * Sets HTTP headers to send to the client + * + * The headers provided will be merged with any existing headers. They will + * be overwritten if they have already been set. + * + * @param {Object} data headers to set (key-value) + * + * @return {UserRequest} self + */ + setHeaders: function( data ) + { + for ( header in data ) + { + this.headers[ header ] = data[ header ]; + } + + return this; + }, + + + /** + * Tells the client not to cache the response + * + * @return {UserRequest} self + */ + noCache: function() + { + // the first two are for IE6, the others are HTTP/1.0 + this.headers[ 'Cache-Control' ] = + 'private, max-age=0, no-store, no-cache, must-revalidate, ' + + 'post-check=0, pre-check=0'; + + return this; + }, + + + /** + * Send headers to the client + * + * @return undefined + */ + _sendHeaders: function() + { + this.headers[ 'Content-Type' ] = this.contentType; + + this.response.writeHead( this.responseCode, this.headers ); + this.headersSent = true; + + // we don't need this function anymore + this._sendHeaders = function() {} + }, + + + /** + * Write data to the client + * + * @param String chunk data to write + * @param String encoding + * + * @return UserRequest self + */ + write: function( chunk, encoding ) + { + encoding = encoding || 'utf8'; + + this.responseLength += chunk.length; + + this._sendHeaders(); + this.response.write( chunk, encoding ); + + return this; + }, + + + error: function( error, data, encoding ) + { + this.setResponseCode( 503 ); + this.end( + JSON.stringify( { + error: error, + data: data + } ), + encoding + ); + }, + + + tryAgain: function( data, encoding ) + { + this.setResponseCode( 503 ); + this.end( + JSON.stringify( { + error: 'EAGAIN', + data: data + } ), + encoding + ); + }, + + + ok: function( data, encoding ) + { + this.setResponseCode( 200 ); + this.end( + JSON.stringify( { + error: null, + data: data, + } ) + ); + }, + + + accepted: function( data, encoding ) + { + this.setResponseCode( 202 ); + this.end( + JSON.stringify( { + error: null, + data: data, + } ) + ); + }, + + + /** + * End client response + * + * @param String chunk data to write + * @param String encoding + * + * @return UserRequest self + */ + end: function( data, encoding ) + { + data = data || ''; + + clearTimeout( this._timeout ); + this._timeout = null; + + this.responseLength += data.length; + + this._sendHeaders(); + this.response.end( data, encoding ); + + this.emit( 'end' ); + + return this; + }, + + + /** + * Returns the length of the response + * + * Note that this is not very accurate, as this doesn't take into account + * multi-byte characters. + * + * @return Integer response length + * + * @todo multibyte + */ + getResponseLength: function() + { + return this.responseLength; + }, + + + /** + * Returns request cookies as an object + * + * @return Object request cookies + */ + getCookies: function() + { + var cookies = {}, + cookie_data = this.request.headers.cookie; + + if ( cookie_data === undefined ) + { + return {}; + } + + // parse the cookies into an easily accessible object + cookie_data.split( ';' ).forEach( function( val ) + { + var data = val.split( '=', 2 ); + cookies[ data[0].trim() ] = data[1]; + }); + + // any future calls to this function will simply return the already + // generated cookies array to save us some time + this.getCookies = function() + { + return cookies; + } + + return cookies; + }, + + + /** + * Returns the current session + * + * @return UserSession + */ + getSession: function() + { + return this.session; + }, + + + /** + * Returns the request object + * + * @return http.ServerRequest + */ + getRequest: function() + { + return this.request; + }, + + + 'public getRemoteAddr': function() + { + // since we may be proxied, let the proxy forward header take precidence + return this.request.headers['x-forwarded-for'] + || this.request.connection.remoteAddress; + }, + + + 'public getUserAgent': function() + { + return this.request.headers['user-agent']; + }, + + + 'public getSessionId': function() + { + return this.getCookies().PHPSESSID || null; + }, + + + 'public getSessionIdName': function() + { + return 'PHPSESSID'; + } +} ); + diff --git a/src/server/request/UserResponse.js b/src/server/request/UserResponse.js new file mode 100644 index 0000000..30aac8c --- /dev/null +++ b/src/server/request/UserResponse.js @@ -0,0 +1,275 @@ +/** + * Response to user request + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class, + UserRequest = require( './UserRequest' ); + + +/** + * Abstraction for standard user request responses + * + * This provides consistency throughout the system and permits hiding the + * actual user request. This may be extended by subtypes to alter its + * behavior in a well-defined manner. + */ +module.exports = Class( 'UserResponse' ) + .extend( +{ + /** + * User request to accept a response + * @type {UserRequest} + */ + 'private _request': null, + + + __construct: function( request ) + { + if ( !Class.isA( UserRequest, request ) ) + { + throw TypeError( + "UserRequest expected; given " + request + ); + } + + this._request = request; + }, + + + /** + * Satisfy a request successfully + * + * This is an HTTP/1.1 200 OK response. + * + * @param {*} data return data + * + * @return {undefined} only one response should be made + */ + 'public ok': function( data ) + { + this._reply( 200, null, data ); + }, + + + /** + * Request was accepted for processing and may or may not eventually + * complete + * + * This is an HTTP/1.1 202 Accepted response. + * + * @param {*} data return data + * + * @return {undefined} only one response should be made + */ + 'public accepted': function( data ) + { + this._reply( 202, null, data ); + }, + + + /** + * Client request was unknown or invalid + * + * This is an HTTP/1.1 400 Bad Request response. + * + * @param {*} data return data + * @param {?string} error error code or description + * + * @return {undefined} only one response should be made + */ + 'public requestError': function( data, error ) + { + this._reply( 400, error, data ); + }, + + + /** + * Request cannot be completed due to a conflict with the current state + * of the resource + * + * This is an HTTP/1.1 409 Conflict response. + * + * @param {*} data return data + * @param {?string} error error code or description + * + * @return {undefined} only one response should be made + */ + 'public stateError': function( data, error ) + { + this._reply( 409, error, data ); + }, + + + /** + * User is not authorized to perform the action + * + * This is an HTTP/1.1 403 Forbidden response. + * + * @param {*} data return data + * @param {?string} error error code or description + * + * @return {undefined} only one response should be made + */ + 'public forbidden': function( data, error ) + { + this._reply( 403, error, data ); + }, + + + /** + * Requested resource was not found + * + * This is an HTTP/1.1 404 Not Found response. + * + * @param {*} data return data + * @param {?string} error error code or description + * + * @return {undefined} only one response should be made + */ + 'public notFound': function( data, error ) + { + this._reply( 404, error, data ); + }, + + + /** + * An internal error occurred while processing an otherwise valid + * request + * + * This is an HTTP/1.1 500 Internal Server Error response. + * + * @param {*} data return data + * @param {?string} error error code or description + * + * @return {undefined} only one response should be made + */ + 'public internalError': function( data, error ) + { + this._reply( 500, error, data ); + }, + + + /** + * Requested service is unavailable and the request could not be + * fulfilled + * + * This is an HTTP/1.1 503 Service Unavailable response. + * + * @param {*} data return data + * @param {?string} error error code or description + * + * @return {undefined} only one response should be made + */ + 'public unavailable': function( data, error ) + { + this._reply( 503, error, data ); + }, + + + /** + * Requested service is temporarily unavailable and the request should + * be retried + * + * This invokes `#unavailable` with a static error code of `EAGAIN` + * (motivated by the standard Unix error constant name). + * + * @param {*} data return data + * + * @return {undefined} only one response should be made + */ + 'public tryAgain': function( data ) + { + this.unavailable( data, 'EAGAIN' ); + }, + + + + /** + * Prepare to complete a request with the given code and data + * + * The majority of this logic is delegated to `#endRequest`, allowing + * subtypes to hook or override the response logic. + * + * @param {number} code status code + * @param {?string} error error string, if applicable + * @param {*} data generic return data + */ + 'protected _reply': function( code, error, data ) + { + code = +code; + error = ''+( error || '' ) || null; + + // this may be overridden by subtypes + this.endRequest( code, error, data ); + }, + + + /** + * Complete a request with the given code and data + * + * The returned are serialized, so keep in mind that certain data + * may be lost (for example, the distinction between `null` and + * `undefined`). + * + * A subtype may override this method to hook or fundamentally alter the + * behavior of a response. If a subtype only wishes to modify or + * augment the response, it should instead override `#createRreply`. + * + * @param {number} code status code + * @param {?string} error error string, if applicable + * @param {*} data generic return data + */ + 'virtual protected endRequest': function( code, error, data ) + { + this._request + .setResponseCode( code ) + .end( + JSON.stringify( + this.createReply( code, error, data ) + ) + ); + }, + + + /** + * Produce a response to be returned to the client + * + * The standard reply consists of an error string (or null) and generic + * response data, output as a JSON object literal. + * + * If a subtype wishes to augment the response in any way, this is the + * place to do it. + * + * The return value will be serialized. + * + * @param {?string} error error string, if applicable + * @param {*} data generic return data + * + * @return {*} response value + */ + 'virtual protected createReply': function( code, error, data ) + { + return { + error: error, + data: data, + }; + }, +} ); + diff --git a/src/server/request/UserSession.js b/src/server/request/UserSession.js new file mode 100644 index 0000000..1497355 --- /dev/null +++ b/src/server/request/UserSession.js @@ -0,0 +1,296 @@ +/** + * UserSession class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * @todo this is very tightly coupled with LoVullo's system + */ + +// php compatibility +var php = require( 'php' ), + Class = require( 'easejs' ).Class; + + +/** + * Stores/retrieves user PHP session data from memcached + */ +module.exports = Class.extend( require( 'events' ).EventEmitter, +{ + /** + * Session id + * @type {string} + */ + 'private _id': '', + + /** + * Memcache client + * @tyep {memcache.Client} + */ + 'private _memcache': null, + + /** + * Session data + * @type {object} + */ + 'private _data': {}, + + + /** + * Initializes a session from an existing PHP session + * + * @param String id session id + * @param memcache.Client memcache memcache client used to access session + * + * @return undefined + */ + __construct: function( id, memcache ) + { + this._id = id || ''; + this._memcache = memcache; + this._data = {}; + + // parse the session data + var _self = this; + this._getSessionData( function( data ) + { + _self._data = ( data === null ) ? {} : data; + + // session data is now available + _self.emit( 'ready', data ); + } ); + }, + + + /** + * Returns the session data + * + * @return Object session data + */ + getData: function() + { + return this._data; + }, + + + /** + * Returns whether the user is currently logged in + * + * This is determined simply by whether the agent id is available. + * + * @return Boolean + */ + isLoggedIn: function() + { + return ( this._data.agentID !== undefined ) + ? true + : false; + }, + + + /** + * Gets the agent id, if available + * + * @return Integer|undefined agent id or undefined if unavailable + */ + agentId: function() + { + return this._data.agentID || undefined; + }, + + + /** + * Gets the agent name, if available + * + * @return String|undefined agent name or undefined if unavailable + */ + agentName: function() + { + return this._data.agentNAME || undefined; + }, + + + /** + * Whether the user is logged in as an internal user rather than a broker + * + * @return {boolean} true if internal user, otherwise false + */ + isInternal: function() + { + return ( this.agentId() === '900000' ) + ? true + : false; + }, + + + 'public setAgentId': function( id ) + { + this._data.agentID = id; + return this; + }, + + + /** + * Gets the broker entity id, if available + * + * @return Integer|undefined agent entity id or undefined if unavailable + */ + agentEntityId: function() + { + return this._data.broker_entity_id || undefined; + }, + + + /** + * Set session redirect for Symfony + * + * This will perform a redirect after login. + * + * @param {string} url url to redirect to + * @param {function()=} callback optional continuation + * + * @return {UserSession} self + */ + 'public setRedirect': function( url, callback ) + { + this._appendSessionData( + { s2_legacy_redirect: url }, + callback + ); + + return this; + }, + + + 'public setReturnQuoteNumber': function( number, callback ) + { + this._appendSessionData( + { lvqs_return_qn: +number }, + callback + ); + + return this; + }, + + + 'public getReturnQuoteNumber': function() + { + return this._data.lvqs_return_qn || 0; + }, + + + 'public clearReturnQuoteNumber': function( callback ) + { + this._data.lvqs_return_qn = 0; + this.setReturnQuoteNumber( 0, callback ); + + return this; + }, + + + /** + * Appends data to a session + * + * XXX: Note that this is not a reliable means of modifying session + * data---it servese its purpose for appending string data to a session, but + * should not be used for anything else (such as modifying existing session + * values). + * + * @param {Object} data key-value string data + * @param {function()=} callback optional continuation + * + * @return {undefined} + */ + 'private _appendSessionData': function( data, callback ) + { + var _self = this; + + this._memcache.get( this._id, function( orig ) + { + if ( orig === null ) + { + // well we gave it a good shot! (right now we don't indicate + // error, because there's not much we can do about that...maybe + // in the future we'll want to be notified of errors)...we just + // don't want to potentially clear out all the session data + callback && callback(); + return; + } + + var newdata = orig; + + for ( var key in data ) + { + newdata += key + '|' + php.serialize( data[ key ] ); + } + + _self._memcache.set( _self._id, newdata, function() + { + callback && callback(); + } ); + } ); + }, + + + /** + * Parses PHP session data from memcache and returns an object with the data + * + * @param {function(data)} callback function to call with parsed data + * + * @return Object parsed session data + */ + 'private _getSessionData': function( callback ) + { + this._memcache.get( this._id, function( data ) + { + if ( data === null ) + { + // failure + callback( null ); + return; + } + + var session_data = {}; + + if ( data ) + { + // Due to limitations of Javascript's regex engine, we need to do + // this in a series of steps. First, split the string to find the + // keys and their serialized values. + var splits = data.split( /(\w+?)\|/ ), + len = splits.length; + + // associate the keys with their values + for ( var i = 1; i < len; i++ ) + { + var key = splits[ i ], + val = splits[ ++i ]; + + // the values are serialized PHP data; unserialize them + val = php.unserialize( val ); + + // add to the session data + session_data[ key ] = val; + } + } + + // return the parsed session data + callback( session_data ); + } ); + }, +} ); + diff --git a/src/server/service/RatingService.js b/src/server/service/RatingService.js new file mode 100644 index 0000000..107755c --- /dev/null +++ b/src/server/service/RatingService.js @@ -0,0 +1,416 @@ +/** + * Rating service + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class; + + +/** + * XXX: Half-assed, quick refactoring to extract from Server class; this is not + * yet complete! + * + * TODO: Logging should be implemented by observers + */ +module.exports = Class( 'RatingService', +{ + _logger: null, + + _dao: null, + + _server: null, + + _raters: null, + + + __construct: function( logger, dao, server, raters ) + { + this._logger = logger; + this._dao = dao; + this._server = server; + this._raters = raters; + }, + + + /** + * Sends rates to the client + * + * Note that the continuation will be called after all data saving is + * complete; the request will be sent back to the client before then. + * + * @param {UserRequest} request user request to satisfy + * @param {UserResponse} response pending response + * @param {Quote} quote quote to export + * @param {string} cmd applicable of command request + * @param {Function} callback continuation after saving is complete + * + * @return Server self to allow for method chaining + */ + 'public request': function( request, response, quote, cmd, callback ) + { + // cmd represents a request for a single rater + if ( !cmd && this._isQuoteValid( quote ) ) + { + // send an empty reply (keeps what is currently in the bucket) + this._server.sendResponse( request, quote, { + data: {}, + }, [] ); + + callback(); + return this; + } + + var program = quote.getProgram(); + + try + { + this._performRating( request, program, quote, cmd, callback ); + } + catch ( err ) + { + this._sendRatingError( request, quote, program, err ); + callback(); + } + + return this; + }, + + + _getProgramRater: function( program, quote ) + { + var rater = this._raters.byId( program.getId() ); + + // if a rater could not be found, we can't do any rating + if ( rater === null ) + { + this._logger.log( this._logger.PRIORITY_ERROR, + "Rating for quote %d (program %s) failed; missing module", + quote.getId(), + program.getId() + ); + } + + return rater; + }, + + + _isQuoteValid: function( quote ) + { + // quotes are valid for 30 days + var re_date = Math.round( ( ( new Date() ).getTime() / 1000 ) - + ( 60 * 60 * 24 * 30 ) + ); + + if ( quote.getLastPremiumDate() > re_date ) + { + this._logger.log( this._logger.PRIORITY_INFO, + "Skipping '%s' rating for quote #%s; quote is still valid", + quote.getProgramId(), + quote.getId() + ); + + return true; + } + + return false; + }, + + + _performRating: function( request, program, quote, indv, c ) + { + var _self = this; + + var rater = this._getProgramRater( program ); + if ( !rater ) + { + this._server.sendError( request, 'Unable to perform rating.' ); + c(); + } + + this._logger.log( this._logger.PRIORITY_INFO, + "Performing '%s' rating for quote #%s", + quote.getProgramId(), + quote.getId() + ); + + rater.rate( quote, request.getSession(), indv, + function( rate_data, actions ) + { + actions = actions || []; + + _self._postProcessRaterData( + rate_data, actions, program, quote + ); + + var class_dest = {}; + + rate_data = _self._cleanRateData( + rate_data, + class_dest + ); + + // TODO: move me during refactoring + _self._dao.saveQuoteClasses( quote, class_dest ); + + + // save all data server-side (important: do after + // post-processing); async + _self._saveRatingData( quote, rate_data, indv, function() + { + // we're done + c(); + } ); + + // no need to wait for the save; send the response + _self._server.sendResponse( request, quote, { + data: rate_data + }, actions ); + }, + function( message ) + { + _self._sendRatingError( request, quote, program, + Error( message ) + ); + + c(); + } + ); + }, + + + /** + * Saves rating data + * + * Data will be merged with existing bucket data and saved. The idea behind + * this is to allow us to reference the data (e.g. for reporting) even if + * the client does not save it. + * + * @param {Quote} quote quote to save data to + * @param {Object} data rating data + * + * @return {undefined} + */ + _saveRatingData: function( quote, data, indv, c ) + { + // only update the last premium calc date on the initial request + if ( !indv ) + { + var cur_date = Math.round( + ( new Date() ).getTime() / 1000 + ); + + quote.setLastPremiumDate( cur_date ); + quote.setRatedDate( cur_date ); + + function done() + { + c(); + } + + // save the last prem status (we pass an empty object as the save + // data argument to ensure that we do not save the actual bucket + // data, which may cause a race condition with the below merge call) + this._dao.saveQuote( quote, done, done, {} ); + } + else + { + c(); + } + + // we're not going to worry about whether or not this fails; if it does, + // an error will be automatically logged, but we still want to give the + // user a rate (if this save fails, it's likely we have bigger problems + // anyway); this can also be done concurrently with the above request + // since it only modifies a portion of the bucket + this._dao.mergeBucket( quote, data ); + }, + + + /** + * Process rater data returned from a rater + * + * @param {Object} data rating data returned + * @param {Array} actions actions to send to client + * @param {Program} program program used to perform rating + * @param {Quote} quote quote used for rating + * + * @return {undefined} + */ + _postProcessRaterData: function( data, actions, program, quote ) + { + var meta = data._cmpdata || {}; + + // the metadata will not be provided to the client + delete data._cmpdata; + + // rating worksheets are returned as metadata + this._processWorksheetData( quote.getId(), data ); + + if ( ( program.ineligibleLockCount > 0 ) + && ( +meta.count_ineligible >= program.ineligibleLockCount ) + ) + { + // lock the quote client-side (we don't send them the reason; they + // don't need it) to the current step + actions.push( { action: 'lock' } ); + + var lock_reason = 'Supplier ineligibility restriction'; + var lock_step = quote.getCurrentStepId(); + + // the next step is the step that displays the rating results + quote.setExplicitLock( lock_reason, ( lock_step + 1 ) ); + + // important: only save the lock state, not the step states, as we + // have a race condition with async. rating (the /visit request may + // be made while we're rating, and when we come back we would then + // update the step id with a prior, incorrect step) + this._dao.saveQuoteLockState( quote ); + } + + // if any have been deferred, instruct the client to request them + // individually + if ( Array.isArray( meta.deferred ) && ( meta.deferred.length > 0 ) ) + { + var torate = []; + + meta.deferred.forEach( function( alias ) + { + actions.push( { action: 'indvRate', id: alias } ); + torate.push( alias ); + } ); + + // we log that we're performing rating, so we should also log when + // it is deferred (otherwise the logs will be rather confusing) + this._logger.log( this._logger.PRIORITY_INFO, + "'%s' rating deferred for quote #%s; will rate: %s", + quote.getProgramId(), + quote.getId(), + torate.join( ',' ) + ); + } + }, + + + _sendRatingError: function( request, quote, program, err ) + { + // well that's no good + this._logger.log( this._logger.PRIORITY_ERROR, + "Rating for quote %d (program %s) failed: %s", + quote.getId(), + program.getId(), + err.message + '\n-!' + err.stack.replace( /\n/g, '\n-!' ) + ); + + this._server.sendError( request, + 'There was a problem during the rating process. Unable to ' + + 'continue. Please contact LoVullo Associates for assistance.' + + + // show details for internal users + ( ( request.getSession().isInternal() ) + ? '

[Internal] ' + err.message + '

' + + '
' + err.stack.replace( /\n/g, '
' ) + : '' + ) + ); + }, + + + _processWorksheetData: function( qid, data ) + { + // TODO: this should be done earlier on, so that this is not necessary + var wre = /^(.+)___worksheet$/, + worksheets = {}; + + // extract worksheets for each supplier + for ( var field in data ) + { + var match; + if ( match = field.match( wre ) ) + { + var name = match[ 1 ]; + + worksheets[ name ] = data[ field ]; + delete data[ field ]; + } + } + + var _self = this; + this._dao.setWorksheets( qid, worksheets, function( err ) + { + if ( err ) + { + _self._logger.log( this._logger.PRIORITY_ERROR, + "Failed to save rating worksheets for quote %d", + quote.getId(), + err.message + '\n-!' + err.stack.replace( /\n/g, '\n-!' ) + ); + } + } ); + }, + + + serveWorksheet: function( request, quote, supplier, index ) + { + var qid = quote.getId(), + _self = this; + + this._dao.getWorksheet( qid, supplier, index, function( data ) + { + _self._server.sendResponse( request, quote, { + data: data + } ); + } ); + }, + + + /** + * Prepares rate data to be sent back to the client + * + * There are certain data saved server-side that there is no use serving to + * the client. + * + * @param {Object} data rate data + * + * @return {Object} modified rate data + */ + 'private _cleanRateData': function( data, classes ) + { + classes = classes || {}; + + var result = {}; + + // clear class data + for ( var key in data ) + { + var mdata; + + // supplier___classes + if ( mdata = key.match( /^(.*)___classes$/ ) ) + { + classes[ mdata[ 1 ] ] = data[ key ]; + continue; + } + + result[ key ] = data[ key ]; + } + + return result; + }, +} ); + diff --git a/src/server/service/Service.js b/src/server/service/Service.js new file mode 100644 index 0000000..7830006 --- /dev/null +++ b/src/server/service/Service.js @@ -0,0 +1,33 @@ +/** + * Generic service interface + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Interface = require( 'easejs' ).Interface; + + +/** + * Any service responsible for fulfilling a user request for a given + * document (quote) + */ +module.exports = Interface( 'Service', +{ + 'public request': [ 'request', 'response', 'quote', 'cmd', 'callback' ], +} ); + diff --git a/src/server/service/TokenDao.js b/src/server/service/TokenDao.js new file mode 100644 index 0000000..26f9fb0 --- /dev/null +++ b/src/server/service/TokenDao.js @@ -0,0 +1,233 @@ +/** + * Token state management + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Class = require( 'easejs' ).Class; + + +/** + * Manages token updates + * + * Note that this is tightly coupled with MongoDB. + */ +module.exports = Class( 'TokenDao', +{ + /** + * @type {MongoCollection} mongo database collection + */ + 'private _collection': null, + + + /** + * Initialize connection + * + * @param {MongoCollection} collection token Mongo collection + */ + 'public __construct': function( collection ) + { + this._collection = collection; + }, + + + /** + * Create or update a token record + * + * The token entry is entered in the token log, and then the current + * entry is updated to reflect the changes. The operation is atomic. + * + * @param {number} quote_id unique quote identifier + * @param {string} ns token namespace + * @param {string} token token value + * @param {string} data token data, if any + * @param {string} status arbitrary token type + * + * @param {function(*)} callback with error or null (success) + * + * @return {TokenDao} self + */ + 'public updateToken': function( quote_id, ns, token, type, data, callback ) + { + var token_data = {}, + token_log = {}, + root = this._genRoot( ns ) + '.', + current_ts = Math.floor( ( new Date() ).getTime() / 1000 ); + + var token_entry = { + type: type, + timestamp: current_ts, + }; + + if ( data ) + { + token_entry.data = data; + } + + token_data[ root + 'last' ] = token; + token_data[ root + 'lastStatus' ] = token_entry; + token_data[ root + token + '.status' ] = token_entry; + + token_log[ root + token + '.statusLog' ] = token_entry; + + this._collection.update( + { id: +quote_id }, + { + $set: token_data, + $push: token_log + }, + { upsert: true }, + + function ( err, docs ) + { + callback( err || null ); + } + ); + + return this; + }, + + + /** + * Retrieve existing token under the namespace NS, if any, for the quote + * identified by QUOTE_ID + * + * If a TOKEN_ID is provided, only that token will be queried; otherwise, + * the most recently created token will be the subject of the query. + * + * @param {number} quote_id quote identifier + * @param {string} ns token namespace + * @param {string} token_id token identifier (unique to NS) + * + * @param {function(?Error,{{id: string, status: string}})} callback + * + * @return {TokenDao} self + */ + 'public getToken': function( quote_id, ns, token_id, callback ) + { + var _self = this; + + var root = this._genRoot( ns ) + '.', + fields = {}; + + fields[ root + 'last' ] = 1; + fields[ root + 'lastStatus' ] = 1; + + if ( token_id ) + { + // XXX: injectable + fields[ root + token_id ] = 1; + } + + this._collection.findOne( + { id: +quote_id }, + { fields: fields }, + function( err, data ) + { + if ( err ) + { + callback( err, null ); + return; + } + + if ( !data || ( data.length === 0 ) ) + { + callback( null, null ); + return; + } + + var exports = data.exports || {}, + ns_data = exports[ ns ] || {}; + + callback( + null, + ( token_id ) + ? _self._getRequestedToken( token_id, ns_data ) + : _self._getLatestToken( ns_data ) + ); + } + ); + + return this; + }, + + + /** + * Retrieve latest token data, or `null` if none + * + * @param {{last: string, lastStatus: string}} ns_data namespace data + * + * @return {?{{id: string, status: string}}} data of latest token in + * namespace + */ + 'private _getLatestToken': function( ns_data ) + { + var last = ns_data.last; + + if ( !last ) + { + return null; + } + + return { + id: last, + status: ns_data.lastStatus, + }; + }, + + + /** + * Retrieve latest token data, or `null` if none + * + * @param {string} token_id token identifier for namespace associated + * with NS_DATA + * + * @param {{last: string, lastStatus: string}} ns_data namespace data + * + * @return {?{{id: string, status: string}}} data of requested token + */ + 'private _getRequestedToken': function( token_id, ns_data ) + { + var reqtok = ns_data[ token_id ]; + + if ( !reqtok ) + { + return null; + } + + return { + id: token_id, + status: reqtok.status, + }; + }, + + + /** + * Determine token root for the given namespace + * + * @param {string} ns token namespace + * + * @return {string} token root for namespace NS + */ + 'private _genRoot': function( ns ) + { + // XXX: injectable + return 'exports.' + ns; + }, +} ); + diff --git a/src/server/service/TokenedService.js b/src/server/service/TokenedService.js new file mode 100644 index 0000000..bbb6fcf --- /dev/null +++ b/src/server/service/TokenedService.js @@ -0,0 +1,755 @@ +/** + * Tokenized service + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + */ + +var Trait = require( 'easejs' ).Trait, + Class = require( 'easejs' ).Class, + Service = require( './Service' ), + TokenDao = require( './TokenDao' ); + + +/** + * Wrap service with token system + * + * Tokened services provide a queue-like processing system whereby service + * requests are assigned a processing token and continue in the + * background. The service may then be queried for the status of the token + * and to accept the response data once the request has completed. + * + * This implementation _does not_ provide fault tolerance: if the + * tokened service is restarted and therefore unable to receive the response + * from the underlying (wrapped) service, for example, then the token will + * never be marked as "done". Such considerations should be handled by a + * more robust system. Callers still have the ability to observe token + * status timestamps and issue a request to kill a token as a last resort. + * + * TODO: The term "generator" should probably be avoided and replaced with + * another term to denote instantiation, since generators (in the coroutine + * sense) have been implemented in ES6. + */ +module.exports = Trait( 'TokenedService' ) + .implement( Service ) + .extend( +{ + /** + * Token namespace + * @type {string} + */ + 'private _ns': '', + + /** + * DAO for handling token persistence + * @type {TokenDao} + */ + 'private _dao': null, + + /** + * Token generator function + * @type {function(): string} + */ + 'private _tokgen': null, + + /** + * Response capture constructor + * @type {function(UserRequest,function(number,?string,*)): UserResponse} + */ + 'private _captureGen': null, + + + /** + * Initialize tokened service + * + * Each service will ideally have its own unique NAMESPACE to holds its + * tokens to both avoid conflicts and to recognize active tokens. DAO + * is used for persistence (e.g. saving to a database). TOKGEN should + * be a function that returns a new unique identifier for the token. + * + * The CAPTURE_GEN is intended to return a special `UserResponse` object + * that is able to be passed to the underlying (wrapped) service to + * complete its processing, while allowing the token system to continue + * its response processing asynchronously, outside the scope of the + * original user request. + * + * @param {string} namespace token namespace + * @param {TokenDao} dao token persistence DAO + * @param {function(): string} tokgen token generator + * + * @param {function(UserRequest,function(number,?string,*)): * UserResponse} + * capture_gen user response capture constructor + */ + __mixin: function( namespace, dao, tokgen, capture_gen ) + { + if ( !Class.isA( TokenDao, dao ) ) + { + throw TypeError( 'Instance of TokenDao expected' ); + } + + if ( typeof tokgen !== 'function' ) + { + throw TypeError( 'Token generator must be a function' ); + } + + if ( typeof capture_gen !== 'function' ) + { + throw TypeError( + 'Request capture generator must be a function' + ); + } + + this._ns = ''+namespace; + this._dao = dao; + this._tokgen = tokgen; + this._captureGen = capture_gen; + }, + + + /** + * Intercept request to underlying service, assign a token, and continue + * processing outside the scope of the original request + * + * The tokened service introduces additional commands for querying and + * token management: `status`, `accept, and `kill`; all other commands + * are proxied to the underlying service. + * + * TODO: This should be virtual once the ease.js arbitrary super method + * invocation bug on stacked traits is corrected in v0.2.6. + * + * @param {UserRequest} request service request + * @param {UserResponse} response pending response to request + * @param {Quote} quote quote associated with request + * @param {string} cmdstr service command string + * @param {Function} callback continuation after saving is complete + * + * @return {Service} self + */ + 'abstract override public request': function( + request, response, quote, cmdstr, callback + ) + { + cmdstr = ''+( cmdstr || '' ); + + var _self = this; + + var cmd_parts = cmdstr.split( '/' ); + + if ( cmd_parts.length > 2 ) + { + throw Error( "Invalid number of command arguments" ); + } + + var action = cmd_parts[ 0 ] || '', + tokid = cmd_parts[ 1 ] || null; + + this._getQuoteToken( quote, tokid, function( err, token ) + { + if ( tokid ) + { + if ( token === null ) + { + _self.respTokenError( response, tokid ); + } + + switch( action ) + { + case 'status': + _self.respStatus( response, token ); + return; + + case 'accept': + _self._tryAccept( response, quote, token ); + return; + + case 'kill': + _self._tryKillToken( response, quote, token ); + return; + } + } + + _self._handleDefaultRequest( + cmdstr, token, request, response, quote, callback + ); + } ); + + return this; + }, + + + /** + * Handle request before passing to underlying service + * + * If an active token TOKEN already exists, then the request will be + * aborted and the user will be notified to try again; the underlying + * service will not observe the request. + * + * XXX: Too many parameters. + * + * @param {string} cmd service command string + * @param {?Object} token existing active token, if any + * @param {UserRequest} request service request + * @param {UserResponse} response pending response to request + * @param {Quote} quote request quote + * @param {Function} callback continuation after saving is complete + + * + * @return {undefined} + */ + 'private _handleDefaultRequest': function( + cmd, token, request, response, quote, callback + ) + { + if ( this.isActive( token ) ) + { + this.respTryAgain( request, token ); + return; + } + + // at this point, we have no active token; we can process the + // request as desired + this.serveWithNewToken( + cmd, request, response, quote, callback + ); + }, + + + /** + * Retrieve token identified by TOKID for QUOTE + * + * The token will be looked up in the service's namespace. + * + * @param {Quote} quote request quote + * @param {string} tokid token identifier for quote + * + * @param {function(?Error,Object}} callback response continuation + * + * @return {undefined} + */ + 'private _getQuoteToken': function( quote, tokid, callback ) + { + this._dao.getToken( + quote.getId(), + this._ns, + tokid, + function( err, token ) + { + if ( err ) + { + callback( err, null ); + return; + } + + if ( tokid && !token ) + { + callback( + Error( "Token not found: " + tokid ), + null + ); + + return; + } + + callback( null, token ); + } + ); + }, + + + /** + * Predicate determining whether token is being actively processed + * + * XXX: this logic needs to be elsewhere; these are hard-coded! + * + * @param {Object} token token to observe + * + * @return {boolean} whether token is active + */ + 'virtual protected isActive': function( token ) + { + return ( + token + && token.status + && token.status.type !== 'DONE' + && token.status.type !== 'ACCEPTED' + && token.status.type !== 'DEAD' + ); + }, + + + /** + * Process request to kill TOKEN + * + * Only active tokens may be killed. + * + * @param {UserRequest} request service request + * @param {Quote} quote request quote + * @param {Object} token token to kill + * + * @return {undefined} + */ + 'private _tryKillToken': function( request, quote, token ) + { + if ( !this.isActive( token ) ) + { + this.respCannotKill( request, token ); + return; + } + + // this is async + this.killToken( quote, token ); + + request.accepted( { + message: "Token will be killed", + token: token.id, + prevStatus: token.status.type, + prevTimestamp: token.status.timestamp, + } ); + }, + + + /** + * Process request to accept TOKEN + * + * Active and dead tokens have no data available and can therefore not + * be accepted. Tokens that have already been accepted cannot be + * re-accepted. + * + * @param {UserRequest} request service request + * @param {Quote} quote request quote + * @param {Object} token token to accept + * + * @return {undefined} + */ + 'private _tryAccept': function( request, quote, token ) + { + var _self = this; + + if ( this.isActive( token ) ) + { + this.respTryAgain( request, token ); + return; + } + + if ( token.status.type === 'DEAD' ) + { + this.respAcceptDead( request, token ); + return; + } + + if ( token.status.type === 'ACCEPTED' ) + { + this.respAcceptAccepted( request, token ); + return; + } + + // accept the token before replying to ensure that we are the only + // one that will return the data (XXX: this is not atomic) + this.acceptToken( quote, token, function( err ) + { + if ( err ) + { + _self.respTryAgain( request, token ); + return; + } + + _self.respAccept( request, token ); + } ); + }, + + + /** + * Respond with an error stating that the request may be re-attempted at + * a later time + * + * The response will include the status of TOKEN. + * + * @param {UserResponse} response pending service response + * @param {Object} token applicable token + * + * @return {undefined} + */ + 'virtual protected respTryAgain': function( response, token ) + { + response.tryAgain( this.getStatus( token ) ); + }, + + + /** + * Respond with the status of TOKEN + * + * @param {UserResponse} response pending service response + * @param {Object} token applicable token + * + * @return {undefined} + */ + 'virtual protected respStatus': function( response, token ) + { + response.ok( this.getStatus( token ) ); + }, + + + /** + * Respond with an indication of a successful acceptance of TOKEN, along + * with its data + * + * @param {UserResponse} response pending service response + * @param {Object} token applicable token + * + * @return {undefined} + */ + 'virtual protected respAccept': function( response, token ) + { + var token_data = this.getStatus( token ); + token_data.tokenData = token.status.data; + + response.ok( token_data ); + }, + + + /** + * Respond indicating that TOKEN is dead and cannot be accepted + * + * @param {UserResponse} response pending service response + * @param {Object} token applicable token + * + * @return {undefined} + */ + 'virtual protected respAcceptDead': function( response, token ) + { + var token_data = this.getStatus( token ); + token_data.message = "Dead requests cannot be accepted"; + + response.stateError( token_data, 'EDEAD' ); + }, + + + /** + * Respond indicating that TOKEN has already been accepted and cannot be + * re-accepted + * + * @param {UserResponse} response pending service response + * @param {Object} token applicable token + * + * @return {undefined} + */ + 'virtual protected respAcceptAccepted': function( response, token ) + { + var token_data = this.getStatus( token ); + token_data.message = "Request has already been accepted"; + + response.stateError( token_data, 'EACCEPTED' ); + }, + + + /** + * Respond indicating that TOKEN has completed and cannot be killed + * + * @param {UserResponse} response pending service response + * @param {Object} token applicable token + * + * @return {undefined} + */ + 'virtual protected respCannotKill': function( response, token ) + { + var token_data = this.getStatus( token ); + token_data.message = "Completed requests cannot be killed"; + + response.stateError( token_data, 'EDONE' ); + }, + + + /** + * Respond with an indication that the provided token is somehow bad and + * cannot be used to fulfill the request + * + * @param {UserResponse} response pending service response + * @param {Object} token applicable token + * + * @return {undefined} + */ + 'virtual protected respTokenError': function( response, tokid ) + { + response.notFound( + { message: "Bad token: " + tokid }, + 'EBADTOK' + ); + }, + + + /** + * Retrieve TOKEN data formatted for a response to a service request + * + * If TOKEN does not represent a valid token, a special status object + * will be generating indicating that the token is corrupt; this should + * never happen when maintaining encapsulation through this system. + * + * @param {Object} token token to format + * + * @return {undefined}} + */ + 'virtual protected getStatus': function( token ) + { + if ( !token || !token.id || typeof token.id !== 'string' ) + { + return { + token: '0BAD', + status: 'CORRUPT', + timestamp: '0', + }; + } + + return { + token: token.id, + status: token.status.type, + timestamp: token.status.timestamp, + }; + }, + + + /** + * Fulfill a service request by issuing a new token and continuing + * service processing outside the scope of the original REQUEST + * + * Once the underlying service request completes, the token will be + * updated to indicate that processing is complete, and will be assigned + * the data provided by the underlying service. Please note that + * warning in the description of `TokenedService` regarding fault + * tolerance. + * + * @param {string} cmd service command string + * @param {UserRequest} request service request + * @param {UserResponse} response pending response to request + * @param {Quote} quote request quote + * @param {Function} callback continuation after saving is complete + * + * @return {undefined} + */ + 'virtual protected serveWithNewToken': function( + cmd, request, response, quote, callback + ) + { + var _self = this; + var program = quote.getProgram(); + + this.generateToken( program, quote, function( err, token ) + { + // fulfill the request immediate with the new token; the user + // will wait and accept the data separately once it's done + response.accepted( { + tokenId: token.id, + status: token.status, + } ); + + // the original request will be performed in the background with + // our own response object, allowing us to capture the result + var capture_resp = _self._captureGen( + request, + function( code, error, data ) + { + _self.completeToken( quote, token, data, function( e, _ ) + { + // TODO: handle save error (this will at least cause + // it to be logged) + if ( e !== null ) + { + throw e; + } + } ); + } + ); + + _self.request.super.call( + _self, request, capture_resp, quote, cmd, callback + ); + } ); + }, + + + /** + * Generate a new token for QUOTE with a default token status + * + * @param {Program} program QUOTE program + * @param {Quote} quote request quote + * + * @param {function(?Error,Object}} callback continuation + * + * @return {undefined} + */ + 'virtual protected generateToken': function( program, quote, callback ) + { + var tokid = this._tokgen( program, quote ), + status = this.getDefaultTokenStatus(); + + this._dao.updateToken( + quote.getId(), + this._ns, + tokid, + status, + null, + function( err ) + { + if ( err ) + { + callback( err, null ); + return; + } + + callback( + null, + { + id: tokid, + status: status, + } + ); + } + ); + }, + + + /** + * Default status of newly created tokens + * + * This exists to permit subtype overrides. + * + * @return {string} default token status + */ + 'virtual protected getDefaultTokenStatus': function() + { + return 'ACTIVE'; + }, + + + /** + * Mark TOKEN as dead + * + * @param {Quote} quote request quote + * @param {Object} token token to kill + * + * @param {function(?Error,Object)} callback continuation + */ + 'virtual virtual protected killToken': function( quote, token, callback ) + { + callback = callback || function() {}; + + this._dao.updateToken( + quote.getId(), + this._ns, + token.id, + 'DEAD', + null, + function( err ) + { + if ( err ) + { + callback( err, null ); + return; + } + + callback( + null, + { + id: token, + status: 'DEAD', + } + ); + } + ); + }, + + + /** + * Mark TOKEN as having been accepted + * + * XXX: largely duplicated from `#killToken`. + * + * @param {Quote} quote request quote + * @param {Object} token token to accept + * + * @param {function(?Error,Object)} callback continuation + */ + 'virtual protected acceptToken': function( quote, token, callback ) + { + callback = callback || function() {}; + + this._dao.updateToken( + quote.getId(), + this._ns, + token.id, + 'ACCEPTED', + null, + function( err ) + { + if ( err ) + { + callback( err, null ); + return; + } + + callback( + null, + { + id: token, + status: 'ACCEPTED', + } + ); + } + ); + }, + + + /** + * Mark TOKEN as having been completed (ready to accept) + * + * XXX: largely duplicated from `#killToken`. + * + * @param {Quote} quote request quote + * @param {Object} token token to complete + * @param {string} data data from underlying service + * + * @param {function(?Error,Object)} callback continuation + */ + 'virtual protected completeToken': function( quote, token, data, callback ) + { + callback = callback || function() {}; + + this._dao.updateToken( + quote.getId(), + this._ns, + token.id, + 'DONE', + data, + function( err ) + { + if ( err ) + { + callback( err, null ); + return; + } + + callback( + null, + { + id: token, + status: 'DONE', + } + ); + } + ); + }, +} ); + diff --git a/src/server/service/export/ExportService.js b/src/server/service/export/ExportService.js new file mode 100644 index 0000000..f219c02 --- /dev/null +++ b/src/server/service/export/ExportService.js @@ -0,0 +1,232 @@ +/** + * Export to external system + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as + * published by the Free Software Foundation, either version 3 of the + * License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + * + * @todo relies on `c1` key; generalize (do not tie to ConceptOne + * terminology) + */ + +var Class = require( 'easejs' ).Class, + Service = require( '../Service' ), + Program = require( '../../../program/Program' ).Program, + Quote = require( '../../../quote/Quote' ), + UserRequest = require( '../../request/UserRequest' ), + UserResponse = require( '../../request/UserResponse' ); + + +/** + * Triggers external system import + */ +module.exports = Class( 'ExportService' ) + .implement( Service ) + .extend( +{ + /** + * @type {Object} object with `request` method for HTTP requests + */ + 'private _http': null, + + + /** + * Initialize service + * + * @param {Object} http object with `request` method for HTTP requests + */ + __construct: function( http ) + { + this._http = http; + }, + + + /** + * Trigger external system import + * + * This exports by requesting an import from an external + * system. EXPORT_CLIENT can be any client (e.g. ClientRequest) that + * has `response` and `error` events, with `response` yielding a single + * argument with a #setEncoding method and `data` and `end` events. + * + * @param {UserRequest} request user request to satisfy + * @param {UserResponse} response pending response + * @param {Quote} quote quote to export + * @param {string} cmd applicable of command request + * @param {Function} callback continuation after saving is complete + * + * @return {ExportService} self + */ + 'virtual public request': function( request, response, quote, cmd, callback ) + { + cmd = ''+( cmd || '' );; + + if ( !Class.isA( UserRequest, request ) ) + { + throw TypeError( + "UserRequest expected; given " + request + ); + } + + if ( !Class.isA( UserResponse, response ) ) + { + throw TypeError( + "UserResponse expected; given " + response + ); + } + + if ( !Class.isA( Quote, quote ) ) + { + throw TypeError("Quote expected; given " + quote ); + } + + if ( cmd ) + { + throw Error( "Unknown command: " + cmd ); + } + + var program = quote.getProgram(); + + this._handleExportRequest( + request, + response, + program, + quote, + callback + ); + + return this; + }, + + + 'private _handleExportRequest': function( + request, response, program, quote, callback + ) + { + var path; + var export_client = this._http.request( + request, + path = this._genClientPath( + program, + quote, + request.getGetData() + ) + ); + + this._doExport( export_client, function( e, reply ) + { + // TODO: incomplete + if ( e !== null ) + { + response.internalError( e.message ); + return; + } + + response.ok( reply ); + callback && callback(); + } ); + }, + + + /** + * Generate quote request URI + * + * @param {Program} program program associated with quote + * @param {Quote} quote quote to export + * + * @return {string} generated URI + */ + 'private _genClientPath': function( program, quote, params ) + { + var program_id = quote.getProgramId(), + quote_id = quote.getId(), + prefix = ( program.export_path || {} ).c1; + + if ( !prefix ) + { + throw Error( "Program missing export_path.c1" ); + } + + return prefix + + '?pid=' + program_id + + '&qid=' + quote_id + + this._joinParams( params ); + }, + + + /** + * XXX: This does no encoding! Instead, we should use Liza's + * HttpDataApi, which will handle all that stuff for us. + * + * @param {Object.} params key-value + */ + 'private _joinParams': function( params ) + { + var uri = ''; + + for ( var key in params ) + { + uri += ( uri ) ? '&' : ''; + uri += key + '=' + params[ key ]; + } + + return ( uri ) + ? '&' + uri + : ''; + }, + + + /** + * Request external import and await reply + * + * @param {ClientRequest} export_client pre-initialized HTTP client + * + * @param {function(?Error,?string)} callback + * + * @return {ExportService} self + */ + 'private _doExport': function( export_client, callback ) + { + export_client + .on( 'response', function( response ) + { + var data = ''; + + response.setEncoding( 'utf8' ); + + response + .on( 'data', function( chunk ) + { + data += chunk; + } ) + .on( 'end', function() + { + callback( null, data ); + } ); + } ); + + export_client.on( 'error', function( e ) + { + export_client.abort(); + callback( e, null ); + } ); + + export_client.end(); + + return this; + } +} ); + diff --git a/src/step/Step.js b/src/step/Step.js index d7b8e21..fbcf07c 100644 --- a/src/step/Step.js +++ b/src/step/Step.js @@ -95,7 +95,7 @@ module.exports = Class( 'Step' ) * Initializes step * * @param {number} id step identifier - * @param {QuoteClient} quote quote to contain step data + * @param {ClientQuote} quote quote to contain step data * * @return {undefined} */ diff --git a/src/ui/Ui.js b/src/ui/Ui.js new file mode 100644 index 0000000..e354754 --- /dev/null +++ b/src/ui/Ui.js @@ -0,0 +1,1572 @@ +/** + * Program UI class + * + * Copyright (C) 2017 LoVullo Associates, Inc. + * + * This file is part of the Liza Data Collection Framework. + * + * liza is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + * + * @todo this, along with Client, contains one of the largest and most + * coupled messes of the system; refactor + * + * @todo The code was vandalized with internal references and URLs---remove + * them (search "pollute")---and referenced a global variable! This + * might not work for you! + */ + +var Class = require( 'easejs' ).Class, + EventEmitter = require( 'events' ).EventEmitter; + +// XXX: decouple +var DynamicContext = require( './context/DynamicContext' ); + + +/** + * Creates a new Ui instance + * + * @param {Object} options ui options + * + * Supported options: + * content: {jQuery} content to operate on + * styler: {ElementStyler} element styler for misc. elements + * nav: {Nav} navigation object + * navStyler: {NavStyler} navigation styler + * errorBox: {FormErrorBox} error box to use for form errors + * sidebar: {Sidebar} sidebar ui + * dialog: {UiDialog} + * + * stepContainer: {jQuery} for the step HTML + * stepBuilder: {Function} function used to instantiate new steps + * + * @return {Ui} + */ +module.exports = Class( 'Ui' ).extend( EventEmitter, +{ + /** + * The Ui requested a step change + * @type {string} + */ + 'const EVENT_STEP_CHANGE': 'stepChange', + + /** + * Another step is about to be rendered + * @type {string} + */ + 'const EVENT_PRE_RENDER_STEP': 'preRenderStep', + + /** + * A different step has been rendered + * @type {string} + */ + 'const EVENT_RENDER_STEP': 'renderStep', + + /** + * Step has been rendered and all events are complete + * + * At this point, hooks may freely manipulate the step without risk of + * running before the framework is done with the step + * + * @type {string} + */ + 'const EVENT_STEP_READY': 'stepReady', + + /** + * Represents an action trigger + * @type {string} + */ + 'const EVENT_ACTION': 'action', + + + /** + * Content to operate on + * @type {jQuery} + */ + $content: null, + + /** + * Element styler to use for misc. elements in the UI (e.g. dialogs) + * @type {Styler} + */ + styler: null, + + /** + * Object responsible for handling navigation + * @type {Nav} + */ + nav: null, + + /** + * Styles navigation menu + * @type {NavStyler} + */ + navStyler: null, + + /** + * Navigation bar + * @type {jQuery} + */ + $navBar: null, + + /** + * Element to contain the step HTML + * @type {jQuery} + */ + $stepParent: null, + + /** + * Builder used to create new step instances + * @type {Function} + */ + buildStep: null, + + /** + * Holds previously loaded steps in memory + * @type {Object} + */ + stepCache: {}, + + /** + * Object representing the current step + * @type {Step} + */ + currentStep: null, + + /** + * Stores the steps that have already been appended to the DOM once + * @type {boolean} + */ + stepAppended: [], + + /** + * Represents the current quote + * @type {Quote} + */ + quote: null, + + /** + * Event to resume when quote is ready (for step navigation) + * @type {Object} + */ + quoteReadyEvent: null, + + /** + * Functions to call when step is to be saved + * @type {Array.} + */ + saveStepHooks: [], + + /** + * Error box to use for form errors + * @type {FormErrorBox} + */ + errorBox: null, + + /** + * Sidebar + * @type {Sidebar} + */ + sidebar: null, + + /** + * Whether navigation is frozen (prevent navigation) + * @type {boolean} + */ + navFrozen: false, + + /** + * Handles dialog display + * @type {UiDialog} + */ + _dialog: null, + + /** + * Active program + * @type {Program} + */ + 'private _program': null, + + /** + * Handles general UI styling + * @type {UiStyler} + */ + 'private _uiStyler': null, + + /** + * Navigation bar + * @type {UiNavBar} + */ + 'private _navBar': null, + + /** + * Notification bar + * @type {UiNotifyBar} + */ + 'private _notifyBar': null, + + 'private _cmatch': null, + + /** + * Root context + * @type {RootDomContext} + */ + 'private _rootContext': null, + + /** + * Step content cache + * @type {Array.} + */ + 'private _stepContent': [], + + /** + * Track field failures and fixes + * @type {DataValidator} + */ + 'private _dataValidator': null, + + + /** + * Initializes new UI instance + * + * @param {Object} options + * + * @return {undefined} + */ + __construct: function( options ) + { + this.$content = options.content; + this.styler = options.styler; + this.nav = options.nav; + this.navStyler = options.navStyler; + this.$navBar = this.$content.find( 'ul.step-nav' ); + this.$stepParent = options.stepContainer; + this.buildStep = options.stepBuilder; + this.errorBox = options.errorBox; + this.sidebar = options.sidebar; + this._dialog = options.dialog; + this._uiStyler = options.uiStyler; + this._navBar = options.navBar; + this._notifyBar = options.notifyBar; + this._rootContext = options.rootContext; + this._dataValidator = options.dataValidator; + }, + + + /** + * Initializes the UI + * + * @return Ui self to allow for method chaining + */ + init: function() + { + var _self = this; + + this._initStyles(); + this._initKeys(); + + this._initNavBar(); + + this.sidebar.init(); + + // set a context that will automatically adjust itself for the current + // active step (that is, once we are actually on a step) + _self.createDynamicContext( function( context ) + { + _self._uiStyler.setContext( context ); + } ); + + return this; + }, + + + /** + * Initializes styling + * + * This is used (a) because CSS cannot be used for certain conditions and + * (b) because IE6 doesn't support :hover for anything other than links. + * + * @return undefined + */ + _initStyles: function() + { + var ui = this; + + this._uiStyler + .init( this.$content ) + .on( 'questionHover', function( element, hover_over ) + { + ui._renderHelp( element, hover_over ); + }) + .on( 'questionFocus', function( element, has_focus ) + { + ui._renderHelp( element, has_focus ); + }); + }, + + + /** + * Render help text for the provided element + * + * @return {undefined} + */ + _renderHelp: function( element, show ) + { + // dt's are only labels and have no fields, but their sibling + // dd's do + var $element = ( element.nodeName == 'DT' ) + ? $( element ).next( 'dd' ) + : $( element ); + + if ( show ) + { + // set help message + this.sidebar.setHelpText( + this.styler.getHelpMessage( + $element.find( ':widget' ) + ) + ); + } + else + { + var text = '', + $focus = this.$content.find( 'dd.focus:first :widget' ); + + // attempt to fall back on the help for the focused element, + // if any + if ( $focus.length > 0 ) + { + text = this.styler.getHelpMessage( $focus ); + } + + this.sidebar.setHelpText( text ); + } + }, + + + /** + * Hooks the navigation bar to permit navigation + * + * @return void + */ + _initNavBar: function() + { + var _self = this; + this._navBar.on( 'click', function( step_id ) + { + // do not permit navigation via nav bar if the user has not already + // visited the step + if ( _self.nav.isStepVisited( step_id ) ) + { + _self.emit( _self.__self.$('EVENT_STEP_CHANGE'), step_id ); + } + }); + }, + + + /** + * Initializes keypress overrides + * + * This overrides the default enter key behavior to ensure that the correct + * button is "pressed". + * + * @return undefined + */ + _initKeys: function() + { + var ui = this; + + this.$content.find( 'form input' ).live( 'keypress.program', + function( e ) + { + if ( ( e.which && ( e.which == 13 ) ) + || ( e.keyCode && ( e.keyCode == 13 ) ) + ) + { + var $btn = ui.$content.find( 'button.default' ); + + // trigger the change event first to ensure any necessary + // assertions are run + $( this ).change(); + + // don't click it if it's disabled + if ( $btn.attr( 'disabled' ) ) + { + // but don't run the default behavior + return false; + } + + $btn.click(); + return false; + } + + return true; + } + ); + }, + + + /** + * Displays a step to the user + * + * If the step is already cached in memory, it will be immediately + * displayed. If not, it will use the assigned step builder in order to + * instantiate a new step and load it. This is an asynchronous operation. + * + * @param Integer step_id identifier representing step to navigate to + * + * @return Ui self to allow for method chaining + */ + 'public displayStep': function( step_id, callback ) + { + step_id = +step_id; + var step = this.stepCache[ step_id ]; + + // first thing to do is cache the current step and detach it + if ( this.currentStep !== null ) + { + // let the current + this._detachStep( this.currentStep ) + .setActive( false ); + } + + // build the step only if it is not yet loaded + if ( step === undefined ) + { + this._createStep( step_id, callback ); + + return this; + } + + this.currentStep = step; + + this.currentStep.setActive(); + this._renderStep( callback ); + + return this; + }, + + + /** + * Detaches the step STEP from the DOM + * + * @param {jQuery} step step to detach + * + * @return StepUi STEP to allow for method chaining + */ + _detachStep: function( step ) + { + this._getStepContent( step ) + .detach(); + + return step; + }, + + + /** + * Builds and initializes a new step + * + * @param Integer step_id id of the step to load + * + * @return Step new step + */ + _createStep: function( step_id, callback ) + { + var ui = this, + prevstep = this.currentStep; + + // prevent navigation while the step is downloading + this.freezeNav(); + + this.buildStep( step_id, function( stepui ) + { + ui.currentStep = ui.stepCache[ step_id ] = stepui; + ui.currentStep.setActive(); + ui._renderStep( callback ); + + stepui + .on( 'error', function() + { + var args = Array.prototype.slice.call( arguments ); + args.unshift( 'error' ); + + // forward to UI error event + ui.emit.apply( ui, args ); + } ) + .on( 'action', function( type, ref, index ) + { + // foward + ui.emit( ui.__self.$( 'EVENT_ACTION' ), type, ref, index ); + } ) + .on( 'displayChanged', function( id, index, value ) + { + var data = {}; + data[ id ] = []; + data[ id ][ index ] = value; + + ui._uiStyler.register( 'fieldFixed' )( data ); + } ); + + // we're done rendering the step; permit navigation + ui.unfreezeNav( prevstep ); + stepui.init(); + }); + }, + + + /** + * Renders the current step's HTML and styles it + * + * @return Ui self to allow for method chaining + */ + _renderStep: function( callback ) + { + var step = this.currentStep, + step_id = step.getStep().getId(), + prev_content = this._getStepContent( step ); + + var step_content = $( '
' ) + .attr( 'id', '__step' + step.getStep().getId() ) + .append( + prev_content || $( this.currentStep.getContent() ) + ); + + this._setStepContent( step, step_content ); + + // display the step (we have to append the container to the DOM before + // we append the step HTML, or dojo will throw a fit, since it won't see + // any of the elements it's trying to modify as part of the DOM + // document) + this.$stepParent.append( step_content ); + + // if this is the first time rendering the step, call the postAppend() + // method on it + if ( !( this.stepAppended[step_id] ) ) + { + // let the step process anything that should be done after the + // elements have been added to the DOM + this.currentStep.postAppend(); + + // let's not do this again + this.stepAppended[step_id] = true; + this._addNavButtons( this.currentStep ); + } + + // we need to emit this before we display to the user, but *after* the + // steps have had the chance to initialize their elements and add them + // to the DOM (otherwise, selectors would fail if we are trying to + // manipulate the DOM further before displaying it to the user) + this.currentStep.preRender(); + this.emit( + this.__self.$('EVENT_PRE_RENDER_STEP'), + this.currentStep, + step_content + ); + + var ui = this; + + setTimeout( function() + { + // raise the event + ui.emit( ui.__self.$('EVENT_RENDER_STEP'), ui.currentStep ); + }, 50 ); + + this._postRenderStep( function() + { + // call the callback before the timeout, allowing us to do stuff before + // repaint + callback && callback(); + + ui.unfreezeNav( step ); + + ui.currentStep.visit( function() + { + ui.emit( + ui.__self.$('EVENT_STEP_READY'), + ui.currentStep + ); + } ); + } ); + + return this; + }, + + + /** + * Retrieve cached stap content + * + * @param {StepUi=} step step to retrieve cached content of + * + * @return {jQuery} cached step content + */ + _getStepContent: function( step ) + { + var step_id = step.getStep().getId(); + + return this._stepContent[ step_id ]; + }, + + + /** + * Set cached step content + * + * @param {StepUi} step step to retrieve cached content of + * @param {jQuery} $content step content + * + * @return {Ui} self + */ + _setStepContent: function( step, $content ) + { + var step_id = step.getStep().getId(); + + this._stepContent[ step_id ] = $content; + + return this; + }, + + + _postRenderStep: function( callback ) + { + var self = this, + content = this._getStepContent( this.currentStep ); + + if ( content === null ) + { + return; + } + + // if the quote is locked, disable the form elements + var disable = false; + if ( this.quote.isLocked() ) + { + disable = true; + } + + this.currentStep.lock( + this.quote.isLocked() + || ( this.currentStep.getStep().getId() + < this.quote.getExplicitLockStep() + ) + ); + + if ( disable === false ) + { + // focus on the first element + content.find( 'input:first' ).focus(); + + // show buttons + this._getNavButtons( this.currentStep ).show(); + this.currentStep.hideAddRemove( false ); + } + else + { + // hide buttons + if ( this.nav.isLastStep( this.currentStep.getStep().getId() ) + === false + ) + { + this._getNavButtons( this.currentStep ).hide(); + } + else + { + // hide only the first (back) button on the last step + $( this._getNavButtons( this.currentStep )[0] ).hide(); + } + + // hide add/remove buttons on groups + this.currentStep.hideAddRemove( true ); + } + + callback && callback(); + }, + + + /** + * Adds the navigation buttons to the step + * + * @param Step step the step to operate on + * + * @return undefined + */ + _addNavButtons: function( step ) + { + var ui = this, + step_id = step.getStep().getId(); + + var $buttons = $( '