# Torque Design ## Overview Torque allows different groups of people to comment on proposals, and to indicate favored proposals in some specific way. The initial system is designed around the needs of grant-making foundations that are running contests similar to the [100&Change](https://www.macfound.org/programs/100change/) competition run by the [MacArthur Foundation](https://macfound.org/) and [Lever for Change](https://www.leverforchange.org/). Those latter two organizations and their partners are supplying the first use cases, and these requirements are designed with them in mind. However, we are also trying to keep the system generalizable to other proposal evaluation processes, and are interested in suggestions or contributions in that direction. ## Shape of the data. The fundamental content objects are pages and comments. Each wiki page represents one proposal, and every comment is associated with a particular page (even perhaps with specific text on that page). Wiki pages are generated automatically generated from [torque](https://code.librehq.com/ots/mediawiki/torque) that operates on a spreadsheet of proposals, one row per proposal, and thus are not to be edited by users. Users interact with the data is by making per-page comments, and in some cases by marking a page via some kind of selector or scoring interface (e.g., checking a box to indicate "I like this proposal" or something like that). Each comment has its own thread. The system does not use wiki "Talk" pages for commenting, because the UX isn't quite right. Instead, we use a system explicitly adapted for commenting in our style; see our [extenstion](https://code.librehq.com/ots/mediawiki/TeamComments). ### Rich Text Fields Rich text fields, such as executive summary, are stored in markdown. This decision was come to in a meeting with several LFC team members. The reasons for this are: * It is supported by a large number of libraries * It works well with textual data analysis * It is natively stored in airtable rich text fields * It is easy to convert mediawiki text * It is comprehensive enough to handle the kinds of rich text needed by proposal data ## Which proposals to include. LFC Torque competitions should include all proposals that made it to the administrative review stage -- whether they passed that review or not. In other words, we don't include proposals where the registrant never submitted it or where it somehow got disqualified before even getting to admin review, but other than that we include everything (and the result of admin review, that is, the proposal's valid/invalid status, is always one of the fields in a proposal's data). (Note that as of 2020-11-05, we may not meet the above requirements for all competitions currently on the production site, and furthermore, the name of the validity column is not even the same across the competitions that do have it. See issues [#79](https://code.librehq.com/ots/torque-sites/issues/79) and [#80](https://code.librehq.com/ots/torque-sites/issues/80) for more.) ## Users and their capabilities. Every user of the system is authenticated; there is no anonymous access. ### User Groups We use the same authorization groups across competitions whenever possible, since access patterns tend to be the same. The roles we've settled on are found on the [help wiki](https://torque.leverforchange.org/help). See [roles/permissions/files/default-permissions.php](roles/permissions/files/default-permissions.php) for technical details of each role's access permissions. Each of these user groups needs to be set up in torque configuration in order to view data. In the case of group information coming from outside (see below about Okta), a mapping needs to be done to these groups. Note that this "User Groups" section should always be present because older documents may still point to it. (For historical background, see [issue #62](https://code.librehq.com/ots/torque-sites/issues/62) and the [2020-03-17 meeting notes](https://code.librehq.com/ots/torque/wiki/Meeting-Notes#2020-03-17-frankkarl-discussion-re-eo-and-usergroup-permissions).) ### Shared user accounts Some partner organizations may use a shared account -- a single login used by multiple individuals at that organization. This kind of account is no different from any other in terms of how it functions technically. This note is merely here to record that, as a matter of policy, we've decided it's okay to have shared accounts for certain trusted organizations. ## Authentication, Authorization, and Okta Setting up permissions for those who have logged in via Okta happens like this: 1. Individual (or group) is enabled in Torque app in Okta applications section, with appropriate group set. 2. Logins by that person will show up with Board member preferences. ### "Local login" (i.e., regular, non-Okta MediaWiki login) is also possible. After [much](https://code.librehq.com/ots/torque/wiki/Meeting-Notes#2020-11-20-10am-ct-meeting-about-printinguploads) [discussion](https://code.librehq.com/ots/torque/wiki/Meeting-Notes#2020-11-19-11am-ct-regular-check-in-meeting), official policy is give normal wiki-using users Okta accounts, except maybe in the rarest of circumstances. However, regular MediaWiki logins can be created if necessary. These are known as "local logins" (or "local-login" as the adjectival form: in case you were searching for it with the hyphen, congratulations, you found it). We sometimes use local-login accounts for testing, although now that OTS folks have Okta accounts that's less common. *Note that a local-login account must not have "@" in the username.* In other words, just make it a regular-looking MediaWiki username, not an email address the way Okta-based accounts are. The reasons for this are described in [this conversation](https://chat.opentechstrategies.com/#narrow/stream/45-Lever-for.20Change/topic/Accounts/near/97563). Since someone can have both an Okta account and a local-login account, and since competition wikis typically redirect non-logged-in visitors straight to an Okta login page, you need ask for local login page explicitly, like this: https://torque.leverforchange.org/COMPETITION/locallogin.php (Note: The cookie will expire in an hour, and you can turn it off any time via `.../?locallogin=off`.) LFC Torque sites don't have a "logout" button, because normal users never need to log out -- Okta keeps them permanently logged in and re-prompts them for authentication information periodically. But local-login users may need to log out explicitly (for example, to switch over to their Okta-based account). Here's how to log out: https://torque.leverforchange.org/COMPETITION/index.php/Special:UserLogout (Note that Frank Duncan [got a patch merged upstream into MediaWiki's PluggableAuth extension](https://gerrit.wikimedia.org/r/c/mediawiki/extensions/PluggableAuth/+/589849) so that MediaWiki will show a logout button when a user is locally logged in (there's an internal `LocalLogin` variable that tracks this). So once we have deployed a new enough version of MediaWiki, the special logout instructions here will no longer be needed.) ### Granting API access API access is currently grated on a per-competition basis. API users do not use SSO (Okta) authentication; they must be created with a ["local-login" account](https://torque.leverforchange.org/COMPETITION/index.php/Special:CreateAccount) (see above for more about such accounts). To do the following steps, you should be logged in yourself as "Admin" via local-login. (Even if your regular Okta-based account has permission to create new users, there could be [complications](https://chat.opentechstrategies.com/#narrow/stream/45-Lever-for.20Change/topic/API/near/97930) when you try to do some of the steps. Trust us: it's better just to be 'Admin' from the start.) 1) [Create the local-login user](https://torque.leverforchange.org/COMPETITION/index.php/Special:CreateAccount). We typically use a username of the form `Jrandom.API`, though some legacy usernames may not match that form. 2) Send them their password by a secure channel. (For OTS, record the account creation in the appropriate `opass` file too please.) 3) [Put the new user into the appropriate group](https://torque.leverforchange.org/COMPETITION/index.php/Special:UserRights). For LFC staff, that group is usually `administrator` (i.e., sysop). For non-LFC API users, it's usually a dedicated API group that has been configured (as per below) to have access to just certain columns and/or proposals. For example, for the MIT Knowledge Futures Group, we used `API_KFG` as the group. 4) If necessary, [add permissions configuration for the group](https://torque.leverforchange.org/COMPETITION/index.php/TorqueConfig:MainConfig). This involves creating or updating the appropriate row in the Permissions table, and it may also involve creating a new Columns or Proposals listing page, if the listing page you need doesn't already exist. (For example, the first time you grant a non-LFC person API access to a competition, that competition might not already have a Columns page that excludes comment fields.) We had a [long chat on 2020-12-15](https://chat.opentechstrategies.com/#narrow/stream/45-Lever-for.20Change/topic/API/near/97830) about API account creation. If you run into a problem with any of the steps above, or if a step seems to be missing, take a look at that chat transcript and see if it offers any help. ## Automated deployment and content management Torque is deployed using [Ansible](https://www.ansible.com/). But because each Torque site is a wiki, Torque originated pages are set with a flag that disables editing by non Administrator users. ### Preference for using OS distro packages instead of source installs Our long-term preference is to depend on the versions of applications and libraries that are packaged in the underlying operating system distribution (typically Debian GNU/Linux), rather than, say, installing from stored source tarballs via ansible. By using OS-packaged versions, security updates come in as part of standard OS updates. There is still work to be done to reach this goal. For example, we'll need to switch to using one instance of MediaWiki across Torque, albeit with different databases (that is, from an external perspective, it would continue to look like multiple separate wikis). [Issue #56](https://code.librehq.com/ots/torque-sites/issues/56) describes this in detail, and [issue #55](https://code.librehq.com/ots/torque-sites/issues/55) describes a similar situation with SimpleSamlPHP. ## Generating PDF books from sets of articles Currently, book creation in torque is done through SimpleBook, a fork of the now obsolete system that is what Wikipedia itself formerly used: the [Collection](https://www.mediawiki.org/wiki/Extension:Collection) extension. ## Reference These requirements are [discussed in more detail](https://chat.opentechstrategies.com/#narrow/stream/45-Lever-for.20Change/topic/hello/near/69877) in our chat room.