This repository has been archived by the owner on Mar 3, 2022. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 335
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Preparing to release petitions-7.x-3.0-alpha1
- Loading branch information
Ian Rose
committed
Jun 22, 2015
1 parent
af3aa17
commit 1a4ccb0
Showing
138 changed files
with
12,779 additions
and
4,204 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,98 +1,6 @@ | ||
| The 7.x-3.x branch of petitions aims to remove the application's dependency on | ||
| Mongo DB and run on MySQL. Between now and whenever 7.x-3.0 is released this | ||
| branch will be building out and refining a proof-of-concept for what will likely | ||
| turn out to be mysql-based petition nodes and signature entities. | ||
| Mongo DB and run on MySQL. | ||
|
|
||
| For a stable Mongo-based petitions application, please use the 7.x-2.x branch. | ||
|
|
||
|
|
||
| Technical notes on phasing out Mongo dependencies | ||
| ------------------------------------------------- | ||
|
|
||
| Modules to be removed by the code base by 7.x-3.0 (more modules may get added | ||
| to this list): | ||
| - wh_petitions | ||
| - petitions_data | ||
|
|
||
| Any new code added to the 7.x-3.x branch is being written in anticipation of | ||
| mongo storage being turned off, and petition nodes and signature_mail entities | ||
| based on a mysql back end. Here are some examples of how we're keeping things | ||
| organized in anticipation of mongo going away: | ||
|
|
||
| ### Example #1: Keeping things organized in mymodule.mongo2mysql.inc | ||
|
|
||
| Store functions that are intended to be deleted when mongo gets shut off in | ||
| mymodule.mongo2mysql.inc. | ||
|
|
||
| Name any new functions created in *.mongo2mysql.inc with mongo2mysql in the | ||
| function name. This way, when it's time to shut mongo off, all we have to do | ||
| is (1) find any files with mongo2mysql in the name and delete them, then (2) | ||
| grep through the code for mongo2mysql, and remove those function calls. | ||
|
|
||
|
|
||
| ### Example #2: Calling mongo-dependent functions and checking related shunt trips | ||
|
|
||
| While we're in transition phasing mongo out, application functionality | ||
| should be decoupled from a specific storage backend as much as possible. | ||
| If myslq-based data is available, it's authoritiative. But don't assume it's | ||
| available. If mysql-based data is NOT available, try getting mongo-based | ||
| data. But don't assume that's available either. Eventually it will get shut | ||
| off. For example: | ||
|
|
||
| ```php | ||
|
|
||
| function mymodule_does_something_with_petitions() { | ||
|
|
||
| // Load a petition. | ||
|
|
||
| // First try mysql (petition nodes). | ||
| $petition_is_enabled = module_exists('petition'); | ||
| $mysql_shunt_is_tripped = shunt_is_enabled('petition_mysql_save'); | ||
| if ($petition_is_enabled && !$mysql_shunt_is_tripped) { | ||
| $petition = node_load($nid); | ||
| } | ||
|
|
||
| // ******************************************************************* | ||
| // * Note: This entire block of code can simply be deleted after * | ||
| // * mysql is "turned on" and mongo is turned off. * | ||
| // ******************************************************************* | ||
| // | ||
| // If mysql data isn't available or we need to do something with mongo | ||
| // data for some reason, get it from mongo like this. | ||
| $mongo_shunt_is_tripped = shunt_is_enabled('wh_petitions_petition_create'); | ||
| if (!$mongo_shunt_is_tripped) { | ||
| $petition_from_mongo = petitions_data_mongo2mysql_get_petition($mongo_petition_id); | ||
| } | ||
| // Format petition data like a petition node, reconcile the two | ||
| // different petition objects, or throw watchdog errors. | ||
| $petition = mymodule_mongo2mysql_transitional_reconciling_and_formatting_happens_here($petition, $petition2); | ||
|
|
||
| // Now proceed to do stuff with your $petition node (or node-like | ||
| // object)... | ||
|
|
||
| } | ||
|
|
||
| ``` | ||
|
|
||
| For a more complex real-world example of ^^ this, see signatures_queue/includes/process_signatures.inc. | ||
|
|
||
|
|
||
| ### Example #3: Phasing out mongo-dependent functions. | ||
|
|
||
| To phase out mongo-dependent functions or anything else that's meant to be removed: | ||
| - Move functions intended to be removed into example.mongo2myql.inc files | ||
| - Rename functions like this: example_get_example() -> example_mongo2mysql_get_example() | ||
| - Replace calls to example_get_example() throughout the codebase with example_mongo2mysql_get_example() | ||
| - In example.mongo2mysql.inc provide the legacy function, but throw an error explaining what's going on | ||
|
|
||
| See petitions_data_get_petition and petitions_data_mongo2mysql_get_petition | ||
| as an example here: | ||
| https://github.com/bryanhirsch/_petitions/commit/453e2f96793e9ce8a95edbc325497bfebd298942 | ||
|
|
||
| Note: Lines like this [here](https://github.com/bryanhirsch/_petitions/commit/453e2f96793e9ce8a95edbc325497bfebd298942#diff-325f9f5d1e9e226e082ddc1c205b536eR47) | ||
| are NOT safe and future proofed as described in example #2. | ||
| Developers/Contributors, wherever possible, please code defensively and | ||
| write things like #2 along the way so there's less housekeeping to do when it's time to grep for things like | ||
| [this](https://github.com/bryanhirsch/_petitions/commit/453e2f96793e9ce8a95edbc325497bfebd298942#diff-325f9f5d1e9e226e082ddc1c205b536eR47) | ||
| and replace it with its mysql equivalents. | ||
|
|
||
| This document has been moved to an internal doc. Please contact us through the github issue queue to get involved: github.com/Whitehouse/petitions . |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,7 +2,7 @@ | |
|
|
||
| Drupal 7 code base used to build an application that lets users create and sign petitions. | ||
|
|
||
| This application is under active development and will continue to be modified and improved over time. The current release is an "alpha" pending changes that will remove its dependency on MongoDB (see “Roadmap” section below). | ||
| This application is under active development and will continue to be modified and improved over time. The current release is an "alpha". | ||
|
|
||
| ## Goals | ||
|
|
||
|
|
@@ -18,13 +18,12 @@ Releasing the source code for this application is meant to empower other governm | |
|
|
||
| * Drupal 7.x | ||
| * MySQL 5.x | ||
| * MongoDB 2.2.4 | ||
| * PHP 5.2 or 5.3 | ||
|
|
||
| *Recommended:* | ||
|
|
||
| * RAM +512 M | ||
| * Dedicated MongoDB server (this dependency will be removed soon, see “Roadmap”) | ||
| * Dedicated Solr server | ||
|
|
||
| ## Usage | ||
|
|
||
|
|
@@ -36,6 +35,8 @@ For installation instructions, see INSTALL.md. | |
|
|
||
| NOTE: Setting up the application and configuring it for use in your organization’s context requires Drupal development experience. The application ships with a similar design (theme) to what is used on petitions.whitehouse.gov and needs to be customized for use by others using standard Drupal 7 themeing conventions. The application also ships with various user interface elements, user account settings, and other configurations that users should expect to customize using standard Drupal 7 techniques and conventions. | ||
|
|
||
| Keys for the write API are issued, validated, and rate limited by a third-party service. | ||
|
|
||
| ## Roadmap | ||
|
|
||
| Have an idea or question about future features for We the People? Let us know by opening a ticket on GitHub, emailing us directly at [email protected], or tweeting @WHWeb. | ||
|
|
@@ -44,12 +45,6 @@ We the People is a work in progress and currently exists at a very basic level o | |
|
|
||
| The following descriptions are for informational purposes only and should not be interpreted as commitments or guarantees of future code releases in any way. | ||
|
|
||
| *Move from MongoDB to MySQL* | ||
|
|
||
| The current release depends on MongoDB. When we first created the application, we wanted to make sure we had a highly scalable application and database to meet our anticipated performance needs under high loads. We have been running MongoDB in production for over a year, but we have decided that the performance benefits it provides are outweighed by the complexity of trying to extend Drupal features backed by MongoDB. | ||
|
|
||
| We are currently moving to a fully MySQL-backed application to increase the speed the development of new features and other aspects of maintaining the Drupal application. Our next release will be a dev branch that will be fully backed by MySQL, and once there is a tag for that branch, we will no longer maintain the MongoDB branch. | ||
|
|
||
| *Install Profile* | ||
|
|
||
| The codebase is released as-is and currently supports a specific, standalone website. In the future we would like to provide an install profile that supports a wider range of applications. | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
52 changes: 52 additions & 0 deletions
52
modules/custom/migrate_mongo2mysql_petsig/link_responses_to_mysql_petitions.drush
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| <?php | ||
|
|
||
| /** | ||
| * @file | ||
| * Drush script to be run one time only. Creates two-way links between response | ||
| * nodes and petition nodes after complete mongo2mysql migration is complete. | ||
| * Only to be run on sites utilizing MySQL. | ||
| * | ||
| * Instructions on when to use this file can be found in: | ||
| * - /README/PT-1532-migration-release-instructions.txt | ||
| */ | ||
|
|
||
| #!/usr/bin/env drush | ||
|
|
||
| // Select all links from petitions to responses. | ||
| $query = db_query("SELECT 'node' as entity_type, r.type as bundle, 0 as deleted, r.nid as entity_id, r.vid as revision_id, r.language as language, p.entity_id as field_petition_id_target_id | ||
| FROM node r | ||
| INNER JOIN field_data_field_response_id p ON p.field_response_id_target_id=r.nid | ||
| WHERE type='wh_response'"); | ||
|
|
||
| // Create corresponding links from responses to petitions. | ||
| $deltas = array(); | ||
| foreach ($query as $result) { | ||
| if (!isset($deltas[$result->entity_id])) { | ||
| $deltas[$result->entity_id] = 0; | ||
| } | ||
| else { | ||
| $deltas[$result->entity_id]++; | ||
| } | ||
| db_query("INSERT IGNORE INTO field_data_field_petition_id (entity_type, bundle, deleted, entity_id, revision_id, language, delta, field_petition_id_target_id) VALUES (:entity_type, :bundle, :deleted, :entity_id, :revision_id, :language, :delta, :field_petition_id_target_id)", | ||
| array( | ||
| ':entity_type' => $result->entity_type, | ||
| ':bundle' => $result->bundle, | ||
| ':deleted' => $result->deleted, | ||
| ':entity_id' => $result->entity_id, | ||
| ':revision_id' => $result->revision_id, | ||
| ':language' => $result->language, | ||
| ':delta' => $deltas[$result->entity_id], | ||
| ':field_petition_id_target_id' => $result->field_petition_id_target_id, | ||
| )); | ||
| db_query("INSERT IGNORE INTO field_revision_field_petition_id (entity_type, bundle, deleted, entity_id, revision_id, language, delta, field_petition_id_target_id) VALUES (:entity_type, :bundle, :deleted, :entity_id, :revision_id, :language, :delta, :field_petition_id_target_id)", | ||
| array( | ||
| ':entity_type' => $result->entity_type, | ||
| ':bundle' => $result->bundle, | ||
| ':deleted' => $result->deleted, | ||
| ':entity_id' => $result->entity_id, | ||
| ':revision_id' => $result->revision_id, | ||
| ':language' => $result->language, | ||
| ':delta' => $deltas[$result->entity_id], | ||
| ':field_petition_id_target_id' => $result->field_petition_id_target_id, | ||
| )); | ||
| } |
Oops, something went wrong.