covid-19-mrna-4th-place-solution.ipynb

{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Fourth place solution to the \"OpenVaccine: COVID-19 mRNA Vaccine Degradation Prediction\"\n\nThis notebook provides the code that can be used to generate our fourth place solution. Our final submission consisted of a blend of multiple variations of this notebook. The variations were only in the 2 RNN layers of the classification part of our network (LSTM+LSTM, LSTM+GRU, GRU+LSTM, GRU+GRU).\n\nThis notebook is mainly based on [this great notebook](https://www.kaggle.com/mrkmakr/covid-ae-pretrain-gnn-attn-cnn) from [@mrmakr](https://www.kaggle.com/mrkmakr).\n\nDifferent datasets are used:\n* **Augmented data:** the output from [@its7171](https://www.kaggle.com/its7171) [his notebook](https://www.kaggle.com/its7171/how-to-generate-augmentation-data). This dataset contains different possible structures for the same RNA sequence.\n* **BPPs generated with ARNIE:** we generated BPPs with different packages using ARNIE. We [provide a notebook te demonstrate this](https://www.kaggle.com/group16/generating-bpps-with-arnie). We also re-use the structures to calculate a certainty feature (fraction of packages that agree on a certain predicted structure).\n* **Predicted loop types with CapR** [CapR](https://github.com/fukunagatsu/CapR) generates probabilities for each of the loop types."},{"metadata":{},"cell_type":"markdown","source":"# 1 Processing the data\n\nWe will create the following inputs for the train set, public test set and private test set:\n* A `NxLxLxE` adjacency matrix, with `N` the number of samples, `L` the sequence length and `E` the number of adjacency (edge) features.\n* A `NxLxV` node feature matrix, with `N` number of samples, `L` sequence length and `V` number of node features (features per base of the RNA sequence)\n* A `NxT` target matrix, with `T`=5 our 5 targets. We also train on the non-scored targets as that improved overall performance."},{"metadata":{"trusted":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport gc\nimport os\nimport pickle\nfrom tqdm.notebook import tqdm\n\n%matplotlib inline\nimport matplotlib.pyplot as plt\n\nimport warnings\nwarnings.filterwarnings('ignore')","execution_count":1,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 1.1 Loading the competition data"},{"metadata":{"trusted":true},"cell_type":"code","source":"train = pd.read_json(\"../input/stanford-covid-vaccine/train.json\",lines=True)\ntest  = pd.read_json(\"../input/stanford-covid-vaccine/test.json\",lines=True)\nsub = pd.read_csv(\"../input/stanford-covid-vaccine/sample_submission.csv\")\n\ntest_pub = test[test[\"seq_length\"] == 107]\ntest_pri = test[test[\"seq_length\"] == 130]","execution_count":2,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.head(5)","execution_count":3,"outputs":[{"output_type":"execute_result","execution_count":3,"data":{"text/plain":"   index            id                                           sequence  \\\n0      0  id_001f94081  GGAAAAGCUCUAAUAACAGGAGACUAGGACUACGUAUUUCUAGGUA...   \n1      1  id_0049f53ba  GGAAAAAGCGCGCGCGGUUAGCGCGCGCUUUUGCGCGCGCUGUACC...   \n2      2  id_006f36f57  GGAAAGUGCUCAGAUAAGCUAAGCUCGAAUAGCAAUCGAAUAGAAU...   \n3      3  id_0082d463b  GGAAAAGCGCGCGCGCGCGCGCGAAAAAGCGCGCGCGCGCGCGCGC...   \n4      4  id_0087940f4  GGAAAAUAUAUAAUAUAUUAUAUAAAUAUAUUAUAGAAGUAUAAUA...   \n\n                                           structure  \\\n0  .....((((((.......)))).)).((.....((..((((((......   \n1  .....(((((((((((((((((((((((....)))))))))).)))...   \n2  .....((((.((.....((((.(((.....)))..((((......)...   \n3  ......((((((((((((((((......))))))))))))))))((...   \n4  .....(((((((.((((((((((((.(((((((((....)))))))...   \n\n                                 predicted_loop_type  signal_to_noise  \\\n0  EEEEESSSSSSHHHHHHHSSSSBSSXSSIIIIISSIISSSSSSHHH...            6.894   \n1  EEEEESSSSSSSSSSSSSSSSSSSSSSSHHHHSSSSSSSSSSBSSS...            0.193   \n2  EEEEESSSSISSIIIIISSSSMSSSHHHHHSSSMMSSSSHHHHHHS...            8.800   \n3  EEEEEESSSSSSSSSSSSSSSSHHHHHHSSSSSSSSSSSSSSSSSS...            0.104   \n4  EEEEESSSSSSSBSSSSSSSSSSSSBSSSSSSSSSHHHHSSSSSSS...            0.423   \n\n   SN_filter  seq_length  seq_scored  \\\n0          1         107          68   \n1          0         107          68   \n2          1         107          68   \n3          0         107          68   \n4          0         107          68   \n\n                                    reactivity_error  \\\n0  [0.1359, 0.20700000000000002, 0.1633, 0.1452, ...   \n1  [2.8272, 2.8272, 2.8272, 4.7343, 2.5676, 2.567...   \n2  [0.0931, 0.13290000000000002, 0.11280000000000...   \n3  [3.5229, 6.0748, 3.0374, 3.0374, 3.0374, 3.037...   \n4  [1.665, 2.1728, 2.0041, 1.2405, 0.620200000000...   \n\n                                   deg_error_Mg_pH10  \\\n0  [0.26130000000000003, 0.38420000000000004, 0.1...   \n1  [73705.3985, 73705.3985, 73705.3985, 73705.398...   \n2  [0.1365, 0.2237, 0.1812, 0.1333, 0.1148, 0.160...   \n3  [73705.3985, 73705.3985, 73705.3985, 73705.398...   \n4  [4.2139, 3.9637000000000002, 3.2467, 2.4716, 1...   \n\n                                      deg_error_pH10  \\\n0  [0.2631, 0.28600000000000003, 0.0964, 0.1574, ...   \n1  [10.1986, 9.2418, 5.0933, 5.0933, 5.0933, 5.09...   \n2  [0.17020000000000002, 0.178, 0.111, 0.091, 0.0...   \n3  [11.8007, 12.7566, 5.7733, 5.7733, 5.7733, 5.7...   \n4  [3.0942, 3.015, 2.1212, 2.0552, 0.881500000000...   \n\n                                    deg_error_Mg_50C  \\\n0  [0.1501, 0.275, 0.0947, 0.18660000000000002, 0...   \n1  [16.6174, 13.868, 8.1968, 8.1968, 8.1968, 8.19...   \n2  [0.1033, 0.1464, 0.1126, 0.09620000000000001, ...   \n3  [121286.7181, 121286.7182, 121286.7181, 121286...   \n4  [2.6717, 2.4818, 1.9919, 2.5484999999999998, 1...   \n\n                                       deg_error_50C  \\\n0  [0.2167, 0.34750000000000003, 0.188, 0.2124, 0...   \n1  [15.4857, 7.9596, 13.3957, 5.8777, 5.8777, 5.8...   \n2  [0.14980000000000002, 0.1761, 0.1517, 0.116700...   \n3  [15.3995, 8.1124, 7.7824, 7.7824, 7.7824, 7.78...   \n4  [1.3285, 3.6173, 1.3057, 1.3021, 1.1507, 1.150...   \n\n                                          reactivity  \\\n0  [0.3297, 1.5693000000000001, 1.1227, 0.8686, 0...   \n1  [0.0, 0.0, 0.0, 2.2965, 0.0, 0.0, 0.0, 0.0, 0....   \n2  [0.44820000000000004, 1.4822, 1.1819, 0.743400...   \n3  [0.0, 2.2399, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0....   \n4  [0.8267, 2.6577, 2.8481, 0.40090000000000003, ...   \n\n                                         deg_Mg_pH10  \\\n0  [0.7556, 2.983, 0.2526, 1.3789, 0.637600000000...   \n1  [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...   \n2  [0.2504, 1.4021, 0.9804, 0.49670000000000003, ...   \n3  [0.0, -0.5083, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0...   \n4  [2.1058, 3.138, 2.5437000000000003, 1.0932, 0....   \n\n                                            deg_pH10  \\\n0  [2.3375, 3.5060000000000002, 0.3008, 1.0108, 0...   \n1  [4.947, 4.4523, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...   \n2  [2.243, 2.9361, 1.0553, 0.721, 0.6396000000000...   \n3  [3.4248, 6.8128, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,...   \n4  [4.7366, 4.6243, 1.2068, 1.1538, 0.0, 0.0, 0.7...   \n\n                                          deg_Mg_50C  \\\n0  [0.35810000000000003, 2.9683, 0.2589, 1.4552, ...   \n1  [4.8511, 4.0426, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,...   \n2  [0.5163, 1.6823000000000001, 1.0426, 0.7902, 0...   \n3  [0.0, -0.8365, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0...   \n4  [2.2052, 1.7947000000000002, 0.7457, 3.1233, 0...   \n\n                                             deg_50C  \n0  [0.6382, 3.4773, 0.9988, 1.3228, 0.78770000000...  \n1  [7.6692, 0.0, 10.9561, 0.0, 0.0, 0.0, 0.0, 0.0...  \n2  [0.9501000000000001, 1.7974999999999999, 1.499...  \n3  [7.6692, -1.3223, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0...  \n4  [0.0, 5.1198, -0.3551, -0.3518, 0.0, 0.0, 0.0,...  ","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>index</th>\n      <th>id</th>\n      <th>sequence</th>\n      <th>structure</th>\n      <th>predicted_loop_type</th>\n      <th>signal_to_noise</th>\n      <th>SN_filter</th>\n      <th>seq_length</th>\n      <th>seq_scored</th>\n      <th>reactivity_error</th>\n      <th>deg_error_Mg_pH10</th>\n      <th>deg_error_pH10</th>\n      <th>deg_error_Mg_50C</th>\n      <th>deg_error_50C</th>\n      <th>reactivity</th>\n      <th>deg_Mg_pH10</th>\n      <th>deg_pH10</th>\n      <th>deg_Mg_50C</th>\n      <th>deg_50C</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>id_001f94081</td>\n      <td>GGAAAAGCUCUAAUAACAGGAGACUAGGACUACGUAUUUCUAGGUA...</td>\n      <td>.....((((((.......)))).)).((.....((..((((((......</td>\n      <td>EEEEESSSSSSHHHHHHHSSSSBSSXSSIIIIISSIISSSSSSHHH...</td>\n      <td>6.894</td>\n      <td>1</td>\n      <td>107</td>\n      <td>68</td>\n      <td>[0.1359, 0.20700000000000002, 0.1633, 0.1452, ...</td>\n      <td>[0.26130000000000003, 0.38420000000000004, 0.1...</td>\n      <td>[0.2631, 0.28600000000000003, 0.0964, 0.1574, ...</td>\n      <td>[0.1501, 0.275, 0.0947, 0.18660000000000002, 0...</td>\n      <td>[0.2167, 0.34750000000000003, 0.188, 0.2124, 0...</td>\n      <td>[0.3297, 1.5693000000000001, 1.1227, 0.8686, 0...</td>\n      <td>[0.7556, 2.983, 0.2526, 1.3789, 0.637600000000...</td>\n      <td>[2.3375, 3.5060000000000002, 0.3008, 1.0108, 0...</td>\n      <td>[0.35810000000000003, 2.9683, 0.2589, 1.4552, ...</td>\n      <td>[0.6382, 3.4773, 0.9988, 1.3228, 0.78770000000...</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1</td>\n      <td>id_0049f53ba</td>\n      <td>GGAAAAAGCGCGCGCGGUUAGCGCGCGCUUUUGCGCGCGCUGUACC...</td>\n      <td>.....(((((((((((((((((((((((....)))))))))).)))...</td>\n      <td>EEEEESSSSSSSSSSSSSSSSSSSSSSSHHHHSSSSSSSSSSBSSS...</td>\n      <td>0.193</td>\n      <td>0</td>\n      <td>107</td>\n      <td>68</td>\n      <td>[2.8272, 2.8272, 2.8272, 4.7343, 2.5676, 2.567...</td>\n      <td>[73705.3985, 73705.3985, 73705.3985, 73705.398...</td>\n      <td>[10.1986, 9.2418, 5.0933, 5.0933, 5.0933, 5.09...</td>\n      <td>[16.6174, 13.868, 8.1968, 8.1968, 8.1968, 8.19...</td>\n      <td>[15.4857, 7.9596, 13.3957, 5.8777, 5.8777, 5.8...</td>\n      <td>[0.0, 0.0, 0.0, 2.2965, 0.0, 0.0, 0.0, 0.0, 0....</td>\n      <td>[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...</td>\n      <td>[4.947, 4.4523, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...</td>\n      <td>[4.8511, 4.0426, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,...</td>\n      <td>[7.6692, 0.0, 10.9561, 0.0, 0.0, 0.0, 0.0, 0.0...</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>2</td>\n      <td>id_006f36f57</td>\n      <td>GGAAAGUGCUCAGAUAAGCUAAGCUCGAAUAGCAAUCGAAUAGAAU...</td>\n      <td>.....((((.((.....((((.(((.....)))..((((......)...</td>\n      <td>EEEEESSSSISSIIIIISSSSMSSSHHHHHSSSMMSSSSHHHHHHS...</td>\n      <td>8.800</td>\n      <td>1</td>\n      <td>107</td>\n      <td>68</td>\n      <td>[0.0931, 0.13290000000000002, 0.11280000000000...</td>\n      <td>[0.1365, 0.2237, 0.1812, 0.1333, 0.1148, 0.160...</td>\n      <td>[0.17020000000000002, 0.178, 0.111, 0.091, 0.0...</td>\n      <td>[0.1033, 0.1464, 0.1126, 0.09620000000000001, ...</td>\n      <td>[0.14980000000000002, 0.1761, 0.1517, 0.116700...</td>\n      <td>[0.44820000000000004, 1.4822, 1.1819, 0.743400...</td>\n      <td>[0.2504, 1.4021, 0.9804, 0.49670000000000003, ...</td>\n      <td>[2.243, 2.9361, 1.0553, 0.721, 0.6396000000000...</td>\n      <td>[0.5163, 1.6823000000000001, 1.0426, 0.7902, 0...</td>\n      <td>[0.9501000000000001, 1.7974999999999999, 1.499...</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>3</td>\n      <td>id_0082d463b</td>\n      <td>GGAAAAGCGCGCGCGCGCGCGCGAAAAAGCGCGCGCGCGCGCGCGC...</td>\n      <td>......((((((((((((((((......))))))))))))))))((...</td>\n      <td>EEEEEESSSSSSSSSSSSSSSSHHHHHHSSSSSSSSSSSSSSSSSS...</td>\n      <td>0.104</td>\n      <td>0</td>\n      <td>107</td>\n      <td>68</td>\n      <td>[3.5229, 6.0748, 3.0374, 3.0374, 3.0374, 3.037...</td>\n      <td>[73705.3985, 73705.3985, 73705.3985, 73705.398...</td>\n      <td>[11.8007, 12.7566, 5.7733, 5.7733, 5.7733, 5.7...</td>\n      <td>[121286.7181, 121286.7182, 121286.7181, 121286...</td>\n      <td>[15.3995, 8.1124, 7.7824, 7.7824, 7.7824, 7.78...</td>\n      <td>[0.0, 2.2399, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0....</td>\n      <td>[0.0, -0.5083, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0...</td>\n      <td>[3.4248, 6.8128, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,...</td>\n      <td>[0.0, -0.8365, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0...</td>\n      <td>[7.6692, -1.3223, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0...</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>4</td>\n      <td>id_0087940f4</td>\n      <td>GGAAAAUAUAUAAUAUAUUAUAUAAAUAUAUUAUAGAAGUAUAAUA...</td>\n      <td>.....(((((((.((((((((((((.(((((((((....)))))))...</td>\n      <td>EEEEESSSSSSSBSSSSSSSSSSSSBSSSSSSSSSHHHHSSSSSSS...</td>\n      <td>0.423</td>\n      <td>0</td>\n      <td>107</td>\n      <td>68</td>\n      <td>[1.665, 2.1728, 2.0041, 1.2405, 0.620200000000...</td>\n      <td>[4.2139, 3.9637000000000002, 3.2467, 2.4716, 1...</td>\n      <td>[3.0942, 3.015, 2.1212, 2.0552, 0.881500000000...</td>\n      <td>[2.6717, 2.4818, 1.9919, 2.5484999999999998, 1...</td>\n      <td>[1.3285, 3.6173, 1.3057, 1.3021, 1.1507, 1.150...</td>\n      <td>[0.8267, 2.6577, 2.8481, 0.40090000000000003, ...</td>\n      <td>[2.1058, 3.138, 2.5437000000000003, 1.0932, 0....</td>\n      <td>[4.7366, 4.6243, 1.2068, 1.1538, 0.0, 0.0, 0.7...</td>\n      <td>[2.2052, 1.7947000000000002, 0.7457, 3.1233, 0...</td>\n      <td>[0.0, 5.1198, -0.3551, -0.3518, 0.0, 0.0, 0.0,...</td>\n    </tr>\n  </tbody>\n</table>\n</div>"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"test.head(5)","execution_count":4,"outputs":[{"output_type":"execute_result","execution_count":4,"data":{"text/plain":"   index            id                                           sequence  \\\n0      0  id_00073f8be  GGAAAAGUACGACUUGAGUACGGAAAACGUACCAACUCGAUUAAAA...   \n1      1  id_000ae4237  GGAAACGGGUUCCGCGGAUUGCUGCUAAUAAGAGUAAUCUCUAAAU...   \n2      2  id_00131c573  GGAAAACAAAACGGCCUGGAAGACGAAGGAAUUCGGCGCGAAGGCC...   \n3      3  id_00181fd34  GGAAAGGAUCUCUAUCGAAGGAUAGAGAUCGCUCGCGACGGCACGA...   \n4      4  id_0020473f7  GGAAACCCGCCCGCGCCCGCCCGCGCUGCUGCCGUGCCUCCUCUCC...   \n\n                                           structure  \\\n0  ......((((((((((.(((((.....))))))))((((((((......   \n1  .....((((..((((((...(((((.....((((....)))).......   \n2  ...........((.(((.(.(..((..((..((((...))))..))...   \n3  ......((((((((((....))))))))))((((((..((.(((.....   \n4  .....(((((((((((((((((((((((((((((((((((((((((...   \n\n                                 predicted_loop_type  seq_length  seq_scored  \n0  EEEEEESSSSSSSSSSBSSSSSHHHHHSSSSSSSSSSSSSSSSHHH...         107          68  \n1  EEEEESSSSIISSSSSSIIISSSSSIIIIISSSSHHHHSSSSIIII...         130          91  \n2  EEEEEEEEEEESSISSSISISIISSIISSIISSSSHHHSSSSIISS...         107          68  \n3  EEEEEESSSSSSSSSSHHHHSSSSSSSSSSSSSSSSIISSISSSHH...         107          68  \n4  EEEEESSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS...         130          91  ","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>index</th>\n      <th>id</th>\n      <th>sequence</th>\n      <th>structure</th>\n      <th>predicted_loop_type</th>\n      <th>seq_length</th>\n      <th>seq_scored</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>id_00073f8be</td>\n      <td>GGAAAAGUACGACUUGAGUACGGAAAACGUACCAACUCGAUUAAAA...</td>\n      <td>......((((((((((.(((((.....))))))))((((((((......</td>\n      <td>EEEEEESSSSSSSSSSBSSSSSHHHHHSSSSSSSSSSSSSSSSHHH...</td>\n      <td>107</td>\n      <td>68</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1</td>\n      <td>id_000ae4237</td>\n      <td>GGAAACGGGUUCCGCGGAUUGCUGCUAAUAAGAGUAAUCUCUAAAU...</td>\n      <td>.....((((..((((((...(((((.....((((....)))).......</td>\n      <td>EEEEESSSSIISSSSSSIIISSSSSIIIIISSSSHHHHSSSSIIII...</td>\n      <td>130</td>\n      <td>91</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>2</td>\n      <td>id_00131c573</td>\n      <td>GGAAAACAAAACGGCCUGGAAGACGAAGGAAUUCGGCGCGAAGGCC...</td>\n      <td>...........((.(((.(.(..((..((..((((...))))..))...</td>\n      <td>EEEEEEEEEEESSISSSISISIISSIISSIISSSSHHHSSSSIISS...</td>\n      <td>107</td>\n      <td>68</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>3</td>\n      <td>id_00181fd34</td>\n      <td>GGAAAGGAUCUCUAUCGAAGGAUAGAGAUCGCUCGCGACGGCACGA...</td>\n      <td>......((((((((((....))))))))))((((((..((.(((.....</td>\n      <td>EEEEEESSSSSSSSSSHHHHSSSSSSSSSSSSSSSSIISSISSSHH...</td>\n      <td>107</td>\n      <td>68</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>4</td>\n      <td>id_0020473f7</td>\n      <td>GGAAACCCGCCCGCGCCCGCCCGCGCUGCUGCCGUGCCUCCUCUCC...</td>\n      <td>.....(((((((((((((((((((((((((((((((((((((((((...</td>\n      <td>EEEEESSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS...</td>\n      <td>130</td>\n      <td>91</td>\n    </tr>\n  </tbody>\n</table>\n</div>"},"metadata":{}}]},{"metadata":{},"cell_type":"markdown","source":"## 1.2 Appending the augmentation data"},{"metadata":{"trusted":true},"cell_type":"code","source":"aug_df = pd.read_csv('../input/covid19-mrna-augmentation-data-and-features/aug_data1.csv')\naug_df = aug_df.drop_duplicates(subset=['id', 'structure'])\n\n\ndef aug_data(df):\n    target_df = df.copy()\n    new_df = aug_df[aug_df['id'].isin(target_df['id'])]\n                         \n    del target_df['structure']\n    del target_df['predicted_loop_type']\n    new_df = new_df.merge(target_df, on=['id','sequence'], how='left')\n\n    df['cnt'] = df['id'].map(new_df[['id','cnt']].set_index('id').to_dict()['cnt'])\n    df['log_gamma'] = 100\n    df['score'] = 1.0\n    df = df.append(new_df[df.columns])\n    return df\n\ntrain = aug_data(train).reset_index(drop=True)\ntest_pub = aug_data(test_pub).reset_index(drop=True)\ntest_pri = aug_data(test_pri).reset_index(drop=True)","execution_count":5,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 1.3 Creating adjacency matrices\n\nWe extract 5 edge features:\n* 3 different BPPs: the provided ones, generated with rnasoft (ARNIE) and generated with contrafold (ARNIE)\n* whether there is a base pairing indicated in the structure ( `(` and `)` )\n* the distances (manhattan) between bases, normalized by sequence length"},{"metadata":{"trusted":true},"cell_type":"code","source":"# The provided BPPs\nAs = []\nfor id in tqdm(train[\"id\"]):\n    a = np.load(f\"../input/stanford-covid-vaccine/bpps/{id}.npy\").astype(np.float16)\n    As.append(a)\nAs = np.array(As)\n\nAs_pub = []\nfor id in tqdm(test_pub[\"id\"]):\n    a = np.load(f\"../input/stanford-covid-vaccine/bpps/{id}.npy\").astype(np.float16)\n    As_pub.append(a)\nAs_pub = np.array(As_pub)\n\nAs_pri = []\nfor id in tqdm(test_pri[\"id\"]):\n    a = np.load(f\"../input/stanford-covid-vaccine/bpps/{id}.npy\").astype(np.float16)\n    As_pri.append(a)\nAs_pri = np.array(As_pri)\n\n# BPPs generated with ARNIE (package='contrafold2')\nAs_cf = []\nfor id in tqdm(train[\"id\"]):\n    a = np.load(f\"../input/covid19-mrna-augmentation-data-and-features/bpps_contrafold_gamma1/bpps_contrafold_gamma1/bpps_contrafold_gamma1/{id}.npy\").astype(np.float16)\n    As_cf.append(a)\nAs_cf = np.array(As_cf)\n\nAs_pub_cf = []\nfor id in tqdm(test_pub[\"id\"]):\n    a = np.load(f\"../input/covid19-mrna-augmentation-data-and-features/bpps_contrafold_gamma1/bpps_contrafold_gamma1/bpps_contrafold_gamma1/{id}.npy\").astype(np.float16)\n    As_pub_cf.append(a)\nAs_pub_cf = np.array(As_pub_cf)\n\nAs_pri_cf = []\nfor id in tqdm(test_pri[\"id\"]):\n    a = np.load(f\"../input/covid19-mrna-augmentation-data-and-features/bpps_contrafold_gamma1/bpps_contrafold_gamma1/bpps_contrafold_gamma1/{id}.npy\").astype(np.float16)\n    As_pri_cf.append(a)\nAs_pri_cf = np.array(As_pri_cf)\n\n# BPPs generated with ARNIE (package='rnasoft')\nAs_rs = []\nfor id in tqdm(train[\"id\"]):\n    a = np.load(f\"../input/covid19-mrna-augmentation-data-and-features/bpps_rnasoft_gamma1/bpps_rnasoft_gamma1/bpps_rnasoft_gamma1/{id}.npy\").astype(np.float16)\n    As_rs.append(a)\nAs_rs = np.array(As_rs)\n\nAs_pub_rs = []\nfor id in tqdm(test_pub[\"id\"]):\n    a = np.load(f\"../input/covid19-mrna-augmentation-data-and-features/bpps_rnasoft_gamma1/bpps_rnasoft_gamma1/bpps_rnasoft_gamma1/{id}.npy\").astype(np.float16)\n    As_pub_rs.append(a)\nAs_pub_rs = np.array(As_pub_rs)\n\nAs_pri_rs = []\nfor id in tqdm(test_pri[\"id\"]):\n    a = np.load(f\"../input/covid19-mrna-augmentation-data-and-features/bpps_rnasoft_gamma1/bpps_rnasoft_gamma1/bpps_rnasoft_gamma1/{id}.npy\").astype(np.float16)\n    As_pri_rs.append(a)\nAs_pri_rs = np.array(As_pri_rs)","execution_count":6,"outputs":[{"output_type":"display_data","data":{"text/plain":"HBox(children=(FloatProgress(value=0.0, max=4800.0), HTML(value='')))","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"a8c544b46fac48bd851ccc5024cda5fe"}},"metadata":{}},{"output_type":"stream","text":"\n","name":"stdout"},{"output_type":"display_data","data":{"text/plain":"HBox(children=(FloatProgress(value=0.0, max=1258.0), HTML(value='')))","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"0520eabba8cb4b6a9e9539576ee44704"}},"metadata":{}},{"output_type":"stream","text":"\n","name":"stdout"},{"output_type":"display_data","data":{"text/plain":"HBox(children=(FloatProgress(value=0.0, max=6010.0), HTML(value='')))","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"792128756bd9414496df0f4db81f9529"}},"metadata":{}},{"output_type":"stream","text":"\n","name":"stdout"},{"output_type":"display_data","data":{"text/plain":"HBox(children=(FloatProgress(value=0.0, max=4800.0), HTML(value='')))","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"4eff2508ad5645db863276bbda448aa9"}},"metadata":{}},{"output_type":"stream","text":"\n","name":"stdout"},{"output_type":"display_data","data":{"text/plain":"HBox(children=(FloatProgress(value=0.0, max=1258.0), HTML(value='')))","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"d85db62f2f2a4b4681b916df0e73407f"}},"metadata":{}},{"output_type":"stream","text":"\n","name":"stdout"},{"output_type":"display_data","data":{"text/plain":"HBox(children=(FloatProgress(value=0.0, max=6010.0), HTML(value='')))","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"ebb13c46ee454747aae433447b1ebfcd"}},"metadata":{}},{"output_type":"stream","text":"\n","name":"stdout"},{"output_type":"display_data","data":{"text/plain":"HBox(children=(FloatProgress(value=0.0, max=4800.0), HTML(value='')))","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"1966dcddc7a242e08a299f3ae67435e7"}},"metadata":{}},{"output_type":"stream","text":"\n","name":"stdout"},{"output_type":"display_data","data":{"text/plain":"HBox(children=(FloatProgress(value=0.0, max=1258.0), HTML(value='')))","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"b711c07156fe4847a8ee5da92d10c578"}},"metadata":{}},{"output_type":"stream","text":"\n","name":"stdout"},{"output_type":"display_data","data":{"text/plain":"HBox(children=(FloatProgress(value=0.0, max=6010.0), HTML(value='')))","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"1079d7cf23d44fbf88b88145077bf4f8"}},"metadata":{}},{"output_type":"stream","text":"\n","name":"stdout"}]},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"def get_structure_adj(train):\n    Ss = []\n    for i in tqdm(range(len(train))):\n        seq_length = train[\"seq_length\"].iloc[i]\n        structure = train[\"structure\"].iloc[i]\n        sequence = train[\"sequence\"].iloc[i]\n\n        cue = []\n        a_structures = {\n            (\"A\", \"U\") : np.zeros([seq_length, seq_length]),\n            (\"C\", \"G\") : np.zeros([seq_length, seq_length]),\n            (\"U\", \"G\") : np.zeros([seq_length, seq_length]),\n            (\"U\", \"A\") : np.zeros([seq_length, seq_length]),\n            (\"G\", \"C\") : np.zeros([seq_length, seq_length]),\n            (\"G\", \"U\") : np.zeros([seq_length, seq_length]),\n        }\n        a_structure = np.zeros([seq_length, seq_length])\n        for i in range(seq_length):\n            if structure[i] == \"(\":\n                cue.append(i)\n            elif structure[i] == \")\":\n                start = cue.pop()\n                a_structures[(sequence[start], sequence[i])][start, i] = 1\n                a_structures[(sequence[i], sequence[start])][i, start] = 1\n        \n        a_strc = np.stack([a for a in a_structures.values()], axis = 2)\n        a_strc = np.sum(a_strc, axis = 2, keepdims = True)\n        Ss.append(a_strc)\n    \n    Ss = np.array(Ss)\n    return Ss.astype(np.float16)\n\nSs = get_structure_adj(train)\nSs_pub = get_structure_adj(test_pub)\nSs_pri = get_structure_adj(test_pri)","execution_count":7,"outputs":[{"output_type":"display_data","data":{"text/plain":"HBox(children=(FloatProgress(value=0.0, max=4800.0), HTML(value='')))","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"9f841ffa965d455c9c295ae6197e2bfa"}},"metadata":{}},{"output_type":"stream","text":"\n","name":"stdout"},{"output_type":"display_data","data":{"text/plain":"HBox(children=(FloatProgress(value=0.0, max=1258.0), HTML(value='')))","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"5d93b9e1c25a4df0a54d477f765a02ea"}},"metadata":{}},{"output_type":"stream","text":"\n","name":"stdout"},{"output_type":"display_data","data":{"text/plain":"HBox(children=(FloatProgress(value=0.0, max=6010.0), HTML(value='')))","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"f9d35b2ec2ed43ab963eef0bc4c6da37"}},"metadata":{}},{"output_type":"stream","text":"\n","name":"stdout"}]},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_distance_matrix(As):\n    idx = np.arange(As.shape[1])\n    Ds = []\n    for i in range(len(idx)):\n        d = np.abs(idx[i] - idx)\n        Ds.append(d)\n\n    Ds = np.array(Ds) + 1\n    Ds = Ds / Ds.shape[1]\n    Ds = Ds[None, :,:]\n    Ds = np.repeat(Ds, len(As), axis = 0)\n    \n    Dss = []\n    for i in [1]:\n        Dss.append(Ds ** i)\n    Ds = np.stack(Dss, axis = 3)\n    return Ds.astype(np.float16)\n\nDs = get_distance_matrix(As)\nDs_pub = get_distance_matrix(As_pub)\nDs_pri = get_distance_matrix(As_pri)","execution_count":8,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"As = np.concatenate([As[:,:,:,None], As_cf[:,:,:,None], As_rs[:,:,:,None], Ss, Ds], axis = 3).astype(np.float16)\ndel Ss, Ds, As_cf, As_rs\n\nAs_pub = np.concatenate([As_pub[:,:,:,None], As_pub_cf[:,:,:,None], As_pub_rs[:,:,:,None], Ss_pub, Ds_pub], axis = 3).astype(np.float16)\ndel Ss_pub, Ds_pub, As_pub_cf, As_pub_rs\n\nAs_pri = np.concatenate([As_pri[:,:,:,None], As_pri_cf[:,:,:,None], As_pri_rs[:,:,:,None], Ss_pri, Ds_pri], axis = 3).astype(np.float16)\ndel Ss_pri, Ds_pri, As_pri_cf, As_pri_rs\n\nprint(As.shape, As_pub.shape, As_pri.shape)","execution_count":9,"outputs":[{"output_type":"stream","text":"(4800, 107, 107, 5) (1258, 107, 107, 5) (6010, 130, 130, 5)\n","name":"stdout"}]},{"metadata":{"trusted":true},"cell_type":"code","source":"f, ax = plt.subplots(1, 5, figsize=(15, 3))\nfor i in range(As.shape[-1]):\n    ax[i].imshow(As[0, :, :, i].astype(np.float32))\nplt.show()","execution_count":10,"outputs":[{"output_type":"display_data","data":{"text/plain":"<Figure size 1080x216 with 5 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAA24AAACzCAYAAADxGU8LAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOy9abAsyXUe9p3M6u67vHffvsy8ZR5mMAswGCwECGBIgoQNWABN0FwsMCjRDG4KBBSWA3Y4TEKWJYYirDAsBWTulBBaCJtyiAiuIkVxAwXS5ACDdbDNDGYeZn3z9n27W2ce/8ilsqqruqq7q27f7pdfxL3VVZWdlV19uiq/Ouc7h5gZEREREREREREREREREdsXYtoDiIiIiIiIiIiIiIiIiBiOSNwiIiIiIiIiIiIiIiK2OSJxi4iIiIiIiIiIiIiI2OaIxC0iIiIiIiIiIiIiImKbIxK3iIiIiIiIiIiIiIiIbY5I3CIiIiIiIiIiIiIiIrY5WiNuRPReIvoGEZ0kog+3dZyIiKYQbTZi1hBtNmLWEG02YtYQbTZiO4HaqONGRBLAMwD+KwCnAHwOwN9i5icbP1hERAOINhsxa4g2GzFriDYbMWuINhux3dCWx+2tAE4y83PMvAHg3wP4vpaOFRHRBKLNRswaos1GzBqizUbMGqLNRmwrJC31ewTAy8H6KQBvK2vcpR4vYHm0IxCZZQsew7kBAWj79GyHYxDhBl++yMwHJjjKFtis/RdtNgLADVzZUpsFtrfdkpSAEIAg8PrGaO/tJIBmsFItjS4CANZwCxu8ThN0sTU2GxFhMVWbtddOcvNVdy0tXJZso3ToPLAvbcPIbwvfkz9OsD08M4TBfsLtuW0huGhfeLyS7eY159bzbdL9lNuePU0cnD722/1rAETZ14SwbX7dvCbbb7jfbwP74wm7Ha6Nf3+4zL0msq/Jj+vFl/u4eFmV2mxbxK3ogJm7PhF9AMAHAGABS3gbvWt4j0KCBIGVglhcBCsF3tjIdzvGSCsmJEKapR5xQiDk6O+pi9Byi8YuJMDaNWj22K5vEqBOMvl3kCfgResksucyf26FxJ+p33hx/EGYIxVsm8xmzZvMIukArO3EsmXiNu5DDffdtjVBH/Zb24oHMUW2NEX8Gf9m6zYLjGG3wXmiTrc5u634jpNDhwEpoa9chd64Va/LJIHcvw/6xk3w+jpY9ycb46ioun/M2nEq8Dh/ctIu2rHZiIgSTM1mxbvNdikBKc2SyK9DCjOPksKs24dWEAIshVknAhIJJgIkgYUAJBmiJgkszB8kgcmtA+zaCJht0nwKv18QmACWhnCl21075NaD/cK9x74O18OlZEPmgjZ+v+CgPds2DM9mBNs/GHIm2ZAmuy6EBhEghHkthNkv7T4pGIIYUmhIuxTESEhDCo1EaCSkzDahkZBGRygIMHqyDwFGIhQ6ZP4EsX/dEX1IuPU+OqQgSfv1LikIaHSD/V24vrR9r0YHjA4BEkCHCB0S6EBCEuE73ntmqEG2RdxOATgWrB8FcDpswMwfA/AxAFihvbXvSCQdkeJmbmRVfXgCNAKo6HfeAirH3sKN3pI2AM18B/n3MxcQ09x3kP9OxvmOBtGazYLsBVlhayZf4xxjK4jTNOzVYat+k1uLSpsFJrFbc864qWttBVhrYHMT+lZN0tbpgrodM0FZXwf3t5i0RYyDdm02IqJ5jG6zwtosmygAFzREUoKZQazNfEACzATSbg4jDFHRDIY25E1rkBBgBRA0GAKQAAXPH03f9gUToNiQNSYQM6AseWPzR5rNnESbQ0IzHD9lAVAwpWJh32Pf66eA9rZAvo/0vWZJhqBp27Ow49TWL+a8aGweShET2Hqv/OcIxsl2nCQIrAUgNLQmf3AhzCklJijtTiVZHkjQADQRiAmaCX1IJFDQTNAgbGqJjlB+2dfSi8k6UNiEhCSNTZ0Aog+whCANAQYYENCQEFD2MyhoCBaQYGy4zwSEJ8ieQGMnEoxNUgDLykekbRG3zwG4n4heBeAVAD8M4G9P1KNWYCaIpSXo27e37unjqMfZiqf6VWNq+dgkzWfkzdHCmUY5xkTr46F5mwUAZoheB3p9fVs8MS+E82ZPc+K7JV6L7eNtawit2SygIXq9Zu22oh917nz9voggVnaAFhfRP/XKhAObAG3b7fxJAtqx2YiI9jCGzRrPGitlyJtmkJnSpyQuCP9jwBCsTA/CkDe37sgbMdiSPmgYL55m37dpnDIFBtlj+SMjDU+073DkjQKS5MiaNm3ZBfNR5p1g151w4wk+kz2WI2R+BJxva/YQWfImKB1PcGx24xYEaENydfoBDHnTAlJoQ95gCB60gHDn245TEKdjFjAETIf7NQRrQ+ogDXljiQ4pT9422TiSdM4pq9y6/cAy/PCApd8paQMBm2B0GIa8VaAV4sbMfSL6ewD+GMa8/g0zf72Bjms/jZ0KXKjZfE0OBzCPT7Zbs1kAem2tiW7agbVZ7s/NxLAYzDB3u/lBmzYL5vbstoGwv+TwIfDqKvqnzzY0qG2K+SFsAFq22YiIFjCOzRKRCY+EjViwjoeAOoGJMtN9JpGd/jODWKTeF80gKcB9mIfnbEIkiWFCI4O+03+hBCUgb5n9lokJ24OwfbLnhW5AlgRi4Fj+deBV823CYwl4sgY24ZdOE+a8cISAvPkPz3Z/ug5h3ik0PHljZkBqsBaQnh0a8saAIXtunO7aWkbkrGhPk0AijGcOABQJdEj5dprtegAZ5n0k08a9BgPKuB3RgQZIQ7NxZyowOtavOgxtedzAzH8I4A/b6n/bgQit6oMiWke02YhZw8zZrItIGJdECwm5awV867bxBs75Q7J5xMzZbMQdj3FslqS0hMaQNgCDYZNKpVxEwoZDOhgWYbxQNmySGcShx82SnaKwSQ04psDShi7aEZl1NuRRBySJYZI82XUKvGK+95QZmt5cmCSy5M29n8VgmKVPYWDDJn1qFetx8+SN7Gd0RxNklDGENGzSfEHQmmxkKfmwSUPUDMNiYu99AyxRCyJUEyj0tYAQ7MMlNQtoYh82ucnSh00CrmsTNhlC5+Q9yq67sMkuA4oYwon+SGMzCJusQmsFuGcKQhoR/qSYxwmwkNVtIrYeRJN/N+EVdJ4wn3q2uYDcvx/yvnvKGxCBkoLniS7JjyCg24G6eQu8vt7SKCMiUiT3noD+jjdOexgRswQCwkQkPjeD1bz5DLjMxvPDGlDaRL8wA1rbP/Oa7BLKbDe6NTbaOGXcY6SMNo4YZh+7P4BsGxP+aImeI2ZsSBwZ549pH6y7dhS0d2GUzqnlXxe0JW3baErb62x7TxoZxpPIAHHgcXPbtVsSmMk+dzaaN3PaCFoLaLtfsV1qo2tzf0oLr21z2/os7dLs29QSfRbY1BIahL6W2GTzp5nMEoRNnUCB7J8wfyygmKDZvNYQUCBssIRmgQ24vsz2TRbYBGGTXSqEKXncZglyxzJoeQn9M0PCbqrCe+ZxAgxA7lqBvnmrPT1bxFigbhei14O6fr28UVVm0zn1VojFRei16I3Zjrj+nffi3LcK3Pvh5wr3U9KBWNkBdelydoe9vnK/P5oWbpbQZibiiLHxzAfvwkd/4OP45fsfmPZQImYGlCYiQRCk6MiaUjYZxwiaNw5S/gPWEycyCUtMbKNJQ+89YKYxHAuqpXnLrbN1XDnvmg+btG3DY9XVvGVDK8MQTrSveQMyYZOjaN6ct81p3jLtLCbVvFWxidkibi2kRE7uPQG+dRv9cxeKG4xbDmCbgTpdkBS1dStiaQliz270z5xr9bNTkiCT6ESYp1Ol6/YpltfZEZlU+3cQr5T79oLX1qFu3ChuMExruU3SitfCiL896nTNpP/yldn5jHcQnvunj2LXM8D9Hz2Jom9UrqwAgPn+cqAk8ZOeuYO9hsWHY9sPd39mJ579rMa/+O73AHh+2sOJmBUQjKeNdY6cCetxG0PzJiic65vrBuDJW6Oat1BLBtOndqsi3TeJ5s2TN9d+CzVvbEsEjKN565C22jayX4tnX41p3maHuNWZpI17085l5pKvfQCbe5cg/uoJ8LXr4GFP55tJNT9VuNCjuklF5O5dgJTgGzdbJ6ysFMLiiqZmFIas54rs5tfnFMk9x6B3LkN/7Wnw6hp4s1/+eyiz2Vkibf43W+/3J5aWjM2urs3OZ5wWtjBT4dr3vhXX70lw8Jcew6HHNXa8dBvqQu4hmbVL7UIf8+MScstKEmw5gvqkEdsD137k7bj2aoHj//gxfO53H8HRp/tQJyNpixgFZOq0uSyQ7oEzrKdtu2jeiH2pgCrNW5jFPtN7meaN4EsFsGtaonkDpwRvVM0b3K1hFM2bsK8LNG+CuLRUQFOaN1A/Uyogr3nj4FsvwrYhbiTsF88Fk8tJJ5yu7hg05P79uH1iF9b2Suz6axoMycljDiYLlCTQG5u1SBh1uqDlZfDGJtT1inMz8kAKvsd8tr+qdbctxLS8oW1OgAOdlty1gv5de7C50kXnazDlMIZhHmw2SKVc2TZJQIsLgFLlXsiIFMH1sDVbIYK8/16ce0uCjRNrOAhg+bceH3ySGHjQCzVrbv+ceqNI3hmZiGcF4vUP4dw7NB64/xQA4MhHHpvyiCJmEgRznZXpKisVJCxBtlQAsSF1teq8aZSWCsjVeWOyHjjFxXXeNIFESN4cOQuCH8vqvBEGSweEZCwkevk6b7Bhk2V13igdS1WdNx826cbpwiYL6ryZSNLBOm++PRH6WiARGn1I70VLoLHp6rpp+DpvUui0VECuzpsAQXjWmtZ583cy+zm6wECdt6q78rZJTuKeOIqlpYKdE0wuXNgYa1C3i5d/4n4kqwq7/t3jEIuLmQlyBkTl+2YFRKBO14RH1pkYCAnxqmPQV64OPhVvAq649qyf1xAkIBYWGu4zOD8kcPl9rwGtK3T+5POgXq/8PUIOP7ezQuiEDYWtM14iyMOHoG/egrp6rf2xzQO0AtjUaWsLlHSw99cuoXcZuP/HvliccASGuFC3W7ofzHNL2sTCAri/OZflVWYSQuJnfucT6J1LgHedmvZoImYaBJICkNJ43lyCkvzS6lq5vwkoZebBSoGVBpQyyUiUAmu7HiYosUu/bhOV+IQltg0pm9zEJS9xiUfsuonOM166TJKSzOvBdeT2oWRfmIxk8P1kPH9hG7beQE1BW/Lt0wQl6Xa27VnnEpZoyiQsUVqYxCRaQLl1m8REaYG+NslE+vZ1n6VdCp+oxC03WfhkJQN/OgnWE7/cyC03IbHGHWyyzCQsmZlQSbG4CKCGN2FUaAWxtAS65whOv/sAjv32GeDSVXCvB726WhiWAwBzkSZ9hEmPXFkB7dkF9eIp8EZLEyVPomf8vFpQ0gEEpSFeTcES3OTwIVx9xwns+9TL0NeuA51usVcizC45rXPbZBhmTe+DWFoCLS9DnT0XJ78jwGXQbdxuYb37b30Yqz97HWf/wR4cfeqb6AtZ+P3IlRUwM3h19c76/oQEdZJmi5vnMUth0dsA6p3fgvf80l/gZz/0d3Dfl17AHWSNEW3Aadx01nsyqHljsJ0TZcImgWY0b8QIB9CU5g3u+XtmQM1o3mwUqU2wkoZNNqF5I3IJTzC0VMA4mjdpE5cMaN5yNKysVACCUz4boZJEPkVq0zcb6vVAJ45i49AO7HqhDz530WjagOKQTGCuyEUdiJ07gU4CvnW73RTb83ReCebzmNytDfdNSI4fhdq7gqXTa9AXL5VPbEMP2zT1mFv8vYqlJeOlWV+/syb9k8LabSuaMSEh9u2FEoRznzuM+554Gv2rV4ubLiwET5fvoDBBq2lD24lW5uU6uwW48cNvx8U3En7lsf8Sr/n0SfSr5BMREXUgDAkgZrALmyzSvDlZAFBP8+bv+UYsRprLNW+KfdikQ13Nm9O4bWvNm4DV9SENk2xI85YIDXKlAhrWvLmwSWXPl4LOaN5mxuPGGxut3GzE4gLWjq5A9QQWfv+zwMKCmbgUTPZG0dXMDYSE2LUCXluDunipvePM3RNgas9WSGDz7r1QiwmSP/8CuNM1E+0CTxRJWbpv6mhLA0gEWl4G1teHl0OIKEB7dktSArt2Irm6hhP/2xNQZb95IUHLS9DXb5oQobm6LgyH07TFhw3bB2f+Rh/3Hj+P5N0vFWY7jYgYHWRIhFUjGU0bmtG8aRrwx4ykedOckrchmjfnJyvVvBENatwa0rw55xnDOg0Dkuc1bzokb/AlA6o0b0QEbXnuUM0b0nGOo3nT5EhesebNIV8qoItZ8bixo/LNQ129hs6ffB4du16YDp8IILF9b6YtkR5KEsjjR6FeOdN+Mdt5m5y1aLPQCvTpL/sfZ2G46x1qsxASyaEDUBcubt/Pvp3Rot3y5gbUM9/MHSsL6vVA3W51UqgqzOCDIOr1WntAGTE+HvipL0x7CBHzBoJhBkHGbAJ8Nsl0PVfnzT6EZdb+oazZrtMi3q79qHXehM02KYJYxsQuRRA2yYZADYZKBuTNrdu+XLmANADIEihGGh5p32JS/iMlXJnQyuBYImgDsvXhhoRNKtseAXkLx2nrvCkICGFYJtsBi6BUAIQOQint+BWZ8gF2PJoIiYvBFIBWhEQoQHfMEqZ+3EA9OAA6l1qkqFRA1S1iexC3aaMoc+F2AonGxyeWl0FLS9Cnz7anaYtoD9vcZjO19hqCWFgALS5CXbwcSduMgjc2TCmLSdAmaWujbxcWFUnb9kT8TiIaB4GlsJ6qYOuomjcZeObCsElgYs0bC5jQzbKwySrNmwtPHKJ5Kw2bdFvLNG8En22yTPNWGjZpeyrVvNmwyVE0b65UAOyYEyj0LTHLeNxkGjbZgcKmHk/zNhset4hitJR9kXo98zSo369dkDsiohaszbJudjJEna6p09bvz22WwTsC2/yBQxukDSTmtw5dRETEIKzHjWFS95vffkOaN+ddKiwVMKHmjYKwSSbj1SvTvPmQx3LNmw+btJ61jIbNbS/SvOk0bLJM8+bDJrUlX9qSN063F2regrDJ2po3WyqgSvPmlo68ASjVvAnSxhvHg5q3KkTitp1Bwhha056LpSXw+nqseRXRPNqy2eVF8No69K1bjfYb0SCGeavy+8bxbLVdOLwVb5vAXCVlioiIqAeilLxxkI4QTtOGYs2bZu9p85o3M6W35E0bomXJVxDAmB66TPNGYbFuM8aM5s3HO6YhiZNq3oCU2GUyTAITa95MHwF582GR7D9HG5q3gQyTYThkTc3bpk1YonPfngJFj9tMQ6tWEgWqK1ea7zQiAmjPZmONtu2PYeQkv28c0tY2+Wmj/+2YNCgiIqJdEAGJNHXW4EIAKeOOGqp5c542r3kjv16oedMMksH1ixmUqwdGROA+QNJq33y6+5zmLRMWiQHNG7ElS46CeWdiDc1bRs+Gas0bU9B30KaO5o2zx86Pk7WACDRvzAwWbE9JheatolRAXvOmmaBQX/OWJ3N5ROI2CWZQIB8REREREREREdEemAgk0km5D5sMUKh5s6TMkBvlyd1QzZsrFeB7MixiQPNGlMk2SWNo3sx6oH9zYZOB5s170ibVvAX9eM2bfU+l5o2Q0bwZspaOhQQZ/hSUCmBmaG5H8yaYa2veosdthiAWFgApmw8HiwQzoiW0ZrNzVqw9YkK0UnYjXhcjIiJagrQkSQgboufCJgmZUgF5zZsjZflSAWhA86Z1aakAhyY0b56clWne7HvH0rwBGYI3oHljthk2OXUc1tS8OeZVpnmjLdK8oeK2dOcSt7Kbdl0dhZCjh8FUTBTEgf3ghS7w7HOj9Vt12G7XZHJz440TltnDMLus+32O+r3XaN+WzYpuxyQiidkjZxs2OUfhtbLutbaN65UthB3tKyIiomkwASxsKn4FkBT+GpbSmgY1b0HZgdY1bxR428bVvFnd2YDmjYaETdbVvLmQzirNm4vVHEHzplxo64iaN0EagrUhdTU0b/PncbOEiXo9iMWF8bUv4URASPMD2BySrjk/eahD2vLvGTL5kA8/CD53EerUK9X9joiBGm2RtG0tnB0Mm8RWIf+dCQnqJOa7rWuzo37vVaTtdQ+Bz11qxWZjttNthPD66FCXTOWySFKSgJLEfL917bHh6xV1uuD+Jrgfr4MRERFtgIzHzYYjch+ZumuMAs0bEVinE3YSZEgZUK15o2xR7krNm2aQFF7zBjbjBeB1ap5Q5TVvnsjV07w5YlVL8xYSuaJSAXnNm+9vDM1bMJYyzRuk8bBNqnkTlvBpEl7zBgCKBDqk0rBJNutVdyZRsb8URHSMiP4zET1FRF8nog/Z7XuJ6E+J6Fm73DPuMXIH9OFTYmkJ4r57sPbW+yfvttO1T143y48bTlLcOOqMl0RoqYUQS0tIXnUPcP4y9I2bjU1S3AQpIsWW26zp3C+T40fBb3vd5F12unYiXeEtCG22bmmJGm2dzdKFy9DXrzdrs51uI33NE6Zit+7YSQJ5YB/Eq45ld4zxnVOvB5CA3ii51k6CmjZOSWKu9bP48Kql8jBtYJo2CwA3fvjteObfvGWmzlnEdNGozRKM7UkyRZ7dUghAChM2KbLrEMKEVUpptpEwnjpbKgDSRAn4kjtKAaxTT5xSgFKA0na7NutaG4+T1sYzpq0EQWkb+pgyK9IMUia8krTZR+zWAbJtybdFGi5pl2S9XxRsd2GP2b+gXw1zfA5e2/Xw/c5z53iU3+/7oKB/stttGCeb/fDrhigzW+0b29dsSgQwE7Q2oZCKCVoLKE1QWpg/Nq/7WqDPIngtTfikFtBswiY1CJpN0pG+llAQ2LTtNlmaP51AwaxXedzGJm4A+gD+Z2Z+DYC3A/jviei1AD4M4JPMfD+AT9r1RkH3HMHGoR1QvUmGb+Ez6tS8kVO9Y4YZf4Y3JHC3A3Xp8qBnbFwIaUljA+dnvjA1m00OH4LauwK12ACZFlapW+a5G4Wo5d9Xsx0vdKGbtlkp099jRIip2a1+68PYvP9ucLczcV/k7KvpTIujhFzWaTfO8SNByGNqNqve+S24+EbCvcfPN911xHyjUZtlSZ68ZUic+3Pkjcjc9xx5cw/9PXmTWfLm5paOrGlOyRuzmRs48saWqJWRN52SN9LakzhP3hgpeWNLyhzRY+NFTMkbSsmbJ2c5AudJWY6MOUJGIZnLkDpK2+tse68RY9iYVeupc5d9t127ZUrezFJ48qa18CROWWKnLJlzf8oSNEPOrObNkTdOyVufhSdxfS09YXPkzYVN1ikHMPbMnpnPMPMX7esbAJ4CcATA9wH4uG32cQDfP+4xcgcEWIO6XZx+9wFs7kiw8Pufnbzb9fXhOgfmwRDJYRMP55FzIXEVkwR96xbUM98cbTJT5RFZ6IGVar9QcR3P4zbCltusAwlcfccJqOUOkj//wsTdjWyz+fXCMdobQ422+tYtqG8816g+SCwuAEo1RwTnCFOxWyJQp4u1n72Ol967CP21pyfuUq+tNX9NcpOcmmGbjWvaXDRG2x48IWfKSzi1a62QeM8v/QU29/aRvPulmTpnEdNFozZLJnyQJQVLYcib87I5z5vzuPmlNJ4253lzZM3dowMyF3ra0qVOPW9KgXXgeVOGxJFd+nVH3jIeOPPnPW8663kzBDD1vIUELft6kLx5smb3ISRzRa/zhM+RN+9VC8kb5drmPW92adfZeuFY58ibpkHypoUnb87rpgPPm2LrdbOeN+eNc6TNLTcdoSshb1VXrUZi6YjoBIA3AXgcwCFmPgOYHwIRHWziGAAg9+/Hyz9xP4799hnwuYvAwsLWaGGIQEmnetJhJxEkaHjblrVH+vbt0fobFW09Od9CbJnN7lrB5fe9Bvs+9TL0xUvgTrd9Qg34i3ytiaol4CNNahv+7mNh7XrYMru9/17s/bVLOPsP9uC+J56G2sqERnUTP02iGW0KbZ+TeK2tDfH6h/Azv/MJ/OyH/g5e8+mTmN0zFjFtNGGz7KJHyHqHiI3mzaXiz2vedPZaMlAqIK95I2E9boHmDSjWvJHI+nBE2h5AqeaN2HoOg76rNG/Z/TktGVLNGzMC19GYmjfbiJ2Pyh2LbdKUSTRvsBUAzAnzmjfWAjLQvDExZFAqYBTNW4e01bZlNW9hfb8iTEzciGgHgN8C8D8y83WqGTJCRB8A8AEAWMBSZXv52gdw+8QuHPrcGnDpKnhtHa1U+s3DZR9TdSYS5osb2tY/IZ7R20pb4UZbiK2y2eSeY+jftQe7v34D+tp1kyVRV5y3JibIjojVslmq/h0JCdHtxIQhU8ZW2e3a974V596S4JXfOISjT30T/atX64UhTmq3/tpS47o+SttZRbzW1rbZaz/ydpx7h8YHf/2DuO9LL6B/6fIEo464k9GEzfZ6u21yEkPIWJIJKZRIyZsrFWBRWOcNMHMGVyqA2JAyKcFQJu19WOcNCLJN1qjzRgRfKsCGQ6ZZJi3ZKSsVUFHnjW2GSk+SGCasM1inzOU7JH22t5I6b+69LFJPmyst4KX9Ghi1zhscoXSlAgAgqPPmlkWlAorqvAni0lIBmgU0pXXeNln6UgFVV/yJRFBE1IEx8H/HzL9tN58jorvs/rsAFAaaM/PHmPktzPyWDnqVx9rcu4S1vRLyU18Cr66aPqo8BA3oDnw8cdUTz3Ai4TIIVvWZf++sYLYnEltms3rnMjZXuuAvfd08aAC25Mk5CTIi5lolAuwlYEhbEgTqxqQh08RW2u31exJsnFjD4Z9/DP3zF2sOsEE97SjZJuvq2mYV8Vpby2avvVrggftP456ffQz9V0439Aki7jQ0ZrPd5TQpiUjDJcfSvFmtW2HYZJOat3zYZE7z5sIbx9K8ZUIeCzRvQftpat582GQNzZuuoXlzCUrCsMkyzZsPl+QWywGQeQzxrwE8xcz/PNj1HwD8GICP2OXvjXuMEOKvnsCuvyaIxUXo1dV6N7QGbnrDwtqo10vro4UhO8OePpMwadzDfmfp5jxLY81hq21Wf+1pdL5m7aSudqsJmx32QCMsbF0zJI01gze2ILwzohBbbbcHf+kxHARsBsYSWxqnPEoVhj08SBI/Gant3asb4r5dEa+1tarEUM8AACAASURBVHH8Hz/WRDcRdzCatlnnWcqEJALGWyUNQfBhk4D/vadhgum7CMiEz5l1Tj1sQCa7pClAnR57oM5b7tri2gfnIhPWCWEImSlvEIzRNPafsKjOm/1w6TsK1plS75oPm3T7gmNV1nkjH/iYuqW07USkYwTGqPNG7MMmQ09bVamA+nXeJATCMZZjklDJbwfwowC+SkRP2G3/K4xxf4KIfgrASwDeX6u3ITfj5N4T4GvXoS5drk/awq47XROzWmfyXCc8hQii14MOa2iFE5fce8XyMs7++Btw9386DfXSKfMZqsbQ4E2bOl0fC32Ho1mbHQK5by94dQ369u2xEm6MbLN17CVP1IbYg1hYwI3veQN2PX4K6uy56jDJcQrSD0Fmoh7R/LUWKDy3z/3TR3HocY3l33q8OgFOUde9Hoio0bDaAQJZcW2Wu3eb391mf8tJW2PX2q3UFLaDLbvW3v2Znfjc7z6CIx+J5C1iIjRnswRbeJrcqpmSkw0/NOxpMs0b6xw5a1DzRm7djM8wJjbHbFjzBgre0rDmzXFdQzbTsMlGNW9AGjZpNW+YUPNWdekfm7gx818B2e8+wLvG6HBwm5CQO5bBt26noWb5dlU3OOdhqHMfrSJtQQYx7vdr3ViTV92D2w8exK7nNs3nqJqMNh3aI6Sv93Gno3GbLQKZsEJeWy+vs1bHZgGTCaoOaoeW1bOB5PAhqGMHsXR6Dby6WlOX16B9EVUf8w5C63ZLBLl/P65/573Y9Qyw46XbhTH21OsZz2sZaUsSE6ZTGaE/GuraAnW6oIWeuTZP45rnvdkNHHe2SduWXGuTe0/gmQ/ehWc/q3H06YazhUbccWjaZlmSCTUkS04kwIrN7XIUzVuQxYOAZjVvfr5pPEqkuVzzpsxxa2veiAFlNG+ppi0gSQyvefN6tbqaN7Jta2jewCnBG6p5E7C6PjOuJjVvidAgVyqgjuatAtu6QjNJCVpeQv/chfInmBWJPkxmvZoFVysylDnjH0grXTIRl7t3Ye3e/bjyQAeHf/XzUHWyUgLN3bTdD3pWQ4VmESQgej2oGzeGPACosNlOkobgVh6vggSG+2vYlVhehrprP27es4Qdv/8E1JCJuu+/4cx+4e8sogWE59V9f7t34ty3Ctz/0ZNQFy4MvkdI80Bis19qu5Qkprh2E7YQ2m0Ygp4ffzi+hR5oaRHqwqWtjy4gMr/bqt9LRGPYuHs3PvoDH8e/+O73QJ18ftrDiYhIQTDEQALeR6QteQubKZgQPqYsmXKaN2gQB64ZACQDMmUPxUpZ8paugxms2ZA72P1svGZwpFHTAFMlmOP6dSFsYhMbNmnJG9tsmazYkDeG+RzKkDdoAglD3tw5MOQsCH4UMCTJkTNLtFz0YWY9JGMh0cut+7BJweYZGgX+PkvEyPdNlqDah8X+VJN9CGk9j26cLmxSaGhNJrrE8lwNgJig7JjIej3zYZOJ0OhD+q80gcamTVACDXSEajVUsnXw5gb6Z84W76yZ7rwWaamafJZpJSpq+Jz6qYex8qLCoV94rPoZdKg9agrMkbRtNbSCun69eF9dm60bWjkKaav5nmvvewTLZ9ex/JuPV9qsD2dseJLceK2tiHIwA6ygnn0O9374ucLABLGwAFpeggoz9RXYUWPhkQVhtwMat3D8AJK7D4PX1qDOTanoMnOsQbjFEH/1BH75/gcARNIWsf0Qhkp68saZoEJDfhSsd4qb1bxptmGTOkvqgMY0b2DyYYROEgaRfsJxNG82IWNwmbcEyjoew2OZlP8YqnmD86Z5DR1ZrVw6RrDzqFlypmz7Cs2bgoAQhmWyHXCV5o0VmfIBdjyaCImLwRSAVtlzX4RtTdxKUTfdeR2dQFWdHHcsS4DE0hJAZOpOlfQt9+/DKz/6IO7+yxsQL52rjtIcRx8UFr/Ov9eG6zWmNZmDtNRTh7OjJsoB1C36a/dTYn7mw8J75coKrr3nNdj9xEXg/KVaNjsywbLngAQNvldIU4RbCOgbN0brN6I1yJUVsFLQ129md4wasj4K3PXMXeO0Mr8bn+dZwIUjUpJAHj4EvnkTerXFchVDHu7J3bsAKbPEdlwEn3lbYPa1dhEFePYX3waxdwP3/ciXpj2UuQQTTAgg+1QZMEzDkhOkHiCX6TATNtmI5o3B1hmQCZsEGtO8hQNgYCBs0r0X4cgo3J/TkoWat8yA0pBK37KkVMBYmjfrYRsIm3THrqF5I3IJTzBU8+ZKBdi3IYFCX4usx63tOm5TQ5PeqWH95PUKzi9aBCIkR49g8/h+LF7QEC+chbpYI532qJoIsuneUUwEXOp2Vg1oLWY9nfZ2QHgOqyZkLWRLZRufXQZ56CD40F70ring3AWoayUew0lQYbNioWd+V3V1fRHtwnmHmQGlTLj5Vg/B2wsGr5E2O6/o9QCloFfX2st86khbAcTOnQAJ8EYD5ydeayNaBiUJTn/orWDZR/LNhWkPZ67BEoCyHicXNlmgeYO9zo2leWMGkyjXvDnZAbD1mjdKwyab0rx58kbIaths+OMkmjdP3igIm3TEuknNm9DQREM1b1VosPjOFsElQmhqglvHaxG00evrpVkhxeIi1u4/hKv3L2LPb3/FkLYWJuKubodzhQ/s73bNZKuBMMmwRkjEmPC10qaUIIZ1qXdaLCxAHz+EW69aQfcvvgp19Vq973pUj4ArTl9ms0uL4I0N6Nu3R+s3onnYGz91u+DV1Wz23DK0dX0oqTNIgiAWF4ym7fKV9rRlIWkrsFuxshO8vt6Il5iSTulxpoZ43Z8rUK+H//2Dv4Z9X5C45x99etrDmWMYcsYSto4bLHkzr1lQeZ0327a0zpsQ8HXepBytzpu2D+LK6rz5Gm9BXbeyOm92CWZDbBwRc3XedK7Om4av8+Zqubn3DNR5c0TMETy7Tv44CGq4AWFdt6o6b6FmLlMfTgOunhtpsm3t0q5X1Xljplp13vpaQLFZmvpu2TpvM61xK8U0bybhsZ0uDYa03X7X67B4+hb2feKp4RPQCcNPqkLUGgk1q6nHiqiBprWLIx8/OHZge5QkWHvnI1g4fQNLf/TMcI3OpOn+tRrKW9XFS+P3HdEsguRL0/z9h8fOl4cQ+/YCG5tQV662a7dWA5hBEDreSLFnIU1ik6iRi2gZ+tYt/MprX4d9/c9MeyjzDRsq6d0/ztc1quaNGNy3njCt03cSZdxRo2neyK+bpU7Jndsucw/KijRvfYAkZcM6XWQPw7ALdtvYfUCTjMWOjNiepyLNm/NEclbzRmx8jwOlAhjDNW9MXrdWqXmrKhXgnZ42bNLWcxtL85YrFaApe66LsH2I27DkHOG+cSbAeY2WkOXZFusmCclNCPTqKpY/+wKwugZV4JHzE49wHFuFUYmi02NF0jYcw87rpDab71/IYm3YuP3BfL+LX3gBvFYSZub0Nqy3lxcgYmyQlEgOHQZrXZjIQ66sQK+vg9fXm/n9N6CPHajppxX05avlnuS27bbB67fXn8YC9xFbhJiwbGvgMiIioE+VmrcwdNCFL0oqD5sMUKl5g9VOOc2bRJqNsihsMuy8SPNGlMk2SYHmjQVM6GZZqQA7IrOeHSWcvs1u9yGTlPZN2jJel40SqNa8Bf2wRpAtEgOat4FSAYSM5s2QtfTbIEGGmtmwSUBACG29ZxhZ8za/Grdx4bQ2RY//R9EYOE/bwgIgJfStW+UZzQqyoG17xLpvk6HhumaNwHkrlpdBnQTq6rXi1O/2mCRiPbW5gw2vwWZOk2Uz525bhNkkh2XLnSWdWJiwaFbuCxEREbVg9FjsiUgtzVvw/lLNm+agVABZz1QdzZslK8M0by6jojQePNIBo8hr3rQuLRXgzwFQqHkDkw21LNG8+TT/qScur3nzpQJG1bxp4/Vz5C0keJ6q5UsFhK7RYZq3oFSAc6GVad5oiOZtdjxuQ1LxT/TUNOdloKRTmsK8MOV/GexEQuzZDSz0oJ+/VdzOeffyteSazNY1rK/8vjrFn6N3pR6GeduasFkX0mgvqtwvON6odkQEcWAfeHkRuHqttA1JOahHa8tmY+a6rYMg6CtXTVbcACQlxMoOqMtXmvsuqqIoav5GnOfPaTZKQwrLwrvbsq9xynGEu2Pdt4iIuQSTdaYFYYCkLWlxYYK6uM6bC/kD4HVhGfImhdGbCREEGDqm48gZiuu8afaeNqd5I6iAvOnBUgG5z1Za542CxCWWqBmilyNvQUhi7TpvZDVmuTpvyCcsCcbLlBKzceq8uTECLlukI5XsP8ckdd6U86rl6rwJqr4fbB/iVoZJb2q59xcSM3fDHzGEQD5wH/jiZaiz5wonIpQkYF3wdLjpicTQrJg8fD2PSNomx6TnMG+zZSFrY9iRfOjV4AuXoV58ufj9drLd+uQ350GJ2Brw+gb0xuBDJu73m0lnH6Lhh3Hc7wP9fqEt+mttQZmJxq9pde22Yn/UtEVEzC84HyZILp29CwnkbPidYx4J4Ovx2GuNKaIt6mveiEwiDbcOpJo352kr07zlSwXkNW/MIBapV0gzSIp6mjcf3uhiKmm45o2R0ZI5YhVq3jyBCg7lDhOGTVZq3nx/zWvemBksbMRTDc1bDJUEMKwGDwC4mF8Apk6b1kMzqVGvB3nwAPj8Jeibtp5bXsAOZHUZbhyTaJ4i7hzUmRwGSUZMuFW5NlMsLEAcPgg+ezG12SIU1ARszWbj7+DORE63CQx5ODHs/eGmomstsH01bjGyISJifuE9bukG0jYEzxEXZTxOA5o3hg+bnEjzhoBfAEbzpsNi3KmWajTNm2FBvn9hPICtaN5c2CTBlwoIwyNTcEAKMVTzlg2btN+VDbNky6nG1rzZceY1b8ISsrqat6o7zOyVAxgHJTV4zL7cBNmmXS1t3usZcpdIqKtXh2stWpqYUqcbPm4oPnbEHYVhddrEwgJoeQkQAura9amI08XyMjJF4/OINru9sJXfx7Drc120da3t9Zo/F9OydZdWvGw9YuYhlpZw62++DXL/vmkP5Y6HSflvCIIryO0Tb5DxyDG5bbZUALn3UXGpAElgIdJSAUL4Oavpn9JSAUJYT5wtFUBieKkApYaXCjA57+2feZjrSwXYfb5UgNa+VAApWxogLBXAuVIBDF8qwOndnObNlwJw6/n0/8G6C6N0Ti3/OtNHum6WaakA18bv1xSMx5JGVzAdwXZfOiAtFYBcqQCthS8VoEpKBbgyAlWYb+LmJoq6WNMGAKLX85m9AJMuV9++nXkqHN7c5MEDoN0r6D//4tDJAnULyFVDT2rlvj2m/9zY/LGTzmzdkPOTiWET/FnCOBOj8EHCMA1d0C/3+8ZjET58CJsfPgjatYL+cy8Mf8pfdN4bsFnqdEEnjkIsL5nfWpHNFv1eZgmzPPY8hBw9UUmNz09JkrUxrYzt2gcJlCSZa3GtYwpZbreT2q6QkPv3pbZZeK0ttuehYJ6Kt42STuZ7paQDsbiYazRHdnyngQj0qmP4k5/7Ray98cS0R3PHI63X5l7D13RzJK5OnTdQSt5MHTdkyZu0xEwIsF3CLyvqvAVkzmnePHlTCqw0oJQhb0qZOm9KZeu62eVIdd40pyRMOQJoiVlQ583VbhupzltI5ope5wmftuTN124LyNtAW/Lth9Z507k6b5oGyZsWnrwpLaCYTH03LQIvbDHmM1SyZpgM9XrZkMi8lywMZyEymrbzl6CuXq0cQtP6Bep0QQs9ULeD/rnzxZMSmxlu5tL9hiR5XlLPj+txHbEMxdDkM0RG03b2ItS169XHbjgRiVhYAO1aAS0tov/UyeLvlcgUep51vc+sh33a77tUl1v2HqAWSaIkGazLFoT3DqT8r4OSEPUB1C3xYiF37zLdb/bRP32m/Fo7Q3UuqdO151d7gsz9/mC90Vm34zsYL/3DR/GTP/THeP+jP4ju2a9WhltFtAhL1sxrG/bHwc46mjebkKOW5i343foQy3A4ec2bIJOIBDY8koRP2uc1b26kec0bieGlAnKaN7AhoMQwnsOg7wHNG2WOnNtvPV0Fmjf2282+Opo3c56CNuGxQs0b26Qpo2jeKF2HMO8U2mX9F8Z7KTVYC8gCzVvVZXg+iVvNmw9v9rNtBxJ5pCn/xZ7d4IuXh+uDWgL1emasa+vlWchcaumiukazgmkXqm4SbX2OfJmBkuQzYnnZZI+8YG12VDI8yfiJIBYXjc3euAmsrhUf39ns5mxMfuca9vuuTZ5GJPaZ6xLRwHH8etOEaBTSJiTkrhVTw84+cS4jbXATnTqo0lhvAcJMyqxUSpoj5gZH/nINv37tPTj88mPTHsodDwZS8qaRZiOsoXkDl5cKmFjzprmkVIAyae9DzZv9HLU1b0TwpQJsOGSaZdKSnbJSARWaN7YZKv1nZ2Q0bz60Mew9ZYb1NW+2r4zmDciUCjAbRte8mWOlmje3zJQKcJq3CswncauLuh4GKYGFnskeudXEQkjzg2AN3iy/8fuaW6NMDqadhj1//HkhbW2i5jmiTgJeXjTZI7f6QYOUoCSBXl0d6rkZy2anjWn/ZtpAWNy9tQcOxQ8YCtelNNkjJ8UoWUuJQJ0E6Hagr10fbpNOk1f3XDWh4ZsQJOykzD2eruOpjJgpyE99EYc/Ne1RRDj49POWELBPJQ/zV1LnDaK8VEB4xXF13oxHy5IpF9JdVudNBmTKDsMnJAnWneaNSBhSJ6XJSsl6SJ03XV4qIFfnjcl6yxT7EghgKq3zliYT8bQrLRXgyFlYqDtfOiAgY7CXQFfnbVipgLx3jkPXYp06b24cBaUC3MEdeQtLBcxuqGTbqcLzT0ELjiMWF6FXV43u7flb9unt1t3wKElMce+1NXiNRcE482FI1R0HN/ApQiwuQq+tz9bEfRi24sl6RTY6Zwvq6jVTp22LNSvU6Zri3tevD7fZTnewtuEMgLpd4yGcF5sFIHetAN0O1Lnz1Y3HvS6HdsCc/lZCaDthWF+fmCD7UMya35PYsQNU4xyMGtLpQxSnZS/2+5KHDpr6ffnQyIiIiOYRhkq6qZZzw/lUi+G6C7fjdJtPUlFO3qBhdHCa0myOQBpBAed4St9FAMJ082adw5FkyZvNhOj35+u85a6Frr1fJ1siIEPeXObJYIymsT8/Jtwxc2TbMHhHbt15OJ13Dd4vlp7iIALSE7OBUgH2WD7np6+rF4ZwOoJVo86bG6cjb8Q+bNKTNy0ghTbkreL2Mv1HgUVwYvM2JnVO0D5sgk0EsbSE2+96HeTBA+n2Lbz5UpJAHj4EceKY2TCEaI1E2oSc+mSZkgRi507o1dX5mQCHiXDagLPbIf2LhQWsv/tNkAcCm93C75p6PeAND2DtOx9Oj11ms5szVnhYSIiFBROqPC82S4TkrsOAUlAXLtVqP/IDnyHk3YXqkSDIA/tMtly/bwLbIOs9rEvadu40iXO63cq2tb2SQkLu22s+41aFJOYf0hBB7NgBed8JqPMXzfU2Ym7w3Ecexa6/itkjtytcQhIIQEsUJCgpWiefsAR+3e53yUlctkkZJiMJsk2KbIKSwoQldpkmLAkSl0ibnMrONzLZJoOlT1jiEpiUJSxR2meZJK1BfbNf9HU222RBwhKfnMQnLAm2BUlD3LpQdl1l99VJWOJCJrNtyL+G264sUR6asIQQJizhXMISzZRJWKKUgLbJSbQWVbxtcuJGRJKIvkREf2DX9xLRnxLRs3a5Z8QO0ZrWKXzKO+Rmmhw9go23vwaLp28Zfc5Wgwjy+FEgkcDla1t//BZBna55al2m1duKMTRtsyMmPxgZ/kFGuc3KQwfBj9yPhdM3wGtbb7OUJFDf+hrojsTiS/NlsxAy1V3Nic1Spwu5dw94ddUkaKpDcsbxslW8jzoJxI5lYGOz3L7HydpYE9TpQuzdDQBGC9oktM3StlU2kzuO3LsHYnkJuHp9at7txq+1EQCAF/7Jo9Bdxtf/44PTHsrcoTGbtV43dqGRlCNrufV0P6WlAsJsk2S2+VIBPsTSPhwrKhXgyJvIrkMIQ97CUgH2IRtJacI4XXkBYLBUQEjeWBvyFpYK0DpbKoDzWSZhthWVCgjIG9i+5iGlAtyfz1wJhCQsJXacJWb5tpm+7dKFWurgtdsfkDcaIG/IkDf75SFTKoDJnAqbZTIsFRD4IAvRhMftQwCeCtY/DOCTzHw/gE/a9XpwxKq18EhhjNFNsgsmBHL/Pmwe348bx7ugp56HunGjnbGUQUjInTuh9u4ANvtQFy400qfHNAXpRKBux1xIpptFsFGbNTbVYkivQ8kx5MoK+NBe3L57EfyN56Fv3mxnLCWgThfy0EHcuGcBYq0P9eQzW3r8VuG0TxjRs908GrNZShJQtwNaXIS6frOd32JRKGQIZpCUphzLwgL07dvlyT5a0ohRYkgjL/RMhsUGrvViedmEuLO2RH/I9bbFMGbq9UA7loEkgbp4aZre7eautRHGvt70MN733sexdEbg6P8RE5G0gMltlpwnLetd8yGUbj8h/Sup8zasVMBAnTfb1pO3UPNWVOdtWKmAfJ03zeZhVFmdNz2EvOVLBeTqvHktWlmdt4C8OQLmyVvgNSus8xYQsUIyVtI2UyrAtQ+9czmSV1XnzZcK4FypAEvewlIBlcnFKw1wmG0SHQXwPQD+VbD5+wB83L7+OIDvr9VZVfjiJCAyk5XwqXlJ+M4rP/ogrr9qAfs+8eVsPbd8f71eK+OUe3dj9dEHgK8+i/4rpxvpVu7dDbGwYFaGWUTLeii5dw94Y3OqOos2bLa1Cb27gOrhmppr73kN1g/vwNIffdlMwsvatlAfj5IE4p4jeOlvncDe338S/KWvN9NxSY3CrYZYWgJv9qdaYqNRm4V5OCVWdqJ/6pXWrrf+Wj7EbuXhQ6DlJagrV4fbbQugThdizx6oB49BP/eSITcNYP3RhyCPHQGYhxPBNpPcCAl66F7wlWvmO54SmrbbCGDtHa/Ff/yD/wdP/fAJ3P3PImlrGk3ZLCMNk8yQNxsymYZOloVNEsKwydQTl90fkjedmLBITkQ2bLJOnbdM2KQcXLqwyf7m5HXegrBJKPZtSOmhYZO16rwVrCO3b9I6b9k2BWGTKvS82eWIdd6qMGlykp8D8NMAdgbbDjHzGQBg5jNEdLBWT21qg0jYpwXB5CsfVrJ7F0791MO4+y9vQLxwFqpADxAK03mj+YmcfPWroHcsYuGxb0A30L97oqyv3TA/uCq0OJEQ3Q7UlWvbQR/UnM22GR7pQhSGkEKxvIxr73sEu5+4CJy7AFXkOQl1cS2c+/47Xo+1fR0c//Vvot+Ud7rNcOkRxkBJp/zhzdaiMZulTgJ9oyUvG5CS7YpC78ndh8E3b0KvrhVeSzNJQFqwW3noAHhpAeIrJ6EbIOVy315svP4EFp89D33xstlYRM5aTroldu6E2LcH+hvPF18PthbNXWsjcPLn3o67HjqP7/6hn4R44clpD2de0YzNOs8aw6WvMCsaaWZDETQGp16azLp9DZNlMc02yHDJS3wLhkn4wdpkfQwSglTWedM6m1CE9UBCErb35EypAGD0Om/k1s34TJaQ8POnfY9c541yPbhoz8yAJqvz5r4Fl9/FJFhJSwW47KET1XnDcIztcSOi9wE4z8xfGPP9HyCizxPR5zfR4g3GuXOHadpedQ/W3vJqrLyoIF46B3Xx4uCNVcisXqHhG29yzzHjcbtywzypbaB/U0i3P/3sfT4GerqkrXGbneI5TQ4fAj90Astn14Hzl4oLbDsC1AaIQG95HQBg6ZU19JsulTF9sjR98ojJbdb24e12Q902dcra8hJXJC+hThdieQm8tpaStgJy06Y2LDl8yBzm+s3GvP+8to7OpdvQFy9DD9NFt53Nd3MTfGvVZCKeou3OzPxgFiAkTv39bwMLxqVPHwb99RNTjQCYVzRps+r2LTP5t2GQGY1bJlwy8LTl1jNhlU7zRkCjmrcwbNJFSpRp3mQaBTOy5o2d16xE8xaETc6l5g1mv/fAVWje0oyixZjE4/btAP4bIvqvASwAWCGiXwdwjojusk8m7gJQmF+ZmT8G4GMAsEJ7273DDLmBieVl3H7wIK480MGhX3gMhdTCFSls42JJBNHrYeP4fnSfO9dYeKTL0KZvNSy4HwfM2+VGMx82u7AAdewgbt6zhOXffHy4zeYn6E2EaAkJubID59+4Ewc+cwX6a09P1l84NmDqZMmNYcqaNoeJbBYYtFvmKX0uItBCD7S0WJ52v+kC3Pm+kw704X2gU+caC48UO3eCiKC+kvsdFHnbWrZtvbYGTCE5UQFm51q7jSEWFkDHj+BnfuwT+L9+4Ydw8FdieGSLaMxmFw8fY7Y8KDVe53kJtmgEpI5QVOeN3VLaPjBCnTdb0Lu8zpvLhS/ge69b502z8cSV1XnTWU+c6W+EOm+azTGL6rxpW6xcpd620jpvZMMUnQfUEq/SOm92X506b8555j2elsSZvskSTed5s6c6XyrAjTOo81aFsT1uzPz3mfkoM58A8MMA/pyZ/zsA/wHAj9lmPwbg98Y9xlbg7I+/AUzA4V/9fOH+VBtXI9RwDMiDB7D2Xa+D/MzXRyJtlCRDNUvq7Q9j47seaWKIc4OZsNkauq4b3/MG6ERgx+8/UdzAZUHUBfOdBiaO8tUn8MpPPIwD/++XRyNtQg612eTQQSRH7p54fPOEmbDZmpC7TfbG0tIDo2hGx9A/yt27wW96EPy1ZxojbQBw6oOP4Jv/y8PVDbfDA4ktwjzZ7TRx9QffiF/75P+N3/gv3oKDv/rpaQ9nrtGozQbeM4j866zmTctA8+ZT/weeuKY0b34pm9e8bfYzy0Y0b84r1rdLlSsV4LRuVZo3la4LhYyuTSi2Kf4Dz5tdr6N582UHfJsCzVvoeSsqFaAGNW9VaKMA90cAfIKIfgrASwDe38Ix3pIYYgAAIABJREFU6qHGE867/9Np8K3bUCUeIZ/pzPXTsFdAX76KxS8oqBGIodd/FEFIrL/nW7D48g10zpxPvTHbyZux/bB9bLbG97Pr8VPg1VWoITrITPHhhr97PnUGR3+3j/4IdaGqChcn9xwDX78JfeVq7o3teylmFNvHZmtC375tb+I1Q6ZL7NYVmR/5+DduQD79IlRD3jzqdPHCP3wz9n9ZYeWrF7Oe72i3ZZg5u50m9v7nF/D+v/s/YfHCF6M9TQ/j2WyQIN2rtNj7zMyKhtdgpc4v63kr1MCNpnlzmQ69x02SWYqUHDCcpyrFyJo3GXjmrAbO9z+h5o0FTMIPac+Pbc3SEqIqzRvBnM8CzZvz0hmvZnA+nKfNNS3TvFk1yjDNG4i8f43dGHXwvQkKFC12veKn3ghxY+ZPAfiUfX0JwLua6HdilH16NyEgAfXSqXISVNRPwxdP3twY7envkEm4WF4GHTmMZFVBXLqK/qXLDY1y/rBtbbYMQoKE0f6os+eGa4DyuraGbVbfvg39/Iv13zBEs0RJArG0BPSV0V7lEyrEyYrHzNlsCCLzNHaY5rKOHtPZ0hjgfh/qeoEWdAwkhw/h9huPo3OdsPPkdahnvjnYKJI3ADNut1NG/8xZLPzB2cpkBRHNogmb9eF1doJPAml4XUjWLHyInVlLyRsjCJu0nEYCUAQiEwpo1m1YIMESKRt+B+tBChOWOPKmOQibJNvehk0yg8mEL0JZsuZC2ZGSNADZhCWOvAlKwyaZjEcNMP0T21BIbTx9VvPmwzQV+7BJfz4RkDcKwiaZrF7OngevO3Phh/Bhk46g+VDIoHdP3hwZc0lknIbNhb0G4ZCk07BJ3959l46qubBJzSAKyBun210CE2ZYkjkcbXjctjesAJOkNBnWVleHE7xw37RvxMP0H0JCHNyPq2/Yj52/8wX0823cuMMsgxEzA9HtgLpd8MaG0bGUNsx9v9vZZgHQ4iJwcB/633whTnLnFVZbNlTnmrfbouuUC7OddpKjXg9rrz2KF34AeOhDXyz+PTKP7RmMiIiYfbDlQN67ZCf36V1uUPPmNVeh5i3wxE2ieXO10DLkTQoTsihE6l2qq3nT7D1tXvMGFZA3bYiWC1rz/aco1bxRQOIsUcto3gKtmOl7cs3bAHkLxsuUet7G0bwB6ffpyZvXNBZo3ipw5xE3ZoAVWKvqhBnbibQBGJYwgd70EJQGdvzW58HDJjaRtM0kaicd2E6kDRhqs/LAAQCAOvn8Vo4oYqtRJzlR3m6LSNs2yPIJAGc/8GaIPuPBv/eFoaUEImmLiLgz4TNHWg40GCppYCIC0y3EKSehkJwQsmGC5EL7gjBBn4nQMb/ssVy0lkkoIsyyD5OwxHrDTAggIXRHERkNll8HTCISBJ42K81gJr9uljot4u22y2wEG3E2/T0RmXFJQqaUgXDRZjDMJRMWaU+6SM8HsSVLwfkwfDTwRAbn23Vep1QAXFIZ14Yp6DvdB5AlfSnBBFWUCnCeuCG484jbuNgGE4ZhEC+eBZihIjGLcNjmNqsbCluLmDMU2e02IW0AcPcnTbmYMl10RERERFa35giP3zWR5s2FTQb+nWLNG6eeNwpC8oZr3jQol7cwQwABo3mzpMwc3ZA2AMM1b+64vifzIfMEk4gy2SZpDM2bWQ+IrQubJHjNmw+ZzPAkDkgh6mnegn5c2CRbT944mrcqjJ1VcpYglpYglpenNwAiyEMHfYr+NqAuXYa6fKVwX3LkboidOwv3RWxPTN1mhYR84D6jP2sJhZo2C7l7F6jXa+3YES1ijIyPlahL2ogg9+wxWXdbgnr6m1DfeK5wn9yzJ9ptxEgQCwu4/rffjuSuw9MeSkSDcDXYQIEHzr22YZSFdd5E0LaozpsLowzfV1bnTQR13nxmSkJpnTdXKkDYNsPqvNm6bplskwDCum6ZOm+s0zpvpmBZeZ03rX2dN59tsqzOG9v6ba4mHGN4nTefBTKo4+a2B+sujLKsblthnTdNaXudbe+yTXqPGltPnbu1ue01pN53BnE7dAB09K7shKJsciFk+b5xj9/rYfWNxyF2tUieygq7ConrbzsGOhrcFNqYWEU0CnHoAOj43UPT56eNh6fZHxlEkCs78Mr3HII4dKC5fkc4vr7vKOTePZltEdsfrnzK9I7fgbr/aGsPHChJTBhnUWQDEfSrc3YbETEMQoKOH8Hv/Z8fxc03H5/2aCKaQo6IZcibI2MUvA5T/me2hf0MkrdMse6gVIAnb/59hqiBUvJminAjS96kGCwV4MibKxXgyJsja468BWTOad6y5E1Xlwpw5M2tO/IWFuv2hbUdscsW6TZ9oLpUQI7MZciabRuWDih8nSd8rlSAJW9eQ5cnfaWlAuz2iueUcx8qKR9+EHz+MtSly1liU0JymtaAJfccw8bx/ej9+VdGSvlfG0JCPPIA6PlXBjKmJXcdxpXvOoHdf/gk1I0bZuN20D1FDIV43UPgC5ehX36l0h7bSIBAb34Y59+4E3f/yydGSvlf/wCE5OgR6AsXBxI7yN270H/NCYgvfQN9542LNjsToE7X1PWZ0neVHD4EfXgf8MWnGkv5H0IsL+O5D78e9/7GYNF5uXsX+q89AfHFwG4jIirwyk+/DT/945/AT7z1v8XihS9WzdciZggsXfFltwHZsEmrWwvDBEs1b84LU1fzRpwNv7MJOZDAJwsp1bwF128GqjVvgkwiEqBa80bZotwDpQLymjfNICm85s0UE7fvEEEoI8J/aUhiupdz+y05CjRvLpnIOJq3TNgkUK55YxueWUvzVo65JW5iYQHiwH7wuYvQN25WEzKywamNDUBCvvoEGED3uXPoN6iHoCSB3L8PvGMJ6HWB0xehV3MT4IcfxOauBez+4gWom7fSyVScAG9biIUFiEMHwGcvQN+4WU3IiKpLWYwAShJsfucbQMw48Jkrpu5WY52bp3K0uAixvAR99Rr0RvZBRnLkbvDSAjrPnzWT32izswMhp0baqNOFPHzQ3PRPnYNukrQRQe7eDfXgMVx87TKO/9ka6Mz5TJPk6BHwYg+d589l7TYiYgie/fm3A0Lh537+/Thw9jPRbuYJPhwyVTf5dP+OvAXp/z3dsJ4Zn1RjXM2bMlkWveZNpaSgcc2b1Zs1qnkjgi8VoLTXvJksk5TRvNnTVlvzxjZDZV7zxu78TKB5c+93ZRvAyGrenB1Uad4qLgXzGyopJXihC3XpcqmOxqPp4tRCQu7dDb28ANrso//K6Wb6BQAiiB3L0Pv3QO/ZAb3YgbpwIZO1TR46iM09i2BBpr5QTFgyG5AS3O3Ut9nwajAhqNOFOHEM63sTiDU14FGYuH8pIZaWQEuLQKcDfeNGxi7lygp4eREgQv/suTiJmSUQmZo90yBtSQKxexd4sQf0+6PVxKwBsbQEfd8R3LhnCet7COL/+4qJ3rDwdisE+mfORruNqIRYXsbGe78Vdz94Hr1LEgd+9dPRbuYROQ1bbc2bD4+kwpBJvy94b1OaN699I/u6Sc2b5jRski2jKdO8hWGTdTRvjNqat1SP1rzmLXxvPmxyQPPG4Z8ljaHmbQjm1uOmb90Cni0WkGfQNGkjgtyxjNU3vwoLj30DfRei2BBISvDxu9Dfs4jOhVtQTz6TbSAkrrzrXuz53HmoOp8/YmtQI9yvts261OgNEXJKEsiD+/HS99+F47/+TUOcGgZ1u6B9e4DVNfRfPpXbSdh8/b3onjzTyrEjWsaQkg9tgjpdiB3LUPfdBfGVk1BNeohhfhd05DBefM8KdrzCuOujj+UaEDbfcB+6z56OdhtRG/zACfzZv/6X+O4f+kks//Vj1W+ImEmwLwNg/DKhk8jVABtW5814bnLuF8tFzKZgX2bdTf7TMMGyOm/g1OsHDUPeNKXZHAE/b0nDBNPxEOA9a369rM6bUiCmMHjReM7Sj5YJ0zSHogyPMZ44UVDnjW32xsBZlq/zRpkj24bBO3LrhXXeQIOhksBAnTe3w37ztj3nQivDEE74EFmqoeefW+JWGw0+6aIkgTx+FGrvDvQ+9VXojebTRXO/Dzx5ElJK6DBMzn3ZWmH3734Fai3qLLYVmnyi2qAHlXo9qG99Da7es4BjH/ta4w8aHPTt29AvnkJhODIz5ONPot+GBjRiLiF27oTYuxu80AN94emh9dTGBff7UCdfwIlfvgje7KfJvoKHffKzT6LfwnU+Yj7xwj95FO977+P43nf+TYgXnoyatjmFc6Kk+ibr+cEImjdKw+ocARnQvLl9XLQekhQernkjq7Uy7Am+SHdYR831pnPkSussucpr3khYj1ugeXOjq9K8CQo+vRknAQOaN2IYz2HQ98iaN3/ggGg7oizSfWNp3mwjT94CxlekeUP2LAxgfoibc9tOKSyQkgTy8CFAa8gzl1sVqHO/D+SecMv9+wEA6sKFZrVJEe1h2jbb6QKP3A9SjN1P3hhIbtM4cp9TLC8DzNC3b1cXaI6IAGBCxXdALC8B6xvArZZtRyuoq9cym8TiIqA19NpadUhzRITFcx95FLrL+JN//3bc/Wz0tM01XMgigsm8Rj3NG1vPm/Xc5DVumXV3sJqaN7D1vClL3gjWK2Y9VpJMSKFESt7qat40Gw+YMp40tmGTDOtpCzVvsOdmHM2bDYcMNW8gGwrpx4jxNG/Omxlo3nxoY9h7wL4YaFzzdkdp3KhG4bpWIExMLy8vgm+tNqtpqwHq9UA7l0FLC1t63IjJMRWbJTJhYJ0E6/sXkVxbBX/p61s/hm4X1O1s7XEjmsE0yjMImdoMEfSt241r2iphfzvoRLuNGA1v+o5nsHBB4O5/FknbnYCqum2lmreapQLG0bwNKxUwseZtWKmAvOZN8/iat3ypgJzmzYc3jqN5y5cKyGvegvatat4qMPseN+e1AKajs+j1QETQa2tQ3zi55ceHkBCvPgH90mnomzdNSu7ovdjecE+i9HS0QWJxEZQkUNevo/tHn8M0/H1y1wr0jRvm87dQhiOiJTit5hSSKchdK0C3A3XufHXjFsegb96KdhsxMq59xyUcRSRtdwQIgM3q6GRczN73Ax+K55oXad44eA0MlAqAbTOO5g1el5buD3tmIpDSzWjemMORpKUCgOY0b3Z8EIaQGR1cMEbT2H/CcTRvjmAPlApw+4Jjjap5A5DWbxPV99bZJ27Mxrc6BRiS1AezTic0Ww2toE++YJ5KkIikbbvDFaecAmEDTMkB3uxDr65Oz2YBqGs2LJMoTn5nCVOyF7l7F3h9Hfpay+G8FYh2GxERUQfGu8VpcWUgIGbW8zNM8+Yi55AncikBcTqsYo1bfj0lKcRBqYCAyPkWbMmPXZlI88Z6gJwx0KzmzY7PMCY2x2xY8xaGrQYjbETz5szDkM07LFRyq0ELPeP+HXUy03CYEa+v2xpKDdahi5hL0K4VUCeZus16r01Mgx1RE07gPt1BRLuNqAEivPSPvg3qnd8y7ZFETAthqGIYNikQpP5nHxIZhkkWhk3a94d9DrT14Y/Z9cwYXKmAMGySzDYfNumOEYZLCpGGTQobEunCJoUYXirAhlmSlIZ8uVBL2Ou6Upnsk1AqDZtUOhs2yZwtFWDDIV245ECpAM1p2KQLf3RtNDJhk6mmjX3oowubzIZVhn+cDassCJOkTN92af+KwiyrEInbuBDSaC2cRqnujdwZ77D9yRiO0DiZmA1M8TuiThe0tJjaV1M2GxHRJoQEb/bNzbxhjHWtjYgYArG0BPnaB/CTP/THuPCGqDu/I2FJmdej2ddta95SUlesefPLcTRvtm2p5i0kb1I2q3nTAXlzGrdRNG8hectr3gLy5ghYHc2bJ1/DNG8l+rgizZvX5mU8qMXYPnetpuuplR2jTv9Og1QSzkadLuS+PeifOz/SeN0kobTfJIHYsQw+fhfw5MnmwummGBIXsUWoyFAplpdBJ46i/9TJkbwWlCTDtXj2t0LdbsxmGjEehlyfqNeD3L8P/dNnmr2GEZmC8EcOQ518YfqevIi5wc33PoI//oVfxPsf/UEcfjlq2u5MGE8agpBEAptIPk250EcXvpjO18fSvFmSRuS8NsG+XKjkeJo3e1DF7WjeWNsslG67zjwwNlGbubDMmpo352DxY2S3zY19dM2bL80QhFR6TaPIHsvV9BumefM2I/zuUmwf4rYVqHPjF7Iy0x/3N6GuXB2ZtLmnCWWQ+/dB79+D/p5FSCkHUv6PDRKYlg5wrrFdCHFgs2XRsnp1DeKl06NNUIWstFmS0kyA9+0xddqamgBvl3Mb0S7cA4eS6xNvbJjMkQ3bgty9G/q+I3jxPSs48csXB1L+R0SMi51/eRLv+/G/i+7Zr057KBFTBAsGsaNmwUwdnBKOJjVvNs28IwqZ1ly0npKUOpo347pDtlRAU5o3NmGPQ0sFAGNr3sIBmL6RKRXg3pshs5Q9R7U1b4KyeraSUgFD67xVYKJQSSLaTUS/SURPE9FTRPQoEe0loj8lomftck/tDrfRRI31kLEwm9CdpvpzbXYsgRc76Fy41XxY0DTSd29DNGqz28hegQob0wq8ujpGp8M/Iy0uAosLwOpaOWuMmBiNX2u3A4KMwKVgBrdQ4FrdfxTX79uBHa+Mfi2PqIe5tNkaUBcvofNnX4iJwmYQjdosAUyWAPil3T6u5q0kzLIRzRsBwzRv+VIBjWrebBglgGLNWxg22bTmjVFaKmBczVtG21ZSKmCY5q0Kk2rcfh7AHzHzQwDeAOApAB8G8Elmvh/AJ+16NbbRJNi5bisajdapHu65gJBAtwMwQz35TDY0TUyoL7JPMyIANGmz2wQkqJbN8qgPA6p+A0QQy0ugbhf9s+eijbWLubNbT9q2OExRLC/j8ut24OZRgT2/9mnoW7e29Ph3EObPZiPmHc3YrCNUIiRjxZo3lJCxQs2bI18UvC7RvGXeX0fzJraZ5i1D3jTG1ryFJM6TqmLyZvpIyVul5i0gXqHmLUPevJ4tR/RyxC+jeauYSo0dKklEKwC+E8CPm3PNGwA2iOj7ALzTNvs4gE8B+Jlxj7PlKJtE5EO38rG2STKRJk088gBw+iL4woVsv72eqdN28gXw+vp4nccJNYD5tdlh+rOhNjthzb/k6BHoq9egb9wYOK7ctWJSp49re9FmPebVbmsTtgavtZQkeO7Dr8exT65D/uWXczsbsNsIAHNssxFzi8ZtVtgQPeFC4Mz9OK95g84InswLQqqFA8bSvIUaLPJiLPumTJggpR2Oq3kjBvdhPGlap+9MBXfwR6yteSO/bpY5zZtmkMzOb4jFoOatD5CkbFinCM55Av89+U/IZOupmW3EluQWad5sX3nNGzFDo6BUAGO45i04P2WYxON2L4ALAP4tEX2JiP4VES0DOMTMZwDALg9OcIztg7IbuZCjTySEBL35Ycj9+/y3TM+/Ah3qLIggDxyAPHIX+KXTtk5bDHecEHeezRbZjLfZzZG6kwcOgHo9v64vXIS+lU1IIpaXjX7oZvRiNIj5t9u8nRbZLVGqu6zbba+Hc//Dt0G+9gHz3n4f9/7GFXS/+kJKHG2iErlzZ7Tb5jD/Nhsxb2jOZglm4i9gZvSB5w2h5y30sIVeulzYpP/Le94KX1PqlRsIoxwMm8y8tp4330/oeROp582MMwibDJdFYZN2HVKC7DL1vIms5831G4RJDi0VoFQaNqm1KRzuPG8ql23Spe933rXAK1dYKkCnYZOpx60422Q+ZDJ9T0mpgKCPjHeuhsdtEuKWAPgWAL/KzG8CcAsjhD0Q0QeI6PNE9PlNjOlJmjZcTG4N/ZqDWF5Gcs9Rm/4zfZ+6fr3Y+6EU9M2bo4e5RRQh2mxoszW9CpQkEDt3DmzXa2uDXhObxndaBcbnFNFuQz1cTbtNDh/C5re/DqKftXX9taehLl3ONtYazEOyp0aMimizEbOGxmxW3biVkjb3DIowoHkrLBWQ17yVhU0GoY+lmre8Xi6veQvCJf376mje8mGTRZq3MGySaLjmrShssq7mjXm45m1IqYCJNW8lpQLyBC6vYavUvFVgEuJ2CsApZn7crv8mjNGfI6K7AMAuzxe9mZk/xsxvYea3dNArarLtQUnHvKgb8iMk6MhhXH3LYfCX7eShaBIiTFFvdeEC+i++bI5TpZGLqIM7y2YLMjPS/9/e2cbKcV73/X+eZ/aS916RFElZlyIlipRNOZFf4dc4KdAqCmyl+uCgRRClcaLYBpw6ResUQRE7rmsgqYsALfohdvNBbhMJaOHEaJzYUYI4tqzEaWLJejFtvYuS6RdZr7QkkxLFy7vznH54XuaZ2Znd2Xtnd2b2nh+w3J3Z2dlnZw/nzn/P+T9nack+mMJXpFZWQGsXIX322epyXXeSNWfP2hn61CYa0wtVLH7cxrFSFrda2z/WNeOWBks4+8bD+PYvKFz8P+9G+uCJ8te69zLnztmSX4nbpljYmKUkgVpdbXsYQvM0FrN612rItNnMG2fZtHGetzgLVzHZyFY9b1XiLTdxSQ3PGygTb1mWLhJvOsq8+YxbXc+bF3P2wJZ73nzmLU2b97y5zFvwvJkGPG+xmCt7HL92ApsWbsz8FIDvE9Gr3aprADwA4AsAbnDrbgDw+Vo7VHrrk3BM2L9aWZm8HZH1lZVkGHLbeH/QFBfA6+96E84f3INdf3Z35ev02sU4ff1bc2NdmNmpZvn91qDxmPVm3FmhdL0LBBfb+qL9+fWFi2HascMKryliNrn8MtCuC5A+9p3KbfTu3TA/9QbQYClbuSh9sVqOWaCHcevfowZq1y4khw7mV1Z52qYQVN/52Jtx6rUD/NiH7q08f+rdu2H+yRtz5b8St83QeMx2iCc+9DZ88Pg38uc7ofc0GrNk/VcUlUda8TambFJxTsyx8gKKwXqLZZNu2eh43xS9R7at8Rk1L97Cfb5skpM48+bEmc+8Jcrex2WTOiqR9OIt0fmyyUGSv9dOF5g0y7xtDDMRF4u3YZqVTbp1ubLJYWozbkMTMm+U2tLKXObNjIo3Kog3lRbKJlNE2wMqtbdYoE0j3mjCn7qt9nH7twD+DxEtAfg2gPfa8MBniej9AL4H4Odr7WmGU4lTkgCkYM5NLrnwGYmxU6czT+UPosES0p94DZa/fwbqhy9gWFGOo6+6Ehv7VrD3zmeQ1hirsCmai1lgZr/OU2JPXHUmpFHLOwGlYM68WL3RtNOqEyFZuxh8+kU7horPmRw6CF5dxtKjT2I4pWeuD5Aia6ZuPwvT4Ll2xp+lZg8+feEee15+/oWx201TJq527cLj//p1uOibKXY9etqW85aQXHrIxu2JJzCcQbuBtiGtQ2+kFmn2XNsBTnzy7WA9xO/83g3YP7y97eEIzdNYzBJxmIQiTL/BZLM8fsKSQp83MghNtMNkFspdHisOfd7IODFmkE1Y4v6rhyk2nE4Mk2oYhL5j2Wv9eMKoQcZl//yeUgKpkj5vDJBG1rtNuUFrgFPYCUFSWE9beKUBFfJFYbwAkLp9mrgZdxomNOE0tc+n6WifN/++Yc/2Q4b9K2U9b+TG5fvQ1ejzxhog4/dEbpnD8/Y4ukVje7mFssfcb5i+dJay93LfRfx9T2JLwo2ZjwN4S8lT10y1oxAPNf7IbKUx77hfU91+7Zdaw6NTYwyUJMH/lu5QGDz5DIZFb4UjueQANi5cttue+PbEffeSDvT5aixm7c62PJ5KnJ+nludGKfuL0ySRN+14kwTm+Rcq96sv3ANe2QkAth3AotK+aGs2btvGn8O1Bp/fgDl7dvz2dUTg/n3gc+sgIpzfzdh97ymkjzxWvu2Fe8DLNsu2sHG7aOfajqD2nUfy2E7s/19fa3sowgxoMmbtBTlHk45QNsmcu0pnsPN7sSs9RGjDPDKTpBdv7LYzfh9uGyf64OxjbKL1KiePgvALws6LN7bjJOMEhiubREpW4BC55UysePEW9p0C0LCD8WIqbhUAYwWrcvWdgG0ATsq+LkUmyoJ4Q87jRqTAcM8zg9gA7n2ZyWbU4A+MzZoxjL1WMgakVE68Qbtx+ziAE2/sPkdqxRuYXKmlOw7BlxaJN/e9B3Hmvyu/b3ecva+NDbLG6TU9blvNuDUDh38mQslg6tLBiRe/yqZkeeN85S+0mxGM6oJV8MYQ5qWXMPjy3aiUjUrj+X96BBd+4xTShx+d6j16RQcugFvBpfqnYWKMu1pwHg5Hp+PfCmHuWsbw8R+M3W7440cwOPnU4l78oqZwFvJM6FdJgwS8vj46QcgWOP/6Ixg89zLSbz6IIx/7WvW5lgjDq45gcPJpDJ98qrH37xoSt7Phlb/0jbaHIPQAm9liO0Oiu7Zln+Hy17qGMvHm5Jqfhj+IN4bNxuTEmxNc/r2KAg/2NaQK2ygrOthndJxQsBsVxNuYVgFknGjx60wmcsK+DawPzlCW2QLC34Yga6KaQCvOYMVbWHbizHDItAXxxhQ3MLDCMvto2dT/fv9MOZURxBtx9r7GfnlW6EXJMoq+RzgRGx8jyn2qTLwRhe8viDVjt3UNIsIhDvmrGi6Dbgi3GlCSgJKkWliNvIBAS0sTsxA0WAKnafWFcnQhWxulofddCPOjMxNLKpNDB3H67Zfhwr96AGndqahzM6xNUQ6zlWylMD2RuKqF0lBLg4kx7jO5jc80qnSteNIX7oF55aVQ33gYw2l6C8a+J4nDbQcNlgA205XsTkCtrmL9HT+G5RPPwJwaLwT13r0wr7oU6p4p41YQBGFKlDJgozKxRnAlcZF4cxmuyj5vQVQhTFxiFRJC2WTYT1w26d4rvBNZ8WZcKSSArO+Yz/B4cRJ2Gi/HIoWz3nPIxAkSIPxi5ttcEYdyRB7mxVQosYwgIrDJrhNI2Qq4IOJI2eueuM+bHx1RriqRSeWrFBXFRx4wDNIq9HmzGUIrlllHpYyI/+H8sUL03RaXT0ZJAAAgAElEQVTFrnJ7UG6fXiiHUkj3rXuJUfI2VfRCuHkTsDk/hYeG2ZoYJ23mpxYds5+pMSn4xZesaCvOllbYpzl9BrsefA7pmTP134vZ55+nG1/bF8vbSTi6yQGmElc1p9EP+2z6WNb8EcC8fA76iR/ai98+xZ/QKuFc22Ac8Po6lh87BXPqOZiXx//gYc6ehX782enjto9sp3OtIHQQIgDKAF68mRqeN1Mom9ys5y3O8vjtyWcCs2234nkDR2WTrn1AY543w1nZJLEVZVrb8kimvOcN7j3ret6IsrJJtjNJZhk3e/xLyyZreN7YZevCZ2fkPG/B9xbvPVOGmedtAv0QblrZC9ppZ/yqs/2MZhGrmxk0Z84AD26y1K1vmTZSLhe+TZj2IrVuH6lZfZdl+y2JHV5fX+gysxxd+b+zCDR5ro1+GBme/O7o8yU/km2vuN1m51pB6BLEUIphnBCwF+Q03vOmsgK60rJJXxJJo2Ks0vPmBVpUNpltFK0p87xF4yzzvEFVl03mShI343nTXF02GS1vzvNmxSIjU1BVnjcmly1LuZbnLRzHyONW6XnzGdgSz9skeiHcapdHzpI6F3B+CuaqC3a/rszzNMsLxK5ceC7KVNt1WJTPOi52toOoWfTP10VqxJXeu8deBHmPZ1Gosb0gCq0EthuLcv4RhB5CsKWSgLKJrOBDy5dJ5somDeXEW8jUIBJv8ekNdlu49f6MWep5Y8TvVDphyYjnLS7nHON5y0xZ2fMj1X6b9bxFhi+7zPFImvW8EblxxeLNzzwZjdFuHD5hLc9bYTkW1GWet0mlkjWScv2AduyA3ru3sJLKN94Mky7giOwfyzqNsucp2ubM8395DN/7Tz/Z9jB6AQ2W7JTofWSBYvbUX1yJ7/9HidnaEM22T9iEuKLBEsyPTiP90WmYF1/MYrH4uroZ7J5y8jNvwBP/QeJW6A+rX30FTv7eO9oexlxQiqGUgVK2nxspAyIGKbj+boU+b9qlXRRsnzeN6j5vvrzRbZs19Y5uuW0Kj7V/TKPPhVuxgTflH7v+bmE/hT5vCP3gogbcvs+bvy/2eYuWobUtq4z7vMXNurUGJYPRPm+F+9DnzTftjptyx33e0qi/mzGgoX1eDafv85ZbV+jl5vu85fq/Ffu8TYitXmTcJqFWV+0XXzScz/PCcivvtQAXwMklB3DiQ0dBx4G1by3uxVJTqJ077XTofe3XtyAx+8hvHIX6JrB2r8RsLfzESG1MN+/eexZeuT6RHFjDY79+BZL7CfvvX7zeicLikRw6iId+8zBwN3DgnsX/f2srARnWzuUyb4bGe968NyqF87GVtAooeN5GWgVs0fOW29Z+kq173lK249UUyiZDZqvK8xbN4tG45y0kdGbgeSOubhXgs5kuk5qVVcaRU5auHKX3GTdKEtDKip1xclI/oJkNosHMXg9JDqzhxTcfxvuv+zKu+JPnsfznX297SN1GadDyMqBUN8qAtyE+Zj943RdxxWefx/LnJWbrQNqXg7dw8eVd23WqGhaU5MAaXnrTYVx73Z048mc/xI6/urPtIQnCWJJLDuDMWy/FJ677E1x582lc8Nnt0LicoYmdeIvvXQmfz7y5xz7TZu+9pQZRJq2YeYMVXO4GFT32QoyibVS2Pmyfy6Tls2vIvYbCvtktB/+b87yxE5a2z1uUeaMo8+aycrZiw24L7baJPW/K3iPcK5tpIxVl3jT8rN1hGbBVFoatOPJllOyMY6kB2ICNy7IZY0WpMS7bFWXimEPmDamt5qDU1qMSwz7H/gb3nMuYcZx5QxBvIbNWWPYTlhBny5Pod8ZNaejDl8I88VS7F8Db9CLC8+BvH8GvXf0VfOV1qwAeans43YYIydorkJ56bup+hEJzPPjRI/jg1V/GF1+7GxKz9aAdO+xU/m2d7+r4thaohLeMh37rKN7z03+P298wAPBw28MRhIk88LHD+MjVt+CPXn05gAfaHs5c8B43YkJqFKCNzbhN43ljjG8VQC7LA2zd81Z8bzdxRtYuoDnPG5PLvJETR0163gy7VgEmP5EJsHXPm3KZNzeRSpOeNy+w6+aAeivc1K5dUHt2I/3Bk432BRKmgAj6K5cguUPhtve+HcB9bY+o06iVFdDqKtJnTy2096bTEGFw2wHo2xVu/dV3QGK2Bsr+otmqaJtElc9tUXDnWn0H4Y73vRHA/W2PSBDGQ4Tlv70YyR0an/uVawDc2/aI5gcBWjFSA2jXzy1XNgm4DI0K4i0TX5EAilsFsJdm7ureCyYvZoIwc5khjOnzFumzvJBza/zEGVwom0SkCHPLmUihSGDG4wtbMKwgYpMvm4Sa3OfNmLy4YjMiztiV0efKJoHp+7yRX7bjs8oq/vyxwMwdzUjIUXZwR4Qccsu+bDWsHkMvhZvevRsYJOBz5yY22BZmQ3LkME7+8qV4+cQGDt1lwHfJBfA4lCvnxfq6iLaWCDH7yAYO3i0xWwul7TTWbXnKFjyDVoeRc+3dItqEbpPF7HkcvNOA79xGog02I6Zc+WNqAE1srWvTet6i6QwZec+be6PM+wZM53lzQqEpz5ttXA3refO+vFl43pjBpKo9b87bBmB6zxvZiUUqPW8ph1YB2RgxQ89bOf0TbkqD9u4Bv3QW6akftj2aahb4gkPv34cfveUSfPq9n8In3vkvkT56su0hdRsi0OoqsL6O9PTptkezLdH79+GFt0rMTgsNEjsjVxs/NoT2KhNKJBf8XCtxK/QJiVmLdqWSfhIMGBXEG7OduGRinzefejGUiTdU93kDXGKoIMZK+7z5vdfs8xY8bRV93tjfa7cPTNHnzU1uUt3nLVKSfu91+7wZ6zGs7PNm8pk4u7/Jfd5sHan7zEB5nzfjJm5Js2xbZZ83otDXbVLFZK+EGw2WoI5ehvS7j88/01bWe20cC3ohAQCP/P7luObYt/C7V74NPNyeJ+W6UJJAH1hD+tTTkmlrkROfPIx3HTsuMTsFaudOmPX1+Z7LlAYNEnt+F08bHvnkYbzz2DclboXe8OinLsO1x+7Z1jFr591wBYPKC47Ze95gojLKyEfVhOfNe7DICZbwopwHLhZ9UclnPKjCe4143ojBQyfeXOPsrCQxJy+n97wZ97wx2YQmbjvShbLMCZ63UNapomOeZN9T9r2Rm3CmWCoZiTe/HO9rDL0RbvrCPaDVVZjH5+xp87/6SkNTUJLg6T99FZbuXMZjn/5x6OE9bQ+p06iVFdDyTvG0tQglCc7cchjJ11bw0KdfIzFbB+dpm7doo8T+OZrq/L6goo0GSzh7yyEMvraCRz59lcSt0HlosIT0r9eg/mEFD9z4um0dswSGVgaKyZVCWvE2c89bvroyNPRuxPM2UkY5WjaJzXje4tJBn3HTVFI2ia173nSUmSsrm4x3PsHzZhtzZ4qYFWzpZlXZ5CTPG8Eez4n5tp4IN7WyYntend+Y75T/vva1jZ5FXUNpqF278MJzqzh8fAj9t9v3pFwHGizZxpFpKj7MtiACLS3hqVN7cOj4EPo2idmJELXjaXM/kLHhhRVjU8EGT5y6EAclboW+wAbffXYv1o4biVkAitjpG3YVdLwpz1vI4NTxvKmsbDKXVfOeN476wU3reXNUet7cOCkqo6zleYuOWaXnzXBUNklO3DbkefO987TN4JHx1/vjPW9x2WSl542iskkm12agwvMWErOT//71Q7jtvRB85kWkp5+b6/uS1pIpcaidO0B79+DK990jF1Y1ULsvAL98DumZM20PZftCCpQkeNV7vtH2SHoDJQP7R27OFQY0SLo9a+Wc4eEQr/wliVuhP/BwiKPXf6vtYXQCApCQgfGCymXdip43A1cqSNWeN3ghNa3nzWXVcp63SLRtxfMGdlWLhMzzFmfifLZtE5433wstJ960U7xKRQWGNT1vhkOmLXjekEbizVihlYaPVt/zRnGzbmDE8xaXldojUM/z1utSSaWh9+zG8Mmn53sh4dW6iLaAOXsW5tvfaXsY3YcIankZ6XPPy0Vo25hUJoOZBqXn31uQyIpFyUoLgrAgEHE2OYnPThlV6nmz2xOMIRARUigoZnBUNgkAU3veQskiIjVis2FkxpRNZptVet6IM00yzvPG5CdNqe9589lDO6GIqva8hTf3LyOwySRXzvPmM23B80ZhuZbnjRnEatTzNnRlneM8b7mySKDoeSN24jc6HpPotnAzKcyLL81dtAEIqVUhYsEnAmgEZphzc57QQRCaYN4+Xl8eKedaQRAWjEQZGKY4MWQzb8iXTfpSSptxA5Ti0bLJTXjefPnjiOdtUqsAoDHPm28VEPu5Sj1vnGXe6nveXKuAiJwABKznzcTNuNMwocl0njcntOP9kxXacdnkZjxvdjkStoriOVhKUeOfHg8R/Xsiup+I7iOizxDRTiLaR0RfIqIT7n7vVt5j7r8Ahzfe5hfeSltv4YIxj5iViWxagiibTGjBmEvctoX8f1lIFjpmhYWkqZi1pZIpFNl+bgr2XitTuGcoZewMlMRQblkp2+uMlLE+KwXnE3M3Bfc4e0P2y+6eFaznTfnn/Tr/ONqOCtv4xxRtT9F2Krvllynsg30ZZfw6DTD5dQTWFLZjOxWnfazscwjrlL13LQJCqwDltlFuvVIuE6cArQBSNpPmquigdZZZcxOWIE2zMso0tco2NQAbMLNV08ZYUcrW8wbjPODGON8aW29c6rZJ7XZk3HPsbwC5bWy5KVt/HGe34gQsZWxauBHRIQD/DsBbmPm1sFWe1wP4MIBbmfkYgFvdcn9g3vaijZIEes9uqAMXuxX+Z5t+H5eFjVkBlCRQy8vQF6y2PZTGWZi4DY1OHWb+XrquQYOlMJvmIrEwMSuMIDE7OWbJCbaEUiTKQCuDhEx4rJ14s7dMvClygs2JOKtJOBNviq1Hyos3xfbmBZbicB+ElRdvkcjKxFu2rlTgqUi8xeLMCbJyAReJt7B9JuJYZ+uCeAuvs0INlIk3VuTePxJv2gkzpcDuPog3ra0nzos3L9a8eIvEnPe85cWbAdLUirc0BRu3bJygMyYTb36ZM4HmxZsVesjEm/E3BGEHY5+nFNlzNeZC3FLGDbbUcpmIEgArAJ4A8G4AN7vnbwbwc7X2pPToH/ZZ4YWIUMrTf/oqPPjfr8Dw5HftisUSs83FrNAZztxyGI/c+OpF9rT1M27jc+3inEMa4+wth/Doza9rexizop8xK4wl/es1fPt/X9X2MGZFIzFLsKWSiTI58aaL4i0IOCfe/E0bu6wNyD1HsXjTTsDlxBvnxBtGxBtHmTAvoBisuSDMCuJNF4Sce2yC4PLL0euj52LxZnxGzYu3cE/R6wicxJk3J8585i1R9l4VxJvWYRlagxKdrdO2P2juXmv4/sy5zFt0H8RbWhBvaQpyog2pAYapzbgNDcgY0NCAUgM1NPnMW4l4oxHxlpVbjgvSTcHMPyCi/wbgewBeBvA3zPw3RLTGzE+6bZ4koovr7dA4JUyznRRELiIq0fv34ZHfvxxLdy7j8PHhbI5Ri8e/8ZgFJJ5aRu/fhxOftH3aDh1fzMmEZhK3wOyz6PJ/oxK9fx8e+eRhDL62goOzittFO9cKraL378Ojn7oM6h9WsHZ88VokNRmzBLazSoKi2RZR6nljYHrPm/OOTfS8MbuJL7bgeYum/8953uJJNTbreUvtLIvB85Zm/rbGPG+Gs1YBGuM9b+7z1fa8EZW2CrCzTJLNlFa1ChjjeZuk3LZSKrkX9peIowAOAlglovdM8foPENFdRHTXBtZtWhPI7meJXEiMkBw5jNNXH8M1xx7Cwf93Djtv+XrbQ2qc5mPW/TeeR8wKIyRHDuNHP30M7zr2IA599RyWP794MQvMIG7niZxrR/Bx+85jD+HQ353D8p/PIG69z6Olc1OvY1YYwcfstccewKW3rWPlc3e0PaTGaTJmz71wDgMVedxc2WSZ581n3ao8b3HZZG3Pm8rEIKikbHKMh6225y2UR+azayg+F722Kc9b8L75io5xnre4bLKO581wVjbJjLGet3Flk2WeN0Y9z9sEtlKo/DMATjLzsy5oPwfgJwE8TUSXuF8mLgHwTNmLmflGADcCwG7axzRI7AGb5WQkLi0qFCDCyV++FJ9+76fwu1e+DXo4m+aZlCThP0RLNBqz4aJIYmr+zClmO+LtbDhu55CJkXNtOXM71w7Aw43FOdcK7TGvc237NBazr7hqPyswBioNmauhUUiQRtk21OvzppyAcE26fUZubKsA43xhvlWAiTJvcZ83IJqt0mZ8ookRRxt5u3HHfd5sRi56bz8eP+FGsYVBNMNkSN9VtQqo6PMGTYChbDZH/0GivcVTMxIQMmth2c8mGS178UZM8aisSI5HVjiv+u3DMrkWAVGTbts/gfMZT7txOD4M29NvlrNKfg/ATxDRChERgGsAPAjgCwBucNvcAODzE/dEAJ8/P/sZJOVCopTBbQfw8mUb+C8/8y9mU6bqZvvj4YzKL+vTXMwCMrlCiwxuO4CXD88wZoGuiDag6bidpWfV//op/y9Kmfm5Fsj68S3SuVZojdW/uwgvX35+tjHbDRqLWQKwQw+RkMFApUjIYEmlOc/bQKe1PW9+tknveaOC5w1+0hLvedMFz5tGfc9bmOAkuo3zvGn/eIznrZCVyy9nXrewn4LnLWTedDQZSTzbZJnnLVqG1rassszz5jNvyWDznjdj8p631ISM22Y8b5PYisftDiL6vwDuATAE8A3YXxsuAPBZIno/7H+En5+8s/CPMEeSA2t48LePILld4dBdBsNZNdhmBlAjGmdMozErtEJyYA0PfvQI9O0KB++eYcwCbV/4BnoVtx05Zl0jObCGh37rKPTtNNtzLWBLe1qmVzErlJJccgAPfOwwkts1Dt4545jtAE3GrM235DNuG0YjgQkJtSE0EmVm43nzvrAUzsfGbpp8tiWJ3ru2Fc8bu8ybsS8uetxyy+6o1PW8gZ3XK2X3Y6DLvKUuY6XJlhRqZJmtzXjeyDUJ1xoMl2nrgOdtZpOTAAAzfxzAxwur12F/qRA6THLJAbz45sP4tau/gtve+3bwXffN9g07ckEnMdtffMx+8Oov49ZffcfsY7ZDSNz2l+TAGl5602G856f/Hne8743gu++f7RvKuVbYIsmhgzjz1kvxkatvwed+5Rrwnfe2PaS50GTMJirF0GgoMlBMGKgUG0YHf1uCFENoK8yYrYgDQtmkvQd82WQKrwfYiTgrBBgAefHm/WPxxCCFskmOShJD2aQviaRRMeZFXqhsVHkxF5dN5ssf3Rq3bZjMxHDmc1NWvMWeN6RkhaOJJ+ooL5v04g1MmZiKPW9w5afKl6ACpCMxhQllk8Z6DBnueWYQGyuItS2/JON/KLMHhmDfN4xRqVyTbmg7biZXlpqyFW/sPkc6+fy9eM04hFqc+NBRvP+6L+Mrr1sFsH0ugIX+8shvHMUHr/sivvja3ZCYFfrCY79+Ba697k7c/oYBgBmLNkFogId+8zA+cd2f4I9efTmA7SHamoQIGJD1symXATesxs4waQWG28EYzxu0saINdlkFH9oYzxsjm20S1gLgZ3LcsueNEb+T235GnrfCezGRLVFswvPGHI/EZuZM6jxtDXvelM28MVS5522WGTehnzz/l8dAx4G//aU3A3io7eEIwkRO/cWVUN8EvvSv3gaJWaEvnPzMG5DcT3jkV14J4OH5vXF3/JlCz1j96iuAu4Gbf+FnATzQ9nB6CYGtcANgQDCkYIgzoWZQq1WALV/MXkRM+UycE2+24bMK4i0TX5EAKrYKQDQNvx+bFzMqEm+MUDYJ2N2Fsklkp5lRIZdJLMoJt2jr0uVMPlEkMGMhF7ZgJ37cQiyOYjFlSz4L4sqYvLhiMyLOGLDijSkv6oiiMkmASeWWoSj69ADIL9vx2RpVm71jTdkY/YebgAi3bciZO16BtW8NYb4lF8BCP3j5jouwdq/ErNAvdtyziv33byC9f46iDRDRJmya+/7xVThwD8McF9G2WciVQw6QYgM6lE3W8bxxameO9OWS4zxvQbzV8bxFqR1G3vPmBp1534BMmNXxvLlSx3Get2JPuHGeN+KsbJK8L68pz5uboROwmTIm1ZznzR9L2A9Bhqs9bymHsslsjPZzTdJuIty2IYd/5x/bHoIgTMVl/1liVugfB/+rxK3QL45++GttD6H3EGyp5AY0BkhtNk2hludNuwlLiq0Ctux58/kfQ5l4G+N5A1xiqI7nze+9pucteNoqPG/s77U/mlN43tzkJtWet0hJ+r035XkzNCK6NuN5m/TDmwg3QRAEQRAEQWgAIlsqqclgw6oNe78FzxspgzTyvBETUqNm43kjWIHntm3U80YIs1lSNJtiGNeWPW/uTVOejeeNTV7UGZM18fbbb9LzhjCRynhEuAmCIAj9QjxcgiB0FAIwUENsmAQDSpGSwgC2Jm4DGgMym/K8AQiet9QAWhmwUfmySWDrnjcnmILgaNLzVtUqoCHPm206h3zZZFOeNzYITbr9+mLZJDbvecsUKMYiwm2WKG1zzIt8gSEXUIuFL8+Q71ToMoX4VLt2ARsbMOfOtTQgQRAEC4GhwUAk3nzZJIBQNlnmeTNEI60CxnneNHEom5zkefOZttY9b/44zcDz5jNWM/G8OZEGoFy8+d55pa0Cpve8VdFv4dZx0aCWBuDUgDfOtz2U2UHuf2yHvwehPpQMADbg4bDtocyOjp83hClRGmr/XvBLLwOLLNy2ww+BgrAAhFklWQNqCG9xquN5S5TB0CgkSKNs26jnrbRVwATPW9Zge0rPm8uqzcLzBnaZshl43mCcGIvFW9OeN6SReDO2J1saPlp9zxtFIm6hMm7+gqvrf8CIoPftRfr8jwBTQz43jXIRFx8jcv/jGhwPJUnzF/hKgxTl9ysX2rOHCGplBebs2XaOtfveAcz2u59FLJVlKRclZjv+OWjHDtCPXQHz8Ml2sm1KW39D/GNH0+daIlAyGP0BcIvfDQ2WbKnU+rpdoTRokGTL/pfmRf4RRxBmAAEY0BDKe9ycNih63rQyUG55as8bUN4qwHneiAgpVDOet1CyiEiNWIFFJiub3IznzVeMBs8bUfainAdues+b/9tMvlUAMXjoxJvLhtkySP/m2fdX3/NGYdneFzxvhkE6FqoMYjXqeRsCpK33bZJy65dwY5/e7a5oo8ESaGkAc/rFdkQbUP6+zO5nhAZw/xl8yrhRTAo2hXUd/a4XBnfBZl4+196xdifAkfdvejyz+Hxl+1yUmO3q53A/jtEFq+CTP0Dqxca8cWb1HE2fa0mBhxujz23xuxnZp0nB56PP4rwcgiBMh8+4+ctzK8ps2aSfRdK3ClDgLXneqloFGAMoxY143nz5Y65ZdMjeZWWTdkDZmAlR7gBFIefWFD1vPiNYKJvEZjxv7jMFz1vqxFFV2WT8HZZ53kw8MUl2zWIzclFmrqxVQNiTE9r+eAD2RzKiMNskTTi190u4eTp6MUFJAtK2dnWhyyM9Hf0ehCnwGQP361GrSDwJdSCCuuAC0GAApAbp6dPtjmcecdvWDw7yf1IQpodsds3qEhpbNgkg53kzaXWrgDLPmx7TKqDoeWOjslYBU3jebE+4rFVALqvmxRtTedlkE543N06Kyihred7ir6TK82Y4KpskJ24rPG9elBXLJt19EIyb8bwZE8omJ9E/4dbhPyS0vAw+fx589mzbQ5ktHf4OhOlQSwPwcCjlUEKvoLWLwC+cRvrU020PZbY0mb0TBGEuZBm3fDZexVk0IGTcNJsg3sqadW/W88ZkhY33vCmyGbiZeN5UXniVlk1O4Xmz+4nEW26c9T1v4EgMlnnetArXtJknrabnzXDItFV63qL0Wbb/OFZGPW+T6J9w6yJEUMvLMC++KKJG6AdEoKUlmPV1iVmh89BgKfyCqdcuRvrdH5SXDwqCILSMLX8cQhfK75Rfjjxvmg02oIPnDWYArRkbRkeCZZOeN1j/VM7zxgyOyia9l2wWnreRsslss8meN69lKjxvTL5RuBsAcdZ7LvbEJQiThVR63oqtAoqeNyKwySQXKbKiDFGmrcrzRvmm3BM9b4YnXpN1R7iNMVpTkgRF20mYYc7JBbDQI5jBG0OJWaHb+Ik50jT8UTTPv2BFm8SuIAgdhMBYotTlXjLqeN7KMm51PW+MGp63ilYBEz1vbGd3BOp73kZaBaDC88Zw+67vefOtAsKeUgKpEs8bZ2WTW/K8+fECrmwS5Z43YGuet4XJuJFv5NChkpGi0GzbHyQI0yIxK/QFPwEIs535VBAEocMoGICAFAQNBYP5eN6wWc+bE0uVnrdofannLVc2OaXnLZRZ5uRREH5B2G3G86aysskyzxuYMjE10ipgk543P2EJKYT+b8wgNvU8b72ZnGTMr6ednOhDfu0VBEGYLW6ip85XXQiCIDiyjFvR4zbe82aIkBplBdyMPG9KZeIt53lzamHTnjcgm63SizGOSiJreN68eCPmzJfmKwe9OozFWzTDZOaVm87zBk2AIet5gys5LXreIp+azRpSfrnK85amVsDGIy163gp/0/yMnuPojnDrI13vJycIgrAAUJJYD0GXqi4EQRBKIHJ93HwNomOS582XQ24YOxNGledNKcaQVRByihjDkGXDWM+bYWsWm4nnzb+d97wxMLZVgHuv8E7OWmZcKSQwxvMWdhovR0Ku6HkjN76C5802/44mLIEa9byZgrgqtgooet5I5cr7o1GNet5I5Scs8aWcY+iOcBvXTLSLAqnj/eQ6idJSnif0i443oF54vMdtOMRog0ehChosiQ9QEFrCzyqpvaBwV+ZFz5stnVQw4Kk8bxtGI4EJQm4IjUSZ+XnejBNGlJVDbsrz5pcZ1a0Ctuh5A1e3CrAzU1J1q4DwNVV43gxnZZPkJmliS78AAA0XSURBVEzR2pZHMuU9b3DvWcvzNv68PdEFR0R/SETPENF90bp9RPQlIjrh7vdGz32EiB4looeJ6F2T9l94s/LVWttvtUuQkj+KU0Kq/Ptt/H3mGbN9ouL/l9ANJG7LoUHihJucb2sj51pBKGUeMUsAlpBCkbEzR4KhYB/bdQxNBgNKMaBhEHkDNXTrUiveKIWGQaJSJ+iMnbFS2ef9LaFoGYxEGWjl3kcZKGJoxVBunW0REN+7Ej5iO6WEewwn+uxj9+EUwIrDPSsnqNxjkBVxvlIzPPbCi6JtVLY+bB9u5G7ZOuReQ2Hf7Jb9vr3njZ2wtJ43yvZLBGjK3kM7tRjds/e7ec+b8o/tMpSyrQRIAVoB2vXEdeItLAOhbBKGszJKZqtUUwOwscvG2NsE6qihmwBcW1j3YQC3MvMxALe6ZRDRVQCuB/Aa95o/ICJd4z3sH+WKP8y8cb57mZqujacHzLFX2E2YR8z2DbnwnZ75HrObIHGbRyYj2RQ8vzYfN0FiVugXN2HGMeszbktIgzhbKtwPSu/TnHgbqBSJsgIvUSl2qCESlSIhY58jg0S5G6WRYDMj4s3eMvGmnGhTyljxpuw68uLNrSMn0qDY3ohz4g1UIt7cjcM9R4+RF3jF9SOP8+Itfys+F4kzZTNtuXU6/3xRvGX3yj3nRZkVauzuEe61FW9a58Vb8d5V7nCaAmka3RsgTa14S1OwccsTTt0ThRszfxXAc4XV7wZws3t8M4Cfi9b/MTOvM/NJAI8CeNuk9xCEJpGYFfqIxK3QNyRmhb4xj5glAAOXUfPibSdtYIA6Ii4v3naqjWw9pXa/RfFGefGWlIm3IOCceHO3ongjZXLiDSPiDXnxphEEWybeOMqEIRJvXMiqTS/eRl5fKd68WHPZuEniTcfCrCDeIrFWKt5UUbwVRFwyCFahUfGW5sVbarDlUskK1pj5SQBw9xe79YcAfD/a7nG3bjLjyriUljIvYas0H7OCMHskboXpUa0msiRmhb7RaMwSYEsfI/GmyFihFom3Ytmkhi2r9GWTCpx53lw5pC+bjMWbLZfMxJvdzoTX+nJJf9O+TDIqmwzijZAXb1QQb3HZJMF66ChbP1LSGAszissss+0qyyajMsvKbUP5Y345e55C2STisklypZShNNLtMy6X9OLNlUsG8RbKJaOySRWVTZIKrQVIa9u6wJdaAnnRFmXifNnkvCcnKVNXpUMgog8A+AAA7MSK87CVT/ZhzXyAzCjWc7o50cPmY1ZYfLoZs8B2itvufgedhQYJ+HznJs/aPjErLAqbitmDhxQGbuYNA0ZKBksMnCdAg2wDbspbR4qtAqr6vBmm0CpAkYHi6j5v2k1YUmwVUNXnjZndxCV28hEGyvu8TdMqoKzPm9MwlX3enByIWwVkG0Vryvq8ReMs6/MGVd0qIDdTZAqbYRzb580N1Pd502xlig4jru7zZqxoLu3zNqN2AE8T0SXM/CQRXQLgGbf+cQCXRdtdCuCJsh0w840AbgSA3bSPx3nGOtnHTZiedi8imo9ZYfFp/8JX4rb976B38Pp6m28vMSv0jUZj9vWvH/AADLjJROwkh/kZJie2CiCDDc5aAACZAPPrlZtp17DKtlNuyv9ohklyYs9urEDKII36vBHb/nHQxoo2JyNV6L0WiTXGdK0CCHYWSiDr8+YOxMQ+b4z4ndz20RqCa6PgBGBxXGP6vHmBGT8fvxeTm22SGEh5a33emOOR2NkoTQpmkxd1cBnOrXrcKvgCgBvc4xsAfD5afz0R7SCiowCOAfj6Jt9DEJpEYlboIxK3Qt+QmBX6RqMxaz1uwADWW2ZLJvOet2LZZFgXe91KPG++bDJ7PL3nTRU8b+Qej5RNbsbzFnxucenijDxvXnsWyyYb8LzB+d5Gyiab8LyNnbAkxZYzbkT0GQD/DMBFRPQ4gI8D+D0AnyWi9wP4HoCfBwBmvp+IPgvgAQBDAP+GuYH6RimVEaagEzErCFMicSs0xpz+ZkrMCn1jHjFLRLZazqZbopo+lzpzZZNLsNmy1HWhTkHQUDC2AzYMaKTP24DSUDYJIJRNlvV5M0ShbFKRnYJeEYdySQZyfd40cSib9Dur6vPmM222PJGixBW7kkLO5qaIyiJzWbU6fd5cqeO4Pm+5Ekv/hhV93oizsknyvejK+ryl1u83VZ83ZoSySWYwqeo+b66fG4CRPm9b9rgx8y9WPHVNxfafAPCJSfsdYUyT7fABRbwJNZhbzApCg2y7uPV/1OW83jjz+pu57WJW6D3ziFmbcaNQG2gi8Vb0vA1gvW5pwePWhOctUQZDo5AgjUolRz1vFJVN1vW8ZQ22p/e8AXZ1Lc+b33tNz1s2kUm55439faasZ+d5AzbleZtE05OTbJ5xHrf59f8SBEEQ5oEItpkhfzMFoT0IhAEpaDA2bF4LG9ys5037ZVjPm1YGyi1P7XkDbFZtTp43GMq2pexBE543dlrKijjKXpTzwDXgeSMGD514c02zG/O8Tfjb2B3hJgiCIAiCIAg9ZwCNDUoxYCANKsDeD2D8NBfh6l0jL+TSQr2ccstWlGVlkykpDGDLJxWs580QZ0LN2JLLcUJOMWUzTkLZskllwEblyybhPoMvmwRHE45EAshwJt7YbRd/3nx1pc3euc/vyybDtr4UEtlvfaNCzq0xZWWUo2WT8Mc6kk8UCcxYyIUtbF2pXfAZN00lZZOws1lGkDH57B2bkQlJ2FUcNuJx6wTicRMEQRAEQRA6DgHQ1iBlxRuyssmtet4GlIayyUmeN5NGZZNjPG9W3/D8PG+q0Cqg6Hljyjxx03re/HdQ5Xlz4yxrFTDJ82bFKqo9b4ajskly20/veduyx20uEI33uCUD+4HGlFMKgiAIggD791T+XgpCKxAICbS9wmdtJxdxos007HkzruzQZ9w0myDeEpUGETeN5833eWNmrzNARG7WyYY8b+z9Zu7jxGIsEm3Tet7ICeT4tTnPW5yJ89m2mp43cCYGSz1v2ilepaLkaU3Pm2EnECeft2lSLeU8IKJnAbwE4FTbY+kIF0GORUyd43E5M79iHoMBJGYrkLjN6FzMAgARnQHw8Dzfs+NIzGZ0NWblXJtHYjZDYrYfSMzmmXQ8xsZsJ4QbABDRXcz8lrbH0QXkWOTp6vHo6rjaQo5HRlePRVfH1RZyPDK6fCy6PLZ5I8cio8vHostjmzdyLPJs9XioyZsIgiAIgiAIgiAIbSLCTRAEQRAEQRAEoeN0Sbjd2PYAOoQcizxdPR5dHVdbyPHI6Oqx6Oq42kKOR0aXj0WXxzZv5FhkdPlYdHls80aORZ4tHY/OeNwEQRAEQRAEQRCEcrqUcRMEQRAEQRAEQRBKaF24EdG1RPQwET1KRB9uezxtQETfIaJ7ieg4Ed3l1u0joi8R0Ql3v7ftcc4CIvpDInqGiO6L1lV+diL6iIuVh4noXe2MWuJ2O8cs0M+4lZiVmJWY7RcSsxKzfWQ7x+08YrZV4UZEGsD/APCzAK4C8ItEdFWbY2qRq5n5jdEUoR8GcCszHwNwq1teRG4CcG1hXelnd7FxPYDXuNf8gYuhuSJxG9iuMQv0LG4lZgMSs3kkZruPxGweidl+sF3j9ibMOGbbzri9DcCjzPxtZj4P4I8BvLvlMXWFdwO42T2+GcDPtTiWmcHMXwXwXGF11Wd/N4A/ZuZ1Zj4J4FHYGJo3ErflbIuYBXoZtxKz5UjMSsz2DYlZidk+si3idh4x27ZwOwTg+9Hy427ddoMB/A0R3U1EH3Dr1pj5SQBw9xe3Nrr5U/XZuxIvXRlHm0jMjtLluO3CGNpGYnYUidluIzE7isRs95G4zdNozCaND286qGTddpzm8qeY+QkiuhjAl4joobYH1FG6Ei9dGUebSMzWpwvx0oUxtI3EbH26EC9dGEPbSMzWpwvx0oUxdAGJ23psKl7azrg9DuCyaPlSAE+0NJbWYOYn3P0zAP4MNlX6NBFdAgDu/pn2Rjh3qj57V+KlK+NoDYnZUroct10YQ6tIzJYiMdthJGZLkZjtOBK3IzQas20LtzsBHCOio0S0BGvS+0LLY5orRLRKRLv8YwDvBHAf7HG4wW12A4DPtzPCVqj67F8AcD0R7SCiowCOAfh6C+Pb1nErMVtJl+NWYlZitgyJ2Y4iMVuJxGyHkbgtpdmYZeZWbwD+OYBHADwG4KNtj6eFz38FgG+62/3+GADYDzv7zAl3v6/tsc7o838GwJMANmB/fXj/uM8O4KMuVh4G8LMtjnvbxu12j1n3WXsXtxKzErMSs/25ScxKzPbxtt3jdh4xS+6FgiAIgiAIgiAIQkdpu1RSEARBEARBEARBmIAIN0EQBEEQBEEQhI4jwk0QBEEQBEEQBKHjiHATBEEQBEEQBEHoOCLcBEEQBEEQBEEQOo4IN0EQBEEQBEEQhI4jwk0QBEEQBEEQBKHjiHATBEEQBEEQBEHoOP8fbw9zpX+j58AAAAAASUVORK5CYII=\n"},"metadata":{"needs_background":"light"}}]},{"metadata":{},"cell_type":"markdown","source":"# 1.4 Creating node features\n\nWe extract 25 features per base in our sequence (107x25 features for train & public test, 130x25 features for private test):\n* One-hot-encoded base (AGCU) (4 features)\n* One-hot-encoded loop type (provided) (7 features)\n* one-hot-encoded positional feature (index % 3) (3 features)\n* certainty: fraction of packages (3 different ones) that predict `(`, `)` or `.`  (3 features)\n* CapR loop type probabilities (6 features)\n* BPP sum and number of zeroese (2 features)"},{"metadata":{"trusted":true},"cell_type":"code","source":"arnie_train_features = pickle.load(open('../input/covid19-mrna-augmentation-data-and-features/arnie/arnie/train_features.p', 'rb'))\narnie_test_features = pickle.load(open('../input/covid19-mrna-augmentation-data-and-features/arnie/arnie/test_features.p', 'rb'))\ncapr_train_features = pickle.load(open('../input/covid19-mrna-augmentation-data-and-features/CapR_train.pickle', 'rb'))\ncapr_test_features = pickle.load(open('../input/covid19-mrna-augmentation-data-and-features/CapR_test.pickle', 'rb'))","execution_count":11,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"## sequence\ndef return_ohe(n, i):\n    tmp = [0] * n\n    tmp[i] = 1\n    return tmp\n\ndef get_input(train):\n    mapping = {}\n    vocab = [\"A\", \"G\", \"C\", \"U\"]\n    for i, s in enumerate(vocab):\n        mapping[s] = return_ohe(len(vocab), i)\n    X_node = np.stack(train[\"sequence\"].apply(lambda x : list(map(lambda y : mapping[y], list(x)))))\n\n    mapping = {}\n    vocab = [\"S\", \"M\", \"I\", \"B\", \"H\", \"E\", \"X\"]\n    for i, s in enumerate(vocab):\n        mapping[s] = return_ohe(len(vocab), i)\n    X_loop = np.stack(train[\"predicted_loop_type\"].apply(lambda x : list(map(lambda y : mapping[y], list(x)))))\n    \n    X_pos1 = np.stack(train[\"sequence\"].apply(lambda x : np.arange(len(x)) % 3 == 0))[:,:,np.newaxis]\n    X_pos2 = np.stack(train[\"sequence\"].apply(lambda x : np.arange(len(x)) % 3 == 1))[:,:,np.newaxis]\n    X_pos3 = np.stack(train[\"sequence\"].apply(lambda x : np.arange(len(x)) % 3 == 2))[:,:,np.newaxis]\n\n    X_certainty = np.zeros((X_pos1.shape[:2] + (3, )))\n    X_loop_capr = np.zeros((X_pos1.shape[:2] + (6, )))\n    X_bpp_sum = np.zeros((X_pos1.shape[:2] + (1, )))\n    X_bpp_nb_zeroes = np.zeros((X_pos1.shape[:2] + (1, )))\n\n    for k, mol_id in enumerate(train['id'].values):\n        if mol_id in arnie_train_features:\n            arr1 = np.array(list(arnie_train_features[mol_id]['mfe']['vienna_2']))\n            arr2 = np.array(list(arnie_train_features[mol_id]['mfe']['contrafold_2']))\n            arr3 = np.array(list(arnie_train_features[mol_id]['mfe']['rnastructure']))\n        else:\n            arr1 = np.array(list(arnie_test_features[mol_id]['mfe']['vienna_2']))\n            arr2 = np.array(list(arnie_test_features[mol_id]['mfe']['contrafold_2']))\n            arr3 = np.array(list(arnie_test_features[mol_id]['mfe']['rnastructure']))\n\n        X_certainty[k, :, 0] = ((arr1 == '(') + (arr2 == '(') + (arr3 == '(')).astype(int) / 3\n        X_certainty[k, :, 1] = ((arr1 == ')') + (arr2 == ')') + (arr3 == ')')).astype(int) / 3\n        X_certainty[k, :, 2] = ((arr1 == '.') + (arr2 == '.') + (arr3 == '.')).astype(int) / 3\n        \n        if mol_id in capr_train_features:\n            X_loop_capr[k] = np.array([list(map(float, i.split()[1:])) for i in capr_train_features[mol_id].split('\\n')[1:-2]]).T\n        else:\n            X_loop_capr[k] = np.array([list(map(float, i.split()[1:])) for i in capr_test_features[mol_id].split('\\n')[1:-2]]).T\n\n        bpp = np.load(f\"../input/stanford-covid-vaccine/bpps/{mol_id}.npy\").astype(np.float16)\n        X_bpp_sum[k] = bpp.sum(axis=1)[:, None]\n        X_bpp_nb_zeroes[k] = ((bpp == 0).sum(axis=1) / X_pos1.shape[1])[:, None]\n\n\n    X_node = np.concatenate([X_node, X_loop, X_pos1, X_pos2, X_pos3, X_certainty, X_loop_capr, X_bpp_sum, X_bpp_nb_zeroes], axis = 2)\n    return X_node\n\nX_node = get_input(train).astype(np.float16)\nX_node_pub = get_input(test_pub).astype(np.float16)\nX_node_pri = get_input(test_pri).astype(np.float16)","execution_count":12,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(X_node.shape, X_node_pub.shape, X_node_pri.shape)","execution_count":13,"outputs":[{"output_type":"stream","text":"(4800, 107, 25) (1258, 107, 25) (6010, 130, 25)\n","name":"stdout"}]},{"metadata":{"trusted":true},"cell_type":"code","source":"del arnie_train_features, arnie_test_features, capr_train_features, capr_test_features","execution_count":14,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 1.5 Extracting the targets\n\n5 targets, including the 2 non-scored ones (deg_pH10 and deg_50C) as they increase the performance of the 3 others when trained jointly on all 5."},{"metadata":{"trusted":true},"cell_type":"code","source":"targets = ['reactivity', 'deg_Mg_pH10', 'deg_pH10', 'deg_Mg_50C', 'deg_50C']\n\ny_train = []\nseq_len = train[\"seq_length\"].iloc[0]\nseq_len_target = train[\"seq_scored\"].iloc[0]\nignore = -10000\nignore_length = seq_len - seq_len_target\nfor target in targets:\n    y = np.vstack(train[target])\n    dummy = np.zeros([y.shape[0], ignore_length]) + ignore\n    y = np.hstack([y, dummy])\n    y_train.append(y)\ny = np.stack(y_train, axis = 2)\nprint(y.shape)","execution_count":15,"outputs":[{"output_type":"stream","text":"(4800, 107, 5)\n","name":"stdout"}]},{"metadata":{"trusted":true},"cell_type":"code","source":"N_NODE_FEATURES = X_node.shape[2]\nN_EDGE_FEATURES = As.shape[3]","execution_count":16,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 2. Creating the model & training"},{"metadata":{"trusted":true},"cell_type":"code","source":"import tensorflow as tf\nfrom tensorflow.keras import layers as L\nimport tensorflow_addons as tfa\nfrom tensorflow.keras import backend as K\nfrom sklearn.utils import shuffle\nfrom sklearn.model_selection import GroupKFold","execution_count":17,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"ae_epochs = 25\nae_epochs_each = 5\nae_batch_size = 32\n\nepochs_list = [30, 10, 3, 3, 5, 5]\nbatch_size_list = [8, 16, 32, 64, 128, 256]","execution_count":18,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def mcrmse(t, p, seq_len_target = seq_len_target):\n    score = np.mean(np.sqrt(tf.keras.losses.mean_squared_error(t, p))[:, :seq_len_target])\n    return score\n\ndef rmse(y_actual, y_pred):\n    mse = tf.keras.losses.mean_squared_error(y_actual, y_pred)\n    return K.sqrt(mse)\n\ndef mcrmse_loss(y_actual, y_pred, num_scored=5):\n    y_actual = y_actual[:, :68, :]\n    y_pred = y_pred[:, :68, :]\n    score = 0\n    for i in range(num_scored):\n        score += rmse(y_actual[:, :, i], y_pred[:, :, i]) / num_scored\n    return score\n\ndef attention(x_inner, x_outer, n_factor, dropout):\n    x_Q =  L.Conv1D(n_factor, 1, activation='linear', \n                  kernel_initializer='glorot_uniform',\n                  bias_initializer='glorot_uniform',\n                 )(x_inner)\n    x_K =  L.Conv1D(n_factor, 1, activation='linear', \n                  kernel_initializer='glorot_uniform',\n                  bias_initializer='glorot_uniform',\n                 )(x_outer)\n    x_V =  L.Conv1D(n_factor, 1, activation='linear', \n                  kernel_initializer='glorot_uniform',\n                  bias_initializer='glorot_uniform',\n                 )(x_outer)\n    x_KT = L.Permute((2, 1))(x_K)\n    res = L.Lambda(lambda c: K.batch_dot(c[0], c[1]) / np.sqrt(n_factor))([x_Q, x_KT])\n    att = L.Lambda(lambda c: K.softmax(c, axis=-1))(res)\n    att = L.Lambda(lambda c: K.batch_dot(c[0], c[1]))([att, x_V])\n    return att\n\ndef multi_head_attention(x, y, n_factor, n_head, dropout):\n    if n_head == 1:\n        att = attention(x, y, n_factor, dropout)\n    else:\n        n_factor_head = n_factor // n_head\n        heads = [attention(x, y, n_factor_head, dropout) for i in range(n_head)]\n        att = L.Concatenate()(heads)\n        att = L.Dense(n_factor, \n                      kernel_initializer='glorot_uniform',\n                      bias_initializer='glorot_uniform',\n                     )(att)\n    x = L.Add()([x, att])\n    x = L.LayerNormalization()(x)\n    if dropout > 0:\n        x = L.Dropout(dropout)(x)\n    return x\n\ndef res(x, unit, kernel = 3, rate = 0.1):\n    h = L.Conv1D(unit, kernel, 1, padding = \"same\", activation = None)(x)\n    h = L.LayerNormalization()(h)\n    h = L.LeakyReLU()(h)\n    h = L.Dropout(rate)(h)\n    return L.Add()([x, h])\n\ndef forward(x, unit, kernel = 3, rate = 0.1):\n#     h = L.Dense(unit, None)(x)\n    h = L.Conv1D(unit, kernel, 1, padding = \"same\", activation = None)(x)\n    h = L.LayerNormalization()(h)\n    h = L.Dropout(rate)(h)\n#         h = tf.keras.activations.swish(h)\n    h = L.LeakyReLU()(h)\n    h = res(h, unit, kernel, rate)\n    return h\n\ndef adj_attn(x, adj, unit, n = 2, rate = 0.1):\n    x_a = x\n    x_as = []\n    for i in range(n):\n        x_a = forward(x_a, unit)\n        x_a = tf.matmul(adj, x_a)\n        x_as.append(x_a)\n    if n == 1:\n        x_a = x_as[0]\n    else:\n        x_a = L.Concatenate()(x_as)\n    x_a = forward(x_a, unit)\n    return x_a\n\n\ndef get_base(config):\n    node = tf.keras.Input(shape = (None, N_NODE_FEATURES), name = \"node\")\n    adj = tf.keras.Input(shape = (None, None, N_EDGE_FEATURES), name = \"adj\")\n    \n    adj_learned = L.Dense(1, \"relu\")(adj)\n    adj_all = L.Concatenate(axis = 3)([adj, adj_learned])\n        \n    xs = []\n    xs.append(node)\n    x1 = forward(node, 128, kernel = 3, rate = 0.0)\n    x2 = forward(x1, 64, kernel = 6, rate = 0.0)\n    x3 = forward(x2, 32, kernel = 15, rate = 0.0)\n    x4 = forward(x3, 16, kernel = 30, rate = 0.0)\n    x = L.Concatenate()([x1, x2, x3, x4])\n    \n    for unit in [128, 64]:\n        x_as = []\n        for i in range(adj_all.shape[3]):\n            x_a = adj_attn(x, adj_all[:, :, :, i], unit, rate = 0.0)\n            x_as.append(x_a)\n        x_c = forward(x, unit, kernel = 30)\n        \n        x = L.Concatenate()(x_as + [x_c])\n        x = forward(x, unit)\n        x = multi_head_attention(x, x, unit, 4, 0.0)\n        xs.append(x)\n        \n    x = L.Concatenate()(xs)\n\n    model = tf.keras.Model(inputs = [node, adj], outputs = [x])\n    return model\n\n\ndef get_ae_model(base, config):\n    node = tf.keras.Input(shape = (None, N_NODE_FEATURES), name = \"node\")\n    adj = tf.keras.Input(shape = (None, None, N_EDGE_FEATURES), name = \"adj\")\n\n    x = base([L.SpatialDropout1D(0.3)(node), adj])\n    x = forward(x, 64, rate = 0.3)\n    p = L.Dense(N_NODE_FEATURES, \"sigmoid\")(x)\n    \n    loss = tf.reduce_mean(tf.math.abs(node - p))\n    model = tf.keras.Model(inputs = [node, adj], outputs = [loss])\n    \n    opt = get_optimizer()\n    model.compile(optimizer = opt, loss = lambda t, y : y)\n    return model\n\ndef gru_layer(hidden_dim, dropout):\n    return L.Bidirectional(L.GRU(hidden_dim, dropout=dropout, return_sequences=True, kernel_initializer = 'orthogonal'))\ndef lstm_layer(hidden_dim, dropout):\n    return L.Bidirectional(L.LSTM(hidden_dim, dropout=dropout, return_sequences=True, kernel_initializer = 'orthogonal'))\n\ndef get_model(base, config):\n    node = tf.keras.Input(shape = (None, N_NODE_FEATURES), name = \"node\")\n    adj = tf.keras.Input(shape = (None, None, N_EDGE_FEATURES), name = \"adj\")\n    \n    x = base([node, adj])\n    x = lstm_layer(256, 0.5)(x)\n    x = lstm_layer(256, 0.5)(x)\n    x = L.Dense(5, None)(x)\n\n    model = tf.keras.Model(inputs = [node, adj], outputs = [x])\n    \n    opt = get_optimizer()\n    model.compile(optimizer = opt, loss = mcrmse_loss)\n    return model\n\ndef get_optimizer():\n    adam = tf.optimizers.Adam()\n    return adam","execution_count":19,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"config = {}\n\nif ae_epochs > 0:\n    base = get_base(config)\n    ae_model = get_ae_model(base, config)\n    ## TODO : simultaneous train\n    for i in range(ae_epochs//ae_epochs_each):\n        print(f\"------ {i} ------\")\n        print(\"--- train ---\")\n        X_node_shuff, As_shuff = shuffle(X_node, As)\n        ae_model.fit([X_node_shuff, As_shuff], [np.zeros((len(X_node)))],\n                  epochs = ae_epochs_each,\n                  batch_size = ae_batch_size)\n        \n        print(\"--- public ---\")\n        X_node_pub_shuff, As_pub_shuff = shuffle(X_node_pub, As_pub)\n        ae_model.fit([X_node_pub_shuff, As_pub_shuff], [np.zeros((len(X_node_pub)))],\n                  epochs = ae_epochs_each,\n                  batch_size = ae_batch_size)\n        \n        print(\"--- private ---\")\n        X_node_pri_shuff, As_pri_shuff = shuffle(X_node_pri, As_pri)\n        ae_model.fit([X_node_pri_shuff, As_pri_shuff], [np.zeros((len(X_node_pri)))],\n                  epochs = ae_epochs_each,\n                  batch_size = ae_batch_size)\n        gc.collect()\n    print(\"****** save ae model ******\")\n    base.save_weights(\"base_ae_lstm_lstm\")","execution_count":20,"outputs":[{"output_type":"stream","text":"------ 0 ------\n--- train ---\n150/150 [==============================] - 15s 102ms/step - loss: 0.1468\n--- public ---\n40/40 [==============================] - 5s 116ms/step - loss: 0.0769\n--- private ---\n188/188 [==============================] - 23s 120ms/step - loss: 0.0518\n****** save ae model ******\n","name":"stdout"}]},{"metadata":{"trusted":true},"cell_type":"code","source":"del ae_model\n\ngc.collect()","execution_count":21,"outputs":[{"output_type":"execute_result","execution_count":21,"data":{"text/plain":"10440"},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"np.save('X_node_pub.npy', X_node_pub)\nnp.save('X_node_pri.npy', X_node_pri)\nnp.save('As_pub.npy', As_pub)\nnp.save('As_pri.npy', As_pri)\n\ndel X_node_pub, X_node_pri, As_pub, As_pri, X_node_shuff, As_shuff, X_node_pub_shuff, As_pub_shuff, X_node_pri_shuff, As_pri_shuff","execution_count":22,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"kfold = GroupKFold(5)\n\nconfig = {}\nscores = []\npreds = np.zeros([len(X_node), X_node.shape[1], 5])\nX_node, As, y, weights, groups_shuffled, SN_filter_mask = shuffle(X_node, As, y, train.signal_to_noise.values, train['id'], (train['SN_filter'] == 1).values)\ndel train\nfor i, (tr_idx, va_idx) in enumerate(kfold.split(X_node, As, groups_shuffled)):\n    print(f\"------ fold {i} start -----\")\n    X_node_tr = X_node[tr_idx]\n    X_node_va = X_node[va_idx]\n    As_tr = As[tr_idx]\n    As_va = As[va_idx]  \n    y_tr = y[tr_idx]\n    y_va = y[va_idx]\n    w_trn = np.log(weights[tr_idx]+1.105)/2\n    \n    base = get_base(config)\n    if ae_epochs > 0:\n        print(\"****** load ae model ******\")\n        base.load_weights(\"base_ae_lstm_lstm\")\n    model = get_model(base, config)\n    for epochs, batch_size in zip(epochs_list, batch_size_list):\n        print(f\"epochs : {epochs}, batch_size : {batch_size}\")\n        model.fit([X_node_tr, As_tr], [y_tr],\n                  validation_data=([X_node_va[SN_filter_mask[va_idx]], As_va[SN_filter_mask[va_idx]]], [y_va[SN_filter_mask[va_idx]]]),\n                  epochs = epochs,\n                  sample_weight=w_trn/2,\n                  batch_size = batch_size, validation_freq = 3)\n        \n    model.save_weights(f\"model{i}_lstm_lstm\")\n    p = model.predict([X_node_va, As_va])\n    for col in range(5):\n        print(targets[col], ((p[:, :68, col][SN_filter_mask[va_idx]] - y_va[:,:68,col][SN_filter_mask[va_idx]]) ** 2).mean() ** .5)\n    scores.append(np.mean(mcrmse_loss(y_va, p)))\n    \n    print(f\"fold {i}: mcrmse {scores[-1]}\")\n    preds[va_idx] = p\n    \n    gc.collect()","execution_count":23,"outputs":[{"output_type":"stream","text":"------ fold 0 start -----\n****** load ae model ******\nepochs : 1, batch_size : 256\n15/15 [==============================] - 8s 551ms/step - loss: 0.2328\nreactivity 0.3952884187219692\ndeg_Mg_pH10 0.4957223950654\ndeg_pH10 0.4877043945404378\ndeg_Mg_50C 0.4600390590920768\ndeg_50C 0.3753641773868789\nfold 0: mcrmse 0.5705066919326782\n------ fold 1 start -----\n****** load ae model ******\nepochs : 1, batch_size : 256\n15/15 [==============================] - 9s 567ms/step - loss: 0.2531\nreactivity 0.394420553697772\ndeg_Mg_pH10 0.5016855179916028\ndeg_pH10 0.5140094322446686\ndeg_Mg_50C 0.4500155234393754\ndeg_50C 0.37748828697821446\nfold 1: mcrmse 0.622940719127655\n------ fold 2 start -----\n****** load ae model ******\nepochs : 1, batch_size : 256\n15/15 [==============================] - 8s 561ms/step - loss: 0.2678\nreactivity 0.43470002689847964\ndeg_Mg_pH10 0.526774612698896\ndeg_pH10 0.515111054528476\ndeg_Mg_50C 0.47215475408942414\ndeg_50C 0.3955008263504551\nfold 2: mcrmse 0.6284493803977966\n------ fold 3 start -----\n****** load ae model ******\nepochs : 1, batch_size : 256\n15/15 [==============================] - 8s 553ms/step - loss: 0.2495\nreactivity 0.4607629544127128\ndeg_Mg_pH10 0.5246239577253572\ndeg_pH10 0.5060399017105729\ndeg_Mg_50C 0.4758396789908668\ndeg_50C 0.4669103990568079\nfold 3: mcrmse 0.6534403562545776\n------ fold 4 start -----\n****** load ae model ******\nepochs : 1, batch_size : 256\n15/15 [==============================] - 8s 565ms/step - loss: 0.2426\nreactivity 0.3918471635807466\ndeg_Mg_pH10 0.49427080320747036\ndeg_pH10 0.4775118773273991\ndeg_Mg_50C 0.4354167678900868\ndeg_50C 0.372986241542789\nfold 4: mcrmse 0.5544297099113464\n","name":"stdout"}]},{"metadata":{"trusted":true},"cell_type":"code","source":"preds_df = []\nfor p_ix, _id in zip(range(preds.shape[0]), groups_shuffled):\n    for i in range(68):\n        preds_df.append([f'{_id}_{i}', preds[p_ix, i, 0], preds[p_ix, i, 1], \n                         preds[p_ix, i, 3], y[p_ix, i, 0], y[p_ix, i, 1],\n                         y[p_ix, i, 3], SN_filter_mask[p_ix]])\npreds_df = pd.DataFrame(preds_df, columns=['id_seqpos', 'reactivity', 'deg_Mg_pH10', \n                                           'deg_Mg_50C', 'reactivity_gt', 'deg_Mg_pH10_gt', \n                                           'deg_Mg_50C_gt', 'SN_filter'])\npreds_df = preds_df.groupby('id_seqpos').mean()\nrmses = []\nmses = []\nfor col in ['reactivity', 'deg_Mg_pH10', 'deg_Mg_50C']:\n    _rmse = ((preds_df[preds_df['SN_filter'] == 1][col] - preds_df[preds_df['SN_filter'] == 1][col+'_gt']) ** 2).mean() ** .5\n    _mse = ((preds_df[preds_df['SN_filter'] == 1][col] - preds_df[preds_df['SN_filter'] == 1][col+'_gt']) ** 2).mean()\n    rmses.append(_rmse)\n    mses.append(_mse)\n    print(col, _rmse, _mse)\nprint(np.mean(rmses), np.mean(mses))","execution_count":24,"outputs":[{"output_type":"stream","text":"reactivity 0.4142755960986902 0.1716242695229251\ndeg_Mg_pH10 0.5077816073306024 0.2578421607432501\ndeg_Mg_50C 0.4571073540264081 0.208947133105024\n0.45972151915190024 0.21280452112373305\n","name":"stdout"}]},{"metadata":{},"cell_type":"markdown","source":"# 3. Inference"},{"metadata":{"trusted":true},"cell_type":"code","source":"del X_node, As","execution_count":25,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"X_node_pub = np.load('X_node_pub.npy')\nX_node_pri = np.load('X_node_pri.npy')\nAs_pub = np.load('As_pub.npy')\nAs_pri = np.load('As_pri.npy')","execution_count":26,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"p_pub = 0\np_pri = 0\nfor i in range(5):\n    config = {}\n    base = get_base(config)\n    if ae_epochs > 0:\n        print(\"****** load ae model ******\")\n        base.load_weights(\"base_ae_lstm_lstm\")\n    model = get_model(base, config)\n    model.load_weights(f\"model{i}_lstm_lstm\")\n    p_pub += model.predict([X_node_pub, As_pub]) / 5\n    p_pri += model.predict([X_node_pri, As_pri]) / 5\n\nfor i, target in enumerate(targets):\n    test_pub[target] = [list(p_pub[k, :, i]) for k in range(p_pub.shape[0])]\n    test_pri[target] = [list(p_pri[k, :, i]) for k in range(p_pri.shape[0])]","execution_count":27,"outputs":[{"output_type":"stream","text":"****** load ae model ******\n****** load ae model ******\n****** load ae model ******\n****** load ae model ******\n****** load ae model ******\n","name":"stdout"}]},{"metadata":{"trusted":true},"cell_type":"code","source":"preds_ls = []\nfor df, preds in [(test_pub, p_pub), (test_pri, p_pri)]:\n    for i, uid in enumerate(df.id):\n        single_pred = preds[i]\n\n        single_df = pd.DataFrame(single_pred, columns=targets)\n        single_df['id_seqpos'] = [f'{uid}_{x}' for x in range(single_df.shape[0])]\n\n        preds_ls.append(single_df)\n\npreds_df = pd.concat(preds_ls)\npreds_df = preds_df.groupby('id_seqpos').mean()\npreds_df.to_csv(\"submission_lstm_lstm.csv\")\npreds_df.head()","execution_count":28,"outputs":[{"output_type":"execute_result","execution_count":28,"data":{"text/plain":"                  reactivity  deg_Mg_pH10  deg_pH10  deg_Mg_50C   deg_50C\nid_seqpos                                                                \nid_00073f8be_0      0.549876     0.523977  0.955659    0.633907  0.604943\nid_00073f8be_1      0.616222     0.575488  0.955579    0.713923  0.661382\nid_00073f8be_10     0.256883     0.369454  0.324495    0.342207  0.290394\nid_00073f8be_100    0.492760     0.493667  0.510421    0.605250  0.542404\nid_00073f8be_101    0.482626     0.483296  0.473538    0.594109  0.528667","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>reactivity</th>\n      <th>deg_Mg_pH10</th>\n      <th>deg_pH10</th>\n      <th>deg_Mg_50C</th>\n      <th>deg_50C</th>\n    </tr>\n    <tr>\n      <th>id_seqpos</th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>id_00073f8be_0</th>\n      <td>0.549876</td>\n      <td>0.523977</td>\n      <td>0.955659</td>\n      <td>0.633907</td>\n      <td>0.604943</td>\n    </tr>\n    <tr>\n      <th>id_00073f8be_1</th>\n      <td>0.616222</td>\n      <td>0.575488</td>\n      <td>0.955579</td>\n      <td>0.713923</td>\n      <td>0.661382</td>\n    </tr>\n    <tr>\n      <th>id_00073f8be_10</th>\n      <td>0.256883</td>\n      <td>0.369454</td>\n      <td>0.324495</td>\n      <td>0.342207</td>\n      <td>0.290394</td>\n    </tr>\n    <tr>\n      <th>id_00073f8be_100</th>\n      <td>0.492760</td>\n      <td>0.493667</td>\n      <td>0.510421</td>\n      <td>0.605250</td>\n      <td>0.542404</td>\n    </tr>\n    <tr>\n      <th>id_00073f8be_101</th>\n      <td>0.482626</td>\n      <td>0.483296</td>\n      <td>0.473538</td>\n      <td>0.594109</td>\n      <td>0.528667</td>\n    </tr>\n  </tbody>\n</table>\n</div>"},"metadata":{}}]}],"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","version":"3.7.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":4}