revise import headers

[jeremydouglass]

Currently pylint is giving (unused-import) style warnings for many files. This isn't critical during early development but I don't see a reason why we can't drop all unused imports and then add them back when/if we actually need them.

Also, many files import almost the same packages, but in slightly different orders with subtle changes in what is added and dropped.

Although pylint (wrong-import-order) is quite opinionated -- I believe it requires built in, then third party, then module imports, with froms second in each group -- it might be useful to conform to it, as this would give consistency over a large number of big import headers.

[scottkleinman]

I've wanted to clean this up for a long time, but I'm not seeing those warnings. (I'm not sure if my VS Code settings have them disabled.)

[jeremydouglass]

Here are the warnings from pylint app when unused-import is not in the .pylintrc disable list:

180806 Reason4:we1s-wms $ pylint app
************* Module app.corpus.helpers.methods
app/corpus/helpers/methods.py:9:0: W0611: Unused import shutil (unused-import)
app/corpus/helpers/methods.py:15:0: W0611: Unused import tabulator (unused-import)
app/corpus/helpers/methods.py:16:0: W0611: Unused Stream imported from tabulator (unused-import)
app/corpus/helpers/methods.py:18:0: W0611: Unused pandas imported as pd (unused-import)
app/corpus/helpers/methods.py:19:0: W0611: Unused Storage imported from tableschema_pandas (unused-import)
************* Module app.projects.helpers.methods
app/projects/helpers/methods.py:6:0: W0611: Unused import shutil (unused-import)
app/projects/helpers/methods.py:12:0: W0611: Unused import tabulator (unused-import)
app/projects/helpers/methods.py:13:0: W0611: Unused Stream imported from tabulator (unused-import)
app/projects/helpers/methods.py:17:0: W0611: Unused pandas imported as pd (unused-import)
app/projects/helpers/methods.py:18:0: W0611: Unused Storage imported from tableschema_pandas (unused-import)
************* Module app.projects.helpers.workspace
app/projects/helpers/workspace.py:4:0: W0611: Unused import requests (unused-import)
app/projects/helpers/workspace.py:6:0: W0611: Unused import subprocess (unused-import)
app/projects/helpers/workspace.py:13:0: W0611: Unused import pymongo (unused-import)
app/projects/helpers/workspace.py:15:0: W0611: Unused BSON imported from bson (unused-import)
app/projects/helpers/workspace.py:15:0: W0611: Unused Binary imported from bson (unused-import)
************* Module app.scripts.helpers.methods
app/scripts/helpers/methods.py:6:0: W0611: Unused import shutil (unused-import)
app/scripts/helpers/methods.py:12:0: W0611: Unused import tabulator (unused-import)
app/scripts/helpers/methods.py:13:0: W0611: Unused Stream imported from tabulator (unused-import)
app/scripts/helpers/methods.py:17:0: W0611: Unused pandas imported as pd (unused-import)
app/scripts/helpers/methods.py:18:0: W0611: Unused Storage imported from tableschema_pandas (unused-import)
************* Module app.scripts.helpers.workspace
app/scripts/helpers/workspace.py:1:0: W0611: Unused import datetime (unused-import)
app/scripts/helpers/workspace.py:4:0: W0611: Unused import re (unused-import)
app/scripts/helpers/workspace.py:6:0: W0611: Unused import shutil (unused-import)
app/scripts/helpers/workspace.py:7:0: W0611: Unused import subprocess (unused-import)
app/scripts/helpers/workspace.py:13:0: W0611: Unused import pymongo (unused-import)
app/scripts/helpers/workspace.py:15:0: W0611: Unused BSON imported from bson (unused-import)
app/scripts/helpers/workspace.py:15:0: W0611: Unused Binary imported from bson (unused-import)
************* Module app.sources.helpers.methods
app/sources/helpers/methods.py:6:0: W0611: Unused import shutil (unused-import)
app/sources/helpers/methods.py:13:0: W0611: Unused Stream imported from tabulator (unused-import)
app/sources/helpers/methods.py:16:0: W0611: Unused BSON imported from bson (unused-import)
app/sources/helpers/methods.py:17:0: W0611: Unused json_util imported from bson (unused-import)
app/sources/helpers/methods.py:20:0: W0611: Unused pandas imported as pd (unused-import)
app/sources/helpers/methods.py:23:0: W0611: Unused import pymongo (unused-import)

[jeremydouglass]

...and here is the same, only this time without disabling wrong-import-order instead:

************* Module app.corpus
app/corpus/__init__.py:7:0: C0411: standard import "import shutil" should be placed before "import requests" (wrong-import-order)
app/corpus/__init__.py:8:0: C0411: standard import "import zipfile" should be placed before "import requests" (wrong-import-order)
app/corpus/__init__.py:11:0: C0411: standard import "import subprocess" should be placed before "import requests" (wrong-import-order)
app/corpus/__init__.py:14:0: C0411: standard import "from datetime import datetime" should be placed before "import requests" (wrong-import-order)
app/corpus/__init__.py:15:0: C0411: standard import "from random import randint" should be placed before "import requests" (wrong-import-order)
app/corpus/__init__.py:16:0: C0411: standard import "from pathlib import Path" should be placed before "import requests" (wrong-import-order)
app/corpus/__init__.py:32:0: C0411: third party import "from bson import BSON" should be placed before "from app.corpus.helpers import methods" (wrong-import-order)
app/corpus/__init__.py:33:0: C0411: third party import "from bson import json_util" should be placed before "from app.corpus.helpers import methods" (wrong-import-order)
************* Module app.corpus.helpers.methods
app/corpus/helpers/methods.py:4:0: C0411: standard import "import itertools" should be placed before "import dateutil.parser" (wrong-import-order)
app/corpus/helpers/methods.py:5:0: C0411: standard import "import json" should be placed before "import dateutil.parser" (wrong-import-order)
app/corpus/helpers/methods.py:6:0: C0411: standard import "import os" should be placed before "import dateutil.parser" (wrong-import-order)
app/corpus/helpers/methods.py:7:0: C0411: standard import "import re" should be placed before "import dateutil.parser" (wrong-import-order)
app/corpus/helpers/methods.py:9:0: C0411: standard import "import shutil" should be placed before "import dateutil.parser" (wrong-import-order)
app/corpus/helpers/methods.py:10:0: C0411: standard import "import zipfile" should be placed before "import dateutil.parser" (wrong-import-order)
app/corpus/helpers/methods.py:12:0: C0411: standard import "from datetime import datetime" should be placed before "import dateutil.parser" (wrong-import-order)
************* Module app.projects
app/projects/__init__.py:6:0: C0411: standard import "import shutil" should be placed before "import requests" (wrong-import-order)
app/projects/__init__.py:7:0: C0411: standard import "import zipfile" should be placed before "import requests" (wrong-import-order)
app/projects/__init__.py:10:0: C0411: standard import "from datetime import datetime" should be placed before "import requests" (wrong-import-order)
app/projects/__init__.py:13:0: C0411: standard import "import uuid" should be placed before "import requests" (wrong-import-order)
app/projects/__init__.py:15:0: C0411: standard import "import subprocess" should be placed before "import requests" (wrong-import-order)
app/projects/__init__.py:16:0: C0411: standard import "from pathlib import Path" should be placed before "import requests" (wrong-import-order)
app/projects/__init__.py:22:0: C0411: third party import "from flask import Blueprint, render_template, request, url_for, current_app, send_file" should be placed before "from app.projects.helpers import workspace" (wrong-import-order)
app/projects/__init__.py:23:0: C0411: third party import "from werkzeug.utils import secure_filename" should be placed before "from app.projects.helpers import workspace" (wrong-import-order)
app/projects/__init__.py:25:0: C0411: third party import "import pymongo" should be placed before "from app.projects.helpers import workspace" (wrong-import-order)
app/projects/__init__.py:26:0: C0411: third party import "from pymongo import MongoClient" should be placed before "from app.projects.helpers import workspace" (wrong-import-order)
app/projects/__init__.py:32:0: C0411: third party import "from bson import BSON, Binary, json_util, ObjectId" should be placed before "from app.projects.helpers import workspace" (wrong-import-order)
************* Module app.projects.helpers.methods
app/projects/helpers/methods.py:6:0: C0411: standard import "import shutil" should be placed before "import requests" (wrong-import-order)
app/projects/helpers/methods.py:7:0: C0411: standard import "import zipfile" should be placed before "import requests" (wrong-import-order)
app/projects/helpers/methods.py:10:0: C0411: standard import "from datetime import datetime" should be placed before "import requests" (wrong-import-order)
************* Module app.projects.helpers.workspace
app/projects/helpers/workspace.py:5:0: C0411: standard import "import shutil" should be placed before "import requests" (wrong-import-order)
app/projects/helpers/workspace.py:6:0: C0411: standard import "import subprocess" should be placed before "import requests" (wrong-import-order)
app/projects/helpers/workspace.py:7:0: C0411: standard import "import zipfile" should be placed before "import requests" (wrong-import-order)
app/projects/helpers/workspace.py:9:0: C0411: standard import "from datetime import datetime" should be placed before "import requests" (wrong-import-order)
app/projects/helpers/workspace.py:10:0: C0411: standard import "from pathlib import Path" should be placed before "import requests" (wrong-import-order)
************* Module app.scripts
app/scripts/__init__.py:6:0: C0411: standard import "import shutil" should be placed before "import requests" (wrong-import-order)
app/scripts/__init__.py:7:0: C0411: standard import "import zipfile" should be placed before "import requests" (wrong-import-order)
app/scripts/__init__.py:10:0: C0411: standard import "from datetime import datetime" should be placed before "import requests" (wrong-import-order)
app/scripts/__init__.py:14:0: C0411: standard import "import subprocess" should be placed before "import requests" (wrong-import-order)
app/scripts/__init__.py:15:0: C0411: standard import "from pathlib import Path" should be placed before "import requests" (wrong-import-order)
************* Module app.scripts.helpers.methods
app/scripts/helpers/methods.py:6:0: C0411: standard import "import shutil" should be placed before "import requests" (wrong-import-order)
app/scripts/helpers/methods.py:7:0: C0411: standard import "import zipfile" should be placed before "import requests" (wrong-import-order)
app/scripts/helpers/methods.py:10:0: C0411: standard import "from datetime import datetime" should be placed before "import requests" (wrong-import-order)
************* Module app.scripts.helpers.workspace
app/scripts/helpers/workspace.py:6:0: C0411: standard import "import shutil" should be placed before "import requests" (wrong-import-order)
app/scripts/helpers/workspace.py:7:0: C0411: standard import "import subprocess" should be placed before "import requests" (wrong-import-order)
app/scripts/helpers/workspace.py:8:0: C0411: standard import "import zipfile" should be placed before "import requests" (wrong-import-order)
app/scripts/helpers/workspace.py:10:0: C0411: standard import "from pathlib import Path" should be placed before "import requests" (wrong-import-order)
************* Module app.sources
app/sources/__init__.py:6:0: C0411: standard import "import shutil" should be placed before "import requests" (wrong-import-order)
app/sources/__init__.py:27:0: C0411: third party import "from bson import BSON" should be placed before "from app.sources.helpers import methods" (wrong-import-order)
app/sources/__init__.py:28:0: C0411: third party import "from bson import json_util" should be placed before "from app.sources.helpers import methods" (wrong-import-order)
************* Module app.sources.helpers.methods
app/sources/helpers/methods.py:6:0: C0411: standard import "import shutil" should be placed before "import requests" (wrong-import-order)
app/sources/helpers/methods.py:7:0: C0411: standard import "import zipfile" should be placed before "import requests" (wrong-import-order)
app/sources/helpers/methods.py:10:0: C0411: standard import "from datetime import datetime" should be placed before "import requests" (wrong-import-order)
************* Module app.tasks
app/tasks/__init__.py:6:0: C0411: standard import "import shutil" should be placed before "import requests" (wrong-import-order)
app/tasks/__init__.py:7:0: C0411: standard import "import zipfile" should be placed before "import requests" (wrong-import-order)
app/tasks/__init__.py:10:0: C0411: standard import "import subprocess" should be placed before "import requests" (wrong-import-order)
app/tasks/__init__.py:11:0: C0411: standard import "from datetime import datetime" should be placed before "import requests" (wrong-import-order)
app/tasks/__init__.py:12:0: C0411: standard import "from pathlib import Path" should be placed before "import requests" (wrong-import-order)