Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

minor version 1.3.1: adds click 8.0 dep #87

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
Changelog
=========

1.3.1 (2022-03-02)
------------------
- Bumped Click dependency to `"click~=8.0"`

0.9 (2019-01-16)
----------------
- Support for loading CSVs directly from URLs, thanks @betatim - #38
Expand Down
13 changes: 0 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,10 +129,8 @@ Options:
constants. Use one of QUOTE_MINIMAL (0),
QUOTE_ALL (1), QUOTE_NONNUMERIC (2) or
QUOTE_NONE (3).

--skip-errors Skip lines with too many fields instead of
stopping the import

--replace-tables Replace tables if they already exist
-t, --table TEXT Table to use (instead of using CSV filename)
-c, --extract-column TEXT One or more columns to 'extract' into a
Expand All @@ -147,29 +145,21 @@ Options:
table, with an id column primary key and a
state_name column containing the strings from
the original column.

-d, --date TEXT One or more columns to parse into ISO
formatted dates

-dt, --datetime TEXT One or more columns to parse into ISO
formatted datetimes

-df, --datetime-format TEXT One or more custom date format strings to try
when parsing dates/datetimes

-pk, --primary-key TEXT One or more columns to use as the primary key
-f, --fts TEXT One or more columns to use to populate a full-
text index

-i, --index TEXT Add index on this column (or a compound index
with -i col1,col2)

--shape TEXT Custom shape for the DB table - format is
csvcol:dbcol(TYPE),...

--filename-column TEXT Add a column with this name and populate with
CSV file name

--fixed-column <TEXT TEXT>... Populate column with a fixed string
--fixed-column-int <TEXT INTEGER>...
Populate column with a fixed integer
Expand All @@ -178,15 +168,12 @@ Options:
--no-index-fks Skip adding index to foreign key columns
created using --extract-column (default is to
add them)

--no-fulltext-fks Skip adding full-text index on values
extracted using --extract-column (default is
to add them)

--just-strings Import all columns as text strings by default
(and, if specified, still obey --shape,
--date/datetime, and --datetime-format)

--version Show the version and exit.
--help Show this message and exit.

Expand Down
4 changes: 2 additions & 2 deletions csvs_to_sqlite/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ def load_csv(
filepath,
sep=separator,
quoting=quoting,
error_bad_lines=not skip_errors,
on_bad_lines='skip' if skip_errors else 'error',
low_memory=True,
encoding=encoding,
usecols=usecols,
Expand Down Expand Up @@ -284,7 +284,7 @@ def get_create_table_sql(
# http://pandas.pydata.org/pandas-docs/stable/gotchas.html#support-for-integer-na
sql_type_overrides = sql_type_overrides or {}
if isinstance(df, pd.DataFrame):
columns_and_types = df.dtypes.iteritems()
columns_and_types = df.dtypes.items()
elif isinstance(df, pd.Series):
columns_and_types = [(df.name, df.dtype)]
for column, dtype in columns_and_types:
Expand Down
6 changes: 4 additions & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
import io
import os

VERSION = "1.3"
VERSION = "1.3.1"


def get_long_description():
Expand All @@ -23,7 +23,7 @@ def get_long_description():
license="Apache License, Version 2.0",
packages=find_packages(),
install_requires=[
"click~=7.0",
"click~=8.0",
"dateparser>=1.0",
"pandas>=1.0",
"py-lru-cache~=0.1.4",
Expand All @@ -45,5 +45,7 @@ def get_long_description():
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",

],
)
15 changes: 9 additions & 6 deletions tests/test_csvs_to_sqlite.py
Original file line number Diff line number Diff line change
Expand Up @@ -766,13 +766,16 @@ def test_just_strings_with_date_specified():
assert isinstance(gross, text_type)


def test_if_cog_needs_to_be_run():
_stdout = sys.stdout
sys.stdout = StringIO()
def test_foo(capsys):
print('123')
out, err = capsys.readouterr()
assert out == "123\n"


def test_if_cog_needs_to_be_run(capsys):
readme = pathlib.Path(__file__).parent.parent / "README.md"
result = Cog().main(["cog", str(readme)])
output = sys.stdout.getvalue()
sys.stdout = _stdout
output, err = capsys.readouterr()
assert (
output == readme.read_text()
), "Run 'cog -r README.md' to update help in README"
) , "Run 'cog -r README.md' to update help in README"