Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: refine print statement #46

Merged
merged 3 commits into from
Aug 10, 2023
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Changelog db-rocket

## Version 1.3.1
gyg-martin-jewell marked this conversation as resolved.
Show resolved Hide resolved

- Adding Markdown documentation to package description
gyg-martin-jewell marked this conversation as resolved.
Show resolved Hide resolved

## Version 1.3.0

- Remove `rocket trigger` CLI
Expand Down
15 changes: 7 additions & 8 deletions rocket/rocket.py
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ def _deploy(self, watch, modified_files):
if modified_files:
logger.info(f"Found changes in {modified_files}. Overwriting them.")
for file in modified_files:
logger.info(f"Sync {file}")
logger.info(f"Finished sync of {file}")
execute_shell_command(
f"databricks fs cp --recursive --overwrite {file} {self.dbfs_folder}/{os.path.relpath(file, self.project_location)}"
)
Expand All @@ -119,6 +119,7 @@ def _deploy(self, watch, modified_files):
f"databricks fs cp --overwrite {self.wheel_path} {self.dbfs_folder}/{self.wheel_file}"
)
if watch:
logger.info("You have watch activated. Your project will be automatically synchronised with databricks. Starting sync process:")
package_dirs = extract_python_package_dirs(self.project_location)
for package_dir in package_dirs:
python_files = extract_python_files_from_folder(package_dir)
Expand All @@ -127,7 +128,7 @@ def helper(file):
execute_shell_command(
f"databricks fs cp --recursive --overwrite {file} {self.dbfs_folder}/{os.path.relpath(file, self.project_location)}"
)
logger.info(f"Sync {file}")
logger.info(f"Finished sync of {file}")
execute_for_each_multithreaded(python_files, helper)
except Exception as e:
raise Exception(
Expand All @@ -139,23 +140,21 @@ def helper(file):
install_cmd = _add_index_urls_to_cmd(install_cmd, self.index_urls)
project_name = extract_project_name_from_wheel(self.wheel_file)

if modified_files:
logger.info("Changes are applied")
elif watch:
if watch:
logger.info(
f"""You have watch activated. Your project will be automatically synchronised with databricks. Add following in one cell:
f"""Sync completed. To use your library in your databricks notebook & automatically apply local changes add following in one cell:
%pip install --upgrade pip
%pip install {install_cmd} --force-reinstall
%pip uninstall -y {project_name}

and then in new Python cell:
and then in a new Python cell:
%load_ext autoreload
%autoreload 2
import sys
import os
sys.path.append(os.path.abspath('{base_path}')""")
else:
logger.info(f"""Install your library in your databricks notebook by running:
logger.info(f"""Uploaded your library to databricks. Install your library in your databricks notebook by running:
%pip install --upgrade pip
%pip install {install_cmd} --force-reinstall""")

Expand Down
11 changes: 9 additions & 2 deletions setup.py
Original file line number Diff line number Diff line change
@@ -1,15 +1,22 @@
import setuptools

# load the README file and use it as the long_description for PyPI
with open("README.md", encoding="utf8") as f:
readme = f.read()
Comment on lines +3 to +5

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this duplicates this PR, should it be here?


setuptools.setup(
name="databricks-rocket",
version="1.3.0",
version="1.3.2",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could pull this from the changelog too, the same as above

author="GetYourGuide",
author_email="[email protected]",
description="Keep your local python scripts installed and in sync with a databricks notebook. Shortens the feedback loop to develop projects using a hybrid enviroment",
url="https://github.com/getyourguide/databricks-rocket",
long_description=readme,
long_description_content_type="text/markdown",
url="https://github.com/getyourguide/db-rocket",
packages=setuptools.find_packages(),
install_requires=["fire", "watchdog~=2.1.9", "build", "databricks_cli"],
entry_points={
"console_scripts": ["rocket=rocket.rocket:main", "dbrocket=rocket.rocket:main"]
},
license="Apache 2.0",
)