Use logging module instead of prints (tektronix#345)

* docs: Update link to badge

* feat: Switched to using the Python logging module for printing out information to stdout.

This provides users with more control over stdout and also allows for logging outputs to files.

* feat: Added the pyvisa log to the file.

Also updated the tests to work properly with the new logging

* feat: Added individual VISA command logging for each VISA device.

* refactor: Update linting settings and update code to use Path objects instead of builtins where possible.

* docs: Updated documentation with instructions and examples on how to configure logging.

* refactor: Update code based on Pull Request build failures

* test: Add tests for the logging configuration function.

* docs: Update IP addresses used in examples

* test: Update some tests to be less flaky

* refactor: Add leading tabs to the standard logging output of the multi-line VISA write.

Also updated unit test simulated devices and locked the version of pyright.

.gitignore

@@ -5,6 +5,7 @@ tm_devices.yaml
 tm_devices.yml
 *.log
 *.dat
+**/logs/**
 
 # Poetry lock file
 poetry.lock
@@ -58,6 +59,7 @@ coverage.xml
 .pytest_cache/
 .results*/
 tests/samples/generated_stubs/*
+tests/generated_**/*
 tests/verify_devices.yaml
 prof/
 

CHANGELOG.md

@@ -26,6 +26,12 @@ Things to be included in the next release go here.
 - Added the `get_errors()` method to the `Device` class to enable easy access to the current error code and messages on any device.
 - Added more details to the Architectural Overview page of the documentation as well as highlighting to the device driver diagram on the page.
 - Added regex matching to the `verify_values()` helper function to allow for more flexible value verification.
+- A main logfile is now created by default (can be disabled if desired) that contains all the logging output of the entire tm_devices package during execution.
+    - Use the `configure_logging()` function to set the logging levels for stdout and file logging.
+    - The default settings will log all messages to the log file and maintain the current printout functionality on stdout.
+- A logfile is now created that contains each command sent to a VISA device.
+    - This file is located next to the main log file and will start with the same name, but have the unique address of the device appended.
+    - This file will only be created if file logging is enabled for the package (which is the default behavior).
 
 ### Changed
 
@@ -46,17 +52,23 @@ However, please read through all changes to be aware of what may potentially imp
 - _**<span style="color:red">BREAKING CHANGE</span>**_: Changed the behavior of the `expect_esr()` method to expect an integer error code input and an optional tuple of error messages to compare against the actual error code and messages returned by the `_get_errors()` private method.
 - _**<span style="color:orange">minor breaking change</span>**_: Converted the `device_type` property into an abstract, cached property to force all children of the `Device` class to specify what type of device they are.
 - Updated the auto-generated command mixin classes to no longer use an `__init__()` method to enable the driver API documentation to render in a more usable way.
+- Switched from using standard `print()` calls to using the `logging` module for all logging in the `tm_devices` package.
+    - A configuration function provides the ability to set different logging levels for stdout and file logging.
+    - The config file and environment variable can also be used to control the logging functionality.
+    - The debug logging from the `pyvisa` package is also included in the log file by default.
 
 ### Removed
 
-- _**<span style="color:red">BREAKING CHANGE</span>**_: Removed previously deprecated `TekScopeSW` alias to the `TekScopePC` class
+- _**<span style="color:red">BREAKING CHANGE</span>**_: Removed previously deprecated `TekScopeSW` alias to the `TekScopePC` class.
 - _**<span style="color:red">BREAKING CHANGE</span>**_: Removed previously deprecated `write_buffers()` from the `TSPControl` class.
 - _**<span style="color:red">BREAKING CHANGE</span>**_: Removed Internal AFG methods from the `TekScopePC` driver, since they wouldn't have worked due to its lack of an IAFG.
 - _**<span style="color:red">BREAKING CHANGE</span>**_: Removed previously deprecated `DEVICE_DRIVER_MODEL_MAPPING` constant.
 - _**<span style="color:red">BREAKING CHANGE</span>**_: Removed the `DEVICE_TYPE_CLASSES` constant and the `device_type_classes.py` module.
 - _**<span style="color:red">BREAKING CHANGE</span>**_: Removed many hacky implementations of `total_channels` and `all_channel_names_list` properties from drivers that don't need them anymore.
 - _**<span style="color:red">BREAKING CHANGE</span>**_: Removed the `verify_values()`, `raise_failure()`, and `raise_error()` methods from all device drivers.
     - These methods have been converted to helper functions and can be imported from the `tm_devices.helpers` subpackage now.
+- _**<span style="color:red">BREAKING CHANGE</span>**_: Removed the `print_with_timestamp()` function since this functionality is now handled by the `logging` module.
+- _**<span style="color:red">BREAKING CHANGE</span>**_: Removed the `get_timestamp_string()` function since this functionality is now handled by the `logging` module.
 
 ---
 

docs/basic_usage.md

@@ -10,7 +10,7 @@ This will print the available VISA devices to the console when run from a shell
 ```console
 $ list-visa-resources
 [
-  "TCPIP0::192.168.0.100::inst0::INSTR",
+  "TCPIP0::192.168.0.1::inst0::INSTR",
   "ASRL4::INSTR"
 ]
 ```
@@ -59,6 +59,23 @@ outside the Python code for ease of automation
 --8<-- "examples/miscellaneous/adding_devices_with_env_var.py"
 ```
 
+## Customize logging and console output
+
+The amount of console output and logging saved to the log file can be customized as needed. This
+configuration can be done in the Python code itself as demonstrated here, or by using the
+[config file](configuration.md#config-options) or
+[environment variable](configuration.md#environment-variable).
+
+!!! important
+    If any configuration is performed in the Python code prior to instantiating the
+    [`DeviceManager`][tm_devices.DeviceManager], all other logging configuration methods
+    (config file, env var) will be ignored.
+
+```python
+# fmt: off
+--8<-- "examples/miscellaneous/customize_logging.py"
+```
+
 ## Disable command checking
 
 This removes an extra query that verifies the property was set to the expected

docs/configuration.md

@@ -215,8 +215,7 @@ devices:
 
 ### Config Options
 
-These options are flags that enable/disable runtime behaviors of the Device
-Manager.
+These options are used to configure runtime behaviors of `tm_devices`.
 
 #### Yaml Options Syntax
 
@@ -230,6 +229,12 @@ options:
   retry_visa_connection: false
   default_visa_timeout: 5000
   check_for_updates: false
+  log_console_level: INFO
+  log_file_level: DEBUG
+  log_file_directory: ./logs
+  log_file_name: tm_devices_<timestamp>.log
+  log_colored_output: false
+  log_pyvisa_messages: false
 ```
 
 These are all `false` by default if not defined, set to `true` to modify the
@@ -259,6 +264,30 @@ runtime behavior configuration.
 - `check_for_updates`
     - This config option will enable a check for any available updates on pypi.org for the
         package when the `DeviceManager` is instantiated.
+- `log_console_level`
+    - This config option is used to set the log level for the console output.
+        The default value of this config option is "INFO". See the
+        [`configure_logging()`][tm_devices.helpers.logging.configure_logging] function for more information.
+- `log_file_level`
+    - This config option is used to set the log level for the file output.
+        The default value of this config option is "DEBUG". See the
+        [`configure_logging()`][tm_devices.helpers.logging.configure_logging] function for more information.
+- `log_file_directory`
+    - This config option is used to set the directory where the log files will be saved.
+        The default value of this config option is "./logs". See the
+        [`configure_logging()`][tm_devices.helpers.logging.configure_logging] function for more information.
+- `log_file_name`
+    - This config option is used to set the name of the log file.
+        The default value of this config option is a timestamped filename with the .log extension. See the
+        [`configure_logging()`][tm_devices.helpers.logging.configure_logging] function for more information.
+- `log_colored_output`
+    - This config option is used to enable or disable colored output in the console.
+        The default value of this config option is false. See the
+        [`configure_logging()`][tm_devices.helpers.logging.configure_logging] function for more information.
+- `log_pyvisa_messages`
+    - This config option is used to enable or disable logging of PyVISA messages within the
+        configured log file. The default value of this config option is false. See the
+        [`configure_logging()`][tm_devices.helpers.logging.configure_logging] function for more information.
 
 ### Sample Config File
 
@@ -320,6 +349,12 @@ options:
   retry_visa_connection: false
   default_visa_timeout: 10000  # 10 second default VISA timeout
   check_for_updates: false
+  log_console_level: NONE    # completely disable console output
+  log_file_level: DEBUG
+  log_file_directory: ./logs
+  log_file_name: custom_logfile.log    # customize the log file name
+  log_colored_output: false
+  log_pyvisa_messages: true  # log PyVISA messages in the log file
 ```
 
 #### TOML
@@ -393,8 +428,14 @@ standalone = false
 verbose_mode = false
 verbose_visa = false
 retry_visa_connection = false
-default_visa_timeout = 10000  # 10 second default VISA timeout
+default_visa_timeout = 10000         # 10 second default VISA timeout
 check_for_updates = false
+log_console_level = "NONE"           # completely disable console output
+log_file_level = "DEBUG"
+log_file_directory = "./logs"
+log_file_name = "custom_logfile.log" # customize the log file name
+log_colored_output = false
+log_pyvisa_messages = true           # log PyVISA messages in the log file
 ```
 
 ---

docs/key_features.md

@@ -14,3 +14,4 @@ complex examples.
         IntelliSense.
 - Organize connections to an entire "Test Bench" of devices with one package!
     - Full support for all VISA connection types (some require external drivers).
+- Customizable logging to both the console and a log file.

docs/known_words.txt

@@ -19,11 +19,13 @@ ci
 classdiagram
 codebase
 codecov
+colored
 config
 conftest
 cookiecutter
 cov
 csv
+customizable
 deps
 dev
 disable_command_verification
@@ -33,6 +35,7 @@ docstring
 docstrings
 en
 enum
+env
 executables
 filepath
 generate_function

docs/macros.py

@@ -265,8 +265,8 @@ def define_env(env: MacrosPlugin) -> None:
             used to perform a transformation
     """
     # Read in the current package version number to use in templates and files
-    with open(
-        pathlib.Path(f"{pathlib.Path(__file__).parents[1]}") / "pyproject.toml", "rb"
+    with (pathlib.Path(f"{pathlib.Path(__file__).parents[1]}") / "pyproject.toml").open(
+        "rb"
     ) as file_handle:
         pyproject_data = tomli.load(file_handle)
         package_version = "v" + pyproject_data["tool"]["poetry"]["version"]

examples/miscellaneous/customize_logging.py

@@ -0,0 +1,21 @@
+"""The console output and level of logging outputs in the log file can be configured as needed."""
+
+from tm_devices import configure_logging, DeviceManager, LoggingLevels
+from tm_devices.drivers import MSO6B
+
+# NOTE: This configuration will prevent any logging config options from a config file or
+# environment variable from being used.
+configure_logging(
+    log_console_level=LoggingLevels.NONE,  # completely disable console logging
+    log_file_level=LoggingLevels.DEBUG,  # log everything to the file
+    log_file_directory="./log_files",  # save the log file in the "./log_files" directory
+    log_file_name="custom_log_filename.log",  # customize the filename
+    log_pyvisa_messages=True,  # include all the pyvisa debug messages in the same log file
+)
+
+with DeviceManager(verbose=False) as dm:
+    scope: MSO6B = dm.add_scope("192.168.0.1")
+    scope.curve_query(1)
+    scope.check_port_connection(4000)
+    scope.check_network_connection()
+    scope.check_visa_connection()

examples/miscellaneous/register_dm_atexit.py

@@ -12,7 +12,7 @@
 atexit.register(dm.close)
 
 # Add a device
-scope: MSO6B = dm.add_scope("192.168.1.102")
+scope: MSO6B = dm.add_scope("192.168.0.1")
 
 # Use the device
 print(scope)

examples/scopes/tekscope/basic_curve_query.py

@@ -1,9 +1,11 @@
 """An example showing a basic curve query."""
 
+from pathlib import Path
+
 from tm_devices import DeviceManager
 from tm_devices.drivers import AFG3KC, MSO5
 
-EXAMPLE_CSV_FILE = "example_curve_query.csv"
+EXAMPLE_CSV_FILE = Path("example_curve_query.csv")
 
 with DeviceManager(verbose=True) as dm:
     scope: MSO5 = dm.add_scope("MSO56-100083")
@@ -16,7 +18,7 @@
     curve_returned = scope.curve_query(1, output_csv_file=EXAMPLE_CSV_FILE)
 
 # Read in the curve query from file
-with open(EXAMPLE_CSV_FILE, encoding="utf-8") as csv_content:
+with EXAMPLE_CSV_FILE.open(encoding="utf-8") as csv_content:
     curve_saved = [int(i) for i in csv_content.read().split(",")]
 
 # Verify query saved to csv is the same as the one returned from curve_query function call

examples/scopes/tekscope/basic_save_recall.py

@@ -5,7 +5,7 @@
 
 with DeviceManager(verbose=True) as dm:
     # Get a scope
-    scope: MSO6B = dm.add_scope("192.168.1.177")
+    scope: MSO6B = dm.add_scope("192.168.0.1")
 
     # Send some commands
     scope.add_new_math("MATH1", "CH1")  # add MATH1 to CH1

examples/scopes/tekscope/generate_internal_afg_signal.py

@@ -5,7 +5,7 @@
 
 with DeviceManager(verbose=True) as dm:
     # Create a connection to the scope and indicate that it is a MSO5 scope for type hinting
-    scope: MSO5 = dm.add_scope("192.168.1.102")
+    scope: MSO5 = dm.add_scope("192.168.0.1")
 
     # Generate the signal using individual PI commands.
     scope.commands.afg.frequency.write(10e6)  # set frequency

examples/scopes/tekscope/save_screenshot.py

@@ -5,7 +5,7 @@
 
 with DeviceManager(verbose=True) as dm:
     # Add a scope
-    scope: MSO6B = dm.add_scope("192.168.1.5")
+    scope: MSO6B = dm.add_scope("192.168.0.1")
 
     # Send some commands
     scope.add_new_math("MATH1", "CH1")  # add MATH1 to CH1

examples/source_measure_units/2400/smu_2450_leakage_current.py

@@ -13,7 +13,7 @@
 from tm_devices.drivers import SMU2450
 
 with DeviceManager(verbose=False) as device_manager:
-    inst: SMU2450 = device_manager.add_smu("192.168.1.4", alias="my2450")
+    inst: SMU2450 = device_manager.add_smu("192.168.0.1", alias="my2450")
 
     # Reset the instrument, which also clears the buffer.
     inst.commands.reset()

examples/source_measure_units/2400/smu_2450_measuring_lowr_devices.py

@@ -21,7 +21,7 @@
 with DeviceManager() as device_manager:
     print(device_manager.get_available_devices())
 
-    inst: SMU2450 = device_manager.add_smu("192.168.4.74", alias="my2450")
+    inst: SMU2450 = device_manager.add_smu("192.168.0.1", alias="my2450")
 
     # Configure the Simple Loop trigger model template to make 100 readings.
     inst.commands.trigger.model.load_simple_loop(100)

examples/source_measure_units/2400/smu_2450_rechargeable_battery.py

@@ -19,7 +19,7 @@
 with DeviceManager() as device_manager:
     print(device_manager.get_available_devices())
 
-    inst: SMU2450 = device_manager.add_smu("192.168.4.74", alias="my2450")
+    inst: SMU2450 = device_manager.add_smu("192.168.0.1", alias="my2450")
 
     # Clear the buffer.
     inst.commands.buffer_var["defbuffer1"].clear()

examples/source_measure_units/2400/smu_2450_solar_cell.py

@@ -15,7 +15,7 @@
 with DeviceManager() as device_manager:
     print(device_manager.get_available_devices())
 
-    inst: SMU2450 = device_manager.add_smu("192.168.4.74", alias="my2450")
+    inst: SMU2450 = device_manager.add_smu("192.168.0.1", alias="my2450")
 
     # Define the number of points in the sweep.
     POINTS = 56