Merge branch 'main' into docstrings

CHANGELOG.md

@@ -288,4 +288,34 @@ We modified exception messages. If your survey run generates exceptions, run `re
 
 ### Fixed
 
-- A package that was missing for working with Anthropic models.
+- A package that was missing for working with Anthropic models.
+
+
+
+## [0.1.21] - 2024-05-TBD ['In progress' indicates changes that are not live yet.]
+
+### Added 
+
+- [In progress] Prompt visibility features.
+
+- [In progress] Methods for piping responses to questions into other questions. Details TBD.
+
+- Methods for adding, sampling and shuffling `Results` objects.
+
+- Option to flatten lists of lists in `Results` - eg, responses to `QuestionList`: `results.to_list(flatten=True)`
+
+### Changed
+
+- Optional parameter `survey.run(cache=False)` if you do not want to access any cached results in running a survey.
+
+- Instructions passed to an agent at creation are now a column of results: `agent_instruction`
+
+- [In progress] `QuestionMultipleChoice` is being modified allow non-responsive answers. Previously, an error was thrown if the agent did not select one of the given options. Details TBD.
+
+### Fixed
+
+### Deprecated
+
+- [In progress] Questions method `compose_questions()` is being deprecated. Please use new piping methods instead. Details TBD.
+
+### Removed

docs/agents.rst

@@ -99,6 +99,9 @@ For example:
 
 When the agent is assigned to a survey, the special instruction will be added to the prompts for generating responses.
 
+The instructions are stored in the `instruction` field of the agent and can be accessed directly in results.
+
+
 Controlling the presentation of the persona
 -------------------------------------------
 The `traits_presentation_template` parameter can be used to create a narrative persona for an agent.

docs/data.rst

@@ -150,6 +150,20 @@ The `unset_session_cache` function is used to unset the cache for a session:
 This will unset the cache for the current session, and you will need to pass the cache object to the `run` method during the session.
 
 
+
+Avoiding cache persistence 
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+We can avoid cache persistence by passing `cache=False` to the `run` method:
+
+.. code-block:: python
+
+   from edsl import QuestionFreeText
+   q = QuestionFreeText.example()
+   results = q.run(cache = False)
+
+
+
+
 For developers
 ^^^^^^^^^^^^^^
 

docs/exceptions.rst

@@ -0,0 +1,104 @@
+.. _exceptions:
+
+Exceptions
+==========
+
+Details on exceptions raised during the execution of a survey can be found in the `Results` object that is returned when a survey is run. 
+The `Results` method `show_exceptions()` can be called to display these exceptions in a table.
+
+Here's an example of a poorly written question that is likely to raise an exception:
+
+.. code-block:: python
+
+    from edsl.questions import QuestionMultipleChoice
+
+    q = QuestionMultipleChoice(
+        question_name = "bad_instruction",
+        question_text = "What is your favorite color?",
+        question_options = ["breakfast", "lunch", "dinner"] # Non-sensical options for the question
+    )
+
+    results = q.run()
+
+
+The above code will likely raise a `QuestionAnswerValidationError` exception because the question options are not related to the question text.
+
+This is the initial exception message that will be displayed:
+
+.. code-block:: text
+
+    Exceptions were raised in the following interviews: [0]
+    The returned results have a ".show_exceptions()" attribute e.g., 
+
+    >>> results = suvey.by(agents).by(scenarios).run() 
+    >>> results.show_exceptions()
+
+    Exceptions details are available here: 
+
+    >>> from edsl import shared_globals
+    >>> shared_globals['edsl_runner_exceptions'].show_exceptions()
+
+    For more details see documentation: https://docs.expectedparrot.com/en/latest/exceptions.html
+
+
+We can then call `results.show_exceptions()` to see the details of the exceptions that were raised:
+
+.. code-block:: python
+
+    results.show_exceptions()
+
+
+This will display a table showing the question name, the exception that was raised, the time the exception was raised, and the traceback of the exception:
+
+.. code-block:: text
+
+    ┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┓
+    ┃ Question     ┃                                  ┃                    ┃                      ┃
+    ┃ name         ┃ Exception                        ┃               Time ┃ Traceback            ┃
+    ┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━┩
+    │ bad_instruc… │ QuestionAnswerValidationError('… │  1715454910.892732 │                      │
+    │              │ code must be a string, a         │                    │                      │
+    │              │ bytes-like object or a real      │                    │                      │
+    │              │ number (got This question seems  │                    │                      │
+    │              │ to be malformed. Favorite colors │                    │                      │
+    │              │ are not typically associated     │                    │                      │
+    │              │ with meals. Please provide       │                    │                      │
+    │              │ appropriate options for favorite │                    │                      │
+    │              │ colors.).')                      │                    │                      │
+    │ bad_instruc… │ QuestionAnswerValidationError("… │ 1715454913.7596118 │                      │
+    │              │ code must be a string, a         │                    │                      │
+    │              │ bytes-like object or a real      │                    │                      │
+    │              │ number (got The question is      │                    │                      │
+    │              │ about my favorite color, which   │                    │                      │
+    │              │ isn't represented by any of the  │                    │                      │
+    │              │ options provided as they are     │                    │                      │
+    │              │ meals of the day. Please provide │                    │                      │
+    │              │ color options for a valid        │                    │                      │
+    │              │ selection.).")                   │                    │                      │
+    │ bad_instruc… │ QuestionAnswerValidationError('… │   1715454917.53185 │                      │
+    │              │ code must be a string, a         │                    │                      │
+    │              │ bytes-like object or a real      │                    │                      │
+    │              │ number (got Invalid).')          │                    │                      │
+    │ bad_instruc… │ QuestionAnswerValidationError('… │  1715454923.154378 │                      │
+    │              │ code must be a string, a         │                    │                      │
+    │              │ bytes-like object or a real      │                    │                      │
+    │              │ number (got The options provided │                    │                      │
+    │              │ do not include colors, they are  │                    │                      │
+    │              │ meal times. Therefore, I cannot  │                    │                      │
+    │              │ select a favorite color from     │                    │                      │
+    │              │ these options.).')               │                    │                      │
+    │ bad_instruc… │ QuestionAnswerValidationError('… │ 1715454933.1015732 │                      │
+    │              │ code must be a string, a         │                    │                      │
+    │              │ bytes-like object or a real      │                    │                      │
+    │              │ number (got Invalid).')          │                    │                      │
+    │ bad_instruc… │ InterviewErrorPriorTaskCanceled… │  1715454933.102806 │                      │
+    │              │ tasks failed for                 │                    │                      │
+    │              │ bad_instruction')                │                    │                      │
+    └──────────────┴──────────────────────────────────┴────────────────────┴──────────────────────┘
+
+
+
+.. .. automodule:: edsl.results.Results
+..    :members: show_exceptions
+..    :undoc-members:
+..    :show-inheritance:

docs/index.rst

@@ -125,6 +125,7 @@ Information about additional functionality for developers.
    language_models
    results
    data
+   exceptions
 
 .. toctree::
    :maxdepth: 2
@@ -143,6 +144,7 @@ Information about additional functionality for developers.
    :caption: Notebooks
    :hidden:
 
+   notebooks/random_numbers.ipynb
    notebooks/data_labeling_agent.ipynb
    notebooks/concept_induction.ipynb
    notebooks/testing_training_data.ipynb