diff --git a/pep-0236.txt b/pep-0236.txt index 5aed23a5498..c19bab14aea 100644 --- a/pep-0236.txt +++ b/pep-0236.txt @@ -5,369 +5,375 @@ Last-Modified: $Date$ Author: Tim Peters Status: Final Type: Standards Track +Content-Type: text/x-rst Created: 26-Feb-2001 Python-Version: 2.1 Post-History: 26-Feb-2001 Motivation +========== - From time to time, Python makes an incompatible change to the - advertised semantics of core language constructs, or changes their - accidental (implementation-dependent) behavior in some way. While this - is never done capriciously, and is always done with the aim of - improving the language over the long term, over the short term it's - contentious and disrupting. +From time to time, Python makes an incompatible change to the advertised +semantics of core language constructs, or changes their accidental +(implementation-dependent) behavior in some way. While this is never done +capriciously, and is always done with the aim of improving the language over +the long term, over the short term it's contentious and disrupting. - PEP 5, Guidelines for Language Evolution[1] suggests ways to ease - the pain, and this PEP introduces some machinery in support of that. +PEP 5, Guidelines for Language Evolution [1]_ suggests ways to ease the pain, +and this PEP introduces some machinery in support of that. - PEP 227, Statically Nested Scopes[2] is the first application, and - will be used as an example here. +PEP 227, Statically Nested Scopes [2]_ is the first application, and will be +used as an example here. Intent +====== - [Note: This is policy, and so should eventually move into PEP 5 [1]] +[Note: This is policy, and so should eventually move into PEP 5 [1]_] - When an incompatible change to core language syntax or semantics is - being made: +When an incompatible change to core language syntax or semantics is being +made: - 1. The release C that introduces the change does not change the - syntax or semantics by default. +1. The release C that introduces the change does not change the syntax or + semantics by default. - 2. A future release R is identified in which the new syntax or semantics - will be enforced. +2. A future release R is identified in which the new syntax or semantics will + be enforced. - 3. The mechanisms described in PEP 3, Warning Framework[3] are - used to generate warnings, whenever possible, about constructs - or operations whose meaning may[4] change in release R. +3. The mechanisms described in PEP 3, Warning Framework [3]_ are used to + generate warnings, whenever possible, about constructs or operations whose + meaning may [4]_ change in release R. - 4. The new future_statement (see below) can be explicitly included in a - module M to request that the code in module M use the new syntax or - semantics in the current release C. +4. The new future_statement (see below) can be explicitly included in a module + M to request that the code in module M use the new syntax or semantics in + the current release C. - So old code continues to work by default, for at least one release, - although it may start to generate new warning messages. Migration to - the new syntax or semantics can proceed during that time, using the - future_statement to make modules containing it act as if the new syntax - or semantics were already being enforced. +So old code continues to work by default, for at least one release, although +it may start to generate new warning messages. Migration to the new syntax or +semantics can proceed during that time, using the future_statement to make +modules containing it act as if the new syntax or semantics were already being +enforced. - Note that there is no need to involve the future_statement machinery - in new features unless they can break existing code; fully backward- - compatible additions can-- and should --be introduced without a - corresponding future_statement. +Note that there is no need to involve the future_statement machinery in new +features unless they can break existing code; fully backward- compatible +additions can-- and should --be introduced without a corresponding +future_statement. Syntax +====== - A future_statement is simply a from/import statement using the reserved - module name __future__: +A future_statement is simply a from/import statement using the reserved module +name ``__future__``:: - future_statement: "from" "__future__" "import" feature ["as" name] - ("," feature ["as" name])* + future_statement: "from" "__future__" "import" feature ["as" name] + (","feature ["as" name])* - feature: identifier - name: identifier + feature: identifier + name: identifier - In addition, all future_statments must appear near the top of the - module. The only lines that can appear before a future_statement are: +In addition, all future_statments must appear near the top of the module. The +only lines that can appear before a future_statement are: - + The module docstring (if any). - + Comments. - + Blank lines. - + Other future_statements. ++ The module docstring (if any). ++ Comments. ++ Blank lines. ++ Other future_statements. - Example: - """This is a module docstring.""" +Example:: - # This is a comment, preceded by a blank line and followed by - # a future_statement. - from __future__ import nested_scopes + """This is a module docstring.""" - from math import sin - from __future__ import alabaster_weenoblobs # compile-time error! - # That was an error because preceded by a non-future_statement. + # This is a comment, preceded by a blank line and followed by + # a future_statement. + from __future__ import nested_scopes + + from math import sin + from __future__ import alabaster_weenoblobs # compile-time error! + # That was an error because preceded by a non-future_statement. Semantics +========= - A future_statement is recognized and treated specially at compile time: - changes to the semantics of core constructs are often implemented by - generating different code. It may even be the case that a new feature - introduces new incompatible syntax (such as a new reserved word), in - which case the compiler may need to parse the module differently. Such - decisions cannot be pushed off until runtime. +A future_statement is recognized and treated specially at compile time: +changes to the semantics of core constructs are often implemented by +generating different code. It may even be the case that a new feature +introduces new incompatible syntax (such as a new reserved word), in which +case the compiler may need to parse the module differently. Such decisions +cannot be pushed off until runtime. - For any given release, the compiler knows which feature names have been - defined, and raises a compile-time error if a future_statement contains - a feature not known to it[5]. +For any given release, the compiler knows which feature names have been +defined, and raises a compile-time error if a future_statement contains a +feature not known to it [5]_. - The direct runtime semantics are the same as for any import statement: - there is a standard module __future__.py, described later, and it will - be imported in the usual way at the time the future_statement is - executed. +The direct runtime semantics are the same as for any ``import`` statement: +there is a standard module ``__future__.py``, described later, and it will be +imported in the usual way at the time the future_statement is executed. - The *interesting* runtime semantics depend on the specific feature(s) - "imported" by the future_statement(s) appearing in the module. +The *interesting* runtime semantics depend on the specific feature(s) +"imported" by the future_statement(s) appearing in the module. - Note that there is nothing special about the statement: +Note that there is nothing special about the statement:: - import __future__ [as name] + import __future__ [as name] - That is not a future_statement; it's an ordinary import statement, with - no special semantics or syntax restrictions. +That is not a future_statement; it's an ordinary import statement, with no +special semantics or syntax restrictions. Example +======= - Consider this code, in file scope.py: +Consider this code, in file scope.py:: - x = 42 - def f(): - x = 666 - def g(): - print "x is", x - g() - f() + x = 42 + def f(): + x = 666 + def g(): + print "x is", x + g() + f() - Under 2.0, it prints: +Under 2.0, it prints:: - x is 42 + x is 42 - Nested scopes[2] are being introduced in 2.1. But under 2.1, it still - prints +Nested scopes [2]_ are being introduced in 2.1. But under 2.1, it still +prints:: - x is 42 + x is 42 - and also generates a warning. +and also generates a warning. - In 2.2, and also in 2.1 *if* "from __future__ import nested_scopes" is - included at the top of scope.py, it prints +In 2.2, and also in 2.1 *if* ``from __future__ import nested_scopes`` is +included at the top of ``scope.py``, it prints:: - x is 666 + x is 666 Standard Module __future__.py +============================= - Lib/__future__.py is a real module, and serves three purposes: +``Lib/__future__.py`` is a real module, and serves three purposes: - 1. To avoid confusing existing tools that analyze import statements and - expect to find the modules they're importing. +1. To avoid confusing existing tools that analyze import statements and expect + to find the modules they're importing. - 2. To ensure that future_statements run under releases prior to 2.1 - at least yield runtime exceptions (the import of __future__ will - fail, because there was no module of that name prior to 2.1). +2. To ensure that future_statements run under releases prior to 2.1 at least + yield runtime exceptions (the import of ``__future__`` will fail, because + there was no module of that name prior to 2.1). - 3. To document when incompatible changes were introduced, and when they - will be-- or were --made mandatory. This is a form of executable - documentation, and can be inspected programatically via importing - __future__ and examining its contents. +3. To document when incompatible changes were introduced, and when they will + be-- or were --made mandatory. This is a form of executable documentation, + and can be inspected programatically via importing ``__future__`` and + examining its contents. - Each statement in __future__.py is of the form: +Each statement in ``__future__.py`` is of the form:: - FeatureName = "_Feature(" OptionalRelease "," MandatoryRelease ")" + FeatureName = "_Feature(" OptionalRelease "," MandatoryRelease ")" - where, normally, OptionalRelease < MandatoryRelease, and both are - 5-tuples of the same form as sys.version_info: +where, normally, *OptionalRelease* < *MandatoryRelease*, and both are +5-tuples of the same form as ``sys.version_info``:: (PY_MAJOR_VERSION, # the 2 in 2.1.0a3; an int PY_MINOR_VERSION, # the 1; an int PY_MICRO_VERSION, # the 0; an int PY_RELEASE_LEVEL, # "alpha", "beta", "candidate" or "final"; string - PY_RELEASE_SERIAL # the 3; an int - ) + PY_RELEASE_SERIAL # the 3; an int ) - OptionalRelease records the first release in which +*OptionalRelease* records the first release in which:: - from __future__ import FeatureName + from __future__ import FeatureName - was accepted. +was accepted. - In the case of MandatoryReleases that have not yet occurred, - MandatoryRelease predicts the release in which the feature will become - part of the language. +In the case of *MandatoryReleases* that have not yet occurred, +*MandatoryRelease* predicts the release in which the feature will become part +of the language. - Else MandatoryRelease records when the feature became part of the - language; in releases at or after that, modules no longer need +Else *MandatoryRelease* records when the feature became part of the language; +in releases at or after that, modules no longer need:: - from __future__ import FeatureName + from __future__ import FeatureName - to use the feature in question, but may continue to use such imports. +to use the feature in question, but may continue to use such imports. - MandatoryRelease may also be None, meaning that a planned feature got - dropped. +*MandatoryRelease* may also be ``None``, meaning that a planned feature got +dropped. - Instances of class _Feature have two corresponding methods, - .getOptionalRelease() and .getMandatoryRelease(). +Instances of ``class _Feature`` have two corresponding methods, +``.getOptionalRelease()`` and ``.getMandatoryRelease()``. - No feature line will ever be deleted from __future__.py. +No feature line will ever be deleted from ``__future__.py``. - Example line: +Example line:: - nested_scopes = _Feature((2, 1, 0, "beta", 1), (2, 2, 0, "final", 0)) + nested_scopes = _Feature((2, 1, 0, "beta", 1), (2, 2, 0, "final", 0)) - This means that +This means that:: - from __future__ import nested_scopes + from __future__ import nested_scopes - will work in all releases at or after 2.1b1, and that nested_scopes are - intended to be enforced starting in release 2.2. +will work in all releases at or after 2.1b1, and that nested_scopes are +intended to be enforced starting in release 2.2. Resolved Problem: Runtime Compilation +====================================== - Several Python features can compile code during a module's runtime: +Several Python features can compile code during a module's runtime: - 1. The exec statement. - 2. The execfile() function. - 3. The compile() function. - 4. The eval() function. - 5. The input() function. +1. The ``exec`` statement. +2. The ``execfile()`` function. +3. The ``compile()`` function. +4. The ``eval()`` function. +5. The ``input()`` function. - Since a module M containing a future_statement naming feature F - explicitly requests that the current release act like a future release - with respect to F, any code compiled dynamically from text passed to - one of these from within M should probably also use the new syntax or - semantics associated with F. The 2.1 release does behave this way. +Since a module M containing a future_statement naming feature F explicitly +requests that the current release act like a future release with respect to F, +any code compiled dynamically from text passed to one of these from within M +should probably also use the new syntax or semantics associated with F. The +2.1 release does behave this way. - This isn't always desired, though. For example, doctest.testmod(M) - compiles examples taken from strings in M, and those examples should - use M's choices, not necessarily the doctest module's choices. In the - 2.1 release, this isn't possible, and no scheme has yet been suggested - for working around this. NOTE: PEP 264 later addressed this in a - flexible way, by adding optional arguments to compile(). +This isn't always desired, though. For example, ``doctest.testmod(M)`` +compiles examples taken from strings in M, and those examples should use M's +choices, not necessarily the doctest module's choices. In the 2.1 release, +this isn't possible, and no scheme has yet been suggested for working around +this. NOTE: PEP 264 later addressed this in a flexible way, by adding +optional arguments to ``compile()``. - In any case, a future_statement appearing "near the top" (see Syntax - above) of text compiled dynamically by an exec, execfile() or compile() - applies to the code block generated, but has no further effect on the - module that executes such an exec, execfile() or compile(). This - can't be used to affect eval() or input(), however, because they only - allow expression input, and a future_statement is not an expression. +In any case, a future_statement appearing "near the top" (see Syntax above) of +text compiled dynamically by an ``exec``, ``execfile()`` or ``compile()`` +applies to the code block generated, but has no further effect on the module +that executes such an ``exec``, ``execfile()`` or ``compile()``. This can't +be used to affect ``eval()`` or ``input()``, however, because they only allow +expression input, and a future_statement is not an expression. Resolved Problem: Native Interactive Shells +============================================ - There are two ways to get an interactive shell: +There are two ways to get an interactive shell: - 1. By invoking Python from a command line without a script argument. +1. By invoking Python from a command line without a script argument. - 2. By invoking Python from a command line with the -i switch and with a - script argument. +2. By invoking Python from a command line with the ``-i`` switch and with a + script argument. - An interactive shell can be seen as an extreme case of runtime - compilation (see above): in effect, each statement typed at an - interactive shell prompt runs a new instance of exec, compile() or - execfile(). A future_statement typed at an interactive shell applies to - the rest of the shell session's life, as if the future_statement had - appeared at the top of a module. +An interactive shell can be seen as an extreme case of runtime compilation +(see above): in effect, each statement typed at an interactive shell prompt +runs a new instance of ``exec``, ``compile()`` or ``execfile()``. A +future_statement typed at an interactive shell applies to the rest of the +shell session's life, as if the future_statement had appeared at the top of a +module. Resolved Problem: Simulated Interactive Shells +=============================================== - Interactive shells "built by hand" (by tools such as IDLE and the Emacs - Python-mode) should behave like native interactive shells (see above). - However, the machinery used internally by native interactive shells has - not been exposed, and there isn't a clear way for tools building their - own interactive shells to achieve the desired behavior. +Interactive shells "built by hand" (by tools such as IDLE and the Emacs +Python-mode) should behave like native interactive shells (see above). +However, the machinery used internally by native interactive shells has not +been exposed, and there isn't a clear way for tools building their own +interactive shells to achieve the desired behavior. - NOTE: PEP 264 later addressed this, by adding intelligence to the - standard codeop.py. Simulated shells that don't use the standard - library shell helpers can get a similar effect by exploiting the - new optional arguments to compile() added by PEP 264. +NOTE: PEP 264 later addressed this, by adding intelligence to the standard +``codeop.py``. Simulated shells that don't use the standard library shell +helpers can get a similar effect by exploiting the new optional arguments to +``compile()`` added by PEP 264. Questions and Answers +===================== - Q: What about a "from __past__" version, to get back *old* behavior? +What about a "from __past__" version, to get back *old* behavior? +----------------------------------------------------------------- - A: Outside the scope of this PEP. Seems unlikely to the author, - though. Write a PEP if you want to pursue it. +Outside the scope of this PEP. Seems unlikely to the author, though. Write a +PEP if you want to pursue it. - Q: What about incompatibilities due to changes in the Python virtual - machine? +What about incompatibilities due to changes in the Python virtual machine? +-------------------------------------------------------------------------- - A: Outside the scope of this PEP, although PEP 5 [1] suggests a grace - period there too, and the future_statement may also have a role to - play there. +Outside the scope of this PEP, although PEP 5 [1]_ suggests a grace period +there too, and the future_statement may also have a role to play there. - Q: What about incompatibilities due to changes in Python's C API? +What about incompatibilities due to changes in Python's C API? +-------------------------------------------------------------- - A: Outside the scope of this PEP. +Outside the scope of this PEP. - Q: I want to wrap future_statements in try/except blocks, so I can - use different code depending on which version of Python I'm running. - Why can't I? +I want to wrap future_statements in try/except blocks, so I can use different code depending on which version of Python I'm running. Why can't I? +------------------------------------------------------------------------------------------------------------------------------------------------- - A: Sorry! try/except is a runtime feature; future_statements are - primarily compile-time gimmicks, and your try/except happens long - after the compiler is done. That is, by the time you do - try/except, the semantics in effect for the module are already a - done deal. Since the try/except wouldn't accomplish what it - *looks* like it should accomplish, it's simply not allowed. We - also want to keep these special statements very easy to find and to - recognize. +Sorry! ``try/except`` is a runtime feature; future_statements are primarily +compile-time gimmicks, and your ``try/except`` happens long after the compiler +is done. That is, by the time you do ``try/except``, the semantics in effect +for the module are already a done deal. Since the ``try/except`` wouldn't +accomplish what it *looks* like it should accomplish, it's simply not allowed. +We also want to keep these special statements very easy to find and to +recognize. - Note that you *can* import __future__ directly, and use the - information in it, along with sys.version_info, to figure out where - the release you're running under stands in relation to a given - feature's status. +Note that you *can* import ``__future__`` directly, and use the information in +it, along with ``sys.version_info``, to figure out where the release you're +running under stands in relation to a given feature's status. - Q: Going back to the nested_scopes example, what if release 2.2 comes - along and I still haven't changed my code? How can I keep the 2.1 - behavior then? +Going back to the nested_scopes example, what if release 2.2 comes along and I still haven't changed my code? How can I keep the 2.1 behavior then? +---------------------------------------------------------------------------------------------------------------------------------------------------- - A: By continuing to use 2.1, and not moving to 2.2 until you do change - your code. The purpose of future_statement is to make life easier - for people who keep current with the latest release in a timely - fashion. We don't hate you if you don't, but your problems are - much harder to solve, and somebody with those problems will need to - write a PEP addressing them. future_statement is aimed at a - different audience. +By continuing to use 2.1, and not moving to 2.2 until you do change your +code. The purpose of future_statement is to make life easier for people who +keep current with the latest release in a timely fashion. We don't hate you +if you don't, but your problems are much harder to solve, and somebody with +those problems will need to write a PEP addressing them. future_statement is +aimed at a different audience. - Q: Overloading "import" sucks. Why not introduce a new statement for - this? +Overloading ``import`` sucks. Why not introduce a new statement for this? +-------------------------------------------------------------------------- - A: Like maybe "lambda lambda nested_scopes"? That is, unless we - introduce a new keyword, we can't introduce an entirely new - statement. But if we introduce a new keyword, that in itself - would break old code. That would be too ironic to bear. Yes, - overloading "import" does suck, but not as energetically as the - alternatives -- as is, future_statements are 100% backward - compatible. +Like maybe ``lambda lambda nested_scopes``? That is, unless we introduce a +new keyword, we can't introduce an entirely new statement. But if we +introduce a new keyword, that in itself would break old code. That would be +too ironic to bear. Yes, overloading ``import`` does suck, but not as +energetically as the alternatives -- as is, future_statements are 100% +backward compatible. Copyright +========= - This document has been placed in the public domain. +This document has been placed in the public domain. References and Footnotes +======================== + +.. [1] PEP 5, Guidelines for Language Evolution, Prescod + http://www.python.org/dev/peps/pep-0005/ - [1] PEP 5, Guidelines for Language Evolution, Prescod - http://www.python.org/dev/peps/pep-0005/ +.. [2] PEP 227, Statically Nested Scopes, Hylton + http://www.python.org/dev/peps/pep-0227/ - [2] PEP 227, Statically Nested Scopes, Hylton - http://www.python.org/dev/peps/pep-0227/ +.. [3] PEP 230, Warning Framework, Van Rossum + http://www.python.org/dev/peps/pep-0230/ - [3] PEP 230, Warning Framework, Van Rossum - http://www.python.org/dev/peps/pep-0230/ +.. [4] Note that this is *may* and not *will*: better safe than sorry. Of course + spurious warnings won't be generated when avoidable with reasonable cost. - [4] Note that this is "may" and not "will": better safe than sorry. Of - course spurious warnings won't be generated when avoidable with - reasonable cost. +.. [5] This ensures that a future_statement run under a release prior to the + first one in which a given feature is known (but >= 2.1) will raise a + compile-time error rather than silently do a wrong thing. If transported + to a release prior to 2.1, a runtime error will be raised because of the + failure to import ``__future__`` (no such module existed in the standard + distribution before the 2.1 release, and the double underscores make it a + reserved name). - [5] This ensures that a future_statement run under a release prior to - the first one in which a given feature is known (but >= 2.1) will - raise a compile-time error rather than silently do a wrong thing. - If transported to a release prior to 2.1, a runtime error will be - raised because of the failure to import __future__ (no such module - existed in the standard distribution before the 2.1 release, and - the double underscores make it a reserved name). - -Local Variables: -mode: indented-text -indent-tabs-mode: nil -End: +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + End: diff --git a/pep-0247.txt b/pep-0247.txt index 104a8216c50..97b4c4f2ad1 100644 --- a/pep-0247.txt +++ b/pep-0247.txt @@ -5,169 +5,169 @@ Last-Modified: $Date$ Author: A.M. Kuchling Status: Final Type: Informational +Content-Type: text/x-rst Created: 23-Mar-2001 Post-History: 20-Sep-2001 + Abstract +======== - There are several different modules available that implement - cryptographic hashing algorithms such as MD5 or SHA. This - document specifies a standard API for such algorithms, to make it - easier to switch between different implementations. +There are several different modules available that implement cryptographic +hashing algorithms such as MD5 or SHA. This document specifies a standard API +for such algorithms, to make it easier to switch between different +implementations. Specification +============= - All hashing modules should present the same interface. Additional - methods or variables can be added, but those described in this - document should always be present. +All hashing modules should present the same interface. Additional methods or +variables can be added, but those described in this document should always be +present. - Hash function modules define one function: +Hash function modules define one function: - new([string]) (unkeyed hashes) - new([key] , [string]) (keyed hashes) +| ``new([string]) (unkeyed hashes)`` +| ``new([key] , [string]) (keyed hashes)`` - Create a new hashing object and return it. The first form is - for hashes that are unkeyed, such as MD5 or SHA. For keyed - hashes such as HMAC, 'key' is a required parameter containing - a string giving the key to use. In both cases, the optional - 'string' parameter, if supplied, will be immediately hashed - into the object's starting state, as if obj.update(string) was - called. + Create a new hashing object and return it. The first form is for hashes + that are unkeyed, such as MD5 or SHA. For keyed hashes such as HMAC, *key* + is a required parameter containing a string giving the key to use. In both + cases, the optional *string* parameter, if supplied, will be immediately + hashed into the object's starting state, as if ``obj.update(string)`` + was called. - After creating a hashing object, arbitrary strings can be fed - into the object using its update() method, and the hash value - can be obtained at any time by calling the object's digest() - method. + After creating a hashing object, arbitrary strings can be fed into the + object using its ``update()`` method, and the hash value can be obtained at + any time by calling the object's ``digest()`` method. - Arbitrary additional keyword arguments can be added to this - function, but if they're not supplied, sensible default values - should be used. For example, 'rounds' and 'digest_size' - keywords could be added for a hash function which supports a - variable number of rounds and several different output sizes, - and they should default to values believed to be secure. + Arbitrary additional keyword arguments can be added to this function, but if + they're not supplied, sensible default values should be used. For example, + ``rounds`` and ``digest_size`` keywords could be added for a hash function + which supports a variable number of rounds and several different output + sizes, and they should default to values believed to be secure. - Hash function modules define one variable: +Hash function modules define one variable: - digest_size +| ``digest_size`` - An integer value; the size of the digest produced by the - hashing objects created by this module, measured in bytes. - You could also obtain this value by creating a sample object - and accessing its 'digest_size' attribute, but it can be - convenient to have this value available from the module. - Hashes with a variable output size will set this variable to - None. + An integer value; the size of the digest produced by the hashing objects + created by this module, measured in bytes. You could also obtain this value + by creating a sample object and accessing its ``digest_size`` attribute, but + it can be convenient to have this value available from the module. Hashes + with a variable output size will set this variable to ``None``. - Hashing objects require a single attribute: +Hashing objects require a single attribute: - digest_size +| ``digest_size`` - This attribute is identical to the module-level digest_size - variable, measuring the size of the digest produced by the - hashing object, measured in bytes. If the hash has a variable - output size, this output size must be chosen when the hashing - object is created, and this attribute must contain the - selected size. Therefore, None is *not* a legal value for this - attribute. + This attribute is identical to the module-level ``digest_size`` variable, + measuring the size of the digest produced by the hashing object, measured in + bytes. If the hash has a variable output size, this output size must be + chosen when the hashing object is created, and this attribute must contain + the selected size. Therefore, ``None`` is *not* a legal value for this + attribute. - Hashing objects require the following methods: +Hashing objects require the following methods: - copy() +| ``copy()`` - Return a separate copy of this hashing object. An update to - this copy won't affect the original object. + Return a separate copy of this hashing object. An update to this copy won't + affect the original object. - digest() +| ``digest()`` - Return the hash value of this hashing object as a string - containing 8-bit data. The object is not altered in any way - by this function; you can continue updating the object after - calling this function. + Return the hash value of this hashing object as a string containing 8-bit + data. The object is not altered in any way by this function; you can + continue updating the object after calling this function. - hexdigest() +| ``hexdigest()`` - Return the hash value of this hashing object as a string - containing hexadecimal digits. Lowercase letters should be used - for the digits 'a' through 'f'. Like the .digest() method, this - method mustn't alter the object. + Return the hash value of this hashing object as a string containing + hexadecimal digits. Lowercase letters should be used for the digits ``a`` + through ``f``. Like the ``.digest()`` method, this method mustn't alter the + object. - update(string) +| ``update(string)`` - Hash 'string' into the current state of the hashing object. - update() can be called any number of times during a hashing - object's lifetime. + Hash *string* into the current state of the hashing object. ``update()`` can + be called any number of times during a hashing object's lifetime. - Hashing modules can define additional module-level functions or - object methods and still be compliant with this specification. +Hashing modules can define additional module-level functions or object methods +and still be compliant with this specification. - Here's an example, using a module named 'MD5': +Here's an example, using a module named ``MD5``:: - >>> from Crypto.Hash import MD5 - >>> m = MD5.new() - >>> m.digest_size - 16 - >>> m.update('abc') - >>> m.digest() - '\x90\x01P\x98<\xd2O\xb0\xd6\x96?}(\xe1\x7fr' - >>> m.hexdigest() - '900150983cd24fb0d6963f7d28e17f72' - >>> MD5.new('abc').digest() - '\x90\x01P\x98<\xd2O\xb0\xd6\x96?}(\xe1\x7fr' + >>> from Crypto.Hash import MD5 + >>> m = MD5.new() + >>> m.digest_size + 16 + >>> m.update('abc') + >>> m.digest() + '\x90\x01P\x98<\xd2O\xb0\xd6\x96?}(\xe1\x7fr' + >>> m.hexdigest() + '900150983cd24fb0d6963f7d28e17f72' + >>> MD5.new('abc').digest() + '\x90\x01P\x98<\xd2O\xb0\xd6\x96?}(\xe1\x7fr' Rationale - - The digest size is measured in bytes, not bits, even though hash - algorithm sizes are usually quoted in bits; MD5 is a 128-bit - algorithm and not a 16-byte one, for example. This is because, in - the sample code I looked at, the length in bytes is often needed - (to seek ahead or behind in a file; to compute the length of an - output string) while the length in bits is rarely used. - Therefore, the burden will fall on the few people actually needing - the size in bits, who will have to multiply digest_size by 8. - - It's been suggested that the update() method would be better named - append(). However, that method is really causing the current - state of the hashing object to be updated, and update() is already - used by the md5 and sha modules included with Python, so it seems - simplest to leave the name update() alone. - - The order of the constructor's arguments for keyed hashes was a - sticky issue. It wasn't clear whether the key should come first - or second. It's a required parameter, and the usual convention is - to place required parameters first, but that also means that the - 'string' parameter moves from the first position to the second. - It would be possible to get confused and pass a single argument to - a keyed hash, thinking that you're passing an initial string to an - unkeyed hash, but it doesn't seem worth making the interface - for keyed hashes more obscure to avoid this potential error. +========= + +The digest size is measured in bytes, not bits, even though hash algorithm +sizes are usually quoted in bits; MD5 is a 128-bit algorithm and not a 16-byte +one, for example. This is because, in the sample code I looked at, the length +in bytes is often needed (to seek ahead or behind in a file; to compute the +length of an output string) while the length in bits is rarely used. Therefore, +the burden will fall on the few people actually needing the size in bits, who +will have to multiply ``digest_size`` by 8. + +It's been suggested that the ``update()`` method would be better named +``append()``. However, that method is really causing the current state of the +hashing object to be updated, and ``update()`` is already used by the md5 and +sha modules included with Python, so it seems simplest to leave the name +``update()`` alone. + +The order of the constructor's arguments for keyed hashes was a sticky issue. +It wasn't clear whether the *key* should come first or second. It's a required +parameter, and the usual convention is to place required parameters first, but +that also means that the *string* parameter moves from the first position to +the second. It would be possible to get confused and pass a single argument to +a keyed hash, thinking that you're passing an initial string to an unkeyed +hash, but it doesn't seem worth making the interface for keyed hashes more +obscure to avoid this potential error. Changes +======= + +2001-09-17: Renamed ``clear()`` to ``reset()``; added ``digest_size`` attribute +to objects; added ``.hexdigest()`` method. - 2001-09-17: Renamed clear() to reset(); added digest_size attribute - to objects; added .hexdigest() method. - 2001-09-20: Removed reset() method completely. - 2001-09-28: Set digest_size to None for variable-size hashes. +2001-09-20: Removed ``reset()`` method completely. + +2001-09-28: Set ``digest_size`` to ``None`` for variable-size hashes. Acknowledgements +================ - Thanks to Aahz, Andrew Archibald, Rich Salz, Itamar - Shtull-Trauring, and the readers of the python-crypto list for - their comments on this PEP. +Thanks to Aahz, Andrew Archibald, Rich Salz, Itamar Shtull-Trauring, and the +readers of the python-crypto list for their comments on this PEP. Copyright +========= + +This document has been placed in the public domain. - This document has been placed in the public domain. - -Local Variables: -mode: indented-text -indent-tabs-mode: nil -End: +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + End: diff --git a/pep-0342.txt b/pep-0342.txt index c0fbb41ff28..ad2a795d0b1 100644 --- a/pep-0342.txt +++ b/pep-0342.txt @@ -5,591 +5,604 @@ Last-Modified: $Date$ Author: Guido van Rossum, Phillip J. Eby Status: Final Type: Standards Track -Content-Type: text/plain +Content-Type: text/x-rst Created: 10-May-2005 Python-Version: 2.5 Post-History: + Introduction +============ + +This PEP proposes some enhancements to the API and syntax of generators, to +make them usable as simple coroutines. It is basically a combination of ideas +from these two PEPs, which may be considered redundant if this PEP is +accepted: - This PEP proposes some enhancements to the API and syntax of - generators, to make them usable as simple coroutines. It is - basically a combination of ideas from these two PEPs, which - may be considered redundant if this PEP is accepted: +- PEP 288, Generators Attributes and Exceptions. The current PEP covers its + second half, generator exceptions (in fact the ``throw()`` method name was + taken from PEP 288). PEP 342 replaces generator attributes, however, with a + concept from an earlier revision of PEP 288, the *yield expression*. - - PEP 288, Generators Attributes and Exceptions. The current PEP - covers its second half, generator exceptions (in fact the - throw() method name was taken from PEP 288). PEP 342 replaces - generator attributes, however, with a concept from an earlier - revision of PEP 288, the "yield expression". +- PEP 325, Resource-Release Support for Generators. PEP 342 ties up a few + loose ends in the PEP 325 spec, to make it suitable for actual + implementation. - - PEP 325, Resource-Release Support for Generators. PEP 342 - ties up a few loose ends in the PEP 325 spec, to make it suitable - for actual implementation. Motivation +========== + +Coroutines are a natural way of expressing many algorithms, such as +simulations, games, asynchronous I/O, and other forms of event-driven +programming or co-operative multitasking. Python's generator functions are +almost coroutines -- but not quite -- in that they allow pausing execution to +produce a value, but do not provide for values or exceptions to be passed in +when execution resumes. They also do not allow execution to be paused within +the ``try`` portion of ``try/finally`` blocks, and therefore make it difficult +for an aborted coroutine to clean up after itself. + +Also, generators cannot yield control while other functions are executing, +unless those functions are themselves expressed as generators, and the outer +generator is written to yield in response to values yielded by the inner +generator. This complicates the implementation of even relatively simple use +cases like asynchronous communications, because calling any functions either +requires the generator to *block* (i.e. be unable to yield control), or else a +lot of boilerplate looping code must be added around every needed function +call. + +However, if it were possible to pass values or exceptions *into* a generator at +the point where it was suspended, a simple co-routine scheduler or *trampoline +function* would let coroutines *call* each other without blocking -- a +tremendous boon for asynchronous applications. Such applications could then +write co-routines to do non-blocking socket I/O by yielding control to an I/O +scheduler until data has been sent or becomes available. Meanwhile, code that +performs the I/O would simply do something like this:: + + data = (yield nonblocking_read(my_socket, nbytes)) + +in order to pause execution until the ``nonblocking_read()`` coroutine produced +a value. + +In other words, with a few relatively minor enhancements to the language and to +the implementation of the generator-iterator type, Python will be able to +support performing asynchronous operations without needing to write the entire +application as a series of callbacks, and without requiring the use of +resource-intensive threads for programs that need hundreds or even thousands of +co-operatively multitasking pseudothreads. Thus, these enhancements will give +standard Python many of the benefits of the Stackless Python fork, without +requiring any significant modification to the CPython core or its APIs. In +addition, these enhancements should be readily implementable by any Python +implementation (such as Jython) that already supports generators. - Coroutines are a natural way of expressing many algorithms, such as - simulations, games, asynchronous I/O, and other forms of event- - driven programming or co-operative multitasking. Python's generator - functions are almost coroutines -- but not quite -- in that they - allow pausing execution to produce a value, but do not provide for - values or exceptions to be passed in when execution resumes. They - also do not allow execution to be paused within the "try" portion of - try/finally blocks, and therefore make it difficult for an aborted - coroutine to clean up after itself. - - Also, generators cannot yield control while other functions are - executing, unless those functions are themselves expressed as - generators, and the outer generator is written to yield in response - to values yielded by the inner generator. This complicates the - implementation of even relatively simple use cases like asynchronous - communications, because calling any functions either requires the - generator to "block" (i.e. be unable to yield control), or else a - lot of boilerplate looping code must be added around every needed - function call. - - However, if it were possible to pass values or exceptions *into* a - generator at the point where it was suspended, a simple co-routine - scheduler or "trampoline function" would let coroutines "call" each - other without blocking -- a tremendous boon for asynchronous - applications. Such applications could then write co-routines to - do non-blocking socket I/O by yielding control to an I/O scheduler - until data has been sent or becomes available. Meanwhile, code that - performs the I/O would simply do something like this: - - data = (yield nonblocking_read(my_socket, nbytes)) - - in order to pause execution until the nonblocking_read() coroutine - produced a value. - - In other words, with a few relatively minor enhancements to the - language and to the implementation of the generator-iterator type, - Python will be able to support performing asynchronous operations - without needing to write the entire application as a series of - callbacks, and without requiring the use of resource-intensive threads - for programs that need hundreds or even thousands of co-operatively - multitasking pseudothreads. Thus, these enhancements will give - standard Python many of the benefits of the Stackless Python fork, - without requiring any significant modification to the CPython core - or its APIs. In addition, these enhancements should be readily - implementable by any Python implementation (such as Jython) that - already supports generators. Specification Summary - - By adding a few simple methods to the generator-iterator type, and - with two minor syntax adjustments, Python developers will be able - to use generator functions to implement co-routines and other forms - of co-operative multitasking. These methods and adjustments are: - - 1. Redefine "yield" to be an expression, rather than a statement. - The current yield statement would become a yield expression - whose value is thrown away. A yield expression's value is - None whenever the generator is resumed by a normal next() call. - - 2. Add a new send() method for generator-iterators, which resumes - the generator and "sends" a value that becomes the result of the - current yield-expression. The send() method returns the next - value yielded by the generator, or raises StopIteration if the - generator exits without yielding another value. - - 3. Add a new throw() method for generator-iterators, which raises - an exception at the point where the generator was paused, and - which returns the next value yielded by the generator, raising - StopIteration if the generator exits without yielding another - value. (If the generator does not catch the passed-in exception, - or raises a different exception, then that exception propagates - to the caller.) - - 4. Add a close() method for generator-iterators, which raises - GeneratorExit at the point where the generator was paused. If - the generator then raises StopIteration (by exiting normally, or - due to already being closed) or GeneratorExit (by not catching - the exception), close() returns to its caller. If the generator - yields a value, a RuntimeError is raised. If the generator - raises any other exception, it is propagated to the caller. - close() does nothing if the generator has already exited due to - an exception or normal exit. - - 5. Add support to ensure that close() is called when a generator - iterator is garbage-collected. - - 6. Allow "yield" to be used in try/finally blocks, since garbage - collection or an explicit close() call would now allow the - finally clause to execute. - - A prototype patch implementing all of these changes against the - current Python CVS HEAD is available as SourceForge patch #1223381 - (http://python.org/sf/1223381). +===================== + +By adding a few simple methods to the generator-iterator type, and with two +minor syntax adjustments, Python developers will be able to use generator +functions to implement co-routines and other forms of co-operative +multitasking. These methods and adjustments are: + +1. Redefine ``yield`` to be an expression, rather than a statement. The current + yield statement would become a yield expression whose value is thrown away. + A yield expression's value is ``None`` whenever the generator is resumed by + a normal ``next()`` call. + +2. Add a new ``send()`` method for generator-iterators, which resumes the + generator and *sends* a value that becomes the result of the current + yield-expression. The ``send()`` method returns the next value yielded by + the generator, or raises ``StopIteration`` if the generator exits without + yielding another value. + +3. Add a new ``throw()`` method for generator-iterators, which raises an + exception at the point where the generator was paused, and which returns the + next value yielded by the generator, raising ``StopIteration`` if the + generator exits without yielding another value. (If the generator does not + catch the passed-in exception, or raises a different exception, then that + exception propagates to the caller.) + +4. Add a ``close()`` method for generator-iterators, which raises + ``GeneratorExit`` at the point where the generator was paused. If the + generator then raises ``StopIteration`` (by exiting normally, or due to + already being closed) or ``GeneratorExit`` (by not catching the exception), + ``close()`` returns to its caller. If the generator yields a value, a + ``RuntimeError`` is raised. If the generator raises any other exception, it + is propagated to the caller. ``close()`` does nothing if the generator has + already exited due to an exception or normal exit. + +5. Add support to ensure that ``close()`` is called when a generator iterator + is garbage-collected. + +6. Allow ``yield`` to be used in ``try/finally`` blocks, since garbage + collection or an explicit ``close()`` call would now allow the ``finally`` + clause to execute. + +A prototype patch implementing all of these changes against the current Python +CVS HEAD is available as SourceForge patch #1223381 +(http://python.org/sf/1223381). Specification: Sending Values into Generators +============================================= + +New generator method: ``send(value)`` +------------------------------------- + +A new method for generator-iterators is proposed, called ``send()``. It +takes exactly one argument, which is the value that should be *sent in* to +the generator. Calling ``send(None)`` is exactly equivalent to calling a +generator's ``next()`` method. Calling ``send()`` with any other value is +the same, except that the value produced by the generator's current +yield expression will be different. + +Because generator-iterators begin execution at the top of the generator's +function body, there is no yield expression to receive a value when the +generator has just been created. Therefore, calling ``send()`` with a +non-``None`` argument is prohibited when the generator iterator has just +started, and a ``TypeError`` is raised if this occurs (presumably due to a +logic error of some kind). Thus, before you can communicate with a +coroutine you must first call ``next()`` or ``send(None)`` to advance its +execution to the first yield expression. + +As with the ``next()`` method, the ``send()`` method returns the next value +yielded by the generator-iterator, or raises ``StopIteration`` if the +generator exits normally, or has already exited. If the generator raises an +uncaught exception, it is propagated to ``send()``'s caller. + +New syntax: Yield Expressions +----------------------------- + +The yield-statement will be allowed to be used on the right-hand side of an +assignment; in that case it is referred to as yield-expression. The value +of this yield-expression is ``None`` unless ``send()`` was called with a +non-``None`` argument; see below. + +A yield-expression must always be parenthesized except when it occurs at the +top-level expression on the right-hand side of an assignment. So + +:: + + x = yield 42 + x = yield + x = 12 + (yield 42) + x = 12 + (yield) + foo(yield 42) + foo(yield) + +are all legal, but + +:: + + x = 12 + yield 42 + x = 12 + yield + foo(yield 42, 12) + foo(yield, 12) + +are all illegal. (Some of the edge cases are motivated by the current +legality of ``yield 12, 42``.) + +Note that a yield-statement or yield-expression without an expression is now +legal. This makes sense: when the information flow in the ``next()`` call +is reversed, it should be possible to yield without passing an explicit +value (``yield`` is of course equivalent to ``yield None``). + +When ``send(value)`` is called, the yield-expression that it resumes will +return the passed-in value. When ``next()`` is called, the resumed +yield-expression will return ``None``. If the yield-expression is a +yield-statement, this returned value is ignored, similar to ignoring the +value returned by a function call used as a statement. + +In effect, a yield-expression is like an inverted function call; the +argument to yield is in fact returned (yielded) from the currently executing +function, and the *return value* of yield is the argument passed in via +``send()``. + +Note: the syntactic extensions to yield make its use very similar to that in +Ruby. This is intentional. Do note that in Python the block passes a value +to the generator using ``send(EXPR)`` rather than ``return EXPR``, and the +underlying mechanism whereby control is passed between the generator and the +block is completely different. Blocks in Python are not compiled into +thunks; rather, ``yield`` suspends execution of the generator's frame. Some +edge cases work differently; in Python, you cannot save the block for later +use, and you cannot test whether there is a block or not. (XXX - this stuff +about blocks seems out of place now, perhaps Guido can edit to clarify.) - New generator method: send(value) - - A new method for generator-iterators is proposed, called send(). It - takes exactly one argument, which is the value that should be "sent - in" to the generator. Calling send(None) is exactly equivalent to - calling a generator's next() method. Calling send() with any other - value is the same, except that the value produced by the generator's - current yield expression will be different. - - Because generator-iterators begin execution at the top of the - generator's function body, there is no yield expression to receive - a value when the generator has just been created. Therefore, - calling send() with a non-None argument is prohibited when the - generator iterator has just started, and a TypeError is raised if - this occurs (presumably due to a logic error of some kind). Thus, - before you can communicate with a coroutine you must first call - next() or send(None) to advance its execution to the first yield - expression. - - As with the next() method, the send() method returns the next value - yielded by the generator-iterator, or raises StopIteration if the - generator exits normally, or has already exited. If the generator - raises an uncaught exception, it is propagated to send()'s caller. - - New syntax: Yield Expressions - - The yield-statement will be allowed to be used on the right-hand - side of an assignment; in that case it is referred to as - yield-expression. The value of this yield-expression is None - unless send() was called with a non-None argument; see below. - - A yield-expression must always be parenthesized except when it - occurs at the top-level expression on the right-hand side of an - assignment. So - - x = yield 42 - x = yield - x = 12 + (yield 42) - x = 12 + (yield) - foo(yield 42) - foo(yield) - - are all legal, but - - x = 12 + yield 42 - x = 12 + yield - foo(yield 42, 12) - foo(yield, 12) - - are all illegal. (Some of the edge cases are motivated by the - current legality of "yield 12, 42".) - - Note that a yield-statement or yield-expression without an - expression is now legal. This makes sense: when the information - flow in the next() call is reversed, it should be possible to - yield without passing an explicit value ("yield" is of course - equivalent to "yield None"). - - When send(value) is called, the yield-expression that it resumes - will return the passed-in value. When next() is called, the resumed - yield-expression will return None. If the yield-expression is a - yield-statement, this returned value is ignored, similar to ignoring - the value returned by a function call used as a statement. - - In effect, a yield-expression is like an inverted function call; the - argument to yield is in fact returned (yielded) from the currently - executing function, and the "return value" of yield is the argument - passed in via send(). - - Note: the syntactic extensions to yield make its use very similar - to that in Ruby. This is intentional. Do note that in Python the - block passes a value to the generator using "send(EXPR)" rather - than "return EXPR", and the underlying mechanism whereby control - is passed between the generator and the block is completely - different. Blocks in Python are not compiled into thunks; rather, - yield suspends execution of the generator's frame. Some edge - cases work differently; in Python, you cannot save the block for - later use, and you cannot test whether there is a block or not. - (XXX - this stuff about blocks seems out of place now, perhaps - Guido can edit to clarify.) Specification: Exceptions and Cleanup +===================================== + +Let a generator object be the iterator produced by calling a generator +function. Below, *g* always refers to a generator object. + +New syntax: ``yield`` allowed inside ``try-finally`` +---------------------------------------------------- + +The syntax for generator functions is extended to allow a yield-statement +inside a ``try-finally`` statement. + +New generator method: ``throw(type, value=None, traceback=None)`` +----------------------------------------------------------------- + +``g.throw(type, value, traceback)`` causes the specified exception to be +thrown at the point where the generator *g* is currently suspended (i.e. at +a yield-statement, or at the start of its function body if ``next()`` has +not been called yet). If the generator catches the exception and yields +another value, that is the return value of ``g.throw()``. If it doesn't +catch the exception, the ``throw()`` appears to raise the same exception +passed it (it *falls through*). If the generator raises another exception +(this includes the ``StopIteration`` produced when it returns) that +exception is raised by the ``throw()`` call. In summary, ``throw()`` +behaves like ``next()`` or ``send()``, except it raises an exception at the +suspension point. If the generator is already in the closed state, +``throw()`` just raises the exception it was passed without executing any of +the generator's code. + +The effect of raising the exception is exactly as if the statement:: + + raise type, value, traceback + +was executed at the suspension point. The type argument must not be +``None``, and the type and value must be compatible. If the value is not an +instance of the type, a new exception instance is created using the value, +following the same rules that the ``raise`` statement uses to create an +exception instance. The traceback, if supplied, must be a valid Python +traceback object, or a ``TypeError`` occurs. + +Note: The name of the ``throw()`` method was selected for several reasons. +``Raise`` is a keyword and so cannot be used as a method name. Unlike +``raise`` (which immediately raises an exception from the current execution +point), ``throw()`` first resumes the generator, and only then raises the +exception. The word *throw* is suggestive of putting the exception in +another location, and is already associated with exceptions in other +languages. + +Alternative method names were considered: ``resolve()``, ``signal()``, +``genraise()``, ``raiseinto()``, and ``flush()``. None of these seem to fit +as well as ``throw()``. + +New standard exception: ``GeneratorExit`` +----------------------------------------- + +A new standard exception is defined, ``GeneratorExit``, inheriting from +``Exception``. A generator should handle this by re-raising it (or just not +catching it) or by raising ``StopIteration``. + +New generator method: ``close()`` +--------------------------------- + +``g.close()`` is defined by the following pseudo-code:: + + def close(self): + try: + self.throw(GeneratorExit) + except (GeneratorExit, StopIteration): + pass + else: + raise RuntimeError("generator ignored GeneratorExit") + # Other exceptions are not caught + +New generator method: __del__() +------------------------------- + +``g.__del__()`` is a wrapper for ``g.close()``. This will be called when +the generator object is garbage-collected (in CPython, this is when its +reference count goes to zero). If ``close()`` raises an exception, a +traceback for the exception is printed to ``sys.stderr`` and further +ignored; it is not propagated back to the place that triggered the garbage +collection. This is consistent with the handling of exceptions in +``__del__()`` methods on class instances. + +If the generator object participates in a cycle, ``g.__del__()`` may not be +called. This is the behavior of CPython's current garbage collector. The +reason for the restriction is that the GC code needs to *break* a cycle at +an arbitrary point in order to collect it, and from then on no Python code +should be allowed to see the objects that formed the cycle, as they may be +in an invalid state. Objects *hanging off* a cycle are not subject to this +restriction. + +Note that it is unlikely to see a generator object participate in a cycle in +practice. However, storing a generator object in a global variable creates +a cycle via the generator frame's ``f_globals`` pointer. Another way to +create a cycle would be to store a reference to the generator object in a +data structure that is passed to the generator as an argument (e.g., if an +object has a method that's a generator, and keeps a reference to a running +iterator created by that method). Neither of these cases are very likely +given the typical patterns of generator use. + +Also, in the CPython implementation of this PEP, the frame object used by +the generator should be released whenever its execution is terminated due to +an error or normal exit. This will ensure that generators that cannot be +resumed do not remain part of an uncollectable reference cycle. This allows +other code to potentially use ``close()`` in a ``try/finally`` or ``with`` +block (per PEP 343) to ensure that a given generator is properly finalized. - Let a generator object be the iterator produced by calling a - generator function. Below, 'g' always refers to a generator - object. - - New syntax: yield allowed inside try-finally - - The syntax for generator functions is extended to allow a - yield-statement inside a try-finally statement. - - New generator method: throw(type, value=None, traceback=None) - - g.throw(type, value, traceback) causes the specified exception to - be thrown at the point where the generator g is currently - suspended (i.e. at a yield-statement, or at the start of its - function body if next() has not been called yet). If the - generator catches the exception and yields another value, that is - the return value of g.throw(). If it doesn't catch the exception, - the throw() appears to raise the same exception passed it (it - "falls through"). If the generator raises another exception (this - includes the StopIteration produced when it returns) that - exception is raised by the throw() call. In summary, throw() - behaves like next() or send(), except it raises an exception at the - suspension point. If the generator is already in the closed - state, throw() just raises the exception it was passed without - executing any of the generator's code. - - The effect of raising the exception is exactly as if the - statement: - - raise type, value, traceback - - was executed at the suspension point. The type argument must - not be None, and the type and value must be compatible. If the - value is not an instance of the type, a new exception instance - is created using the value, following the same rules that the raise - statement uses to create an exception instance. The traceback, if - supplied, must be a valid Python traceback object, or a TypeError - occurs. - - Note: The name of the throw() method was selected for several - reasons. Raise is a keyword and so cannot be used as a method - name. Unlike raise (which immediately raises an exception from the - current execution point), throw() first resumes the generator, and - only then raises the exception. The word throw is suggestive of - putting the exception in another location, and is already associated - with exceptions in other languages. - - Alternative method names were considered: resolve(), signal(), - genraise(), raiseinto(), and flush(). None of these seem to fit - as well as throw(). - - New standard exception: GeneratorExit - - A new standard exception is defined, GeneratorExit, inheriting - from Exception. A generator should handle this by re-raising it - (or just not catching it) or by raising StopIteration. - - New generator method: close() - - g.close() is defined by the following pseudo-code: - - def close(self): - try: - self.throw(GeneratorExit) - except (GeneratorExit, StopIteration): - pass - else: - raise RuntimeError("generator ignored GeneratorExit") - # Other exceptions are not caught - - New generator method: __del__() - - g.__del__() is a wrapper for g.close(). This will be called when - the generator object is garbage-collected (in CPython, this is - when its reference count goes to zero). If close() raises an - exception, a traceback for the exception is printed to sys.stderr - and further ignored; it is not propagated back to the place that - triggered the garbage collection. This is consistent with the - handling of exceptions in __del__() methods on class instances. - - If the generator object participates in a cycle, g.__del__() may - not be called. This is the behavior of CPython's current garbage - collector. The reason for the restriction is that the GC code - needs to "break" a cycle at an arbitrary point in order to collect - it, and from then on no Python code should be allowed to see the - objects that formed the cycle, as they may be in an invalid state. - Objects "hanging off" a cycle are not subject to this restriction. - - Note that it is unlikely to see a generator object participate in - a cycle in practice. However, storing a generator object in a - global variable creates a cycle via the generator frame's - f_globals pointer. Another way to create a cycle would be to - store a reference to the generator object in a data structure that - is passed to the generator as an argument (e.g., if an object has - a method that's a generator, and keeps a reference to a running - iterator created by that method). Neither of these cases - are very likely given the typical patterns of generator use. - - Also, in the CPython implementation of this PEP, the frame object - used by the generator should be released whenever its execution is - terminated due to an error or normal exit. This will ensure that - generators that cannot be resumed do not remain part of an - uncollectable reference cycle. This allows other code to - potentially use close() in a try/finally or "with" block (per PEP - 343) to ensure that a given generator is properly finalized. Optional Extensions +=================== + +The Extended ``continue`` Statement +----------------------------------- - The Extended 'continue' Statement +An earlier draft of this PEP proposed a new ``continue EXPR`` syntax for use +in for-loops (carried over from PEP 340), that would pass the value of +*EXPR* into the iterator being looped over. This feature has been withdrawn +for the time being, because the scope of this PEP has been narrowed to focus +only on passing values into generator-iterators, and not other kinds of +iterators. It was also felt by some on the Python-Dev list that adding new +syntax for this particular feature would be premature at best. - An earlier draft of this PEP proposed a new "continue EXPR" - syntax for use in for-loops (carried over from PEP 340), that - would pass the value of EXPR into the iterator being looped over. - This feature has been withdrawn for the time being, because the - scope of this PEP has been narrowed to focus only on passing values - into generator-iterators, and not other kinds of iterators. It - was also felt by some on the Python-Dev list that adding new syntax - for this particular feature would be premature at best. Open Issues +=========== + +Discussion on python-dev has revealed some open issues. I list them here, with +my preferred resolution and its motivation. The PEP as currently written +reflects this preferred resolution. + +1. What exception should be raised by ``close()`` when the generator yields + another value as a response to the ``GeneratorExit`` exception? + + I originally chose ``TypeError`` because it represents gross misbehavior of + the generator function, which should be fixed by changing the code. But the + ``with_template`` decorator class in PEP 343 uses ``RuntimeError`` for + similar offenses. Arguably they should all use the same exception. I'd + rather not introduce a new exception class just for this purpose, since it's + not an exception that I want people to catch: I want it to turn into a + traceback which is seen by the programmer who then fixes the code. So now I + believe they should both raise ``RuntimeError``. There are some precedents + for that: it's raised by the core Python code in situations where endless + recursion is detected, and for uninitialized objects (and for a variety of + miscellaneous conditions). + +2. Oren Tirosh has proposed renaming the ``send()`` method to ``feed()``, for + compatibility with the *consumer interface* (see + http://effbot.org/zone/consumer.htm for the specification.) + + However, looking more closely at the consumer interface, it seems that the + desired semantics for ``feed()`` are different than for ``send()``, because + ``send()`` can't be meaningfully called on a just-started generator. Also, + the consumer interface as currently defined doesn't include handling for + ``StopIteration``. + + Therefore, it seems like it would probably be more useful to create a simple + decorator that wraps a generator function to make it conform to the consumer + interface. For example, it could *warm up* the generator with an initial + ``next()`` call, trap StopIteration, and perhaps even provide ``reset()`` by + re-invoking the generator function. - Discussion on python-dev has revealed some open issues. I list - them here, with my preferred resolution and its motivation. The - PEP as currently written reflects this preferred resolution. - - 1. What exception should be raised by close() when the generator - yields another value as a response to the GeneratorExit - exception? - - I originally chose TypeError because it represents gross - misbehavior of the generator function, which should be fixed by - changing the code. But the with_template decorator class in - PEP 343 uses RuntimeError for similar offenses. Arguably they - should all use the same exception. I'd rather not introduce a - new exception class just for this purpose, since it's not an - exception that I want people to catch: I want it to turn into a - traceback which is seen by the programmer who then fixes the - code. So now I believe they should both raise RuntimeError. - There are some precedents for that: it's raised by the core - Python code in situations where endless recursion is detected, - and for uninitialized objects (and for a variety of - miscellaneous conditions). - - 2. Oren Tirosh has proposed renaming the send() method to feed(), - for compatibility with the "consumer interface" (see - http://effbot.org/zone/consumer.htm for the specification.) - - However, looking more closely at the consumer interface, it seems - that the desired semantics for feed() are different than for - send(), because send() can't be meaningfully called on a just- - started generator. Also, the consumer interface as currently - defined doesn't include handling for StopIteration. - - Therefore, it seems like it would probably be more useful to - create a simple decorator that wraps a generator function to make - it conform to the consumer interface. For example, it could - "warm up" the generator with an initial next() call, trap - StopIteration, and perhaps even provide reset() by re-invoking - the generator function. Examples - - 1. A simple "consumer" decorator that makes a generator function - automatically advance to its first yield point when initially - called: - - def consumer(func): - def wrapper(*args,**kw): - gen = func(*args, **kw) - gen.next() - return gen - wrapper.__name__ = func.__name__ - wrapper.__dict__ = func.__dict__ - wrapper.__doc__ = func.__doc__ - return wrapper - - 2. An example of using the "consumer" decorator to create a - "reverse generator" that receives images and creates thumbnail - pages, sending them on to another consumer. Functions like - this can be chained together to form efficient processing - pipelines of "consumers" that each can have complex internal - state: - - @consumer - def thumbnail_pager(pagesize, thumbsize, destination): - while True: - page = new_image(pagesize) - rows, columns = pagesize / thumbsize - pending = False - try: - for row in xrange(rows): - for column in xrange(columns): - thumb = create_thumbnail((yield), thumbsize) - page.write( - thumb, col*thumbsize.x, row*thumbsize.y - ) - pending = True - except GeneratorExit: - # close() was called, so flush any pending output - if pending: - destination.send(page) - - # then close the downstream consumer, and exit - destination.close() - return - else: - # we finished a page full of thumbnails, so send it - # downstream and keep on looping +======== + +1. A simple *consumer* decorator that makes a generator function automatically + advance to its first yield point when initially called:: + + def consumer(func): + def wrapper(*args,**kw): + gen = func(*args, **kw) + gen.next() + return gen + wrapper.__name__ = func.__name__ + wrapper.__dict__ = func.__dict__ + wrapper.__doc__ = func.__doc__ + return wrapper + +2. An example of using the *consumer* decorator to create a *reverse generator* + that receives images and creates thumbnail pages, sending them on to another + consumer. Functions like this can be chained together to form efficient + processing pipelines of *consumers* that each can have complex internal + state:: + + @consumer + def thumbnail_pager(pagesize, thumbsize, destination): + while True: + page = new_image(pagesize) + rows, columns = pagesize / thumbsize + pending = False + try: + for row in xrange(rows): + for column in xrange(columns): + thumb = create_thumbnail((yield), thumbsize) + page.write( + thumb, col*thumbsize.x, row*thumbsize.y ) + pending = True + except GeneratorExit: + # close() was called, so flush any pending output + if pending: destination.send(page) - @consumer - def jpeg_writer(dirname): - fileno = 1 - while True: - filename = os.path.join(dirname,"page%04d.jpg" % fileno) - write_jpeg((yield), filename) - fileno += 1 - - - # Put them together to make a function that makes thumbnail - # pages from a list of images and other parameters. - # - def write_thumbnails(pagesize, thumbsize, images, output_dir): - pipeline = thumbnail_pager( - pagesize, thumbsize, jpeg_writer(output_dir) - ) - - for image in images: - pipeline.send(image) - - pipeline.close() - - 3. A simple co-routine scheduler or "trampoline" that lets - coroutines "call" other coroutines by yielding the coroutine - they wish to invoke. Any non-generator value yielded by - a coroutine is returned to the coroutine that "called" the - one yielding the value. Similarly, if a coroutine raises an - exception, the exception is propagated to its "caller". In - effect, this example emulates simple tasklets as are used - in Stackless Python, as long as you use a yield expression to - invoke routines that would otherwise "block". This is only - a very simple example, and far more sophisticated schedulers - are possible. (For example, the existing GTasklet framework - for Python (http://www.gnome.org/~gjc/gtasklet/gtasklets.html) - and the peak.events framework (http://peak.telecommunity.com/) - already implement similar scheduling capabilities, but must - currently use awkward workarounds for the inability to pass - values or exceptions into generators.) - - import collections - - class Trampoline: - """Manage communications between coroutines""" - - running = False - - def __init__(self): - self.queue = collections.deque() - - def add(self, coroutine): - """Request that a coroutine be executed""" - self.schedule(coroutine) - - def run(self): - result = None - self.running = True - try: - while self.running and self.queue: - func = self.queue.popleft() - result = func() - return result - finally: - self.running = False - - def stop(self): + # then close the downstream consumer, and exit + destination.close() + return + else: + # we finished a page full of thumbnails, so send it + # downstream and keep on looping + destination.send(page) + + @consumer + def jpeg_writer(dirname):: + fileno = 1 + while True: + filename = os.path.join(dirname,"page%04d.jpg" % fileno) + write_jpeg((yield), filename) + fileno += 1 + + + # Put them together to make a function that makes thumbnail + # pages from a list of images and other parameters. + # + def write_thumbnails(pagesize, thumbsize, images, output_dir): + pipeline = thumbnail_pager( + pagesize, thumbsize, jpeg_writer(output_dir) + ) + + for image in images: + pipeline.send(image) + + pipeline.close() + +3. A simple co-routine scheduler or *trampoline* that lets coroutines *call* + other coroutines by yielding the coroutine they wish to invoke. Any + non-generator value yielded by a coroutine is returned to the coroutine that + *called* the one yielding the value. Similarly, if a coroutine raises an + exception, the exception is propagated to its *caller*. In effect, this + example emulates simple tasklets as are used in Stackless Python, as long as + you use a yield expression to invoke routines that would otherwise *block*. + This is only a very simple example, and far more sophisticated schedulers + are possible. (For example, the existing GTasklet framework for Python + (http://www.gnome.org/~gjc/gtasklet/gtasklets.html) and the peak.events + framework (http://peak.telecommunity.com/) already implement similar + scheduling capabilities, but must currently use awkward workarounds for the + inability to pass values or exceptions into generators.) + + :: + + import collections + + class Trampoline: + """Manage communications between coroutines""" + + running = False + + def __init__(self): + self.queue = collections.deque() + + def add(self, coroutine): + """Request that a coroutine be executed""" + self.schedule(coroutine) + + def run(self): + result = None + self.running = True + try: + while self.running and self.queue:: + func = self.queue.popleft() + result = func() + return result + finally: self.running = False - def schedule(self, coroutine, stack=(), val=None, *exc): - def resume(): - value = val - try: - if exc: - value = coroutine.throw(value,*exc) - else: - value = coroutine.send(value) - except: - if stack: - # send the error back to the "caller" - self.schedule( - stack[0], stack[1], *sys.exc_info() - ) - else: - # Nothing left in this pseudothread to - # handle it, let it propagate to the - # run loop - raise - - if isinstance(value, types.GeneratorType): - # Yielded to a specific coroutine, push the - # current one on the stack, and call the new - # one with no args - self.schedule(value, (coroutine,stack)) - - elif stack: - # Yielded a result, pop the stack and send the - # value to the caller - self.schedule(stack[0], stack[1], value) - - # else: this pseudothread has ended - - self.queue.append(resume) - - 4. A simple "echo" server, and code to run it using a trampoline - (presumes the existence of "nonblocking_read", - "nonblocking_write", and other I/O coroutines, that e.g. raise - ConnectionLost if the connection is closed): - - # coroutine function that echos data back on a connected - # socket - # - def echo_handler(sock): - while True: - try: - data = yield nonblocking_read(sock) - yield nonblocking_write(sock, data) - except ConnectionLost: - pass # exit normally if connection lost - - # coroutine function that listens for connections on a - # socket, and then launches a service "handler" coroutine - # to service the connection - # - def listen_on(trampoline, sock, handler): - while True: - # get the next incoming connection - connected_socket = yield nonblocking_accept(sock) - - # start another coroutine to handle the connection - trampoline.add( handler(connected_socket) ) - - # Create a scheduler to manage all our coroutines - t = Trampoline() - - # Create a coroutine instance to run the echo_handler on - # incoming connections - # - server = listen_on( - t, listening_socket("localhost","echo"), echo_handler - ) - - # Add the coroutine to the scheduler - t.add(server) - - # loop forever, accepting connections and servicing them - # "in parallel" - # - t.run() + def stop(self): + self.running = False + + def schedule(self, coroutine, stack=(), val=None, *exc): + def resume(): + value = val + try: + if exc: + value = coroutine.throw(value,*exc) + else: + value = coroutine.send(value) + except: + if stack: + # send the error back to the "caller" + self.schedule( + stack[0], stack[1], *sys.exc_info() + ) + else: + # Nothing left in this pseudothread to + # handle it, let it propagate to the + # run loop + raise + + if isinstance(value, types.GeneratorType): + # Yielded to a specific coroutine, push the + # current one on the stack, and call the new + # one with no args + self.schedule(value, (coroutine,stack)) + + elif stack: + # Yielded a result, pop the stack and send the + # value to the caller + self.schedule(stack[0], stack[1], value) + + # else: this pseudothread has ended + + self.queue.append(resume) + +4. A simple *echo* server, and code to run it using a trampoline (presumes the + existence of ``nonblocking_read``, ``nonblocking_write``, and other I/O + coroutines, that e.g. raise ``ConnectionLost`` if the connection is + closed):: + + # coroutine function that echos data back on a connected + # socket + # + def echo_handler(sock): + while True: + try: + data = yield nonblocking_read(sock) + yield nonblocking_write(sock, data) + except ConnectionLost: + pass # exit normally if connection lost + + # coroutine function that listens for connections on a + # socket, and then launches a service "handler" coroutine + # to service the connection + # + def listen_on(trampoline, sock, handler): + while True: + # get the next incoming connection + connected_socket = yield nonblocking_accept(sock) + + # start another coroutine to handle the connection + trampoline.add( handler(connected_socket) ) + + # Create a scheduler to manage all our coroutines + t = Trampoline() + + # Create a coroutine instance to run the echo_handler on + # incoming connections + # + server = listen_on( + t, listening_socket("localhost","echo"), echo_handler + ) + + # Add the coroutine to the scheduler + t.add(server) + + # loop forever, accepting connections and servicing them + # "in parallel" + # + t.run() Reference Implementation +======================== - A prototype patch implementing all of the features described in this - PEP is available as SourceForge patch #1223381 - (http://python.org/sf/1223381). +A prototype patch implementing all of the features described in this PEP is +available as SourceForge patch #1223381 (http://python.org/sf/1223381). - This patch was committed to CVS 01-02 August 2005. +This patch was committed to CVS 01-02 August 2005. Acknowledgements +================ + +Raymond Hettinger (PEP 288) and Samuele Pedroni (PEP 325) first formally +proposed the ideas of communicating values or exceptions into generators, and +the ability to *close* generators. Timothy Delaney suggested the title of this +PEP, and Steven Bethard helped edit a previous version. See also the +Acknowledgements section of PEP 340. - Raymond Hettinger (PEP 288) and Samuele Pedroni (PEP 325) first - formally proposed the ideas of communicating values or exceptions - into generators, and the ability to "close" generators. Timothy - Delaney suggested the title of this PEP, and Steven Bethard helped - edit a previous version. See also the Acknowledgements section - of PEP 340. References +========== - TBD. +TBD. Copyright +========= + +This document has been placed in the public domain. + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: - This document has been placed in the public domain. diff --git a/pep-0344.txt b/pep-0344.txt index 64957482fd9..47c882f2a2e 100644 --- a/pep-0344.txt +++ b/pep-0344.txt @@ -5,541 +5,560 @@ Last-Modified: $Date$ Author: Ka-Ping Yee Status: Superseded Type: Standards Track -Content-Type: text/plain +Content-Type: text/x-rst Created: 12-May-2005 Python-Version: 2.5 Post-History: Numbering Note +============== - This PEP has been renumbered to PEP 3134. The text below is the - last version submitted under the old number. +This PEP has been renumbered to PEP 3134. The text below is the last version +submitted under the old number. Abstract +======== - This PEP proposes three standard attributes on exception instances: - the '__context__' attribute for implicitly chained exceptions, the - '__cause__' attribute for explicitly chained exceptions, and the - '__traceback__' attribute for the traceback. A new "raise ... from" - statement sets the '__cause__' attribute. +This PEP proposes three standard attributes on exception instances: the +``__context__`` attribute for implicitly chained exceptions, the +``__cause__`` attribute for explicitly chained exceptions, and the +``__traceback__`` attribute for the traceback. A new ``raise ... from`` +statement sets the ``__cause__`` attribute. Motivation - - During the handling of one exception (exception A), it is possible - that another exception (exception B) may occur. In today's Python - (version 2.4), if this happens, exception B is propagated outward - and exception A is lost. In order to debug the problem, it is - useful to know about both exceptions. The '__context__' attribute - retains this information automatically. - - Sometimes it can be useful for an exception handler to intentionally - re-raise an exception, either to provide extra information or to - translate an exception to another type. The '__cause__' attribute - provides an explicit way to record the direct cause of an exception. - - In today's Python implementation, exceptions are composed of three - parts: the type, the value, and the traceback. The 'sys' module, - exposes the current exception in three parallel variables, exc_type, - exc_value, and exc_traceback, the sys.exc_info() function returns a - tuple of these three parts, and the 'raise' statement has a - three-argument form accepting these three parts. Manipulating - exceptions often requires passing these three things in parallel, - which can be tedious and error-prone. Additionally, the 'except' - statement can only provide access to the value, not the traceback. - Adding the '__traceback__' attribute to exception values makes all - the exception information accessible from a single place. +========== + +During the handling of one exception (exception A), it is possible that another +exception (exception B) may occur. In today's Python (version 2.4), if this +happens, exception B is propagated outward and exception A is lost. In order +to debug the problem, it is useful to know about both exceptions. The +``__context__`` attribute retains this information automatically. + +Sometimes it can be useful for an exception handler to intentionally re-raise +an exception, either to provide extra information or to translate an exception +to another type. The ``__cause__`` attribute provides an explicit way to +record the direct cause of an exception. + +In today's Python implementation, exceptions are composed of three parts: the +type, the value, and the traceback. The ``sys`` module, exposes the current +exception in three parallel variables, ``exc_type``, ``exc_value``, and +``exc_traceback``, the ``sys.exc_info()`` function returns a tuple of these +three parts, and the ``raise`` statement has a three-argument form accepting +these three parts. Manipulating exceptions often requires passing these three +things in parallel, which can be tedious and error-prone. Additionally, the +``except`` statement can only provide access to the value, not the traceback. +Adding the ``__traceback__`` attribute to exception values makes all the +exception information accessible from a single place. History +======= - Raymond Hettinger [1] raised the issue of masked exceptions on - Python-Dev in January 2003 and proposed a PyErr_FormatAppend() - function that C modules could use to augment the currently active - exception with more information. Brett Cannon [2] brought up - chained exceptions again in June 2003, prompting a long discussion. +Raymond Hettinger [1]_ raised the issue of masked exceptions on Python-Dev in +January 2003 and proposed a ``PyErr_FormatAppend()`` function that C modules +could use to augment the currently active exception with more information. +Brett Cannon [2]_ brought up chained exceptions again in June 2003, prompting +a long discussion. - Greg Ewing [3] identified the case of an exception occurring in a - 'finally' block during unwinding triggered by an original exception, - as distinct from the case of an exception occurring in an 'except' - block that is handling the original exception. +Greg Ewing [3]_ identified the case of an exception occurring in a ``finally`` +block during unwinding triggered by an original exception, as distinct from +the case of an exception occurring in an ``except`` block that is handling the +original exception. - Greg Ewing [4] and Guido van Rossum [5], and probably others, have - previously mentioned adding a traceback attribute to Exception - instances. This is noted in PEP 3000. +Greg Ewing [4]_ and Guido van Rossum [5]_, and probably others, have +previously mentioned adding a traceback attribute to ``Exception`` instances. +This is noted in PEP 3000. - This PEP was motivated by yet another recent Python-Dev reposting - of the same ideas [6] [7]. +This PEP was motivated by yet another recent Python-Dev reposting of the same +ideas [6]_ [7]_. Rationale - - The Python-Dev discussions revealed interest in exception chaining - for two quite different purposes. To handle the unexpected raising - of a secondary exception, the exception must be retained implicitly. - To support intentional translation of an exception, there must be a - way to chain exceptions explicitly. This PEP addresses both. - - Several attribute names for chained exceptions have been suggested - on Python-Dev [2], including 'cause', 'antecedent', 'reason', - 'original', 'chain', 'chainedexc', 'exc_chain', 'excprev', - 'previous', and 'precursor'. For an explicitly chained exception, - this PEP suggests '__cause__' because of its specific meaning. For - an implicitly chained exception, this PEP proposes the name - '__context__' because the intended meaning is more specific than - temporal precedence but less specific than causation: an exception - occurs in the context of handling another exception. - - This PEP suggests names with leading and trailing double-underscores - for these three attributes because they are set by the Python VM. - Only in very special cases should they be set by normal assignment. - - This PEP handles exceptions that occur during 'except' blocks and - 'finally' blocks in the same way. Reading the traceback makes it - clear where the exceptions occurred, so additional mechanisms for - distinguishing the two cases would only add unnecessary complexity. - - This PEP proposes that the outermost exception object (the one - exposed for matching by 'except' clauses) be the most recently - raised exception for compatibility with current behaviour. - - This PEP proposes that tracebacks display the outermost exception - last, because this would be consistent with the chronological order - of tracebacks (from oldest to most recent frame) and because the - actual thrown exception is easier to find on the last line. - - To keep things simpler, the C API calls for setting an exception - will not automatically set the exception's '__context__'. Guido - van Rossum has expressed concerns with making such changes [8]. - - As for other languages, Java and Ruby both discard the original - exception when another exception occurs in a 'catch'/'rescue' or - 'finally'/'ensure' clause. Perl 5 lacks built-in structured - exception handling. For Perl 6, RFC number 88 [9] proposes an exception - mechanism that implicitly retains chained exceptions in an array - named @@. In that RFC, the most recently raised exception is - exposed for matching, as in this PEP; also, arbitrary expressions - (possibly involving @@) can be evaluated for exception matching. - - Exceptions in C# contain a read-only 'InnerException' property that - may point to another exception. Its documentation [10] says that - "When an exception X is thrown as a direct result of a previous - exception Y, the InnerException property of X should contain a - reference to Y." This property is not set by the VM automatically; - rather, all exception constructors take an optional 'innerException' - argument to set it explicitly. The '__cause__' attribute fulfills - the same purpose as InnerException, but this PEP proposes a new form - of 'raise' rather than extending the constructors of all exceptions. - C# also provides a GetBaseException method that jumps directly to - the end of the InnerException chain; this PEP proposes no analog. - - The reason all three of these attributes are presented together in - one proposal is that the '__traceback__' attribute provides - convenient access to the traceback on chained exceptions. +========= + +The Python-Dev discussions revealed interest in exception chaining for two +quite different purposes. To handle the unexpected raising of a secondary +exception, the exception must be retained implicitly. To support intentional +translation of an exception, there must be a way to chain exceptions +explicitly. This PEP addresses both. + +Several attribute names for chained exceptions have been suggested on Python- +Dev [2]_, including ``cause``, ``antecedent``, ``reason``, ``original``, +``chain``, ``chainedexc``, ``xc_chain``, ``excprev``, ``previous`` and +``precursor``. For an explicitly chained exception, this PEP suggests +``__cause__`` because of its specific meaning. For an implicitly chained +exception, this PEP proposes the name ``__context__`` because the intended +meaning is more specific than temporal precedence but less specific than +causation: an exception occurs in the context of handling another exception. + +This PEP suggests names with leading and trailing double-underscores for these +three attributes because they are set by the Python VM. Only in very special +cases should they be set by normal assignment. + +This PEP handles exceptions that occur during ``except`` blocks and +``finally`` blocks in the same way. Reading the traceback makes it clear +where the exceptions occurred, so additional mechanisms for distinguishing +the two cases would only add unnecessary complexity. + +This PEP proposes that the outermost exception object (the one exposed for +matching by ``except`` clauses) be the most recently raised exception for +compatibility with current behaviour. + +This PEP proposes that tracebacks display the outermost exception last, +because this would be consistent with the chronological order of tracebacks +(from oldest to most recent frame) and because the actual thrown exception is +easier to find on the last line. + +To keep things simpler, the C API calls for setting an exception will not +automatically set the exception's ``__context__``. Guido van Rossum has +expressed concerns with making such changes [8]_. + +As for other languages, Java and Ruby both discard the original exception when +another exception occurs in a ``catch/rescue`` or ``finally/ensure`` clause. +Perl 5 lacks built-in structured exception handling. For Perl 6, RFC number +88 [9]_ proposes an exception mechanism that implicitly retains chained +exceptions in an array named ``@@``. In that RFC, the most recently raised +exception is exposed for matching, as in this PEP; also, arbitrary expressions +(possibly involving ``@@``) can be evaluated for exception matching. + +Exceptions in C# contain a read-only ``InnerException`` property that may +point to another exception. Its documentation [10]_ says that "When an +exception X is thrown as a direct result of a previous exception Y, the +``InnerException`` property of X should contain a reference to Y." This +property is not set by the VM automatically; rather, all exception +constructors take an optional ``innerException`` argument to set it +explicitly. The ``__cause__`` attribute fulfills the same purpose as +``InnerException``, but this PEP proposes a new form of ``raise`` rather than +extending the constructors of all exceptions. C# also provides a +``GetBaseException`` method that jumps directly to the end of the +``InnerException`` chain; this PEP proposes no analog. + +The reason all three of these attributes are presented together in one proposal +is that the ``__traceback__`` attribute provides convenient access to the +traceback on chained exceptions. Implicit Exception Chaining +=========================== - Here is an example to illustrate the '__context__' attribute. +Here is an example to illustrate the ``__context__`` attribute:: - def compute(a, b): - try: - a/b - except Exception, exc: - log(exc) + def compute(a, b): + try: + a/b + except Exception, exc: + log(exc) - def log(exc): - file = open('logfile.txt') # oops, forgot the 'w' - print >>file, exc - file.close() + def log(exc): + file = open('logfile.txt') # oops, forgot the 'w' + print >>file, exc + file.close() - Calling compute(0, 0) causes a ZeroDivisionError. The compute() - function catches this exception and calls log(exc), but the log() - function also raises an exception when it tries to write to a - file that wasn't opened for writing. +Calling ``compute(0, 0)`` causes a ``ZeroDivisionError``. The ``compute()`` +function catches this exception and calls ``log(exc)``, but the ``log()`` +function also raises an exception when it tries to write to a file that wasn't +opened for writing. - In today's Python, the caller of compute() gets thrown an IOError. - The ZeroDivisionError is lost. With the proposed change, the - instance of IOError has an additional '__context__' attribute that - retains the ZeroDivisionError. +In today's Python, the caller of ``compute()`` gets thrown an ``IOError``. The +``ZeroDivisionError`` is lost. With the proposed change, the instance of +``IOError`` has an additional ``__context__`` attribute that retains the +``ZeroDivisionError``. - The following more elaborate example demonstrates the handling of a - mixture of 'finally' and 'except' clauses: +The following more elaborate example demonstrates the handling of a mixture of +``finally`` and ``except`` clauses:: - def main(filename): - file = open(filename) # oops, forgot the 'w' + def main(filename): + file = open(filename) # oops, forgot the 'w' + try: try: - try: - compute() - except Exception, exc: - log(file, exc) + compute() + except Exception, exc: + log(file, exc) finally: - file.clos() # oops, misspelled 'close' + file.clos() # oops, misspelled 'close' - def compute(): - 1/0 + def compute(): + 1/0 - def log(file, exc): - try: - print >>file, exc # oops, file is not writable - except: - display(exc) + def log(file, exc): + try: + print >>file, exc # oops, file is not writable + except: + display(exc) - def display(exc): - print ex # oops, misspelled 'exc' + def display(exc): + print ex # oops, misspelled 'exc' - Calling main() with the name of an existing file will trigger four - exceptions. The ultimate result will be an AttributeError due to - the misspelling of 'clos', whose __context__ points to a NameError - due to the misspelling of 'ex', whose __context__ points to an - IOError due to the file being read-only, whose __context__ points to - a ZeroDivisionError, whose __context__ attribute is None. +Calling ``main()`` with the name of an existing file will trigger four +exceptions. The ultimate result will be an ``AttributeError`` due to the +misspelling of ``clos``, whose ``__context__`` points to a ``NameError`` due +to the misspelling of ``ex``, whose ``__context__`` points to an ``IOError`` +due to the file being read-only, whose ``__context__`` points to a +``ZeroDivisionError``, whose ``__context__`` attribute is ``None``. - The proposed semantics are as follows: +The proposed semantics are as follows: - 1. Each thread has an exception context initially set to None. +1. Each thread has an exception context initially set to ``None``. - 2. Whenever an exception is raised, if the exception instance does - not already have a '__context__' attribute, the interpreter sets - it equal to the thread's exception context. +2. Whenever an exception is raised, if the exception instance does not + already have a ``__context__`` attribute, the interpreter sets it equal to + the thread's exception context. - 3. Immediately after an exception is raised, the thread's exception - context is set to the exception. +3. Immediately after an exception is raised, the thread's exception context is + set to the exception. - 4. Whenever the interpreter exits an 'except' block by reaching the - end or executing a 'return', 'yield', 'continue', or 'break' - statement, the thread's exception context is set to None. +4. Whenever the interpreter exits an ``except`` block by reaching the end or + executing a ``return``, ``yield``, ``continue``, or ``break`` statement, + the thread's exception context is set to ``None``. Explicit Exception Chaining +=========================== - The '__cause__' attribute on exception objects is always initialized - to None. It is set by a new form of the 'raise' statement: +The ``__cause__`` attribute on exception objects is always initialized to +``None``. It is set by a new form of the ``raise`` statement:: - raise EXCEPTION from CAUSE + raise EXCEPTION from CAUSE - which is equivalent to: +which is equivalent to:: - exc = EXCEPTION - exc.__cause__ = CAUSE - raise exc + exc = EXCEPTION + exc.__cause__ = CAUSE + raise exc - In the following example, a database provides implementations for a - few different kinds of storage, with file storage as one kind. The - database designer wants errors to propagate as DatabaseError objects - so that the client doesn't have to be aware of the storage-specific - details, but doesn't want to lose the underlying error information. +In the following example, a database provides implementations for a few +different kinds of storage, with file storage as one kind. The database +designer wants errors to propagate as ``DatabaseError`` objects so that the +client doesn't have to be aware of the storage-specific details, but doesn't +want to lose the underlying error information:: - class DatabaseError(StandardError): - pass + class DatabaseError(StandardError): + pass - class FileDatabase(Database): - def __init__(self, filename): - try: - self.file = open(filename) - except IOError, exc: - raise DatabaseError('failed to open') from exc + class FileDatabase(Database): + def __init__(self, filename): + try: + self.file = open(filename) + except IOError, exc: + raise DatabaseError('failed to open') from exc - If the call to open() raises an exception, the problem will be - reported as a DatabaseError, with a __cause__ attribute that reveals - the IOError as the original cause. +If the call to ``open()`` raises an exception, the problem will be reported as +a ``DatabaseError``, with a ``__cause__`` attribute that reveals the +``IOError`` as the original cause. Traceback Attribute +=================== - The following example illustrates the '__traceback__' attribute. +The following example illustrates the ``__traceback__`` attribute:: - def do_logged(file, work): - try: - work() - except Exception, exc: - write_exception(file, exc) - raise exc + def do_logged(file, work): + try: + work() + except Exception, exc: + write_exception(file, exc) + raise exc - from traceback import format_tb + from traceback import format_tb - def write_exception(file, exc): - ... - type = exc.__class__ - message = str(exc) - lines = format_tb(exc.__traceback__) - file.write(... type ... message ... lines ...) - ... + def write_exception(file, exc): + ... + type = exc.__class__ + message = str(exc) + lines = format_tb(exc.__traceback__) + file.write(... type ... message ... lines ...) + ... - In today's Python, the do_logged() function would have to extract - the traceback from sys.exc_traceback or sys.exc_info()[2] and pass - both the value and the traceback to write_exception(). With the - proposed change, write_exception() simply gets one argument and - obtains the exception using the '__traceback__' attribute. +In today's Python, the ``do_logged()`` function would have to extract the +traceback from ``sys.exc_traceback`` or ``sys.exc_info()`` [2]_ and pass both +the value and the traceback to ``write_exception()``. With the proposed +change, ``write_exception()`` simply gets one argument and obtains the +exception using the ``__traceback__`` attribute. - The proposed semantics are as follows: +The proposed semantics are as follows: - 1. Whenever an exception is caught, if the exception instance does - not already have a '__traceback__' attribute, the interpreter - sets it to the newly caught traceback. +1. Whenever an exception is caught, if the exception instance does not already + have a ``__traceback__`` attribute, the interpreter sets it to the newly + caught traceback. Enhanced Reporting +================== - The default exception handler will be modified to report chained - exceptions. The chain of exceptions is traversed by following the - '__cause__' and '__context__' attributes, with '__cause__' taking - priority. In keeping with the chronological order of tracebacks, - the most recently raised exception is displayed last; that is, the - display begins with the description of the innermost exception and - backs up the chain to the outermost exception. The tracebacks are - formatted as usual, with one of the lines: +The default exception handler will be modified to report chained exceptions. +The chain of exceptions is traversed by following the ``__cause__`` and +``__context__`` attributes, with ``__cause__`` taking priority. In keeping +with the chronological order of tracebacks, the most recently raised exception +is displayed last; that is, the display begins with the description of the +innermost exception and backs up the chain to the outermost exception. The +tracebacks are formatted as usual, with one of the lines:: - The above exception was the direct cause of the following exception: + The above exception was the direct cause of the following exception: - or +or - During handling of the above exception, another exception occurred: +:: - between tracebacks, depending whether they are linked by __cause__ - or __context__ respectively. Here is a sketch of the procedure: + During handling of the above exception, another exception occurred: - def print_chain(exc): - if exc.__cause__: - print_chain(exc.__cause__) - print '\nThe above exception was the direct cause...' - elif exc.__context__: - print_chain(exc.__context__) - print '\nDuring handling of the above exception, ...' - print_exc(exc) +between tracebacks, depending whether they are linked by ``__cause__`` or +``__context__`` respectively. Here is a sketch of the procedure:: - In the 'traceback' module, the format_exception, print_exception, - print_exc, and print_last functions will be updated to accept an - optional 'chain' argument, True by default. When this argument is - True, these functions will format or display the entire chain of - exceptions as just described. When it is False, these functions - will format or display only the outermost exception. + def print_chain(exc): + if exc.__cause__: + print_chain(exc.__cause__) + print '\nThe above exception was the direct cause...' + elif exc.__context__: + print_chain(exc.__context__) + print '\nDuring handling of the above exception, ...' + print_exc(exc) - The 'cgitb' module should also be updated to display the entire - chain of exceptions. +In the ``traceback`` module, the ``format_exception``, ``print_exception``, +``print_exc``, and ``print_last functions`` will be updated to accept an +optional ``chain`` argument, ``True`` by default. When this argument is +``True``, these functions will format or display the entire chain of +exceptions as just described. When it is ``False``, these functions will +format or display only the outermost exception. + +The ``cgitb`` module should also be updated to display the entire chain of +exceptions. C API +===== - The PyErr_Set* calls for setting exceptions will not set the - '__context__' attribute on exceptions. PyErr_NormalizeException - will always set the 'traceback' attribute to its 'tb' argument and - the '__context__' and '__cause__' attributes to None. +The ``PyErr_Set*`` calls for setting exceptions will not set the +``__context__`` attribute on exceptions. ``PyErr_NormalizeException`` will +always set the ``traceback`` attribute to its ``tb`` argument and the +``__context__`` and ``__cause__`` attributes to ``None``. - A new API function, PyErr_SetContext(context), will help C - programmers provide chained exception information. This function - will first normalize the current exception so it is an instance, - then set its '__context__' attribute. A similar API function, - PyErr_SetCause(cause), will set the '__cause__' attribute. +A new API function, ``PyErr_SetContext(context)``, will help C programmers +provide chained exception information. This function will first normalize the +current exception so it is an instance, then set its ``__context__`` +attribute. A similar API function, ``PyErr_SetCause(cause)``, will set the +``__cause__`` attribute. Compatibility +============= - Chained exceptions expose the type of the most recent exception, so - they will still match the same 'except' clauses as they do now. +Chained exceptions expose the type of the most recent exception, so they will +still match the same ``except`` clauses as they do now. - The proposed changes should not break any code unless it sets or - uses attributes named '__context__', '__cause__', or '__traceback__' - on exception instances. As of 2005-05-12, the Python standard - library contains no mention of such attributes. +The proposed changes should not break any code unless it sets or uses +attributes named ``__context__``, ``__cause__``, or ``__traceback__`` on +exception instances. As of 2005-05-12, the Python standard library contains +no mention of such attributes. Open Issue: Extra Information +============================== - Walter Dörwald [11] expressed a desire to attach extra information - to an exception during its upward propagation without changing its - type. This could be a useful feature, but it is not addressed by - this PEP. It could conceivably be addressed by a separate PEP - establishing conventions for other informational attributes on - exceptions. +Walter Dörwald [11]_ expressed a desire to attach extra information to an +exception during its upward propagation without changing its type. This could +be a useful feature, but it is not addressed by this PEP. It could +conceivably be addressed by a separate PEP establishing conventions for other +informational attributes on exceptions. Open Issue: Suppressing Context +================================ - As written, this PEP makes it impossible to suppress '__context__', - since setting exc.__context__ to None in an 'except' or 'finally' - clause will only result in it being set again when exc is raised. +As written, this PEP makes it impossible to suppress ``__context__``, since +setting ``exc.__context__`` to ``None`` in an ``except`` or ``finally`` clause +will only result in it being set again when ``exc`` is raised. Open Issue: Limiting Exception Types +===================================== - To improve encapsulation, library implementors may want to wrap all - implementation-level exceptions with an application-level exception. - One could try to wrap exceptions by writing this: +To improve encapsulation, library implementors may want to wrap all +implementation-level exceptions with an application-level exception. One could +try to wrap exceptions by writing this:: - try: - ... implementation may raise an exception ... - except: - import sys - raise ApplicationError from sys.exc_value + try: + ... implementation may raise an exception ... + except: + import sys + raise ApplicationError from sys.exc_value - or this: +or this - try: - ... implementation may raise an exception ... - except Exception, exc: - raise ApplicationError from exc +:: - but both are somewhat flawed. It would be nice to be able to name - the current exception in a catch-all 'except' clause, but that isn't - addressed here. Such a feature would allow something like this: + try: + ... implementation may raise an exception ... + except Exception, exc: + raise ApplicationError from exc - try: - ... implementation may raise an exception ... - except *, exc: - raise ApplicationError from exc +but both are somewhat flawed. It would be nice to be able to name the current +exception in a catch-all ``except`` clause, but that isn't addressed here. +Such a feature would allow something like this:: + try: + ... implementation may raise an exception ... + except *, exc: + raise ApplicationError from exc -Open Issue: yield - The exception context is lost when a 'yield' statement is executed; - resuming the frame after the 'yield' does not restore the context. - Addressing this problem is out of the scope of this PEP; it is not a - new problem, as demonstrated by the following example: - - >>> def gen(): - ... try: - ... 1/0 - ... except: - ... yield 3 - ... raise - ... - >>> g = gen() - >>> g.next() - 3 - >>> g.next() - TypeError: exceptions must be classes, instances, or strings - (deprecated), not NoneType +Open Issue: yield +================== + +The exception context is lost when a ``yield`` statement is executed; resuming +the frame after the ``yield`` does not restore the context. Addressing this +problem is out of the scope of this PEP; it is not a new problem, as +demonstrated by the following example:: + + >>> def gen(): + ... try: + ... 1/0 + ... except: + ... yield 3 + ... raise + ... + >>> g = gen() + >>> g.next() + 3 + >>> g.next() + TypeError: exceptions must be classes, instances, or strings + (deprecated), not NoneType Open Issue: Garbage Collection +=============================== - The strongest objection to this proposal has been that it creates - cycles between exceptions and stack frames [12]. Collection of - cyclic garbage (and therefore resource release) can be greatly - delayed. +The strongest objection to this proposal has been that it creates cycles +between exceptions and stack frames [12]_. Collection of cyclic garbage (and +therefore resource release) can be greatly delayed:: - >>> try: - >>> 1/0 - >>> except Exception, err: - >>> pass + >>> try: + >>> 1/0 + >>> except Exception, err: + >>> pass - will introduce a cycle from err -> traceback -> stack frame -> err, - keeping all locals in the same scope alive until the next GC happens. +will introduce a cycle from err -> traceback -> stack frame -> err, keeping +all locals in the same scope alive until the next GC happens. - Today, these locals would go out of scope. There is lots of code - which assumes that "local" resources -- particularly open files -- will - be closed quickly. If closure has to wait for the next GC, a program - (which runs fine today) may run out of file handles. +Today, these locals would go out of scope. There is lots of code which +assumes that "local" resources -- particularly open files -- will be closed +quickly. If closure has to wait for the next GC, a program (which runs fine +today) may run out of file handles. - Making the __traceback__ attribute a weak reference would avoid the - problems with cyclic garbage. Unfortunately, it would make saving - the Exception for later (as unittest does) more awkward, and it would - not allow as much cleanup of the sys module. +Making the ``__traceback__`` attribute a weak reference would avoid the +problems with cyclic garbage. Unfortunately, it would make saving the +``Exception`` for later (as ``unittest`` does) more awkward, and it would not +allow as much cleanup of the ``sys`` module. - A possible alternate solution, suggested by Adam Olsen, would be to - instead turn the reference from the stack frame to the 'err' variable - into a weak reference when the variable goes out of scope [13]. +A possible alternate solution, suggested by Adam Olsen, would be to instead +turn the reference from the stack frame to the ``err`` variable into a weak +reference when the variable goes out of scope [13]_. Possible Future Compatible Changes +================================== - These changes are consistent with the appearance of exceptions as - a single object rather than a triple at the interpreter level. +These changes are consistent with the appearance of exceptions as a single +object rather than a triple at the interpreter level. - - If PEP 340 or PEP 343 is accepted, replace the three (type, value, - traceback) arguments to __exit__ with a single exception argument. +- If PEP 340 or PEP 343 is accepted, replace the three (``type``, ``value``, + ``traceback``) arguments to ``__exit__`` with a single exception argument. - - Deprecate sys.exc_type, sys.exc_value, sys.exc_traceback, and - sys.exc_info() in favour of a single member, sys.exception. +- Deprecate ``sys.exc_type``, ``sys.exc_value``, ``sys.exc_traceback``, and + ``sys.exc_info()`` in favour of a single member, ``sys.exception``. - - Deprecate sys.last_type, sys.last_value, and sys.last_traceback - in favour of a single member, sys.last_exception. +- Deprecate ``sys.last_type``, ``sys.last_value``, and ``sys.last_traceback`` + in favour of a single member, ``sys.last_exception``. - - Deprecate the three-argument form of the 'raise' statement in - favour of the one-argument form. +- Deprecate the three-argument form of the ``raise`` statement in favour of + the one-argument form. - - Upgrade cgitb.html() to accept a single value as its first - argument as an alternative to a (type, value, traceback) tuple. +- Upgrade ``cgitb.html()`` to accept a single value as its first argument as + an alternative to a ``(type, value, traceback)`` tuple. Possible Future Incompatible Changes +==================================== - These changes might be worth considering for Python 3000. +These changes might be worth considering for Python 3000. - - Remove sys.exc_type, sys.exc_value, sys.exc_traceback, and - sys.exc_info(). +- Remove ``sys.exc_type``, ``sys.exc_value``, ``sys.exc_traceback``, and + ``sys.exc_info()``. - - Remove sys.last_type, sys.last_value, and sys.last_traceback. +- Remove ``sys.last_type``, ``sys.last_value``, and ``sys.last_traceback``. - - Replace the three-argument sys.excepthook with a one-argument - API, and changing the 'cgitb' module to match. +- Replace the three-argument ``sys.excepthook`` with a one-argument API, and + changing the ``cgitb`` module to match. - - Remove the three-argument form of the 'raise' statement. +- Remove the three-argument form of the ``raise`` statement. - - Upgrade traceback.print_exception to accept an 'exception' - argument instead of the type, value, and traceback arguments. +- Upgrade ``traceback.print_exception`` to accept an ``exception`` argument + instead of the ``type``, ``value``, and ``traceback`` arguments. Acknowledgements +================ - Brett Cannon, Greg Ewing, Guido van Rossum, Jeremy Hylton, Phillip - J. Eby, Raymond Hettinger, Walter Dörwald, and others. +Brett Cannon, Greg Ewing, Guido van Rossum, Jeremy Hylton, Phillip J. Eby, +Raymond Hettinger, Walter Dörwald, and others. References +========== - [1] Raymond Hettinger, "Idea for avoiding exception masking" - http://mail.python.org/pipermail/python-dev/2003-January/032492.html +.. [1] Raymond Hettinger, "Idea for avoiding exception masking" + http://mail.python.org/pipermail/python-dev/2003-January/032492.html - [2] Brett Cannon explains chained exceptions - http://mail.python.org/pipermail/python-dev/2003-June/036063.html +.. [2] Brett Cannon explains chained exceptions + http://mail.python.org/pipermail/python-dev/2003-June/036063.html - [3] Greg Ewing points out masking caused by exceptions during finally - http://mail.python.org/pipermail/python-dev/2003-June/036290.html +.. [3] Greg Ewing points out masking caused by exceptions during finally + http://mail.python.org/pipermail/python-dev/2003-June/036290.html - [4] Greg Ewing suggests storing the traceback in the exception object - http://mail.python.org/pipermail/python-dev/2003-June/036092.html +.. [4] Greg Ewing suggests storing the traceback in the exception object + http://mail.python.org/pipermail/python-dev/2003-June/036092.html - [5] Guido van Rossum mentions exceptions having a traceback attribute - http://mail.python.org/pipermail/python-dev/2005-April/053060.html +.. [5] Guido van Rossum mentions exceptions having a traceback attribute + http://mail.python.org/pipermail/python-dev/2005-April/053060.html - [6] Ka-Ping Yee, "Tidier Exceptions" - http://mail.python.org/pipermail/python-dev/2005-May/053671.html +.. [6] Ka-Ping Yee, "Tidier Exceptions" + http://mail.python.org/pipermail/python-dev/2005-May/053671.html - [7] Ka-Ping Yee, "Chained Exceptions" - http://mail.python.org/pipermail/python-dev/2005-May/053672.html +.. [7] Ka-Ping Yee, "Chained Exceptions" + http://mail.python.org/pipermail/python-dev/2005-May/053672.html - [8] Guido van Rossum discusses automatic chaining in PyErr_Set* - http://mail.python.org/pipermail/python-dev/2003-June/036180.html +.. [8] Guido van Rossum discusses automatic chaining in ``PyErr_Set*`` + http://mail.python.org/pipermail/python-dev/2003-June/036180.html - [9] Tony Olensky, "Omnibus Structured Exception/Error Handling Mechanism" - http://dev.perl.org/perl6/rfc/88.html +.. [9] Tony Olensky, "Omnibus Structured Exception/Error Handling Mechanism" + http://dev.perl.org/perl6/rfc/88.html - [10] MSDN .NET Framework Library, "Exception.InnerException Property" +.. [10] MSDN .NET Framework Library, "Exception.InnerException Property" http://msdn.microsoft.com/library/en-us/cpref/html/frlrfsystemexceptionclassinnerexceptiontopic.asp - [11] Walter Dörwald suggests wrapping exceptions to add details +.. [11] Walter Dörwald suggests wrapping exceptions to add details http://mail.python.org/pipermail/python-dev/2003-June/036148.html - [12] Guido van Rossum restates the objection to cyclic trash +.. [12] Guido van Rossum restates the objection to cyclic trash http://mail.python.org/pipermail/python-3000/2007-January/005322.html - [13] Adam Olsen suggests using a weakref from stack frame to exception +.. [13] Adam Olsen suggests using a weakref from stack frame to exception http://mail.python.org/pipermail/python-3000/2007-January/005363.html Copyright +========= + +This document has been placed in the public domain. - This document has been placed in the public domain. - -Local Variables: -mode: indented-text -indent-tabs-mode: nil -sentence-end-double-space: t -fill-column: 70 -coding: utf-8 -End: +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: diff --git a/pep-3134.txt b/pep-3134.txt index 5d26ffb6166..ce51b5726d9 100644 --- a/pep-3134.txt +++ b/pep-3134.txt @@ -5,550 +5,574 @@ Last-Modified: $Date$ Author: Ka-Ping Yee Status: Final Type: Standards Track -Content-Type: text/plain +Content-Type: text/x-rst Created: 12-May-2005 Python-Version: 3.0 Post-History: Numbering Note +============== - This PEP started its life as PEP 344. Since it is now targeted - for Python 3000, it has been moved into the 3xxx space. +This PEP started its life as PEP 344. Since it is now targeted for Python +3000, it has been moved into the 3xxx space. Abstract +======== - This PEP proposes three standard attributes on exception instances: - the '__context__' attribute for implicitly chained exceptions, the - '__cause__' attribute for explicitly chained exceptions, and the - '__traceback__' attribute for the traceback. A new "raise ... from" - statement sets the '__cause__' attribute. +This PEP proposes three standard attributes on exception instances: the +``__context__`` attribute for implicitly chained exceptions, the ``__cause__`` +attribute for explicitly chained exceptions, and the ``__traceback__`` +attribute for the traceback. A new ``raise ... from`` statement sets the +``__cause__`` attribute. Motivation - - During the handling of one exception (exception A), it is possible - that another exception (exception B) may occur. In today's Python - (version 2.4), if this happens, exception B is propagated outward - and exception A is lost. In order to debug the problem, it is - useful to know about both exceptions. The '__context__' attribute - retains this information automatically. - - Sometimes it can be useful for an exception handler to intentionally - re-raise an exception, either to provide extra information or to - translate an exception to another type. The '__cause__' attribute - provides an explicit way to record the direct cause of an exception. - - In today's Python implementation, exceptions are composed of three - parts: the type, the value, and the traceback. The 'sys' module, - exposes the current exception in three parallel variables, exc_type, - exc_value, and exc_traceback, the sys.exc_info() function returns a - tuple of these three parts, and the 'raise' statement has a - three-argument form accepting these three parts. Manipulating - exceptions often requires passing these three things in parallel, - which can be tedious and error-prone. Additionally, the 'except' - statement can only provide access to the value, not the traceback. - Adding the '__traceback__' attribute to exception values makes all - the exception information accessible from a single place. +========== + +During the handling of one exception (exception A), it is possible that another +exception (exception B) may occur. In today's Python (version 2.4), if this +happens, exception B is propagated outward and exception A is lost. In order +to debug the problem, it is useful to know about both exceptions. The +``__context__`` attribute retains this information automatically. + +Sometimes it can be useful for an exception handler to intentionally re-raise +an exception, either to provide extra information or to translate an exception +to another type. The ``__cause__`` attribute provides an explicit way to +record the direct cause of an exception. + +In today's Python implementation, exceptions are composed of three parts: the +type, the value, and the traceback. The ``sys`` module, exposes the current +exception in three parallel variables, ``exc_type``, ``exc_value``, and +``exc_traceback``, the ``sys.exc_info()`` function returns a tuple of these +three parts, and the ``raise`` statement has a three-argument form accepting +these three parts. Manipulating exceptions often requires passing these three +things in parallel, which can be tedious and error-prone. Additionally, the +``except`` statement can only provide access to the value, not the traceback. +Adding the ``__traceback__`` attribute to exception values makes all the +exception information accessible from a single place. History +======= - Raymond Hettinger [1] raised the issue of masked exceptions on - Python-Dev in January 2003 and proposed a PyErr_FormatAppend() - function that C modules could use to augment the currently active - exception with more information. Brett Cannon [2] brought up - chained exceptions again in June 2003, prompting a long discussion. +Raymond Hettinger [1]_ raised the issue of masked exceptions on Python-Dev in +January 2003 and proposed a ``PyErr_FormatAppend()`` function that C modules +could use to augment the currently active exception with more information. +Brett Cannon [2]_ brought up chained exceptions again in June 2003, prompting +a long discussion. - Greg Ewing [3] identified the case of an exception occurring in a - 'finally' block during unwinding triggered by an original exception, - as distinct from the case of an exception occurring in an 'except' - block that is handling the original exception. +Greg Ewing [3]_ identified the case of an exception occurring in a ``finally`` +block during unwinding triggered by an original exception, as distinct from +the case of an exception occurring in an ``except`` block that is handling the +original exception. - Greg Ewing [4] and Guido van Rossum [5], and probably others, have - previously mentioned adding a traceback attribute to Exception - instances. This is noted in PEP 3000. +Greg Ewing [4]_ and Guido van Rossum [5]_, and probably others, have +previously mentioned adding a traceback attribute to Exception instances. +This is noted in PEP 3000. - This PEP was motivated by yet another recent Python-Dev reposting - of the same ideas [6] [7]. +This PEP was motivated by yet another recent Python-Dev reposting of the same +ideas [6]_ [7]_. Rationale - - The Python-Dev discussions revealed interest in exception chaining - for two quite different purposes. To handle the unexpected raising - of a secondary exception, the exception must be retained implicitly. - To support intentional translation of an exception, there must be a - way to chain exceptions explicitly. This PEP addresses both. - - Several attribute names for chained exceptions have been suggested - on Python-Dev [2], including 'cause', 'antecedent', 'reason', - 'original', 'chain', 'chainedexc', 'exc_chain', 'excprev', - 'previous', and 'precursor'. For an explicitly chained exception, - this PEP suggests '__cause__' because of its specific meaning. For - an implicitly chained exception, this PEP proposes the name - '__context__' because the intended meaning is more specific than - temporal precedence but less specific than causation: an exception - occurs in the context of handling another exception. - - This PEP suggests names with leading and trailing double-underscores - for these three attributes because they are set by the Python VM. - Only in very special cases should they be set by normal assignment. - - This PEP handles exceptions that occur during 'except' blocks and - 'finally' blocks in the same way. Reading the traceback makes it - clear where the exceptions occurred, so additional mechanisms for - distinguishing the two cases would only add unnecessary complexity. - - This PEP proposes that the outermost exception object (the one - exposed for matching by 'except' clauses) be the most recently - raised exception for compatibility with current behaviour. - - This PEP proposes that tracebacks display the outermost exception - last, because this would be consistent with the chronological order - of tracebacks (from oldest to most recent frame) and because the - actual thrown exception is easier to find on the last line. - - To keep things simpler, the C API calls for setting an exception - will not automatically set the exception's '__context__'. Guido - van Rossum has expressed concerns with making such changes [8]. - - As for other languages, Java and Ruby both discard the original - exception when another exception occurs in a 'catch'/'rescue' or - 'finally'/'ensure' clause. Perl 5 lacks built-in structured - exception handling. For Perl 6, RFC number 88 [9] proposes an exception - mechanism that implicitly retains chained exceptions in an array - named @@. In that RFC, the most recently raised exception is - exposed for matching, as in this PEP; also, arbitrary expressions - (possibly involving @@) can be evaluated for exception matching. - - Exceptions in C# contain a read-only 'InnerException' property that - may point to another exception. Its documentation [10] says that - "When an exception X is thrown as a direct result of a previous - exception Y, the InnerException property of X should contain a - reference to Y." This property is not set by the VM automatically; - rather, all exception constructors take an optional 'innerException' - argument to set it explicitly. The '__cause__' attribute fulfills - the same purpose as InnerException, but this PEP proposes a new form - of 'raise' rather than extending the constructors of all exceptions. - C# also provides a GetBaseException method that jumps directly to - the end of the InnerException chain; this PEP proposes no analog. - - The reason all three of these attributes are presented together in - one proposal is that the '__traceback__' attribute provides - convenient access to the traceback on chained exceptions. +========= + +The Python-Dev discussions revealed interest in exception chaining for two +quite different purposes. To handle the unexpected raising of a secondary +exception, the exception must be retained implicitly. To support intentional +translation of an exception, there must be a way to chain exceptions +explicitly. This PEP addresses both. + +Several attribute names for chained exceptions have been suggested on +Python-Dev [2]_, including ``cause``, ``antecedent``, ``reason``, ``original``, +``chain``, ``chainedexc``, ``exc_chain``, ``excprev``, ``previous``, and +``precursor``. For an explicitly chained exception, this PEP suggests +``__cause__`` because of its specific meaning. For an implicitly chained +exception, this PEP proposes the name ``__context__`` because the intended +meaning is more specific than temporal precedence but less specific than +causation: an exception occurs in the context of handling another exception. + +This PEP suggests names with leading and trailing double-underscores for these +three attributes because they are set by the Python VM. Only in very special +cases should they be set by normal assignment. + +This PEP handles exceptions that occur during ``except`` blocks and ``finally`` +blocks in the same way. Reading the traceback makes it clear where the +exceptions occurred, so additional mechanisms for distinguishing the two cases +would only add unnecessary complexity. + +This PEP proposes that the outermost exception object (the one exposed for +matching by ``except`` clauses) be the most recently raised exception for +compatibility with current behaviour. + +This PEP proposes that tracebacks display the outermost exception last, because +this would be consistent with the chronological order of tracebacks (from +oldest to most recent frame) and because the actual thrown exception is easier +to find on the last line. + +To keep things simpler, the C API calls for setting an exception will not +automatically set the exception's ``__context__``. Guido van Rossum has +expressed concerns with making such changes [8]_. + +As for other languages, Java and Ruby both discard the original exception when +another exception occurs in a ``catch``/``rescue`` or ``finally``/``ensure`` +clause. Perl 5 lacks built-in structured exception handling. For Perl 6, RFC +number 88 [9]_ proposes an exception mechanism that implicitly retains chained +exceptions in an array named ``@@``. In that RFC, the most recently raised +exception is exposed for matching, as in this PEP; also, arbitrary expressions +(possibly involving ``@@``) can be evaluated for exception matching. + +Exceptions in C# contain a read-only ``InnerException`` property that may point +to another exception. Its documentation [10]_ says that "When an exception X +is thrown as a direct result of a previous exception Y, the ``InnerException`` +property of X should contain a reference to Y." This property is not set by +the VM automatically; rather, all exception constructors take an optional +``innerException`` argument to set it explicitly. The ``__cause__`` attribute +fulfills the same purpose as ``InnerException``, but this PEP proposes a new +form of ``raise`` rather than extending the constructors of all exceptions. C# +also provides a ``GetBaseException`` method that jumps directly to the end of +the ``InnerException`` chain; this PEP proposes no analog. + +The reason all three of these attributes are presented together in one proposal +is that the ``__traceback__`` attribute provides convenient access to the +traceback on chained exceptions. Implicit Exception Chaining +=========================== - Here is an example to illustrate the '__context__' attribute. +Here is an example to illustrate the ``__context__`` attribute:: - def compute(a, b): - try: - a/b - except Exception, exc: - log(exc) - - def log(exc): - file = open('logfile.txt') # oops, forgot the 'w' - print >>file, exc - file.close() - - Calling compute(0, 0) causes a ZeroDivisionError. The compute() - function catches this exception and calls log(exc), but the log() - function also raises an exception when it tries to write to a - file that wasn't opened for writing. - - In today's Python, the caller of compute() gets thrown an IOError. - The ZeroDivisionError is lost. With the proposed change, the - instance of IOError has an additional '__context__' attribute that - retains the ZeroDivisionError. - - The following more elaborate example demonstrates the handling of a - mixture of 'finally' and 'except' clauses: - - def main(filename): - file = open(filename) # oops, forgot the 'w' + def compute(a, b): + try: + a/b + except Exception, exc: + log(exc) + + def log(exc): + file = open('logfile.txt') # oops, forgot the 'w' + print >>file, exc + file.close() + +Calling ``compute(0, 0)`` causes a ``ZeroDivisionError``. The ``compute()`` +function catches this exception and calls ``log(exc)``, but the ``log()`` +function also raises an exception when it tries to write to a file that wasn't +opened for writing. + +In today's Python, the caller of ``compute()`` gets thrown an ``IOError``. The +``ZeroDivisionError`` is lost. With the proposed change, the instance of +``IOError`` has an additional ``__context__`` attribute that retains the +``ZeroDivisionError``. + +The following more elaborate example demonstrates the handling of a mixture of +``finally`` and ``except`` clauses:: + + def main(filename): + file = open(filename) # oops, forgot the 'w' try: try: compute() except Exception, exc: log(file, exc) finally: - file.clos() # oops, misspelled 'close' + file.clos() # oops, misspelled 'close' - def compute(): - 1/0 + def compute(): + 1/0 - def log(file, exc): - try: - print >>file, exc # oops, file is not writable - except: - display(exc) + def log(file, exc): + try: + print >>file, exc # oops, file is not writable + except: + display(exc) - def display(exc): - print ex # oops, misspelled 'exc' + def display(exc): + print ex # oops, misspelled 'exc' - Calling main() with the name of an existing file will trigger four - exceptions. The ultimate result will be an AttributeError due to - the misspelling of 'clos', whose __context__ points to a NameError - due to the misspelling of 'ex', whose __context__ points to an - IOError due to the file being read-only, whose __context__ points to - a ZeroDivisionError, whose __context__ attribute is None. +Calling ``main()`` with the name of an existing file will trigger four +exceptions. The ultimate result will be an ``AttributeError`` due to the +misspelling of ``clos``, whose ``__context__`` points to a ``NameError`` due +to the misspelling of ``ex``, whose ``__context__`` points to an ``IOError`` +due to the file being read-only, whose ``__context__`` points to a +``ZeroDivisionError``, whose ``__context__`` attribute is ``None``. - The proposed semantics are as follows: +The proposed semantics are as follows: - 1. Each thread has an exception context initially set to None. +1. Each thread has an exception context initially set to ``None``. - 2. Whenever an exception is raised, if the exception instance does - not already have a '__context__' attribute, the interpreter sets - it equal to the thread's exception context. +2. Whenever an exception is raised, if the exception instance does not already + have a ``__context__`` attribute, the interpreter sets it equal to the + thread's exception context. - 3. Immediately after an exception is raised, the thread's exception - context is set to the exception. +3. Immediately after an exception is raised, the thread's exception context is + set to the exception. - 4. Whenever the interpreter exits an 'except' block by reaching the - end or executing a 'return', 'yield', 'continue', or 'break' - statement, the thread's exception context is set to None. +4. Whenever the interpreter exits an ``except`` block by reaching the end or + executing a ``return``, ``yield``, ``continue``, or ``break`` statement, the + thread's exception context is set to ``None``. Explicit Exception Chaining +=========================== - The '__cause__' attribute on exception objects is always initialized - to None. It is set by a new form of the 'raise' statement: +The ``__cause__`` attribute on exception objects is always initialized to +``None``. It is set by a new form of the ``raise`` statement:: - raise EXCEPTION from CAUSE + raise EXCEPTION from CAUSE - which is equivalent to: +which is equivalent to:: - exc = EXCEPTION - exc.__cause__ = CAUSE - raise exc + exc = EXCEPTION + exc.__cause__ = CAUSE + raise exc - In the following example, a database provides implementations for a - few different kinds of storage, with file storage as one kind. The - database designer wants errors to propagate as DatabaseError objects - so that the client doesn't have to be aware of the storage-specific - details, but doesn't want to lose the underlying error information. +In the following example, a database provides implementations for a few +different kinds of storage, with file storage as one kind. The database +designer wants errors to propagate as ``DatabaseError`` objects so that the +client doesn't have to be aware of the storage-specific details, but doesn't +want to lose the underlying error information. - class DatabaseError(Exception): - pass +:: - class FileDatabase(Database): - def __init__(self, filename): - try: - self.file = open(filename) - except IOError, exc: - raise DatabaseError('failed to open') from exc + class DatabaseError(Exception): + pass + + class FileDatabase(Database): + def __init__(self, filename): + try: + self.file = open(filename) + except IOError, exc: + raise DatabaseError('failed to open') from exc - If the call to open() raises an exception, the problem will be - reported as a DatabaseError, with a __cause__ attribute that reveals - the IOError as the original cause. +If the call to ``open()`` raises an exception, the problem will be reported as +a ``DatabaseError``, with a ``__cause__`` attribute that reveals the +``IOError`` as the original cause. Traceback Attribute +=================== - The following example illustrates the '__traceback__' attribute. +The following example illustrates the ``__traceback__`` attribute. - def do_logged(file, work): - try: - work() - except Exception, exc: - write_exception(file, exc) - raise exc +:: - from traceback import format_tb + def do_logged(file, work): + try: + work() + except Exception, exc: + write_exception(file, exc) + raise exc - def write_exception(file, exc): - ... - type = exc.__class__ - message = str(exc) - lines = format_tb(exc.__traceback__) - file.write(... type ... message ... lines ...) - ... + from traceback import format_tb + + def write_exception(file, exc): + ... + type = exc.__class__ + message = str(exc) + lines = format_tb(exc.__traceback__) + file.write(... type ... message ... lines ...) + ... - In today's Python, the do_logged() function would have to extract - the traceback from sys.exc_traceback or sys.exc_info()[2] and pass - both the value and the traceback to write_exception(). With the - proposed change, write_exception() simply gets one argument and - obtains the exception using the '__traceback__' attribute. +In today's Python, the ``do_logged()`` function would have to extract the +traceback from ``sys.exc_traceback`` or ``sys.exc_info()`` [2]_ and pass both +the value and the traceback to ``write_exception()``. With the proposed +change, ``write_exception()`` simply gets one argument and obtains the +exception using the ``__traceback__`` attribute. - The proposed semantics are as follows: +The proposed semantics are as follows: - 1. Whenever an exception is caught, if the exception instance does - not already have a '__traceback__' attribute, the interpreter - sets it to the newly caught traceback. +1. Whenever an exception is caught, if the exception instance does not already + have a ``__traceback__`` attribute, the interpreter sets it to the newly + caught traceback. Enhanced Reporting +================== + +The default exception handler will be modified to report chained exceptions. +The chain of exceptions is traversed by following the ``__cause__`` and +``__context__`` attributes, with ``__cause__`` taking priority. In keeping +with the chronological order of tracebacks, the most recently raised exception +is displayed last; that is, the display begins with the description of the +innermost exception and backs up the chain to the outermost exception. The +tracebacks are formatted as usual, with one of the lines:: - The default exception handler will be modified to report chained - exceptions. The chain of exceptions is traversed by following the - '__cause__' and '__context__' attributes, with '__cause__' taking - priority. In keeping with the chronological order of tracebacks, - the most recently raised exception is displayed last; that is, the - display begins with the description of the innermost exception and - backs up the chain to the outermost exception. The tracebacks are - formatted as usual, with one of the lines: + The above exception was the direct cause of the following exception: - The above exception was the direct cause of the following exception: +or - or +:: - During handling of the above exception, another exception occurred: + During handling of the above exception, another exception occurred: - between tracebacks, depending whether they are linked by __cause__ - or __context__ respectively. Here is a sketch of the procedure: +between tracebacks, depending whether they are linked by ``__cause__`` or +``__context__`` respectively. Here is a sketch of the procedure:: - def print_chain(exc): - if exc.__cause__: - print_chain(exc.__cause__) - print '\nThe above exception was the direct cause...' - elif exc.__context__: - print_chain(exc.__context__) - print '\nDuring handling of the above exception, ...' - print_exc(exc) + def print_chain(exc): + if exc.__cause__: + print_chain(exc.__cause__) + print '\nThe above exception was the direct cause...' + elif exc.__context__: + print_chain(exc.__context__) + print '\nDuring handling of the above exception, ...' + print_exc(exc) - In the 'traceback' module, the format_exception, print_exception, - print_exc, and print_last functions will be updated to accept an - optional 'chain' argument, True by default. When this argument is - True, these functions will format or display the entire chain of - exceptions as just described. When it is False, these functions - will format or display only the outermost exception. +In the ``traceback`` module, the ``format_exception``, ``print_exception``, +``print_exc``, and ``print_last`` functions will be updated to accept an +optional ``chain`` argument, ``True`` by default. When this argument is +``True``, these functions will format or display the entire chain of exceptions +as just described. When it is ``False``, these functions will format or +display only the outermost exception. - The 'cgitb' module should also be updated to display the entire - chain of exceptions. +The ``cgitb`` module should also be updated to display the entire chain of +exceptions. C API +===== - The PyErr_Set* calls for setting exceptions will not set the - '__context__' attribute on exceptions. PyErr_NormalizeException - will always set the 'traceback' attribute to its 'tb' argument and - the '__context__' and '__cause__' attributes to None. +The ``PyErr_Set*`` calls for setting exceptions will not set the +``__context__`` attribute on exceptions. ``PyErr_NormalizeException`` will +always set the ``traceback`` attribute to its ``tb`` argument and the +``__context__`` and ``__cause__`` attributes to ``None``. - A new API function, PyErr_SetContext(context), will help C - programmers provide chained exception information. This function - will first normalize the current exception so it is an instance, - then set its '__context__' attribute. A similar API function, - PyErr_SetCause(cause), will set the '__cause__' attribute. +A new API function, ``PyErr_SetContext(context)``, will help C programmers +provide chained exception information. This function will first normalize the +current exception so it is an instance, then set its ``__context__`` attribute. +A similar API function, ``PyErr_SetCause(cause)``, will set the ``__cause__`` +attribute. Compatibility +============= - Chained exceptions expose the type of the most recent exception, so - they will still match the same 'except' clauses as they do now. +Chained exceptions expose the type of the most recent exception, so they will +still match the same ``except`` clauses as they do now. - The proposed changes should not break any code unless it sets or - uses attributes named '__context__', '__cause__', or '__traceback__' - on exception instances. As of 2005-05-12, the Python standard - library contains no mention of such attributes. +The proposed changes should not break any code unless it sets or uses +attributes named ``__context__``, ``__cause__``, or ``__traceback__`` on +exception instances. As of 2005-05-12, the Python standard library contains no +mention of such attributes. Open Issue: Extra Information +============================== - Walter Dörwald [11] expressed a desire to attach extra information - to an exception during its upward propagation without changing its - type. This could be a useful feature, but it is not addressed by - this PEP. It could conceivably be addressed by a separate PEP - establishing conventions for other informational attributes on - exceptions. +Walter Dörwald [11]_ expressed a desire to attach extra information to an +exception during its upward propagation without changing its type. This could +be a useful feature, but it is not addressed by this PEP. It could conceivably +be addressed by a separate PEP establishing conventions for other informational +attributes on exceptions. Open Issue: Suppressing Context +================================ - As written, this PEP makes it impossible to suppress '__context__', - since setting exc.__context__ to None in an 'except' or 'finally' - clause will only result in it being set again when exc is raised. +As written, this PEP makes it impossible to suppress ``__context__``, since +setting ``exc.__context__`` to ``None`` in an ``except`` or ``finally`` clause +will only result in it being set again when ``exc`` is raised. Open Issue: Limiting Exception Types +===================================== - To improve encapsulation, library implementors may want to wrap all - implementation-level exceptions with an application-level exception. - One could try to wrap exceptions by writing this: +To improve encapsulation, library implementors may want to wrap all +implementation-level exceptions with an application-level exception. One could +try to wrap exceptions by writing this:: - try: - ... implementation may raise an exception ... - except: - import sys - raise ApplicationError from sys.exc_value + try: + ... implementation may raise an exception ... + except: + import sys + raise ApplicationError from sys.exc_value - or this: +or this:: - try: - ... implementation may raise an exception ... - except Exception, exc: - raise ApplicationError from exc + try: + ... implementation may raise an exception ... + except Exception, exc: + raise ApplicationError from exc - but both are somewhat flawed. It would be nice to be able to name - the current exception in a catch-all 'except' clause, but that isn't - addressed here. Such a feature would allow something like this: +but both are somewhat flawed. It would be nice to be able to name the current +exception in a catch-all ``except`` clause, but that isn't addressed here. +Such a feature would allow something like this:: - try: - ... implementation may raise an exception ... - except *, exc: - raise ApplicationError from exc + try: + ... implementation may raise an exception ... + except *, exc: + raise ApplicationError from exc Open Issue: yield - - The exception context is lost when a 'yield' statement is executed; - resuming the frame after the 'yield' does not restore the context. - Addressing this problem is out of the scope of this PEP; it is not a - new problem, as demonstrated by the following example: - - >>> def gen(): - ... try: - ... 1/0 - ... except: - ... yield 3 - ... raise - ... - >>> g = gen() - >>> g.next() - 3 - >>> g.next() - TypeError: exceptions must be classes, instances, or strings - (deprecated), not NoneType +================== + +The exception context is lost when a ``yield`` statement is executed; resuming +the frame after the ``yield`` does not restore the context. Addressing this +problem is out of the scope of this PEP; it is not a new problem, as +demonstrated by the following example:: + + >>> def gen(): + ... try: + ... 1/0 + ... except: + ... yield 3 + ... raise + ... + >>> g = gen() + >>> g.next() + 3 + >>> g.next() + TypeError: exceptions must be classes, instances, or strings + (deprecated), not NoneType Open Issue: Garbage Collection +=============================== + +The strongest objection to this proposal has been that it creates cycles +between exceptions and stack frames [12]_. Collection of cyclic garbage (and +therefore resource release) can be greatly delayed. - The strongest objection to this proposal has been that it creates - cycles between exceptions and stack frames [12]. Collection of - cyclic garbage (and therefore resource release) can be greatly - delayed. +:: - >>> try: - >>> 1/0 - >>> except Exception, err: - >>> pass + >>> try: + >>> 1/0 + >>> except Exception, err: + >>> pass - will introduce a cycle from err -> traceback -> stack frame -> err, - keeping all locals in the same scope alive until the next GC happens. +will introduce a cycle from err -> traceback -> stack frame -> err, keeping all +locals in the same scope alive until the next GC happens. - Today, these locals would go out of scope. There is lots of code - which assumes that "local" resources -- particularly open files -- will - be closed quickly. If closure has to wait for the next GC, a program - (which runs fine today) may run out of file handles. +Today, these locals would go out of scope. There is lots of code which assumes +that "local" resources -- particularly open files -- will be closed quickly. +If closure has to wait for the next GC, a program (which runs fine today) may +run out of file handles. - Making the __traceback__ attribute a weak reference would avoid the - problems with cyclic garbage. Unfortunately, it would make saving - the Exception for later (as unittest does) more awkward, and it would - not allow as much cleanup of the sys module. +Making the ``__traceback__`` attribute a weak reference would avoid the +problems with cyclic garbage. Unfortunately, it would make saving the +``Exception`` for later (as ``unittest`` does) more awkward, and it would not +allow as much cleanup of the ``sys`` module. - A possible alternate solution, suggested by Adam Olsen, would be to - instead turn the reference from the stack frame to the 'err' variable - into a weak reference when the variable goes out of scope [13]. +A possible alternate solution, suggested by Adam Olsen, would be to instead +turn the reference from the stack frame to the ``err`` variable into a weak +reference when the variable goes out of scope [13]_. Possible Future Compatible Changes +================================== - These changes are consistent with the appearance of exceptions as - a single object rather than a triple at the interpreter level. +These changes are consistent with the appearance of exceptions as a single +object rather than a triple at the interpreter level. - - If PEP 340 or PEP 343 is accepted, replace the three (type, value, - traceback) arguments to __exit__ with a single exception argument. +- If PEP 340 or PEP 343 is accepted, replace the three (``type``, ``value``, + ``traceback``) arguments to ``__exit__`` with a single exception argument. - - Deprecate sys.exc_type, sys.exc_value, sys.exc_traceback, and - sys.exc_info() in favour of a single member, sys.exception. +- Deprecate ``sys.exc_type``, ``sys.exc_value``, ``sys.exc_traceback``, and + ``sys.exc_info()`` in favour of a single member, ``sys.exception``. - - Deprecate sys.last_type, sys.last_value, and sys.last_traceback - in favour of a single member, sys.last_exception. +- Deprecate ``sys.last_type``, ``sys.last_value``, and ``sys.last_traceback`` + in favour of a single member, ``sys.last_exception``. - - Deprecate the three-argument form of the 'raise' statement in - favour of the one-argument form. +- Deprecate the three-argument form of the ``raise`` statement in favour of the + one-argument form. - - Upgrade cgitb.html() to accept a single value as its first - argument as an alternative to a (type, value, traceback) tuple. +- Upgrade ``cgitb.html()`` to accept a single value as its first argument as an + alternative to a ``(type, value, traceback)`` tuple. Possible Future Incompatible Changes +==================================== - These changes might be worth considering for Python 3000. +These changes might be worth considering for Python 3000. - - Remove sys.exc_type, sys.exc_value, sys.exc_traceback, and - sys.exc_info(). +- Remove ``sys.exc_type``, ``sys.exc_value``, ``sys.exc_traceback``, and + ``sys.exc_info()``. - - Remove sys.last_type, sys.last_value, and sys.last_traceback. +- Remove ``sys.last_type``, ``sys.last_value``, and ``sys.last_traceback``. - - Replace the three-argument sys.excepthook with a one-argument - API, and changing the 'cgitb' module to match. +- Replace the three-argument ``sys.excepthook`` with a one-argument API, and + changing the ``cgitb`` module to match. - - Remove the three-argument form of the 'raise' statement. +- Remove the three-argument form of the ``raise`` statement. - - Upgrade traceback.print_exception to accept an 'exception' - argument instead of the type, value, and traceback arguments. +- Upgrade ``traceback.print_exception`` to accept an ``exception`` argument + instead of the ``type``, ``value``, and ``traceback`` arguments. Implementation +============== - The __traceback__ and __cause__ attributes and the new raise syntax were - implemented in revision 57783 [14]. +The ``__traceback__`` and ``__cause__`` attributes and the new raise syntax +were implemented in revision 57783 [14]_. Acknowledgements +================ - Brett Cannon, Greg Ewing, Guido van Rossum, Jeremy Hylton, Phillip - J. Eby, Raymond Hettinger, Walter Dörwald, and others. +Brett Cannon, Greg Ewing, Guido van Rossum, Jeremy Hylton, Phillip J. Eby, +Raymond Hettinger, Walter Dörwald, and others. References +========== - [1] Raymond Hettinger, "Idea for avoiding exception masking" - http://mail.python.org/pipermail/python-dev/2003-January/032492.html +.. [1] Raymond Hettinger, "Idea for avoiding exception masking" + http://mail.python.org/pipermail/python-dev/2003-January/032492.html - [2] Brett Cannon explains chained exceptions - http://mail.python.org/pipermail/python-dev/2003-June/036063.html +.. [2] Brett Cannon explains chained exceptions + http://mail.python.org/pipermail/python-dev/2003-June/036063.html - [3] Greg Ewing points out masking caused by exceptions during finally - http://mail.python.org/pipermail/python-dev/2003-June/036290.html +.. [3] Greg Ewing points out masking caused by exceptions during finally + http://mail.python.org/pipermail/python-dev/2003-June/036290.html - [4] Greg Ewing suggests storing the traceback in the exception object - http://mail.python.org/pipermail/python-dev/2003-June/036092.html +.. [4] Greg Ewing suggests storing the traceback in the exception object + http://mail.python.org/pipermail/python-dev/2003-June/036092.html - [5] Guido van Rossum mentions exceptions having a traceback attribute - http://mail.python.org/pipermail/python-dev/2005-April/053060.html +.. [5] Guido van Rossum mentions exceptions having a traceback attribute + http://mail.python.org/pipermail/python-dev/2005-April/053060.html - [6] Ka-Ping Yee, "Tidier Exceptions" - http://mail.python.org/pipermail/python-dev/2005-May/053671.html +.. [6] Ka-Ping Yee, "Tidier Exceptions" + http://mail.python.org/pipermail/python-dev/2005-May/053671.html - [7] Ka-Ping Yee, "Chained Exceptions" - http://mail.python.org/pipermail/python-dev/2005-May/053672.html +.. [7] Ka-Ping Yee, "Chained Exceptions" + http://mail.python.org/pipermail/python-dev/2005-May/053672.html - [8] Guido van Rossum discusses automatic chaining in PyErr_Set* - http://mail.python.org/pipermail/python-dev/2003-June/036180.html +.. [8] Guido van Rossum discusses automatic chaining in ``PyErr_Set*`` + http://mail.python.org/pipermail/python-dev/2003-June/036180.html - [9] Tony Olensky, "Omnibus Structured Exception/Error Handling Mechanism" - http://dev.perl.org/perl6/rfc/88.html +.. [9] Tony Olensky, "Omnibus Structured Exception/Error Handling Mechanism" + http://dev.perl.org/perl6/rfc/88.html - [10] MSDN .NET Framework Library, "Exception.InnerException Property" +.. [10] MSDN .NET Framework Library, "Exception.InnerException Property" http://msdn.microsoft.com/library/en-us/cpref/html/frlrfsystemexceptionclassinnerexceptiontopic.asp - [11] Walter Dörwald suggests wrapping exceptions to add details +.. [11] Walter Dörwald suggests wrapping exceptions to add details http://mail.python.org/pipermail/python-dev/2003-June/036148.html - [12] Guido van Rossum restates the objection to cyclic trash +.. [12] Guido van Rossum restates the objection to cyclic trash http://mail.python.org/pipermail/python-3000/2007-January/005322.html - [13] Adam Olsen suggests using a weakref from stack frame to exception +.. [13] Adam Olsen suggests using a weakref from stack frame to exception http://mail.python.org/pipermail/python-3000/2007-January/005363.html - [14] Patch to implement the bulk of the PEP +.. [14] Patch to implement the bulk of the PEP http://svn.python.org/view/python/branches/py3k/Include/?rev=57783&view=rev + Copyright +========= + +This document has been placed in the public domain. - This document has been placed in the public domain. - -Local Variables: -mode: indented-text -indent-tabs-mode: nil -sentence-end-double-space: t -fill-column: 70 -coding: utf-8 -End: +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: