Move timeoutDeferred to an addTimeout method on Deferred, and make it…

… chainable Signed-off-by: cyli <cyli@twistedmatrix.com>
zmyer · Sep 30, 2016 · f4b3869 · f4b3869
1 parent aa33c19
commit f4b3869
Show file tree

Hide file tree

Showing 7 changed files with 458 additions and 427 deletions.
diff --git a/docs/core/howto/defer.rst b/docs/core/howto/defer.rst
@@ -332,6 +332,7 @@ Now ``isValidUser`` could be either ``synchronousIsValidUser`` or ``asynchronous
 
 It is also possible to modify ``synchronousIsValidUser`` to return a Deferred, see :doc:`Generating Deferreds <gendefer>` for more information.
 
+.. _core-howto-defer-deferreds-cancellation:
 
 Cancellation
 ------------
@@ -456,8 +457,75 @@ Now if someone calls ``cancel()`` on the ``Deferred`` returned from ``HTTPClient
 Care should be taken not to ``callback()`` a Deferred that has already been cancelled.
 
 
+.. _core-howto-defer-deferreds-timeouts:
+
+Timeouts
+--------
+
+Timeouts are a special case of :ref:`Cancellation <core-howto-defer-deferreds-cancellation>`.
+Let's say we have a :api:`twisted.internet.defer.Deferred <Deferred>` representing a task that may take a long time.
+We want to put an upper bound on that task, so we want the :api:`twisted.internet.defer.Deferred <Deferred>` to time
+out X seconds in the future.
+
+A convenient API to do so is :api:`twisted.internet.defer.Deferred.addTimeout <Deferred.addTimeout>`.
+By default, it will fail with a :api:`twisted.internet.defer.TimeoutError <TimeoutError>` if the :api:`twisted.internet.defer.Deferred <Deferred>` hasn't fired (with either an errback or a callback) within ``timeout`` seconds.
+
+.. code-block:: python
+
+    import random
+    from twisted.internet import task
+
+    def f():
+        return "Hopefully this will be called in 3 seconds or less"
+
+    def main(reactor):
+        delay = random.uniform(1, 5)
+
+        def called(result):
+            print("{0} seconds later:".format(delay), result)
+
+        d = task.deferLater(reactor, delay, f)
+        d.addTimeout(3, reactor).addBoth(called)
+
+        return d
+
+    # f() will be timed out if the random delay is greater than 3 seconds
+    task.react(main)
+
+
+:api:`twisted.internet.defer.Deferred.addTimeout <Deferred.addTimeout>` uses the :api:`twisted.internet.defer.Deferred.cancel <Deferred.cancel>` function under the hood, but can distinguish between a user's call to :api:`twisted.internet.defer.Deferred.cancel <Deferred.cancel>` and a cancellation due to a timeout.
+By default, :api:`twisted.internet.defer.Deferred.addTimeout <Deferred.addTimeout>` translates a :api:`twisted.internet.defer.CancelledError <CancelledError>` produced by the timeout into a :api:`twisted.internet.error.TimeoutError <TimeoutError>`.
+
+However, if you provided a custom :ref:`cancellation <core-howto-defer-deferreds-cancellation>` when creating the :api:`twisted.internet.defer.Deferred <Deferred>`, then cancelling it may not produce a :api:`twisted.internet.defer.CancelledError <CancelledError>`.  In this case, the default behavior of :api:`twisted.internet.defer.Deferred.addTimeout <Deferred.addTimeout>` is to preserve whatever callback or errback value your custom cancellation function produced.  This can be useful if, for instance, a cancellation or timeout should produce a default value instead of an error.
+
+:api:`twisted.internet.defer.Deferred.addTimeout <Deferred.addTimeout>` also takes an optional callable ``onTimeoutCancel`` which is called immediately after the deferred times out.  ``onTimeoutCancel`` is not called if it the deferred is otherwise cancelled before the timeout. It takes an arbitrary value, which is the value of the deferred at that exact time (probably a :api:`twisted.internet.defer.CancelledError <CancelledError>` :api:`twisted.python.failure.Failure <Failure>`), and the ``timeout``.  This can be useful if, for instance, the cancellation or timeout does not result in an error but you want to log the timeout anyway.  It can also be used to alter the return value.
+
+.. code-block:: python
+
+    from twisted.internet import task, defer
+
+    def logTimeout(result, timeout):
+        print("Got {0!r} but actually timed out after {1} seconds".format(
+            result, timeout))
+        return result + " (timed out)"
+
+    def main(reactor):
+        # generate a deferred with a custom canceller function, and never
+        # never callback or errback it to guarantee it gets timed out
+        d = defer.Deferred(lambda c: c.callback("Everything's ok!"))
+        d.addTimeout(2, reactor, onTimeoutCancel=logTimeout)
+        d.addBoth(print)
+        return d
+
+    task.react(main)
+
+
+Note that the exact place in the callback chain that :api:`twisted.internet.defer.Deferred.addTimeout <Deferred.addTimeout>` is added determines how much of the callback chain should be timed out.  The timeout encompasses all the callbacks and errbacks added to the :api:`twisted.internet.defer.Deferred <Deferred>` before the call to :api:`twisted.internet.defer.Deferred.addTimeout <addTimeout>`, and none of the callbacks and errbacks added after the call.  The timeout also starts counting down as soon as soon as it's invoked.
+
+
 .. _core-howto-defer-deferredlist:
 
+
 DeferredList
 ------------
 

diff --git a/docs/core/howto/time.rst b/docs/core/howto/time.rst
@@ -130,61 +130,7 @@ As with all reactor-based code, in order for scheduling to work the reactor must
 
 
 
-Timing out outstanding tasks
-============================
+See also
+--------
 
-Let's say we have a :api:`twisted.internet.defer.Deferred <Deferred>` representing a task that may take a long time.
-We want to put an upper bound on that task, so we want the :api:`twisted.internet.defer.Deferred <Deferred>` to time
-out X seconds in the future.
-
-A convenient API to do so is :api:`twisted.internet.task.timeoutDeferred <twisted.internet.task.timeoutDeferred>`.
-By default, it will fail with a :api:`twisted.internet.error.TimeoutError <TimeoutError>` if the :api:`twisted.internet.defer.Deferred <Deferred>` hasn't fired (with either an errback or a callback) within ``timeout`` seconds.
-
-.. code-block:: python
-
-    import random
-    from twisted.internet import task
-
-    def f():
-        return "Hopefully this will be called in 3 seconds or less"
-
-    def main(reactor):
-        delay = random.uniform(1, 5)
-        d = task.deferLater(reactor, delay, f)
-        task.timeoutDeferred(d, 3, reactor)
-
-        def called(result):
-            print("{0} seconds later:".format(delay), result)
-        d.addBoth(called)
-
-        return d
-
-    # f() will be timed out if the random delay is greater than 3 seconds
-    task.react(main)
-
-
-:api:`twisted.internet.task.timeoutDeferred <timeoutDeferred>` uses the :api:`twisted.internet.defer.Deferred.cancel <Deferred.cancel>` function under the hood, but can distinguish between a user's call to :api:`twisted.internet.defer.Deferred.cancel <Deferred.cancel>` and a cancellation due to a timeout.
-By default, :api:`twisted.internet.task.timeoutDeferred <timeoutDeferred>` translates a :api:`twisted.internet.defer.CancelledError <CancelledError>` produced by the timeout into a :api:`twisted.internet.error.TimeoutError <TimeoutError>`.
-
-However, if you provided a custom cancellation callable when creating the :api:`twisted.internet.defer.Deferred <Deferred>`, then cancelling it may not produce a :api:`twisted.internet.defer.CancelledError <CancelledError>`.  In this case, the default behavior of :api:`twisted.internet.task.timeoutDeferred <timeoutDeferred>` is to preserve whatever callback or errback value your custom cancellation function produced.  This can be useful if, for instance, a cancellation or timeout should produce a default value instead of an error.
-
-:api:`twisted.internet.task.timeoutDeferred <timeoutDeferred>` also takes an optional callable ``onTimeoutCancel`` which is called immediately after the deferred times out.  ``onTimeoutCancel`` is not called if it the deferred is otherwise cancelled before the timeout. It takes an arbitrary value, which is the value of the deferred at that exact time (probably a :api:`twisted.internet.defer.CancelledError <CancelledError>` :api:`twisted.python.failure.Failure <Failure>`), and the ``timeout``.  This can be useful if, for instance, the cancellation or timeout does not result in an error but you want to log the timeout anyway.  It can also be used to alter the return value.
-
-.. code-block:: python
-
-    from twisted.internet import task, defer
-
-    def logTimeout(result, timeout):
-        print("Got {0!r} but actually timed out after {1} seconds".format(
-            result, timeout))
-        return result + " (timed out)"
-
-    def main(reactor):
-        # generate a deferred with a custom canceller function, and never
-        # never callback or errback it to guarantee it gets timed out
-        d = defer.Deferred(lambda c: c.callback("Everything's ok!"))
-        task.timeoutDeferred(d, 2, reactor, onTimeoutCancel=logTimeout)
-        d.addBoth(print)
-        return d
-
-    task.react(main)
+#. :ref:`Timing out Deferreds <core-howto-defer-deferreds-timeouts>`
diff --git a/src/twisted/internet/defer.py b/src/twisted/internet/defer.py
@@ -47,7 +47,7 @@ class CancelledError(Exception):
 
 class TimeoutError(Exception):
     """
-    This exception is deprecated.
+    This error is raised by default when a L{Deferred} times out.
     """
 
 
@@ -340,6 +340,68 @@ def addBoth(self, callback, *args, **kw):
                                  callbackKeywords=kw, errbackKeywords=kw)
 
 
+    def addTimeout(self, timeout, clock, onTimeoutCancel=None):
+        """
+        Time out this L{Deferred} by scheduling it to be cancelled after
+        C{timeout} seconds.
+
+        The timeout encompasses all the callbacks and errbacks added to this
+        L{defer.Deferred} before the call to L{addTimeout}, and none added
+        after the call.
+
+        If this L{Deferred} gets timed out, it errbacks with a L{TimeoutError},
+        unless a cancelable function was passed to its initialization or unless
+        a different C{onTimeoutCancel} callable is provided.
+
+        @param timeout: number of seconds to wait before timing out this
+            L{Deferred}
+        @type timeout: L{int}
+
+        @param clock: The object which will be used to schedule the timeout.
+        @type clock: L{twisted.internet.interfaces.IReactorTime}
+
+        @param onTimeoutCancel: A callable which is called immediately after
+            this L{Deferred} times out, and not if this L{Deferred} is
+            otherwise cancelled before the timeout. It takes an arbitrary
+            value, which is the value of this L{Deferred} at that exact point
+            in time (probably a L{CancelledError} L{Failure}), and the
+            C{timeout}.  The default callable (if none is provided) will
+            translate a L{CancelledError} L{Failure} into a L{TimeoutError}.
+        @type onTimeoutCancel: L{callable}
+
+        @return: C{self}.
+        @rtype: a L{Deferred}
+
+        @since: 16.5
+        """
+        timedOut = [False]
+
+        def timeItOut():
+            timedOut[0] = True
+            self.cancel()
+
+        delayedCall = clock.callLater(timeout, timeItOut)
+
+        def convertCancelled(value):
+            # if C{deferred} was timed out, call the translation function,
+            # if provdied, otherwise just use L{cancelledToTimedOutError}
+            if timedOut[0]:
+                toCall = onTimeoutCancel or _cancelledToTimedOutError
+                return toCall(value, timeout)
+            return value
+
+        self.addBoth(convertCancelled)
+
+        def cancelTimeout(result):
+            # stop the pending call to cancel the deferred if it's been fired
+            if delayedCall.active():
+                delayedCall.cancel()
+            return result
+
+        self.addBoth(cancelTimeout)
+        return self
+
+
     def chainDeferred(self, d):
         """
         Chain another L{Deferred} to this L{Deferred}.
@@ -678,6 +740,29 @@ def __send__(self, value=None):
 
 
 
+def _cancelledToTimedOutError(value, timeout):
+    """
+    A default translation function that translates L{Failure}s that are
+    L{CancelledError}s to L{TimeoutError}s.
+
+    @param value: Anything
+    @type value: Anything
+
+    @param timeout: The timeout
+    @type timeout: L{int}
+
+    @rtype: C{value}
+    @raise: L{TimeoutError}
+
+    @since: 16.5
+    """
+    if isinstance(value, failure.Failure):
+        value.trap(CancelledError)
+        raise TimeoutError(timeout, "Deferred")
+    return value
+
+
+
 def ensureDeferred(coro):
     """
     Transform a coroutine that uses L{Deferred}s into a L{Deferred} itself.

diff --git a/src/twisted/internet/task.py b/src/twisted/internet/task.py
@@ -24,7 +24,7 @@
 
 from twisted.internet import base, defer
 from twisted.internet.interfaces import IReactorTime
-from twisted.internet.error import ReactorNotRunning, TimeoutError
+from twisted.internet.error import ReactorNotRunning
 
 
 class LoopingCall:
@@ -937,91 +937,11 @@ def cbFinish(result):
     sys.exit(codes[0])
 
 
-def _cancelledToTimedOutError(value, timeout):
-    """
-    A translation function that translate L{Failure}'s that are L{defer.CancelledError} to L{TimeoutError}.
-
-    @param value: Anything
-    @type value: Anything
-
-    @param timeout: The timeout
-    @type timeout: L{int}
-
-    @rtype: C{value}
-    @raise: L{TimeoutError}
-
-    @since: 16.5
-    """
-    if isinstance(value, Failure):
-        value.trap(defer.CancelledError)
-        raise TimeoutError(timeout, "Deferred")
-    return value
-
-
-def timeoutDeferred(deferred, timeout, clock, onTimeoutCancel=None):
-    """
-    Timeout a L{defer.Deferred} by scheduling it to be cancelled after C{timeout} seconds.
-
-    If it gets timed out, it errbacks with a L{TimeoutError}, unless a
-    cancelable function is passed to the C{deferred}'s initialization or unless
-    a different C{onTimeoutCancel} callable is provided.
-
-    (see the documentation for L{defer.Deferred} for more details about
-    cancellation functions.)
-
-    @param deferred: The L{defer.Deferred} to time out (cancel)
-    @type deferred: L{defer.Deferred}
-
-    @param timeout: number of seconds to wait before timing out the deferred
-    @type timeout: L{int}
-
-    @param clock: The object which will be used to schedule the timeout.
-    @type clock: L{IReactorTime}
-
-    @param onTimeoutCancel: A callable which is called immediately after C{deferred}
-        times out, and not if C{deferred} is otherwise cancelled before the timeout.
-        It takes an arbitrary value, which is the value of the deferred at that
-        exact time (probably a L{defer.CancelledError} L{Failure}), and the
-        C{timeout}.  The default callable (if none is provided) will translate
-        a L{defer.CancelledError} L{Failure} into a L{TimeoutError}.
-    @type onTimeoutCancel: L{callable}
-
-    @rtype: L{None}
-
-    @since: 16.5
-    """
-    timedOut = [False]
-
-    def timeItOut():
-        timedOut[0] = True
-        deferred.cancel()
-
-    delayedCall = clock.callLater(timeout, timeItOut)
-
-    def convertCancelled(value):
-        # if C{deferred} was timed out, call the translation function,
-        # if provdied, otherwise just use L{cancelledToTimedOutError}
-        if timedOut[0]:
-            toCall = onTimeoutCancel or _cancelledToTimedOutError
-            return toCall(value, timeout)
-        return value
-
-    deferred.addBoth(convertCancelled)
-
-    def cancelTimeout(result):
-        # stop the pending call to cancel the deferred if it's been fired
-        if delayedCall.active():
-            delayedCall.cancel()
-        return result
-
-    deferred.addBoth(cancelTimeout)
-
-
 __all__ = [
     'LoopingCall',
 
     'Clock',
 
     'SchedulerStopped', 'Cooperator', 'coiterate',
 
-    'deferLater', 'react', 'timeoutDeferred']
+    'deferLater', 'react']