Skip to content

Commit f3f2dfe

Browse files
asvetlovclaude
andcommitted
gh-160760: asyncio: run newly scheduled callbacks eagerly before polling for I/O
After running the ready callbacks, BaseEventLoop._run_once() now keeps running the callbacks they schedule, in batches, before it polls for I/O again. This saves one I/O poll per hop of a callback chain, for example when a completed future wakes up a task. Eager processing is bounded by the new asyncio.EventLoopSettings, set with loop.set_settings() and read with loop.get_settings(): eager_max_time (default 100 microseconds) and eager_max_callbacks (default 100). asyncio.sleep(0) ends eager processing, so that the task resumes only after the next I/O poll. When stop() is called during eager processing, run_forever() runs one more iteration, which polls for I/O and runs what the loop would have run without eager processing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent c71bf4d commit f3f2dfe

16 files changed

Lines changed: 697 additions & 19 deletions

‎Doc/library/asyncio-eventloop.rst‎

Lines changed: 98 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -130,7 +130,12 @@ Running and stopping the loop
130130
the loop will run the current batch of callbacks and then exit.
131131
Note that new callbacks scheduled by callbacks will not run in this
132132
case; instead, they will run the next time :meth:`run_forever` or
133-
:meth:`run_until_complete` is called.
133+
:meth:`run_until_complete` is called. If
134+
:ref:`eager callback processing <asyncio-eager-callbacks>` ran in
135+
the current or the previous iteration of the loop, the loop runs one
136+
more iteration after :meth:`stop` is called, which polls for I/O and
137+
runs all ready callbacks, including those scheduled after
138+
:meth:`stop`.
134139

135140
.. method:: loop.stop()
136141

@@ -213,7 +218,10 @@ Scheduling callbacks
213218
.. method:: loop.call_soon(callback, *args, context=None)
214219

215220
Schedule the *callback* :term:`callback` to be called with
216-
*args* arguments at the next iteration of the event loop.
221+
*args* arguments after the callbacks that are already scheduled.
222+
If it is scheduled by another callback, it may run in the same
223+
iteration of the event loop, see
224+
:ref:`eager callback processing <asyncio-eager-callbacks>`.
217225

218226
Return an instance of :class:`asyncio.Handle`,
219227
which can be used later to cancel the callback.
@@ -1578,6 +1586,94 @@ Enabling debug mode
15781586
The :ref:`debug mode of asyncio <asyncio-debug-mode>`.
15791587

15801588

1589+
.. _asyncio-eager-callbacks:
1590+
1591+
Eager callback processing
1592+
^^^^^^^^^^^^^^^^^^^^^^^^^
1593+
1594+
Each iteration of the event loop polls for I/O events and then runs the
1595+
callbacks that are ready. Callbacks scheduled by these callbacks, for
1596+
example tasks woken up by a completed future, do not have to wait for
1597+
the next I/O poll: the loop keeps running them, in the order they were
1598+
scheduled, for a short time. This saves one I/O poll per hop and speeds
1599+
up code that runs many short callbacks.
1600+
1601+
Eager processing in the current iteration ends when:
1602+
1603+
* there are no more ready callbacks;
1604+
* *eager_max_callbacks* additional callbacks have run;
1605+
* more than *eager_max_time* seconds have passed; the time is checked
1606+
between batches of callbacks, and a batch ends at the latest where the
1607+
callbacks scheduled while it runs begin;
1608+
* :func:`asyncio.sleep` is called with a delay of 0 or less; the rest of
1609+
the current batch then runs after the next I/O poll; or
1610+
* :meth:`loop.stop` is called; the loop then runs one more iteration,
1611+
which polls for I/O and runs the remaining ready callbacks without
1612+
eager processing, before :meth:`loop.run_forever` returns.
1613+
1614+
I/O events and timers are not processed while eager processing runs,
1615+
so larger limits can delay them.
1616+
1617+
.. note::
1618+
1619+
Eager processing can end before all ready callbacks have run: when
1620+
``asyncio.sleep(0)`` is called or a limit is reached, the callbacks
1621+
still queued are postponed to the next iteration of the loop, after
1622+
the I/O poll. How far a chain of callbacks gets before pending I/O
1623+
events and timers are handled therefore depends on its position in
1624+
the queue and on timing. Application code should not rely on this
1625+
order, but some unit tests do. Such tests can disable eager
1626+
processing::
1627+
1628+
loop.set_settings(asyncio.EventLoopSettings(eager_max_time=0,
1629+
eager_max_callbacks=0))
1630+
1631+
.. class:: EventLoopSettings(*, eager_max_time, eager_max_callbacks)
1632+
1633+
Immutable settings of an event loop.
1634+
1635+
*eager_max_time* is the time in seconds after which eager processing
1636+
in one iteration of the loop stops. It is checked between batches of
1637+
callbacks, so eager processing can take longer if the callbacks are
1638+
slow. It must be a non-negative finite :class:`int` or :class:`float`;
1639+
``0`` disables eager processing.
1640+
1641+
*eager_max_callbacks* is the maximum number of callbacks run eagerly
1642+
in one iteration of the loop. It must be a non-negative integer;
1643+
``0`` disables eager processing.
1644+
1645+
:exc:`TypeError` is raised if a value has a wrong type and
1646+
:exc:`ValueError` if it is out of range.
1647+
1648+
.. attribute:: DEFAULT
1649+
1650+
The default settings of a new event loop:
1651+
``EventLoopSettings(eager_max_time=0.0001, eager_max_callbacks=100)``.
1652+
1653+
.. versionadded:: next
1654+
1655+
.. method:: loop.get_settings()
1656+
1657+
Return the :class:`EventLoopSettings` of the event loop.
1658+
1659+
The default value is :attr:`EventLoopSettings.DEFAULT`.
1660+
1661+
.. versionadded:: next
1662+
1663+
.. method:: loop.set_settings(settings)
1664+
1665+
Set the :class:`EventLoopSettings` of the event loop.
1666+
1667+
The new settings are used starting from the next iteration of the loop.
1668+
1669+
To change a single setting, use :func:`copy.replace`::
1670+
1671+
loop.set_settings(copy.replace(asyncio.EventLoopSettings.DEFAULT,
1672+
eager_max_callbacks=1000))
1673+
1674+
.. versionadded:: next
1675+
1676+
15811677
Running subprocesses
15821678
^^^^^^^^^^^^^^^^^^^^
15831679

‎Doc/library/asyncio-llapi-index.rst‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,19 @@ See also the main documentation section about the
7878
- Get the current debug mode.
7979

8080

81+
.. rubric:: Loop settings
82+
.. list-table::
83+
:widths: 50 50
84+
:class: full-width-table
85+
86+
* - :meth:`loop.set_settings`
87+
- Set the :ref:`eager callback processing <asyncio-eager-callbacks>`
88+
settings.
89+
90+
* - :meth:`loop.get_settings`
91+
- Get the current :class:`EventLoopSettings`.
92+
93+
8194
.. rubric:: Scheduling Callbacks
8295
.. list-table::
8396
:widths: 50 50

‎Doc/library/asyncio-task.rst‎

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -511,6 +511,9 @@ Sleeping
511511
Setting the delay to 0 provides an optimized path to allow other
512512
tasks to run. This can be used by long-running functions to avoid
513513
blocking the event loop for the full duration of the function call.
514+
It also ends :ref:`eager callback processing <asyncio-eager-callbacks>`
515+
in the current iteration of the loop, so the task resumes only after
516+
the loop has polled for I/O.
514517

515518
.. _asyncio_example_sleep:
516519

@@ -538,6 +541,10 @@ Sleeping
538541
.. versionchanged:: 3.13
539542
Raises :exc:`ValueError` if *delay* is :data:`~math.nan`.
540543

544+
.. versionchanged:: next
545+
A delay of 0 or less ends eager callback processing in the current
546+
iteration of the event loop.
547+
541548

542549
Running tasks concurrently
543550
==========================
@@ -798,8 +805,8 @@ Timeouts
798805
as measured by the event loop's clock:
799806

800807
- If ``when`` is ``None``, the timeout will never trigger.
801-
- If ``when < loop.time()``, the timeout will trigger on the next
802-
iteration of the event loop.
808+
- If ``when < loop.time()``, the timeout will trigger as soon as
809+
possible, like a callback scheduled with :meth:`loop.call_soon`.
803810

804811
.. method:: when() -> float | None
805812

@@ -1436,7 +1443,7 @@ Task object
14361443
otherwise, return ``True``.
14371444

14381445
The method arranges for a :exc:`CancelledError` exception to be thrown
1439-
into the wrapped coroutine on the next cycle of the event loop.
1446+
into the wrapped coroutine the next time the task runs.
14401447

14411448
The coroutine then has a chance to clean up or even deny the
14421449
request by suppressing the exception with a :keyword:`try` ...

‎Doc/whatsnew/3.16.rst‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -144,6 +144,15 @@ asyncio
144144
socket file created for *path*.
145145
(Contributed by Sam Bull in :gh:`94984`.)
146146

147+
* The event loop now runs callbacks scheduled by other callbacks eagerly,
148+
within a small time and count budget, before polling for I/O again.
149+
This speeds up code that resolves futures or wakes tasks in chains.
150+
The limits can be changed with the new :meth:`loop.set_settings()
151+
<asyncio.loop.set_settings>` method and :class:`asyncio.EventLoopSettings`
152+
class. ``asyncio.sleep(0)`` ends eager processing in the current
153+
iteration. See :ref:`asyncio-eager-callbacks`.
154+
(Contributed by Andrew Svetlov in :gh:`160760`.)
155+
147156

148157
bdb
149158
---

‎Lib/asyncio/base_events.py‎

Lines changed: 75 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -434,6 +434,8 @@ def __init__(self):
434434
self._thread_id = None
435435
self._clock_resolution = time.get_clock_info('monotonic').resolution
436436
self._exception_handler = None
437+
self._settings = events.EventLoopSettings.DEFAULT
438+
self._eager_state = constants.EAGER_IDLE
437439
self.set_debug(coroutines._is_debug_mode())
438440
# The preserved state of async generator hooks.
439441
self._old_agen_hooks = None
@@ -667,6 +669,7 @@ def _run_forever_cleanup(self):
667669
loop cleanup logic.
668670
"""
669671
self._stopping = False
672+
self._eager_state = constants.EAGER_IDLE
670673
self._thread_id = None
671674
events._set_running_loop(None)
672675
self._set_coroutine_origin_tracking(False)
@@ -682,6 +685,8 @@ def run_forever(self):
682685
while True:
683686
self._run_once()
684687
if self._stopping:
688+
if self._eager_state >= 0:
689+
self._run_once()
685690
break
686691
finally:
687692
self._run_forever_cleanup()
@@ -2009,10 +2014,14 @@ def _timer_handle_cancelled(self, handle):
20092014
def _run_once(self):
20102015
"""Run one full iteration of the event loop.
20112016
2012-
This calls all currently ready callbacks, polls for I/O,
2013-
schedules the resulting callbacks, and finally schedules
2014-
'call_later' callbacks.
2017+
This polls for I/O, schedules the resulting callbacks and the
2018+
'call_later' callbacks that are due, and calls all ready
2019+
callbacks. Then it calls the callbacks scheduled by them
2020+
eagerly, within the limits of the loop settings.
20152021
"""
2022+
left = self._eager_state
2023+
if left >= 0 and self._stopping:
2024+
nready = len(self._ready)
20162025

20172026
sched_count = len(self._scheduled)
20182027
if (sched_count > _MIN_SCHEDULED_TIMER_HANDLES and
@@ -2068,11 +2077,59 @@ def _run_once(self):
20682077

20692078
# This is the only place where callbacks are actually *called*.
20702079
# All other places just add them to ready.
2071-
# Note: We run all currently scheduled callbacks, but not any
2072-
# callbacks scheduled by callbacks run this time around --
2073-
# they will be run the next time (after another I/O poll).
2080+
# Note: We run all currently scheduled callbacks. Callbacks
2081+
# scheduled by them are run eagerly in batches, within the limits
2082+
# of the loop settings; the rest will be run the next time (after
2083+
# another I/O poll).
20742084
# Use an idiom that is thread-safe without using locks.
2085+
self._eager_state = constants.EAGER_IDLE
2086+
settings = self._settings
2087+
if left >= 0 and self._stopping:
2088+
# The final iteration after stop() during eager processing: run
2089+
# what the loop would run without eager processing, and keep the
2090+
# callbacks scheduled for later iterations queued.
2091+
left = min(left, nready)
2092+
nnew = len(self._ready) - nready
2093+
self._run_handles(left)
2094+
later = [self._ready.popleft() for i in range(nready - left)]
2095+
self._run_handles(nnew)
2096+
self._ready.extendleft(reversed(later))
2097+
return
20752098
ntodo = len(self._ready)
2099+
if left > 0:
2100+
left = min(left, ntodo)
2101+
self._run_handles(left)
2102+
if self._stopping:
2103+
return
2104+
ntodo -= left
2105+
left = len(self._ready) - ntodo
2106+
self._run_handles(ntodo)
2107+
if (not self._ready or self._eager_state == constants.EAGER_BREAK
2108+
or self._stopping or settings.eager_max_time <= 0
2109+
or not settings.eager_max_callbacks):
2110+
if left > 0:
2111+
self._eager_state = left
2112+
return
2113+
2114+
# Eager processing: run the callbacks scheduled by the callbacks
2115+
# above, in batches, within the limits of the loop settings.
2116+
deadline = (self.time() + settings.eager_max_time
2117+
+ self._clock_resolution)
2118+
remaining = settings.eager_max_callbacks
2119+
while True:
2120+
if left <= 0:
2121+
left = len(self._ready)
2122+
ntodo = min(left, remaining, constants.EAGER_BATCH_SIZE)
2123+
remaining -= ntodo
2124+
left -= self._run_handles(ntodo, True)
2125+
if (not remaining or not self._ready
2126+
or self._eager_state == constants.EAGER_BREAK
2127+
or self._stopping or self.time() > deadline):
2128+
break
2129+
if left > 0 or self._stopping:
2130+
self._eager_state = left
2131+
2132+
def _run_handles(self, ntodo, eager=False):
20762133
for i in range(ntodo):
20772134
handle = self._ready.popleft()
20782135
if handle._cancelled:
@@ -2090,7 +2147,11 @@ def _run_once(self):
20902147
self._current_handle = None
20912148
else:
20922149
handle._run()
2150+
if eager and self._eager_state == constants.EAGER_BREAK:
2151+
ntodo = i + 1
2152+
break
20932153
handle = None # Needed to break cycles when an exception occurs.
2154+
return ntodo
20942155

20952156
def _set_coroutine_origin_tracking(self, enabled):
20962157
if bool(enabled) == bool(self._coroutine_origin_tracking_enabled):
@@ -2115,3 +2176,11 @@ def set_debug(self, enabled):
21152176

21162177
if self.is_running():
21172178
self.call_soon_threadsafe(self._set_coroutine_origin_tracking, enabled)
2179+
2180+
def get_settings(self):
2181+
return self._settings
2182+
2183+
def set_settings(self, settings):
2184+
if not isinstance(settings, events.EventLoopSettings):
2185+
raise TypeError('settings must be EventLoopSettings instance')
2186+
self._settings = settings

‎Lib/asyncio/constants.py‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,17 @@
3333
# Default timeout for joining the threads in the threadpool
3434
THREAD_JOIN_TIMEOUT = 300
3535

36+
# Maximum number of callbacks run during eager processing between checks
37+
# of the eager_max_time deadline. A batch also ends where the callbacks
38+
# scheduled during it begin, so for chains of callbacks the deadline is
39+
# checked after every callback.
40+
EAGER_BATCH_SIZE = 100
41+
42+
# Values of BaseEventLoop._eager_state other than a number of callbacks.
43+
EAGER_IDLE = -1
44+
# Set by asyncio.sleep(0) to end eager processing.
45+
EAGER_BREAK = -2
46+
3647
# The enum should be here to break circular dependencies between
3748
# base_events and sslproto
3849
class _SendfileMode(enum.Enum):

0 commit comments

Comments
 (0)