New release v2.16.3 (#82619 )

Update Ansible release version to v2.16.3rc1.post0. (#82591 )
New release v2.16.3rc1 (#82587 )
2024-01-29 14:40:01 -06:00 · 2024-01-22 21:00:14 +00:00 · 2024-01-22 13:00:19 -06:00 · 2024-01-22 09:59:21 -06:00 · 2024-01-19 14:52:11 -08:00 · 2024-01-19 14:48:07 -08:00
250 changed files with 4068 additions and 947 deletions
--- a/changelogs/CHANGELOG-v2.16.rst
+++ b/changelogs/CHANGELOG-v2.16.rst
@ -0,0 +1,388 @@
+=============================================
+ansible-core 2.16 "All My Love" Release Notes
+=============================================
+
+.. contents:: Topics
+
+
+v2.16.3
+=======
+
+Release Summary
+---------------
+
+| Release Date: 2024-01-29
+| `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+
+Security Fixes
+--------------
+
+- ANSIBLE_NO_LOG - Address issue where ANSIBLE_NO_LOG was ignored (CVE-2024-0690)
+
+Bugfixes
+--------
+
+- Run all handlers with the same ``listen`` topic, even when notified from another handler (https://github.com/ansible/ansible/issues/82363).
+- ``ansible-galaxy role import`` - fix using the ``role_name`` in a standalone role's ``galaxy_info`` metadata by disabling automatic removal of the ``ansible-role-`` prefix. This matches the behavior of the Galaxy UI which also no longer implicitly removes the ``ansible-role-`` prefix. Use the ``--role-name`` option or add a ``role_name`` to the ``galaxy_info`` dictionary in the role's ``meta/main.yml`` to use an alternate role name.
+- ``ansible-test sanity --test runtime-metadata`` - add ``action_plugin`` as a valid field for modules in the schema (https://github.com/ansible/ansible/pull/82562).
+- ansible-config init will now dedupe ini entries from plugins.
+- ansible-galaxy role import - exit with 1 when the import fails (https://github.com/ansible/ansible/issues/82175).
+- ansible-galaxy role install - normalize tarfile paths and symlinks using ``ansible.utils.path.unfrackpath`` and consider them valid as long as the realpath is in the tarfile's role directory (https://github.com/ansible/ansible/issues/81965).
+- delegate_to when set to an empty or undefined variable will now give a proper error.
+- dwim functions for lookups should be better at detectging role context even in abscense of tasks/main.
+- roles, code cleanup and performance optimization of dependencies, now cached,  and ``public`` setting is now determined once, at role instantiation.
+- roles, the ``static`` property is now correctly set, this will fix issues with ``public`` and ``DEFAULT_PRIVATE_ROLE_VARS`` controls on exporting vars.
+- unsafe data - Enable directly using ``AnsibleUnsafeText`` with Python ``pathlib`` (https://github.com/ansible/ansible/issues/82414)
+
+v2.16.2
+=======
+
+Release Summary
+---------------
+
+| Release Date: 2023-12-11
+| `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+
+Bugfixes
+--------
+
+- unsafe data - Address an incompatibility when iterating or getting a single index from ``AnsibleUnsafeBytes``
+- unsafe data - Address an incompatibility with ``AnsibleUnsafeText`` and ``AnsibleUnsafeBytes`` when pickling with ``protocol=0``
+
+v2.16.1
+=======
+
+Release Summary
+---------------
+
+| Release Date: 2023-12-04
+| `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+
+Breaking Changes / Porting Guide
+--------------------------------
+
+- assert - Nested templating may result in an inability for the conditional to be evaluated. See the porting guide for more information.
+
+Security Fixes
+--------------
+
+- templating - Address issues where internal templating can cause unsafe variables to lose their unsafe designation (CVE-2023-5764)
+
+Bugfixes
+--------
+
+- Fix issue where an ``include_tasks`` handler in a role was not able to locate a file in ``tasks/`` when ``tasks_from`` was used as a role entry point and ``main.yml`` was not present (https://github.com/ansible/ansible/issues/82241)
+- Plugin loader does not dedupe nor cache filter/test plugins by file basename, but full path name.
+- Restoring the ability of filters/tests can have same file base name but different tests/filters defined inside.
+- ansible-pull now will expand relative paths for the ``-d|--directory`` option is now expanded before use.
+- ansible-pull will now correctly handle become and connection password file options for ansible-playbook.
+- flush_handlers - properly handle a handler failure in a nested block when ``force_handlers`` is set (http://github.com/ansible/ansible/issues/81532)
+- module no_log will no longer affect top level booleans, for example ``no_log_module_parameter='a'`` will no longer hide ``changed=False`` as a 'no log value' (matches 'a').
+- role params now have higher precedence than host facts again, matching documentation, this had unintentionally changed in 2.15.
+- wait_for should not handle 'non mmapable files' again.
+
+v2.16.0
+=======
+
+Release Summary
+---------------
+
+| Release Date: 2023-11-06
+| `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+
+Minor Changes
+-------------
+
+- Add Python type hints to the Display class (https://github.com/ansible/ansible/issues/80841)
+- Add ``GALAXY_COLLECTIONS_PATH_WARNING`` option to disable the warning given by ``ansible-galaxy collection install`` when installing a collection to a path that isn't in the configured collection paths.
+- Add ``python3.12`` to the default ``INTERPRETER_PYTHON_FALLBACK`` list.
+- Add ``utcfromtimestamp`` and ``utcnow`` to ``ansible.module_utils.compat.datetime`` to return fixed offset datetime objects.
+- Add a general ``GALAXY_SERVER_TIMEOUT`` config option for distribution servers (https://github.com/ansible/ansible/issues/79833).
+- Added Python type annotation to connection plugins
+- CLI argument parsing - Automatically prepend to the help of CLI arguments that support being specified multiple times. (https://github.com/ansible/ansible/issues/22396)
+- DEFAULT_TRANSPORT now defaults to 'ssh', the old 'smart' option is being deprecated as versions of OpenSSH without control persist are basically not present anymore.
+- Documentation for set filters ``intersect``, ``difference``, ``symmetric_difference`` and ``union`` now states that the returned list items are in arbitrary order.
+- Record ``removal_date`` in runtime metadata as a string instead of a date.
+- Remove the ``CleansingNodeVisitor`` class and its usage due to the templating changes that made it superfluous. Also simplify the ``Conditional`` class.
+- Removed ``exclude`` and ``recursive-exclude`` commands for generated files from the ``MANIFEST.in`` file. These excludes were unnecessary since releases are expected to be built with a clean worktree.
+- Removed ``exclude`` commands for sanity test files from the ``MANIFEST.in`` file. These tests were previously excluded because they did not pass when run from an sdist. However, sanity tests are not expected to pass from an sdist, so excluding some (but not all) of the failing tests makes little sense.
+- Removed redundant ``include`` commands from the ``MANIFEST.in`` file. These includes either duplicated default behavior or another command.
+- The ``ansible-core`` sdist no longer contains pre-generated man pages. Instead, a ``packaging/cli-doc/build.py`` script is included in the sdist. This script can generate man pages and standalone RST documentation for ``ansible-core`` CLI programs.
+- The ``docs`` and ``examples`` directories are no longer included in the ``ansible-core`` sdist. These directories have been moved to the https://github.com/ansible/ansible-documentation repository.
+- The minimum required ``setuptools`` version is now 66.1.0, as it is the oldest version to support Python 3.12.
+- Update ``ansible_service_mgr`` fact to include init system for SMGL OS family
+- Use ``ansible.module_utils.common.text.converters`` instead of ``ansible.module_utils._text``.
+- Use ``importlib.resources.abc.TraversableResources`` instead of deprecated ``importlib.abc.TraversableResources`` where available (https:/github.com/ansible/ansible/pull/81082).
+- Use ``include`` where ``recursive-include`` is unnecessary in the ``MANIFEST.in`` file.
+- Use ``package_data`` instead of ``include_package_data`` for ``setup.cfg`` to avoid ``setuptools`` warnings.
+- Utilize gpg check provided internally by the ``transaction.run`` method as oppose to calling it manually.
+- ``Templar`` - do not add the ``dict`` constructor to ``globals`` as all required Jinja2 versions already do so
+- ansible-doc - allow to filter listing of collections and metadata dump by more than one collection (https://github.com/ansible/ansible/pull/81450).
+- ansible-galaxy - Add a plural option to improve ignoring multiple signature error status codes when installing or verifying collections. A space-separated list of error codes can follow --ignore-signature-status-codes in addition to specifying --ignore-signature-status-code multiple times (for example, ``--ignore-signature-status-codes NO_PUBKEY UNEXPECTED``).
+- ansible-galaxy - Remove internal configuration argument ``v3`` (https://github.com/ansible/ansible/pull/80721)
+- ansible-galaxy - add note to the collection dependency resolver error message about pre-releases if ``--pre`` was not provided (https://github.com/ansible/ansible/issues/80048).
+- ansible-galaxy - used to crash out with a "Errno 20 Not a directory" error when extracting files from a role when hitting a file with an illegal name (https://github.com/ansible/ansible/pull/81553). Now it gives a warning identifying the culprit file and the rule violation (e.g., ``my$class.jar`` has a ``$`` in the name) before crashing out, giving the user a chance to remove the invalid file and try again. (https://github.com/ansible/ansible/pull/81555).
+- ansible-test - Add Alpine 3.18 to remotes
+- ansible-test - Add Fedora 38 container.
+- ansible-test - Add Fedora 38 remote.
+- ansible-test - Add FreeBSD 13.2 remote.
+- ansible-test - Add new pylint checker for new ``# deprecated:`` comments within code to trigger errors when time to remove code that has no user facing deprecation message. Only supported in ansible-core, not collections.
+- ansible-test - Add support for RHEL 8.8 remotes.
+- ansible-test - Add support for RHEL 9.2 remotes.
+- ansible-test - Add support for testing with Python 3.12.
+- ansible-test - Allow float values for the ``--timeout`` option to the ``env`` command. This simplifies testing.
+- ansible-test - Enable ``thread`` code coverage in addition to the existing ``multiprocessing`` coverage.
+- ansible-test - Make Python 3.12 the default version used in the ``base`` and ``default`` containers.
+- ansible-test - RHEL 8.8 provisioning can now be used with the ``--python 3.11`` option.
+- ansible-test - RHEL 9.2 provisioning can now be used with the ``--python 3.11`` option.
+- ansible-test - Refactored ``env`` command logic and timeout handling.
+- ansible-test - Remove Fedora 37 remote support.
+- ansible-test - Remove Fedora 37 test container.
+- ansible-test - Remove Python 3.8 and 3.9 from RHEL 8.8.
+- ansible-test - Remove obsolete embedded script for configuring WinRM on Windows remotes.
+- ansible-test - Removed Ubuntu 20.04 LTS image from the `--remote` option.
+- ansible-test - Removed `freebsd/12.4` remote.
+- ansible-test - Removed `freebsd/13.1` remote.
+- ansible-test - Removed test remotes: rhel/8.7, rhel/9.1
+- ansible-test - Removed the deprecated ``--docker-no-pull`` option.
+- ansible-test - Removed the deprecated ``--no-pip-check`` option.
+- ansible-test - Removed the deprecated ``foreman`` test plugin.
+- ansible-test - Removed the deprecated ``govcsim`` support from the ``vcenter`` test plugin.
+- ansible-test - Replace the ``pytest-forked`` pytest plugin with a custom plugin.
+- ansible-test - The ``no-get-exception`` sanity test is now limited to plugins in collections. Previously any Python file in a collection was checked for ``get_exception`` usage.
+- ansible-test - The ``replace-urlopen`` sanity test is now limited to plugins in collections. Previously any Python file in a collection was checked for ``urlopen`` usage.
+- ansible-test - The ``use-compat-six`` sanity test is now limited to plugins in collections. Previously any Python file in a collection was checked for ``six`` usage.
+- ansible-test - The openSUSE test container has been updated to openSUSE Leap 15.5.
+- ansible-test - Update pip to ``23.1.2`` and setuptools to ``67.7.2``.
+- ansible-test - Update the ``default`` containers.
+- ansible-test - Update the ``nios-test-container`` to version 2.0.0, which supports API version 2.9.
+- ansible-test - Update the logic used to detect when ``ansible-test`` is running from source.
+- ansible-test - Updated the CloudStack test container to version 1.6.1.
+- ansible-test - Updated the distro test containers to version 6.3.0 to include coverage 7.3.2 for Python 3.8+. The alpine3 container is now based on 3.18 instead of 3.17 and includes Python 3.11 instead of Python 3.10.
+- ansible-test - Use ``datetime.datetime.now`` with ``tz`` specified instead of ``datetime.datetime.utcnow``.
+- ansible-test - Use a context manager to perform cleanup at exit instead of using the built-in ``atexit`` module.
+- ansible-test - When invoking ``sleep`` in containers during container setup, the ``env`` command is used to avoid invoking the shell builtin, if present.
+- ansible-test - remove Alpine 3.17 from remotes
+- ansible-test — Python 3.8–3.12 will use ``coverage`` v7.3.2.
+- ansible-test — ``coverage`` v6.5.0 is to be used only under Python 3.7.
+- ansible-vault create: Now raises an error when opening the editor without tty. The flag --skip-tty-check restores previous behaviour.
+- ansible_user_module - tweaked macos user defaults to reflect expected defaults (https://github.com/ansible/ansible/issues/44316)
+- apt - return calculated diff while running apt clean operation.
+- blockinfile - add append_newline and prepend_newline options (https://github.com/ansible/ansible/issues/80835).
+- cli - Added short option '-J' for asking for vault password (https://github.com/ansible/ansible/issues/80523).
+- command - Add option ``expand_argument_vars`` to disable argument expansion and use literal values - https://github.com/ansible/ansible/issues/54162
+- config lookup new option show_origin to also return the origin of a configuration value.
+- display methods for warning and deprecation are now proxied to main process when issued from a fork. This allows for the deduplication of warnings and deprecations to work globally.
+- dnf5 - enable environment groups installation testing in CI as its support was added.
+- dnf5 - enable now implemented ``cacheonly`` functionality
+- executor now skips persistent connection when it detects an action that does not require a connection.
+- find module - Add ability to filter based on modes
+- gather_facts now will use gather_timeout setting to limit parallel execution of modules that do not themselves use gather_timeout.
+- group - remove extraneous warning shown when user does not exist (https://github.com/ansible/ansible/issues/77049).
+- include_vars - os.walk now follows symbolic links when traversing directories (https://github.com/ansible/ansible/pull/80460)
+- module compression is now sourced directly via config, bypassing play_context possibly stale values.
+- reboot - show last error message in verbose logs (https://github.com/ansible/ansible/issues/81574).
+- service_facts now returns more info for rcctl managed systesm (OpenBSD).
+- tasks - the ``retries`` keyword can be specified without ``until`` in which case the task is retried until it succeeds but at most ``retries`` times (https://github.com/ansible/ansible/issues/20802)
+- user - add new option ``password_expire_warn`` (supported on Linux only) to set the number of days of warning before a password change is required (https://github.com/ansible/ansible/issues/79882).
+- yum_repository - Align module documentation with parameters
+
+Breaking Changes / Porting Guide
+--------------------------------
+
+- Any plugin using the config system and the `cli` entry to use the `timeout` from the command line, will see the value change if the use had configured it in any of the lower precedence methods. If relying on this behaviour to consume the global/generic timeout from the DEFAULT_TIMEOUT constant, please consult the documentation on plugin configuration to add the overlaping entries.
+- ansible-test - Test plugins that rely on containers no longer support reusing running containers. The previous behavior was an undocumented, untested feature.
+- service module will not permanently configure variables/flags for openbsd when doing enable/disable operation anymore, this module was never meant to do this type of work, just to manage the service state itself. A rcctl_config or similar module should be created and used instead.
+
+Deprecated Features
+-------------------
+
+- Deprecated ini config option ``collections_paths``, use the singular form ``collections_path`` instead
+- Deprecated the env var ``ANSIBLE_COLLECTIONS_PATHS``, use the singular form ``ANSIBLE_COLLECTIONS_PATH`` instead
+- Old style vars plugins which use the entrypoints `get_host_vars` or `get_group_vars` are deprecated. The plugin should be updated to inherit from `BaseVarsPlugin` and define a `get_vars` method as the entrypoint.
+- Support for Windows Server 2012 and 2012 R2 has been removed as the support end of life from Microsoft is October 10th 2023. These versions of Windows will no longer be tested in this Ansible release and it cannot be guaranteed that they will continue to work going forward.
+- ``STRING_CONVERSION_ACTION`` config option is deprecated as it is no longer used in the Ansible Core code base.
+- the 'smart' option for setting a connection plugin is being removed as it's main purpose (choosing between ssh and paramiko) is now irrelevant.
+- vault and unfault filters - the undocumented ``vaultid`` parameter is deprecated and will be removed in ansible-core 2.20. Use ``vault_id`` instead.
+- yum_repository - deprecated parameter 'keepcache' (https://github.com/ansible/ansible/issues/78693).
+
+Removed Features (previously deprecated)
+----------------------------------------
+
+- ActionBase - remove deprecated ``_remote_checksum`` method
+- PlayIterator - remove deprecated ``cache_block_tasks`` and ``get_original_task`` methods
+- Remove deprecated ``FileLock`` class
+- Removed Python 3.9 as a supported version on the controller. Python 3.10 or newer is required.
+- Removed ``include`` which has been deprecated in Ansible 2.12. Use ``include_tasks`` or ``import_tasks`` instead.
+- ``Templar`` - remove deprecated ``shared_loader_obj`` parameter of ``__init__``
+- ``fetch_url`` - remove auto disabling ``decompress`` when gzip is not available
+- ``get_action_args_with_defaults`` - remove deprecated ``redirected_names`` method parameter
+- ansible-test - Removed support for the remote Windows targets 2012 and 2012-R2
+- inventory_cache - remove deprecated ``default.fact_caching_prefix`` ini configuration option, use ``defaults.fact_caching_prefix`` instead.
+- module_utils/basic.py - Removed Python 3.5 as a supported remote version. Python 2.7 or Python 3.6+ is now required.
+- stat - removed unused `get_md5` parameter.
+
+Security Fixes
+--------------
+
+- ansible-galaxy - Prevent roles from using symlinks to overwrite files outside of the installation directory (CVE-2023-5115)
+
+Bugfixes
+--------
+
+- Allow for searching handler subdir for included task via include_role (https://github.com/ansible/ansible/issues/81722)
+- AnsibleModule.run_command - Only use selectors when needed, and rely on Python stdlib subprocess for the simple task of collecting stdout/stderr when prompt matching is not required.
+- Cache host_group_vars after instantiating it once and limit the amount of repetitive work it needs to do every time it runs.
+- Call PluginLoader.all() once for vars plugins, and load vars plugins that run automatically or are enabled specifically by name subsequently.
+- Display - Defensively configure writing to stdout and stderr with a custom encoding error handler that will replace invalid characters while providing a deprecation warning that non-utf8 text will result in an error in a future version.
+- Exclude internal options from man pages and docs.
+- Fix ``ansible-config init`` man page option indentation.
+- Fix ``ast`` deprecation warnings for ``Str`` and ``value.s`` when using Python 3.12.
+- Fix ``run_once`` being incorrectly interpreted on handlers (https://github.com/ansible/ansible/issues/81666)
+- Fix exceptions caused by various inputs when performing arg splitting or parsing key/value pairs. Resolves issue https://github.com/ansible/ansible/issues/46379 and issue https://github.com/ansible/ansible/issues/61497
+- Fix incorrect parsing of multi-line Jinja2 blocks when performing arg splitting or parsing key/value pairs.
+- Fix post-validating looped task fields so the strategy uses the correct values after task execution.
+- Fixed `pip` module failure in case of usage quotes for `virtualenv_command` option for the venv command. (https://github.com/ansible/ansible/issues/76372)
+- From issue https://github.com/ansible/ansible/issues/80880, when notifying a handler from another handler, handler notifications must be registered immediately as the flush_handler call is not recursive.
+- Import ``FILE_ATTRIBUTES`` from ``ansible.module_utils.common.file`` in ``ansible.module_utils.basic`` instead of defining it twice.
+- Inventory scripts parser not treat exception when getting hostsvar (https://github.com/ansible/ansible/issues/81103)
+- On Python 3 use datetime methods ``fromtimestamp`` and ``now`` with UTC timezone instead of ``utcfromtimestamp`` and ``utcnow``, which are deprecated in Python 3.12.
+- PluginLoader - fix Jinja plugin performance issues (https://github.com/ansible/ansible/issues/79652)
+- PowerShell - Remove some code which is no longer valid for dotnet 5+
+- Prevent running same handler multiple times when included via ``include_role`` (https://github.com/ansible/ansible/issues/73643)
+- Prompting - add a short sleep between polling for user input to reduce CPU consumption (https://github.com/ansible/ansible/issues/81516).
+- Properly disable ``jinja2_native`` in the template module when jinja2 override is used in the template (https://github.com/ansible/ansible/issues/80605)
+- Properly template tags in parent blocks (https://github.com/ansible/ansible/issues/81053)
+- Remove unreachable parser error for removed ``static`` parameter of ``include_role``
+- Replace uses of ``configparser.ConfigParser.readfp()`` which was removed in Python 3.12 with ``configparser.ConfigParser.read_file()`` (https://github.com/ansible/ansible/issues/81656)
+- Set filters ``intersect``, ``difference``, ``symmetric_difference`` and ``union`` now always return a ``list``, never a ``set``. Previously, a ``set`` would be returned if the inputs were a hashable type such as ``str``, instead of a collection, such as a ``list`` or ``tuple``.
+- Set filters ``intersect``, ``difference``, ``symmetric_difference`` and ``union`` now use set operations when the given items are hashable. Previously, list operations were performed unless the inputs were a hashable type such as ``str``, instead of a collection, such as a ``list`` or ``tuple``.
+- Switch result queue from a ``multiprocessing.queues.Queue` to ``multiprocessing.queues.SimpleQueue``, primarily to allow properly handling pickling errors, to prevent an infinite hang waiting for task results
+- The ``ansible-config init`` command now has a documentation description.
+- The ``ansible-galaxy collection download`` command now has a documentation description.
+- The ``ansible-galaxy collection install`` command documentation is now visible (previously hidden by a decorator).
+- The ``ansible-galaxy collection verify`` command now has a documentation description.
+- The ``ansible-galaxy role install`` command documentation is now visible (previously hidden by a decorator).
+- The ``ansible-inventory`` command command now has a documentation description (previously used as the epilog).
+- The ``hostname`` module now also updates both current and permanent hostname on OpenBSD. Before it only updated the permanent hostname (https://github.com/ansible/ansible/issues/80520).
+- Update module_utils.urls unit test to work with cryptography >= 41.0.0.
+- When generating man pages, use ``func`` to find the command function instead of looking it up by the command name.
+- ``StrategyBase._process_pending_results`` - create a ``Templar`` on demand for templating ``changed_when``/``failed_when``.
+- ``ansible-galaxy`` now considers all collection paths when identifying which collection requirements are already installed. Use the ``COLLECTIONS_PATHS`` and ``COLLECTIONS_SCAN_SYS_PATHS`` config options to modify these. Previously only the install path was considered when resolving the candidates. The install path will remain the only one potentially modified. (https://github.com/ansible/ansible/issues/79767, https://github.com/ansible/ansible/issues/81163)
+- ``ansible.module_utils.service`` - ensure binary data transmission in ``daemonize()``
+- ``ansible.module_utils.service`` - fix inter-process communication in ``daemonize()``
+- ``import_role`` reverts to previous behavior of exporting vars at compile time.
+- ``pkg_mgr`` - fix the default dnf version detection
+- ansiballz - Prevent issue where the time on the control host could change part way through building the ansiballz file, potentially causing a pre-1980 date to be used during ansiballz unpacking leading to a zip file error (https://github.com/ansible/ansible/issues/80089)
+- ansible terminal color settings were incorrectly limited to 16 options via 'choices', removing so all 256 can be accessed.
+- ansible-console - fix filtering by collection names when a collection search path was set (https://github.com/ansible/ansible/pull/81450).
+- ansible-galaxy - Enabled the ``data`` tarfile filter during role installation for Python versions that support it. A probing mechanism is used to avoid Python versions with a broken implementation.
+- ansible-galaxy - Fix issue installing collections containing directories with more than 100 characters on python versions before 3.10.6
+- ansible-galaxy - Fix variable type error when installing subdir collections (https://github.com/ansible/ansible/issues/80943)
+- ansible-galaxy - Provide a better error message when using a requirements file with an invalid format - https://github.com/ansible/ansible/issues/81901
+- ansible-galaxy - fix installing collections from directories that have a trailing path separator (https://github.com/ansible/ansible/issues/77803).
+- ansible-galaxy - fix installing signed collections (https://github.com/ansible/ansible/issues/80648).
+- ansible-galaxy - reduce API calls to servers by fetching signatures only for final candidates.
+- ansible-galaxy - started allowing the use of pre-releases for collections that do not have any stable versions published. (https://github.com/ansible/ansible/pull/81606)
+- ansible-galaxy - started allowing the use of pre-releases for dependencies on any level of the dependency tree that specifically demand exact pre-release versions of collections and not version ranges. (https://github.com/ansible/ansible/pull/81606)
+- ansible-galaxy collection verify - fix verifying signed collections when the keyring is not configured.
+- ansible-galaxy info - fix reporting no role found when lookup_role_by_name returns None.
+- ansible-inventory - index available_hosts for major performance boost when dumping large inventories
+- ansible-test - Add a ``pylint`` plugin to work around a known issue on Python 3.12.
+- ansible-test - Add support for ``argcomplete`` version 3.
+- ansible-test - All containers created by ansible-test now include the current test session ID in their name. This avoids conflicts between concurrent ansible-test invocations using the same container host.
+- ansible-test - Always use ansible-test managed entry points for ansible-core CLI tools when not running from source. This fixes issues where CLI entry points created during install are not compatible with ansible-test.
+- ansible-test - Fix a traceback that occurs when attempting to test Ansible source using a different ansible-test. A clear error message is now given when this scenario occurs.
+- ansible-test - Fix handling of timeouts exceeding one day.
+- ansible-test - Fix parsing of cgroup entries which contain a ``:`` in the path (https://github.com/ansible/ansible/issues/81977).
+- ansible-test - Fix several possible tracebacks when using the ``-e`` option with sanity tests.
+- ansible-test - Fix various cases where the test timeout could expire without terminating the tests.
+- ansible-test - Include missing ``pylint`` requirements for Python 3.10.
+- ansible-test - Pre-build a PyYAML wheel before installing requirements to avoid a potential Cython build failure.
+- ansible-test - Remove redundant warning about missing programs before attempting to execute them.
+- ansible-test - The ``import`` sanity test now checks the collection loader for remote-only Python support when testing ansible-core.
+- ansible-test - Unit tests now report warnings generated during test runs. Previously only warnings generated during test collection were reported.
+- ansible-test - Update ``pylint`` to 2.17.2 to resolve several possible false positives.
+- ansible-test - Update ``pylint`` to 2.17.3 to resolve several possible false positives.
+- ansible-test - Update ``pylint`` to version 3.0.1.
+- ansible-test - Use ``raise ... from ...`` when raising exceptions from within an exception handler.
+- ansible-test - When bootstrapping remote FreeBSD instances, use the OS packaged ``setuptools`` instead of installing the latest version from PyPI.
+- ansible-test local change detection - use ``git merge-base <branch> HEAD`` instead of ``git merge-base --fork-point <branch>`` (https://github.com/ansible/ansible/pull/79734).
+- ansible-vault - fail when the destination file location is not writable before performing encryption (https://github.com/ansible/ansible/issues/81455).
+- apt - ignore fail_on_autoremove and allow_downgrade parameters when using aptitude (https://github.com/ansible/ansible/issues/77868).
+- blockinfile - avoid crash with Python 3 if creating the directory fails when ``create=true`` (https://github.com/ansible/ansible/pull/81662).
+- connection timeouts defined in ansible.cfg will now be properly used, the --timeout cli option was obscuring them by always being set.
+- copy - print correct destination filename when using `content` and `--diff` (https://github.com/ansible/ansible/issues/79749).
+- copy unit tests - Fixing "dir all perms" documentation and formatting for easier reading.
+- core will now also look at the connection plugin to force 'local' interpreter for networking path compatibility as just ansible_network_os could be misleading.
+- deb822_repository - use http-agent for receiving content (https://github.com/ansible/ansible/issues/80809).
+- debconf - idempotency in questions with type 'password' (https://github.com/ansible/ansible/issues/47676).
+- distribution facts - fix Source Mage family mapping
+- dnf - fix a failure when a package from URI was specified and ``update_only`` was set (https://github.com/ansible/ansible/issues/81376).
+- dnf5 - Update dnf5 module to handle API change for setting the download directory (https://github.com/ansible/ansible/issues/80887)
+- dnf5 - Use ``transaction.check_gpg_signatures`` API call to check package signatures AND possibly to recover from when keys are missing.
+- dnf5 - fix module and package names in the message following failed module respawn attempt
+- dnf5 - use the logs API to determine transaction problems
+- dpkg_selections - check if the package exists before performing the selection operation (https://github.com/ansible/ansible/issues/81404).
+- encrypt - deprecate passlib_or_crypt API (https://github.com/ansible/ansible/issues/55839).
+- fetch - Handle unreachable errors properly (https://github.com/ansible/ansible/issues/27816)
+- file modules - Make symbolic modes with X use the computed permission, not original file (https://github.com/ansible/ansible/issues/80128)
+- file modules - fix validating invalid symbolic modes.
+- first found lookup has been updated to use the normalized argument parsing (pythonic) matching the documented examples.
+- first found lookup, fixed an issue with subsequent items clobbering information from previous ones.
+- first_found lookup now gets 'untemplated' loop entries and handles templating itself as task_executor was removing even 'templatable' entries and breaking functionality. https://github.com/ansible/ansible/issues/70772
+- galaxy - check if the target for symlink exists (https://github.com/ansible/ansible/pull/81586).
+- galaxy - cross check the collection type and collection source (https://github.com/ansible/ansible/issues/79463).
+- gather_facts parallel option was doing the reverse of what was stated, now it does run modules in parallel when True and serially when False.
+- handlers - fix ``v2_playbook_on_notify`` callback not being called when notifying handlers
+- handlers - the ``listen`` keyword can affect only one handler with the same name, the last one defined as it is a case with the ``notify`` keyword (https://github.com/ansible/ansible/issues/81013)
+- include_role - expose variables from parent roles to role's handlers (https://github.com/ansible/ansible/issues/80459)
+- inventory_ini - handle SyntaxWarning while parsing ini file in inventory (https://github.com/ansible/ansible/issues/81457).
+- iptables - remove default rule creation when creating iptables chain to be more similar to the command line utility (https://github.com/ansible/ansible/issues/80256).
+- lib/ansible/utils/encrypt.py - remove unused private ``_LOCK`` (https://github.com/ansible/ansible/issues/81613)
+- lookup/url.py - Fix incorrect var/env/ini entry for `force_basic_auth`
+- man page build - Remove the dependency on the ``docs`` directory for building man pages.
+- man page build - Sub commands of ``ansible-galaxy role`` and ``ansible-galaxy collection`` are now documented.
+- module responses - Ensure that module responses are utf-8 adhereing to JSON RFC and expectations of the core code.
+- module/role argument spec - validate the type for options that are None when the option is required or has a non-None default (https://github.com/ansible/ansible/issues/79656).
+- modules/user.py - Add check for valid directory when creating new user homedir (allows /dev/null as skeleton) (https://github.com/ansible/ansible/issues/75063)
+- paramiko_ssh, psrp, and ssh connection plugins - ensure that all values for options that should be strings are actually converted to strings (https://github.com/ansible/ansible/pull/81029).
+- password_hash - fix salt format for ``crypt``  (only used if ``passlib`` is not installed) for the ``bcrypt`` algorithm.
+- pep517 build backend - Copy symlinks when copying the source tree. This avoids tracebacks in various scenarios, such as when a venv is present in the source tree.
+- pep517 build backend - Use the documented ``import_module`` import from ``importlib``.
+- pip module - Update module to prefer use of the python ``packaging`` and ``importlib.metadata`` modules due to ``pkg_resources`` being deprecated (https://github.com/ansible/ansible/issues/80488)
+- pkg_mgr.py - Fix `ansible_pkg_mgr` incorrect in TencentOS Server Linux
+- pkg_mgr.py - Fix `ansible_pkg_mgr` is unknown in Kylin Linux (https://github.com/ansible/ansible/issues/81332)
+- powershell modules - Only set an rc of 1 if the PowerShell pipeline signaled an error occurred AND there are error records present. Previously it would do so only if the error signal was present without checking the error count.
+- replace - handle exception when bad escape character is provided in replace (https://github.com/ansible/ansible/issues/79364).
+- role deduplication - don't deduplicate before a role has had a task run for that particular host (https://github.com/ansible/ansible/issues/81486).
+- service module, does not permanently configure flags flags on Openbsd when enabling/disabling a service.
+- service module, enable/disable is not a exclusive action in checkmode anymore.
+- setup gather_timeout - Fix timeout in get_mounts_facts for linux.
+- setup module (fact gathering) will now try to be smarter about different versions of facter emitting error when --puppet flag is used w/o puppet.
+- syntax check - Limit ``--syntax-check`` to ``ansible-playbook`` only, as that is the only CLI affected by this argument (https://github.com/ansible/ansible/issues/80506)
+- tarfile - handle data filter deprecation warning message for extract and extractall (https://github.com/ansible/ansible/issues/80832).
+- template - Fix for formatting issues when a template path contains valid jinja/strftime pattern (especially line break one) and using the template path in ansible_managed (https://github.com/ansible/ansible/pull/79129)
+- templating - In the template action and lookup, use local jinja2 environment overlay overrides instead of mutating the templars environment
+- templating - prevent setting arbitrary attributes on Jinja2 environments via Jinja2 overrides in templates
+- templating escape and single var optimization now use correct delimiters when custom ones are provided either via task or template header.
+- unarchive - fix unarchiving sources that are copied to the remote node using a relative temporory directory path (https://github.com/ansible/ansible/issues/80710).
+- uri - fix search for JSON type to include complex strings containing '+'
+- uri/urls - Add compat function to handle the ability to parse the filename from a Content-Disposition header (https://github.com/ansible/ansible/issues/81806)
+- urls.py - fixed cert_file and key_file parameters when running on Python 3.12 - https://github.com/ansible/ansible/issues/80490
+- user - set expiration value correctly when unable to retrieve the current value from the system (https://github.com/ansible/ansible/issues/71916)
+- validate-modules sanity test - replace semantic markup parsing and validating code with the code from `antsibull-docs-parser 0.2.0 <https://github.com/ansible-community/antsibull-docs-parser/releases/tag/0.2.0>`__ (https://github.com/ansible/ansible/pull/80406).
+- vars_prompt - internally convert the ``unsafe`` value to ``bool``
+- vault and unvault filters now properly take ``vault_id`` parameter.
+- win_fetch - Add support for using file with wildcards in file name. (https://github.com/ansible/ansible/issues/73128)
+- winrm - Better handle send input failures when communicating with hosts under load
+
+Known Issues
+------------
+
+- ansible-galaxy - dies in the middle of installing a role when that role contains Java inner classes (files with $ in the file name).  This is by design, to exclude temporary or backup files. (https://github.com/ansible/ansible/pull/81553).
+- ansible-test - The ``pep8`` sanity test is unable to detect f-string spacing issues (E201, E202) on Python 3.10 and 3.11. They are correctly detected under Python 3.12. See (https://github.com/PyCQA/pycodestyle/issues/1190).
--- a/changelogs/changelog.yaml
+++ b/changelogs/changelog.yaml
@ -1,2 +1,902 @@
 ancestor: 2.15.0
-releases: {}
+releases:
+  2.16.0:
+    changes:
+      bugfixes:
+      - ansible-test - Fix parsing of cgroup entries which contain a ``:`` in the
+        path (https://github.com/ansible/ansible/issues/81977).
+      release_summary: '| Release Date: 2023-11-06
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+        '
+    codename: All My Love
+    fragments:
+    - 2.16.0_summary.yaml
+    - ansible-test-cgroup-split.yml
+    release_date: '2023-11-06'
+  2.16.0b1:
+    changes:
+      breaking_changes:
+      - Any plugin using the config system and the `cli` entry to use the `timeout`
+        from the command line, will see the value change if the use had configured
+        it in any of the lower precedence methods. If relying on this behaviour to
+        consume the global/generic timeout from the DEFAULT_TIMEOUT constant, please
+        consult the documentation on plugin configuration to add the overlaping entries.
+      - ansible-test - Test plugins that rely on containers no longer support reusing
+        running containers. The previous behavior was an undocumented, untested feature.
+      - service module will not permanently configure variables/flags for openbsd
+        when doing enable/disable operation anymore, this module was never meant to
+        do this type of work, just to manage the service state itself. A rcctl_config
+        or similar module should be created and used instead.
+      bugfixes:
+      - Allow for searching handler subdir for included task via include_role (https://github.com/ansible/ansible/issues/81722)
+      - AnsibleModule.run_command - Only use selectors when needed, and rely on Python
+        stdlib subprocess for the simple task of collecting stdout/stderr when prompt
+        matching is not required.
+      - Display - Defensively configure writing to stdout and stderr with a custom
+        encoding error handler that will replace invalid characters while providing
+        a deprecation warning that non-utf8 text will result in an error in a future
+        version.
+      - Exclude internal options from man pages and docs.
+      - Fix ``ansible-config init`` man page option indentation.
+      - Fix ``ast`` deprecation warnings for ``Str`` and ``value.s`` when using Python
+        3.12.
+      - Fix exceptions caused by various inputs when performing arg splitting or parsing
+        key/value pairs. Resolves issue https://github.com/ansible/ansible/issues/46379
+        and issue https://github.com/ansible/ansible/issues/61497
+      - Fix incorrect parsing of multi-line Jinja2 blocks when performing arg splitting
+        or parsing key/value pairs.
+      - Fix post-validating looped task fields so the strategy uses the correct values
+        after task execution.
+      - Fixed `pip` module failure in case of usage quotes for `virtualenv_command`
+        option for the venv command. (https://github.com/ansible/ansible/issues/76372)
+      - From issue https://github.com/ansible/ansible/issues/80880, when notifying
+        a handler from another handler, handler notifications must be registered immediately
+        as the flush_handler call is not recursive.
+      - Import ``FILE_ATTRIBUTES`` from ``ansible.module_utils.common.file`` in ``ansible.module_utils.basic``
+        instead of defining it twice.
+      - Inventory scripts parser not treat exception when getting hostsvar (https://github.com/ansible/ansible/issues/81103)
+      - On Python 3 use datetime methods ``fromtimestamp`` and ``now`` with UTC timezone
+        instead of ``utcfromtimestamp`` and ``utcnow``, which are deprecated in Python
+        3.12.
+      - PluginLoader - fix Jinja plugin performance issues (https://github.com/ansible/ansible/issues/79652)
+      - PowerShell - Remove some code which is no longer valid for dotnet 5+
+      - Prevent running same handler multiple times when included via ``include_role``
+        (https://github.com/ansible/ansible/issues/73643)
+      - Prompting - add a short sleep between polling for user input to reduce CPU
+        consumption (https://github.com/ansible/ansible/issues/81516).
+      - Properly disable ``jinja2_native`` in the template module when jinja2 override
+        is used in the template (https://github.com/ansible/ansible/issues/80605)
+      - Remove unreachable parser error for removed ``static`` parameter of ``include_role``
+      - Replace uses of ``configparser.ConfigParser.readfp()`` which was removed in
+        Python 3.12 with ``configparser.ConfigParser.read_file()`` (https://github.com/ansible/ansible/issues/81656)
+      - Set filters ``intersect``, ``difference``, ``symmetric_difference`` and ``union``
+        now always return a ``list``, never a ``set``. Previously, a ``set`` would
+        be returned if the inputs were a hashable type such as ``str``, instead of
+        a collection, such as a ``list`` or ``tuple``.
+      - Set filters ``intersect``, ``difference``, ``symmetric_difference`` and ``union``
+        now use set operations when the given items are hashable. Previously, list
+        operations were performed unless the inputs were a hashable type such as ``str``,
+        instead of a collection, such as a ``list`` or ``tuple``.
+      - Switch result queue from a ``multiprocessing.queues.Queue` to ``multiprocessing.queues.SimpleQueue``,
+        primarily to allow properly handling pickling errors, to prevent an infinite
+        hang waiting for task results
+      - The ``ansible-config init`` command now has a documentation description.
+      - The ``ansible-galaxy collection download`` command now has a documentation
+        description.
+      - The ``ansible-galaxy collection install`` command documentation is now visible
+        (previously hidden by a decorator).
+      - The ``ansible-galaxy collection verify`` command now has a documentation description.
+      - The ``ansible-galaxy role install`` command documentation is now visible (previously
+        hidden by a decorator).
+      - The ``ansible-inventory`` command command now has a documentation description
+        (previously used as the epilog).
+      - The ``hostname`` module now also updates both current and permanent hostname
+        on OpenBSD. Before it only updated the permanent hostname (https://github.com/ansible/ansible/issues/80520).
+      - Update module_utils.urls unit test to work with cryptography >= 41.0.0.
+      - When generating man pages, use ``func`` to find the command function instead
+        of looking it up by the command name.
+      - '``StrategyBase._process_pending_results`` - create a ``Templar`` on demand
+        for templating ``changed_when``/``failed_when``.'
+      - '``ansible-galaxy`` now considers all collection paths when identifying which
+        collection requirements are already installed. Use the ``COLLECTIONS_PATHS``
+        and ``COLLECTIONS_SCAN_SYS_PATHS`` config options to modify these. Previously
+        only the install path was considered when resolving the candidates. The install
+        path will remain the only one potentially modified. (https://github.com/ansible/ansible/issues/79767,
+        https://github.com/ansible/ansible/issues/81163)'
+      - '``ansible.module_utils.service`` - ensure binary data transmission in ``daemonize()``'
+      - '``ansible.module_utils.service`` - fix inter-process communication in ``daemonize()``'
+      - '``pkg_mgr`` - fix the default dnf version detection'
+      - ansiballz - Prevent issue where the time on the control host could change
+        part way through building the ansiballz file, potentially causing a pre-1980
+        date to be used during ansiballz unpacking leading to a zip file error (https://github.com/ansible/ansible/issues/80089)
+      - ansible terminal color settings were incorrectly limited to 16 options via
+        'choices', removing so all 256 can be accessed.
+      - ansible-console - fix filtering by collection names when a collection search
+        path was set (https://github.com/ansible/ansible/pull/81450).
+      - ansible-galaxy - Enabled the ``data`` tarfile filter during role installation
+        for Python versions that support it. A probing mechanism is used to avoid
+        Python versions with a broken implementation.
+      - ansible-galaxy - Fix issue installing collections containing directories with
+        more than 100 characters on python versions before 3.10.6
+      - ansible-galaxy - Fix variable type error when installing subdir collections
+        (https://github.com/ansible/ansible/issues/80943)
+      - ansible-galaxy - fix installing collections from directories that have a trailing
+        path separator (https://github.com/ansible/ansible/issues/77803).
+      - ansible-galaxy - fix installing signed collections (https://github.com/ansible/ansible/issues/80648).
+      - ansible-galaxy - reduce API calls to servers by fetching signatures only for
+        final candidates.
+      - ansible-galaxy - started allowing the use of pre-releases for collections
+        that do not have any stable versions published. (https://github.com/ansible/ansible/pull/81606)
+      - ansible-galaxy - started allowing the use of pre-releases for dependencies
+        on any level of the dependency tree that specifically demand exact pre-release
+        versions of collections and not version ranges. (https://github.com/ansible/ansible/pull/81606)
+      - ansible-galaxy collection verify - fix verifying signed collections when the
+        keyring is not configured.
+      - ansible-test - Add support for ``argcomplete`` version 3.
+      - ansible-test - All containers created by ansible-test now include the current
+        test session ID in their name. This avoids conflicts between concurrent ansible-test
+        invocations using the same container host.
+      - ansible-test - Always use ansible-test managed entry points for ansible-core
+        CLI tools when not running from source. This fixes issues where CLI entry
+        points created during install are not compatible with ansible-test.
+      - ansible-test - Fix a traceback that occurs when attempting to test Ansible
+        source using a different ansible-test. A clear error message is now given
+        when this scenario occurs.
+      - ansible-test - Fix handling of timeouts exceeding one day.
+      - ansible-test - Fix several possible tracebacks when using the ``-e`` option
+        with sanity tests.
+      - ansible-test - Fix various cases where the test timeout could expire without
+        terminating the tests.
+      - ansible-test - Pre-build a PyYAML wheel before installing requirements to
+        avoid a potential Cython build failure.
+      - ansible-test - Remove redundant warning about missing programs before attempting
+        to execute them.
+      - ansible-test - The ``import`` sanity test now checks the collection loader
+        for remote-only Python support when testing ansible-core.
+      - ansible-test - Unit tests now report warnings generated during test runs.
+        Previously only warnings generated during test collection were reported.
+      - ansible-test - Update ``pylint`` to 2.17.2 to resolve several possible false
+        positives.
+      - ansible-test - Update ``pylint`` to 2.17.3 to resolve several possible false
+        positives.
+      - ansible-test - Use ``raise ... from ...`` when raising exceptions from within
+        an exception handler.
+      - ansible-test - When bootstrapping remote FreeBSD instances, use the OS packaged
+        ``setuptools`` instead of installing the latest version from PyPI.
+      - ansible-test local change detection - use ``git merge-base <branch> HEAD``
+        instead of ``git merge-base --fork-point <branch>`` (https://github.com/ansible/ansible/pull/79734).
+      - ansible-vault - fail when the destination file location is not writable before
+        performing encryption (https://github.com/ansible/ansible/issues/81455).
+      - apt - ignore fail_on_autoremove and allow_downgrade parameters when using
+        aptitude (https://github.com/ansible/ansible/issues/77868).
+      - blockinfile - avoid crash with Python 3 if creating the directory fails when
+        ``create=true`` (https://github.com/ansible/ansible/pull/81662).
+      - connection timeouts defined in ansible.cfg will now be properly used, the
+        --timeout cli option was obscuring them by always being set.
+      - copy - print correct destination filename when using `content` and `--diff`
+        (https://github.com/ansible/ansible/issues/79749).
+      - copy unit tests - Fixing "dir all perms" documentation and formatting for
+        easier reading.
+      - core will now also look at the connection plugin to force 'local' interpreter
+        for networking path compatibility as just ansible_network_os could be misleading.
+      - deb822_repository - use http-agent for receiving content (https://github.com/ansible/ansible/issues/80809).
+      - debconf - idempotency in questions with type 'password' (https://github.com/ansible/ansible/issues/47676).
+      - distribution facts - fix Source Mage family mapping
+      - dnf - fix a failure when a package from URI was specified and ``update_only``
+        was set (https://github.com/ansible/ansible/issues/81376).
+      - dnf5 - Update dnf5 module to handle API change for setting the download directory
+        (https://github.com/ansible/ansible/issues/80887)
+      - dnf5 - Use ``transaction.check_gpg_signatures`` API call to check package
+        signatures AND possibly to recover from when keys are missing.
+      - dnf5 - fix module and package names in the message following failed module
+        respawn attempt
+      - dnf5 - use the logs API to determine transaction problems
+      - dpkg_selections - check if the package exists before performing the selection
+        operation (https://github.com/ansible/ansible/issues/81404).
+      - encrypt - deprecate passlib_or_crypt API (https://github.com/ansible/ansible/issues/55839).
+      - fetch - Handle unreachable errors properly (https://github.com/ansible/ansible/issues/27816)
+      - file modules - Make symbolic modes with X use the computed permission, not
+        original file (https://github.com/ansible/ansible/issues/80128)
+      - file modules - fix validating invalid symbolic modes.
+      - first found lookup has been updated to use the normalized argument parsing
+        (pythonic) matching the documented examples.
+      - first found lookup, fixed an issue with subsequent items clobbering information
+        from previous ones.
+      - first_found lookup now gets 'untemplated' loop entries and handles templating
+        itself as task_executor was removing even 'templatable' entries and breaking
+        functionality. https://github.com/ansible/ansible/issues/70772
+      - galaxy - check if the target for symlink exists (https://github.com/ansible/ansible/pull/81586).
+      - galaxy - cross check the collection type and collection source (https://github.com/ansible/ansible/issues/79463).
+      - gather_facts parallel option was doing the reverse of what was stated, now
+        it does run modules in parallel when True and serially when False.
+      - handlers - fix ``v2_playbook_on_notify`` callback not being called when notifying
+        handlers
+      - handlers - the ``listen`` keyword can affect only one handler with the same
+        name, the last one defined as it is a case with the ``notify`` keyword (https://github.com/ansible/ansible/issues/81013)
+      - include_role - expose variables from parent roles to role's handlers (https://github.com/ansible/ansible/issues/80459)
+      - inventory_ini - handle SyntaxWarning while parsing ini file in inventory (https://github.com/ansible/ansible/issues/81457).
+      - iptables - remove default rule creation when creating iptables chain to be
+        more similar to the command line utility (https://github.com/ansible/ansible/issues/80256).
+      - lib/ansible/utils/encrypt.py - remove unused private ``_LOCK`` (https://github.com/ansible/ansible/issues/81613)
+      - lookup/url.py - Fix incorrect var/env/ini entry for `force_basic_auth`
+      - man page build - Remove the dependency on the ``docs`` directory for building
+        man pages.
+      - man page build - Sub commands of ``ansible-galaxy role`` and ``ansible-galaxy
+        collection`` are now documented.
+      - module responses - Ensure that module responses are utf-8 adhereing to JSON
+        RFC and expectations of the core code.
+      - module/role argument spec - validate the type for options that are None when
+        the option is required or has a non-None default (https://github.com/ansible/ansible/issues/79656).
+      - modules/user.py - Add check for valid directory when creating new user homedir
+        (allows /dev/null as skeleton) (https://github.com/ansible/ansible/issues/75063)
+      - paramiko_ssh, psrp, and ssh connection plugins - ensure that all values for
+        options that should be strings are actually converted to strings (https://github.com/ansible/ansible/pull/81029).
+      - password_hash - fix salt format for ``crypt``  (only used if ``passlib`` is
+        not installed) for the ``bcrypt`` algorithm.
+      - pep517 build backend - Copy symlinks when copying the source tree. This avoids
+        tracebacks in various scenarios, such as when a venv is present in the source
+        tree.
+      - pep517 build backend - Use the documented ``import_module`` import from ``importlib``.
+      - pip module - Update module to prefer use of the python ``packaging`` and ``importlib.metadata``
+        modules due to ``pkg_resources`` being deprecated (https://github.com/ansible/ansible/issues/80488)
+      - pkg_mgr.py - Fix `ansible_pkg_mgr` incorrect in TencentOS Server Linux
+      - pkg_mgr.py - Fix `ansible_pkg_mgr` is unknown in Kylin Linux (https://github.com/ansible/ansible/issues/81332)
+      - powershell modules - Only set an rc of 1 if the PowerShell pipeline signaled
+        an error occurred AND there are error records present. Previously it would
+        do so only if the error signal was present without checking the error count.
+      - replace - handle exception when bad escape character is provided in replace
+        (https://github.com/ansible/ansible/issues/79364).
+      - role deduplication - don't deduplicate before a role has had a task run for
+        that particular host (https://github.com/ansible/ansible/issues/81486).
+      - service module, does not permanently configure flags flags on Openbsd when
+        enabling/disabling a service.
+      - service module, enable/disable is not a exclusive action in checkmode anymore.
+      - setup gather_timeout - Fix timeout in get_mounts_facts for linux.
+      - setup module (fact gathering) will now try to be smarter about different versions
+        of facter emitting error when --puppet flag is used w/o puppet.
+      - syntax check - Limit ``--syntax-check`` to ``ansible-playbook`` only, as that
+        is the only CLI affected by this argument (https://github.com/ansible/ansible/issues/80506)
+      - tarfile - handle data filter deprecation warning message for extract and extractall
+        (https://github.com/ansible/ansible/issues/80832).
+      - template - Fix for formatting issues when a template path contains valid jinja/strftime
+        pattern (especially line break one) and using the template path in ansible_managed
+        (https://github.com/ansible/ansible/pull/79129)
+      - templating - In the template action and lookup, use local jinja2 environment
+        overlay overrides instead of mutating the templars environment
+      - templating - prevent setting arbitrary attributes on Jinja2 environments via
+        Jinja2 overrides in templates
+      - templating escape and single var optimization now use correct delimiters when
+        custom ones are provided either via task or template header.
+      - unarchive - fix unarchiving sources that are copied to the remote node using
+        a relative temporory directory path (https://github.com/ansible/ansible/issues/80710).
+      - uri - fix search for JSON type to include complex strings containing '+'
+      - urls.py - fixed cert_file and key_file parameters when running on Python 3.12
+        - https://github.com/ansible/ansible/issues/80490
+      - user - set expiration value correctly when unable to retrieve the current
+        value from the system (https://github.com/ansible/ansible/issues/71916)
+      - validate-modules sanity test - replace semantic markup parsing and validating
+        code with the code from `antsibull-docs-parser 0.2.0 <https://github.com/ansible-community/antsibull-docs-parser/releases/tag/0.2.0>`__
+        (https://github.com/ansible/ansible/pull/80406).
+      - vars_prompt - internally convert the ``unsafe`` value to ``bool``
+      - vault and unvault filters now properly take ``vault_id`` parameter.
+      - win_fetch - Add support for using file with wildcards in file name. (https://github.com/ansible/ansible/issues/73128)
+      deprecated_features:
+      - Deprecated ini config option ``collections_paths``, use the singular form
+        ``collections_path`` instead
+      - Deprecated the env var ``ANSIBLE_COLLECTIONS_PATHS``, use the singular form
+        ``ANSIBLE_COLLECTIONS_PATH`` instead
+      - Support for Windows Server 2012 and 2012 R2 has been removed as the support
+        end of life from Microsoft is October 10th 2023. These versions of Windows
+        will no longer be tested in this Ansible release and it cannot be guaranteed
+        that they will continue to work going forward.
+      - '``STRING_CONVERSION_ACTION`` config option is deprecated as it is no longer
+        used in the Ansible Core code base.'
+      - the 'smart' option for setting a connection plugin is being removed as it's
+        main purpose (choosing between ssh and paramiko) is now irrelevant.
+      - vault and unfault filters - the undocumented ``vaultid`` parameter is deprecated
+        and will be removed in ansible-core 2.20. Use ``vault_id`` instead.
+      - yum_repository - deprecated parameter 'keepcache' (https://github.com/ansible/ansible/issues/78693).
+      known_issues:
+      - ansible-galaxy - dies in the middle of installing a role when that role contains
+        Java inner classes (files with $ in the file name).  This is by design, to
+        exclude temporary or backup files. (https://github.com/ansible/ansible/pull/81553).
+      - ansible-test - The ``pep8`` sanity test is unable to detect f-string spacing
+        issues (E201, E202) on Python 3.10 and 3.11. They are correctly detected under
+        Python 3.12. See (https://github.com/PyCQA/pycodestyle/issues/1190).
+      minor_changes:
+      - Add Python type hints to the Display class (https://github.com/ansible/ansible/issues/80841)
+      - Add ``GALAXY_COLLECTIONS_PATH_WARNING`` option to disable the warning given
+        by ``ansible-galaxy collection install`` when installing a collection to a
+        path that isn't in the configured collection paths.
+      - Add ``python3.12`` to the default ``INTERPRETER_PYTHON_FALLBACK`` list.
+      - Add ``utcfromtimestamp`` and ``utcnow`` to ``ansible.module_utils.compat.datetime``
+        to return fixed offset datetime objects.
+      - Add a general ``GALAXY_SERVER_TIMEOUT`` config option for distribution servers
+        (https://github.com/ansible/ansible/issues/79833).
+      - Added Python type annotation to connection plugins
+      - CLI argument parsing - Automatically prepend to the help of CLI arguments
+        that support being specified multiple times. (https://github.com/ansible/ansible/issues/22396)
+      - DEFAULT_TRANSPORT now defaults to 'ssh', the old 'smart' option is being deprecated
+        as versions of OpenSSH without control persist are basically not present anymore.
+      - Documentation for set filters ``intersect``, ``difference``, ``symmetric_difference``
+        and ``union`` now states that the returned list items are in arbitrary order.
+      - Record ``removal_date`` in runtime metadata as a string instead of a date.
+      - Remove the ``CleansingNodeVisitor`` class and its usage due to the templating
+        changes that made it superfluous. Also simplify the ``Conditional`` class.
+      - Removed ``exclude`` and ``recursive-exclude`` commands for generated files
+        from the ``MANIFEST.in`` file. These excludes were unnecessary since releases
+        are expected to be built with a clean worktree.
+      - Removed ``exclude`` commands for sanity test files from the ``MANIFEST.in``
+        file. These tests were previously excluded because they did not pass when
+        run from an sdist. However, sanity tests are not expected to pass from an
+        sdist, so excluding some (but not all) of the failing tests makes little sense.
+      - Removed redundant ``include`` commands from the ``MANIFEST.in`` file. These
+        includes either duplicated default behavior or another command.
+      - The ``ansible-core`` sdist no longer contains pre-generated man pages. Instead,
+        a ``packaging/cli-doc/build.py`` script is included in the sdist. This script
+        can generate man pages and standalone RST documentation for ``ansible-core``
+        CLI programs.
+      - The ``docs`` and ``examples`` directories are no longer included in the ``ansible-core``
+        sdist. These directories have been moved to the https://github.com/ansible/ansible-documentation
+        repository.
+      - The minimum required ``setuptools`` version is now 66.1.0, as it is the oldest
+        version to support Python 3.12.
+      - Update ``ansible_service_mgr`` fact to include init system for SMGL OS family
+      - Use ``ansible.module_utils.common.text.converters`` instead of ``ansible.module_utils._text``.
+      - Use ``importlib.resources.abc.TraversableResources`` instead of deprecated
+        ``importlib.abc.TraversableResources`` where available (https:/github.com/ansible/ansible/pull/81082).
+      - Use ``include`` where ``recursive-include`` is unnecessary in the ``MANIFEST.in``
+        file.
+      - Use ``package_data`` instead of ``include_package_data`` for ``setup.cfg``
+        to avoid ``setuptools`` warnings.
+      - Utilize gpg check provided internally by the ``transaction.run`` method as
+        oppose to calling it manually.
+      - '``Templar`` - do not add the ``dict`` constructor to ``globals`` as all required
+        Jinja2 versions already do so'
+      - ansible-doc - allow to filter listing of collections and metadata dump by
+        more than one collection (https://github.com/ansible/ansible/pull/81450).
+      - ansible-galaxy - Add a plural option to improve ignoring multiple signature
+        error status codes when installing or verifying collections. A space-separated
+        list of error codes can follow --ignore-signature-status-codes in addition
+        to specifying --ignore-signature-status-code multiple times (for example,
+        ``--ignore-signature-status-codes NO_PUBKEY UNEXPECTED``).
+      - ansible-galaxy - Remove internal configuration argument ``v3`` (https://github.com/ansible/ansible/pull/80721)
+      - ansible-galaxy - add note to the collection dependency resolver error message
+        about pre-releases if ``--pre`` was not provided (https://github.com/ansible/ansible/issues/80048).
+      - ansible-galaxy - used to crash out with a "Errno 20 Not a directory" error
+        when extracting files from a role when hitting a file with an illegal name
+        (https://github.com/ansible/ansible/pull/81553). Now it gives a warning identifying
+        the culprit file and the rule violation (e.g., ``my$class.jar`` has a ``$``
+        in the name) before crashing out, giving the user a chance to remove the invalid
+        file and try again. (https://github.com/ansible/ansible/pull/81555).
+      - ansible-test - Add Alpine 3.18 to remotes
+      - ansible-test - Add Fedora 38 container.
+      - ansible-test - Add Fedora 38 remote.
+      - ansible-test - Add FreeBSD 13.2 remote.
+      - ansible-test - Add new pylint checker for new ``# deprecated:`` comments within
+        code to trigger errors when time to remove code that has no user facing deprecation
+        message. Only supported in ansible-core, not collections.
+      - ansible-test - Add support for RHEL 8.8 remotes.
+      - ansible-test - Add support for RHEL 9.2 remotes.
+      - ansible-test - Add support for testing with Python 3.12.
+      - ansible-test - Allow float values for the ``--timeout`` option to the ``env``
+        command. This simplifies testing.
+      - ansible-test - Enable ``thread`` code coverage in addition to the existing
+        ``multiprocessing`` coverage.
+      - ansible-test - RHEL 8.8 provisioning can now be used with the ``--python 3.11``
+        option.
+      - ansible-test - RHEL 9.2 provisioning can now be used with the ``--python 3.11``
+        option.
+      - ansible-test - Refactored ``env`` command logic and timeout handling.
+      - ansible-test - Remove Fedora 37 remote support.
+      - ansible-test - Remove Fedora 37 test container.
+      - ansible-test - Remove Python 3.8 and 3.9 from RHEL 8.8.
+      - ansible-test - Remove obsolete embedded script for configuring WinRM on Windows
+        remotes.
+      - ansible-test - Removed Ubuntu 20.04 LTS image from the `--remote` option.
+      - ansible-test - Removed `freebsd/12.4` remote.
+      - ansible-test - Removed `freebsd/13.1` remote.
+      - 'ansible-test - Removed test remotes: rhel/8.7, rhel/9.1'
+      - ansible-test - Removed the deprecated ``--docker-no-pull`` option.
+      - ansible-test - Removed the deprecated ``--no-pip-check`` option.
+      - ansible-test - Removed the deprecated ``foreman`` test plugin.
+      - ansible-test - Removed the deprecated ``govcsim`` support from the ``vcenter``
+        test plugin.
+      - ansible-test - Replace the ``pytest-forked`` pytest plugin with a custom plugin.
+      - ansible-test - The ``no-get-exception`` sanity test is now limited to plugins
+        in collections. Previously any Python file in a collection was checked for
+        ``get_exception`` usage.
+      - ansible-test - The ``replace-urlopen`` sanity test is now limited to plugins
+        in collections. Previously any Python file in a collection was checked for
+        ``urlopen`` usage.
+      - ansible-test - The ``use-compat-six`` sanity test is now limited to plugins
+        in collections. Previously any Python file in a collection was checked for
+        ``six`` usage.
+      - ansible-test - The openSUSE test container has been updated to openSUSE Leap
+        15.5.
+      - ansible-test - Update pip to ``23.1.2`` and setuptools to ``67.7.2``.
+      - ansible-test - Update the ``default`` containers.
+      - ansible-test - Update the ``nios-test-container`` to version 2.0.0, which
+        supports API version 2.9.
+      - ansible-test - Update the logic used to detect when ``ansible-test`` is running
+        from source.
+      - ansible-test - Updated the CloudStack test container to version 1.6.1.
+      - ansible-test - Updated the distro test containers to version 6.3.0 to include
+        coverage 7.3.2 for Python 3.8+. The alpine3 container is now based on 3.18
+        instead of 3.17 and includes Python 3.11 instead of Python 3.10.
+      - ansible-test - Use ``datetime.datetime.now`` with ``tz`` specified instead
+        of ``datetime.datetime.utcnow``.
+      - ansible-test - Use a context manager to perform cleanup at exit instead of
+        using the built-in ``atexit`` module.
+      - ansible-test - remove Alpine 3.17 from remotes
+      - "ansible-test \u2014 Python 3.8\u20133.12 will use ``coverage`` v7.3.2."
+      - "ansible-test \u2014 ``coverage`` v6.5.0 is to be used only under Python 3.7."
+      - 'ansible-vault create: Now raises an error when opening the editor without
+        tty. The flag --skip-tty-check restores previous behaviour.'
+      - ansible_user_module - tweaked macos user defaults to reflect expected defaults
+        (https://github.com/ansible/ansible/issues/44316)
+      - apt - return calculated diff while running apt clean operation.
+      - blockinfile - add append_newline and prepend_newline options (https://github.com/ansible/ansible/issues/80835).
+      - cli - Added short option '-J' for asking for vault password (https://github.com/ansible/ansible/issues/80523).
+      - command - Add option ``expand_argument_vars`` to disable argument expansion
+        and use literal values - https://github.com/ansible/ansible/issues/54162
+      - config lookup new option show_origin to also return the origin of a configuration
+        value.
+      - display methods for warning and deprecation are now proxied to main process
+        when issued from a fork. This allows for the deduplication of warnings and
+        deprecations to work globally.
+      - dnf5 - enable environment groups installation testing in CI as its support
+        was added.
+      - dnf5 - enable now implemented ``cacheonly`` functionality
+      - executor now skips persistent connection when it detects an action that does
+        not require a connection.
+      - find module - Add ability to filter based on modes
+      - gather_facts now will use gather_timeout setting to limit parallel execution
+        of modules that do not themselves use gather_timeout.
+      - group - remove extraneous warning shown when user does not exist (https://github.com/ansible/ansible/issues/77049).
+      - include_vars - os.walk now follows symbolic links when traversing directories
+        (https://github.com/ansible/ansible/pull/80460)
+      - module compression is now sourced directly via config, bypassing play_context
+        possibly stale values.
+      - reboot - show last error message in verbose logs (https://github.com/ansible/ansible/issues/81574).
+      - service_facts now returns more info for rcctl managed systesm (OpenBSD).
+      - tasks - the ``retries`` keyword can be specified without ``until`` in which
+        case the task is retried until it succeeds but at most ``retries`` times (https://github.com/ansible/ansible/issues/20802)
+      - user - add new option ``password_expire_warn`` (supported on Linux only) to
+        set the number of days of warning before a password change is required (https://github.com/ansible/ansible/issues/79882).
+      - yum_repository - Align module documentation with parameters
+      release_summary: '| Release Date: 2023-09-26
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+        '
+      removed_features:
+      - ActionBase - remove deprecated ``_remote_checksum`` method
+      - PlayIterator - remove deprecated ``cache_block_tasks`` and ``get_original_task``
+        methods
+      - Remove deprecated ``FileLock`` class
+      - Removed Python 3.9 as a supported version on the controller. Python 3.10 or
+        newer is required.
+      - Removed ``include`` which has been deprecated in Ansible 2.12. Use ``include_tasks``
+        or ``import_tasks`` instead.
+      - '``Templar`` - remove deprecated ``shared_loader_obj`` parameter of ``__init__``'
+      - '``fetch_url`` - remove auto disabling ``decompress`` when gzip is not available'
+      - '``get_action_args_with_defaults`` - remove deprecated ``redirected_names``
+        method parameter'
+      - ansible-test - Removed support for the remote Windows targets 2012 and 2012-R2
+      - inventory_cache - remove deprecated ``default.fact_caching_prefix`` ini configuration
+        option, use ``defaults.fact_caching_prefix`` instead.
+      - module_utils/basic.py - Removed Python 3.5 as a supported remote version.
+        Python 2.7 or Python 3.6+ is now required.
+      - stat - removed unused `get_md5` parameter.
+    codename: All My Love
+    fragments:
+    - 2.16.0b1_summary.yaml
+    - 20802-until-default.yml
+    - 22396-indicate-which-args-are-multi.yml
+    - 27816-fetch-unreachable.yml
+    - 50603-tty-check.yaml
+    - 71916-user-expires-int.yml
+    - 73643-handlers-prevent-multiple-runs.yml
+    - 74723-support-wildcard-win_fetch.yml
+    - 75063-allow-dev-nul-as-skeleton-for-new-homedir.yml
+    - 76372-fix-pip-virtualenv-command-parsing.yml
+    - 78487-galaxy-collections-path-warnings.yml
+    - 79129-ansible-managed-filename-format.yaml
+    - 79364_replace.yml
+    - 79677-fix-argspec-type-check.yml
+    - 79734-ansible-test-change-detection.yml
+    - 79844-fix-timeout-mounts-linux.yml
+    - 79999-ansible-user-tweak-macos-defaults.yaml
+    - 80089-prevent-module-build-date-issue.yml
+    - 80128-symbolic-modes-X-use-computed.yml
+    - 80257-iptables-chain-creation-does-not-populate-a-rule.yml
+    - 80258-defensive-display-non-utf8.yml
+    - 80334-reduce-ansible-galaxy-api-calls.yml
+    - 80406-validate-modules-semantic-markup.yml
+    - 80449-fix-symbolic-mode-error-msg.yml
+    - 80459-handlers-nested-includes-vars.yml
+    - 80460-add-symbolic-links-with-dir.yml
+    - 80476-fix-loop-task-post-validation.yml
+    - 80488-pip-pkg-resources.yml
+    - 80506-syntax-check-playbook-only.yml
+    - 80520-fix-current-hostname-openbsd.yml
+    - 80523_-_adding_short_option_for_--ask-vault-pass.yml
+    - 80605-template-overlay-native-jinja.yml
+    - 80648-fix-ansible-galaxy-cache-signatures-bug.yml
+    - 80721-ansible-galaxy.yml
+    - 80738-abs-unarachive-src.yml
+    - 80841-display-type-annotation.yml
+    - 80880-register-handlers-immediately-if-iterating-handlers.yml
+    - 80887-dnf5-api-change.yml
+    - 80943-ansible-galaxy-collection-subdir-install.yml
+    - 80968-replace-deprecated-ast-attr.yml
+    - 80985-fix-smgl-family-mapping.yml
+    - 81005-use-overlay-overrides.yml
+    - 81013-handlers-listen-last-defined-only.yml
+    - 81029-connection-types.yml
+    - 81064-daemonize-fixes.yml
+    - 81082-deprecated-importlib-abc.yml
+    - 81083-add-blockinfile-append-and-prepend-new-line-options.yml
+    - 81104-inventory-script-plugin-raise-execution-error.yml
+    - 81319-cloudstack-test-container-bump-version.yml
+    - 81332-fix-pkg-mgr-in-kylin.yml
+    - 81450-list-filters.yml
+    - 81494-remove-duplicated-file-attribute-constant.yml
+    - 81555-add-warning-for-illegal-filenames-in-roles.yaml
+    - 81584-daemonize-follow-up-fixes.yml
+    - 81606-ansible-galaxy-collection-pre-releases.yml
+    - 81613-remove-unusued-private-lock.yml
+    - 81656-cf_readfp-deprecated.yml
+    - 81662-blockinfile-exc.yml
+    - 81722-handler-subdir-include_tasks.yml
+    - CleansingNodeVisitor-removal.yml
+    - a-g-col-install-directory-with-trailing-sep.yml
+    - a-g-col-prevent-reinstalling-satisfied-req.yml
+    - a_test_rmv_alpine_317.yml
+    - add-missing-cli-docs.yml
+    - ag-ignore-multiple-signature-statuses.yml
+    - ansible-galaxy-server-timeout.yml
+    - ansible-runtime-metadata-removal-date.yml
+    - ansible-test-added-fedora-38.yml
+    - ansible-test-argcomplete-3.yml
+    - ansible-test-atexit.yml
+    - ansible-test-coverage-update.yml
+    - ansible-test-default-containers.yml
+    - ansible-test-deprecated-cleanup.yml
+    - ansible-test-distro-containers.yml
+    - ansible-test-entry-points.yml
+    - ansible-test-explain-traceback.yml
+    - ansible-test-fedora-37.yml
+    - ansible-test-freebsd-bootstrap-setuptools.yml
+    - ansible-test-import-sanity-fix.yml
+    - ansible-test-layout-detection.yml
+    - ansible-test-long-timeout-fix.yml
+    - ansible-test-minimum-setuptools.yml
+    - ansible-test-nios-container.yml
+    - ansible-test-pylint-update.yml
+    - ansible-test-pytest-forked.yml
+    - ansible-test-python-3.12.yml
+    - ansible-test-pyyaml-build.yml
+    - ansible-test-remove-old-rhel-remotes.yml
+    - ansible-test-remove-ubuntu-2004.yml
+    - ansible-test-rhel-9.2-python-3.11.yml
+    - ansible-test-rhel-9.2.yml
+    - ansible-test-sanity-scope.yml
+    - ansible-test-source-detection.yml
+    - ansible-test-thread-coverage.yml
+    - ansible-test-timeout-fix.yml
+    - ansible-test-unique-container-names.yml
+    - ansible-test-use-raise-from.yml
+    - ansible-test-utcnow.yml
+    - ansible-test-winrm-config.yml
+    - ansible-vault.yml
+    - ansible_test_alpine_3.18.yml
+    - apt_fail_on_autoremove.yml
+    - aptclean_diff.yml
+    - basestrategy-lazy-templar.yml
+    - ci_freebsd_new.yml
+    - collections_paths-deprecation.yml
+    - colors.yml
+    - command-expand-args.yml
+    - config_origins_option.yml
+    - connection-type-annotation.yml
+    - copy_diff.yml
+    - deb822_open_url.yml
+    - debconf.yml
+    - deprecated_string_conversion_action.yml
+    - display_proxy.yml
+    - dnf-update-only-latest.yml
+    - dnf5-cacheonly.yml
+    - dnf5-fix-interpreter-fail-msg.yml
+    - dnf5-gpg-check-api.yml
+    - dnf5-gpg-check-builtin.yml
+    - dnf5-logs-api.yml
+    - dnf5-test-env-groups.yml
+    - dotnet-preparation.yml
+    - dpkg_selections.yml
+    - fbsd13_1_remove.yml
+    - fetch_url-remove-auto-disable-decompress.yml
+    - find-mode.yml
+    - first_found_fixes.yml
+    - first_found_template_fix.yml
+    - fix-display-prompt-cpu-consumption.yml
+    - fix-handlers-callback.yml
+    - fix-pkg-mgr-in-TencentOS.yml
+    - fix-setuptools-warnings.yml
+    - fix-url-lookup-plugin-docs.yml
+    - forced_local+fix+.yml
+    - freebsd_12_4_removal.yml
+    - galaxy_check_type.yml
+    - galaxy_symlink.yml
+    - gather_facts_fix_parallel.yml
+    - get_action_args_with_defaults-remove-deprecated-arg.yml
+    - group_warning.yml
+    - inventory_cache-remove-deprecated-default-section.yml
+    - inventory_ini.yml
+    - jinja_plugin_cache_cleanup.yml
+    - long-collection-paths-fix.yml
+    - man-page-build-docs-dependency.yml
+    - man-page-subcommands.yml
+    - manifest-in-cleanup.yml
+    - mc_from_config.yml
+    - missing-doc-func.yml
+    - no-arbitrary-j2-override.yml
+    - omit-man-pages-from-sdist.yml
+    - parsing-splitter-fixes.yml
+    - passlib_or_crypt.yml
+    - password_hash-fix-crypt-salt-bcrypt.yml
+    - pep517-backend-import-fix.yml
+    - pep517-backend-traceback-fix.yml
+    - pep8-known-issue.yml
+    - persist_skip.yml
+    - pkg_mgr-default-dnf.yml
+    - powershell-module-error-handling.yml
+    - pre-release-hint-for-dep-resolution-error.yml
+    - pylint-deprecated-comment-checker.yml
+    - reboot.yml
+    - remove-deprecated-actionbase-_remote_checksum.yml
+    - remove-deprecated-datetime-methods.yml
+    - remove-deprecated-filelock-class.yml
+    - remove-docs-examples.yml
+    - remove-include.yml
+    - remove-play_iterator-deprecated-methods.yml
+    - remove-python3.5.yml
+    - remove-python3.9-controller-support.yml
+    - remove-templar-shared_loader_obj-arg.yml
+    - remove-unreachable-include_role-static-err.yml
+    - remove_md5.yml
+    - role-deduplication-condition.yml
+    - run-command-selectors-prompt-only.yml
+    - server2012-deprecation.yml
+    - service_facts_rcctl.yml
+    - service_facts_simpleinit_msb.yml
+    - service_fix_obsd.yml
+    - set-filters.yml
+    - setup_facter_fix.yml
+    - simple-result-queue.yml
+    - smart_connection_bye.yml
+    - suppressed-options.yml
+    - tarfile_extract_warn.yml
+    - templar-globals-dict.yml
+    - templating_fixes.yml
+    - text-converters.yml
+    - timeout_config_fix.yml
+    - update-maybe-json-uri.yml
+    - urls-client-cert-py12.yml
+    - urls-unit-test-latest-cryptography.yml
+    - user-add-password-exp-warning.yml
+    - v2.16.0-initial-commit.yaml
+    - vault_unvault_id_fix.yml
+    - yum-repository-docs-fixes.yml
+    - yum_repository_keepcache.yml
+    release_date: '2023-09-26'
+  2.16.0b2:
+    changes:
+      bugfixes:
+      - '``import_role`` reverts to previous behavior of exporting vars at compile
+        time.'
+      - ansible-galaxy info - fix reporting no role found when lookup_role_by_name
+        returns None.
+      - uri/urls - Add compat function to handle the ability to parse the filename
+        from a Content-Disposition header (https://github.com/ansible/ansible/issues/81806)
+      - winrm - Better handle send input failures when communicating with hosts under
+        load
+      minor_changes:
+      - ansible-test - When invoking ``sleep`` in containers during container setup,
+        the ``env`` command is used to avoid invoking the shell builtin, if present.
+      release_summary: '| Release Date: 2023-10-03
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+        '
+      security_fixes:
+      - ansible-galaxy - Prevent roles from using symlinks to overwrite files outside
+        of the installation directory (CVE-2023-5115)
+    codename: All My Love
+    fragments:
+    - 2.16.0b2_summary.yaml
+    - 81806-py2-content-disposition.yml
+    - ansible-test-container-sleep.yml
+    - cve-2023-5115.yml
+    - fix-ansible-galaxy-info-no-role-found.yml
+    - import_role_goes_public.yml
+    - winrm-send-input.yml
+    release_date: '2023-10-03'
+  2.16.0rc1:
+    changes:
+      bugfixes:
+      - Cache host_group_vars after instantiating it once and limit the amount of
+        repetitive work it needs to do every time it runs.
+      - Call PluginLoader.all() once for vars plugins, and load vars plugins that
+        run automatically or are enabled specifically by name subsequently.
+      - Fix ``run_once`` being incorrectly interpreted on handlers (https://github.com/ansible/ansible/issues/81666)
+      - Properly template tags in parent blocks (https://github.com/ansible/ansible/issues/81053)
+      - ansible-galaxy - Provide a better error message when using a requirements
+        file with an invalid format - https://github.com/ansible/ansible/issues/81901
+      - ansible-inventory - index available_hosts for major performance boost when
+        dumping large inventories
+      - ansible-test - Add a ``pylint`` plugin to work around a known issue on Python
+        3.12.
+      - ansible-test - Include missing ``pylint`` requirements for Python 3.10.
+      - ansible-test - Update ``pylint`` to version 3.0.1.
+      deprecated_features:
+      - Old style vars plugins which use the entrypoints `get_host_vars` or `get_group_vars`
+        are deprecated. The plugin should be updated to inherit from `BaseVarsPlugin`
+        and define a `get_vars` method as the entrypoint.
+      minor_changes:
+      - ansible-test - Make Python 3.12 the default version used in the ``base`` and
+        ``default`` containers.
+      release_summary: '| Release Date: 2023-10-16
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+        '
+    codename: All My Love
+    fragments:
+    - 2.16.0rc1_summary.yaml
+    - 79945-host_group_vars-improvements.yml
+    - 81053-templated-tags-inheritance.yml
+    - 81666-handlers-run_once.yml
+    - 81901-galaxy-requirements-format.yml
+    - ansible-test-pylint3-update.yml
+    - ansible-test-python-3.12-compat.yml
+    - ansible-test-python-default.yml
+    - inv_available_hosts_to_frozenset.yml
+    release_date: '2023-10-16'
+  2.16.1:
+    changes:
+      release_summary: '| Release Date: 2023-12-04
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+        '
+    codename: All My Love
+    fragments:
+    - 2.16.1_summary.yaml
+    release_date: '2023-12-04'
+  2.16.1rc1:
+    changes:
+      breaking_changes:
+      - assert - Nested templating may result in an inability for the conditional
+        to be evaluated. See the porting guide for more information.
+      bugfixes:
+      - Fix issue where an ``include_tasks`` handler in a role was not able to locate
+        a file in ``tasks/`` when ``tasks_from`` was used as a role entry point and
+        ``main.yml`` was not present (https://github.com/ansible/ansible/issues/82241)
+      - Plugin loader does not dedupe nor cache filter/test plugins by file basename,
+        but full path name.
+      - Restoring the ability of filters/tests can have same file base name but different
+        tests/filters defined inside.
+      - ansible-pull now will expand relative paths for the ``-d|--directory`` option
+        is now expanded before use.
+      - ansible-pull will now correctly handle become and connection password file
+        options for ansible-playbook.
+      - flush_handlers - properly handle a handler failure in a nested block when
+        ``force_handlers`` is set (http://github.com/ansible/ansible/issues/81532)
+      - module no_log will no longer affect top level booleans, for example ``no_log_module_parameter='a'``
+        will no longer hide ``changed=False`` as a 'no log value' (matches 'a').
+      - role params now have higher precedence than host facts again, matching documentation,
+        this had unintentionally changed in 2.15.
+      - wait_for should not handle 'non mmapable files' again.
+      release_summary: '| Release Date: 2023-11-27
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+        '
+      security_fixes:
+      - templating - Address issues where internal templating can cause unsafe variables
+        to lose their unsafe designation (CVE-2023-5764)
+    codename: All My Love
+    fragments:
+    - 2.16.1rc1_summary.yaml
+    - 81532-fix-nested-flush_handlers.yml
+    - 82241-handler-include-tasks-from.yml
+    - cve-2023-5764.yml
+    - j2_load_fix.yml
+    - no_log_booly.yml
+    - pull_file_secrets.yml
+    - pull_unfrack_dest.yml
+    - restore_role_param_precedence.yml
+    - wait_for_mmap.yml
+    release_date: '2023-11-27'
+  2.16.2:
+    changes:
+      bugfixes:
+      - unsafe data - Address an incompatibility when iterating or getting a single
+        index from ``AnsibleUnsafeBytes``
+      - unsafe data - Address an incompatibility with ``AnsibleUnsafeText`` and ``AnsibleUnsafeBytes``
+        when pickling with ``protocol=0``
+      release_summary: '| Release Date: 2023-12-11
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+        '
+    codename: All My Love
+    fragments:
+    - 2.16.2_summary.yaml
+    - unsafe-fixes-2.yml
+    release_date: '2023-12-11'
+  2.16.3:
+    changes:
+      release_summary: '| Release Date: 2024-01-29
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+        '
+    codename: All My Love
+    fragments:
+    - 2.16.3_summary.yaml
+    release_date: '2024-01-29'
+  2.16.3rc1:
+    changes:
+      bugfixes:
+      - Run all handlers with the same ``listen`` topic, even when notified from another
+        handler (https://github.com/ansible/ansible/issues/82363).
+      - '``ansible-galaxy role import`` - fix using the ``role_name`` in a standalone
+        role''s ``galaxy_info`` metadata by disabling automatic removal of the ``ansible-role-``
+        prefix. This matches the behavior of the Galaxy UI which also no longer implicitly
+        removes the ``ansible-role-`` prefix. Use the ``--role-name`` option or add
+        a ``role_name`` to the ``galaxy_info`` dictionary in the role''s ``meta/main.yml``
+        to use an alternate role name.'
+      - '``ansible-test sanity --test runtime-metadata`` - add ``action_plugin`` as
+        a valid field for modules in the schema (https://github.com/ansible/ansible/pull/82562).'
+      - ansible-config init will now dedupe ini entries from plugins.
+      - ansible-galaxy role import - exit with 1 when the import fails (https://github.com/ansible/ansible/issues/82175).
+      - ansible-galaxy role install - normalize tarfile paths and symlinks using ``ansible.utils.path.unfrackpath``
+        and consider them valid as long as the realpath is in the tarfile's role directory
+        (https://github.com/ansible/ansible/issues/81965).
+      - delegate_to when set to an empty or undefined variable will now give a proper
+        error.
+      - dwim functions for lookups should be better at detectging role context even
+        in abscense of tasks/main.
+      - roles, code cleanup and performance optimization of dependencies, now cached,  and
+        ``public`` setting is now determined once, at role instantiation.
+      - roles, the ``static`` property is now correctly set, this will fix issues
+        with ``public`` and ``DEFAULT_PRIVATE_ROLE_VARS`` controls on exporting vars.
+      - unsafe data - Enable directly using ``AnsibleUnsafeText`` with Python ``pathlib``
+        (https://github.com/ansible/ansible/issues/82414)
+      release_summary: '| Release Date: 2024-01-22
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+        '
+      security_fixes:
+      - ANSIBLE_NO_LOG - Address issue where ANSIBLE_NO_LOG was ignored (CVE-2024-0690)
+    codename: All My Love
+    fragments:
+    - 2.16.3rc1_summary.yaml
+    - 82175-fix-ansible-galaxy-role-import-rc.yml
+    - 82363-multiple-handlers-with-recursive-notification.yml
+    - ansible-galaxy-role-install-symlink.yml
+    - cve-2024-0690.yml
+    - dedupe_config_init.yml
+    - delegate_to_invalid.yml
+    - dwim_is_role_fix.yml
+    - fix-default-ansible-galaxy-role-import-name.yml
+    - fix-runtime-metadata-modules-action_plugin.yml
+    - role_fixes.yml
+    - unsafe-intern.yml
+    release_date: '2024-01-22'
--- a/changelogs/fragments/2.16.0_summary.yaml
+++ b/changelogs/fragments/2.16.0_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2023-11-06
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
--- a/changelogs/fragments/2.16.0b1_summary.yaml
+++ b/changelogs/fragments/2.16.0b1_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2023-09-26
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
--- a/changelogs/fragments/2.16.0b2_summary.yaml
+++ b/changelogs/fragments/2.16.0b2_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2023-10-03
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
--- a/changelogs/fragments/2.16.0rc1_summary.yaml
+++ b/changelogs/fragments/2.16.0rc1_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2023-10-16
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
--- a/changelogs/fragments/2.16.1_summary.yaml
+++ b/changelogs/fragments/2.16.1_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2023-12-04
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
--- a/changelogs/fragments/2.16.1rc1_summary.yaml
+++ b/changelogs/fragments/2.16.1rc1_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2023-11-27
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
--- a/changelogs/fragments/2.16.2_summary.yaml
+++ b/changelogs/fragments/2.16.2_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2023-12-11
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
--- a/changelogs/fragments/2.16.3_summary.yaml
+++ b/changelogs/fragments/2.16.3_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2024-01-29
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
--- a/changelogs/fragments/2.16.3rc1_summary.yaml
+++ b/changelogs/fragments/2.16.3rc1_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2024-01-22
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
--- a/changelogs/fragments/79945-host_group_vars-improvements.yml
+++ b/changelogs/fragments/79945-host_group_vars-improvements.yml
@ -0,0 +1,5 @@
+bugfixes:
+  - Cache host_group_vars after instantiating it once and limit the amount of repetitive work it needs to do every time it runs.
+  - Call PluginLoader.all() once for vars plugins, and load vars plugins that run automatically or are enabled specifically by name subsequently.
+deprecated_features:
+  - Old style vars plugins which use the entrypoints `get_host_vars` or `get_group_vars` are deprecated. The plugin should be updated to inherit from `BaseVarsPlugin` and define a `get_vars` method as the entrypoint.
--- a/changelogs/fragments/81053-templated-tags-inheritance.yml
+++ b/changelogs/fragments/81053-templated-tags-inheritance.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - Properly template tags in parent blocks (https://github.com/ansible/ansible/issues/81053)
--- a/changelogs/fragments/81532-fix-nested-flush_handlers.yml
+++ b/changelogs/fragments/81532-fix-nested-flush_handlers.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - flush_handlers - properly handle a handler failure in a nested block when ``force_handlers`` is set (http://github.com/ansible/ansible/issues/81532)
--- a/changelogs/fragments/81584-daemonize-follow-up-fixes.yml
+++ b/changelogs/fragments/81584-daemonize-follow-up-fixes.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - "``ansible.module_utils.service`` - ensure binary data transmission in ``daemonize()``"
--- a/changelogs/fragments/81606-ansible-galaxy-collection-pre-releases.yml
+++ b/changelogs/fragments/81606-ansible-galaxy-collection-pre-releases.yml
@ -0,0 +1,16 @@
+---
+
+bugfixes:
+- >-
+  ansible-galaxy - started allowing the use of pre-releases
+  for dependencies on any level of the dependency tree that
+  specifically demand exact pre-release versions of
+  collections and not version ranges.
+  (https://github.com/ansible/ansible/pull/81606)
+- >-
+  ansible-galaxy - started allowing the use of pre-releases
+  for collections that do not have any stable versions
+  published.
+  (https://github.com/ansible/ansible/pull/81606)
+
+...
--- a/changelogs/fragments/81666-handlers-run_once.yml
+++ b/changelogs/fragments/81666-handlers-run_once.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - Fix ``run_once`` being incorrectly interpreted on handlers (https://github.com/ansible/ansible/issues/81666)
--- a/changelogs/fragments/81722-handler-subdir-include_tasks.yml
+++ b/changelogs/fragments/81722-handler-subdir-include_tasks.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - Allow for searching handler subdir for included task via include_role (https://github.com/ansible/ansible/issues/81722)
--- a/changelogs/fragments/81806-py2-content-disposition.yml
+++ b/changelogs/fragments/81806-py2-content-disposition.yml
@ -0,0 +1,3 @@
+bugfixes:
+- uri/urls - Add compat function to handle the ability to parse the filename from a Content-Disposition header
+  (https://github.com/ansible/ansible/issues/81806)
--- a/changelogs/fragments/81901-galaxy-requirements-format.yml
+++ b/changelogs/fragments/81901-galaxy-requirements-format.yml
@ -0,0 +1,2 @@
+bugfixes:
+- ansible-galaxy - Provide a better error message when using a requirements file with an invalid format - https://github.com/ansible/ansible/issues/81901
--- a/changelogs/fragments/82175-fix-ansible-galaxy-role-import-rc.yml
+++ b/changelogs/fragments/82175-fix-ansible-galaxy-role-import-rc.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-galaxy role import - exit with 1 when the import fails (https://github.com/ansible/ansible/issues/82175).
--- a/changelogs/fragments/82241-handler-include-tasks-from.yml
+++ b/changelogs/fragments/82241-handler-include-tasks-from.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - Fix issue where an ``include_tasks`` handler in a role was not able to locate a file in ``tasks/`` when ``tasks_from`` was used as a role entry point and ``main.yml`` was not present (https://github.com/ansible/ansible/issues/82241)
--- a/changelogs/fragments/82363-multiple-handlers-with-recursive-notification.yml
+++ b/changelogs/fragments/82363-multiple-handlers-with-recursive-notification.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - Run all handlers with the same ``listen`` topic, even when notified from another handler (https://github.com/ansible/ansible/issues/82363).
--- a/changelogs/fragments/ansible-galaxy-role-install-symlink.yml
+++ b/changelogs/fragments/ansible-galaxy-role-install-symlink.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-galaxy role install - normalize tarfile paths and symlinks using ``ansible.utils.path.unfrackpath`` and consider them valid as long as the realpath is in the tarfile's role directory (https://github.com/ansible/ansible/issues/81965).
--- a/changelogs/fragments/ansible-test-cgroup-split.yml
+++ b/changelogs/fragments/ansible-test-cgroup-split.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-test - Fix parsing of cgroup entries which contain a ``:`` in the path (https://github.com/ansible/ansible/issues/81977).
--- a/changelogs/fragments/ansible-test-container-sleep.yml
+++ b/changelogs/fragments/ansible-test-container-sleep.yml
@ -0,0 +1,3 @@
+minor_changes:
+- ansible-test - When invoking ``sleep`` in containers during container setup, the ``env`` command is used to avoid invoking
+                 the shell builtin, if present.
--- a/changelogs/fragments/ansible-test-coverage-update.yml
+++ b/changelogs/fragments/ansible-test-coverage-update.yml
@ -1,7 +1,7 @@
 ---

 minor_changes:
- ansible-test — Python 3.8–3.12 will use ``coverage`` v7.3.0.
+- ansible-test — Python 3.8–3.12 will use ``coverage`` v7.3.2.
 - ansible-test — ``coverage`` v6.5.0 is to be used only under Python 3.7.

 ...
--- a/changelogs/fragments/ansible-test-distro-containers.yml
+++ b/changelogs/fragments/ansible-test-distro-containers.yml
@ -1,2 +1,4 @@
 minor_changes:
  - ansible-test - The openSUSE test container has been updated to openSUSE Leap 15.5.
+  - ansible-test - Updated the distro test containers to version 6.3.0 to include coverage 7.3.2 for Python 3.8+.
+                   The alpine3 container is now based on 3.18 instead of 3.17 and includes Python 3.11 instead of Python 3.10.
--- a/changelogs/fragments/ansible-test-pylint3-update.yml
+++ b/changelogs/fragments/ansible-test-pylint3-update.yml
@ -0,0 +1,3 @@
+bugfixes:
+  - ansible-test - Update ``pylint`` to version 3.0.1.
+  - ansible-test - Include missing ``pylint`` requirements for Python 3.10.
--- a/changelogs/fragments/ansible-test-python-3.12-compat.yml
+++ b/changelogs/fragments/ansible-test-python-3.12-compat.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-test - Add a ``pylint`` plugin to work around a known issue on Python 3.12.
--- a/changelogs/fragments/ansible-test-python-default.yml
+++ b/changelogs/fragments/ansible-test-python-default.yml
@ -0,0 +1,2 @@
+minor_changes:
+  - ansible-test - Make Python 3.12 the default version used in the ``base`` and ``default`` containers.
--- a/changelogs/fragments/cve-2023-5115.yml
+++ b/changelogs/fragments/cve-2023-5115.yml
@ -0,0 +1,3 @@
+security_fixes:
+- ansible-galaxy - Prevent roles from using symlinks to overwrite
+  files outside of the installation directory (CVE-2023-5115)
--- a/changelogs/fragments/cve-2023-5764.yml
+++ b/changelogs/fragments/cve-2023-5764.yml
@ -0,0 +1,6 @@
+security_fixes:
+- templating - Address issues where internal templating can cause unsafe
+  variables to lose their unsafe designation (CVE-2023-5764)
+breaking_changes:
+- assert - Nested templating may result in an inability for the conditional
+  to be evaluated. See the porting guide for more information.
--- a/changelogs/fragments/cve-2024-0690.yml
+++ b/changelogs/fragments/cve-2024-0690.yml
@ -0,0 +1,2 @@
+security_fixes:
+- ANSIBLE_NO_LOG - Address issue where ANSIBLE_NO_LOG was ignored (CVE-2024-0690)
--- a/changelogs/fragments/dedupe_config_init.yml
+++ b/changelogs/fragments/dedupe_config_init.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-config init will now dedupe ini entries from plugins.
--- a/changelogs/fragments/delegate_to_invalid.yml
+++ b/changelogs/fragments/delegate_to_invalid.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - delegate_to when set to an empty or undefined variable will now give a proper error.
--- a/changelogs/fragments/dwim_is_role_fix.yml
+++ b/changelogs/fragments/dwim_is_role_fix.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - dwim functions for lookups should be better at detectging role context even in abscense of tasks/main.
--- a/changelogs/fragments/fix-ansible-galaxy-info-no-role-found.yml
+++ b/changelogs/fragments/fix-ansible-galaxy-info-no-role-found.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-galaxy info - fix reporting no role found when lookup_role_by_name returns None.
--- a/changelogs/fragments/fix-default-ansible-galaxy-role-import-name.yml
+++ b/changelogs/fragments/fix-default-ansible-galaxy-role-import-name.yml
@ -0,0 +1,8 @@
+bugfixes:
+  - >-
+    ``ansible-galaxy role import`` - fix using the ``role_name`` in a standalone role's
+    ``galaxy_info`` metadata by disabling automatic removal of the ``ansible-role-`` prefix.
+    This matches the behavior of the Galaxy UI which also no longer implicitly removes the
+    ``ansible-role-`` prefix.
+    Use the ``--role-name`` option or add a ``role_name`` to the ``galaxy_info`` dictionary
+    in the role's ``meta/main.yml`` to use an alternate role name.
--- a/changelogs/fragments/fix-runtime-metadata-modules-action_plugin.yml
+++ b/changelogs/fragments/fix-runtime-metadata-modules-action_plugin.yml
@ -0,0 +1,4 @@
+bugfixes:
+- >-
+  ``ansible-test sanity --test runtime-metadata`` - add ``action_plugin`` as a valid field
+  for modules in the schema (https://github.com/ansible/ansible/pull/82562).
--- a/changelogs/fragments/import_role_goes_public.yml
+++ b/changelogs/fragments/import_role_goes_public.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - "``import_role`` reverts to previous behavior of exporting vars at compile time."
--- a/changelogs/fragments/inv_available_hosts_to_frozenset.yml
+++ b/changelogs/fragments/inv_available_hosts_to_frozenset.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-inventory - index available_hosts for major performance boost when dumping large inventories
--- a/changelogs/fragments/j2_load_fix.yml
+++ b/changelogs/fragments/j2_load_fix.yml
@ -0,0 +1,3 @@
+bugfixes:
+  - Plugin loader does not dedupe nor cache filter/test plugins by file basename, but full path name.
+  - Restoring the ability of filters/tests can have same file base name but different tests/filters defined inside.
--- a/changelogs/fragments/no_log_booly.yml
+++ b/changelogs/fragments/no_log_booly.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - module no_log will no longer affect top level booleans, for example ``no_log_module_parameter='a'`` will no longer hide ``changed=False`` as a 'no log value' (matches 'a').
--- a/changelogs/fragments/pull_file_secrets.yml
+++ b/changelogs/fragments/pull_file_secrets.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-pull will now correctly handle become and connection password file options for ansible-playbook.
--- a/changelogs/fragments/pull_unfrack_dest.yml
+++ b/changelogs/fragments/pull_unfrack_dest.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-pull now will expand relative paths for the ``-d|--directory`` option is now expanded before use.
--- a/changelogs/fragments/restore_role_param_precedence.yml
+++ b/changelogs/fragments/restore_role_param_precedence.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - role params now have higher precedence than host facts again, matching documentation, this had unintentionally changed in 2.15.
--- a/changelogs/fragments/role_fixes.yml
+++ b/changelogs/fragments/role_fixes.yml
@ -0,0 +1,3 @@
+bugfixes:
+  - roles, the ``static`` property is now correctly set, this will fix issues with ``public`` and ``DEFAULT_PRIVATE_ROLE_VARS`` controls on exporting vars.
+  - roles, code cleanup and performance optimization of dependencies, now cached,  and ``public`` setting is now determined once, at role instantiation.
--- a/changelogs/fragments/unsafe-fixes-2.yml
+++ b/changelogs/fragments/unsafe-fixes-2.yml
@ -0,0 +1,3 @@
+bugfixes:
+- unsafe data - Address an incompatibility with ``AnsibleUnsafeText`` and ``AnsibleUnsafeBytes`` when pickling with ``protocol=0``
+- unsafe data - Address an incompatibility when iterating or getting a single index from ``AnsibleUnsafeBytes``
--- a/changelogs/fragments/unsafe-intern.yml
+++ b/changelogs/fragments/unsafe-intern.yml
@ -0,0 +1,3 @@
+bugfixes:
+- unsafe data - Enable directly using ``AnsibleUnsafeText`` with Python ``pathlib``
+  (https://github.com/ansible/ansible/issues/82414)
--- a/changelogs/fragments/wait_for_mmap.yml
+++ b/changelogs/fragments/wait_for_mmap.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - wait_for should not handle 'non mmapable files' again.
--- a/changelogs/fragments/winrm-send-input.yml
+++ b/changelogs/fragments/winrm-send-input.yml
@ -0,0 +1,2 @@
+bugfixes:
+- winrm - Better handle send input failures when communicating with hosts under load
--- a/lib/ansible/cli/config.py
+++ b/lib/ansible/cli/config.py
@ -314,7 +314,7 @@ class ConfigCLI(CLI):

        return data

-    def _get_settings_ini(self, settings):
+    def _get_settings_ini(self, settings, seen):

        sections = {}
        for o in sorted(settings.keys()):
@ -327,7 +327,7 @@ class ConfigCLI(CLI):

            if not opt.get('description'):
                # its a plugin
-                new_sections = self._get_settings_ini(opt)
+                new_sections = self._get_settings_ini(opt, seen)
                for s in new_sections:
                    if s in sections:
                        sections[s].extend(new_sections[s])
@ -343,37 +343,45 @@ class ConfigCLI(CLI):

            if 'ini' in opt and opt['ini']:
                entry = opt['ini'][-1]
+                if entry['section'] not in seen:
+                    seen[entry['section']] = []
                if entry['section'] not in sections:
                    sections[entry['section']] = []

-                default = opt.get('default', '')
-                if opt.get('type', '') == 'list' and not isinstance(default, string_types):
-                    # python lists are not valid ini ones
-                    default = ', '.join(default)
-                elif default is None:
-                    default = ''
+                # avoid dupes
+                if entry['key'] not in seen[entry['section']]:
+                    seen[entry['section']].append(entry['key'])

-                if context.CLIARGS['commented']:
-                    entry['key'] = ';%s' % entry['key']
+                    default = opt.get('default', '')
+                    if opt.get('type', '') == 'list' and not isinstance(default, string_types):
+                        # python lists are not valid ini ones
+                        default = ', '.join(default)
+                    elif default is None:
+                        default = ''

-                key = desc + '\n%s=%s' % (entry['key'], default)
-                sections[entry['section']].append(key)
+                    if context.CLIARGS['commented']:
+                        entry['key'] = ';%s' % entry['key']
+
+                    key = desc + '\n%s=%s' % (entry['key'], default)
+
+                    sections[entry['section']].append(key)

        return sections

    def execute_init(self):
        """Create initial configuration"""

+        seen = {}
        data = []
        config_entries = self._list_entries_from_args()
        plugin_types = config_entries.pop('PLUGINS', None)

        if context.CLIARGS['format'] == 'ini':
-            sections = self._get_settings_ini(config_entries)
+            sections = self._get_settings_ini(config_entries, seen)

            if plugin_types:
                for ptype in plugin_types:
-                    plugin_sections = self._get_settings_ini(plugin_types[ptype])
+                    plugin_sections = self._get_settings_ini(plugin_types[ptype], seen)
                    for s in plugin_sections:
                        if s in sections:
                            sections[s].extend(plugin_sections[s])
--- a/lib/ansible/cli/galaxy.py
+++ b/lib/ansible/cli/galaxy.py
@ -835,7 +835,7 @@ class GalaxyCLI(CLI):
            for role_req in file_requirements:
                requirements['roles'] += parse_role_req(role_req)

-        else:
+        elif isinstance(file_requirements, dict):
            # Newer format with a collections and/or roles key
            extra_keys = set(file_requirements.keys()).difference(set(['roles', 'collections']))
            if extra_keys:
@ -854,6 +854,9 @@ class GalaxyCLI(CLI):
                for collection_req in file_requirements.get('collections') or []
            ]

+        else:
+            raise AnsibleError(f"Expecting requirements yaml to be a list or dictionary but got {type(file_requirements).__name__}")
+
        return requirements

    def _init_coll_req_dict(self, coll_req):
@ -1265,6 +1268,9 @@ class GalaxyCLI(CLI):

                if remote_data:
                    role_info.update(remote_data)
+                else:
+                    data = u"- the role %s was not found" % role
+                    break

            elif context.CLIARGS['offline'] and not gr._exists:
                data = u"- the role %s was not found" % role
@ -1786,6 +1792,7 @@ class GalaxyCLI(CLI):
        github_user = to_text(context.CLIARGS['github_user'], errors='surrogate_or_strict')
        github_repo = to_text(context.CLIARGS['github_repo'], errors='surrogate_or_strict')

+        rc = 0
        if context.CLIARGS['check_status']:
            task = self.api.get_import_task(github_user=github_user, github_repo=github_repo)
        else:
@ -1803,7 +1810,7 @@ class GalaxyCLI(CLI):
                    display.display('%s.%s' % (t['summary_fields']['role']['namespace'], t['summary_fields']['role']['name']), color=C.COLOR_CHANGED)
                display.display(u'\nTo properly namespace this role, remove each of the above and re-import %s/%s from scratch' % (github_user, github_repo),
                                color=C.COLOR_CHANGED)
-                return 0
+                return rc
            # found a single role as expected
            display.display("Successfully submitted import request %d" % task[0]['id'])
            if not context.CLIARGS['wait']:
@ -1820,12 +1827,13 @@ class GalaxyCLI(CLI):
                    if msg['id'] not in msg_list:
                        display.display(msg['message_text'], color=colors[msg['message_type']])
                        msg_list.append(msg['id'])
-                if task[0]['state'] in ['SUCCESS', 'FAILED']:
+                if (state := task[0]['state']) in ['SUCCESS', 'FAILED']:
+                    rc = ['SUCCESS', 'FAILED'].index(state)
                    finished = True
                else:
                    time.sleep(10)

-        return 0
+        return rc

    def execute_setup(self):
        """ Setup an integration from Github or Travis for Ansible Galaxy roles"""
--- a/lib/ansible/cli/inventory.py
+++ b/lib/ansible/cli/inventory.py
@ -325,7 +325,7 @@ class InventoryCLI(CLI):
            return results

        hosts = self.inventory.get_hosts(top.name)
-        results = format_group(top, [h.name for h in hosts])
+        results = format_group(top, frozenset(h.name for h in hosts))

        # populate meta
        results['_meta'] = {'hostvars': {}}
@ -381,7 +381,7 @@ class InventoryCLI(CLI):

            return results

-        available_hosts = [h.name for h in self.inventory.get_hosts(top.name)]
+        available_hosts = frozenset(h.name for h in self.inventory.get_hosts(top.name))
        return format_group(top, available_hosts)

    def toml_inventory(self, top):
@ -425,7 +425,7 @@ class InventoryCLI(CLI):

            return results

-        available_hosts = [h.name for h in self.inventory.get_hosts(top.name)]
+        available_hosts = frozenset(h.name for h in self.inventory.get_hosts(top.name))
        results = format_group(top, available_hosts)

        return results
--- a/lib/ansible/cli/pull.py
+++ b/lib/ansible/cli/pull.py
@ -29,6 +29,7 @@ from ansible.plugins.loader import module_loader
 from ansible.utils.cmd_functions import run_cmd
 from ansible.utils.display import Display

+
 display = Display()


@ -102,8 +103,8 @@ class PullCLI(CLI):
                                      'This is a useful way to disperse git requests')
        self.parser.add_argument('-f', '--force', dest='force', default=False, action='store_true',
                                 help='run the playbook even if the repository could not be updated')
-        self.parser.add_argument('-d', '--directory', dest='dest', default=None,
-                                 help='absolute path of repository checkout directory (relative paths are not supported)')
+        self.parser.add_argument('-d', '--directory', dest='dest', default=None, type=opt_help.unfrack_path(),
+                                 help='path to the directory to which Ansible will checkout the repository.')
        self.parser.add_argument('-U', '--url', dest='url', default=None, help='URL of the playbook repository')
        self.parser.add_argument('--full', dest='fullclone', action='store_true', help='Do a full clone, instead of a shallow one.')
        self.parser.add_argument('-C', '--checkout', dest='checkout',
@ -134,7 +135,6 @@ class PullCLI(CLI):
            hostname = socket.getfqdn()
            # use a hostname dependent directory, in case of $HOME on nfs
            options.dest = os.path.join(C.ANSIBLE_HOME, 'pull', hostname)
-        options.dest = os.path.expandvars(os.path.expanduser(options.dest))

        if os.path.exists(options.dest) and not os.path.isdir(options.dest):
            raise AnsibleOptionsError("%s is not a valid or accessible directory." % options.dest)
@ -275,8 +275,15 @@ class PullCLI(CLI):
            for vault_id in context.CLIARGS['vault_ids']:
                cmd += " --vault-id=%s" % vault_id

+        if context.CLIARGS['become_password_file']:
+            cmd += " --become-password-file=%s" % context.CLIARGS['become_password_file']
+
+        if context.CLIARGS['connection_password_file']:
+            cmd += " --connection-password-file=%s" % context.CLIARGS['connection_password_file']
+
        for ev in context.CLIARGS['extra_vars']:
            cmd += ' -e %s' % shlex.quote(ev)
+
        if context.CLIARGS['become_ask_pass']:
            cmd += ' --ask-become-pass'
        if context.CLIARGS['skip_tags']:
@ -307,6 +314,7 @@ class PullCLI(CLI):
        if context.CLIARGS['purge']:
            os.chdir('/')
            try:
+                display.debug("removing: %s" % context.CLIARGS['dest'])
                shutil.rmtree(context.CLIARGS['dest'])
            except Exception as e:
                display.error(u"Failed to remove %s: %s" % (context.CLIARGS['dest'], to_text(e)))
--- a/lib/ansible/config/base.yml
+++ b/lib/ansible/config/base.yml
@ -934,9 +934,9 @@ DEFAULT_PRIVATE_ROLE_VARS:
  name: Private role variables
  default: False
  description:
-    - Makes role variables inaccessible from other roles.
-    - This was introduced as a way to reset role variables to default values if
-      a role is used more than once in a playbook.
+    - By default, imported roles publish their variables to the play and other roles, this setting can avoid that.
+    - This was introduced as a way to reset role variables to default values if a role is used more than once in a playbook.
+    - Included roles only make their variables public at execution, unlike imported roles which happen at playbook compile time.
  env: [{name: ANSIBLE_PRIVATE_ROLE_VARS}]
  ini:
  - {key: private_role_vars, section: defaults}
--- a/lib/ansible/executor/play_iterator.py
+++ b/lib/ansible/executor/play_iterator.py
@ -52,7 +52,7 @@ class FailedStates(IntFlag):
    TASKS = 2
    RESCUE = 4
    ALWAYS = 8
-    HANDLERS = 16
+    HANDLERS = 16  # NOTE not in use anymore


 class HostState:
@ -431,22 +431,18 @@ class PlayIterator:
                    state.update_handlers = False
                    state.cur_handlers_task = 0

-                if state.fail_state & FailedStates.HANDLERS == FailedStates.HANDLERS:
-                    state.update_handlers = True
-                    state.run_state = IteratingStates.COMPLETE
-                else:
-                    while True:
-                        try:
-                            task = state.handlers[state.cur_handlers_task]
-                        except IndexError:
-                            task = None
-                            state.run_state = state.pre_flushing_run_state
-                            state.update_handlers = True
+                while True:
+                    try:
+                        task = state.handlers[state.cur_handlers_task]
+                    except IndexError:
+                        task = None
+                        state.run_state = state.pre_flushing_run_state
+                        state.update_handlers = True
+                        break
+                    else:
+                        state.cur_handlers_task += 1
+                        if task.is_host_notified(host):
                            break
-                        else:
-                            state.cur_handlers_task += 1
-                            if task.is_host_notified(host):
-                                break

            elif state.run_state == IteratingStates.COMPLETE:
                return (state, None)
@ -487,20 +483,16 @@ class PlayIterator:
            else:
                state.fail_state |= FailedStates.ALWAYS
                state.run_state = IteratingStates.COMPLETE
-        elif state.run_state == IteratingStates.HANDLERS:
-            state.fail_state |= FailedStates.HANDLERS
-            state.update_handlers = True
-            if state._blocks[state.cur_block].rescue:
-                state.run_state = IteratingStates.RESCUE
-            elif state._blocks[state.cur_block].always:
-                state.run_state = IteratingStates.ALWAYS
-            else:
-                state.run_state = IteratingStates.COMPLETE
        return state

    def mark_host_failed(self, host):
        s = self.get_host_state(host)
        display.debug("marking host %s failed, current state: %s" % (host, s))
+        if s.run_state == IteratingStates.HANDLERS:
+            # we are failing `meta: flush_handlers`, so just reset the state to whatever
+            # it was before and let `_set_failed_state` figure out the next state
+            s.run_state = s.pre_flushing_run_state
+            s.update_handlers = True
        s = self._set_failed_state(s)
        display.debug("^ failed state is now: %s" % s)
        self.set_state_for_host(host.name, s)
@ -516,8 +508,6 @@ class PlayIterator:
            return True
        elif state.run_state == IteratingStates.ALWAYS and self._check_failed_state(state.always_child_state):
            return True
-        elif state.run_state == IteratingStates.HANDLERS and state.fail_state & FailedStates.HANDLERS == FailedStates.HANDLERS:
-            return True
        elif state.fail_state != FailedStates.NONE:
            if state.run_state == IteratingStates.RESCUE and state.fail_state & FailedStates.RESCUE == 0:
                return False
--- a/lib/ansible/executor/task_executor.py
+++ b/lib/ansible/executor/task_executor.py
@ -408,11 +408,7 @@ class TaskExecutor:
        """This method is responsible for effectively pre-validating Task.delegate_to and will
        happen before Task.post_validate is executed
        """
-        delegated_vars, delegated_host_name = self._variable_manager.get_delegated_vars_and_hostname(
-            templar,
-            self._task,
-            variables
-        )
+        delegated_vars, delegated_host_name = self._variable_manager.get_delegated_vars_and_hostname(templar, self._task, variables)
        # At the point this is executed it is safe to mutate self._task,
        # since `self._task` is either a copy referred to by `tmp_task` in `_run_loop`
        # or just a singular non-looped task
--- a/lib/ansible/galaxy/api.py
+++ b/lib/ansible/galaxy/api.py
@ -483,8 +483,6 @@ class GalaxyAPI:
        }
        if role_name:
            args['alternate_role_name'] = role_name
-        elif github_repo.startswith('ansible-role'):
-            args['alternate_role_name'] = github_repo[len('ansible-role') + 1:]
        data = self._call_galaxy(url, args=urlencode(args), method="POST")
        if data.get('results', None):
            return data['results']
--- a/lib/ansible/galaxy/collection/init.py
+++ b/lib/ansible/galaxy/collection/init.py
@ -1817,15 +1817,15 @@ def _resolve_depenency_map(
        elif not req.specifier.contains(RESOLVELIB_VERSION.vstring):
            raise AnsibleError(f"ansible-galaxy requires {req.name}{req.specifier}")

-    if allow_pre_release:
-        pre_release_hint = ''
-    else:
-        pre_release_hint = 'Hint: Pre-releases are not installed by default unless the specific version is given. To enable pre-releases, use --pre.'
+    pre_release_hint = '' if allow_pre_release else (
+        'Hint: Pre-releases hosted on Galaxy or Automation Hub are not '
+        'installed by default unless a specific version is requested. '
+        'To enable pre-releases globally, use --pre.'
+    )

    collection_dep_resolver = build_collection_dependency_resolver(
        galaxy_apis=galaxy_apis,
        concrete_artifacts_manager=concrete_artifacts_manager,
-        user_requirements=requested_requirements,
        preferred_candidates=preferred_candidates,
        with_deps=not no_deps,
        with_pre_releases=allow_pre_release,
--- a/lib/ansible/galaxy/dependency_resolution/init.py
+++ b/lib/ansible/galaxy/dependency_resolution/init.py
@ -13,10 +13,7 @@ if t.TYPE_CHECKING:
    from ansible.galaxy.collection.concrete_artifact_manager import (
        ConcreteArtifactsManager,
    )
-    from ansible.galaxy.dependency_resolution.dataclasses import (
-        Candidate,
-        Requirement,
-    )
+    from ansible.galaxy.dependency_resolution.dataclasses import Candidate

 from ansible.galaxy.collection.galaxy_api_proxy import MultiGalaxyAPIProxy
 from ansible.galaxy.dependency_resolution.providers import CollectionDependencyProvider
@ -27,7 +24,6 @@ from ansible.galaxy.dependency_resolution.resolvers import CollectionDependencyR
 def build_collection_dependency_resolver(
        galaxy_apis,  # type: t.Iterable[GalaxyAPI]
        concrete_artifacts_manager,  # type: ConcreteArtifactsManager
-        user_requirements,  # type: t.Iterable[Requirement]
        preferred_candidates=None,  # type: t.Iterable[Candidate]
        with_deps=True,  # type: bool
        with_pre_releases=False,  # type: bool
@ -44,7 +40,6 @@ def build_collection_dependency_resolver(
        CollectionDependencyProvider(
            apis=MultiGalaxyAPIProxy(galaxy_apis, concrete_artifacts_manager, offline=offline),
            concrete_artifacts_manager=concrete_artifacts_manager,
-            user_requirements=user_requirements,
            preferred_candidates=preferred_candidates,
            with_deps=with_deps,
            with_pre_releases=with_pre_releases,
--- a/lib/ansible/galaxy/dependency_resolution/dataclasses.py
+++ b/lib/ansible/galaxy/dependency_resolution/dataclasses.py
@ -564,6 +564,27 @@ class _ComputedReqKindsMixin:
    def is_online_index_pointer(self):
        return not self.is_concrete_artifact

+    @property
+    def is_pinned(self):
+        """Indicate if the version set is considered pinned.
+
+        This essentially computes whether the version field of the current
+        requirement explicitly requests a specific version and not an allowed
+        version range.
+
+        It is then used to help the resolvelib-based dependency resolver judge
+        whether it's acceptable to consider a pre-release candidate version
+        despite pre-release installs not being requested by the end-user
+        explicitly.
+
+        See https://github.com/ansible/ansible/pull/81606 for extra context.
+        """
+        version_string = self.ver[0]
+        return version_string.isdigit() or not (
+            version_string == '*' or
+            version_string.startswith(('<', '>', '!='))
+        )
+
    @property
    def source_info(self):
        return self._source_info
--- a/lib/ansible/galaxy/dependency_resolution/providers.py
+++ b/lib/ansible/galaxy/dependency_resolution/providers.py
@ -51,7 +51,6 @@ class CollectionDependencyProviderBase(AbstractProvider):
            self,  # type: CollectionDependencyProviderBase
            apis,  # type: MultiGalaxyAPIProxy
            concrete_artifacts_manager=None,  # type: ConcreteArtifactsManager
-            user_requirements=None,  # type: t.Iterable[Requirement]
            preferred_candidates=None,  # type: t.Iterable[Candidate]
            with_deps=True,  # type: bool
            with_pre_releases=False,  # type: bool
@ -87,58 +86,12 @@ class CollectionDependencyProviderBase(AbstractProvider):
            Requirement.from_requirement_dict,
            art_mgr=concrete_artifacts_manager,
        )
-        self._pinned_candidate_requests = set(
-            # NOTE: User-provided signatures are supplemental, so signatures
-            # NOTE: are not used to determine if a candidate is user-requested
-            Candidate(req.fqcn, req.ver, req.src, req.type, None)
-            for req in (user_requirements or ())
-            if req.is_concrete_artifact or (
-                req.ver != '*' and
-                not req.ver.startswith(('<', '>', '!='))
-            )
-        )
        self._preferred_candidates = set(preferred_candidates or ())
        self._with_deps = with_deps
        self._with_pre_releases = with_pre_releases
        self._upgrade = upgrade
        self._include_signatures = include_signatures

-    def _is_user_requested(self, candidate):  # type: (Candidate) -> bool
-        """Check if the candidate is requested by the user."""
-        if candidate in self._pinned_candidate_requests:
-            return True
-
-        if candidate.is_online_index_pointer and candidate.src is not None:
-            # NOTE: Candidate is a namedtuple, it has a source server set
-            # NOTE: to a specific GalaxyAPI instance or `None`. When the
-            # NOTE: user runs
-            # NOTE:
-            # NOTE:     $ ansible-galaxy collection install ns.coll
-            # NOTE:
-            # NOTE: then it's saved in `self._pinned_candidate_requests`
-            # NOTE: as `('ns.coll', '*', None, 'galaxy')` but then
-            # NOTE: `self.find_matches()` calls `self.is_satisfied_by()`
-            # NOTE: with Candidate instances bound to each specific
-            # NOTE: server available, those look like
-            # NOTE: `('ns.coll', '*', GalaxyAPI(...), 'galaxy')` and
-            # NOTE: wouldn't match the user requests saved in
-            # NOTE: `self._pinned_candidate_requests`. This is why we
-            # NOTE: normalize the collection to have `src=None` and try
-            # NOTE: again.
-            # NOTE:
-            # NOTE: When the user request comes from `requirements.yml`
-            # NOTE: with the `source:` set, it'll match the first check
-            # NOTE: but it still can have entries with `src=None` so this
-            # NOTE: normalized check is still necessary.
-            # NOTE:
-            # NOTE: User-provided signatures are supplemental, so signatures
-            # NOTE: are not used to determine if a candidate is user-requested
-            return Candidate(
-                candidate.fqcn, candidate.ver, None, candidate.type, None
-            ) in self._pinned_candidate_requests
-
-        return False
-
    def identify(self, requirement_or_candidate):
        # type: (t.Union[Candidate, Requirement]) -> str
        """Given requirement or candidate, return an identifier for it.
@ -342,25 +295,79 @@ class CollectionDependencyProviderBase(AbstractProvider):
        latest_matches = []
        signatures = []
        extra_signature_sources = []  # type: list[str]
+
+        discarding_pre_releases_acceptable = any(
+            not is_pre_release(candidate_version)
+            for candidate_version, _src_server in coll_versions
+        )
+
+        # NOTE: The optimization of conditionally looping over the requirements
+        # NOTE: is used to skip having to compute the pinned status of all
+        # NOTE: requirements and apply version normalization to the found ones.
+        all_pinned_requirement_version_numbers = {
+            # NOTE: Pinned versions can start with a number, but also with an
+            # NOTE: equals sign. Stripping it at the beginning should be
+            # NOTE: enough. If there's a space after equals, the second strip
+            # NOTE: will take care of it.
+            # NOTE: Without this conversion, requirements versions like
+            # NOTE: '1.2.3-alpha.4' work, but '=1.2.3-alpha.4' don't.
+            requirement.ver.lstrip('=').strip()
+            for requirement in requirements
+            if requirement.is_pinned
+        } if discarding_pre_releases_acceptable else set()
+
        for version, src_server in coll_versions:
            tmp_candidate = Candidate(fqcn, version, src_server, 'galaxy', None)

-            unsatisfied = False
            for requirement in requirements:
-                unsatisfied |= not self.is_satisfied_by(requirement, tmp_candidate)
+                candidate_satisfies_requirement = self.is_satisfied_by(
+                    requirement, tmp_candidate,
+                )
+                if not candidate_satisfies_requirement:
+                    break
+
+                should_disregard_pre_release_candidate = (
+                    # NOTE: Do not discard pre-release candidates in the
+                    # NOTE: following cases:
+                    # NOTE:   * the end-user requested pre-releases explicitly;
+                    # NOTE:   * the candidate is a concrete artifact (e.g. a
+                    # NOTE:     Git repository, subdirs, a tarball URL, or a
+                    # NOTE:     local dir or file etc.);
+                    # NOTE:   * the candidate's pre-release version exactly
+                    # NOTE:     matches a version specifically requested by one
+                    # NOTE:     of the requirements in the current match
+                    # NOTE:     discovery round (i.e. matching a requirement
+                    # NOTE:     that is not a range but an explicit specific
+                    # NOTE:     version pin). This works when some requirements
+                    # NOTE:     request version ranges but others (possibly on
+                    # NOTE:     different dependency tree level depths) demand
+                    # NOTE:     pre-release dependency versions, even if those
+                    # NOTE:     dependencies are transitive.
+                    is_pre_release(tmp_candidate.ver)
+                    and discarding_pre_releases_acceptable
+                    and not (
+                        self._with_pre_releases
+                        or tmp_candidate.is_concrete_artifact
+                        or version in all_pinned_requirement_version_numbers
+                    )
+                )
+                if should_disregard_pre_release_candidate:
+                    break
+
                # FIXME
-                # unsatisfied |= not self.is_satisfied_by(requirement, tmp_candidate) or not (
-                #    requirement.src is None or  # if this is true for some candidates but not all it will break key param - Nonetype can't be compared to str
+                # candidate_is_from_requested_source = (
+                #    requirement.src is None  # if this is true for some candidates but not all it will break key param - Nonetype can't be compared to str
                #    or requirement.src == candidate.src
                # )
-                if unsatisfied:
-                    break
+                # if not candidate_is_from_requested_source:
+                #     break
+
                if not self._include_signatures:
                    continue

                extra_signature_sources.extend(requirement.signature_sources or [])

-            if not unsatisfied:
+            else:  # candidate satisfies requirements, `break` never happened
                if self._include_signatures:
                    for extra_source in extra_signature_sources:
                        signatures.append(get_signature_from_source(extra_source))
@ -405,21 +412,6 @@ class CollectionDependencyProviderBase(AbstractProvider):
        :returns: Indication whether the `candidate` is a viable \
                  solution to the `requirement`.
        """
-        # NOTE: Only allow pre-release candidates if we want pre-releases
-        # NOTE: or the req ver was an exact match with the pre-release
-        # NOTE: version. Another case where we'd want to allow
-        # NOTE: pre-releases is when there are several user requirements
-        # NOTE: and one of them is a pre-release that also matches a
-        # NOTE: transitive dependency of another requirement.
-        allow_pre_release = self._with_pre_releases or not (
-            requirement.ver == '*' or
-            requirement.ver.startswith('<') or
-            requirement.ver.startswith('>') or
-            requirement.ver.startswith('!=')
-        ) or self._is_user_requested(candidate)
-        if is_pre_release(candidate.ver) and not allow_pre_release:
-            return False
-
        # NOTE: This is a set of Pipenv-inspired optimizations. Ref:
        # https://github.com/sarugaku/passa/blob/2ac00f1/src/passa/models/providers.py#L58-L74
        if (
--- a/lib/ansible/galaxy/role.py
+++ b/lib/ansible/galaxy/role.py
@ -42,6 +42,7 @@ from ansible.module_utils.compat.version import LooseVersion
 from ansible.module_utils.urls import open_url
 from ansible.playbook.role.requirement import RoleRequirement
 from ansible.utils.display import Display
+from ansible.utils.path import is_subpath, unfrackpath

 display = Display()

@ -393,37 +394,41 @@ class GalaxyRole(object):
                            # we only extract files, and remove any relative path
                            # bits that might be in the file for security purposes
                            # and drop any containing directory, as mentioned above
-                            if member.isreg() or member.issym():
-                                n_member_name = to_native(member.name)
-                                n_archive_parent_dir = to_native(archive_parent_dir)
-                                n_parts = n_member_name.replace(n_archive_parent_dir, "", 1).split(os.sep)
-                                n_final_parts = []
-                                for n_part in n_parts:
-                                    # TODO if the condition triggers it produces a broken installation.
-                                    # It will create the parent directory as an empty file and will
-                                    # explode if the directory contains valid files.
-                                    # Leaving this as is since the whole module needs a rewrite.
-                                    #
-                                    # Check if we have any files with illegal names,
-                                    # and display a warning if so. This could help users
-                                    # to debug a broken installation.
-                                    if n_part == '..':
-                                        display.warning(f"Illegal filename '{n_part}': '..' is not allowed")
-                                        continue
-                                    if n_part.startswith('~'):
-                                        display.warning(f"Illegal filename '{n_part}': names cannot start with '~'")
-                                        continue
-                                    if '$' in n_part:
-                                        display.warning(f"Illegal filename '{n_part}': names cannot contain '$'")
-                                        continue
-                                    n_final_parts.append(n_part)
-                                member.name = os.path.join(*n_final_parts)
+                            if not (member.isreg() or member.issym()):
+                                continue

-                                if _check_working_data_filter():
-                                    # deprecated: description='extract fallback without filter' python_version='3.11'
-                                    role_tar_file.extract(member, to_native(self.path), filter='data')  # type: ignore[call-arg]
+                            for attr in ('name', 'linkname'):
+                                if not (attr_value := getattr(member, attr, None)):
+                                    continue
+
+                                if attr_value.startswith(os.sep) and not is_subpath(attr_value, archive_parent_dir):
+                                    err = f"Invalid {attr} for tarfile member: path {attr_value} is not a subpath of the role {archive_parent_dir}"
+                                    raise AnsibleError(err)
+
+                                if attr == 'linkname':
+                                    # Symlinks are relative to the link
+                                    relative_to_archive_dir = os.path.dirname(getattr(member, 'name', ''))
+                                    archive_dir_path = os.path.join(archive_parent_dir, relative_to_archive_dir, attr_value)
                                else:
-                                    role_tar_file.extract(member, to_native(self.path))
+                                    # Normalize paths that start with the archive dir
+                                    attr_value = attr_value.replace(archive_parent_dir, "", 1)
+                                    attr_value = os.path.join(*attr_value.split(os.sep))  # remove leading os.sep
+                                    archive_dir_path = os.path.join(archive_parent_dir, attr_value)
+
+                                resolved_archive = unfrackpath(archive_parent_dir)
+                                resolved_path = unfrackpath(archive_dir_path)
+                                if not is_subpath(resolved_path, resolved_archive):
+                                    err = f"Invalid {attr} for tarfile member: path {resolved_path} is not a subpath of the role {resolved_archive}"
+                                    raise AnsibleError(err)
+
+                                relative_path = os.path.join(*resolved_path.replace(resolved_archive, "", 1).split(os.sep)) or '.'
+                                setattr(member, attr, relative_path)
+
+                            if _check_working_data_filter():
+                                # deprecated: description='extract fallback without filter' python_version='3.11'
+                                role_tar_file.extract(member, to_native(self.path), filter='data')  # type: ignore[call-arg]
+                            else:
+                                role_tar_file.extract(member, to_native(self.path))

                        # write out the install info file for later use
                        self._write_galaxy_install_info()
--- a/lib/ansible/inventory/group.py
+++ b/lib/ansible/inventory/group.py
@ -18,6 +18,7 @@ from __future__ import (absolute_import, division, print_function)
 __metaclass__ = type

 from collections.abc import Mapping, MutableMapping
+from enum import Enum
 from itertools import chain

 from ansible import constants as C
@ -53,8 +54,14 @@ def to_safe_group_name(name, replacer="_", force=False, silent=False):
    return name


+class InventoryObjectType(Enum):
+    HOST = 0
+    GROUP = 1
+
+
 class Group:
    ''' a group of ansible hosts '''
+    base_type = InventoryObjectType.GROUP

    # __slots__ = [ 'name', 'hosts', 'vars', 'child_groups', 'parent_groups', 'depth', '_hosts_cache' ]

--- a/lib/ansible/inventory/host.py
+++ b/lib/ansible/inventory/host.py
@ -21,7 +21,7 @@ __metaclass__ = type

 from collections.abc import Mapping, MutableMapping

-from ansible.inventory.group import Group
+from ansible.inventory.group import Group, InventoryObjectType
 from ansible.parsing.utils.addresses import patterns
 from ansible.utils.vars import combine_vars, get_unique_id

@ -31,6 +31,7 @@ __all__ = ['Host']

 class Host:
    ''' a single ansible host '''
+    base_type = InventoryObjectType.HOST

    # __slots__ = [ 'name', 'vars', 'groups' ]

--- a/lib/ansible/keyword_desc.yml
+++ b/lib/ansible/keyword_desc.yml
@ -5,7 +5,7 @@ action: "The 'action' to execute for a task, it normally translates into a C(mod
 args: "A secondary way to add arguments into a task. Takes a dictionary in which keys map to options and values."
 always: List of tasks, in a block, that execute no matter if there is an error in the block or not.
 any_errors_fatal: Force any un-handled task errors on any host to propagate to all hosts and end the play.
-async: Run a task asynchronously if the C(action) supports this; value is maximum runtime in seconds.
+async: Run a task asynchronously if the C(action) supports this; the value is the maximum runtime in seconds.
 become: Boolean that controls if privilege escalation is used or not on :term:`Task` execution. Implemented by the become plugin. See :ref:`become_plugins`.
 become_exe: Path to the executable used to elevate privileges. Implemented by the become plugin. See :ref:`become_plugins`.
 become_flags: A string of flag(s) to pass to the privilege escalation program when :term:`become` is True.
@ -23,25 +23,25 @@ collections: |


 connection: Allows you to change the connection plugin used for tasks to execute on the target. See :ref:`using_connection`.
-debugger: Enable debugging tasks based on state of the task result. See :ref:`playbook_debugger`.
+debugger: Enable debugging tasks based on the state of the task result. See :ref:`playbook_debugger`.
 delay: Number of seconds to delay between retries. This setting is only used in combination with :term:`until`.
 delegate_facts: Boolean that allows you to apply facts to a delegated host instead of inventory_hostname.
 delegate_to: Host to execute task instead of the target (inventory_hostname). Connection vars from the delegated host will also be used for the task.
 diff: "Toggle to make tasks return 'diff' information or not."
-environment: A dictionary that gets converted into environment vars to be provided for the task upon execution. This can ONLY be used with modules. This isn't supported for any other type of plugins nor Ansible itself nor its configuration, it just sets the variables for the code responsible for executing the task. This is not a recommended way to pass in confidential data.
+environment: A dictionary that gets converted into environment vars to be provided for the task upon execution. This can ONLY be used with modules. This is not supported for any other type of plugins nor Ansible itself nor its configuration, it just sets the variables for the code responsible for executing the task. This is not a recommended way to pass in confidential data.
 fact_path: Set the fact path option for the fact gathering plugin controlled by :term:`gather_facts`.
 failed_when: "Conditional expression that overrides the task's normal 'failed' status."
 force_handlers: Will force notified handler execution for hosts even if they failed during the play. Will not trigger if the play itself fails.
 gather_facts: "A boolean that controls if the play will automatically run the 'setup' task to gather facts for the hosts."
-gather_subset: Allows you to pass subset options to the  fact gathering plugin controlled by :term:`gather_facts`.
+gather_subset: Allows you to pass subset options to the fact gathering plugin controlled by :term:`gather_facts`.
 gather_timeout: Allows you to set the timeout for the fact gathering plugin controlled by :term:`gather_facts`.
 handlers: "A section with tasks that are treated as handlers, these won't get executed normally, only when notified after each section of tasks is complete. A handler's `listen` field is not templatable."
 hosts: "A list of groups, hosts or host pattern that translates into a list of hosts that are the play's target."
 ignore_errors: Boolean that allows you to ignore task failures and continue with play. It does not affect connection errors.
 ignore_unreachable: Boolean that allows you to ignore task failures due to an unreachable host and continue with the play. This does not affect other task errors (see :term:`ignore_errors`) but is useful for groups of volatile/ephemeral hosts.
 loop: "Takes a list for the task to iterate over, saving each list element into the ``item`` variable (configurable via loop_control)"
-loop_control: Several keys here allow you to modify/set loop behaviour in a task. See :ref:`loop_control`.
-max_fail_percentage: can be used to abort the run after a given percentage of hosts in the current batch has failed. This only works on linear or linear derived strategies.
+loop_control: Several keys here allow you to modify/set loop behavior in a task. See :ref:`loop_control`.
+max_fail_percentage: can be used to abort the run after a given percentage of hosts in the current batch has failed. This only works on linear or linear-derived strategies.
 module_defaults: Specifies default parameter values for modules.
 name: "Identifier. Can be used for documentation, or in tasks/handlers."
 no_log: Boolean that controls information disclosure.
@ -56,13 +56,13 @@ register: Name of variable that will contain task status and module return data.
 rescue: List of tasks in a :term:`block` that run if there is a task error in the main :term:`block` list.
 retries: "Number of retries before giving up in a :term:`until` loop. This setting is only used in combination with :term:`until`."
 roles: List of roles to be imported into the play
-run_once: Boolean that will bypass the host loop, forcing the task to attempt to execute on the first host available and afterwards apply any results and facts to all active hosts in the same batch.
+run_once: Boolean that will bypass the host loop, forcing the task to attempt to execute on the first host available and afterward apply any results and facts to all active hosts in the same batch.
 serial: Explicitly define how Ansible batches the execution of the current play on the play's target. See :ref:`rolling_update_batch_size`.
-strategy: Allows you to choose the connection plugin to use for the play.
+strategy: Allows you to choose the strategy plugin to use for the play. See :ref:`strategy_plugins`.
 tags: Tags applied to the task or included tasks, this allows selecting subsets of tasks from the command line.
 tasks: Main list of tasks to execute in the play, they run after :term:`roles` and before :term:`post_tasks`.
-timeout: Time limit for task to execute in, if exceeded Ansible will interrupt and fail the task.
-throttle: Limit number of concurrent task runs on task, block and playbook level. This is independent of the forks and serial settings, but cannot be set higher than those limits. For example, if forks is set to 10 and the throttle is set to 15, at most 10 hosts will be operated on in parallel.
+timeout: Time limit for the task to execute in, if exceeded Ansible will interrupt and fail the task.
+throttle: Limit the number of concurrent task runs on task, block and playbook level. This is independent of the forks and serial settings, but cannot be set higher than those limits. For example, if forks is set to 10 and the throttle is set to 15, at most 10 hosts will be operated on in parallel.
 until: "This keyword implies a ':term:`retries` loop' that will go on until the condition supplied here is met or we hit the :term:`retries` limit."
 vars: Dictionary/map of variables
 vars_files: List of files that contain vars to include in the play.
--- a/lib/ansible/module_utils/basic.py
+++ b/lib/ansible/module_utils/basic.py
@ -1489,7 +1489,19 @@ class AnsibleModule(object):
        if deprecations:
            kwargs['deprecations'] = deprecations

+        # preserve bools/none from no_log
+        # TODO: once python version on target high enough, dict comprh
+        preserved = {}
+        for k, v in kwargs.items():
+            if v is None or isinstance(v, bool):
+                preserved[k] = v
+
+        # strip no_log collisions
        kwargs = remove_values(kwargs, self.no_log_values)
+
+        # return preserved
+        kwargs.update(preserved)
+
        print('\n%s' % self.jsonify(kwargs))

    def exit_json(self, **kwargs):
--- a/lib/ansible/module_utils/common/json.py
+++ b/lib/ansible/module_utils/common/json.py
@ -30,7 +30,7 @@ def _preprocess_unsafe_encode(value):
    Used in ``AnsibleJSONEncoder.iterencode``
    """
    if _is_unsafe(value):
-        value = {'__ansible_unsafe': to_text(value, errors='surrogate_or_strict', nonstring='strict')}
+        value = {'__ansible_unsafe': to_text(value._strip_unsafe(), errors='surrogate_or_strict', nonstring='strict')}
    elif is_sequence(value):
        value = [_preprocess_unsafe_encode(v) for v in value]
    elif isinstance(value, Mapping):
@ -63,7 +63,7 @@ class AnsibleJSONEncoder(json.JSONEncoder):
                value = {'__ansible_vault': to_text(o._ciphertext, errors='surrogate_or_strict', nonstring='strict')}
        elif getattr(o, '__UNSAFE__', False):
            # unsafe object, this will never be triggered, see ``AnsibleJSONEncoder.iterencode``
-            value = {'__ansible_unsafe': to_text(o, errors='surrogate_or_strict', nonstring='strict')}
+            value = {'__ansible_unsafe': to_text(o._strip_unsafe(), errors='surrogate_or_strict', nonstring='strict')}
        elif isinstance(o, Mapping):
            # hostvars and other objects
            value = dict(o)
--- a/lib/ansible/module_utils/service.py
+++ b/lib/ansible/module_utils/service.py
@ -215,9 +215,10 @@ def daemonize(module, cmd):
                for out in list(fds):
                    if out in rfd:
                        data = os.read(out.fileno(), chunk)
-                        if not data:
+                        if data:
+                            output[out] += to_bytes(data, errors=errors)
+                        else:
                            fds.remove(out)
-                        output[out] += data
            else:
                break

@ -248,7 +249,7 @@ def daemonize(module, cmd):
                data = os.read(pipe[0], chunk)
                if not data:
                    break
-                return_data += data
+                return_data += to_bytes(data, errors=errors)

        # Note: no need to specify encoding on py3 as this module sends the
        # pickle to itself (thus same python interpreter so we aren't mixing
--- a/lib/ansible/module_utils/urls.py
+++ b/lib/ansible/module_utils/urls.py
@ -129,13 +129,13 @@ if not HAS_SSLCONTEXT:
        try:
            from urllib3.contrib.pyopenssl import PyOpenSSLContext
        except Exception:
-            from requests.packages.urllib3.contrib.pyopenssl import PyOpenSSLContext
+            from requests.packages.urllib3.contrib.pyopenssl import PyOpenSSLContext  # type: ignore[no-redef]
        HAS_URLLIB3_PYOPENSSLCONTEXT = True
    except Exception:
        # urllib3<1.15,>=1.6
        try:
            try:
-                from urllib3.contrib.pyopenssl import ssl_wrap_socket
+                from urllib3.contrib.pyopenssl import ssl_wrap_socket  # type: ignore[attr-defined]
            except Exception:
                from requests.packages.urllib3.contrib.pyopenssl import ssl_wrap_socket
            HAS_URLLIB3_SSL_WRAP_SOCKET = True
@ -771,6 +771,18 @@ def extract_pem_certs(b_data):
        yield match.group(0)


+def _py2_get_param(headers, param, header='content-type'):
+    m = httplib.HTTPMessage(io.StringIO())
+    cd = headers.getheader(header) or ''
+    try:
+        m.plisttext = cd[cd.index(';'):]
+        m.parseplist()
+    except ValueError:
+        return None
+
+    return m.getparam(param)
+
+
 def get_response_filename(response):
    url = response.geturl()
    path = urlparse(url)[2]
@ -778,7 +790,12 @@ def get_response_filename(response):
    if filename:
        filename = unquote(filename)

-    return response.headers.get_param('filename', header='content-disposition') or filename
+    if PY2:
+        get_param = functools.partial(_py2_get_param, response.headers)
+    else:
+        get_param = response.headers.get_param
+
+    return get_param('filename', header='content-disposition') or filename


 def parse_content_type(response):
--- a/lib/ansible/modules/import_playbook.py
+++ b/lib/ansible/modules/import_playbook.py
@ -41,7 +41,7 @@ seealso:
 - module: ansible.builtin.import_tasks
 - module: ansible.builtin.include_role
 - module: ansible.builtin.include_tasks
- ref: playbooks_reuse_includes
+- ref: playbooks_reuse
  description: More information related to including and importing playbooks, roles and tasks.
 '''

--- a/lib/ansible/modules/import_role.py
+++ b/lib/ansible/modules/import_role.py
@ -78,7 +78,7 @@ seealso:
 - module: ansible.builtin.import_tasks
 - module: ansible.builtin.include_role
 - module: ansible.builtin.include_tasks
- ref: playbooks_reuse_includes
+- ref: playbooks_reuse
  description: More information related to including and importing playbooks, roles and tasks.
 '''

--- a/lib/ansible/modules/import_tasks.py
+++ b/lib/ansible/modules/import_tasks.py
@ -45,7 +45,7 @@ seealso:
 - module: ansible.builtin.import_role
 - module: ansible.builtin.include_role
 - module: ansible.builtin.include_tasks
- ref: playbooks_reuse_includes
+- ref: playbooks_reuse
  description: More information related to including and importing playbooks, roles and tasks.
 '''

--- a/lib/ansible/modules/include_role.py
+++ b/lib/ansible/modules/include_role.py
@ -91,7 +91,7 @@ seealso:
 - module: ansible.builtin.import_role
 - module: ansible.builtin.import_tasks
 - module: ansible.builtin.include_tasks
- ref: playbooks_reuse_includes
+- ref: playbooks_reuse
  description: More information related to including and importing playbooks, roles and tasks.
 '''

--- a/lib/ansible/modules/include_tasks.py
+++ b/lib/ansible/modules/include_tasks.py
@ -49,7 +49,7 @@ seealso:
 - module: ansible.builtin.import_role
 - module: ansible.builtin.import_tasks
 - module: ansible.builtin.include_role
- ref: playbooks_reuse_includes
+- ref: playbooks_reuse
  description: More information related to including and importing playbooks, roles and tasks.
 '''

--- a/lib/ansible/modules/pip.py
+++ b/lib/ansible/modules/pip.py
@ -121,6 +121,8 @@ attributes:
    platform:
        platforms: posix
 notes:
+   - Python installations marked externally-managed (as defined by PEP668) cannot be updated by pip versions >= 23.0.1 without the use of
+     a virtual environment or setting the environment variable ``PIP_BREAK_SYSTEM_PACKAGES=1``.
   - The virtualenv (U(http://www.virtualenv.org/)) must be
     installed on the remote host if the virtualenv parameter is specified and
     the virtualenv needs to be created.
--- a/lib/ansible/modules/wait_for.py
+++ b/lib/ansible/modules/wait_for.py
@ -238,7 +238,7 @@ import traceback

 from ansible.module_utils.basic import AnsibleModule, missing_required_lib
 from ansible.module_utils.common.sys_info import get_platform_subclass
-from ansible.module_utils.common.text.converters import to_bytes
+from ansible.module_utils.common.text.converters import to_bytes, to_native
 from ansible.module_utils.compat.datetime import utcnow


@ -585,17 +585,30 @@ def main():
                    if not b_compiled_search_re:
                        # nope, succeed!
                        break
+
                    try:
                        with open(b_path, 'rb') as f:
-                            with contextlib.closing(mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)) as mm:
-                                search = b_compiled_search_re.search(mm)
+                            try:
+                                with contextlib.closing(mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)) as mm:
+                                    search = b_compiled_search_re.search(mm)
+                                    if search:
+                                        if search.groupdict():
+                                            match_groupdict = search.groupdict()
+                                        if search.groups():
+                                            match_groups = search.groups()
+                                        break
+                            except (ValueError, OSError) as e:
+                                module.debug('wait_for failed to use mmap on "%s": %s. Falling back to file read().' % (path, to_native(e)))
+                                # cannot mmap this file, try normal read
+                                search = re.search(b_compiled_search_re, f.read())
                                if search:
                                    if search.groupdict():
                                        match_groupdict = search.groupdict()
                                    if search.groups():
                                        match_groups = search.groups()
-
                                    break
+                            except Exception as e:
+                                module.warn('wait_for failed on "%s", unexpected exception(%s): %s.).' % (path, to_native(e.__class__), to_native(e)))
                    except IOError:
                        pass
            elif port:
--- a/lib/ansible/parsing/yaml/dumper.py
+++ b/lib/ansible/parsing/yaml/dumper.py
@ -24,7 +24,7 @@ import yaml
 from ansible.module_utils.six import text_type, binary_type
 from ansible.module_utils.common.yaml import SafeDumper
 from ansible.parsing.yaml.objects import AnsibleUnicode, AnsibleSequence, AnsibleMapping, AnsibleVaultEncryptedUnicode
-from ansible.utils.unsafe_proxy import AnsibleUnsafeText, AnsibleUnsafeBytes, NativeJinjaUnsafeText, NativeJinjaText
+from ansible.utils.unsafe_proxy import AnsibleUnsafeText, AnsibleUnsafeBytes, NativeJinjaUnsafeText, NativeJinjaText, _is_unsafe
 from ansible.template import AnsibleUndefined
 from ansible.vars.hostvars import HostVars, HostVarsVars
 from ansible.vars.manager import VarsWithSources
@ -47,10 +47,14 @@ def represent_vault_encrypted_unicode(self, data):


 def represent_unicode(self, data):
+    if _is_unsafe(data):
+        data = data._strip_unsafe()
    return yaml.representer.SafeRepresenter.represent_str(self, text_type(data))


 def represent_binary(self, data):
+    if _is_unsafe(data):
+        data = data._strip_unsafe()
    return yaml.representer.SafeRepresenter.represent_binary(self, binary_type(data))


--- a/lib/ansible/playbook/base.py
+++ b/lib/ansible/playbook/base.py
@ -731,7 +731,7 @@ class Base(FieldAttributeBase):

    # flags and misc. settings
    environment = FieldAttribute(isa='list', extend=True, prepend=True)
-    no_log = FieldAttribute(isa='bool')
+    no_log = FieldAttribute(isa='bool', default=C.DEFAULT_NO_LOG)
    run_once = FieldAttribute(isa='bool')
    ignore_errors = FieldAttribute(isa='bool')
    ignore_unreachable = FieldAttribute(isa='bool')
--- a/lib/ansible/playbook/conditional.py
+++ b/lib/ansible/playbook/conditional.py
@ -21,7 +21,7 @@ __metaclass__ = type

 import typing as t

-from ansible.errors import AnsibleError, AnsibleUndefinedVariable
+from ansible.errors import AnsibleError, AnsibleUndefinedVariable, AnsibleTemplateError
 from ansible.module_utils.common.text.converters import to_native
 from ansible.playbook.attribute import FieldAttribute
 from ansible.template import Templar
@ -102,14 +102,14 @@ class Conditional:
                    return False

            # If the result of the first-pass template render (to resolve inline templates) is marked unsafe,
-            # explicitly disable lookups on the final pass to prevent evaluation of untrusted content in the
-            # constructed template.
-            disable_lookups = hasattr(conditional, '__UNSAFE__')
+            # explicitly fail since the next templating operation would never evaluate
+            if hasattr(conditional, '__UNSAFE__'):
+                raise AnsibleTemplateError('Conditional is marked as unsafe, and cannot be evaluated.')

            # NOTE The spaces around True and False are intentional to short-circuit literal_eval for
            #      jinja2_native=False and avoid its expensive calls.
            return templar.template(
                "{%% if %s %%} True {%% else %%} False {%% endif %%}" % conditional,
-                disable_lookups=disable_lookups).strip() == "True"
+            ).strip() == "True"
        except AnsibleUndefinedVariable as e:
            raise AnsibleUndefinedVariable("error while evaluating conditional (%s): %s" % (original, e))
--- a/lib/ansible/playbook/handler.py
+++ b/lib/ansible/playbook/handler.py
@ -53,6 +53,9 @@ class Handler(Task):
    def remove_host(self, host):
        self.notified_hosts = [h for h in self.notified_hosts if h != host]

+    def clear_hosts(self):
+        self.notified_hosts = []
+
    def is_host_notified(self, host):
        return host in self.notified_hosts

--- a/lib/ansible/playbook/included_file.py
+++ b/lib/ansible/playbook/included_file.py
@ -146,9 +146,12 @@ class IncludedFile:
                                    cumulative_path = parent_include_dir
                                include_target = templar.template(include_result['include'])
                                if original_task._role:
-                                    new_basedir = os.path.join(original_task._role._role_path, 'tasks', cumulative_path)
-                                    candidates = [loader.path_dwim_relative(original_task._role._role_path, 'tasks', include_target),
-                                                  loader.path_dwim_relative(new_basedir, 'tasks', include_target)]
+                                    dirname = 'handlers' if isinstance(original_task, Handler) else 'tasks'
+                                    new_basedir = os.path.join(original_task._role._role_path, dirname, cumulative_path)
+                                    candidates = [
+                                        loader.path_dwim_relative(original_task._role._role_path, dirname, include_target, is_role=True),
+                                        loader.path_dwim_relative(new_basedir, dirname, include_target, is_role=True)
+                                    ]
                                    for include_file in candidates:
                                        try:
                                            # may throw OSError
--- a/lib/ansible/playbook/play_context.py
+++ b/lib/ansible/playbook/play_context.py
@ -318,10 +318,6 @@ class PlayContext(Base):
            display.warning('The "%s" connection plugin has an improperly configured remote target value, '
                            'forcing "inventory_hostname" templated value instead of the string' % new_info.connection)

-        # set no_log to default if it was not previously set
-        if new_info.no_log is None:
-            new_info.no_log = C.DEFAULT_NO_LOG
-
        if task.check_mode is not None:
            new_info.check_mode = task.check_mode

--- a/lib/ansible/playbook/role/init.py
+++ b/lib/ansible/playbook/role/init.py
@ -100,19 +100,30 @@ def hash_params(params):

 class Role(Base, Conditional, Taggable, CollectionSearch, Delegatable):

-    def __init__(self, play=None, from_files=None, from_include=False, validate=True, public=True):
+    def __init__(self, play=None, from_files=None, from_include=False, validate=True, public=None, static=True):
        self._role_name = None
        self._role_path = None
        self._role_collection = None
        self._role_params = dict()
        self._loader = None
-        self.public = public
-        self.static = True
+        self.static = static
+
+        # includes (static=false) default to private, while imports (static=true) default to public
+        # but both can be overriden by global config if set
+        if public is None:
+            global_private, origin = C.config.get_config_value_and_origin('DEFAULT_PRIVATE_ROLE_VARS')
+            if origin == 'default':
+                self.public = static
+            else:
+                self.public = not global_private
+        else:
+            self.public = public

        self._metadata = RoleMetadata()
        self._play = play
        self._parents = []
        self._dependencies = []
+        self._all_dependencies = None
        self._task_blocks = []
        self._handler_blocks = []
        self._compiled_handler_blocks = None
@ -169,7 +180,7 @@ class Role(Base, Conditional, Taggable, CollectionSearch, Delegatable):
        return self._get_hash_dict() == other._get_hash_dict()

    @staticmethod
-    def load(role_include, play, parent_role=None, from_files=None, from_include=False, validate=True, public=True):
+    def load(role_include, play, parent_role=None, from_files=None, from_include=False, validate=True, public=None, static=True):
        if from_files is None:
            from_files = {}
        try:
@ -177,7 +188,7 @@ class Role(Base, Conditional, Taggable, CollectionSearch, Delegatable):
            #  for the in-flight in role cache as a sentinel that we're already trying to load
            #  that role?)
            # see https://github.com/ansible/ansible/issues/61527
-            r = Role(play=play, from_files=from_files, from_include=from_include, validate=validate, public=public)
+            r = Role(play=play, from_files=from_files, from_include=from_include, validate=validate, public=public, static=static)
            r._load_role_data(role_include, parent_role=parent_role)

            role_path = r.get_role_path()
@ -428,7 +439,7 @@ class Role(Base, Conditional, Taggable, CollectionSearch, Delegatable):

        deps = []
        for role_include in self._metadata.dependencies:
-            r = Role.load(role_include, play=self._play, parent_role=self)
+            r = Role.load(role_include, play=self._play, parent_role=self, static=self.static)
            deps.append(r)

        return deps
@ -528,15 +539,15 @@ class Role(Base, Conditional, Taggable, CollectionSearch, Delegatable):
        Returns a list of all deps, built recursively from all child dependencies,
        in the proper order in which they should be executed or evaluated.
        '''
+        if self._all_dependencies is None:

-        child_deps = []
+            self._all_dependencies = []
+            for dep in self.get_direct_dependencies():
+                for child_dep in dep.get_all_dependencies():
+                    self._all_dependencies.append(child_dep)
+                self._all_dependencies.append(dep)

-        for dep in self.get_direct_dependencies():
-            for child_dep in dep.get_all_dependencies():
-                child_deps.append(child_dep)
-            child_deps.append(dep)
-
-        return child_deps
+        return self._all_dependencies

    def get_task_blocks(self):
        return self._task_blocks[:]
--- a/lib/ansible/playbook/role_include.py
+++ b/lib/ansible/playbook/role_include.py
@ -49,10 +49,10 @@ class IncludeRole(TaskInclude):

    # =================================================================================
    # ATTRIBUTES
+    public = NonInheritableFieldAttribute(isa='bool', default=None, private=False, always_post_validate=True)

    # private as this is a 'module options' vs a task property
    allow_duplicates = NonInheritableFieldAttribute(isa='bool', default=True, private=True, always_post_validate=True)
-    public = NonInheritableFieldAttribute(isa='bool', default=False, private=True, always_post_validate=True)
    rolespec_validate = NonInheritableFieldAttribute(isa='bool', default=True, private=True, always_post_validate=True)

    def __init__(self, block=None, role=None, task_include=None):
@ -88,11 +88,11 @@ class IncludeRole(TaskInclude):

        # build role
        actual_role = Role.load(ri, myplay, parent_role=self._parent_role, from_files=from_files,
-                                from_include=True, validate=self.rolespec_validate, public=self.public)
+                                from_include=True, validate=self.rolespec_validate, public=self.public, static=self.statically_loaded)
        actual_role._metadata.allow_duplicates = self.allow_duplicates

-        if self.statically_loaded or self.public:
-            myplay.roles.append(actual_role)
+        # add role to play
+        myplay.roles.append(actual_role)

        # save this for later use
        self._role_path = actual_role._role_path
@ -124,10 +124,6 @@ class IncludeRole(TaskInclude):

        ir = IncludeRole(block, role, task_include=task_include).load_data(data, variable_manager=variable_manager, loader=loader)

-        # dyanmic role!
-        if ir.action in C._ACTION_INCLUDE_ROLE:
-            ir.static = False
-
        # Validate options
        my_arg_names = frozenset(ir.args.keys())

--- a/lib/ansible/playbook/taggable.py
+++ b/lib/ansible/playbook/taggable.py
@ -23,6 +23,17 @@ from ansible.errors import AnsibleError
 from ansible.module_utils.six import string_types
 from ansible.playbook.attribute import FieldAttribute
 from ansible.template import Templar
+from ansible.utils.sentinel import Sentinel
+
+
+def _flatten_tags(tags: list) -> list:
+    rv = set()
+    for tag in tags:
+        if isinstance(tag, list):
+            rv.update(tag)
+        else:
+            rv.add(tag)
+    return list(rv)


 class Taggable:
@ -34,11 +45,7 @@ class Taggable:
        if isinstance(ds, list):
            return ds
        elif isinstance(ds, string_types):
-            value = ds.split(',')
-            if isinstance(value, list):
-                return [x.strip() for x in value]
-            else:
-                return [ds]
+            return [x.strip() for x in ds.split(',')]
        else:
            raise AnsibleError('tags must be specified as a list', obj=ds)

@ -47,16 +54,12 @@ class Taggable:

        if self.tags:
            templar = Templar(loader=self._loader, variables=all_vars)
-            tags = templar.template(self.tags)
-
-            _temp_tags = set()
-            for tag in tags:
-                if isinstance(tag, list):
-                    _temp_tags.update(tag)
-                else:
-                    _temp_tags.add(tag)
-            tags = _temp_tags
-            self.tags = list(tags)
+            obj = self
+            while obj is not None:
+                if (_tags := getattr(obj, "_tags", Sentinel)) is not Sentinel:
+                    obj._tags = _flatten_tags(templar.template(_tags))
+                obj = obj._parent
+            tags = set(self.tags)
        else:
            # this makes isdisjoint work for untagged
            tags = self.untagged
--- a/lib/ansible/playbook/task.py
+++ b/lib/ansible/playbook/task.py
@ -289,6 +289,30 @@ class Task(Base, Conditional, Taggable, CollectionSearch, Notifiable, Delegatabl

        super(Task, self).post_validate(templar)

+    def _post_validate_args(self, attr, value, templar):
+        # smuggle an untemplated copy of the task args for actions that need more control over the templating of their
+        # input (eg, debug's var/msg, assert's "that" conditional expressions)
+        self.untemplated_args = value
+
+        # now recursively template the args dict
+        args = templar.template(value)
+
+        # FIXME: could we just nuke this entirely and/or wrap it up in ModuleArgsParser or something?
+        if '_variable_params' in args:
+            variable_params = args.pop('_variable_params')
+            if isinstance(variable_params, dict):
+                if C.INJECT_FACTS_AS_VARS:
+                    display.warning("Using a variable for a task's 'args' is unsafe in some situations "
+                                    "(see https://docs.ansible.com/ansible/devel/reference_appendices/faq.html#argsplat-unsafe)")
+                variable_params.update(args)
+                args = variable_params
+            else:
+                # if we didn't get a dict, it means there's garbage remaining after k=v parsing, just give up
+                # see https://github.com/ansible/ansible/issues/79862
+                raise AnsibleError(f"invalid or malformed argument: '{variable_params}'")
+
+        return args
+
    def _post_validate_loop(self, attr, value, templar):
        '''
        Override post validation for the loop field, which is templated
--- a/lib/ansible/plugins/action/assert.py
+++ b/lib/ansible/plugins/action/assert.py
@ -64,8 +64,29 @@ class ActionModule(ActionBase):

        quiet = boolean(self._task.args.get('quiet', False), strict=False)

+        # directly access 'that' via untemplated args from the task so we can intelligently trust embedded
+        # templates and preserve the original inputs/locations for better messaging on assert failures and
+        # errors.
+        # FIXME: even in devel, things like `that: item` don't always work properly (truthy string value
+        # is not really an embedded expression)
+        # we could fix that by doing direct var lookups on the inputs
+        # FIXME: some form of this code should probably be shared between debug, assert, and
+        # Task.post_validate, since they
+        # have a lot of overlapping needs
+        try:
+            thats = self._task.untemplated_args['that']
+        except KeyError:
+            # in the case of "we got our entire args dict from a template", we can just consult the
+            # post-templated dict (the damage has likely already been done for embedded templates anyway)
+            thats = self._task.args['that']
+
+        # FIXME: this is a case where we only want to resolve indirections, NOT recurse containers
+        # (and even then, the leaf-most expression being wrapped is at least suboptimal
+        # (since its expression will be "eaten").
+        if isinstance(thats, str):
+            thats = self._templar.template(thats)
+
        # make sure the 'that' items are a list
-        thats = self._task.args['that']
        if not isinstance(thats, list):
            thats = [thats]

--- a/lib/ansible/plugins/callback/init.py
+++ b/lib/ansible/plugins/callback/init.py
@ -38,7 +38,7 @@ from ansible.parsing.yaml.objects import AnsibleUnicode
 from ansible.plugins import AnsiblePlugin
 from ansible.utils.color import stringc
 from ansible.utils.display import Display
-from ansible.utils.unsafe_proxy import AnsibleUnsafeText, NativeJinjaUnsafeText
+from ansible.utils.unsafe_proxy import AnsibleUnsafeText, NativeJinjaUnsafeText, _is_unsafe
 from ansible.vars.clean import strip_internal_keys, module_response_deepcopy

 import yaml
@ -113,6 +113,8 @@ def _munge_data_for_lossy_yaml(scalar):

 def _pretty_represent_str(self, data):
    """Uses block style for multi-line strings"""
+    if _is_unsafe(data):
+        data = data._strip_unsafe()
    data = text_type(data)
    if _should_use_block(data):
        style = '|'
--- a/lib/ansible/plugins/connection/winrm.py
+++ b/lib/ansible/plugins/connection/winrm.py
@ -170,6 +170,7 @@ import json
 import tempfile
 import shlex
 import subprocess
+import time
 import typing as t

 from inspect import getfullargspec
@ -199,6 +200,7 @@ from ansible.utils.display import Display
 try:
    import winrm
    from winrm import Response
+    from winrm.exceptions import WinRMError, WinRMOperationTimeoutError
    from winrm.protocol import Protocol
    import requests.exceptions
    HAS_WINRM = True
@ -494,6 +496,43 @@ class Connection(ConnectionBase):
        else:
            raise AnsibleError('No transport found for WinRM connection')

+    def _winrm_write_stdin(self, command_id: str, stdin_iterator: t.Iterable[tuple[bytes, bool]]) -> None:
+        for (data, is_last) in stdin_iterator:
+            for attempt in range(1, 4):
+                try:
+                    self._winrm_send_input(self.protocol, self.shell_id, command_id, data, eof=is_last)
+
+                except WinRMOperationTimeoutError:
+                    # A WSMan OperationTimeout can be received for a Send
+                    # operation when the server is under severe load. On manual
+                    # testing the input is still processed and it's safe to
+                    # continue. As the calling method still tries to wait for
+                    # the proc to end if this failed it shouldn't hurt to just
+                    # treat this as a warning.
+                    display.warning(
+                        "WSMan OperationTimeout during send input, attempting to continue. "
+                        "If this continues to occur, try increasing the connection_timeout "
+                        "value for this host."
+                    )
+                    if not is_last:
+                        time.sleep(5)
+
+                except WinRMError as e:
+                    # Error 170 == ERROR_BUSY. This could be the result of a
+                    # timed out Send from above still being processed on the
+                    # server. Add a 5 second delay and try up to 3 times before
+                    # fully giving up.
+                    # pywinrm does not expose the internal WSMan fault details
+                    # through an actual object but embeds it as a repr.
+                    if attempt == 3 or "'wsmanfault_code': '170'" not in str(e):
+                        raise
+
+                    display.warning(f"WSMan send failed on attempt {attempt} as the command is busy, trying to send data again")
+                    time.sleep(5)
+                    continue
+
+                break
+
    def _winrm_send_input(self, protocol: winrm.Protocol, shell_id: str, command_id: str, stdin: bytes, eof: bool = False) -> None:
        rq = {'env:Envelope': protocol._get_soap_header(
            resource_uri='http://schemas.microsoft.com/wbem/wsman/1/windows/shell/cmd',
@ -529,8 +568,7 @@ class Connection(ConnectionBase):

            try:
                if stdin_iterator:
-                    for (data, is_last) in stdin_iterator:
-                        self._winrm_send_input(self.protocol, self.shell_id, command_id, data, eof=is_last)
+                    self._winrm_write_stdin(command_id, stdin_iterator)

            except Exception as ex:
                display.warning("ERROR DURING WINRM SEND INPUT - attempting to recover: %s %s"
--- a/lib/ansible/plugins/filter/core.py
+++ b/lib/ansible/plugins/filter/core.py
@ -37,6 +37,7 @@ from ansible.utils.display import Display
 from ansible.utils.encrypt import do_encrypt, PASSLIB_AVAILABLE
 from ansible.utils.hashing import md5s, checksum_s
 from ansible.utils.unicode import unicode_wrap
+from ansible.utils.unsafe_proxy import _is_unsafe
 from ansible.utils.vars import merge_hash

 display = Display()
@ -215,6 +216,8 @@ def from_yaml(data):
        # The ``text_type`` call here strips any custom
        # string wrapper class, so that CSafeLoader can
        # read the data
+        if _is_unsafe(data):
+            data = data._strip_unsafe()
        return yaml_load(text_type(to_text(data, errors='surrogate_or_strict')))
    return data

@ -224,6 +227,8 @@ def from_yaml_all(data):
        # The ``text_type`` call here strips any custom
        # string wrapper class, so that CSafeLoader can
        # read the data
+        if _is_unsafe(data):
+            data = data._strip_unsafe()
        return yaml_load_all(text_type(to_text(data, errors='surrogate_or_strict')))
    return data

--- a/lib/ansible/plugins/loader.py
+++ b/lib/ansible/plugins/loader.py
@ -238,6 +238,10 @@ class PluginLoader:
        self._module_cache = MODULE_CACHE[class_name]
        self._paths = PATH_CACHE[class_name]
        self._plugin_path_cache = PLUGIN_PATH_CACHE[class_name]
+        try:
+            self._plugin_instance_cache = {} if self.type == 'vars' else None
+        except ValueError:
+            self._plugin_instance_cache = None

        self._searched_paths = set()

@ -262,6 +266,7 @@ class PluginLoader:
            self._module_cache = MODULE_CACHE[self.class_name]
            self._paths = PATH_CACHE[self.class_name]
            self._plugin_path_cache = PLUGIN_PATH_CACHE[self.class_name]
+            self._plugin_instance_cache = {} if self.type == 'vars' else None
            self._searched_paths = set()

    def __setstate__(self, data):
@ -866,23 +871,35 @@ class PluginLoader:
        collection_list = kwargs.pop('collection_list', None)
        if name in self.aliases:
            name = self.aliases[name]
+
+        if self._plugin_instance_cache and (cached_load_result := self._plugin_instance_cache.get(name)):
+            # Resolving the FQCN is slow, even if we've passed in the resolved FQCN.
+            # Short-circuit here if we've previously resolved this name.
+            # This will need to be restricted if non-vars plugins start using the cache, since
+            # some non-fqcn plugin need to be resolved again with the collections list.
+            return get_with_context_result(*cached_load_result)
+
        plugin_load_context = self.find_plugin_with_context(name, collection_list=collection_list)
        if not plugin_load_context.resolved or not plugin_load_context.plugin_resolved_path:
            # FIXME: this is probably an error (eg removed plugin)
            return get_with_context_result(None, plugin_load_context)

        fq_name = plugin_load_context.resolved_fqcn
-        if '.' not in fq_name:
+        if '.' not in fq_name and plugin_load_context.plugin_resolved_collection:
            fq_name = '.'.join((plugin_load_context.plugin_resolved_collection, fq_name))
-        name = plugin_load_context.plugin_resolved_name
+        resolved_type_name = plugin_load_context.plugin_resolved_name
        path = plugin_load_context.plugin_resolved_path
+        if self._plugin_instance_cache and (cached_load_result := self._plugin_instance_cache.get(fq_name)):
+            # This is unused by vars plugins, but it's here in case the instance cache expands to other plugin types.
+            # We get here if we've seen this plugin before, but it wasn't called with the resolved FQCN.
+            return get_with_context_result(*cached_load_result)
        redirected_names = plugin_load_context.redirect_list or []

        if path not in self._module_cache:
-            self._module_cache[path] = self._load_module_source(name, path)
+            self._module_cache[path] = self._load_module_source(resolved_type_name, path)
            found_in_cache = False

-        self._load_config_defs(name, self._module_cache[path], path)
+        self._load_config_defs(resolved_type_name, self._module_cache[path], path)

        obj = getattr(self._module_cache[path], self.class_name)

@ -899,24 +916,27 @@ class PluginLoader:
                return get_with_context_result(None, plugin_load_context)

        # FIXME: update this to use the load context
-        self._display_plugin_load(self.class_name, name, self._searched_paths, path, found_in_cache=found_in_cache, class_only=class_only)
+        self._display_plugin_load(self.class_name, resolved_type_name, self._searched_paths, path, found_in_cache=found_in_cache, class_only=class_only)

        if not class_only:
            try:
                # A plugin may need to use its _load_name in __init__ (for example, to set
                # or get options from config), so update the object before using the constructor
                instance = object.__new__(obj)
-                self._update_object(instance, name, path, redirected_names, fq_name)
+                self._update_object(instance, resolved_type_name, path, redirected_names, fq_name)
                obj.__init__(instance, *args, **kwargs)  # pylint: disable=unnecessary-dunder-call
                obj = instance
            except TypeError as e:
                if "abstract" in e.args[0]:
                    # Abstract Base Class or incomplete plugin, don't load
-                    display.v('Returning not found on "%s" as it has unimplemented abstract methods; %s' % (name, to_native(e)))
+                    display.v('Returning not found on "%s" as it has unimplemented abstract methods; %s' % (resolved_type_name, to_native(e)))
                    return get_with_context_result(None, plugin_load_context)
                raise

-        self._update_object(obj, name, path, redirected_names, fq_name)
+        self._update_object(obj, resolved_type_name, path, redirected_names, fq_name)
+        if self._plugin_instance_cache is not None and getattr(obj, 'is_stateless', False):
+            # store under both the originally requested name and the resolved FQ name
+            self._plugin_instance_cache[name] = self._plugin_instance_cache[fq_name] = (obj, plugin_load_context)
        return get_with_context_result(obj, plugin_load_context)

    def _display_plugin_load(self, class_name, name, searched_paths, path, found_in_cache=None, class_only=None):
@ -986,28 +1006,47 @@ class PluginLoader:

        loaded_modules = set()
        for path in all_matches:
+
            name = os.path.splitext(path)[0]
            basename = os.path.basename(name)
+            is_j2 = isinstance(self, Jinja2Loader)

-            if basename in _PLUGIN_FILTERS[self.package]:
+            if is_j2:
+                ref_name = path
+            else:
+                ref_name = basename
+
+            if not is_j2 and basename in _PLUGIN_FILTERS[self.package]:
+                # j2 plugins get processed in own class, here they would just be container files
                display.debug("'%s' skipped due to a defined plugin filter" % basename)
                continue

            if basename == '__init__' or (basename == 'base' and self.package == 'ansible.plugins.cache'):
                # cache has legacy 'base.py' file, which is wrapper for __init__.py
-                display.debug("'%s' skipped due to reserved name" % basename)
+                display.debug("'%s' skipped due to reserved name" % name)
                continue

-            if dedupe and basename in loaded_modules:
-                display.debug("'%s' skipped as duplicate" % basename)
+            if dedupe and ref_name in loaded_modules:
+                # for j2 this is 'same file', other plugins it is basename
+                display.debug("'%s' skipped as duplicate" % ref_name)
                continue

-            loaded_modules.add(basename)
+            loaded_modules.add(ref_name)

            if path_only:
                yield path
                continue

+            if path in legacy_excluding_builtin:
+                fqcn = basename
+            else:
+                fqcn = f"ansible.builtin.{basename}"
+
+            if self._plugin_instance_cache is not None and fqcn in self._plugin_instance_cache:
+                # Here just in case, but we don't call all() multiple times for vars plugins, so this should not be used.
+                yield self._plugin_instance_cache[basename][0]
+                continue
+
            if path not in self._module_cache:
                if self.type in ('filter', 'test'):
                    # filter and test plugin files can contain multiple plugins
@ -1055,11 +1094,12 @@ class PluginLoader:
                except TypeError as e:
                    display.warning("Skipping plugin (%s) as it seems to be incomplete: %s" % (path, to_text(e)))

-            if path in legacy_excluding_builtin:
-                fqcn = basename
-            else:
-                fqcn = f"ansible.builtin.{basename}"
            self._update_object(obj, basename, path, resolved=fqcn)
+
+            if self._plugin_instance_cache is not None and fqcn not in self._plugin_instance_cache:
+                # Use get_with_context to cache the plugin the first time we see it.
+                self.get_with_context(fqcn)[0]
+
            yield obj


--- a/lib/ansible/plugins/lookup/init.py
+++ b/lib/ansible/plugins/lookup/init.py
@ -117,7 +117,7 @@ class LookupBase(AnsiblePlugin):

        result = None
        try:
-            result = self._loader.path_dwim_relative_stack(paths, subdir, needle)
+            result = self._loader.path_dwim_relative_stack(paths, subdir, needle, is_role=bool('role_path' in myvars))
        except AnsibleFileNotFound:
            if not ignore_missing:
                self._display.warning("Unable to find '%s' in expected paths (use -vvvvv to see paths)" % needle)
--- a/lib/ansible/plugins/lookup/first_found.py
+++ b/lib/ansible/plugins/lookup/first_found.py
@ -139,7 +139,6 @@ RETURN = """
    elements: path
 """
 import os
-import re

 from collections.abc import Mapping, Sequence

@ -150,10 +149,22 @@ from ansible.module_utils.six import string_types
 from ansible.plugins.lookup import LookupBase


+def _splitter(value, chars):
+    chars = set(chars)
+    v = ''
+    for c in value:
+        if c in chars:
+            yield v
+            v = ''
+            continue
+        v += c
+    yield v
+
+
 def _split_on(terms, spliters=','):
    termlist = []
    if isinstance(terms, string_types):
-        termlist = re.split(r'[%s]' % ''.join(map(re.escape, spliters)), terms)
+        termlist = list(_splitter(terms, spliters))
    else:
        # added since options will already listify
        for t in terms:
--- a/lib/ansible/plugins/strategy/init.py
+++ b/lib/ansible/plugins/strategy/init.py
@ -55,6 +55,7 @@ from ansible.template import Templar
 from ansible.utils.display import Display
 from ansible.utils.fqcn import add_internal_fqcns
 from ansible.utils.unsafe_proxy import wrap_var
+from ansible.utils.sentinel import Sentinel
 from ansible.utils.vars import combine_vars, isidentifier
 from ansible.vars.clean import strip_internal_keys, module_response_deepcopy

@ -658,6 +659,7 @@ class StrategyBase:
                        # otherwise depending on the setting either error or warn
                        host_state = iterator.get_state_for_host(original_host.name)
                        for notification in result_item['_ansible_notify']:
+                            handler = Sentinel
                            for handler in self.search_handlers_by_notification(notification, iterator):
                                if host_state.run_state == IteratingStates.HANDLERS:
                                    # we're currently iterating handlers, so we need to expand this now
@ -668,8 +670,8 @@ class StrategyBase:
                                else:
                                    iterator.add_notification(original_host.name, notification)
                                    display.vv(f"Notification for handler {notification} has been saved.")
-                                break
-                            else:
+                                    break
+                            if handler is Sentinel:
                                msg = (
                                    f"The requested handler '{notification}' was not found in either the main handlers"
                                    " list nor in the listening handlers list"
@ -800,10 +802,6 @@ class StrategyBase:

            ret_results.append(task_result)

-            if isinstance(original_task, Handler):
-                for handler in (h for b in iterator._play.handlers for h in b.block if h._uuid == original_task._uuid):
-                    handler.remove_host(original_host)
-
            if one_pass or max_passes is not None and (cur_pass + 1) >= max_passes:
                break

@ -1091,9 +1089,6 @@ class StrategyBase:
            header = skip_reason if skipped else msg
            display.vv(f"META: {header}")

-        if isinstance(task, Handler):
-            task.remove_host(target_host)
-
        res = TaskResult(target_host, task, result)
        if skipped:
            self._tqm.send_callback('v2_runner_on_skipped', res)
--- a/lib/ansible/plugins/strategy/free.py
+++ b/lib/ansible/plugins/strategy/free.py
@ -146,6 +146,8 @@ class StrategyModule(StrategyBase):
                        # advance the host, mark the host blocked, and queue it
                        self._blocked_hosts[host_name] = True
                        iterator.set_state_for_host(host.name, state)
+                        if isinstance(task, Handler):
+                            task.remove_host(host)

                        try:
                            action = action_loader.get(task.action, class_only=True, collection_list=task.collections)
--- a/lib/ansible/plugins/strategy/linear.py
+++ b/lib/ansible/plugins/strategy/linear.py
@ -242,6 +242,12 @@ class StrategyModule(StrategyBase):
                        self._queue_task(host, task, task_vars, play_context)
                        del task_vars

+                    if isinstance(task, Handler):
+                        if run_once:
+                            task.clear_hosts()
+                        else:
+                            task.remove_host(host)
+
                    # if we're bypassing the host loop, break out now
                    if run_once:
                        break
--- a/lib/ansible/plugins/vars/init.py
+++ b/lib/ansible/plugins/vars/init.py
@ -30,6 +30,7 @@ class BaseVarsPlugin(AnsiblePlugin):
    """
    Loads variables for groups and/or hosts
    """
+    is_stateless = False

    def __init__(self):
        """ constructor """
--- a/Show More
+++ b/Show More