New release v2.16.2 (#82399 )

Additional Unsafe fixes (#82376 )
* Allow older pickle protocols to pickle unsafe classes. Fixes #82356 * Address issues when iterating or getting single index from AnsibleUnsafeBytes. Fixes #82375 * clog frag
2023-12-11 14:48:58 -06:00 · 2023-12-07 17:17:14 -06:00 · 2023-12-06 16:18:19 -06:00 · 2023-12-04 22:27:00 +00:00 · 2023-12-04 15:46:24 -06:00 · 2023-12-01 09:28:05 -08:00
204 changed files with 3379 additions and 828 deletions
--- a/changelogs/CHANGELOG-v2.16.rst
+++ b/changelogs/CHANGELOG-v2.16.rst
@ -0,0 +1,358 @@
+=============================================
+ansible-core 2.16 "All My Love" Release Notes
+=============================================
+
+.. contents:: Topics
+
+
+v2.16.2
+=======
+
+Release Summary
+---------------
+
+| Release Date: 2023-12-11
+| `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+
+Bugfixes
+--------
+
+- unsafe data - Address an incompatibility when iterating or getting a single index from ``AnsibleUnsafeBytes``
+- unsafe data - Address an incompatibility with ``AnsibleUnsafeText`` and ``AnsibleUnsafeBytes`` when pickling with ``protocol=0``
+
+v2.16.1
+=======
+
+Release Summary
+---------------
+
+| Release Date: 2023-12-04
+| `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+
+Breaking Changes / Porting Guide
+--------------------------------
+
+- assert - Nested templating may result in an inability for the conditional to be evaluated. See the porting guide for more information.
+
+Security Fixes
+--------------
+
+- templating - Address issues where internal templating can cause unsafe variables to lose their unsafe designation (CVE-2023-5764)
+
+Bugfixes
+--------
+
+- Fix issue where an ``include_tasks`` handler in a role was not able to locate a file in ``tasks/`` when ``tasks_from`` was used as a role entry point and ``main.yml`` was not present (https://github.com/ansible/ansible/issues/82241)
+- Plugin loader does not dedupe nor cache filter/test plugins by file basename, but full path name.
+- Restoring the ability of filters/tests can have same file base name but different tests/filters defined inside.
+- ansible-pull now will expand relative paths for the ``-d|--directory`` option is now expanded before use.
+- ansible-pull will now correctly handle become and connection password file options for ansible-playbook.
+- flush_handlers - properly handle a handler failure in a nested block when ``force_handlers`` is set (http://github.com/ansible/ansible/issues/81532)
+- module no_log will no longer affect top level booleans, for example ``no_log_module_parameter='a'`` will no longer hide ``changed=False`` as a 'no log value' (matches 'a').
+- role params now have higher precedence than host facts again, matching documentation, this had unintentionally changed in 2.15.
+- wait_for should not handle 'non mmapable files' again.
+
+v2.16.0
+=======
+
+Release Summary
+---------------
+
+| Release Date: 2023-11-06
+| `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+
+Minor Changes
+-------------
+
+- Add Python type hints to the Display class (https://github.com/ansible/ansible/issues/80841)
+- Add ``GALAXY_COLLECTIONS_PATH_WARNING`` option to disable the warning given by ``ansible-galaxy collection install`` when installing a collection to a path that isn't in the configured collection paths.
+- Add ``python3.12`` to the default ``INTERPRETER_PYTHON_FALLBACK`` list.
+- Add ``utcfromtimestamp`` and ``utcnow`` to ``ansible.module_utils.compat.datetime`` to return fixed offset datetime objects.
+- Add a general ``GALAXY_SERVER_TIMEOUT`` config option for distribution servers (https://github.com/ansible/ansible/issues/79833).
+- Added Python type annotation to connection plugins
+- CLI argument parsing - Automatically prepend to the help of CLI arguments that support being specified multiple times. (https://github.com/ansible/ansible/issues/22396)
+- DEFAULT_TRANSPORT now defaults to 'ssh', the old 'smart' option is being deprecated as versions of OpenSSH without control persist are basically not present anymore.
+- Documentation for set filters ``intersect``, ``difference``, ``symmetric_difference`` and ``union`` now states that the returned list items are in arbitrary order.
+- Record ``removal_date`` in runtime metadata as a string instead of a date.
+- Remove the ``CleansingNodeVisitor`` class and its usage due to the templating changes that made it superfluous. Also simplify the ``Conditional`` class.
+- Removed ``exclude`` and ``recursive-exclude`` commands for generated files from the ``MANIFEST.in`` file. These excludes were unnecessary since releases are expected to be built with a clean worktree.
+- Removed ``exclude`` commands for sanity test files from the ``MANIFEST.in`` file. These tests were previously excluded because they did not pass when run from an sdist. However, sanity tests are not expected to pass from an sdist, so excluding some (but not all) of the failing tests makes little sense.
+- Removed redundant ``include`` commands from the ``MANIFEST.in`` file. These includes either duplicated default behavior or another command.
+- The ``ansible-core`` sdist no longer contains pre-generated man pages. Instead, a ``packaging/cli-doc/build.py`` script is included in the sdist. This script can generate man pages and standalone RST documentation for ``ansible-core`` CLI programs.
+- The ``docs`` and ``examples`` directories are no longer included in the ``ansible-core`` sdist. These directories have been moved to the https://github.com/ansible/ansible-documentation repository.
+- The minimum required ``setuptools`` version is now 66.1.0, as it is the oldest version to support Python 3.12.
+- Update ``ansible_service_mgr`` fact to include init system for SMGL OS family
+- Use ``ansible.module_utils.common.text.converters`` instead of ``ansible.module_utils._text``.
+- Use ``importlib.resources.abc.TraversableResources`` instead of deprecated ``importlib.abc.TraversableResources`` where available (https:/github.com/ansible/ansible/pull/81082).
+- Use ``include`` where ``recursive-include`` is unnecessary in the ``MANIFEST.in`` file.
+- Use ``package_data`` instead of ``include_package_data`` for ``setup.cfg`` to avoid ``setuptools`` warnings.
+- Utilize gpg check provided internally by the ``transaction.run`` method as oppose to calling it manually.
+- ``Templar`` - do not add the ``dict`` constructor to ``globals`` as all required Jinja2 versions already do so
+- ansible-doc - allow to filter listing of collections and metadata dump by more than one collection (https://github.com/ansible/ansible/pull/81450).
+- ansible-galaxy - Add a plural option to improve ignoring multiple signature error status codes when installing or verifying collections. A space-separated list of error codes can follow --ignore-signature-status-codes in addition to specifying --ignore-signature-status-code multiple times (for example, ``--ignore-signature-status-codes NO_PUBKEY UNEXPECTED``).
+- ansible-galaxy - Remove internal configuration argument ``v3`` (https://github.com/ansible/ansible/pull/80721)
+- ansible-galaxy - add note to the collection dependency resolver error message about pre-releases if ``--pre`` was not provided (https://github.com/ansible/ansible/issues/80048).
+- ansible-galaxy - used to crash out with a "Errno 20 Not a directory" error when extracting files from a role when hitting a file with an illegal name (https://github.com/ansible/ansible/pull/81553). Now it gives a warning identifying the culprit file and the rule violation (e.g., ``my$class.jar`` has a ``$`` in the name) before crashing out, giving the user a chance to remove the invalid file and try again. (https://github.com/ansible/ansible/pull/81555).
+- ansible-test - Add Alpine 3.18 to remotes
+- ansible-test - Add Fedora 38 container.
+- ansible-test - Add Fedora 38 remote.
+- ansible-test - Add FreeBSD 13.2 remote.
+- ansible-test - Add new pylint checker for new ``# deprecated:`` comments within code to trigger errors when time to remove code that has no user facing deprecation message. Only supported in ansible-core, not collections.
+- ansible-test - Add support for RHEL 8.8 remotes.
+- ansible-test - Add support for RHEL 9.2 remotes.
+- ansible-test - Add support for testing with Python 3.12.
+- ansible-test - Allow float values for the ``--timeout`` option to the ``env`` command. This simplifies testing.
+- ansible-test - Enable ``thread`` code coverage in addition to the existing ``multiprocessing`` coverage.
+- ansible-test - Make Python 3.12 the default version used in the ``base`` and ``default`` containers.
+- ansible-test - RHEL 8.8 provisioning can now be used with the ``--python 3.11`` option.
+- ansible-test - RHEL 9.2 provisioning can now be used with the ``--python 3.11`` option.
+- ansible-test - Refactored ``env`` command logic and timeout handling.
+- ansible-test - Remove Fedora 37 remote support.
+- ansible-test - Remove Fedora 37 test container.
+- ansible-test - Remove Python 3.8 and 3.9 from RHEL 8.8.
+- ansible-test - Remove obsolete embedded script for configuring WinRM on Windows remotes.
+- ansible-test - Removed Ubuntu 20.04 LTS image from the `--remote` option.
+- ansible-test - Removed `freebsd/12.4` remote.
+- ansible-test - Removed `freebsd/13.1` remote.
+- ansible-test - Removed test remotes: rhel/8.7, rhel/9.1
+- ansible-test - Removed the deprecated ``--docker-no-pull`` option.
+- ansible-test - Removed the deprecated ``--no-pip-check`` option.
+- ansible-test - Removed the deprecated ``foreman`` test plugin.
+- ansible-test - Removed the deprecated ``govcsim`` support from the ``vcenter`` test plugin.
+- ansible-test - Replace the ``pytest-forked`` pytest plugin with a custom plugin.
+- ansible-test - The ``no-get-exception`` sanity test is now limited to plugins in collections. Previously any Python file in a collection was checked for ``get_exception`` usage.
+- ansible-test - The ``replace-urlopen`` sanity test is now limited to plugins in collections. Previously any Python file in a collection was checked for ``urlopen`` usage.
+- ansible-test - The ``use-compat-six`` sanity test is now limited to plugins in collections. Previously any Python file in a collection was checked for ``six`` usage.
+- ansible-test - The openSUSE test container has been updated to openSUSE Leap 15.5.
+- ansible-test - Update pip to ``23.1.2`` and setuptools to ``67.7.2``.
+- ansible-test - Update the ``default`` containers.
+- ansible-test - Update the ``nios-test-container`` to version 2.0.0, which supports API version 2.9.
+- ansible-test - Update the logic used to detect when ``ansible-test`` is running from source.
+- ansible-test - Updated the CloudStack test container to version 1.6.1.
+- ansible-test - Updated the distro test containers to version 6.3.0 to include coverage 7.3.2 for Python 3.8+. The alpine3 container is now based on 3.18 instead of 3.17 and includes Python 3.11 instead of Python 3.10.
+- ansible-test - Use ``datetime.datetime.now`` with ``tz`` specified instead of ``datetime.datetime.utcnow``.
+- ansible-test - Use a context manager to perform cleanup at exit instead of using the built-in ``atexit`` module.
+- ansible-test - When invoking ``sleep`` in containers during container setup, the ``env`` command is used to avoid invoking the shell builtin, if present.
+- ansible-test - remove Alpine 3.17 from remotes
+- ansible-test — Python 3.8–3.12 will use ``coverage`` v7.3.2.
+- ansible-test — ``coverage`` v6.5.0 is to be used only under Python 3.7.
+- ansible-vault create: Now raises an error when opening the editor without tty. The flag --skip-tty-check restores previous behaviour.
+- ansible_user_module - tweaked macos user defaults to reflect expected defaults (https://github.com/ansible/ansible/issues/44316)
+- apt - return calculated diff while running apt clean operation.
+- blockinfile - add append_newline and prepend_newline options (https://github.com/ansible/ansible/issues/80835).
+- cli - Added short option '-J' for asking for vault password (https://github.com/ansible/ansible/issues/80523).
+- command - Add option ``expand_argument_vars`` to disable argument expansion and use literal values - https://github.com/ansible/ansible/issues/54162
+- config lookup new option show_origin to also return the origin of a configuration value.
+- display methods for warning and deprecation are now proxied to main process when issued from a fork. This allows for the deduplication of warnings and deprecations to work globally.
+- dnf5 - enable environment groups installation testing in CI as its support was added.
+- dnf5 - enable now implemented ``cacheonly`` functionality
+- executor now skips persistent connection when it detects an action that does not require a connection.
+- find module - Add ability to filter based on modes
+- gather_facts now will use gather_timeout setting to limit parallel execution of modules that do not themselves use gather_timeout.
+- group - remove extraneous warning shown when user does not exist (https://github.com/ansible/ansible/issues/77049).
+- include_vars - os.walk now follows symbolic links when traversing directories (https://github.com/ansible/ansible/pull/80460)
+- module compression is now sourced directly via config, bypassing play_context possibly stale values.
+- reboot - show last error message in verbose logs (https://github.com/ansible/ansible/issues/81574).
+- service_facts now returns more info for rcctl managed systesm (OpenBSD).
+- tasks - the ``retries`` keyword can be specified without ``until`` in which case the task is retried until it succeeds but at most ``retries`` times (https://github.com/ansible/ansible/issues/20802)
+- user - add new option ``password_expire_warn`` (supported on Linux only) to set the number of days of warning before a password change is required (https://github.com/ansible/ansible/issues/79882).
+- yum_repository - Align module documentation with parameters
+
+Breaking Changes / Porting Guide
+--------------------------------
+
+- Any plugin using the config system and the `cli` entry to use the `timeout` from the command line, will see the value change if the use had configured it in any of the lower precedence methods. If relying on this behaviour to consume the global/generic timeout from the DEFAULT_TIMEOUT constant, please consult the documentation on plugin configuration to add the overlaping entries.
+- ansible-test - Test plugins that rely on containers no longer support reusing running containers. The previous behavior was an undocumented, untested feature.
+- service module will not permanently configure variables/flags for openbsd when doing enable/disable operation anymore, this module was never meant to do this type of work, just to manage the service state itself. A rcctl_config or similar module should be created and used instead.
+
+Deprecated Features
+-------------------
+
+- Deprecated ini config option ``collections_paths``, use the singular form ``collections_path`` instead
+- Deprecated the env var ``ANSIBLE_COLLECTIONS_PATHS``, use the singular form ``ANSIBLE_COLLECTIONS_PATH`` instead
+- Old style vars plugins which use the entrypoints `get_host_vars` or `get_group_vars` are deprecated. The plugin should be updated to inherit from `BaseVarsPlugin` and define a `get_vars` method as the entrypoint.
+- Support for Windows Server 2012 and 2012 R2 has been removed as the support end of life from Microsoft is October 10th 2023. These versions of Windows will no longer be tested in this Ansible release and it cannot be guaranteed that they will continue to work going forward.
+- ``STRING_CONVERSION_ACTION`` config option is deprecated as it is no longer used in the Ansible Core code base.
+- the 'smart' option for setting a connection plugin is being removed as it's main purpose (choosing between ssh and paramiko) is now irrelevant.
+- vault and unfault filters - the undocumented ``vaultid`` parameter is deprecated and will be removed in ansible-core 2.20. Use ``vault_id`` instead.
+- yum_repository - deprecated parameter 'keepcache' (https://github.com/ansible/ansible/issues/78693).
+
+Removed Features (previously deprecated)
+----------------------------------------
+
+- ActionBase - remove deprecated ``_remote_checksum`` method
+- PlayIterator - remove deprecated ``cache_block_tasks`` and ``get_original_task`` methods
+- Remove deprecated ``FileLock`` class
+- Removed Python 3.9 as a supported version on the controller. Python 3.10 or newer is required.
+- Removed ``include`` which has been deprecated in Ansible 2.12. Use ``include_tasks`` or ``import_tasks`` instead.
+- ``Templar`` - remove deprecated ``shared_loader_obj`` parameter of ``__init__``
+- ``fetch_url`` - remove auto disabling ``decompress`` when gzip is not available
+- ``get_action_args_with_defaults`` - remove deprecated ``redirected_names`` method parameter
+- ansible-test - Removed support for the remote Windows targets 2012 and 2012-R2
+- inventory_cache - remove deprecated ``default.fact_caching_prefix`` ini configuration option, use ``defaults.fact_caching_prefix`` instead.
+- module_utils/basic.py - Removed Python 3.5 as a supported remote version. Python 2.7 or Python 3.6+ is now required.
+- stat - removed unused `get_md5` parameter.
+
+Security Fixes
+--------------
+
+- ansible-galaxy - Prevent roles from using symlinks to overwrite files outside of the installation directory (CVE-2023-5115)
+
+Bugfixes
+--------
+
+- Allow for searching handler subdir for included task via include_role (https://github.com/ansible/ansible/issues/81722)
+- AnsibleModule.run_command - Only use selectors when needed, and rely on Python stdlib subprocess for the simple task of collecting stdout/stderr when prompt matching is not required.
+- Cache host_group_vars after instantiating it once and limit the amount of repetitive work it needs to do every time it runs.
+- Call PluginLoader.all() once for vars plugins, and load vars plugins that run automatically or are enabled specifically by name subsequently.
+- Display - Defensively configure writing to stdout and stderr with a custom encoding error handler that will replace invalid characters while providing a deprecation warning that non-utf8 text will result in an error in a future version.
+- Exclude internal options from man pages and docs.
+- Fix ``ansible-config init`` man page option indentation.
+- Fix ``ast`` deprecation warnings for ``Str`` and ``value.s`` when using Python 3.12.
+- Fix ``run_once`` being incorrectly interpreted on handlers (https://github.com/ansible/ansible/issues/81666)
+- Fix exceptions caused by various inputs when performing arg splitting or parsing key/value pairs. Resolves issue https://github.com/ansible/ansible/issues/46379 and issue https://github.com/ansible/ansible/issues/61497
+- Fix incorrect parsing of multi-line Jinja2 blocks when performing arg splitting or parsing key/value pairs.
+- Fix post-validating looped task fields so the strategy uses the correct values after task execution.
+- Fixed `pip` module failure in case of usage quotes for `virtualenv_command` option for the venv command. (https://github.com/ansible/ansible/issues/76372)
+- From issue https://github.com/ansible/ansible/issues/80880, when notifying a handler from another handler, handler notifications must be registered immediately as the flush_handler call is not recursive.
+- Import ``FILE_ATTRIBUTES`` from ``ansible.module_utils.common.file`` in ``ansible.module_utils.basic`` instead of defining it twice.
+- Inventory scripts parser not treat exception when getting hostsvar (https://github.com/ansible/ansible/issues/81103)
+- On Python 3 use datetime methods ``fromtimestamp`` and ``now`` with UTC timezone instead of ``utcfromtimestamp`` and ``utcnow``, which are deprecated in Python 3.12.
+- PluginLoader - fix Jinja plugin performance issues (https://github.com/ansible/ansible/issues/79652)
+- PowerShell - Remove some code which is no longer valid for dotnet 5+
+- Prevent running same handler multiple times when included via ``include_role`` (https://github.com/ansible/ansible/issues/73643)
+- Prompting - add a short sleep between polling for user input to reduce CPU consumption (https://github.com/ansible/ansible/issues/81516).
+- Properly disable ``jinja2_native`` in the template module when jinja2 override is used in the template (https://github.com/ansible/ansible/issues/80605)
+- Properly template tags in parent blocks (https://github.com/ansible/ansible/issues/81053)
+- Remove unreachable parser error for removed ``static`` parameter of ``include_role``
+- Replace uses of ``configparser.ConfigParser.readfp()`` which was removed in Python 3.12 with ``configparser.ConfigParser.read_file()`` (https://github.com/ansible/ansible/issues/81656)
+- Set filters ``intersect``, ``difference``, ``symmetric_difference`` and ``union`` now always return a ``list``, never a ``set``. Previously, a ``set`` would be returned if the inputs were a hashable type such as ``str``, instead of a collection, such as a ``list`` or ``tuple``.
+- Set filters ``intersect``, ``difference``, ``symmetric_difference`` and ``union`` now use set operations when the given items are hashable. Previously, list operations were performed unless the inputs were a hashable type such as ``str``, instead of a collection, such as a ``list`` or ``tuple``.
+- Switch result queue from a ``multiprocessing.queues.Queue` to ``multiprocessing.queues.SimpleQueue``, primarily to allow properly handling pickling errors, to prevent an infinite hang waiting for task results
+- The ``ansible-config init`` command now has a documentation description.
+- The ``ansible-galaxy collection download`` command now has a documentation description.
+- The ``ansible-galaxy collection install`` command documentation is now visible (previously hidden by a decorator).
+- The ``ansible-galaxy collection verify`` command now has a documentation description.
+- The ``ansible-galaxy role install`` command documentation is now visible (previously hidden by a decorator).
+- The ``ansible-inventory`` command command now has a documentation description (previously used as the epilog).
+- The ``hostname`` module now also updates both current and permanent hostname on OpenBSD. Before it only updated the permanent hostname (https://github.com/ansible/ansible/issues/80520).
+- Update module_utils.urls unit test to work with cryptography >= 41.0.0.
+- When generating man pages, use ``func`` to find the command function instead of looking it up by the command name.
+- ``StrategyBase._process_pending_results`` - create a ``Templar`` on demand for templating ``changed_when``/``failed_when``.
+- ``ansible-galaxy`` now considers all collection paths when identifying which collection requirements are already installed. Use the ``COLLECTIONS_PATHS`` and ``COLLECTIONS_SCAN_SYS_PATHS`` config options to modify these. Previously only the install path was considered when resolving the candidates. The install path will remain the only one potentially modified. (https://github.com/ansible/ansible/issues/79767, https://github.com/ansible/ansible/issues/81163)
+- ``ansible.module_utils.service`` - ensure binary data transmission in ``daemonize()``
+- ``ansible.module_utils.service`` - fix inter-process communication in ``daemonize()``
+- ``import_role`` reverts to previous behavior of exporting vars at compile time.
+- ``pkg_mgr`` - fix the default dnf version detection
+- ansiballz - Prevent issue where the time on the control host could change part way through building the ansiballz file, potentially causing a pre-1980 date to be used during ansiballz unpacking leading to a zip file error (https://github.com/ansible/ansible/issues/80089)
+- ansible terminal color settings were incorrectly limited to 16 options via 'choices', removing so all 256 can be accessed.
+- ansible-console - fix filtering by collection names when a collection search path was set (https://github.com/ansible/ansible/pull/81450).
+- ansible-galaxy - Enabled the ``data`` tarfile filter during role installation for Python versions that support it. A probing mechanism is used to avoid Python versions with a broken implementation.
+- ansible-galaxy - Fix issue installing collections containing directories with more than 100 characters on python versions before 3.10.6
+- ansible-galaxy - Fix variable type error when installing subdir collections (https://github.com/ansible/ansible/issues/80943)
+- ansible-galaxy - Provide a better error message when using a requirements file with an invalid format - https://github.com/ansible/ansible/issues/81901
+- ansible-galaxy - fix installing collections from directories that have a trailing path separator (https://github.com/ansible/ansible/issues/77803).
+- ansible-galaxy - fix installing signed collections (https://github.com/ansible/ansible/issues/80648).
+- ansible-galaxy - reduce API calls to servers by fetching signatures only for final candidates.
+- ansible-galaxy - started allowing the use of pre-releases for collections that do not have any stable versions published. (https://github.com/ansible/ansible/pull/81606)
+- ansible-galaxy - started allowing the use of pre-releases for dependencies on any level of the dependency tree that specifically demand exact pre-release versions of collections and not version ranges. (https://github.com/ansible/ansible/pull/81606)
+- ansible-galaxy collection verify - fix verifying signed collections when the keyring is not configured.
+- ansible-galaxy info - fix reporting no role found when lookup_role_by_name returns None.
+- ansible-inventory - index available_hosts for major performance boost when dumping large inventories
+- ansible-test - Add a ``pylint`` plugin to work around a known issue on Python 3.12.
+- ansible-test - Add support for ``argcomplete`` version 3.
+- ansible-test - All containers created by ansible-test now include the current test session ID in their name. This avoids conflicts between concurrent ansible-test invocations using the same container host.
+- ansible-test - Always use ansible-test managed entry points for ansible-core CLI tools when not running from source. This fixes issues where CLI entry points created during install are not compatible with ansible-test.
+- ansible-test - Fix a traceback that occurs when attempting to test Ansible source using a different ansible-test. A clear error message is now given when this scenario occurs.
+- ansible-test - Fix handling of timeouts exceeding one day.
+- ansible-test - Fix parsing of cgroup entries which contain a ``:`` in the path (https://github.com/ansible/ansible/issues/81977).
+- ansible-test - Fix several possible tracebacks when using the ``-e`` option with sanity tests.
+- ansible-test - Fix various cases where the test timeout could expire without terminating the tests.
+- ansible-test - Include missing ``pylint`` requirements for Python 3.10.
+- ansible-test - Pre-build a PyYAML wheel before installing requirements to avoid a potential Cython build failure.
+- ansible-test - Remove redundant warning about missing programs before attempting to execute them.
+- ansible-test - The ``import`` sanity test now checks the collection loader for remote-only Python support when testing ansible-core.
+- ansible-test - Unit tests now report warnings generated during test runs. Previously only warnings generated during test collection were reported.
+- ansible-test - Update ``pylint`` to 2.17.2 to resolve several possible false positives.
+- ansible-test - Update ``pylint`` to 2.17.3 to resolve several possible false positives.
+- ansible-test - Update ``pylint`` to version 3.0.1.
+- ansible-test - Use ``raise ... from ...`` when raising exceptions from within an exception handler.
+- ansible-test - When bootstrapping remote FreeBSD instances, use the OS packaged ``setuptools`` instead of installing the latest version from PyPI.
+- ansible-test local change detection - use ``git merge-base <branch> HEAD`` instead of ``git merge-base --fork-point <branch>`` (https://github.com/ansible/ansible/pull/79734).
+- ansible-vault - fail when the destination file location is not writable before performing encryption (https://github.com/ansible/ansible/issues/81455).
+- apt - ignore fail_on_autoremove and allow_downgrade parameters when using aptitude (https://github.com/ansible/ansible/issues/77868).
+- blockinfile - avoid crash with Python 3 if creating the directory fails when ``create=true`` (https://github.com/ansible/ansible/pull/81662).
+- connection timeouts defined in ansible.cfg will now be properly used, the --timeout cli option was obscuring them by always being set.
+- copy - print correct destination filename when using `content` and `--diff` (https://github.com/ansible/ansible/issues/79749).
+- copy unit tests - Fixing "dir all perms" documentation and formatting for easier reading.
+- core will now also look at the connection plugin to force 'local' interpreter for networking path compatibility as just ansible_network_os could be misleading.
+- deb822_repository - use http-agent for receiving content (https://github.com/ansible/ansible/issues/80809).
+- debconf - idempotency in questions with type 'password' (https://github.com/ansible/ansible/issues/47676).
+- distribution facts - fix Source Mage family mapping
+- dnf - fix a failure when a package from URI was specified and ``update_only`` was set (https://github.com/ansible/ansible/issues/81376).
+- dnf5 - Update dnf5 module to handle API change for setting the download directory (https://github.com/ansible/ansible/issues/80887)
+- dnf5 - Use ``transaction.check_gpg_signatures`` API call to check package signatures AND possibly to recover from when keys are missing.
+- dnf5 - fix module and package names in the message following failed module respawn attempt
+- dnf5 - use the logs API to determine transaction problems
+- dpkg_selections - check if the package exists before performing the selection operation (https://github.com/ansible/ansible/issues/81404).
+- encrypt - deprecate passlib_or_crypt API (https://github.com/ansible/ansible/issues/55839).
+- fetch - Handle unreachable errors properly (https://github.com/ansible/ansible/issues/27816)
+- file modules - Make symbolic modes with X use the computed permission, not original file (https://github.com/ansible/ansible/issues/80128)
+- file modules - fix validating invalid symbolic modes.
+- first found lookup has been updated to use the normalized argument parsing (pythonic) matching the documented examples.
+- first found lookup, fixed an issue with subsequent items clobbering information from previous ones.
+- first_found lookup now gets 'untemplated' loop entries and handles templating itself as task_executor was removing even 'templatable' entries and breaking functionality. https://github.com/ansible/ansible/issues/70772
+- galaxy - check if the target for symlink exists (https://github.com/ansible/ansible/pull/81586).
+- galaxy - cross check the collection type and collection source (https://github.com/ansible/ansible/issues/79463).
+- gather_facts parallel option was doing the reverse of what was stated, now it does run modules in parallel when True and serially when False.
+- handlers - fix ``v2_playbook_on_notify`` callback not being called when notifying handlers
+- handlers - the ``listen`` keyword can affect only one handler with the same name, the last one defined as it is a case with the ``notify`` keyword (https://github.com/ansible/ansible/issues/81013)
+- include_role - expose variables from parent roles to role's handlers (https://github.com/ansible/ansible/issues/80459)
+- inventory_ini - handle SyntaxWarning while parsing ini file in inventory (https://github.com/ansible/ansible/issues/81457).
+- iptables - remove default rule creation when creating iptables chain to be more similar to the command line utility (https://github.com/ansible/ansible/issues/80256).
+- lib/ansible/utils/encrypt.py - remove unused private ``_LOCK`` (https://github.com/ansible/ansible/issues/81613)
+- lookup/url.py - Fix incorrect var/env/ini entry for `force_basic_auth`
+- man page build - Remove the dependency on the ``docs`` directory for building man pages.
+- man page build - Sub commands of ``ansible-galaxy role`` and ``ansible-galaxy collection`` are now documented.
+- module responses - Ensure that module responses are utf-8 adhereing to JSON RFC and expectations of the core code.
+- module/role argument spec - validate the type for options that are None when the option is required or has a non-None default (https://github.com/ansible/ansible/issues/79656).
+- modules/user.py - Add check for valid directory when creating new user homedir (allows /dev/null as skeleton) (https://github.com/ansible/ansible/issues/75063)
+- paramiko_ssh, psrp, and ssh connection plugins - ensure that all values for options that should be strings are actually converted to strings (https://github.com/ansible/ansible/pull/81029).
+- password_hash - fix salt format for ``crypt``  (only used if ``passlib`` is not installed) for the ``bcrypt`` algorithm.
+- pep517 build backend - Copy symlinks when copying the source tree. This avoids tracebacks in various scenarios, such as when a venv is present in the source tree.
+- pep517 build backend - Use the documented ``import_module`` import from ``importlib``.
+- pip module - Update module to prefer use of the python ``packaging`` and ``importlib.metadata`` modules due to ``pkg_resources`` being deprecated (https://github.com/ansible/ansible/issues/80488)
+- pkg_mgr.py - Fix `ansible_pkg_mgr` incorrect in TencentOS Server Linux
+- pkg_mgr.py - Fix `ansible_pkg_mgr` is unknown in Kylin Linux (https://github.com/ansible/ansible/issues/81332)
+- powershell modules - Only set an rc of 1 if the PowerShell pipeline signaled an error occurred AND there are error records present. Previously it would do so only if the error signal was present without checking the error count.
+- replace - handle exception when bad escape character is provided in replace (https://github.com/ansible/ansible/issues/79364).
+- role deduplication - don't deduplicate before a role has had a task run for that particular host (https://github.com/ansible/ansible/issues/81486).
+- service module, does not permanently configure flags flags on Openbsd when enabling/disabling a service.
+- service module, enable/disable is not a exclusive action in checkmode anymore.
+- setup gather_timeout - Fix timeout in get_mounts_facts for linux.
+- setup module (fact gathering) will now try to be smarter about different versions of facter emitting error when --puppet flag is used w/o puppet.
+- syntax check - Limit ``--syntax-check`` to ``ansible-playbook`` only, as that is the only CLI affected by this argument (https://github.com/ansible/ansible/issues/80506)
+- tarfile - handle data filter deprecation warning message for extract and extractall (https://github.com/ansible/ansible/issues/80832).
+- template - Fix for formatting issues when a template path contains valid jinja/strftime pattern (especially line break one) and using the template path in ansible_managed (https://github.com/ansible/ansible/pull/79129)
+- templating - In the template action and lookup, use local jinja2 environment overlay overrides instead of mutating the templars environment
+- templating - prevent setting arbitrary attributes on Jinja2 environments via Jinja2 overrides in templates
+- templating escape and single var optimization now use correct delimiters when custom ones are provided either via task or template header.
+- unarchive - fix unarchiving sources that are copied to the remote node using a relative temporory directory path (https://github.com/ansible/ansible/issues/80710).
+- uri - fix search for JSON type to include complex strings containing '+'
+- uri/urls - Add compat function to handle the ability to parse the filename from a Content-Disposition header (https://github.com/ansible/ansible/issues/81806)
+- urls.py - fixed cert_file and key_file parameters when running on Python 3.12 - https://github.com/ansible/ansible/issues/80490
+- user - set expiration value correctly when unable to retrieve the current value from the system (https://github.com/ansible/ansible/issues/71916)
+- validate-modules sanity test - replace semantic markup parsing and validating code with the code from `antsibull-docs-parser 0.2.0 <https://github.com/ansible-community/antsibull-docs-parser/releases/tag/0.2.0>`__ (https://github.com/ansible/ansible/pull/80406).
+- vars_prompt - internally convert the ``unsafe`` value to ``bool``
+- vault and unvault filters now properly take ``vault_id`` parameter.
+- win_fetch - Add support for using file with wildcards in file name. (https://github.com/ansible/ansible/issues/73128)
+- winrm - Better handle send input failures when communicating with hosts under load
+
+Known Issues
+------------
+
+- ansible-galaxy - dies in the middle of installing a role when that role contains Java inner classes (files with $ in the file name).  This is by design, to exclude temporary or backup files. (https://github.com/ansible/ansible/pull/81553).
+- ansible-test - The ``pep8`` sanity test is unable to detect f-string spacing issues (E201, E202) on Python 3.10 and 3.11. They are correctly detected under Python 3.12. See (https://github.com/PyCQA/pycodestyle/issues/1190).
--- a/changelogs/changelog.yaml
+++ b/changelogs/changelog.yaml
@ -1,2 +1,841 @@
 ancestor: 2.15.0
-releases: {}
+releases:
+  2.16.0:
+    changes:
+      bugfixes:
+      - ansible-test - Fix parsing of cgroup entries which contain a ``:`` in the
+        path (https://github.com/ansible/ansible/issues/81977).
+      release_summary: '| Release Date: 2023-11-06
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+        '
+    codename: All My Love
+    fragments:
+    - 2.16.0_summary.yaml
+    - ansible-test-cgroup-split.yml
+    release_date: '2023-11-06'
+  2.16.0b1:
+    changes:
+      breaking_changes:
+      - Any plugin using the config system and the `cli` entry to use the `timeout`
+        from the command line, will see the value change if the use had configured
+        it in any of the lower precedence methods. If relying on this behaviour to
+        consume the global/generic timeout from the DEFAULT_TIMEOUT constant, please
+        consult the documentation on plugin configuration to add the overlaping entries.
+      - ansible-test - Test plugins that rely on containers no longer support reusing
+        running containers. The previous behavior was an undocumented, untested feature.
+      - service module will not permanently configure variables/flags for openbsd
+        when doing enable/disable operation anymore, this module was never meant to
+        do this type of work, just to manage the service state itself. A rcctl_config
+        or similar module should be created and used instead.
+      bugfixes:
+      - Allow for searching handler subdir for included task via include_role (https://github.com/ansible/ansible/issues/81722)
+      - AnsibleModule.run_command - Only use selectors when needed, and rely on Python
+        stdlib subprocess for the simple task of collecting stdout/stderr when prompt
+        matching is not required.
+      - Display - Defensively configure writing to stdout and stderr with a custom
+        encoding error handler that will replace invalid characters while providing
+        a deprecation warning that non-utf8 text will result in an error in a future
+        version.
+      - Exclude internal options from man pages and docs.
+      - Fix ``ansible-config init`` man page option indentation.
+      - Fix ``ast`` deprecation warnings for ``Str`` and ``value.s`` when using Python
+        3.12.
+      - Fix exceptions caused by various inputs when performing arg splitting or parsing
+        key/value pairs. Resolves issue https://github.com/ansible/ansible/issues/46379
+        and issue https://github.com/ansible/ansible/issues/61497
+      - Fix incorrect parsing of multi-line Jinja2 blocks when performing arg splitting
+        or parsing key/value pairs.
+      - Fix post-validating looped task fields so the strategy uses the correct values
+        after task execution.
+      - Fixed `pip` module failure in case of usage quotes for `virtualenv_command`
+        option for the venv command. (https://github.com/ansible/ansible/issues/76372)
+      - From issue https://github.com/ansible/ansible/issues/80880, when notifying
+        a handler from another handler, handler notifications must be registered immediately
+        as the flush_handler call is not recursive.
+      - Import ``FILE_ATTRIBUTES`` from ``ansible.module_utils.common.file`` in ``ansible.module_utils.basic``
+        instead of defining it twice.
+      - Inventory scripts parser not treat exception when getting hostsvar (https://github.com/ansible/ansible/issues/81103)
+      - On Python 3 use datetime methods ``fromtimestamp`` and ``now`` with UTC timezone
+        instead of ``utcfromtimestamp`` and ``utcnow``, which are deprecated in Python
+        3.12.
+      - PluginLoader - fix Jinja plugin performance issues (https://github.com/ansible/ansible/issues/79652)
+      - PowerShell - Remove some code which is no longer valid for dotnet 5+
+      - Prevent running same handler multiple times when included via ``include_role``
+        (https://github.com/ansible/ansible/issues/73643)
+      - Prompting - add a short sleep between polling for user input to reduce CPU
+        consumption (https://github.com/ansible/ansible/issues/81516).
+      - Properly disable ``jinja2_native`` in the template module when jinja2 override
+        is used in the template (https://github.com/ansible/ansible/issues/80605)
+      - Remove unreachable parser error for removed ``static`` parameter of ``include_role``
+      - Replace uses of ``configparser.ConfigParser.readfp()`` which was removed in
+        Python 3.12 with ``configparser.ConfigParser.read_file()`` (https://github.com/ansible/ansible/issues/81656)
+      - Set filters ``intersect``, ``difference``, ``symmetric_difference`` and ``union``
+        now always return a ``list``, never a ``set``. Previously, a ``set`` would
+        be returned if the inputs were a hashable type such as ``str``, instead of
+        a collection, such as a ``list`` or ``tuple``.
+      - Set filters ``intersect``, ``difference``, ``symmetric_difference`` and ``union``
+        now use set operations when the given items are hashable. Previously, list
+        operations were performed unless the inputs were a hashable type such as ``str``,
+        instead of a collection, such as a ``list`` or ``tuple``.
+      - Switch result queue from a ``multiprocessing.queues.Queue` to ``multiprocessing.queues.SimpleQueue``,
+        primarily to allow properly handling pickling errors, to prevent an infinite
+        hang waiting for task results
+      - The ``ansible-config init`` command now has a documentation description.
+      - The ``ansible-galaxy collection download`` command now has a documentation
+        description.
+      - The ``ansible-galaxy collection install`` command documentation is now visible
+        (previously hidden by a decorator).
+      - The ``ansible-galaxy collection verify`` command now has a documentation description.
+      - The ``ansible-galaxy role install`` command documentation is now visible (previously
+        hidden by a decorator).
+      - The ``ansible-inventory`` command command now has a documentation description
+        (previously used as the epilog).
+      - The ``hostname`` module now also updates both current and permanent hostname
+        on OpenBSD. Before it only updated the permanent hostname (https://github.com/ansible/ansible/issues/80520).
+      - Update module_utils.urls unit test to work with cryptography >= 41.0.0.
+      - When generating man pages, use ``func`` to find the command function instead
+        of looking it up by the command name.
+      - '``StrategyBase._process_pending_results`` - create a ``Templar`` on demand
+        for templating ``changed_when``/``failed_when``.'
+      - '``ansible-galaxy`` now considers all collection paths when identifying which
+        collection requirements are already installed. Use the ``COLLECTIONS_PATHS``
+        and ``COLLECTIONS_SCAN_SYS_PATHS`` config options to modify these. Previously
+        only the install path was considered when resolving the candidates. The install
+        path will remain the only one potentially modified. (https://github.com/ansible/ansible/issues/79767,
+        https://github.com/ansible/ansible/issues/81163)'
+      - '``ansible.module_utils.service`` - ensure binary data transmission in ``daemonize()``'
+      - '``ansible.module_utils.service`` - fix inter-process communication in ``daemonize()``'
+      - '``pkg_mgr`` - fix the default dnf version detection'
+      - ansiballz - Prevent issue where the time on the control host could change
+        part way through building the ansiballz file, potentially causing a pre-1980
+        date to be used during ansiballz unpacking leading to a zip file error (https://github.com/ansible/ansible/issues/80089)
+      - ansible terminal color settings were incorrectly limited to 16 options via
+        'choices', removing so all 256 can be accessed.
+      - ansible-console - fix filtering by collection names when a collection search
+        path was set (https://github.com/ansible/ansible/pull/81450).
+      - ansible-galaxy - Enabled the ``data`` tarfile filter during role installation
+        for Python versions that support it. A probing mechanism is used to avoid
+        Python versions with a broken implementation.
+      - ansible-galaxy - Fix issue installing collections containing directories with
+        more than 100 characters on python versions before 3.10.6
+      - ansible-galaxy - Fix variable type error when installing subdir collections
+        (https://github.com/ansible/ansible/issues/80943)
+      - ansible-galaxy - fix installing collections from directories that have a trailing
+        path separator (https://github.com/ansible/ansible/issues/77803).
+      - ansible-galaxy - fix installing signed collections (https://github.com/ansible/ansible/issues/80648).
+      - ansible-galaxy - reduce API calls to servers by fetching signatures only for
+        final candidates.
+      - ansible-galaxy - started allowing the use of pre-releases for collections
+        that do not have any stable versions published. (https://github.com/ansible/ansible/pull/81606)
+      - ansible-galaxy - started allowing the use of pre-releases for dependencies
+        on any level of the dependency tree that specifically demand exact pre-release
+        versions of collections and not version ranges. (https://github.com/ansible/ansible/pull/81606)
+      - ansible-galaxy collection verify - fix verifying signed collections when the
+        keyring is not configured.
+      - ansible-test - Add support for ``argcomplete`` version 3.
+      - ansible-test - All containers created by ansible-test now include the current
+        test session ID in their name. This avoids conflicts between concurrent ansible-test
+        invocations using the same container host.
+      - ansible-test - Always use ansible-test managed entry points for ansible-core
+        CLI tools when not running from source. This fixes issues where CLI entry
+        points created during install are not compatible with ansible-test.
+      - ansible-test - Fix a traceback that occurs when attempting to test Ansible
+        source using a different ansible-test. A clear error message is now given
+        when this scenario occurs.
+      - ansible-test - Fix handling of timeouts exceeding one day.
+      - ansible-test - Fix several possible tracebacks when using the ``-e`` option
+        with sanity tests.
+      - ansible-test - Fix various cases where the test timeout could expire without
+        terminating the tests.
+      - ansible-test - Pre-build a PyYAML wheel before installing requirements to
+        avoid a potential Cython build failure.
+      - ansible-test - Remove redundant warning about missing programs before attempting
+        to execute them.
+      - ansible-test - The ``import`` sanity test now checks the collection loader
+        for remote-only Python support when testing ansible-core.
+      - ansible-test - Unit tests now report warnings generated during test runs.
+        Previously only warnings generated during test collection were reported.
+      - ansible-test - Update ``pylint`` to 2.17.2 to resolve several possible false
+        positives.
+      - ansible-test - Update ``pylint`` to 2.17.3 to resolve several possible false
+        positives.
+      - ansible-test - Use ``raise ... from ...`` when raising exceptions from within
+        an exception handler.
+      - ansible-test - When bootstrapping remote FreeBSD instances, use the OS packaged
+        ``setuptools`` instead of installing the latest version from PyPI.
+      - ansible-test local change detection - use ``git merge-base <branch> HEAD``
+        instead of ``git merge-base --fork-point <branch>`` (https://github.com/ansible/ansible/pull/79734).
+      - ansible-vault - fail when the destination file location is not writable before
+        performing encryption (https://github.com/ansible/ansible/issues/81455).
+      - apt - ignore fail_on_autoremove and allow_downgrade parameters when using
+        aptitude (https://github.com/ansible/ansible/issues/77868).
+      - blockinfile - avoid crash with Python 3 if creating the directory fails when
+        ``create=true`` (https://github.com/ansible/ansible/pull/81662).
+      - connection timeouts defined in ansible.cfg will now be properly used, the
+        --timeout cli option was obscuring them by always being set.
+      - copy - print correct destination filename when using `content` and `--diff`
+        (https://github.com/ansible/ansible/issues/79749).
+      - copy unit tests - Fixing "dir all perms" documentation and formatting for
+        easier reading.
+      - core will now also look at the connection plugin to force 'local' interpreter
+        for networking path compatibility as just ansible_network_os could be misleading.
+      - deb822_repository - use http-agent for receiving content (https://github.com/ansible/ansible/issues/80809).
+      - debconf - idempotency in questions with type 'password' (https://github.com/ansible/ansible/issues/47676).
+      - distribution facts - fix Source Mage family mapping
+      - dnf - fix a failure when a package from URI was specified and ``update_only``
+        was set (https://github.com/ansible/ansible/issues/81376).
+      - dnf5 - Update dnf5 module to handle API change for setting the download directory
+        (https://github.com/ansible/ansible/issues/80887)
+      - dnf5 - Use ``transaction.check_gpg_signatures`` API call to check package
+        signatures AND possibly to recover from when keys are missing.
+      - dnf5 - fix module and package names in the message following failed module
+        respawn attempt
+      - dnf5 - use the logs API to determine transaction problems
+      - dpkg_selections - check if the package exists before performing the selection
+        operation (https://github.com/ansible/ansible/issues/81404).
+      - encrypt - deprecate passlib_or_crypt API (https://github.com/ansible/ansible/issues/55839).
+      - fetch - Handle unreachable errors properly (https://github.com/ansible/ansible/issues/27816)
+      - file modules - Make symbolic modes with X use the computed permission, not
+        original file (https://github.com/ansible/ansible/issues/80128)
+      - file modules - fix validating invalid symbolic modes.
+      - first found lookup has been updated to use the normalized argument parsing
+        (pythonic) matching the documented examples.
+      - first found lookup, fixed an issue with subsequent items clobbering information
+        from previous ones.
+      - first_found lookup now gets 'untemplated' loop entries and handles templating
+        itself as task_executor was removing even 'templatable' entries and breaking
+        functionality. https://github.com/ansible/ansible/issues/70772
+      - galaxy - check if the target for symlink exists (https://github.com/ansible/ansible/pull/81586).
+      - galaxy - cross check the collection type and collection source (https://github.com/ansible/ansible/issues/79463).
+      - gather_facts parallel option was doing the reverse of what was stated, now
+        it does run modules in parallel when True and serially when False.
+      - handlers - fix ``v2_playbook_on_notify`` callback not being called when notifying
+        handlers
+      - handlers - the ``listen`` keyword can affect only one handler with the same
+        name, the last one defined as it is a case with the ``notify`` keyword (https://github.com/ansible/ansible/issues/81013)
+      - include_role - expose variables from parent roles to role's handlers (https://github.com/ansible/ansible/issues/80459)
+      - inventory_ini - handle SyntaxWarning while parsing ini file in inventory (https://github.com/ansible/ansible/issues/81457).
+      - iptables - remove default rule creation when creating iptables chain to be
+        more similar to the command line utility (https://github.com/ansible/ansible/issues/80256).
+      - lib/ansible/utils/encrypt.py - remove unused private ``_LOCK`` (https://github.com/ansible/ansible/issues/81613)
+      - lookup/url.py - Fix incorrect var/env/ini entry for `force_basic_auth`
+      - man page build - Remove the dependency on the ``docs`` directory for building
+        man pages.
+      - man page build - Sub commands of ``ansible-galaxy role`` and ``ansible-galaxy
+        collection`` are now documented.
+      - module responses - Ensure that module responses are utf-8 adhereing to JSON
+        RFC and expectations of the core code.
+      - module/role argument spec - validate the type for options that are None when
+        the option is required or has a non-None default (https://github.com/ansible/ansible/issues/79656).
+      - modules/user.py - Add check for valid directory when creating new user homedir
+        (allows /dev/null as skeleton) (https://github.com/ansible/ansible/issues/75063)
+      - paramiko_ssh, psrp, and ssh connection plugins - ensure that all values for
+        options that should be strings are actually converted to strings (https://github.com/ansible/ansible/pull/81029).
+      - password_hash - fix salt format for ``crypt``  (only used if ``passlib`` is
+        not installed) for the ``bcrypt`` algorithm.
+      - pep517 build backend - Copy symlinks when copying the source tree. This avoids
+        tracebacks in various scenarios, such as when a venv is present in the source
+        tree.
+      - pep517 build backend - Use the documented ``import_module`` import from ``importlib``.
+      - pip module - Update module to prefer use of the python ``packaging`` and ``importlib.metadata``
+        modules due to ``pkg_resources`` being deprecated (https://github.com/ansible/ansible/issues/80488)
+      - pkg_mgr.py - Fix `ansible_pkg_mgr` incorrect in TencentOS Server Linux
+      - pkg_mgr.py - Fix `ansible_pkg_mgr` is unknown in Kylin Linux (https://github.com/ansible/ansible/issues/81332)
+      - powershell modules - Only set an rc of 1 if the PowerShell pipeline signaled
+        an error occurred AND there are error records present. Previously it would
+        do so only if the error signal was present without checking the error count.
+      - replace - handle exception when bad escape character is provided in replace
+        (https://github.com/ansible/ansible/issues/79364).
+      - role deduplication - don't deduplicate before a role has had a task run for
+        that particular host (https://github.com/ansible/ansible/issues/81486).
+      - service module, does not permanently configure flags flags on Openbsd when
+        enabling/disabling a service.
+      - service module, enable/disable is not a exclusive action in checkmode anymore.
+      - setup gather_timeout - Fix timeout in get_mounts_facts for linux.
+      - setup module (fact gathering) will now try to be smarter about different versions
+        of facter emitting error when --puppet flag is used w/o puppet.
+      - syntax check - Limit ``--syntax-check`` to ``ansible-playbook`` only, as that
+        is the only CLI affected by this argument (https://github.com/ansible/ansible/issues/80506)
+      - tarfile - handle data filter deprecation warning message for extract and extractall
+        (https://github.com/ansible/ansible/issues/80832).
+      - template - Fix for formatting issues when a template path contains valid jinja/strftime
+        pattern (especially line break one) and using the template path in ansible_managed
+        (https://github.com/ansible/ansible/pull/79129)
+      - templating - In the template action and lookup, use local jinja2 environment
+        overlay overrides instead of mutating the templars environment
+      - templating - prevent setting arbitrary attributes on Jinja2 environments via
+        Jinja2 overrides in templates
+      - templating escape and single var optimization now use correct delimiters when
+        custom ones are provided either via task or template header.
+      - unarchive - fix unarchiving sources that are copied to the remote node using
+        a relative temporory directory path (https://github.com/ansible/ansible/issues/80710).
+      - uri - fix search for JSON type to include complex strings containing '+'
+      - urls.py - fixed cert_file and key_file parameters when running on Python 3.12
+        - https://github.com/ansible/ansible/issues/80490
+      - user - set expiration value correctly when unable to retrieve the current
+        value from the system (https://github.com/ansible/ansible/issues/71916)
+      - validate-modules sanity test - replace semantic markup parsing and validating
+        code with the code from `antsibull-docs-parser 0.2.0 <https://github.com/ansible-community/antsibull-docs-parser/releases/tag/0.2.0>`__
+        (https://github.com/ansible/ansible/pull/80406).
+      - vars_prompt - internally convert the ``unsafe`` value to ``bool``
+      - vault and unvault filters now properly take ``vault_id`` parameter.
+      - win_fetch - Add support for using file with wildcards in file name. (https://github.com/ansible/ansible/issues/73128)
+      deprecated_features:
+      - Deprecated ini config option ``collections_paths``, use the singular form
+        ``collections_path`` instead
+      - Deprecated the env var ``ANSIBLE_COLLECTIONS_PATHS``, use the singular form
+        ``ANSIBLE_COLLECTIONS_PATH`` instead
+      - Support for Windows Server 2012 and 2012 R2 has been removed as the support
+        end of life from Microsoft is October 10th 2023. These versions of Windows
+        will no longer be tested in this Ansible release and it cannot be guaranteed
+        that they will continue to work going forward.
+      - '``STRING_CONVERSION_ACTION`` config option is deprecated as it is no longer
+        used in the Ansible Core code base.'
+      - the 'smart' option for setting a connection plugin is being removed as it's
+        main purpose (choosing between ssh and paramiko) is now irrelevant.
+      - vault and unfault filters - the undocumented ``vaultid`` parameter is deprecated
+        and will be removed in ansible-core 2.20. Use ``vault_id`` instead.
+      - yum_repository - deprecated parameter 'keepcache' (https://github.com/ansible/ansible/issues/78693).
+      known_issues:
+      - ansible-galaxy - dies in the middle of installing a role when that role contains
+        Java inner classes (files with $ in the file name).  This is by design, to
+        exclude temporary or backup files. (https://github.com/ansible/ansible/pull/81553).
+      - ansible-test - The ``pep8`` sanity test is unable to detect f-string spacing
+        issues (E201, E202) on Python 3.10 and 3.11. They are correctly detected under
+        Python 3.12. See (https://github.com/PyCQA/pycodestyle/issues/1190).
+      minor_changes:
+      - Add Python type hints to the Display class (https://github.com/ansible/ansible/issues/80841)
+      - Add ``GALAXY_COLLECTIONS_PATH_WARNING`` option to disable the warning given
+        by ``ansible-galaxy collection install`` when installing a collection to a
+        path that isn't in the configured collection paths.
+      - Add ``python3.12`` to the default ``INTERPRETER_PYTHON_FALLBACK`` list.
+      - Add ``utcfromtimestamp`` and ``utcnow`` to ``ansible.module_utils.compat.datetime``
+        to return fixed offset datetime objects.
+      - Add a general ``GALAXY_SERVER_TIMEOUT`` config option for distribution servers
+        (https://github.com/ansible/ansible/issues/79833).
+      - Added Python type annotation to connection plugins
+      - CLI argument parsing - Automatically prepend to the help of CLI arguments
+        that support being specified multiple times. (https://github.com/ansible/ansible/issues/22396)
+      - DEFAULT_TRANSPORT now defaults to 'ssh', the old 'smart' option is being deprecated
+        as versions of OpenSSH without control persist are basically not present anymore.
+      - Documentation for set filters ``intersect``, ``difference``, ``symmetric_difference``
+        and ``union`` now states that the returned list items are in arbitrary order.
+      - Record ``removal_date`` in runtime metadata as a string instead of a date.
+      - Remove the ``CleansingNodeVisitor`` class and its usage due to the templating
+        changes that made it superfluous. Also simplify the ``Conditional`` class.
+      - Removed ``exclude`` and ``recursive-exclude`` commands for generated files
+        from the ``MANIFEST.in`` file. These excludes were unnecessary since releases
+        are expected to be built with a clean worktree.
+      - Removed ``exclude`` commands for sanity test files from the ``MANIFEST.in``
+        file. These tests were previously excluded because they did not pass when
+        run from an sdist. However, sanity tests are not expected to pass from an
+        sdist, so excluding some (but not all) of the failing tests makes little sense.
+      - Removed redundant ``include`` commands from the ``MANIFEST.in`` file. These
+        includes either duplicated default behavior or another command.
+      - The ``ansible-core`` sdist no longer contains pre-generated man pages. Instead,
+        a ``packaging/cli-doc/build.py`` script is included in the sdist. This script
+        can generate man pages and standalone RST documentation for ``ansible-core``
+        CLI programs.
+      - The ``docs`` and ``examples`` directories are no longer included in the ``ansible-core``
+        sdist. These directories have been moved to the https://github.com/ansible/ansible-documentation
+        repository.
+      - The minimum required ``setuptools`` version is now 66.1.0, as it is the oldest
+        version to support Python 3.12.
+      - Update ``ansible_service_mgr`` fact to include init system for SMGL OS family
+      - Use ``ansible.module_utils.common.text.converters`` instead of ``ansible.module_utils._text``.
+      - Use ``importlib.resources.abc.TraversableResources`` instead of deprecated
+        ``importlib.abc.TraversableResources`` where available (https:/github.com/ansible/ansible/pull/81082).
+      - Use ``include`` where ``recursive-include`` is unnecessary in the ``MANIFEST.in``
+        file.
+      - Use ``package_data`` instead of ``include_package_data`` for ``setup.cfg``
+        to avoid ``setuptools`` warnings.
+      - Utilize gpg check provided internally by the ``transaction.run`` method as
+        oppose to calling it manually.
+      - '``Templar`` - do not add the ``dict`` constructor to ``globals`` as all required
+        Jinja2 versions already do so'
+      - ansible-doc - allow to filter listing of collections and metadata dump by
+        more than one collection (https://github.com/ansible/ansible/pull/81450).
+      - ansible-galaxy - Add a plural option to improve ignoring multiple signature
+        error status codes when installing or verifying collections. A space-separated
+        list of error codes can follow --ignore-signature-status-codes in addition
+        to specifying --ignore-signature-status-code multiple times (for example,
+        ``--ignore-signature-status-codes NO_PUBKEY UNEXPECTED``).
+      - ansible-galaxy - Remove internal configuration argument ``v3`` (https://github.com/ansible/ansible/pull/80721)
+      - ansible-galaxy - add note to the collection dependency resolver error message
+        about pre-releases if ``--pre`` was not provided (https://github.com/ansible/ansible/issues/80048).
+      - ansible-galaxy - used to crash out with a "Errno 20 Not a directory" error
+        when extracting files from a role when hitting a file with an illegal name
+        (https://github.com/ansible/ansible/pull/81553). Now it gives a warning identifying
+        the culprit file and the rule violation (e.g., ``my$class.jar`` has a ``$``
+        in the name) before crashing out, giving the user a chance to remove the invalid
+        file and try again. (https://github.com/ansible/ansible/pull/81555).
+      - ansible-test - Add Alpine 3.18 to remotes
+      - ansible-test - Add Fedora 38 container.
+      - ansible-test - Add Fedora 38 remote.
+      - ansible-test - Add FreeBSD 13.2 remote.
+      - ansible-test - Add new pylint checker for new ``# deprecated:`` comments within
+        code to trigger errors when time to remove code that has no user facing deprecation
+        message. Only supported in ansible-core, not collections.
+      - ansible-test - Add support for RHEL 8.8 remotes.
+      - ansible-test - Add support for RHEL 9.2 remotes.
+      - ansible-test - Add support for testing with Python 3.12.
+      - ansible-test - Allow float values for the ``--timeout`` option to the ``env``
+        command. This simplifies testing.
+      - ansible-test - Enable ``thread`` code coverage in addition to the existing
+        ``multiprocessing`` coverage.
+      - ansible-test - RHEL 8.8 provisioning can now be used with the ``--python 3.11``
+        option.
+      - ansible-test - RHEL 9.2 provisioning can now be used with the ``--python 3.11``
+        option.
+      - ansible-test - Refactored ``env`` command logic and timeout handling.
+      - ansible-test - Remove Fedora 37 remote support.
+      - ansible-test - Remove Fedora 37 test container.
+      - ansible-test - Remove Python 3.8 and 3.9 from RHEL 8.8.
+      - ansible-test - Remove obsolete embedded script for configuring WinRM on Windows
+        remotes.
+      - ansible-test - Removed Ubuntu 20.04 LTS image from the `--remote` option.
+      - ansible-test - Removed `freebsd/12.4` remote.
+      - ansible-test - Removed `freebsd/13.1` remote.
+      - 'ansible-test - Removed test remotes: rhel/8.7, rhel/9.1'
+      - ansible-test - Removed the deprecated ``--docker-no-pull`` option.
+      - ansible-test - Removed the deprecated ``--no-pip-check`` option.
+      - ansible-test - Removed the deprecated ``foreman`` test plugin.
+      - ansible-test - Removed the deprecated ``govcsim`` support from the ``vcenter``
+        test plugin.
+      - ansible-test - Replace the ``pytest-forked`` pytest plugin with a custom plugin.
+      - ansible-test - The ``no-get-exception`` sanity test is now limited to plugins
+        in collections. Previously any Python file in a collection was checked for
+        ``get_exception`` usage.
+      - ansible-test - The ``replace-urlopen`` sanity test is now limited to plugins
+        in collections. Previously any Python file in a collection was checked for
+        ``urlopen`` usage.
+      - ansible-test - The ``use-compat-six`` sanity test is now limited to plugins
+        in collections. Previously any Python file in a collection was checked for
+        ``six`` usage.
+      - ansible-test - The openSUSE test container has been updated to openSUSE Leap
+        15.5.
+      - ansible-test - Update pip to ``23.1.2`` and setuptools to ``67.7.2``.
+      - ansible-test - Update the ``default`` containers.
+      - ansible-test - Update the ``nios-test-container`` to version 2.0.0, which
+        supports API version 2.9.
+      - ansible-test - Update the logic used to detect when ``ansible-test`` is running
+        from source.
+      - ansible-test - Updated the CloudStack test container to version 1.6.1.
+      - ansible-test - Updated the distro test containers to version 6.3.0 to include
+        coverage 7.3.2 for Python 3.8+. The alpine3 container is now based on 3.18
+        instead of 3.17 and includes Python 3.11 instead of Python 3.10.
+      - ansible-test - Use ``datetime.datetime.now`` with ``tz`` specified instead
+        of ``datetime.datetime.utcnow``.
+      - ansible-test - Use a context manager to perform cleanup at exit instead of
+        using the built-in ``atexit`` module.
+      - ansible-test - remove Alpine 3.17 from remotes
+      - "ansible-test \u2014 Python 3.8\u20133.12 will use ``coverage`` v7.3.2."
+      - "ansible-test \u2014 ``coverage`` v6.5.0 is to be used only under Python 3.7."
+      - 'ansible-vault create: Now raises an error when opening the editor without
+        tty. The flag --skip-tty-check restores previous behaviour.'
+      - ansible_user_module - tweaked macos user defaults to reflect expected defaults
+        (https://github.com/ansible/ansible/issues/44316)
+      - apt - return calculated diff while running apt clean operation.
+      - blockinfile - add append_newline and prepend_newline options (https://github.com/ansible/ansible/issues/80835).
+      - cli - Added short option '-J' for asking for vault password (https://github.com/ansible/ansible/issues/80523).
+      - command - Add option ``expand_argument_vars`` to disable argument expansion
+        and use literal values - https://github.com/ansible/ansible/issues/54162
+      - config lookup new option show_origin to also return the origin of a configuration
+        value.
+      - display methods for warning and deprecation are now proxied to main process
+        when issued from a fork. This allows for the deduplication of warnings and
+        deprecations to work globally.
+      - dnf5 - enable environment groups installation testing in CI as its support
+        was added.
+      - dnf5 - enable now implemented ``cacheonly`` functionality
+      - executor now skips persistent connection when it detects an action that does
+        not require a connection.
+      - find module - Add ability to filter based on modes
+      - gather_facts now will use gather_timeout setting to limit parallel execution
+        of modules that do not themselves use gather_timeout.
+      - group - remove extraneous warning shown when user does not exist (https://github.com/ansible/ansible/issues/77049).
+      - include_vars - os.walk now follows symbolic links when traversing directories
+        (https://github.com/ansible/ansible/pull/80460)
+      - module compression is now sourced directly via config, bypassing play_context
+        possibly stale values.
+      - reboot - show last error message in verbose logs (https://github.com/ansible/ansible/issues/81574).
+      - service_facts now returns more info for rcctl managed systesm (OpenBSD).
+      - tasks - the ``retries`` keyword can be specified without ``until`` in which
+        case the task is retried until it succeeds but at most ``retries`` times (https://github.com/ansible/ansible/issues/20802)
+      - user - add new option ``password_expire_warn`` (supported on Linux only) to
+        set the number of days of warning before a password change is required (https://github.com/ansible/ansible/issues/79882).
+      - yum_repository - Align module documentation with parameters
+      release_summary: '| Release Date: 2023-09-26
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+        '
+      removed_features:
+      - ActionBase - remove deprecated ``_remote_checksum`` method
+      - PlayIterator - remove deprecated ``cache_block_tasks`` and ``get_original_task``
+        methods
+      - Remove deprecated ``FileLock`` class
+      - Removed Python 3.9 as a supported version on the controller. Python 3.10 or
+        newer is required.
+      - Removed ``include`` which has been deprecated in Ansible 2.12. Use ``include_tasks``
+        or ``import_tasks`` instead.
+      - '``Templar`` - remove deprecated ``shared_loader_obj`` parameter of ``__init__``'
+      - '``fetch_url`` - remove auto disabling ``decompress`` when gzip is not available'
+      - '``get_action_args_with_defaults`` - remove deprecated ``redirected_names``
+        method parameter'
+      - ansible-test - Removed support for the remote Windows targets 2012 and 2012-R2
+      - inventory_cache - remove deprecated ``default.fact_caching_prefix`` ini configuration
+        option, use ``defaults.fact_caching_prefix`` instead.
+      - module_utils/basic.py - Removed Python 3.5 as a supported remote version.
+        Python 2.7 or Python 3.6+ is now required.
+      - stat - removed unused `get_md5` parameter.
+    codename: All My Love
+    fragments:
+    - 2.16.0b1_summary.yaml
+    - 20802-until-default.yml
+    - 22396-indicate-which-args-are-multi.yml
+    - 27816-fetch-unreachable.yml
+    - 50603-tty-check.yaml
+    - 71916-user-expires-int.yml
+    - 73643-handlers-prevent-multiple-runs.yml
+    - 74723-support-wildcard-win_fetch.yml
+    - 75063-allow-dev-nul-as-skeleton-for-new-homedir.yml
+    - 76372-fix-pip-virtualenv-command-parsing.yml
+    - 78487-galaxy-collections-path-warnings.yml
+    - 79129-ansible-managed-filename-format.yaml
+    - 79364_replace.yml
+    - 79677-fix-argspec-type-check.yml
+    - 79734-ansible-test-change-detection.yml
+    - 79844-fix-timeout-mounts-linux.yml
+    - 79999-ansible-user-tweak-macos-defaults.yaml
+    - 80089-prevent-module-build-date-issue.yml
+    - 80128-symbolic-modes-X-use-computed.yml
+    - 80257-iptables-chain-creation-does-not-populate-a-rule.yml
+    - 80258-defensive-display-non-utf8.yml
+    - 80334-reduce-ansible-galaxy-api-calls.yml
+    - 80406-validate-modules-semantic-markup.yml
+    - 80449-fix-symbolic-mode-error-msg.yml
+    - 80459-handlers-nested-includes-vars.yml
+    - 80460-add-symbolic-links-with-dir.yml
+    - 80476-fix-loop-task-post-validation.yml
+    - 80488-pip-pkg-resources.yml
+    - 80506-syntax-check-playbook-only.yml
+    - 80520-fix-current-hostname-openbsd.yml
+    - 80523_-_adding_short_option_for_--ask-vault-pass.yml
+    - 80605-template-overlay-native-jinja.yml
+    - 80648-fix-ansible-galaxy-cache-signatures-bug.yml
+    - 80721-ansible-galaxy.yml
+    - 80738-abs-unarachive-src.yml
+    - 80841-display-type-annotation.yml
+    - 80880-register-handlers-immediately-if-iterating-handlers.yml
+    - 80887-dnf5-api-change.yml
+    - 80943-ansible-galaxy-collection-subdir-install.yml
+    - 80968-replace-deprecated-ast-attr.yml
+    - 80985-fix-smgl-family-mapping.yml
+    - 81005-use-overlay-overrides.yml
+    - 81013-handlers-listen-last-defined-only.yml
+    - 81029-connection-types.yml
+    - 81064-daemonize-fixes.yml
+    - 81082-deprecated-importlib-abc.yml
+    - 81083-add-blockinfile-append-and-prepend-new-line-options.yml
+    - 81104-inventory-script-plugin-raise-execution-error.yml
+    - 81319-cloudstack-test-container-bump-version.yml
+    - 81332-fix-pkg-mgr-in-kylin.yml
+    - 81450-list-filters.yml
+    - 81494-remove-duplicated-file-attribute-constant.yml
+    - 81555-add-warning-for-illegal-filenames-in-roles.yaml
+    - 81584-daemonize-follow-up-fixes.yml
+    - 81606-ansible-galaxy-collection-pre-releases.yml
+    - 81613-remove-unusued-private-lock.yml
+    - 81656-cf_readfp-deprecated.yml
+    - 81662-blockinfile-exc.yml
+    - 81722-handler-subdir-include_tasks.yml
+    - CleansingNodeVisitor-removal.yml
+    - a-g-col-install-directory-with-trailing-sep.yml
+    - a-g-col-prevent-reinstalling-satisfied-req.yml
+    - a_test_rmv_alpine_317.yml
+    - add-missing-cli-docs.yml
+    - ag-ignore-multiple-signature-statuses.yml
+    - ansible-galaxy-server-timeout.yml
+    - ansible-runtime-metadata-removal-date.yml
+    - ansible-test-added-fedora-38.yml
+    - ansible-test-argcomplete-3.yml
+    - ansible-test-atexit.yml
+    - ansible-test-coverage-update.yml
+    - ansible-test-default-containers.yml
+    - ansible-test-deprecated-cleanup.yml
+    - ansible-test-distro-containers.yml
+    - ansible-test-entry-points.yml
+    - ansible-test-explain-traceback.yml
+    - ansible-test-fedora-37.yml
+    - ansible-test-freebsd-bootstrap-setuptools.yml
+    - ansible-test-import-sanity-fix.yml
+    - ansible-test-layout-detection.yml
+    - ansible-test-long-timeout-fix.yml
+    - ansible-test-minimum-setuptools.yml
+    - ansible-test-nios-container.yml
+    - ansible-test-pylint-update.yml
+    - ansible-test-pytest-forked.yml
+    - ansible-test-python-3.12.yml
+    - ansible-test-pyyaml-build.yml
+    - ansible-test-remove-old-rhel-remotes.yml
+    - ansible-test-remove-ubuntu-2004.yml
+    - ansible-test-rhel-9.2-python-3.11.yml
+    - ansible-test-rhel-9.2.yml
+    - ansible-test-sanity-scope.yml
+    - ansible-test-source-detection.yml
+    - ansible-test-thread-coverage.yml
+    - ansible-test-timeout-fix.yml
+    - ansible-test-unique-container-names.yml
+    - ansible-test-use-raise-from.yml
+    - ansible-test-utcnow.yml
+    - ansible-test-winrm-config.yml
+    - ansible-vault.yml
+    - ansible_test_alpine_3.18.yml
+    - apt_fail_on_autoremove.yml
+    - aptclean_diff.yml
+    - basestrategy-lazy-templar.yml
+    - ci_freebsd_new.yml
+    - collections_paths-deprecation.yml
+    - colors.yml
+    - command-expand-args.yml
+    - config_origins_option.yml
+    - connection-type-annotation.yml
+    - copy_diff.yml
+    - deb822_open_url.yml
+    - debconf.yml
+    - deprecated_string_conversion_action.yml
+    - display_proxy.yml
+    - dnf-update-only-latest.yml
+    - dnf5-cacheonly.yml
+    - dnf5-fix-interpreter-fail-msg.yml
+    - dnf5-gpg-check-api.yml
+    - dnf5-gpg-check-builtin.yml
+    - dnf5-logs-api.yml
+    - dnf5-test-env-groups.yml
+    - dotnet-preparation.yml
+    - dpkg_selections.yml
+    - fbsd13_1_remove.yml
+    - fetch_url-remove-auto-disable-decompress.yml
+    - find-mode.yml
+    - first_found_fixes.yml
+    - first_found_template_fix.yml
+    - fix-display-prompt-cpu-consumption.yml
+    - fix-handlers-callback.yml
+    - fix-pkg-mgr-in-TencentOS.yml
+    - fix-setuptools-warnings.yml
+    - fix-url-lookup-plugin-docs.yml
+    - forced_local+fix+.yml
+    - freebsd_12_4_removal.yml
+    - galaxy_check_type.yml
+    - galaxy_symlink.yml
+    - gather_facts_fix_parallel.yml
+    - get_action_args_with_defaults-remove-deprecated-arg.yml
+    - group_warning.yml
+    - inventory_cache-remove-deprecated-default-section.yml
+    - inventory_ini.yml
+    - jinja_plugin_cache_cleanup.yml
+    - long-collection-paths-fix.yml
+    - man-page-build-docs-dependency.yml
+    - man-page-subcommands.yml
+    - manifest-in-cleanup.yml
+    - mc_from_config.yml
+    - missing-doc-func.yml
+    - no-arbitrary-j2-override.yml
+    - omit-man-pages-from-sdist.yml
+    - parsing-splitter-fixes.yml
+    - passlib_or_crypt.yml
+    - password_hash-fix-crypt-salt-bcrypt.yml
+    - pep517-backend-import-fix.yml
+    - pep517-backend-traceback-fix.yml
+    - pep8-known-issue.yml
+    - persist_skip.yml
+    - pkg_mgr-default-dnf.yml
+    - powershell-module-error-handling.yml
+    - pre-release-hint-for-dep-resolution-error.yml
+    - pylint-deprecated-comment-checker.yml
+    - reboot.yml
+    - remove-deprecated-actionbase-_remote_checksum.yml
+    - remove-deprecated-datetime-methods.yml
+    - remove-deprecated-filelock-class.yml
+    - remove-docs-examples.yml
+    - remove-include.yml
+    - remove-play_iterator-deprecated-methods.yml
+    - remove-python3.5.yml
+    - remove-python3.9-controller-support.yml
+    - remove-templar-shared_loader_obj-arg.yml
+    - remove-unreachable-include_role-static-err.yml
+    - remove_md5.yml
+    - role-deduplication-condition.yml
+    - run-command-selectors-prompt-only.yml
+    - server2012-deprecation.yml
+    - service_facts_rcctl.yml
+    - service_facts_simpleinit_msb.yml
+    - service_fix_obsd.yml
+    - set-filters.yml
+    - setup_facter_fix.yml
+    - simple-result-queue.yml
+    - smart_connection_bye.yml
+    - suppressed-options.yml
+    - tarfile_extract_warn.yml
+    - templar-globals-dict.yml
+    - templating_fixes.yml
+    - text-converters.yml
+    - timeout_config_fix.yml
+    - update-maybe-json-uri.yml
+    - urls-client-cert-py12.yml
+    - urls-unit-test-latest-cryptography.yml
+    - user-add-password-exp-warning.yml
+    - v2.16.0-initial-commit.yaml
+    - vault_unvault_id_fix.yml
+    - yum-repository-docs-fixes.yml
+    - yum_repository_keepcache.yml
+    release_date: '2023-09-26'
+  2.16.0b2:
+    changes:
+      bugfixes:
+      - '``import_role`` reverts to previous behavior of exporting vars at compile
+        time.'
+      - ansible-galaxy info - fix reporting no role found when lookup_role_by_name
+        returns None.
+      - uri/urls - Add compat function to handle the ability to parse the filename
+        from a Content-Disposition header (https://github.com/ansible/ansible/issues/81806)
+      - winrm - Better handle send input failures when communicating with hosts under
+        load
+      minor_changes:
+      - ansible-test - When invoking ``sleep`` in containers during container setup,
+        the ``env`` command is used to avoid invoking the shell builtin, if present.
+      release_summary: '| Release Date: 2023-10-03
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+        '
+      security_fixes:
+      - ansible-galaxy - Prevent roles from using symlinks to overwrite files outside
+        of the installation directory (CVE-2023-5115)
+    codename: All My Love
+    fragments:
+    - 2.16.0b2_summary.yaml
+    - 81806-py2-content-disposition.yml
+    - ansible-test-container-sleep.yml
+    - cve-2023-5115.yml
+    - fix-ansible-galaxy-info-no-role-found.yml
+    - import_role_goes_public.yml
+    - winrm-send-input.yml
+    release_date: '2023-10-03'
+  2.16.0rc1:
+    changes:
+      bugfixes:
+      - Cache host_group_vars after instantiating it once and limit the amount of
+        repetitive work it needs to do every time it runs.
+      - Call PluginLoader.all() once for vars plugins, and load vars plugins that
+        run automatically or are enabled specifically by name subsequently.
+      - Fix ``run_once`` being incorrectly interpreted on handlers (https://github.com/ansible/ansible/issues/81666)
+      - Properly template tags in parent blocks (https://github.com/ansible/ansible/issues/81053)
+      - ansible-galaxy - Provide a better error message when using a requirements
+        file with an invalid format - https://github.com/ansible/ansible/issues/81901
+      - ansible-inventory - index available_hosts for major performance boost when
+        dumping large inventories
+      - ansible-test - Add a ``pylint`` plugin to work around a known issue on Python
+        3.12.
+      - ansible-test - Include missing ``pylint`` requirements for Python 3.10.
+      - ansible-test - Update ``pylint`` to version 3.0.1.
+      deprecated_features:
+      - Old style vars plugins which use the entrypoints `get_host_vars` or `get_group_vars`
+        are deprecated. The plugin should be updated to inherit from `BaseVarsPlugin`
+        and define a `get_vars` method as the entrypoint.
+      minor_changes:
+      - ansible-test - Make Python 3.12 the default version used in the ``base`` and
+        ``default`` containers.
+      release_summary: '| Release Date: 2023-10-16
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+        '
+    codename: All My Love
+    fragments:
+    - 2.16.0rc1_summary.yaml
+    - 79945-host_group_vars-improvements.yml
+    - 81053-templated-tags-inheritance.yml
+    - 81666-handlers-run_once.yml
+    - 81901-galaxy-requirements-format.yml
+    - ansible-test-pylint3-update.yml
+    - ansible-test-python-3.12-compat.yml
+    - ansible-test-python-default.yml
+    - inv_available_hosts_to_frozenset.yml
+    release_date: '2023-10-16'
+  2.16.1:
+    changes:
+      release_summary: '| Release Date: 2023-12-04
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+        '
+    codename: All My Love
+    fragments:
+    - 2.16.1_summary.yaml
+    release_date: '2023-12-04'
+  2.16.1rc1:
+    changes:
+      breaking_changes:
+      - assert - Nested templating may result in an inability for the conditional
+        to be evaluated. See the porting guide for more information.
+      bugfixes:
+      - Fix issue where an ``include_tasks`` handler in a role was not able to locate
+        a file in ``tasks/`` when ``tasks_from`` was used as a role entry point and
+        ``main.yml`` was not present (https://github.com/ansible/ansible/issues/82241)
+      - Plugin loader does not dedupe nor cache filter/test plugins by file basename,
+        but full path name.
+      - Restoring the ability of filters/tests can have same file base name but different
+        tests/filters defined inside.
+      - ansible-pull now will expand relative paths for the ``-d|--directory`` option
+        is now expanded before use.
+      - ansible-pull will now correctly handle become and connection password file
+        options for ansible-playbook.
+      - flush_handlers - properly handle a handler failure in a nested block when
+        ``force_handlers`` is set (http://github.com/ansible/ansible/issues/81532)
+      - module no_log will no longer affect top level booleans, for example ``no_log_module_parameter='a'``
+        will no longer hide ``changed=False`` as a 'no log value' (matches 'a').
+      - role params now have higher precedence than host facts again, matching documentation,
+        this had unintentionally changed in 2.15.
+      - wait_for should not handle 'non mmapable files' again.
+      release_summary: '| Release Date: 2023-11-27
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+        '
+      security_fixes:
+      - templating - Address issues where internal templating can cause unsafe variables
+        to lose their unsafe designation (CVE-2023-5764)
+    codename: All My Love
+    fragments:
+    - 2.16.1rc1_summary.yaml
+    - 81532-fix-nested-flush_handlers.yml
+    - 82241-handler-include-tasks-from.yml
+    - cve-2023-5764.yml
+    - j2_load_fix.yml
+    - no_log_booly.yml
+    - pull_file_secrets.yml
+    - pull_unfrack_dest.yml
+    - restore_role_param_precedence.yml
+    - wait_for_mmap.yml
+    release_date: '2023-11-27'
+  2.16.2:
+    changes:
+      bugfixes:
+      - unsafe data - Address an incompatibility when iterating or getting a single
+        index from ``AnsibleUnsafeBytes``
+      - unsafe data - Address an incompatibility with ``AnsibleUnsafeText`` and ``AnsibleUnsafeBytes``
+        when pickling with ``protocol=0``
+      release_summary: '| Release Date: 2023-12-11
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
+
+        '
+    codename: All My Love
+    fragments:
+    - 2.16.2_summary.yaml
+    - unsafe-fixes-2.yml
+    release_date: '2023-12-11'
--- a/changelogs/fragments/2.16.0_summary.yaml
+++ b/changelogs/fragments/2.16.0_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2023-11-06
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
--- a/changelogs/fragments/2.16.0b1_summary.yaml
+++ b/changelogs/fragments/2.16.0b1_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2023-09-26
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
--- a/changelogs/fragments/2.16.0b2_summary.yaml
+++ b/changelogs/fragments/2.16.0b2_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2023-10-03
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
--- a/changelogs/fragments/2.16.0rc1_summary.yaml
+++ b/changelogs/fragments/2.16.0rc1_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2023-10-16
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
--- a/changelogs/fragments/2.16.1_summary.yaml
+++ b/changelogs/fragments/2.16.1_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2023-12-04
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
--- a/changelogs/fragments/2.16.1rc1_summary.yaml
+++ b/changelogs/fragments/2.16.1rc1_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2023-11-27
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
--- a/changelogs/fragments/2.16.2_summary.yaml
+++ b/changelogs/fragments/2.16.2_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2023-12-11
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.16/porting_guides/porting_guide_core_2.16.html>`__
--- a/changelogs/fragments/79945-host_group_vars-improvements.yml
+++ b/changelogs/fragments/79945-host_group_vars-improvements.yml
@ -0,0 +1,5 @@
+bugfixes:
+  - Cache host_group_vars after instantiating it once and limit the amount of repetitive work it needs to do every time it runs.
+  - Call PluginLoader.all() once for vars plugins, and load vars plugins that run automatically or are enabled specifically by name subsequently.
+deprecated_features:
+  - Old style vars plugins which use the entrypoints `get_host_vars` or `get_group_vars` are deprecated. The plugin should be updated to inherit from `BaseVarsPlugin` and define a `get_vars` method as the entrypoint.
--- a/changelogs/fragments/81053-templated-tags-inheritance.yml
+++ b/changelogs/fragments/81053-templated-tags-inheritance.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - Properly template tags in parent blocks (https://github.com/ansible/ansible/issues/81053)
--- a/changelogs/fragments/81532-fix-nested-flush_handlers.yml
+++ b/changelogs/fragments/81532-fix-nested-flush_handlers.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - flush_handlers - properly handle a handler failure in a nested block when ``force_handlers`` is set (http://github.com/ansible/ansible/issues/81532)
--- a/changelogs/fragments/81584-daemonize-follow-up-fixes.yml
+++ b/changelogs/fragments/81584-daemonize-follow-up-fixes.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - "``ansible.module_utils.service`` - ensure binary data transmission in ``daemonize()``"
--- a/changelogs/fragments/81606-ansible-galaxy-collection-pre-releases.yml
+++ b/changelogs/fragments/81606-ansible-galaxy-collection-pre-releases.yml
@ -0,0 +1,16 @@
+---
+
+bugfixes:
+- >-
+  ansible-galaxy - started allowing the use of pre-releases
+  for dependencies on any level of the dependency tree that
+  specifically demand exact pre-release versions of
+  collections and not version ranges.
+  (https://github.com/ansible/ansible/pull/81606)
+- >-
+  ansible-galaxy - started allowing the use of pre-releases
+  for collections that do not have any stable versions
+  published.
+  (https://github.com/ansible/ansible/pull/81606)
+
+...
--- a/changelogs/fragments/81666-handlers-run_once.yml
+++ b/changelogs/fragments/81666-handlers-run_once.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - Fix ``run_once`` being incorrectly interpreted on handlers (https://github.com/ansible/ansible/issues/81666)
--- a/changelogs/fragments/81722-handler-subdir-include_tasks.yml
+++ b/changelogs/fragments/81722-handler-subdir-include_tasks.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - Allow for searching handler subdir for included task via include_role (https://github.com/ansible/ansible/issues/81722)
--- a/changelogs/fragments/81806-py2-content-disposition.yml
+++ b/changelogs/fragments/81806-py2-content-disposition.yml
@ -0,0 +1,3 @@
+bugfixes:
+- uri/urls - Add compat function to handle the ability to parse the filename from a Content-Disposition header
+  (https://github.com/ansible/ansible/issues/81806)
--- a/changelogs/fragments/81901-galaxy-requirements-format.yml
+++ b/changelogs/fragments/81901-galaxy-requirements-format.yml
@ -0,0 +1,2 @@
+bugfixes:
+- ansible-galaxy - Provide a better error message when using a requirements file with an invalid format - https://github.com/ansible/ansible/issues/81901
--- a/changelogs/fragments/82241-handler-include-tasks-from.yml
+++ b/changelogs/fragments/82241-handler-include-tasks-from.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - Fix issue where an ``include_tasks`` handler in a role was not able to locate a file in ``tasks/`` when ``tasks_from`` was used as a role entry point and ``main.yml`` was not present (https://github.com/ansible/ansible/issues/82241)
--- a/changelogs/fragments/ansible-test-cgroup-split.yml
+++ b/changelogs/fragments/ansible-test-cgroup-split.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-test - Fix parsing of cgroup entries which contain a ``:`` in the path (https://github.com/ansible/ansible/issues/81977).
--- a/changelogs/fragments/ansible-test-container-sleep.yml
+++ b/changelogs/fragments/ansible-test-container-sleep.yml
@ -0,0 +1,3 @@
+minor_changes:
+- ansible-test - When invoking ``sleep`` in containers during container setup, the ``env`` command is used to avoid invoking
+                 the shell builtin, if present.
--- a/changelogs/fragments/ansible-test-coverage-update.yml
+++ b/changelogs/fragments/ansible-test-coverage-update.yml
@ -1,7 +1,7 @@
 ---

 minor_changes:
- ansible-test — Python 3.8–3.12 will use ``coverage`` v7.3.0.
+- ansible-test — Python 3.8–3.12 will use ``coverage`` v7.3.2.
 - ansible-test — ``coverage`` v6.5.0 is to be used only under Python 3.7.

 ...
--- a/changelogs/fragments/ansible-test-distro-containers.yml
+++ b/changelogs/fragments/ansible-test-distro-containers.yml
@ -1,2 +1,4 @@
 minor_changes:
  - ansible-test - The openSUSE test container has been updated to openSUSE Leap 15.5.
+  - ansible-test - Updated the distro test containers to version 6.3.0 to include coverage 7.3.2 for Python 3.8+.
+                   The alpine3 container is now based on 3.18 instead of 3.17 and includes Python 3.11 instead of Python 3.10.
--- a/changelogs/fragments/ansible-test-pylint3-update.yml
+++ b/changelogs/fragments/ansible-test-pylint3-update.yml
@ -0,0 +1,3 @@
+bugfixes:
+  - ansible-test - Update ``pylint`` to version 3.0.1.
+  - ansible-test - Include missing ``pylint`` requirements for Python 3.10.
--- a/changelogs/fragments/ansible-test-python-3.12-compat.yml
+++ b/changelogs/fragments/ansible-test-python-3.12-compat.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-test - Add a ``pylint`` plugin to work around a known issue on Python 3.12.
--- a/changelogs/fragments/ansible-test-python-default.yml
+++ b/changelogs/fragments/ansible-test-python-default.yml
@ -0,0 +1,2 @@
+minor_changes:
+  - ansible-test - Make Python 3.12 the default version used in the ``base`` and ``default`` containers.
--- a/changelogs/fragments/cve-2023-5115.yml
+++ b/changelogs/fragments/cve-2023-5115.yml
@ -0,0 +1,3 @@
+security_fixes:
+- ansible-galaxy - Prevent roles from using symlinks to overwrite
+  files outside of the installation directory (CVE-2023-5115)
--- a/changelogs/fragments/cve-2023-5764.yml
+++ b/changelogs/fragments/cve-2023-5764.yml
@ -0,0 +1,6 @@
+security_fixes:
+- templating - Address issues where internal templating can cause unsafe
+  variables to lose their unsafe designation (CVE-2023-5764)
+breaking_changes:
+- assert - Nested templating may result in an inability for the conditional
+  to be evaluated. See the porting guide for more information.
--- a/changelogs/fragments/fix-ansible-galaxy-info-no-role-found.yml
+++ b/changelogs/fragments/fix-ansible-galaxy-info-no-role-found.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-galaxy info - fix reporting no role found when lookup_role_by_name returns None.
--- a/changelogs/fragments/import_role_goes_public.yml
+++ b/changelogs/fragments/import_role_goes_public.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - "``import_role`` reverts to previous behavior of exporting vars at compile time."
--- a/changelogs/fragments/inv_available_hosts_to_frozenset.yml
+++ b/changelogs/fragments/inv_available_hosts_to_frozenset.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-inventory - index available_hosts for major performance boost when dumping large inventories
--- a/changelogs/fragments/j2_load_fix.yml
+++ b/changelogs/fragments/j2_load_fix.yml
@ -0,0 +1,3 @@
+bugfixes:
+  - Plugin loader does not dedupe nor cache filter/test plugins by file basename, but full path name.
+  - Restoring the ability of filters/tests can have same file base name but different tests/filters defined inside.
--- a/changelogs/fragments/no_log_booly.yml
+++ b/changelogs/fragments/no_log_booly.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - module no_log will no longer affect top level booleans, for example ``no_log_module_parameter='a'`` will no longer hide ``changed=False`` as a 'no log value' (matches 'a').
--- a/changelogs/fragments/pull_file_secrets.yml
+++ b/changelogs/fragments/pull_file_secrets.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-pull will now correctly handle become and connection password file options for ansible-playbook.
--- a/changelogs/fragments/pull_unfrack_dest.yml
+++ b/changelogs/fragments/pull_unfrack_dest.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-pull now will expand relative paths for the ``-d|--directory`` option is now expanded before use.
--- a/changelogs/fragments/restore_role_param_precedence.yml
+++ b/changelogs/fragments/restore_role_param_precedence.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - role params now have higher precedence than host facts again, matching documentation, this had unintentionally changed in 2.15.
--- a/changelogs/fragments/unsafe-fixes-2.yml
+++ b/changelogs/fragments/unsafe-fixes-2.yml
@ -0,0 +1,3 @@
+bugfixes:
+- unsafe data - Address an incompatibility with ``AnsibleUnsafeText`` and ``AnsibleUnsafeBytes`` when pickling with ``protocol=0``
+- unsafe data - Address an incompatibility when iterating or getting a single index from ``AnsibleUnsafeBytes``
--- a/changelogs/fragments/wait_for_mmap.yml
+++ b/changelogs/fragments/wait_for_mmap.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - wait_for should not handle 'non mmapable files' again.
--- a/changelogs/fragments/winrm-send-input.yml
+++ b/changelogs/fragments/winrm-send-input.yml
@ -0,0 +1,2 @@
+bugfixes:
+- winrm - Better handle send input failures when communicating with hosts under load
--- a/lib/ansible/cli/galaxy.py
+++ b/lib/ansible/cli/galaxy.py
@ -835,7 +835,7 @@ class GalaxyCLI(CLI):
            for role_req in file_requirements:
                requirements['roles'] += parse_role_req(role_req)

-        else:
+        elif isinstance(file_requirements, dict):
            # Newer format with a collections and/or roles key
            extra_keys = set(file_requirements.keys()).difference(set(['roles', 'collections']))
            if extra_keys:
@ -854,6 +854,9 @@ class GalaxyCLI(CLI):
                for collection_req in file_requirements.get('collections') or []
            ]

+        else:
+            raise AnsibleError(f"Expecting requirements yaml to be a list or dictionary but got {type(file_requirements).__name__}")
+
        return requirements

    def _init_coll_req_dict(self, coll_req):
@ -1265,6 +1268,9 @@ class GalaxyCLI(CLI):

                if remote_data:
                    role_info.update(remote_data)
+                else:
+                    data = u"- the role %s was not found" % role
+                    break

            elif context.CLIARGS['offline'] and not gr._exists:
                data = u"- the role %s was not found" % role
--- a/lib/ansible/cli/inventory.py
+++ b/lib/ansible/cli/inventory.py
@ -325,7 +325,7 @@ class InventoryCLI(CLI):
            return results

        hosts = self.inventory.get_hosts(top.name)
-        results = format_group(top, [h.name for h in hosts])
+        results = format_group(top, frozenset(h.name for h in hosts))

        # populate meta
        results['_meta'] = {'hostvars': {}}
@ -381,7 +381,7 @@ class InventoryCLI(CLI):

            return results

-        available_hosts = [h.name for h in self.inventory.get_hosts(top.name)]
+        available_hosts = frozenset(h.name for h in self.inventory.get_hosts(top.name))
        return format_group(top, available_hosts)

    def toml_inventory(self, top):
@ -425,7 +425,7 @@ class InventoryCLI(CLI):

            return results

-        available_hosts = [h.name for h in self.inventory.get_hosts(top.name)]
+        available_hosts = frozenset(h.name for h in self.inventory.get_hosts(top.name))
        results = format_group(top, available_hosts)

        return results
--- a/lib/ansible/cli/pull.py
+++ b/lib/ansible/cli/pull.py
@ -29,6 +29,7 @@ from ansible.plugins.loader import module_loader
 from ansible.utils.cmd_functions import run_cmd
 from ansible.utils.display import Display

+
 display = Display()


@ -102,8 +103,8 @@ class PullCLI(CLI):
                                      'This is a useful way to disperse git requests')
        self.parser.add_argument('-f', '--force', dest='force', default=False, action='store_true',
                                 help='run the playbook even if the repository could not be updated')
-        self.parser.add_argument('-d', '--directory', dest='dest', default=None,
-                                 help='absolute path of repository checkout directory (relative paths are not supported)')
+        self.parser.add_argument('-d', '--directory', dest='dest', default=None, type=opt_help.unfrack_path(),
+                                 help='path to the directory to which Ansible will checkout the repository.')
        self.parser.add_argument('-U', '--url', dest='url', default=None, help='URL of the playbook repository')
        self.parser.add_argument('--full', dest='fullclone', action='store_true', help='Do a full clone, instead of a shallow one.')
        self.parser.add_argument('-C', '--checkout', dest='checkout',
@ -134,7 +135,6 @@ class PullCLI(CLI):
            hostname = socket.getfqdn()
            # use a hostname dependent directory, in case of $HOME on nfs
            options.dest = os.path.join(C.ANSIBLE_HOME, 'pull', hostname)
-        options.dest = os.path.expandvars(os.path.expanduser(options.dest))

        if os.path.exists(options.dest) and not os.path.isdir(options.dest):
            raise AnsibleOptionsError("%s is not a valid or accessible directory." % options.dest)
@ -275,8 +275,15 @@ class PullCLI(CLI):
            for vault_id in context.CLIARGS['vault_ids']:
                cmd += " --vault-id=%s" % vault_id

+        if context.CLIARGS['become_password_file']:
+            cmd += " --become-password-file=%s" % context.CLIARGS['become_password_file']
+
+        if context.CLIARGS['connection_password_file']:
+            cmd += " --connection-password-file=%s" % context.CLIARGS['connection_password_file']
+
        for ev in context.CLIARGS['extra_vars']:
            cmd += ' -e %s' % shlex.quote(ev)
+
        if context.CLIARGS['become_ask_pass']:
            cmd += ' --ask-become-pass'
        if context.CLIARGS['skip_tags']:
@ -307,6 +314,7 @@ class PullCLI(CLI):
        if context.CLIARGS['purge']:
            os.chdir('/')
            try:
+                display.debug("removing: %s" % context.CLIARGS['dest'])
                shutil.rmtree(context.CLIARGS['dest'])
            except Exception as e:
                display.error(u"Failed to remove %s: %s" % (context.CLIARGS['dest'], to_text(e)))
--- a/lib/ansible/executor/play_iterator.py
+++ b/lib/ansible/executor/play_iterator.py
@ -52,7 +52,7 @@ class FailedStates(IntFlag):
    TASKS = 2
    RESCUE = 4
    ALWAYS = 8
-    HANDLERS = 16
+    HANDLERS = 16  # NOTE not in use anymore


 class HostState:
@ -431,22 +431,18 @@ class PlayIterator:
                    state.update_handlers = False
                    state.cur_handlers_task = 0

-                if state.fail_state & FailedStates.HANDLERS == FailedStates.HANDLERS:
-                    state.update_handlers = True
-                    state.run_state = IteratingStates.COMPLETE
-                else:
-                    while True:
-                        try:
-                            task = state.handlers[state.cur_handlers_task]
-                        except IndexError:
-                            task = None
-                            state.run_state = state.pre_flushing_run_state
-                            state.update_handlers = True
+                while True:
+                    try:
+                        task = state.handlers[state.cur_handlers_task]
+                    except IndexError:
+                        task = None
+                        state.run_state = state.pre_flushing_run_state
+                        state.update_handlers = True
+                        break
+                    else:
+                        state.cur_handlers_task += 1
+                        if task.is_host_notified(host):
                            break
-                        else:
-                            state.cur_handlers_task += 1
-                            if task.is_host_notified(host):
-                                break

            elif state.run_state == IteratingStates.COMPLETE:
                return (state, None)
@ -487,20 +483,16 @@ class PlayIterator:
            else:
                state.fail_state |= FailedStates.ALWAYS
                state.run_state = IteratingStates.COMPLETE
-        elif state.run_state == IteratingStates.HANDLERS:
-            state.fail_state |= FailedStates.HANDLERS
-            state.update_handlers = True
-            if state._blocks[state.cur_block].rescue:
-                state.run_state = IteratingStates.RESCUE
-            elif state._blocks[state.cur_block].always:
-                state.run_state = IteratingStates.ALWAYS
-            else:
-                state.run_state = IteratingStates.COMPLETE
        return state

    def mark_host_failed(self, host):
        s = self.get_host_state(host)
        display.debug("marking host %s failed, current state: %s" % (host, s))
+        if s.run_state == IteratingStates.HANDLERS:
+            # we are failing `meta: flush_handlers`, so just reset the state to whatever
+            # it was before and let `_set_failed_state` figure out the next state
+            s.run_state = s.pre_flushing_run_state
+            s.update_handlers = True
        s = self._set_failed_state(s)
        display.debug("^ failed state is now: %s" % s)
        self.set_state_for_host(host.name, s)
@ -516,8 +508,6 @@ class PlayIterator:
            return True
        elif state.run_state == IteratingStates.ALWAYS and self._check_failed_state(state.always_child_state):
            return True
-        elif state.run_state == IteratingStates.HANDLERS and state.fail_state & FailedStates.HANDLERS == FailedStates.HANDLERS:
-            return True
        elif state.fail_state != FailedStates.NONE:
            if state.run_state == IteratingStates.RESCUE and state.fail_state & FailedStates.RESCUE == 0:
                return False
--- a/lib/ansible/galaxy/collection/init.py
+++ b/lib/ansible/galaxy/collection/init.py
@ -1817,15 +1817,15 @@ def _resolve_depenency_map(
        elif not req.specifier.contains(RESOLVELIB_VERSION.vstring):
            raise AnsibleError(f"ansible-galaxy requires {req.name}{req.specifier}")

-    if allow_pre_release:
-        pre_release_hint = ''
-    else:
-        pre_release_hint = 'Hint: Pre-releases are not installed by default unless the specific version is given. To enable pre-releases, use --pre.'
+    pre_release_hint = '' if allow_pre_release else (
+        'Hint: Pre-releases hosted on Galaxy or Automation Hub are not '
+        'installed by default unless a specific version is requested. '
+        'To enable pre-releases globally, use --pre.'
+    )

    collection_dep_resolver = build_collection_dependency_resolver(
        galaxy_apis=galaxy_apis,
        concrete_artifacts_manager=concrete_artifacts_manager,
-        user_requirements=requested_requirements,
        preferred_candidates=preferred_candidates,
        with_deps=not no_deps,
        with_pre_releases=allow_pre_release,
--- a/lib/ansible/galaxy/dependency_resolution/init.py
+++ b/lib/ansible/galaxy/dependency_resolution/init.py
@ -13,10 +13,7 @@ if t.TYPE_CHECKING:
    from ansible.galaxy.collection.concrete_artifact_manager import (
        ConcreteArtifactsManager,
    )
-    from ansible.galaxy.dependency_resolution.dataclasses import (
-        Candidate,
-        Requirement,
-    )
+    from ansible.galaxy.dependency_resolution.dataclasses import Candidate

 from ansible.galaxy.collection.galaxy_api_proxy import MultiGalaxyAPIProxy
 from ansible.galaxy.dependency_resolution.providers import CollectionDependencyProvider
@ -27,7 +24,6 @@ from ansible.galaxy.dependency_resolution.resolvers import CollectionDependencyR
 def build_collection_dependency_resolver(
        galaxy_apis,  # type: t.Iterable[GalaxyAPI]
        concrete_artifacts_manager,  # type: ConcreteArtifactsManager
-        user_requirements,  # type: t.Iterable[Requirement]
        preferred_candidates=None,  # type: t.Iterable[Candidate]
        with_deps=True,  # type: bool
        with_pre_releases=False,  # type: bool
@ -44,7 +40,6 @@ def build_collection_dependency_resolver(
        CollectionDependencyProvider(
            apis=MultiGalaxyAPIProxy(galaxy_apis, concrete_artifacts_manager, offline=offline),
            concrete_artifacts_manager=concrete_artifacts_manager,
-            user_requirements=user_requirements,
            preferred_candidates=preferred_candidates,
            with_deps=with_deps,
            with_pre_releases=with_pre_releases,
--- a/lib/ansible/galaxy/dependency_resolution/dataclasses.py
+++ b/lib/ansible/galaxy/dependency_resolution/dataclasses.py
@ -564,6 +564,27 @@ class _ComputedReqKindsMixin:
    def is_online_index_pointer(self):
        return not self.is_concrete_artifact

+    @property
+    def is_pinned(self):
+        """Indicate if the version set is considered pinned.
+
+        This essentially computes whether the version field of the current
+        requirement explicitly requests a specific version and not an allowed
+        version range.
+
+        It is then used to help the resolvelib-based dependency resolver judge
+        whether it's acceptable to consider a pre-release candidate version
+        despite pre-release installs not being requested by the end-user
+        explicitly.
+
+        See https://github.com/ansible/ansible/pull/81606 for extra context.
+        """
+        version_string = self.ver[0]
+        return version_string.isdigit() or not (
+            version_string == '*' or
+            version_string.startswith(('<', '>', '!='))
+        )
+
    @property
    def source_info(self):
        return self._source_info
--- a/lib/ansible/galaxy/dependency_resolution/providers.py
+++ b/lib/ansible/galaxy/dependency_resolution/providers.py
@ -51,7 +51,6 @@ class CollectionDependencyProviderBase(AbstractProvider):
            self,  # type: CollectionDependencyProviderBase
            apis,  # type: MultiGalaxyAPIProxy
            concrete_artifacts_manager=None,  # type: ConcreteArtifactsManager
-            user_requirements=None,  # type: t.Iterable[Requirement]
            preferred_candidates=None,  # type: t.Iterable[Candidate]
            with_deps=True,  # type: bool
            with_pre_releases=False,  # type: bool
@ -87,58 +86,12 @@ class CollectionDependencyProviderBase(AbstractProvider):
            Requirement.from_requirement_dict,
            art_mgr=concrete_artifacts_manager,
        )
-        self._pinned_candidate_requests = set(
-            # NOTE: User-provided signatures are supplemental, so signatures
-            # NOTE: are not used to determine if a candidate is user-requested
-            Candidate(req.fqcn, req.ver, req.src, req.type, None)
-            for req in (user_requirements or ())
-            if req.is_concrete_artifact or (
-                req.ver != '*' and
-                not req.ver.startswith(('<', '>', '!='))
-            )
-        )
        self._preferred_candidates = set(preferred_candidates or ())
        self._with_deps = with_deps
        self._with_pre_releases = with_pre_releases
        self._upgrade = upgrade
        self._include_signatures = include_signatures

-    def _is_user_requested(self, candidate):  # type: (Candidate) -> bool
-        """Check if the candidate is requested by the user."""
-        if candidate in self._pinned_candidate_requests:
-            return True
-
-        if candidate.is_online_index_pointer and candidate.src is not None:
-            # NOTE: Candidate is a namedtuple, it has a source server set
-            # NOTE: to a specific GalaxyAPI instance or `None`. When the
-            # NOTE: user runs
-            # NOTE:
-            # NOTE:     $ ansible-galaxy collection install ns.coll
-            # NOTE:
-            # NOTE: then it's saved in `self._pinned_candidate_requests`
-            # NOTE: as `('ns.coll', '*', None, 'galaxy')` but then
-            # NOTE: `self.find_matches()` calls `self.is_satisfied_by()`
-            # NOTE: with Candidate instances bound to each specific
-            # NOTE: server available, those look like
-            # NOTE: `('ns.coll', '*', GalaxyAPI(...), 'galaxy')` and
-            # NOTE: wouldn't match the user requests saved in
-            # NOTE: `self._pinned_candidate_requests`. This is why we
-            # NOTE: normalize the collection to have `src=None` and try
-            # NOTE: again.
-            # NOTE:
-            # NOTE: When the user request comes from `requirements.yml`
-            # NOTE: with the `source:` set, it'll match the first check
-            # NOTE: but it still can have entries with `src=None` so this
-            # NOTE: normalized check is still necessary.
-            # NOTE:
-            # NOTE: User-provided signatures are supplemental, so signatures
-            # NOTE: are not used to determine if a candidate is user-requested
-            return Candidate(
-                candidate.fqcn, candidate.ver, None, candidate.type, None
-            ) in self._pinned_candidate_requests
-
-        return False
-
    def identify(self, requirement_or_candidate):
        # type: (t.Union[Candidate, Requirement]) -> str
        """Given requirement or candidate, return an identifier for it.
@ -342,25 +295,79 @@ class CollectionDependencyProviderBase(AbstractProvider):
        latest_matches = []
        signatures = []
        extra_signature_sources = []  # type: list[str]
+
+        discarding_pre_releases_acceptable = any(
+            not is_pre_release(candidate_version)
+            for candidate_version, _src_server in coll_versions
+        )
+
+        # NOTE: The optimization of conditionally looping over the requirements
+        # NOTE: is used to skip having to compute the pinned status of all
+        # NOTE: requirements and apply version normalization to the found ones.
+        all_pinned_requirement_version_numbers = {
+            # NOTE: Pinned versions can start with a number, but also with an
+            # NOTE: equals sign. Stripping it at the beginning should be
+            # NOTE: enough. If there's a space after equals, the second strip
+            # NOTE: will take care of it.
+            # NOTE: Without this conversion, requirements versions like
+            # NOTE: '1.2.3-alpha.4' work, but '=1.2.3-alpha.4' don't.
+            requirement.ver.lstrip('=').strip()
+            for requirement in requirements
+            if requirement.is_pinned
+        } if discarding_pre_releases_acceptable else set()
+
        for version, src_server in coll_versions:
            tmp_candidate = Candidate(fqcn, version, src_server, 'galaxy', None)

-            unsatisfied = False
            for requirement in requirements:
-                unsatisfied |= not self.is_satisfied_by(requirement, tmp_candidate)
+                candidate_satisfies_requirement = self.is_satisfied_by(
+                    requirement, tmp_candidate,
+                )
+                if not candidate_satisfies_requirement:
+                    break
+
+                should_disregard_pre_release_candidate = (
+                    # NOTE: Do not discard pre-release candidates in the
+                    # NOTE: following cases:
+                    # NOTE:   * the end-user requested pre-releases explicitly;
+                    # NOTE:   * the candidate is a concrete artifact (e.g. a
+                    # NOTE:     Git repository, subdirs, a tarball URL, or a
+                    # NOTE:     local dir or file etc.);
+                    # NOTE:   * the candidate's pre-release version exactly
+                    # NOTE:     matches a version specifically requested by one
+                    # NOTE:     of the requirements in the current match
+                    # NOTE:     discovery round (i.e. matching a requirement
+                    # NOTE:     that is not a range but an explicit specific
+                    # NOTE:     version pin). This works when some requirements
+                    # NOTE:     request version ranges but others (possibly on
+                    # NOTE:     different dependency tree level depths) demand
+                    # NOTE:     pre-release dependency versions, even if those
+                    # NOTE:     dependencies are transitive.
+                    is_pre_release(tmp_candidate.ver)
+                    and discarding_pre_releases_acceptable
+                    and not (
+                        self._with_pre_releases
+                        or tmp_candidate.is_concrete_artifact
+                        or version in all_pinned_requirement_version_numbers
+                    )
+                )
+                if should_disregard_pre_release_candidate:
+                    break
+
                # FIXME
-                # unsatisfied |= not self.is_satisfied_by(requirement, tmp_candidate) or not (
-                #    requirement.src is None or  # if this is true for some candidates but not all it will break key param - Nonetype can't be compared to str
+                # candidate_is_from_requested_source = (
+                #    requirement.src is None  # if this is true for some candidates but not all it will break key param - Nonetype can't be compared to str
                #    or requirement.src == candidate.src
                # )
-                if unsatisfied:
-                    break
+                # if not candidate_is_from_requested_source:
+                #     break
+
                if not self._include_signatures:
                    continue

                extra_signature_sources.extend(requirement.signature_sources or [])

-            if not unsatisfied:
+            else:  # candidate satisfies requirements, `break` never happened
                if self._include_signatures:
                    for extra_source in extra_signature_sources:
                        signatures.append(get_signature_from_source(extra_source))
@ -405,21 +412,6 @@ class CollectionDependencyProviderBase(AbstractProvider):
        :returns: Indication whether the `candidate` is a viable \
                  solution to the `requirement`.
        """
-        # NOTE: Only allow pre-release candidates if we want pre-releases
-        # NOTE: or the req ver was an exact match with the pre-release
-        # NOTE: version. Another case where we'd want to allow
-        # NOTE: pre-releases is when there are several user requirements
-        # NOTE: and one of them is a pre-release that also matches a
-        # NOTE: transitive dependency of another requirement.
-        allow_pre_release = self._with_pre_releases or not (
-            requirement.ver == '*' or
-            requirement.ver.startswith('<') or
-            requirement.ver.startswith('>') or
-            requirement.ver.startswith('!=')
-        ) or self._is_user_requested(candidate)
-        if is_pre_release(candidate.ver) and not allow_pre_release:
-            return False
-
        # NOTE: This is a set of Pipenv-inspired optimizations. Ref:
        # https://github.com/sarugaku/passa/blob/2ac00f1/src/passa/models/providers.py#L58-L74
        if (
--- a/lib/ansible/galaxy/role.py
+++ b/lib/ansible/galaxy/role.py
@ -394,30 +394,36 @@ class GalaxyRole(object):
                            # bits that might be in the file for security purposes
                            # and drop any containing directory, as mentioned above
                            if member.isreg() or member.issym():
-                                n_member_name = to_native(member.name)
-                                n_archive_parent_dir = to_native(archive_parent_dir)
-                                n_parts = n_member_name.replace(n_archive_parent_dir, "", 1).split(os.sep)
-                                n_final_parts = []
-                                for n_part in n_parts:
-                                    # TODO if the condition triggers it produces a broken installation.
-                                    # It will create the parent directory as an empty file and will
-                                    # explode if the directory contains valid files.
-                                    # Leaving this as is since the whole module needs a rewrite.
-                                    #
-                                    # Check if we have any files with illegal names,
-                                    # and display a warning if so. This could help users
-                                    # to debug a broken installation.
-                                    if n_part == '..':
-                                        display.warning(f"Illegal filename '{n_part}': '..' is not allowed")
+                                for attr in ('name', 'linkname'):
+                                    attr_value = getattr(member, attr, None)
+                                    if not attr_value:
                                        continue
-                                    if n_part.startswith('~'):
-                                        display.warning(f"Illegal filename '{n_part}': names cannot start with '~'")
-                                        continue
-                                    if '$' in n_part:
-                                        display.warning(f"Illegal filename '{n_part}': names cannot contain '$'")
-                                        continue
-                                    n_final_parts.append(n_part)
-                                member.name = os.path.join(*n_final_parts)
+                                    n_attr_value = to_native(attr_value)
+                                    n_archive_parent_dir = to_native(archive_parent_dir)
+                                    n_parts = n_attr_value.replace(n_archive_parent_dir, "", 1).split(os.sep)
+                                    n_final_parts = []
+                                    for n_part in n_parts:
+                                        # TODO if the condition triggers it produces a broken installation.
+                                        # It will create the parent directory as an empty file and will
+                                        # explode if the directory contains valid files.
+                                        # Leaving this as is since the whole module needs a rewrite.
+                                        #
+                                        # Check if we have any files with illegal names,
+                                        # and display a warning if so. This could help users
+                                        # to debug a broken installation.
+                                        if not n_part:
+                                            continue
+                                        if n_part == '..':
+                                            display.warning(f"Illegal filename '{n_part}': '..' is not allowed")
+                                            continue
+                                        if n_part.startswith('~'):
+                                            display.warning(f"Illegal filename '{n_part}': names cannot start with '~'")
+                                            continue
+                                        if '$' in n_part:
+                                            display.warning(f"Illegal filename '{n_part}': names cannot contain '$'")
+                                            continue
+                                        n_final_parts.append(n_part)
+                                    setattr(member, attr, os.path.join(*n_final_parts))

                                if _check_working_data_filter():
                                    # deprecated: description='extract fallback without filter' python_version='3.11'
--- a/lib/ansible/inventory/group.py
+++ b/lib/ansible/inventory/group.py
@ -18,6 +18,7 @@ from __future__ import (absolute_import, division, print_function)
 __metaclass__ = type

 from collections.abc import Mapping, MutableMapping
+from enum import Enum
 from itertools import chain

 from ansible import constants as C
@ -53,8 +54,14 @@ def to_safe_group_name(name, replacer="_", force=False, silent=False):
    return name


+class InventoryObjectType(Enum):
+    HOST = 0
+    GROUP = 1
+
+
 class Group:
    ''' a group of ansible hosts '''
+    base_type = InventoryObjectType.GROUP

    # __slots__ = [ 'name', 'hosts', 'vars', 'child_groups', 'parent_groups', 'depth', '_hosts_cache' ]

--- a/lib/ansible/inventory/host.py
+++ b/lib/ansible/inventory/host.py
@ -21,7 +21,7 @@ __metaclass__ = type

 from collections.abc import Mapping, MutableMapping

-from ansible.inventory.group import Group
+from ansible.inventory.group import Group, InventoryObjectType
 from ansible.parsing.utils.addresses import patterns
 from ansible.utils.vars import combine_vars, get_unique_id

@ -31,6 +31,7 @@ __all__ = ['Host']

 class Host:
    ''' a single ansible host '''
+    base_type = InventoryObjectType.HOST

    # __slots__ = [ 'name', 'vars', 'groups' ]

--- a/lib/ansible/keyword_desc.yml
+++ b/lib/ansible/keyword_desc.yml
@ -5,7 +5,7 @@ action: "The 'action' to execute for a task, it normally translates into a C(mod
 args: "A secondary way to add arguments into a task. Takes a dictionary in which keys map to options and values."
 always: List of tasks, in a block, that execute no matter if there is an error in the block or not.
 any_errors_fatal: Force any un-handled task errors on any host to propagate to all hosts and end the play.
-async: Run a task asynchronously if the C(action) supports this; value is maximum runtime in seconds.
+async: Run a task asynchronously if the C(action) supports this; the value is the maximum runtime in seconds.
 become: Boolean that controls if privilege escalation is used or not on :term:`Task` execution. Implemented by the become plugin. See :ref:`become_plugins`.
 become_exe: Path to the executable used to elevate privileges. Implemented by the become plugin. See :ref:`become_plugins`.
 become_flags: A string of flag(s) to pass to the privilege escalation program when :term:`become` is True.
@ -23,25 +23,25 @@ collections: |


 connection: Allows you to change the connection plugin used for tasks to execute on the target. See :ref:`using_connection`.
-debugger: Enable debugging tasks based on state of the task result. See :ref:`playbook_debugger`.
+debugger: Enable debugging tasks based on the state of the task result. See :ref:`playbook_debugger`.
 delay: Number of seconds to delay between retries. This setting is only used in combination with :term:`until`.
 delegate_facts: Boolean that allows you to apply facts to a delegated host instead of inventory_hostname.
 delegate_to: Host to execute task instead of the target (inventory_hostname). Connection vars from the delegated host will also be used for the task.
 diff: "Toggle to make tasks return 'diff' information or not."
-environment: A dictionary that gets converted into environment vars to be provided for the task upon execution. This can ONLY be used with modules. This isn't supported for any other type of plugins nor Ansible itself nor its configuration, it just sets the variables for the code responsible for executing the task. This is not a recommended way to pass in confidential data.
+environment: A dictionary that gets converted into environment vars to be provided for the task upon execution. This can ONLY be used with modules. This is not supported for any other type of plugins nor Ansible itself nor its configuration, it just sets the variables for the code responsible for executing the task. This is not a recommended way to pass in confidential data.
 fact_path: Set the fact path option for the fact gathering plugin controlled by :term:`gather_facts`.
 failed_when: "Conditional expression that overrides the task's normal 'failed' status."
 force_handlers: Will force notified handler execution for hosts even if they failed during the play. Will not trigger if the play itself fails.
 gather_facts: "A boolean that controls if the play will automatically run the 'setup' task to gather facts for the hosts."
-gather_subset: Allows you to pass subset options to the  fact gathering plugin controlled by :term:`gather_facts`.
+gather_subset: Allows you to pass subset options to the fact gathering plugin controlled by :term:`gather_facts`.
 gather_timeout: Allows you to set the timeout for the fact gathering plugin controlled by :term:`gather_facts`.
 handlers: "A section with tasks that are treated as handlers, these won't get executed normally, only when notified after each section of tasks is complete. A handler's `listen` field is not templatable."
 hosts: "A list of groups, hosts or host pattern that translates into a list of hosts that are the play's target."
 ignore_errors: Boolean that allows you to ignore task failures and continue with play. It does not affect connection errors.
 ignore_unreachable: Boolean that allows you to ignore task failures due to an unreachable host and continue with the play. This does not affect other task errors (see :term:`ignore_errors`) but is useful for groups of volatile/ephemeral hosts.
 loop: "Takes a list for the task to iterate over, saving each list element into the ``item`` variable (configurable via loop_control)"
-loop_control: Several keys here allow you to modify/set loop behaviour in a task. See :ref:`loop_control`.
-max_fail_percentage: can be used to abort the run after a given percentage of hosts in the current batch has failed. This only works on linear or linear derived strategies.
+loop_control: Several keys here allow you to modify/set loop behavior in a task. See :ref:`loop_control`.
+max_fail_percentage: can be used to abort the run after a given percentage of hosts in the current batch has failed. This only works on linear or linear-derived strategies.
 module_defaults: Specifies default parameter values for modules.
 name: "Identifier. Can be used for documentation, or in tasks/handlers."
 no_log: Boolean that controls information disclosure.
@ -56,13 +56,13 @@ register: Name of variable that will contain task status and module return data.
 rescue: List of tasks in a :term:`block` that run if there is a task error in the main :term:`block` list.
 retries: "Number of retries before giving up in a :term:`until` loop. This setting is only used in combination with :term:`until`."
 roles: List of roles to be imported into the play
-run_once: Boolean that will bypass the host loop, forcing the task to attempt to execute on the first host available and afterwards apply any results and facts to all active hosts in the same batch.
+run_once: Boolean that will bypass the host loop, forcing the task to attempt to execute on the first host available and afterward apply any results and facts to all active hosts in the same batch.
 serial: Explicitly define how Ansible batches the execution of the current play on the play's target. See :ref:`rolling_update_batch_size`.
-strategy: Allows you to choose the connection plugin to use for the play.
+strategy: Allows you to choose the strategy plugin to use for the play. See :ref:`strategy_plugins`.
 tags: Tags applied to the task or included tasks, this allows selecting subsets of tasks from the command line.
 tasks: Main list of tasks to execute in the play, they run after :term:`roles` and before :term:`post_tasks`.
-timeout: Time limit for task to execute in, if exceeded Ansible will interrupt and fail the task.
-throttle: Limit number of concurrent task runs on task, block and playbook level. This is independent of the forks and serial settings, but cannot be set higher than those limits. For example, if forks is set to 10 and the throttle is set to 15, at most 10 hosts will be operated on in parallel.
+timeout: Time limit for the task to execute in, if exceeded Ansible will interrupt and fail the task.
+throttle: Limit the number of concurrent task runs on task, block and playbook level. This is independent of the forks and serial settings, but cannot be set higher than those limits. For example, if forks is set to 10 and the throttle is set to 15, at most 10 hosts will be operated on in parallel.
 until: "This keyword implies a ':term:`retries` loop' that will go on until the condition supplied here is met or we hit the :term:`retries` limit."
 vars: Dictionary/map of variables
 vars_files: List of files that contain vars to include in the play.
--- a/lib/ansible/module_utils/basic.py
+++ b/lib/ansible/module_utils/basic.py
@ -1489,7 +1489,19 @@ class AnsibleModule(object):
        if deprecations:
            kwargs['deprecations'] = deprecations

+        # preserve bools/none from no_log
+        # TODO: once python version on target high enough, dict comprh
+        preserved = {}
+        for k, v in kwargs.items():
+            if v is None or isinstance(v, bool):
+                preserved[k] = v
+
+        # strip no_log collisions
        kwargs = remove_values(kwargs, self.no_log_values)
+
+        # return preserved
+        kwargs.update(preserved)
+
        print('\n%s' % self.jsonify(kwargs))

    def exit_json(self, **kwargs):
--- a/lib/ansible/module_utils/common/json.py
+++ b/lib/ansible/module_utils/common/json.py
@ -30,7 +30,7 @@ def _preprocess_unsafe_encode(value):
    Used in ``AnsibleJSONEncoder.iterencode``
    """
    if _is_unsafe(value):
-        value = {'__ansible_unsafe': to_text(value, errors='surrogate_or_strict', nonstring='strict')}
+        value = {'__ansible_unsafe': to_text(value._strip_unsafe(), errors='surrogate_or_strict', nonstring='strict')}
    elif is_sequence(value):
        value = [_preprocess_unsafe_encode(v) for v in value]
    elif isinstance(value, Mapping):
@ -63,7 +63,7 @@ class AnsibleJSONEncoder(json.JSONEncoder):
                value = {'__ansible_vault': to_text(o._ciphertext, errors='surrogate_or_strict', nonstring='strict')}
        elif getattr(o, '__UNSAFE__', False):
            # unsafe object, this will never be triggered, see ``AnsibleJSONEncoder.iterencode``
-            value = {'__ansible_unsafe': to_text(o, errors='surrogate_or_strict', nonstring='strict')}
+            value = {'__ansible_unsafe': to_text(o._strip_unsafe(), errors='surrogate_or_strict', nonstring='strict')}
        elif isinstance(o, Mapping):
            # hostvars and other objects
            value = dict(o)
--- a/lib/ansible/module_utils/service.py
+++ b/lib/ansible/module_utils/service.py
@ -215,9 +215,10 @@ def daemonize(module, cmd):
                for out in list(fds):
                    if out in rfd:
                        data = os.read(out.fileno(), chunk)
-                        if not data:
+                        if data:
+                            output[out] += to_bytes(data, errors=errors)
+                        else:
                            fds.remove(out)
-                        output[out] += data
            else:
                break

@ -248,7 +249,7 @@ def daemonize(module, cmd):
                data = os.read(pipe[0], chunk)
                if not data:
                    break
-                return_data += data
+                return_data += to_bytes(data, errors=errors)

        # Note: no need to specify encoding on py3 as this module sends the
        # pickle to itself (thus same python interpreter so we aren't mixing
--- a/lib/ansible/module_utils/urls.py
+++ b/lib/ansible/module_utils/urls.py
@ -129,13 +129,13 @@ if not HAS_SSLCONTEXT:
        try:
            from urllib3.contrib.pyopenssl import PyOpenSSLContext
        except Exception:
-            from requests.packages.urllib3.contrib.pyopenssl import PyOpenSSLContext
+            from requests.packages.urllib3.contrib.pyopenssl import PyOpenSSLContext  # type: ignore[no-redef]
        HAS_URLLIB3_PYOPENSSLCONTEXT = True
    except Exception:
        # urllib3<1.15,>=1.6
        try:
            try:
-                from urllib3.contrib.pyopenssl import ssl_wrap_socket
+                from urllib3.contrib.pyopenssl import ssl_wrap_socket  # type: ignore[attr-defined]
            except Exception:
                from requests.packages.urllib3.contrib.pyopenssl import ssl_wrap_socket
            HAS_URLLIB3_SSL_WRAP_SOCKET = True
@ -771,6 +771,18 @@ def extract_pem_certs(b_data):
        yield match.group(0)


+def _py2_get_param(headers, param, header='content-type'):
+    m = httplib.HTTPMessage(io.StringIO())
+    cd = headers.getheader(header) or ''
+    try:
+        m.plisttext = cd[cd.index(';'):]
+        m.parseplist()
+    except ValueError:
+        return None
+
+    return m.getparam(param)
+
+
 def get_response_filename(response):
    url = response.geturl()
    path = urlparse(url)[2]
@ -778,7 +790,12 @@ def get_response_filename(response):
    if filename:
        filename = unquote(filename)

-    return response.headers.get_param('filename', header='content-disposition') or filename
+    if PY2:
+        get_param = functools.partial(_py2_get_param, response.headers)
+    else:
+        get_param = response.headers.get_param
+
+    return get_param('filename', header='content-disposition') or filename


 def parse_content_type(response):
--- a/lib/ansible/modules/pip.py
+++ b/lib/ansible/modules/pip.py
@ -121,6 +121,8 @@ attributes:
    platform:
        platforms: posix
 notes:
+   - Python installations marked externally-managed (as defined by PEP668) cannot be updated by pip versions >= 23.0.1 without the use of
+     a virtual environment or setting the environment variable ``PIP_BREAK_SYSTEM_PACKAGES=1``.
   - The virtualenv (U(http://www.virtualenv.org/)) must be
     installed on the remote host if the virtualenv parameter is specified and
     the virtualenv needs to be created.
--- a/lib/ansible/modules/wait_for.py
+++ b/lib/ansible/modules/wait_for.py
@ -238,7 +238,7 @@ import traceback

 from ansible.module_utils.basic import AnsibleModule, missing_required_lib
 from ansible.module_utils.common.sys_info import get_platform_subclass
-from ansible.module_utils.common.text.converters import to_bytes
+from ansible.module_utils.common.text.converters import to_bytes, to_native
 from ansible.module_utils.compat.datetime import utcnow


@ -585,17 +585,30 @@ def main():
                    if not b_compiled_search_re:
                        # nope, succeed!
                        break
+
                    try:
                        with open(b_path, 'rb') as f:
-                            with contextlib.closing(mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)) as mm:
-                                search = b_compiled_search_re.search(mm)
+                            try:
+                                with contextlib.closing(mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)) as mm:
+                                    search = b_compiled_search_re.search(mm)
+                                    if search:
+                                        if search.groupdict():
+                                            match_groupdict = search.groupdict()
+                                        if search.groups():
+                                            match_groups = search.groups()
+                                        break
+                            except (ValueError, OSError) as e:
+                                module.debug('wait_for failed to use mmap on "%s": %s. Falling back to file read().' % (path, to_native(e)))
+                                # cannot mmap this file, try normal read
+                                search = re.search(b_compiled_search_re, f.read())
                                if search:
                                    if search.groupdict():
                                        match_groupdict = search.groupdict()
                                    if search.groups():
                                        match_groups = search.groups()
-
                                    break
+                            except Exception as e:
+                                module.warn('wait_for failed on "%s", unexpected exception(%s): %s.).' % (path, to_native(e.__class__), to_native(e)))
                    except IOError:
                        pass
            elif port:
--- a/lib/ansible/parsing/yaml/dumper.py
+++ b/lib/ansible/parsing/yaml/dumper.py
@ -24,7 +24,7 @@ import yaml
 from ansible.module_utils.six import text_type, binary_type
 from ansible.module_utils.common.yaml import SafeDumper
 from ansible.parsing.yaml.objects import AnsibleUnicode, AnsibleSequence, AnsibleMapping, AnsibleVaultEncryptedUnicode
-from ansible.utils.unsafe_proxy import AnsibleUnsafeText, AnsibleUnsafeBytes, NativeJinjaUnsafeText, NativeJinjaText
+from ansible.utils.unsafe_proxy import AnsibleUnsafeText, AnsibleUnsafeBytes, NativeJinjaUnsafeText, NativeJinjaText, _is_unsafe
 from ansible.template import AnsibleUndefined
 from ansible.vars.hostvars import HostVars, HostVarsVars
 from ansible.vars.manager import VarsWithSources
@ -47,10 +47,14 @@ def represent_vault_encrypted_unicode(self, data):


 def represent_unicode(self, data):
+    if _is_unsafe(data):
+        data = data._strip_unsafe()
    return yaml.representer.SafeRepresenter.represent_str(self, text_type(data))


 def represent_binary(self, data):
+    if _is_unsafe(data):
+        data = data._strip_unsafe()
    return yaml.representer.SafeRepresenter.represent_binary(self, binary_type(data))


--- a/lib/ansible/playbook/conditional.py
+++ b/lib/ansible/playbook/conditional.py
@ -21,7 +21,7 @@ __metaclass__ = type

 import typing as t

-from ansible.errors import AnsibleError, AnsibleUndefinedVariable
+from ansible.errors import AnsibleError, AnsibleUndefinedVariable, AnsibleTemplateError
 from ansible.module_utils.common.text.converters import to_native
 from ansible.playbook.attribute import FieldAttribute
 from ansible.template import Templar
@ -102,14 +102,14 @@ class Conditional:
                    return False

            # If the result of the first-pass template render (to resolve inline templates) is marked unsafe,
-            # explicitly disable lookups on the final pass to prevent evaluation of untrusted content in the
-            # constructed template.
-            disable_lookups = hasattr(conditional, '__UNSAFE__')
+            # explicitly fail since the next templating operation would never evaluate
+            if hasattr(conditional, '__UNSAFE__'):
+                raise AnsibleTemplateError('Conditional is marked as unsafe, and cannot be evaluated.')

            # NOTE The spaces around True and False are intentional to short-circuit literal_eval for
            #      jinja2_native=False and avoid its expensive calls.
            return templar.template(
                "{%% if %s %%} True {%% else %%} False {%% endif %%}" % conditional,
-                disable_lookups=disable_lookups).strip() == "True"
+            ).strip() == "True"
        except AnsibleUndefinedVariable as e:
            raise AnsibleUndefinedVariable("error while evaluating conditional (%s): %s" % (original, e))
--- a/lib/ansible/playbook/handler.py
+++ b/lib/ansible/playbook/handler.py
@ -53,6 +53,9 @@ class Handler(Task):
    def remove_host(self, host):
        self.notified_hosts = [h for h in self.notified_hosts if h != host]

+    def clear_hosts(self):
+        self.notified_hosts = []
+
    def is_host_notified(self, host):
        return host in self.notified_hosts

--- a/lib/ansible/playbook/included_file.py
+++ b/lib/ansible/playbook/included_file.py
@ -146,9 +146,12 @@ class IncludedFile:
                                    cumulative_path = parent_include_dir
                                include_target = templar.template(include_result['include'])
                                if original_task._role:
-                                    new_basedir = os.path.join(original_task._role._role_path, 'tasks', cumulative_path)
-                                    candidates = [loader.path_dwim_relative(original_task._role._role_path, 'tasks', include_target),
-                                                  loader.path_dwim_relative(new_basedir, 'tasks', include_target)]
+                                    dirname = 'handlers' if isinstance(original_task, Handler) else 'tasks'
+                                    new_basedir = os.path.join(original_task._role._role_path, dirname, cumulative_path)
+                                    candidates = [
+                                        loader.path_dwim_relative(original_task._role._role_path, dirname, include_target, is_role=True),
+                                        loader.path_dwim_relative(new_basedir, dirname, include_target, is_role=True)
+                                    ]
                                    for include_file in candidates:
                                        try:
                                            # may throw OSError
--- a/lib/ansible/playbook/role_include.py
+++ b/lib/ansible/playbook/role_include.py
@ -49,10 +49,10 @@ class IncludeRole(TaskInclude):

    # =================================================================================
    # ATTRIBUTES
+    public = NonInheritableFieldAttribute(isa='bool', default=None, private=False, always_post_validate=True)

    # private as this is a 'module options' vs a task property
    allow_duplicates = NonInheritableFieldAttribute(isa='bool', default=True, private=True, always_post_validate=True)
-    public = NonInheritableFieldAttribute(isa='bool', default=False, private=True, always_post_validate=True)
    rolespec_validate = NonInheritableFieldAttribute(isa='bool', default=True, private=True, always_post_validate=True)

    def __init__(self, block=None, role=None, task_include=None):
--- a/lib/ansible/playbook/taggable.py
+++ b/lib/ansible/playbook/taggable.py
@ -23,6 +23,17 @@ from ansible.errors import AnsibleError
 from ansible.module_utils.six import string_types
 from ansible.playbook.attribute import FieldAttribute
 from ansible.template import Templar
+from ansible.utils.sentinel import Sentinel
+
+
+def _flatten_tags(tags: list) -> list:
+    rv = set()
+    for tag in tags:
+        if isinstance(tag, list):
+            rv.update(tag)
+        else:
+            rv.add(tag)
+    return list(rv)


 class Taggable:
@ -34,11 +45,7 @@ class Taggable:
        if isinstance(ds, list):
            return ds
        elif isinstance(ds, string_types):
-            value = ds.split(',')
-            if isinstance(value, list):
-                return [x.strip() for x in value]
-            else:
-                return [ds]
+            return [x.strip() for x in ds.split(',')]
        else:
            raise AnsibleError('tags must be specified as a list', obj=ds)

@ -47,16 +54,12 @@ class Taggable:

        if self.tags:
            templar = Templar(loader=self._loader, variables=all_vars)
-            tags = templar.template(self.tags)
-
-            _temp_tags = set()
-            for tag in tags:
-                if isinstance(tag, list):
-                    _temp_tags.update(tag)
-                else:
-                    _temp_tags.add(tag)
-            tags = _temp_tags
-            self.tags = list(tags)
+            obj = self
+            while obj is not None:
+                if (_tags := getattr(obj, "_tags", Sentinel)) is not Sentinel:
+                    obj._tags = _flatten_tags(templar.template(_tags))
+                obj = obj._parent
+            tags = set(self.tags)
        else:
            # this makes isdisjoint work for untagged
            tags = self.untagged
--- a/lib/ansible/playbook/task.py
+++ b/lib/ansible/playbook/task.py
@ -289,6 +289,30 @@ class Task(Base, Conditional, Taggable, CollectionSearch, Notifiable, Delegatabl

        super(Task, self).post_validate(templar)

+    def _post_validate_args(self, attr, value, templar):
+        # smuggle an untemplated copy of the task args for actions that need more control over the templating of their
+        # input (eg, debug's var/msg, assert's "that" conditional expressions)
+        self.untemplated_args = value
+
+        # now recursively template the args dict
+        args = templar.template(value)
+
+        # FIXME: could we just nuke this entirely and/or wrap it up in ModuleArgsParser or something?
+        if '_variable_params' in args:
+            variable_params = args.pop('_variable_params')
+            if isinstance(variable_params, dict):
+                if C.INJECT_FACTS_AS_VARS:
+                    display.warning("Using a variable for a task's 'args' is unsafe in some situations "
+                                    "(see https://docs.ansible.com/ansible/devel/reference_appendices/faq.html#argsplat-unsafe)")
+                variable_params.update(args)
+                args = variable_params
+            else:
+                # if we didn't get a dict, it means there's garbage remaining after k=v parsing, just give up
+                # see https://github.com/ansible/ansible/issues/79862
+                raise AnsibleError(f"invalid or malformed argument: '{variable_params}'")
+
+        return args
+
    def _post_validate_loop(self, attr, value, templar):
        '''
        Override post validation for the loop field, which is templated
--- a/lib/ansible/plugins/action/assert.py
+++ b/lib/ansible/plugins/action/assert.py
@ -64,8 +64,29 @@ class ActionModule(ActionBase):

        quiet = boolean(self._task.args.get('quiet', False), strict=False)

+        # directly access 'that' via untemplated args from the task so we can intelligently trust embedded
+        # templates and preserve the original inputs/locations for better messaging on assert failures and
+        # errors.
+        # FIXME: even in devel, things like `that: item` don't always work properly (truthy string value
+        # is not really an embedded expression)
+        # we could fix that by doing direct var lookups on the inputs
+        # FIXME: some form of this code should probably be shared between debug, assert, and
+        # Task.post_validate, since they
+        # have a lot of overlapping needs
+        try:
+            thats = self._task.untemplated_args['that']
+        except KeyError:
+            # in the case of "we got our entire args dict from a template", we can just consult the
+            # post-templated dict (the damage has likely already been done for embedded templates anyway)
+            thats = self._task.args['that']
+
+        # FIXME: this is a case where we only want to resolve indirections, NOT recurse containers
+        # (and even then, the leaf-most expression being wrapped is at least suboptimal
+        # (since its expression will be "eaten").
+        if isinstance(thats, str):
+            thats = self._templar.template(thats)
+
        # make sure the 'that' items are a list
-        thats = self._task.args['that']
        if not isinstance(thats, list):
            thats = [thats]

--- a/lib/ansible/plugins/callback/init.py
+++ b/lib/ansible/plugins/callback/init.py
@ -38,7 +38,7 @@ from ansible.parsing.yaml.objects import AnsibleUnicode
 from ansible.plugins import AnsiblePlugin
 from ansible.utils.color import stringc
 from ansible.utils.display import Display
-from ansible.utils.unsafe_proxy import AnsibleUnsafeText, NativeJinjaUnsafeText
+from ansible.utils.unsafe_proxy import AnsibleUnsafeText, NativeJinjaUnsafeText, _is_unsafe
 from ansible.vars.clean import strip_internal_keys, module_response_deepcopy

 import yaml
@ -113,6 +113,8 @@ def _munge_data_for_lossy_yaml(scalar):

 def _pretty_represent_str(self, data):
    """Uses block style for multi-line strings"""
+    if _is_unsafe(data):
+        data = data._strip_unsafe()
    data = text_type(data)
    if _should_use_block(data):
        style = '|'
--- a/lib/ansible/plugins/connection/winrm.py
+++ b/lib/ansible/plugins/connection/winrm.py
@ -170,6 +170,7 @@ import json
 import tempfile
 import shlex
 import subprocess
+import time
 import typing as t

 from inspect import getfullargspec
@ -199,6 +200,7 @@ from ansible.utils.display import Display
 try:
    import winrm
    from winrm import Response
+    from winrm.exceptions import WinRMError, WinRMOperationTimeoutError
    from winrm.protocol import Protocol
    import requests.exceptions
    HAS_WINRM = True
@ -494,6 +496,43 @@ class Connection(ConnectionBase):
        else:
            raise AnsibleError('No transport found for WinRM connection')

+    def _winrm_write_stdin(self, command_id: str, stdin_iterator: t.Iterable[tuple[bytes, bool]]) -> None:
+        for (data, is_last) in stdin_iterator:
+            for attempt in range(1, 4):
+                try:
+                    self._winrm_send_input(self.protocol, self.shell_id, command_id, data, eof=is_last)
+
+                except WinRMOperationTimeoutError:
+                    # A WSMan OperationTimeout can be received for a Send
+                    # operation when the server is under severe load. On manual
+                    # testing the input is still processed and it's safe to
+                    # continue. As the calling method still tries to wait for
+                    # the proc to end if this failed it shouldn't hurt to just
+                    # treat this as a warning.
+                    display.warning(
+                        "WSMan OperationTimeout during send input, attempting to continue. "
+                        "If this continues to occur, try increasing the connection_timeout "
+                        "value for this host."
+                    )
+                    if not is_last:
+                        time.sleep(5)
+
+                except WinRMError as e:
+                    # Error 170 == ERROR_BUSY. This could be the result of a
+                    # timed out Send from above still being processed on the
+                    # server. Add a 5 second delay and try up to 3 times before
+                    # fully giving up.
+                    # pywinrm does not expose the internal WSMan fault details
+                    # through an actual object but embeds it as a repr.
+                    if attempt == 3 or "'wsmanfault_code': '170'" not in str(e):
+                        raise
+
+                    display.warning(f"WSMan send failed on attempt {attempt} as the command is busy, trying to send data again")
+                    time.sleep(5)
+                    continue
+
+                break
+
    def _winrm_send_input(self, protocol: winrm.Protocol, shell_id: str, command_id: str, stdin: bytes, eof: bool = False) -> None:
        rq = {'env:Envelope': protocol._get_soap_header(
            resource_uri='http://schemas.microsoft.com/wbem/wsman/1/windows/shell/cmd',
@ -529,8 +568,7 @@ class Connection(ConnectionBase):

            try:
                if stdin_iterator:
-                    for (data, is_last) in stdin_iterator:
-                        self._winrm_send_input(self.protocol, self.shell_id, command_id, data, eof=is_last)
+                    self._winrm_write_stdin(command_id, stdin_iterator)

            except Exception as ex:
                display.warning("ERROR DURING WINRM SEND INPUT - attempting to recover: %s %s"
--- a/lib/ansible/plugins/filter/core.py
+++ b/lib/ansible/plugins/filter/core.py
@ -37,6 +37,7 @@ from ansible.utils.display import Display
 from ansible.utils.encrypt import do_encrypt, PASSLIB_AVAILABLE
 from ansible.utils.hashing import md5s, checksum_s
 from ansible.utils.unicode import unicode_wrap
+from ansible.utils.unsafe_proxy import _is_unsafe
 from ansible.utils.vars import merge_hash

 display = Display()
@ -215,6 +216,8 @@ def from_yaml(data):
        # The ``text_type`` call here strips any custom
        # string wrapper class, so that CSafeLoader can
        # read the data
+        if _is_unsafe(data):
+            data = data._strip_unsafe()
        return yaml_load(text_type(to_text(data, errors='surrogate_or_strict')))
    return data

@ -224,6 +227,8 @@ def from_yaml_all(data):
        # The ``text_type`` call here strips any custom
        # string wrapper class, so that CSafeLoader can
        # read the data
+        if _is_unsafe(data):
+            data = data._strip_unsafe()
        return yaml_load_all(text_type(to_text(data, errors='surrogate_or_strict')))
    return data

--- a/lib/ansible/plugins/loader.py
+++ b/lib/ansible/plugins/loader.py
@ -238,6 +238,10 @@ class PluginLoader:
        self._module_cache = MODULE_CACHE[class_name]
        self._paths = PATH_CACHE[class_name]
        self._plugin_path_cache = PLUGIN_PATH_CACHE[class_name]
+        try:
+            self._plugin_instance_cache = {} if self.type == 'vars' else None
+        except ValueError:
+            self._plugin_instance_cache = None

        self._searched_paths = set()

@ -262,6 +266,7 @@ class PluginLoader:
            self._module_cache = MODULE_CACHE[self.class_name]
            self._paths = PATH_CACHE[self.class_name]
            self._plugin_path_cache = PLUGIN_PATH_CACHE[self.class_name]
+            self._plugin_instance_cache = {} if self.type == 'vars' else None
            self._searched_paths = set()

    def __setstate__(self, data):
@ -866,23 +871,35 @@ class PluginLoader:
        collection_list = kwargs.pop('collection_list', None)
        if name in self.aliases:
            name = self.aliases[name]
+
+        if self._plugin_instance_cache and (cached_load_result := self._plugin_instance_cache.get(name)):
+            # Resolving the FQCN is slow, even if we've passed in the resolved FQCN.
+            # Short-circuit here if we've previously resolved this name.
+            # This will need to be restricted if non-vars plugins start using the cache, since
+            # some non-fqcn plugin need to be resolved again with the collections list.
+            return get_with_context_result(*cached_load_result)
+
        plugin_load_context = self.find_plugin_with_context(name, collection_list=collection_list)
        if not plugin_load_context.resolved or not plugin_load_context.plugin_resolved_path:
            # FIXME: this is probably an error (eg removed plugin)
            return get_with_context_result(None, plugin_load_context)

        fq_name = plugin_load_context.resolved_fqcn
-        if '.' not in fq_name:
+        if '.' not in fq_name and plugin_load_context.plugin_resolved_collection:
            fq_name = '.'.join((plugin_load_context.plugin_resolved_collection, fq_name))
-        name = plugin_load_context.plugin_resolved_name
+        resolved_type_name = plugin_load_context.plugin_resolved_name
        path = plugin_load_context.plugin_resolved_path
+        if self._plugin_instance_cache and (cached_load_result := self._plugin_instance_cache.get(fq_name)):
+            # This is unused by vars plugins, but it's here in case the instance cache expands to other plugin types.
+            # We get here if we've seen this plugin before, but it wasn't called with the resolved FQCN.
+            return get_with_context_result(*cached_load_result)
        redirected_names = plugin_load_context.redirect_list or []

        if path not in self._module_cache:
-            self._module_cache[path] = self._load_module_source(name, path)
+            self._module_cache[path] = self._load_module_source(resolved_type_name, path)
            found_in_cache = False

-        self._load_config_defs(name, self._module_cache[path], path)
+        self._load_config_defs(resolved_type_name, self._module_cache[path], path)

        obj = getattr(self._module_cache[path], self.class_name)

@ -899,24 +916,27 @@ class PluginLoader:
                return get_with_context_result(None, plugin_load_context)

        # FIXME: update this to use the load context
-        self._display_plugin_load(self.class_name, name, self._searched_paths, path, found_in_cache=found_in_cache, class_only=class_only)
+        self._display_plugin_load(self.class_name, resolved_type_name, self._searched_paths, path, found_in_cache=found_in_cache, class_only=class_only)

        if not class_only:
            try:
                # A plugin may need to use its _load_name in __init__ (for example, to set
                # or get options from config), so update the object before using the constructor
                instance = object.__new__(obj)
-                self._update_object(instance, name, path, redirected_names, fq_name)
+                self._update_object(instance, resolved_type_name, path, redirected_names, fq_name)
                obj.__init__(instance, *args, **kwargs)  # pylint: disable=unnecessary-dunder-call
                obj = instance
            except TypeError as e:
                if "abstract" in e.args[0]:
                    # Abstract Base Class or incomplete plugin, don't load
-                    display.v('Returning not found on "%s" as it has unimplemented abstract methods; %s' % (name, to_native(e)))
+                    display.v('Returning not found on "%s" as it has unimplemented abstract methods; %s' % (resolved_type_name, to_native(e)))
                    return get_with_context_result(None, plugin_load_context)
                raise

-        self._update_object(obj, name, path, redirected_names, fq_name)
+        self._update_object(obj, resolved_type_name, path, redirected_names, fq_name)
+        if self._plugin_instance_cache is not None and getattr(obj, 'is_stateless', False):
+            # store under both the originally requested name and the resolved FQ name
+            self._plugin_instance_cache[name] = self._plugin_instance_cache[fq_name] = (obj, plugin_load_context)
        return get_with_context_result(obj, plugin_load_context)

    def _display_plugin_load(self, class_name, name, searched_paths, path, found_in_cache=None, class_only=None):
@ -986,28 +1006,47 @@ class PluginLoader:

        loaded_modules = set()
        for path in all_matches:
+
            name = os.path.splitext(path)[0]
            basename = os.path.basename(name)
+            is_j2 = isinstance(self, Jinja2Loader)

-            if basename in _PLUGIN_FILTERS[self.package]:
+            if is_j2:
+                ref_name = path
+            else:
+                ref_name = basename
+
+            if not is_j2 and basename in _PLUGIN_FILTERS[self.package]:
+                # j2 plugins get processed in own class, here they would just be container files
                display.debug("'%s' skipped due to a defined plugin filter" % basename)
                continue

            if basename == '__init__' or (basename == 'base' and self.package == 'ansible.plugins.cache'):
                # cache has legacy 'base.py' file, which is wrapper for __init__.py
-                display.debug("'%s' skipped due to reserved name" % basename)
+                display.debug("'%s' skipped due to reserved name" % name)
                continue

-            if dedupe and basename in loaded_modules:
-                display.debug("'%s' skipped as duplicate" % basename)
+            if dedupe and ref_name in loaded_modules:
+                # for j2 this is 'same file', other plugins it is basename
+                display.debug("'%s' skipped as duplicate" % ref_name)
                continue

-            loaded_modules.add(basename)
+            loaded_modules.add(ref_name)

            if path_only:
                yield path
                continue

+            if path in legacy_excluding_builtin:
+                fqcn = basename
+            else:
+                fqcn = f"ansible.builtin.{basename}"
+
+            if self._plugin_instance_cache is not None and fqcn in self._plugin_instance_cache:
+                # Here just in case, but we don't call all() multiple times for vars plugins, so this should not be used.
+                yield self._plugin_instance_cache[basename][0]
+                continue
+
            if path not in self._module_cache:
                if self.type in ('filter', 'test'):
                    # filter and test plugin files can contain multiple plugins
@ -1055,11 +1094,12 @@ class PluginLoader:
                except TypeError as e:
                    display.warning("Skipping plugin (%s) as it seems to be incomplete: %s" % (path, to_text(e)))

-            if path in legacy_excluding_builtin:
-                fqcn = basename
-            else:
-                fqcn = f"ansible.builtin.{basename}"
            self._update_object(obj, basename, path, resolved=fqcn)
+
+            if self._plugin_instance_cache is not None and fqcn not in self._plugin_instance_cache:
+                # Use get_with_context to cache the plugin the first time we see it.
+                self.get_with_context(fqcn)[0]
+
            yield obj


--- a/lib/ansible/plugins/lookup/first_found.py
+++ b/lib/ansible/plugins/lookup/first_found.py
@ -139,7 +139,6 @@ RETURN = """
    elements: path
 """
 import os
-import re

 from collections.abc import Mapping, Sequence

@ -150,10 +149,22 @@ from ansible.module_utils.six import string_types
 from ansible.plugins.lookup import LookupBase


+def _splitter(value, chars):
+    chars = set(chars)
+    v = ''
+    for c in value:
+        if c in chars:
+            yield v
+            v = ''
+            continue
+        v += c
+    yield v
+
+
 def _split_on(terms, spliters=','):
    termlist = []
    if isinstance(terms, string_types):
-        termlist = re.split(r'[%s]' % ''.join(map(re.escape, spliters)), terms)
+        termlist = list(_splitter(terms, spliters))
    else:
        # added since options will already listify
        for t in terms:
--- a/lib/ansible/plugins/strategy/init.py
+++ b/lib/ansible/plugins/strategy/init.py
@ -800,10 +800,6 @@ class StrategyBase:

            ret_results.append(task_result)

-            if isinstance(original_task, Handler):
-                for handler in (h for b in iterator._play.handlers for h in b.block if h._uuid == original_task._uuid):
-                    handler.remove_host(original_host)
-
            if one_pass or max_passes is not None and (cur_pass + 1) >= max_passes:
                break

@ -1091,9 +1087,6 @@ class StrategyBase:
            header = skip_reason if skipped else msg
            display.vv(f"META: {header}")

-        if isinstance(task, Handler):
-            task.remove_host(target_host)
-
        res = TaskResult(target_host, task, result)
        if skipped:
            self._tqm.send_callback('v2_runner_on_skipped', res)
--- a/lib/ansible/plugins/strategy/free.py
+++ b/lib/ansible/plugins/strategy/free.py
@ -146,6 +146,8 @@ class StrategyModule(StrategyBase):
                        # advance the host, mark the host blocked, and queue it
                        self._blocked_hosts[host_name] = True
                        iterator.set_state_for_host(host.name, state)
+                        if isinstance(task, Handler):
+                            task.remove_host(host)

                        try:
                            action = action_loader.get(task.action, class_only=True, collection_list=task.collections)
--- a/lib/ansible/plugins/strategy/linear.py
+++ b/lib/ansible/plugins/strategy/linear.py
@ -242,6 +242,12 @@ class StrategyModule(StrategyBase):
                        self._queue_task(host, task, task_vars, play_context)
                        del task_vars

+                    if isinstance(task, Handler):
+                        if run_once:
+                            task.clear_hosts()
+                        else:
+                            task.remove_host(host)
+
                    # if we're bypassing the host loop, break out now
                    if run_once:
                        break
--- a/lib/ansible/plugins/vars/init.py
+++ b/lib/ansible/plugins/vars/init.py
@ -30,6 +30,7 @@ class BaseVarsPlugin(AnsiblePlugin):
    """
    Loads variables for groups and/or hosts
    """
+    is_stateless = False

    def __init__(self):
        """ constructor """
--- a/lib/ansible/plugins/vars/host_group_vars.py
+++ b/lib/ansible/plugins/vars/host_group_vars.py
@ -55,18 +55,29 @@ DOCUMENTATION = '''

 import os
 from ansible.errors import AnsibleParserError
-from ansible.module_utils.common.text.converters import to_bytes, to_native, to_text
+from ansible.module_utils.common.text.converters import to_native
 from ansible.plugins.vars import BaseVarsPlugin
-from ansible.inventory.host import Host
-from ansible.inventory.group import Group
+from ansible.utils.path import basedir
+from ansible.inventory.group import InventoryObjectType
 from ansible.utils.vars import combine_vars

+CANONICAL_PATHS = {}  # type: dict[str, str]
 FOUND = {}  # type: dict[str, list[str]]
+NAK = set()  # type: set[str]
+PATH_CACHE = {}  # type: dict[tuple[str, str], str]


 class VarsModule(BaseVarsPlugin):

    REQUIRES_ENABLED = True
+    is_stateless = True
+
+    def load_found_files(self, loader, data, found_files):
+        for found in found_files:
+            new_data = loader.load_from_file(found, cache=True, unsafe=True)
+            if new_data:  # ignore empty files
+                data = combine_vars(data, new_data)
+        return data

    def get_vars(self, loader, path, entities, cache=True):
        ''' parses the inventory file '''
@ -74,41 +85,68 @@ class VarsModule(BaseVarsPlugin):
        if not isinstance(entities, list):
            entities = [entities]

-        super(VarsModule, self).get_vars(loader, path, entities)
+        # realpath is expensive
+        try:
+            realpath_basedir = CANONICAL_PATHS[path]
+        except KeyError:
+            CANONICAL_PATHS[path] = realpath_basedir = os.path.realpath(basedir(path))

        data = {}
        for entity in entities:
-            if isinstance(entity, Host):
-                subdir = 'host_vars'
-            elif isinstance(entity, Group):
-                subdir = 'group_vars'
-            else:
+            try:
+                entity_name = entity.name
+            except AttributeError:
+                raise AnsibleParserError("Supplied entity must be Host or Group, got %s instead" % (type(entity)))
+
+            try:
+                first_char = entity_name[0]
+            except (TypeError, IndexError, KeyError):
                raise AnsibleParserError("Supplied entity must be Host or Group, got %s instead" % (type(entity)))

            # avoid 'chroot' type inventory hostnames /path/to/chroot
-            if not entity.name.startswith(os.path.sep):
+            if first_char != os.path.sep:
                try:
                    found_files = []
                    # load vars
-                    b_opath = os.path.realpath(to_bytes(os.path.join(self._basedir, subdir)))
-                    opath = to_text(b_opath)
-                    key = '%s.%s' % (entity.name, opath)
-                    if cache and key in FOUND:
-                        found_files = FOUND[key]
-                    else:
-                        # no need to do much if path does not exist for basedir
-                        if os.path.exists(b_opath):
-                            if os.path.isdir(b_opath):
-                                self._display.debug("\tprocessing dir %s" % opath)
-                                found_files = loader.find_vars_files(opath, entity.name)
-                                FOUND[key] = found_files
-                            else:
-                                self._display.warning("Found %s that is not a directory, skipping: %s" % (subdir, opath))
+                    try:
+                        entity_type = entity.base_type
+                    except AttributeError:
+                        raise AnsibleParserError("Supplied entity must be Host or Group, got %s instead" % (type(entity)))

-                    for found in found_files:
-                        new_data = loader.load_from_file(found, cache=True, unsafe=True)
-                        if new_data:  # ignore empty files
-                            data = combine_vars(data, new_data)
+                    if entity_type is InventoryObjectType.HOST:
+                        subdir = 'host_vars'
+                    elif entity_type is InventoryObjectType.GROUP:
+                        subdir = 'group_vars'
+                    else:
+                        raise AnsibleParserError("Supplied entity must be Host or Group, got %s instead" % (type(entity)))
+
+                    if cache:
+                        try:
+                            opath = PATH_CACHE[(realpath_basedir, subdir)]
+                        except KeyError:
+                            opath = PATH_CACHE[(realpath_basedir, subdir)] = os.path.join(realpath_basedir, subdir)
+
+                        if opath in NAK:
+                            continue
+                        key = '%s.%s' % (entity_name, opath)
+                        if key in FOUND:
+                            data = self.load_found_files(loader, data, FOUND[key])
+                            continue
+                    else:
+                        opath = PATH_CACHE[(realpath_basedir, subdir)] = os.path.join(realpath_basedir, subdir)
+
+                    if os.path.isdir(opath):
+                        self._display.debug("\tprocessing dir %s" % opath)
+                        FOUND[key] = found_files = loader.find_vars_files(opath, entity_name)
+                    elif not os.path.exists(opath):
+                        # cache missing dirs so we don't have to keep looking for things beneath the
+                        NAK.add(opath)
+                    else:
+                        self._display.warning("Found %s that is not a directory, skipping: %s" % (subdir, opath))
+                        # cache non-directory matches
+                        NAK.add(opath)
+
+                    data = self.load_found_files(loader, data, found_files)

                except Exception as e:
                    raise AnsibleParserError(to_native(e))
--- a/lib/ansible/release.py
+++ b/lib/ansible/release.py
@ -19,6 +19,6 @@
 from __future__ import (absolute_import, division, print_function)
 __metaclass__ = type

-__version__ = '2.16.0.dev0'
+__version__ = '2.16.2'
 __author__ = 'Ansible, Inc.'
 __codename__ = "All My Love"
--- a/lib/ansible/template/init.py
+++ b/lib/ansible/template/init.py
@ -31,7 +31,7 @@ from contextlib import contextmanager
 from numbers import Number
 from traceback import format_exc

-from jinja2.exceptions import TemplateSyntaxError, UndefinedError
+from jinja2.exceptions import TemplateSyntaxError, UndefinedError, SecurityError
 from jinja2.loaders import FileSystemLoader
 from jinja2.nativetypes import NativeEnvironment
 from jinja2.runtime import Context, StrictUndefined
@ -55,7 +55,7 @@ from ansible.template.vars import AnsibleJ2Vars
 from ansible.utils.display import Display
 from ansible.utils.listify import listify_lookup_plugin_terms
 from ansible.utils.native_jinja import NativeJinjaText
-from ansible.utils.unsafe_proxy import to_unsafe_text, wrap_var
+from ansible.utils.unsafe_proxy import to_unsafe_text, wrap_var, AnsibleUnsafeText, AnsibleUnsafeBytes, NativeJinjaUnsafeText

 display = Display()

@ -365,10 +365,21 @@ class AnsibleContext(Context):
    flag is checked post-templating, and (when set) will result in the
    final templated result being wrapped in AnsibleUnsafe.
    '''
+    _disallowed_callables = frozenset({
+        AnsibleUnsafeText._strip_unsafe.__qualname__,
+        AnsibleUnsafeBytes._strip_unsafe.__qualname__,
+        NativeJinjaUnsafeText._strip_unsafe.__qualname__,
+    })
+
    def __init__(self, *args, **kwargs):
        super(AnsibleContext, self).__init__(*args, **kwargs)
        self.unsafe = False

+    def call(self, obj, *args, **kwargs):
+        if getattr(obj, '__qualname__', None) in self._disallowed_callables or obj in self._disallowed_callables:
+            raise SecurityError(f"{obj!r} is not safely callable")
+        return super().call(obj, *args, **kwargs)
+
    def _is_unsafe(self, val):
        '''
        Our helper function, which will also recursively check dict and
--- a/lib/ansible/utils/unsafe_proxy.py
+++ b/lib/ansible/utils/unsafe_proxy.py
@ -57,7 +57,6 @@ from collections.abc import Mapping, Set

 from ansible.module_utils.common.text.converters import to_bytes, to_text
 from ansible.module_utils.common.collections import is_sequence
-from ansible.module_utils.six import binary_type, text_type
 from ansible.utils.native_jinja import NativeJinjaText


@ -68,16 +67,256 @@ class AnsibleUnsafe(object):
    __UNSAFE__ = True


-class AnsibleUnsafeBytes(binary_type, AnsibleUnsafe):
-    def decode(self, *args, **kwargs):
-        """Wrapper method to ensure type conversions maintain unsafe context"""
-        return AnsibleUnsafeText(super(AnsibleUnsafeBytes, self).decode(*args, **kwargs))
+class AnsibleUnsafeBytes(bytes, AnsibleUnsafe):
+    def _strip_unsafe(self):
+        return super().__bytes__()
+
+    def __reduce__(self, /):
+        return (self.__class__, (self._strip_unsafe(),))
+
+    def __str__(self, /):  # pylint: disable=invalid-str-returned
+        return self.decode()
+
+    def __bytes__(self, /):  # pylint: disable=invalid-bytes-returned
+        return self
+
+    def __repr__(self, /):  # pylint: disable=invalid-repr-returned
+        return AnsibleUnsafeText(super().__repr__())
+
+    def __format__(self, format_spec, /):  # pylint: disable=invalid-format-returned
+        return AnsibleUnsafeText(super().__format__(format_spec))
+
+    def __getitem__(self, key, /):
+        if isinstance(key, int):
+            return super().__getitem__(key)
+        return self.__class__(super().__getitem__(key))
+
+    def __reversed__(self, /):
+        return self[::-1]
+
+    def __add__(self, value, /):
+        return self.__class__(super().__add__(value))
+
+    def __radd__(self, value, /):
+        return self.__class__(value.__add__(self))
+
+    def __mul__(self, value, /):
+        return self.__class__(super().__mul__(value))
+
+    __rmul__ = __mul__
+
+    def __mod__(self, value, /):
+        return self.__class__(super().__mod__(value))
+
+    def __rmod__(self, value, /):
+        return self.__class__(super().__rmod__(value))
+
+    def capitalize(self, /):
+        return self.__class__(super().capitalize())
+
+    def center(self, width, fillchar=b' ', /):
+        return self.__class__(super().center(width, fillchar))
+
+    def decode(self, /, encoding='utf-8', errors='strict'):
+        return AnsibleUnsafeText(super().decode(encoding=encoding, errors=errors))
+
+    def removeprefix(self, prefix, /):
+        return self.__class__(super().removeprefix(prefix))
+
+    def removesuffix(self, suffix, /):
+        return self.__class__(super().removesuffix(suffix))
+
+    def expandtabs(self, /, tabsize=8):
+        return self.__class__(super().expandtabs(tabsize))
+
+    def join(self, iterable_of_bytes, /):
+        return self.__class__(super().join(iterable_of_bytes))
+
+    def ljust(self, width, fillchar=b' ', /):
+        return self.__class__(super().ljust(width, fillchar))
+
+    def lower(self, /):
+        return self.__class__(super().lower())
+
+    def lstrip(self, chars=None, /):
+        return self.__class__(super().lstrip(chars))
+
+    def partition(self, sep, /):
+        cls = self.__class__
+        return tuple(cls(e) for e in super().partition(sep))
+
+    def replace(self, old, new, count=-1, /):
+        return self.__class__(super().replace(old, new, count))
+
+    def rjust(self, width, fillchar=b' ', /):
+        return self.__class__(super().rjust(width, fillchar))
+
+    def rpartition(self, sep, /):
+        cls = self.__class__
+        return tuple(cls(e) for e in super().rpartition(sep))
+
+    def rstrip(self, chars=None, /):
+        return self.__class__(super().rstrip(chars))
+
+    def split(self, /, sep=None, maxsplit=-1):
+        cls = self.__class__
+        return [cls(e) for e in super().split(sep=sep, maxsplit=maxsplit)]
+
+    def rsplit(self, /, sep=None, maxsplit=-1):
+        cls = self.__class__
+        return [cls(e) for e in super().rsplit(sep=sep, maxsplit=maxsplit)]
+
+    def splitlines(self, /, keepends=False):
+        cls = self.__class__
+        return [cls(e) for e in super().splitlines(keepends=keepends)]
+
+    def strip(self, chars=None, /):
+        return self.__class__(super().strip(chars))
+
+    def swapcase(self, /):
+        return self.__class__(super().swapcase())
+
+    def title(self, /):
+        return self.__class__(super().title())
+
+    def translate(self, table, /, delete=b''):
+        return self.__class__(super().translate(table, delete=delete))
+
+    def upper(self, /):
+        return self.__class__(super().upper())
+
+    def zfill(self, width, /):
+        return self.__class__(super().zfill(width))


-class AnsibleUnsafeText(text_type, AnsibleUnsafe):
-    def encode(self, *args, **kwargs):
-        """Wrapper method to ensure type conversions maintain unsafe context"""
-        return AnsibleUnsafeBytes(super(AnsibleUnsafeText, self).encode(*args, **kwargs))
+class AnsibleUnsafeText(str, AnsibleUnsafe):
+    def _strip_unsafe(self, /):
+        return super().__str__()
+
+    def __reduce__(self, /):
+        return (self.__class__, (self._strip_unsafe(),))
+
+    def __str__(self, /):  # pylint: disable=invalid-str-returned
+        return self
+
+    def __repr__(self, /):  # pylint: disable=invalid-repr-returned
+        return self.__class__(super().__repr__())
+
+    def __format__(self, format_spec, /):  # pylint: disable=invalid-format-returned
+        return self.__class__(super().__format__(format_spec))
+
+    def __getitem__(self, key, /):
+        return self.__class__(super().__getitem__(key))
+
+    def __iter__(self, /):
+        cls = self.__class__
+        return (cls(c) for c in super().__iter__())
+
+    def __reversed__(self, /):
+        return self[::-1]
+
+    def __add__(self, value, /):
+        return self.__class__(super().__add__(value))
+
+    def __radd__(self, value, /):
+        return self.__class__(value.__add__(self))
+
+    def __mul__(self, value, /):
+        return self.__class__(super().__mul__(value))
+
+    __rmul__ = __mul__
+
+    def __mod__(self, value, /):
+        return self.__class__(super().__mod__(value))
+
+    def __rmod__(self, value, /):
+        return self.__class__(super().__rmod__(value))
+
+    def capitalize(self, /):
+        return self.__class__(super().capitalize())
+
+    def casefold(self, /):
+        return self.__class__(super().casefold())
+
+    def center(self, width, fillchar=' ', /):
+        return self.__class__(super().center(width, fillchar))
+
+    def encode(self, /, encoding='utf-8', errors='strict'):
+        return AnsibleUnsafeBytes(super().encode(encoding=encoding, errors=errors))
+
+    def removeprefix(self, prefix, /):
+        return self.__class__(super().removeprefix(prefix))
+
+    def removesuffix(self, suffix, /):
+        return self.__class__(super().removesuffix(suffix))
+
+    def expandtabs(self, /, tabsize=8):
+        return self.__class__(super().expandtabs(tabsize))
+
+    def format(self, /, *args, **kwargs):
+        return self.__class__(super().format(*args, **kwargs))
+
+    def format_map(self, mapping, /):
+        return self.__class__(super().format_map(mapping))
+
+    def join(self, iterable, /):
+        return self.__class__(super().join(iterable))
+
+    def ljust(self, width, fillchar=' ', /):
+        return self.__class__(super().ljust(width, fillchar))
+
+    def lower(self, /):
+        return self.__class__(super().lower())
+
+    def lstrip(self, chars=None, /):
+        return self.__class__(super().lstrip(chars))
+
+    def partition(self, sep, /):
+        cls = self.__class__
+        return tuple(cls(e) for e in super().partition(sep))
+
+    def replace(self, old, new, count=-1, /):
+        return self.__class__(super().replace(old, new, count))
+
+    def rjust(self, width, fillchar=' ', /):
+        return self.__class__(super().rjust(width, fillchar))
+
+    def rpartition(self, sep, /):
+        cls = self.__class__
+        return tuple(cls(e) for e in super().rpartition(sep))
+
+    def rstrip(self, chars=None, /):
+        return self.__class__(super().rstrip(chars))
+
+    def split(self, /, sep=None, maxsplit=-1):
+        cls = self.__class__
+        return [cls(e) for e in super().split(sep=sep, maxsplit=maxsplit)]
+
+    def rsplit(self, /, sep=None, maxsplit=-1):
+        cls = self.__class__
+        return [cls(e) for e in super().rsplit(sep=sep, maxsplit=maxsplit)]
+
+    def splitlines(self, /, keepends=False):
+        cls = self.__class__
+        return [cls(e) for e in super().splitlines(keepends=keepends)]
+
+    def strip(self, chars=None, /):
+        return self.__class__(super().strip(chars))
+
+    def swapcase(self, /):
+        return self.__class__(super().swapcase())
+
+    def title(self, /):
+        return self.__class__(super().title())
+
+    def translate(self, table, /):
+        return self.__class__(super().translate(table))
+
+    def upper(self, /):
+        return self.__class__(super().upper())
+
+    def zfill(self, width, /):
+        return self.__class__(super().zfill(width))


 class NativeJinjaUnsafeText(NativeJinjaText, AnsibleUnsafeText):
@ -112,9 +351,9 @@ def wrap_var(v):
        v = _wrap_sequence(v)
    elif isinstance(v, NativeJinjaText):
        v = NativeJinjaUnsafeText(v)
-    elif isinstance(v, binary_type):
+    elif isinstance(v, bytes):
        v = AnsibleUnsafeBytes(v)
-    elif isinstance(v, text_type):
+    elif isinstance(v, str):
        v = AnsibleUnsafeText(v)

    return v
@ -126,3 +365,7 @@ def to_unsafe_bytes(*args, **kwargs):

 def to_unsafe_text(*args, **kwargs):
    return wrap_var(to_text(*args, **kwargs))
+
+
+def _is_unsafe(obj):
+    return getattr(obj, '__UNSAFE__', False) is True
--- a/lib/ansible/utils/vars.py
+++ b/lib/ansible/utils/vars.py
@ -109,6 +109,8 @@ def merge_hash(x, y, recursive=True, list_merge='replace'):
    #  except performance)
    if x == {} or x == y:
        return y.copy()
+    if y == {}:
+        return x

    # in the following we will copy elements from y to x, but
    # we don't want to modify x, so we create a copy of it
--- a/lib/ansible/vars/manager.py
+++ b/lib/ansible/vars/manager.py
@ -184,6 +184,9 @@ class VariableManager:

            See notes in the VarsWithSources docstring for caveats and limitations of the source tracking
            '''
+            if new_data == {}:
+                return data
+
            if C.DEFAULT_DEBUG:
                # Populate var sources dict
                for key in new_data:
@ -197,13 +200,13 @@ class VariableManager:

        if play:
            if not C.DEFAULT_PRIVATE_ROLE_VARS:
-                # first we compile any vars specified in defaults/main.yml
-                # for all roles within the specified play
                for role in play.get_roles():
-                    # role from roles or include_role+public or import_role and completed
-                    if not role.from_include or role.public or (role.static and role._completed.get(to_text(host), False)):
+                    # role is public and
+                    #    either static or dynamic and completed
+                    # role is not set
+                    #    use config option as default
+                    if role.static or role.public and role._completed.get(host.name, False):
                        all_vars = _combine_and_track(all_vars, role.get_default_vars(), "role '%s' defaults" % role.name)
-
        if task:
            # set basedirs
            if C.PLAYBOOK_VARS_ROOT == 'all':  # should be default
@ -386,11 +389,15 @@ class VariableManager:
                raise AnsibleParserError("Error while reading vars files - please supply a list of file names. "
                                         "Got '%s' of type %s" % (vars_files, type(vars_files)))

-            # By default, we now merge in all exported vars from all roles in the play,
-            # unless the user has disabled this via a config option
+            # We now merge in all exported vars from all roles in the play,
+            # unless the user has disabled this
+            # role is public and
+            #    either static or dynamic and completed
+            # role is not set
+            #    use config option as default
            if not C.DEFAULT_PRIVATE_ROLE_VARS:
                for role in play.get_roles():
-                    if not role.from_include or role.public or (role.static and role._completed.get(to_text(host), False)):
+                    if role.static or role.public and role._completed.get(host.name, False):
                        all_vars = _combine_and_track(all_vars, role.get_vars(include_params=False, only_exports=True), "role '%s' exported vars" % role.name)

        # next, we merge in the vars from the role, which will specifically
@ -398,7 +405,7 @@ class VariableManager:
        # vars (which will look at parent blocks/task includes)
        if task:
            if task._role:
-                all_vars = _combine_and_track(all_vars, task._role.get_vars(task.get_dep_chain(), include_params=True, only_exports=False),
+                all_vars = _combine_and_track(all_vars, task._role.get_vars(task.get_dep_chain(), include_params=False, only_exports=False),
                                              "role '%s' all vars" % task._role.name)
            all_vars = _combine_and_track(all_vars, task.get_vars(), "task vars")

@ -415,6 +422,8 @@ class VariableManager:
            # special case for include tasks, where the include params
            # may be specified in the vars field for the task, which should
            # have higher precedence than the vars/np facts above
+            if task._role:
+                all_vars = _combine_and_track(all_vars, task._role.get_role_params(task.get_dep_chain()), "role params")
            all_vars = _combine_and_track(all_vars, task.get_include_params(), "include params")

        # extra vars
--- a/lib/ansible/vars/plugins.py
+++ b/lib/ansible/vars/plugins.py
@ -1,23 +1,57 @@
 # Copyright (c) 2018 Ansible Project
 # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)

-# Make coding more python3-ish
-from __future__ import (absolute_import, division, print_function)
-__metaclass__ = type
+from __future__ import annotations

 import os

+from functools import lru_cache
+
 from ansible import constants as C
 from ansible.errors import AnsibleError
-from ansible.inventory.host import Host
-from ansible.module_utils.common.text.converters import to_bytes
+from ansible.inventory.group import InventoryObjectType
 from ansible.plugins.loader import vars_loader
-from ansible.utils.collection_loader import AnsibleCollectionRef
 from ansible.utils.display import Display
 from ansible.utils.vars import combine_vars

 display = Display()

+cached_vars_plugin_order = None
+
+
+def _load_vars_plugins_order():
+    # find 3rd party legacy vars plugins once, and look them up by name subsequently
+    auto = []
+    for auto_run_plugin in vars_loader.all(class_only=True):
+        needs_enabled = False
+        if hasattr(auto_run_plugin, 'REQUIRES_ENABLED'):
+            needs_enabled = auto_run_plugin.REQUIRES_ENABLED
+        elif hasattr(auto_run_plugin, 'REQUIRES_WHITELIST'):
+            needs_enabled = auto_run_plugin.REQUIRES_WHITELIST
+            display.deprecated("The VarsModule class variable 'REQUIRES_WHITELIST' is deprecated. "
+                               "Use 'REQUIRES_ENABLED' instead.", version=2.18)
+        if needs_enabled:
+            continue
+        auto.append(auto_run_plugin._load_name)
+
+    # find enabled plugins once so we can look them up by resolved fqcn subsequently
+    enabled = []
+    for plugin_name in C.VARIABLE_PLUGINS_ENABLED:
+        if (plugin := vars_loader.get(plugin_name)) is None:
+            enabled.append(plugin_name)
+        else:
+            collection = '.' in plugin.ansible_name and not plugin.ansible_name.startswith('ansible.builtin.')
+            # Warn if a collection plugin has REQUIRES_ENABLED because it has no effect.
+            if collection and (hasattr(plugin, 'REQUIRES_ENABLED') or hasattr(plugin, 'REQUIRES_WHITELIST')):
+                display.warning(
+                    "Vars plugins in collections must be enabled to be loaded, REQUIRES_ENABLED is not supported. "
+                    "This should be removed from the plugin %s." % plugin.ansible_name
+                )
+            enabled.append(plugin.ansible_name)
+
+    global cached_vars_plugin_order
+    cached_vars_plugin_order = auto + enabled
+

 def get_plugin_vars(loader, plugin, path, entities):

@ -25,9 +59,17 @@ def get_plugin_vars(loader, plugin, path, entities):
    try:
        data = plugin.get_vars(loader, path, entities)
    except AttributeError:
+        if hasattr(plugin, 'get_host_vars') or hasattr(plugin, 'get_group_vars'):
+            display.deprecated(
+                f"The vars plugin {plugin.ansible_name} from {plugin._original_path} is relying "
+                "on the deprecated entrypoints 'get_host_vars' and 'get_group_vars'. "
+                "This plugin should be updated to inherit from BaseVarsPlugin and define "
+                "a 'get_vars' method as the main entrypoint instead.",
+                version="2.20",
+            )
        try:
            for entity in entities:
-                if isinstance(entity, Host):
+                if entity.base_type is InventoryObjectType.HOST:
                    data |= plugin.get_host_vars(entity.name)
                else:
                    data |= plugin.get_group_vars(entity.name)
@ -39,59 +81,46 @@ def get_plugin_vars(loader, plugin, path, entities):
    return data


+# optimized for stateless plugins; non-stateless plugin instances will fall out quickly
+@lru_cache(maxsize=10)
+def _plugin_should_run(plugin, stage):
+    # if a plugin-specific setting has not been provided, use the global setting
+    # older/non shipped plugins that don't support the plugin-specific setting should also use the global setting
+    allowed_stages = None
+
+    try:
+        allowed_stages = plugin.get_option('stage')
+    except (AttributeError, KeyError):
+        pass
+
+    if allowed_stages:
+        return allowed_stages in ('all', stage)
+
+    # plugin didn't declare a preference; consult global config
+    config_stage_override = C.RUN_VARS_PLUGINS
+    if config_stage_override == 'demand' and stage == 'inventory':
+        return False
+    elif config_stage_override == 'start' and stage == 'task':
+        return False
+    return True
+
+
 def get_vars_from_path(loader, path, entities, stage):

    data = {}

-    vars_plugin_list = list(vars_loader.all())
-    for plugin_name in C.VARIABLE_PLUGINS_ENABLED:
-        if AnsibleCollectionRef.is_valid_fqcr(plugin_name):
-            vars_plugin = vars_loader.get(plugin_name)
-            if vars_plugin is None:
-                # Error if there's no play directory or the name is wrong?
-                continue
-            if vars_plugin not in vars_plugin_list:
-                vars_plugin_list.append(vars_plugin)
+    if cached_vars_plugin_order is None:
+        _load_vars_plugins_order()

-    for plugin in vars_plugin_list:
-        # legacy plugins always run by default, but they can set REQUIRES_ENABLED=True to opt out.
-
-        builtin_or_legacy = plugin.ansible_name.startswith('ansible.builtin.') or '.' not in plugin.ansible_name
-
-        # builtin is supposed to have REQUIRES_ENABLED=True, the following is for legacy plugins...
-        needs_enabled = not builtin_or_legacy
-        if hasattr(plugin, 'REQUIRES_ENABLED'):
-            needs_enabled = plugin.REQUIRES_ENABLED
-        elif hasattr(plugin, 'REQUIRES_WHITELIST'):
-            display.deprecated("The VarsModule class variable 'REQUIRES_WHITELIST' is deprecated. "
-                               "Use 'REQUIRES_ENABLED' instead.", version="2.18")
-            needs_enabled = plugin.REQUIRES_WHITELIST
-
-        # A collection plugin was enabled to get to this point because vars_loader.all() does not include collection plugins.
-        # Warn if a collection plugin has REQUIRES_ENABLED because it has no effect.
-        if not builtin_or_legacy and (hasattr(plugin, 'REQUIRES_ENABLED') or hasattr(plugin, 'REQUIRES_WHITELIST')):
-            display.warning(
-                "Vars plugins in collections must be enabled to be loaded, REQUIRES_ENABLED is not supported. "
-                "This should be removed from the plugin %s." % plugin.ansible_name
-            )
-        elif builtin_or_legacy and needs_enabled and not plugin.matches_name(C.VARIABLE_PLUGINS_ENABLED):
+    for plugin_name in cached_vars_plugin_order:
+        if (plugin := vars_loader.get(plugin_name)) is None:
            continue

-        has_stage = hasattr(plugin, 'get_option') and plugin.has_option('stage')
-
-        # if a plugin-specific setting has not been provided, use the global setting
-        # older/non shipped plugins that don't support the plugin-specific setting should also use the global setting
-        use_global = (has_stage and plugin.get_option('stage') is None) or not has_stage
-
-        if use_global:
-            if C.RUN_VARS_PLUGINS == 'demand' and stage == 'inventory':
-                continue
-            elif C.RUN_VARS_PLUGINS == 'start' and stage == 'task':
-                continue
-        elif has_stage and plugin.get_option('stage') not in ('all', stage):
+        if not _plugin_should_run(plugin, stage):
            continue

-        data = combine_vars(data, get_plugin_vars(loader, plugin, path, entities))
+        if (new_vars := get_plugin_vars(loader, plugin, path, entities)) != {}:
+            data = combine_vars(data, new_vars)

    return data

@ -105,10 +134,11 @@ def get_vars_from_inventory_sources(loader, sources, entities, stage):
            continue
        if ',' in path and not os.path.exists(path):  # skip host lists
            continue
-        elif not os.path.isdir(to_bytes(path)):
+        elif not os.path.isdir(path):
            # always pass the directory of the inventory source file
            path = os.path.dirname(path)

-        data = combine_vars(data, get_vars_from_path(loader, path, entities, stage))
+        if (new_vars := get_vars_from_path(loader, path, entities, stage)) != {}:
+            data = combine_vars(data, new_vars)

    return data
--- a/setup.cfg
+++ b/setup.cfg
@ -29,6 +29,7 @@ classifiers =
    Programming Language :: Python :: 3
    Programming Language :: Python :: 3.10
    Programming Language :: Python :: 3.11
+    Programming Language :: Python :: 3.12
    Programming Language :: Python :: 3 :: Only
    Topic :: System :: Installation/Setup
    Topic :: System :: Systems Administration
--- a/test/integration/targets/ansible-galaxy-collection-scm/tasks/scm_dependency_deduplication.yml
+++ b/test/integration/targets/ansible-galaxy-collection-scm/tasks/scm_dependency_deduplication.yml
@ -13,22 +13,22 @@
        in command.stdout_lines
      - >-
        "Installing 'namespace_1.collection_1:1.0.0' to
-        '{{ install_path }}/namespace_1/collection_1'"
+        '" ~ install_path ~ "/namespace_1/collection_1'"
        in command.stdout_lines
      - >-
        'Created collection for namespace_1.collection_1:1.0.0 at
-        {{ install_path }}/namespace_1/collection_1'
+        ' ~ install_path ~ '/namespace_1/collection_1'
        in command.stdout_lines
      - >-
        'namespace_1.collection_1:1.0.0 was installed successfully'
        in command.stdout_lines
      - >-
        "Installing 'namespace_2.collection_2:1.0.0' to
-        '{{ install_path }}/namespace_2/collection_2'"
+        '" ~ install_path ~ "/namespace_2/collection_2'"
        in command.stdout_lines
      - >-
        'Created collection for namespace_2.collection_2:1.0.0 at
-        {{ install_path }}/namespace_2/collection_2'
+        ' ~ install_path ~ '/namespace_2/collection_2'
        in command.stdout_lines
      - >-
        'namespace_2.collection_2:1.0.0 was installed successfully'
@ -58,22 +58,22 @@
        in command.stdout_lines
      - >-
        "Installing 'namespace_1.collection_1:1.0.0' to
-        '{{ install_path }}/namespace_1/collection_1'"
+        '" ~ install_path ~ "/namespace_1/collection_1'"
        in command.stdout_lines
      - >-
        'Created collection for namespace_1.collection_1:1.0.0 at
-        {{ install_path }}/namespace_1/collection_1'
+        ' ~ install_path ~ '/namespace_1/collection_1'
        in command.stdout_lines
      - >-
        'namespace_1.collection_1:1.0.0 was installed successfully'
        in command.stdout_lines
      - >-
        "Installing 'namespace_2.collection_2:1.0.0' to
-        '{{ install_path }}/namespace_2/collection_2'"
+        '" ~ install_path ~ "/namespace_2/collection_2'"
        in command.stdout_lines
      - >-
        'Created collection for namespace_2.collection_2:1.0.0 at
-        {{ install_path }}/namespace_2/collection_2'
+        ' ~ install_path ~ '/namespace_2/collection_2'
        in command.stdout_lines
      - >-
        'namespace_2.collection_2:1.0.0 was installed successfully'
--- a/test/integration/targets/ansible-galaxy-collection/tasks/main.yml
+++ b/test/integration/targets/ansible-galaxy-collection/tasks/main.yml
@ -134,6 +134,17 @@
  loop_control:
    loop_var: resolvelib_version

+- name: test choosing pinned pre-releases anywhere in the dependency tree
+  # This is a regression test for the case when the end-user does not
+  # explicitly allow installing pre-release collection versions, but their
+  # precise pins are still selected if met among the dependencies, either
+  # direct or transitive.
+  include_tasks: pinned_pre_releases_in_deptree.yml
+
+- name: test installing prereleases via scm direct requests
+  # In this test suite because the bug relies on the dep having versions on a Galaxy server
+  include_tasks: virtual_direct_requests.yml
+
 - name: publish collection with a dep on another server
  setup_collections:
    server: secondary
--- a/test/integration/targets/ansible-galaxy-collection/tasks/pinned_pre_releases_in_deptree.yml
+++ b/test/integration/targets/ansible-galaxy-collection/tasks/pinned_pre_releases_in_deptree.yml
@ -0,0 +1,79 @@
+---
+
+- name: >-
+    test that the dependency resolver chooses pre-releases if they are pinned
+  environment:
+    ANSIBLE_COLLECTIONS_PATH: '{{ galaxy_dir }}'
+    ANSIBLE_CONFIG: '{{ galaxy_dir }}/ansible.cfg'
+  block:
+  - name: reset installation directory
+    file:
+      state: "{{ item }}"
+      path: "{{ galaxy_dir }}/ansible_collections"
+    loop:
+    - absent
+    - directory
+
+  - name: >-
+      install collections with pre-release versions in the dependency tree
+    command: >-
+      ansible-galaxy collection install
+      meta_ns_with_transitive_wildcard_dep.meta_name_with_transitive_wildcard_dep
+      rc_meta_ns_with_transitive_dev_dep.rc_meta_name_with_transitive_dev_dep:=2.4.5-rc5
+      {{ galaxy_verbosity }}
+    register: prioritize_direct_req
+  - assert:
+      that:
+      - >-
+        "rc_meta_ns_with_transitive_dev_dep.rc_meta_name_with_transitive_dev_dep:2.4.5-rc5 was installed successfully"
+        in prioritize_direct_req.stdout
+      - >-
+        "meta_ns_with_transitive_wildcard_dep.meta_name_with_transitive_wildcard_dep:4.5.6 was installed successfully"
+        in prioritize_direct_req.stdout
+      - >-
+        "ns_with_dev_dep.name_with_dev_dep:6.7.8 was installed successfully"
+        in prioritize_direct_req.stdout
+      - >-
+        "ns_with_wildcard_dep.name_with_wildcard_dep:5.6.7-beta.3 was installed successfully"
+        in prioritize_direct_req.stdout
+      - >-
+        "dev_and_stables_ns.dev_and_stables_name:1.2.3-dev0 was installed successfully"
+        in prioritize_direct_req.stdout
+
+  - name: cleanup
+    file:
+      state: "{{ item }}"
+      path: "{{ galaxy_dir }}/ansible_collections"
+    loop:
+    - absent
+    - directory
+
+  - name: >-
+      install collection that only has pre-release versions published
+      to the index
+    command: >-
+      ansible-galaxy collection install
+      rc_meta_ns_with_transitive_dev_dep.rc_meta_name_with_transitive_dev_dep:*
+      {{ galaxy_verbosity }}
+    register: select_pre_release_if_no_stable
+  - assert:
+      that:
+      - >-
+        "rc_meta_ns_with_transitive_dev_dep.rc_meta_name_with_transitive_dev_dep:2.4.5-rc5 was installed successfully"
+        in select_pre_release_if_no_stable.stdout
+      - >-
+        "ns_with_dev_dep.name_with_dev_dep:6.7.8 was installed successfully"
+        in select_pre_release_if_no_stable.stdout
+      - >-
+        "dev_and_stables_ns.dev_and_stables_name:1.2.3-dev0 was installed successfully"
+        in select_pre_release_if_no_stable.stdout
+  always:
+  - name: cleanup
+    file:
+      state: "{{ item }}"
+      path: "{{ galaxy_dir }}/ansible_collections"
+    loop:
+    - absent
+    - directory
+
+...
--- a/test/integration/targets/ansible-galaxy-collection/tasks/virtual_direct_requests.yml
+++ b/test/integration/targets/ansible-galaxy-collection/tasks/virtual_direct_requests.yml
@ -0,0 +1,77 @@
+- environment:
+    ANSIBLE_CONFIG: '{{ galaxy_dir }}/ansible.cfg'
+  vars:
+    scm_path: "{{ galaxy_dir }}/scms"
+    metadata:
+      collection1: |-
+        name: collection1
+        version: "1.0.0"
+        dependencies:
+          test_prereleases.collection2: '*'
+      collection2: |
+        name: collection2
+        version: "1.0.0-dev0"
+        dependencies: {}
+      namespace_boilerplate: |-
+        namespace: test_prereleases
+        readme: README.md
+        authors:
+          - "ansible-core"
+        description: test prerelease priority with virtual collections
+        license:
+          - GPL-2.0-or-later
+        license_file: ''
+        tags: []
+        repository: https://github.com/ansible/ansible
+        documentation: https://github.com/ansible/ansible
+        homepage: https://github.com/ansible/ansible
+        issues: https://github.com/ansible/ansible
+        build_ignore: []
+  block:
+    - name: Initialize git repository
+      command: 'git init {{ scm_path }}/test_prereleases'
+
+    - name: Configure commiter for the repo
+      shell: git config user.email ansible-test@ansible.com && git config user.name ansible-test
+      args:
+        chdir: "{{ scm_path }}/test_prereleases"
+
+    - name: Add collections to the repo
+      file:
+        path: "{{ scm_path }}/test_prereleases/{{ item }}"
+        state: directory
+      loop:
+        - collection1
+        - collection2
+
+    - name: Add collection metadata
+      copy:
+        dest: "{{ scm_path }}/test_prereleases/{{ item }}/galaxy.yml"
+        content: "{{ metadata[item] + '\n' + metadata['namespace_boilerplate'] }}"
+      loop:
+        - collection1
+        - collection2
+
+    - name: Save the changes
+      shell: git add . && git commit -m "Add collections to test installing a git repo directly takes priority over indirect Galaxy dep"
+      args:
+        chdir: '{{ scm_path }}/test_prereleases'
+
+    - name: Validate the dependency also exists on Galaxy before test
+      command: "ansible-galaxy collection install test_prereleases.collection2"
+      register: prereq
+      failed_when: '"test_prereleases.collection2:1.0.0 was installed successfully" not in prereq.stdout'
+
+    - name: Install collections from source
+      command: "ansible-galaxy collection install git+file://{{ scm_path }}/test_prereleases"
+      register: prioritize_direct_req
+
+    - assert:
+        that:
+          - '"test_prereleases.collection2:1.0.0-dev0 was installed successfully" in prioritize_direct_req.stdout'
+
+  always:
+    - name: Clean up test repos
+      file:
+        path: "{{ scm_path }}"
+        state: absent
--- a/test/integration/targets/ansible-galaxy-collection/vars/main.yml
+++ b/test/integration/targets/ansible-galaxy-collection/vars/main.yml
@ -165,3 +165,41 @@ collection_list:
    name: parent
    dependencies:
      namespace1.name1: '*'
+
+  # non-prerelease is published to test that installing
+  # the pre-release from SCM doesn't accidentally prefer indirect
+  # dependencies from Galaxy
+  - namespace: test_prereleases
+    name: collection2
+    version: 1.0.0
+
+  - namespace: dev_and_stables_ns
+    name: dev_and_stables_name
+    version: 1.2.3-dev0
+  - namespace: dev_and_stables_ns
+    name: dev_and_stables_name
+    version: 1.2.4
+
+  - namespace: ns_with_wildcard_dep
+    name: name_with_wildcard_dep
+    version: 5.6.7-beta.3
+    dependencies:
+      dev_and_stables_ns.dev_and_stables_name: >-
+        *
+  - namespace: ns_with_dev_dep
+    name: name_with_dev_dep
+    version: 6.7.8
+    dependencies:
+      dev_and_stables_ns.dev_and_stables_name: 1.2.3-dev0
+
+  - namespace: rc_meta_ns_with_transitive_dev_dep
+    name: rc_meta_name_with_transitive_dev_dep
+    version: 2.4.5-rc5
+    dependencies:
+      ns_with_dev_dep.name_with_dev_dep: >-
+        *
+  - namespace: meta_ns_with_transitive_wildcard_dep
+    name: meta_name_with_transitive_wildcard_dep
+    version: 4.5.6
+    dependencies:
+      ns_with_wildcard_dep.name_with_wildcard_dep: 5.6.7-beta.3
--- a/test/integration/targets/ansible-galaxy-role/files/create-role-archive.py
+++ b/test/integration/targets/ansible-galaxy-role/files/create-role-archive.py
@ -0,0 +1,45 @@
+#!/usr/bin/env python
+"""Create a role archive which overwrites an arbitrary file."""
+
+import argparse
+import pathlib
+import tarfile
+import tempfile
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument('archive', type=pathlib.Path, help='archive to create')
+    parser.add_argument('content', type=pathlib.Path, help='content to write')
+    parser.add_argument('target', type=pathlib.Path, help='file to overwrite')
+
+    args = parser.parse_args()
+
+    create_archive(args.archive, args.content, args.target)
+
+
+def create_archive(archive_path: pathlib.Path, content_path: pathlib.Path, target_path: pathlib.Path) -> None:
+    with (
+        tarfile.open(name=archive_path, mode='w') as role_archive,
+        tempfile.TemporaryDirectory() as temp_dir_name,
+    ):
+        temp_dir_path = pathlib.Path(temp_dir_name)
+
+        meta_main_path = temp_dir_path / 'meta' / 'main.yml'
+        meta_main_path.parent.mkdir()
+        meta_main_path.write_text('')
+
+        symlink_path = temp_dir_path / 'symlink'
+        symlink_path.symlink_to(target_path)
+
+        role_archive.add(meta_main_path)
+        role_archive.add(symlink_path)
+
+        content_tarinfo = role_archive.gettarinfo(content_path, str(symlink_path))
+
+        with content_path.open('rb') as content_file:
+            role_archive.addfile(content_tarinfo, content_file)
+
+
+if __name__ == '__main__':
+    main()
--- a/test/integration/targets/ansible-galaxy-role/tasks/dir-traversal.yml
+++ b/test/integration/targets/ansible-galaxy-role/tasks/dir-traversal.yml
@ -0,0 +1,44 @@
+- name: create test directories
+  file:
+    path: '{{ remote_tmp_dir }}/dir-traversal/{{ item }}'
+    state: directory
+  loop:
+    - source
+    - target
+    - roles
+
+- name: create test content
+  copy:
+    dest: '{{ remote_tmp_dir }}/dir-traversal/source/content.txt'
+    content: |
+      some content to write
+
+- name: build dangerous dir traversal role
+  script:
+    chdir: '{{ remote_tmp_dir }}/dir-traversal/source'
+    cmd: create-role-archive.py dangerous.tar content.txt {{ remote_tmp_dir }}/dir-traversal/target/target-file-to-overwrite.txt
+    executable: '{{ ansible_playbook_python }}'
+
+- name: install dangerous role
+  command:
+    cmd: ansible-galaxy role install --roles-path '{{ remote_tmp_dir }}/dir-traversal/roles' dangerous.tar
+    chdir: '{{ remote_tmp_dir }}/dir-traversal/source'
+  ignore_errors: true
+  register: galaxy_install_dangerous
+
+- name: check for overwritten file
+  stat:
+    path: '{{ remote_tmp_dir }}/dir-traversal/target/target-file-to-overwrite.txt'
+  register: dangerous_overwrite_stat
+
+- name: get overwritten content
+  slurp:
+    path: '{{ remote_tmp_dir }}/dir-traversal/target/target-file-to-overwrite.txt'
+  register: dangerous_overwrite_content
+  when: dangerous_overwrite_stat.stat.exists
+
+- assert:
+    that:
+      - dangerous_overwrite_content.content|default('')|b64decode == ''
+      - not dangerous_overwrite_stat.stat.exists
+      - galaxy_install_dangerous is failed
--- a/test/integration/targets/ansible-galaxy-role/tasks/main.yml
+++ b/test/integration/targets/ansible-galaxy-role/tasks/main.yml
@ -64,3 +64,5 @@

 - name: Uninstall invalid role
  command: ansible-galaxy role remove invalid-testrole
+
+- import_tasks: dir-traversal.yml
--- a/test/integration/targets/ansible-pull/pull-integration-test/conn_secret.yml
+++ b/test/integration/targets/ansible-pull/pull-integration-test/conn_secret.yml
@ -0,0 +1,12 @@
+- hosts: localhost
+  gather_facts: false
+  tasks:
+    - ping: data='{{ansible_password}}'
+      register: dumb
+      vars:
+        ansible_python_interpreter: '{{ansible_playbook_python}}'
+
+    - name: If we got here, password was passed!
+      assert:
+        that:
+          - "dumb.ping == 'Testing123'"
--- a/test/integration/targets/ansible-pull/pull-integration-test/secret_connection_password
+++ b/test/integration/targets/ansible-pull/pull-integration-test/secret_connection_password
@ -0,0 +1 @@
+Testing123
--- a/test/integration/targets/ansible-pull/runme.sh
+++ b/test/integration/targets/ansible-pull/runme.sh
@ -36,7 +36,8 @@ function pass_tests {
 	fi

 	# test for https://github.com/ansible/ansible/issues/13681
-	if grep -E '127\.0\.0\.1.*ok' "${temp_log}"; then
+	# match play default output stats, was matching limit + docker
+	if grep -E '127\.0\.0\.1\s*: ok=' "${temp_log}"; then
 	    cat "${temp_log}"
 	    echo "Found host 127.0.0.1 in output. Only localhost should be present."
 	    exit 1
@ -84,4 +85,9 @@ pass_tests

 ANSIBLE_CONFIG='' ansible-pull -d "${pull_dir}" -U "${repo_dir}" "$@" multi_play_1.yml multi_play_2.yml | tee "${temp_log}"

-pass_tests_multi
+pass_tests_multi
+
+ANSIBLE_CONFIG='' ansible-pull -d "${pull_dir}" -U "${repo_dir}" conn_secret.yml --connection-password-file "${repo_dir}/secret_connection_password" "$@"
+
+# fail if we try do delete /var/tmp
+ANSIBLE_CONFIG='' ansible-pull -d var/tmp -U "${repo_dir}" --purge "$@"
--- a/test/integration/targets/ansible-test-container/runme.py
+++ b/test/integration/targets/ansible-test-container/runme.py
@ -1052,9 +1052,15 @@ class ApkBootstrapper(Bootstrapper):
        # The `openssl` package is used to generate hashed passwords.
        # crun added as podman won't install it as dep if runc is present
        # but we don't want runc as it fails
-        packages = ['docker', 'podman', 'openssl', 'crun']
+        # The edge `crun` package installed below requires ip6tables, and in
+        # edge, the `iptables` package includes `ip6tables`, but in 3.18 they
+        # are separate. Remove `ip6tables` once we update to 3.19.
+        packages = ['docker', 'podman', 'openssl', 'crun', 'ip6tables']

        run_command('apk', 'add', *packages)
+        # 3.18 only contains crun 1.8.4, to get 1.9.2 to resolve the run/shm issue, install crun from edge
+        # Remove once we update to 3.19
+        run_command('apk', 'upgrade', '-U', '--repository=http://dl-cdn.alpinelinux.org/alpine/edge/community', 'crun')
        run_command('service', 'docker', 'start')
        run_command('modprobe', 'tun')

--- a/test/integration/targets/ansible-vault/roles/test_vault_embedded/tasks/main.yml
+++ b/test/integration/targets/ansible-vault/roles/test_vault_embedded/tasks/main.yml
@ -2,7 +2,7 @@
 - name: Assert that a embedded vault of a string with no newline works
  assert:
    that:
-      - '"{{ vault_encrypted_one_line_var }}" == "Setec Astronomy"'
+      - 'vault_encrypted_one_line_var == "Setec Astronomy"'

 - name: Assert that a multi line embedded vault works, including new line
  assert:
--- a/test/integration/targets/ansible-vault/roles/test_vault_file_encrypted_embedded/tasks/main.yml
+++ b/test/integration/targets/ansible-vault/roles/test_vault_file_encrypted_embedded/tasks/main.yml
@ -2,7 +2,7 @@
 - name: Assert that a vault encrypted file with embedded vault of a string with no newline works
  assert:
    that:
-      - '"{{ vault_file_encrypted_with_encrypted_one_line_var }}" == "Setec Astronomy"'
+      - 'vault_file_encrypted_with_encrypted_one_line_var == "Setec Astronomy"'

 - name: Assert that a vault encrypted file with multi line embedded vault works, including new line
  assert:
--- a/test/integration/targets/apt_repository/tasks/apt.yml
+++ b/test/integration/targets/apt_repository/tasks/apt.yml
@ -50,7 +50,7 @@
    that:
      - 'result.changed'
      - 'result.state == "present"'
-      - 'result.repo == "{{test_ppa_name}}"'
+      - 'result.repo == test_ppa_name'

 - name: 'examine apt cache mtime'
  stat: path='/var/cache/apt/pkgcache.bin'
@ -81,7 +81,7 @@
    that:
      - 'result.changed'
      - 'result.state == "present"'
-      - 'result.repo == "{{test_ppa_name}}"'
+      - 'result.repo == test_ppa_name'

 - name: 'examine apt cache mtime'
  stat: path='/var/cache/apt/pkgcache.bin'
@ -112,7 +112,7 @@
    that:
      - 'result.changed'
      - 'result.state == "present"'
-      - 'result.repo == "{{test_ppa_name}}"'
+      - 'result.repo == test_ppa_name'

 - name: 'examine apt cache mtime'
  stat: path='/var/cache/apt/pkgcache.bin'
@ -151,7 +151,7 @@
    that:
      - 'result.changed'
      - 'result.state == "present"'
-      - 'result.repo == "{{test_ppa_spec}}"'
+      - 'result.repo == test_ppa_spec'
      - '"sources_added" in result'
      - 'result.sources_added | length == 1'
      - '"git" in result.sources_added[0]'
@ -176,7 +176,7 @@
    that:
      - 'result.changed'
      - 'result.state == "absent"'
-      - 'result.repo == "{{test_ppa_spec}}"'
+      - 'result.repo == test_ppa_spec'
      - '"sources_added" in result'
      - 'result.sources_added | length == 0'
      - '"sources_removed" in result'
@ -207,7 +207,7 @@
    that:
      - 'result.changed'
      - 'result.state == "present"'
-      - 'result.repo == "{{test_ppa_spec}}"'
+      - 'result.repo == test_ppa_spec'

 - name: 'examine source file'
  stat: path='/etc/apt/sources.list.d/{{test_ppa_filename}}.list'
--- a/test/integration/targets/assert/assert.out.nested_tmpl.stderr
+++ b/test/integration/targets/assert/assert.out.nested_tmpl.stderr
@ -0,0 +1,4 @@
+ ansible-playbook -i localhost, -c local nested_tmpl.yml
++ set +x
+[WARNING]: conditional statements should not include jinja2 templating
+delimiters such as {{ }} or {% %}. Found: "{{ foo }}" == "bar"
--- a/test/integration/targets/assert/assert.out.nested_tmpl.stdout
+++ b/test/integration/targets/assert/assert.out.nested_tmpl.stdout
@ -0,0 +1,12 @@
+
+PLAY [localhost] ***************************************************************
+
+TASK [assert] ******************************************************************
+ok: [localhost] => {
+    "changed": false,
+    "msg": "All assertions passed"
+}
+
+PLAY RECAP *********************************************************************
+localhost                  : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
+
--- a/test/integration/targets/assert/assert_quiet.out.quiet.stderr
+++ b/test/integration/targets/assert/assert_quiet.out.quiet.stderr
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Matt Martz	71a1f55a34	New release v2.16.2 (#82399 )	2023-12-11 14:48:58 -06:00
Matt Martz	afe3fc184f	Additional Unsafe fixes (#82376 ) * Allow older pickle protocols to pickle unsafe classes. Fixes #82356 * Address issues when iterating or getting single index from AnsibleUnsafeBytes. Fixes #82375 * clog frag	2023-12-07 17:17:14 -06:00
Sviatoslav Sydorenko	d949af5093	[backport][stable-2.16] 🧪 Replace GitHub SVN integration test with local TLS (#82368 ) * [backport][stable-2.16] 🧪 Replace GitHub SVN integration test with local TLS PR #82334 * Run svn integration test locally with TLS This patch uses a `trustme` to make an ephemeral CA, and server, and client TLS artifacts for testing. These are integrated into the Apache web server via it's `mod_ssl`. Resolves #82207 * Replace GitHub SVN HTTPS URL w/ localhost over TLS This change gets rid of the need to use GitHub, which is just about to drop support for SVN [[1]]. Moreover, it eliminates the need to use external network for any SVN commands in the test. [1]: https://github.blog/2023-01-20-sunsetting-subversion-support/ (cherry picked from commit `dd0138ba21`) * Implement compat w/ opensuse15, centos7 & rhel7/8	2023-12-06 16:18:19 -06:00
Matt Martz	94e8d99ef4	Update Ansible release version to v2.16.1.post0. (#82349 )	2023-12-04 22:27:00 +00:00
Matt Martz	6d40beeef4	New release v2.16.1 (#82346 )	2023-12-04 15:46:24 -06:00
Matt Clay	59aa0145d2	Fix various issues in unsafe_proxy (#82326 ) - Use str/bytes directly instead of text_type/binary_type - Fix AnsibleUnsafeBytes.__str__ implementation - Fix AnsibleUnsafeBytes.__format__ return type - Remove invalid methods from AnsibleUnsafeBytes (casefold, format, format_map) - Use `chars` instead of `bytes` to match stdlib naming - Remove commented out code	2023-12-01 09:28:05 -08:00
Matt Martz	6655343d6d	Update Ansible release version to v2.16.1rc1.post0. (#82304 )	2023-11-27 22:05:00 +00:00
Matt Martz	5007068bfe	New release v2.16.1rc1 (#82301 )	2023-11-27 15:24:38 -06:00
Matt Martz	270b39f6ff	Ensure that unsafe is more difficult to lose [stable-2.16] (#82293 ) * Ensure that unsafe is more difficult to lose * Add Task.untemplated_args, and switch assert over to use it * Don't use re in first_found, switch to using native string methods * If nested templating results in unsafe, just error, don't continue * ci_complete	2023-11-27 10:26:45 -08:00
Martin Krizek	f302b2f592	Allow include_tasks handlers for searching role subdirs (#82248 ) (#82268 ) Fixes #82241 (cherry picked from commit `d664f13b4a`)	2023-11-27 17:55:57 +00:00
Brian Coca	bb787c119d	ansible-pull: expand destinantion directoy to avoid purgin in / (#82030 ) (#82221 ) * expand destinantion directoy to avoid purgin in / bad things could happen and help alone is not enough (cherry picked from commit `8825e60add`)	2023-11-27 17:31:33 +00:00
Brian Coca	8a87e1c5d3	no_log avoid masking booleans (#82217 ) (#82235 ) * no_log avoid masking booleans (#82217) * no_log avoid masking booleans * clog * fix issues (cherry picked from commit `6e448edc63`) * unused boil is hot	2023-11-27 08:29:41 -08:00
Brian Coca	fb5d254a79	wait_for, fallback to read for non mmapable files (#82064 ) (#82233 ) * wait_for, fallback to read for non mmapable files (#82064) * also handle oserror, added debug jic (cherry picked from commit `8b102dca4a`) * skip problem versions	2023-11-27 08:28:03 -08:00
Martin Krizek	2f7376ce06	flush_handlers: handle a failure in a nested block with force_handlers (#81572 ) (#82197 ) Fixes #81532 ci_complete (cherry picked from commit `a8b6ef7e7c`)	2023-11-27 08:25:54 -08:00
Maxwell G	32aedb8bab	setup.cfg: add `Programming Language :: Python :: 3.12` classifier (#82152 ) ansible-core 2.16+ supports Python 3.12. This corrects missing metadata.	2023-11-27 08:25:01 -08:00
Brian Coca	71b00cefac	restore role param precedence (#82106 ) (#82138 ) * add test for setfact/param override (cherry picked from commit `20a54eb236`)	2023-11-27 08:24:25 -08:00
Abhijeet Kasurde	6f8261bcf9	[bp-2.16] Fix the strategy plugin description in keywords (#82133 ) Fixes: https://github.com/ansible/ansible-documentation/issues/762 Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com> (cherry picked from commit `8ecc1d485b`)	2023-11-27 08:23:15 -08:00
Jordan Borean	c2a3cad466	Remove duplicate winrm test (#82124 ) (#82127 ) (cherry picked from commit `f8a9fd4758`)	2023-11-27 08:22:29 -08:00
Matt Davis	75bce4b9d5	add pep668 note to pip module docs (#82096 ) (#82098 ) (cherry picked from commit `3ea5304a57`)	2023-11-27 07:38:15 -08:00
Brian Coca	e13569d0e2	ansible-pull now handles all secret files CLI options (#82009 ) (#82070 ) * ansible-pull added missing pasthrough for secrets Both become and connection password file options were missing. Also added test (cherry picked from commit `99e0d25857`)	2023-11-27 07:37:51 -08:00
Brian Coca	6cfa8ec021	Fix Jinja plugin deduplication (#82002 ) (#82053 ) for j2 plugins dedupe on path and not basename for j2 this is a container file , for other plugins file name == plugin name (cherry picked from commit `b4566c18b3`)	2023-11-27 07:37:00 -08:00
Abhijeet Kasurde	859f2876f0	[bp-2.16] Update Python3 (#82203 ) Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com> (cherry picked from commit `c5b68ef16d`)	2023-11-13 13:21:46 -08:00
Matt Martz	862c5824d0	Update Ansible release version to v2.16.0.post0. (#82144 )	2023-11-06 18:41:14 +00:00
Matt Martz	704ee70943	New release v2.16.0 (#82141 )	2023-11-06 11:56:55 -06:00
Matt Davis	a13df22c02	remove args passthru on role runme tests that use grep (#82060 ) (#82061 ) * varying verbosity was masking some first-pass test failures (cherry picked from commit `40263992df`)	2023-10-23 17:59:01 +00:00
Matt Clay	31aac52a72	[stable-2.16] ansible-test - Fix parsing of cgroup entries (cherry picked from commit `e933d9d8a6`) Co-authored-by: Matt Clay <matt@mystile.com>	2023-10-23 09:47:54 -07:00
Matt Clay	00f74e03b2	[stable-2.16] Fix subversion integration test (#82029 ) - Remove dependency on the htpasswd module (and thus passlib) - Fix setup/teardown of the httpd process - Fix cleanup of temporary directories. (cherry picked from commit `09d943445c`) Co-authored-by: Matt Clay <matt@mystile.com>	2023-10-23 09:46:55 -07:00
Matt Martz	f6fb5b2031	Update Ansible release version to v2.16.0rc1.post0. (#81987 )	2023-10-16 22:07:45 +00:00
Matt Martz	2223709bdd	New release v2.16.0rc1 (#81986 )	2023-10-16 16:27:53 -05:00
Matt Clay	33248690fe	ansible-test - Update base/default containers (#81956 )	2023-10-11 16:37:29 -07:00
Matt Clay	f9dfd28cff	[stable-2.16] Update default test container Python to 3.12 and support for PyLint 3.0.1 (#81953 ) (#81955 ) * temporary PyLint plugin to mask 3.12 finalizer noise * ansible-test - Default to Python 3.12 for base/default * ansible-test - Update pylint requirements * ansible-test - Remove obsoleted changelog entry * Add changelog fragment for pylint work-around. (cherry picked from commit `d8484f0af7`)	2023-10-11 13:53:55 -07:00
Martin Krizek	9c91e578d0	[stable-2.16] run_once: unnotify hosts on handlers that are not run (#81667 ) (#81920 ) Fixes #81666 (cherry picked from commit `2d5861c`)	2023-10-09 11:34:58 -05:00
Martin Krizek	f3f0e6a0f8	[stable-2.16] Properly template tags in parent blocks (#81624 ) (#81921 ) When templating tags (which happens outside of standard `post_validate`) we need to template each object in the inheritance chain and set the templated values on those objects individually. That way when `task.tags` is called the `extend` functionality properly picks up the templated values of all parents into one flatten list. Fixes #81053 (cherry picked from commit `9b3ed5e`)	2023-10-09 11:34:48 -05:00
Jordan Borean	e856696f09	ansible-galaxy - fix traceback error for invalid req file (#81917 ) (#81924 ) Provide a better error message when encountering a YAML requirements file that is not a dictionary or list. Fixes: #81901 (cherry picked from commit `8a5ccc9d63`)	2023-10-09 11:34:36 -05:00
Jordan Borean	c70f7d829f	Add test coverage for winrm (#81910 ) - 2.16 (#81914 ) * Add test coverage for winrm (#81910) (cherry picked from commit `282908c57e`) * Add more winrm coverage (#81923) (cherry picked from commit `976067c15f`)	2023-10-09 10:59:32 -05:00
Sloane Hertel	889248bcf7	optimize host_group_vars and vars plugin loading (#79945 ) (#81878 ) * Improve host_group_vars efficiency: * normalize the basedir with `os.path.realpath()` once and cache it * cache missing paths/files * reduce the calls to `isinstance` Add a couple more general improvements in vars/plugins.py get_vars_from_path(): * call `PluginLoader.all()` once for vars plugins and reload specific plugins subsequently * don't reload legacy/builtin vars plugins that are not enabled Add a test for host_group_vars and legacy plugin loading Co-authored-by: Matt Davis <mrd@redhat.com> * changelog * Add a new is_stateless attribute to the vars plugin baseclass update integration tests to be quieter and use the same test pattern Fix deprecation and adjust test that didn't catch the issue (deprecation only occured when the value was False) move realpath cache to host_group_vars (do not smuggle call state as instance data) refactor under a single 'if cache:' statement Call os.path.isdir instead of always calling os.path.exists first. Just call os.path.exists to differentiate between missing and non-directory. remove call to super(VarsModule, self).get_vars() use the entity name as the cache key instead of variable location Remove isinstance checks and use a class attribute just in case any plugins are subclassing Host/Group Replace startswith by checking index 0 of the name instead, since host/group names are required * rename is_stateless to cache_instance to make it more clear what it does * add plugin instance cache using the path to plugin loader reduce loading stage option if a new instance isn't created don't require a known subdir on PluginLoader instantiation for backwards compatibility rename attribute again contain reading from/initializing cached instances to a plugin loader method * Deprecate v2 vars plugins * Refactor to use the cache in existing plugin loader methods Rename the attribute again Refactor host_group_vars with requested changes Make changelog a bugfixes fragment Add a deprecation fragment for v2 vars plugins. Add type hints * unbreak group_vars * Apply suggestions from code review * misc tweaks * always cache instance by both requested and resolved FQ name * add lru_cache to stage calculation to avoid repeated config consultation * handle KeyError from missing stage option --------- Co-authored-by: Matt Davis <mrd@redhat.com> (cherry picked from commit `debf2be913`)	2023-10-09 10:59:22 -05:00
Matt Davis	7d789de400	switch available_hosts to `frozenset` from `list` in ansible-inventory (#81870 ) (#81875 ) * major speedup large inventories, since it's consulted iteratively (cherry picked from commit `c1343cc304`)	2023-10-09 10:59:09 -05:00
Jordan Borean	5665eca5e9	win_fetch - improve test time by not scanning Win dir (#81884 ) (#81888 ) (cherry picked from commit `dfc62589f6`)	2023-10-04 10:24:30 -07:00
Matt Martz	4ac5b51529	Update Ansible release version to v2.16.0b2.post0. (#81868 )	2023-10-03 17:37:33 -04:00
Matt Martz	2dd22e3025	New release v2.16.0b2 (#81864 )	2023-10-03 16:01:03 -04:00
Matt Clay	2578664198	[stable-2.16] ansible-test - Update distro containers (#81851 ) (#81859 ) * ansible-test - Update distro containers * Update the alpine3 container to Python 3.11 * Update changelog. (cherry picked from commit `b7903669b4`)	2023-10-03 18:29:45 +00:00
Brian Coca	9a4bc7e7b3	Restore import_role variable exporting behavior (#81840 ) * Import role public (#81772) revert to previous behavior to push vars to play at compile time add `public` parameter to allow per import control of exporting (vs just the global config) Co-authored-by: tchernomax <maxime.deroucy@gmail.com> Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com> (cherry picked from commit `ab6a544e86`) * adapted to prev version - removed new functionality - restored global config functioning overriding specific public option * remove typoe * quote it	2023-10-03 13:59:01 -04:00
Sloane Hertel	cbae91501a	Update the test containers to include Python 3.12.0 final (#81857 )	2023-10-03 10:51:58 -07:00
Matt Clay	8c4736dd76	[stable-2.16] ansible-test - Invoke container sleep through env (#81853 ) (#81856 ) (cherry picked from commit `20f17687da`)	2023-10-03 16:50:15 +00:00
Abhijeet Kasurde	a8e59b2547	[backport/2.16] Bump Pylint sanity test requirements for 3.12 (#81850 ) Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>	2023-10-03 07:51:55 -07:00
Abhijeet Kasurde	de732ed712	[backport/2.16] Bump sanity test requirements for 3.12 (#81848 )	2023-10-03 07:51:37 -07:00
Sviatoslav Sydorenko	c7793e65ec	Bump `coverage.py` to v7.3.2 in `ansible-test` (#81844 ) (#81845 ) (cherry picked from commit `ec265ec08a`)	2023-10-03 09:13:06 -04:00
Matt Clay	58cf53b3c3	[stable-2.16] ansible-test - Improve compatibility with pylint 3 (#81841 ) (#81842 ) (cherry picked from commit `f31c287348`)	2023-10-02 13:07:27 -07:00
Sloane Hertel	755a7c7387	Fix reporting role as not found when remote_data is None (#81829 ) (#81838 ) (cherry picked from commit `7fab552563`)	2023-10-02 19:57:23 +00:00
Matt Martz	5008c56c50	[stable-2.16] Install upgraded crun from edge (#81833 ) (#81839 ) (cherry picked from commit `e78cc1e`)	2023-10-02 18:58:19 +00:00
Matt Martz	831dc6e444	Add compat function for parsing Content-Disposition header (#81807 ) * py2 compat for get_param * Add tests, and handle ValueError * Add clog frag	2023-10-02 09:30:33 -04:00
Matt Clay	c2567b1d67	[stable-2.16] ansible-test - Update sanity test requirements (#81810 ) (#81811 ) (cherry picked from commit `d6a356d7cd`)	2023-09-28 15:49:58 -07:00
Jordan Borean	9632086901	winrm - make command input more resiliant (#81538 ) (#81794 ) * winrm - make command input more resiliant * Expand warning message (cherry picked from commit `f22231de20`)	2023-09-27 09:10:59 -05:00
Sloane Hertel	42d7a06cd8	Update test container versions for stable-2.16 (#81790 )	2023-09-27 09:10:33 -05:00
Matt Martz	fffb3c403f	[stable-2.16] Prevent roles from using symlinks to overwrite files outside of the installation directory (#81780 ) (#81783 ) * Sanitize linkname during role installs * Add tests * add clog frag (cherry picked from commit `ddf0311`)	2023-09-27 08:44:18 -05:00
Matt Martz	c6affcf7c1	Update Ansible release version to v2.16.0b1.post0. (#81779 )	2023-09-26 11:19:49 -05:00
Matt Martz	00ca6b94d0	New release v2.16.0b1 (#81778 )	2023-09-26 09:37:26 -05:00
Val V	37a0754cbf	Daemonize follow-up fixes (#81584 ) (#81766 ) * Ensure binary data transmission in daemonize * Add changelog fragment (cherry picked from commit `91f94fb59d`)	2023-09-26 08:56:38 -05:00
Martin Krizek	9033002a86	Allow for searching handler subdir for included task via include_role (#81733 ) (#81759 ) Fixes #81722 (cherry picked from commit `1e7f7875c6`)	2023-09-26 08:51:57 -05:00
Sviatoslav Sydorenko	e470954985	Always allow "no-other-choice" pre-release dependencies when resolving collection dependency tree (#81746 ) PR #81606. Prior to this patch, when `--pre` CLI flag was not passed, the dependency resolver would treat concrete collection dependency candidates (Git repositories, subdirs, tarball URLs, or local dirs or files etc) as not meeting the requirements. This patch makes it so pre-releases in any concrete artifact references, and the ones being specifically pinned dependencies or user requests, met anywhere in the dependency tree, are allowed unconditionally. This is achieved by moving the pre-release check from `is_satisfied_by()` to the `find_matches()` hook, following the Pip's example. As a bonus, this change also fixes the situation when a collection pre-releases weren't considered if it didn't have any stable releases. This now works even if `--pre` wasn't requested explicitly. Finally, this patch partially reverts commit `6f4b4c345b`, except for the tests. And it also improves the `--pre` hint warning to explain that it mostly affects Galaxy/Automation Hub-hosted collection releases. Ref #73416 Ref #79112 Fixes #79168 Fixes #80048 Resolves #81605 Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com> (cherry picked from commit `7662a05085`)	2023-09-26 08:37:16 -05:00
Matt Clay	e3eecf096f	[stable-2.16] ansible-test - Skip pylint test on Python 3.12 (#81706 ) (#81757 ) (cherry picked from commit `3794612832`)	2023-09-21 23:05:04 -07:00