New release v2.18.4rc1 (#84842 )

[bp-2.18] facts: use pagesize for darwin (#84809 )
Fixes: #84773 Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com> (cherry picked from commit 50b4e0d279)
2025-03-17 15:08:30 -05:00 · 2025-03-12 15:44:23 -05:00 · 2025-03-12 15:44:14 -05:00 · 2025-03-12 15:43:59 -05:00 · 2025-03-10 11:35:10 -05:00 · 2025-03-10 11:34:57 -05:00
282 changed files with 4823 additions and 916 deletions
--- a/changelogs/CHANGELOG-v2.18.rst
+++ b/changelogs/CHANGELOG-v2.18.rst
@ -0,0 +1,360 @@
+==================================================
+ansible-core 2.18 "Fool in the Rain" Release Notes
+==================================================
+
+.. contents:: Topics
+
+v2.18.4rc1
+==========
+
+Release Summary
+---------------
+
+| Release Date: 2025-03-17
+| `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
+
+Bugfixes
+--------
+
+- Windows - add support for running on system where WDAC is in audit mode with ``Dynamic Code Security`` enabled.
+- dnf5 - fix ``is_installed`` check for packages that are not installed but listed as provided by an installed package (https://github.com/ansible/ansible/issues/84578)
+- dnf5 - libdnf5 - use ``conf.pkg_gpgcheck`` instead of deprecated ``conf.gpgcheck`` which is used only as a fallback
+- facts - gather pagesize and calculate respective values depending upon architecture (https://github.com/ansible/ansible/issues/84773).
+- module respawn - limit to supported Python versions
+
+v2.18.3
+=======
+
+Release Summary
+---------------
+
+| Release Date: 2025-02-24
+| `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
+
+Minor Changes
+-------------
+
+- ansible-test - Automatically retry HTTP GET/PUT/DELETE requests on exceptions.
+- ansible-test - Use Python's ``urllib`` instead of ``curl`` for HTTP requests.
+
+Bugfixes
+--------
+
+- include_vars - fixed erroneous warning if an unreserved variable name contains a single character that matches a reserved variable. (https://github.com/ansible/ansible/issues/84623)
+- linear strategy - fix executing ``end_role`` meta tasks for each host, instead of handling these as implicit run_once tasks (https://github.com/ansible/ansible/issues/84660).
+
+v2.18.2
+=======
+
+Release Summary
+---------------
+
+| Release Date: 2025-01-27
+| `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
+
+Bugfixes
+--------
+
+- Ansible will now also warn when reserved keywords are set via a module (set_fact, include_vars, etc).
+- Ansible.Basic - Fix ``required_if`` check when the option value to check is unset or set to null.
+- Use consistent multiprocessing context for action write locks
+- ansible-test - Fix up coverage reporting to properly translate the temporary path of integration test modules to the expected static test module path.
+- ansible-vault will now correctly handle `--prompt`, previously it would issue an error about stdin if no 2nd argument was passed
+- copy action now prevents user from setting internal options.
+- gather_facts action now defaults to `ansible.legacy.setup` if `smart` was set, no network OS was found and no other alias for `setup` was present.
+- gather_facts action will now issues errors and warnings as appropriate if a network OS is detected but no facts modules are defined for it.
+- ssh - Improve the logic for parsing CLIXML data in stderr when working with Windows host. This fixes issues when the raw stderr contains invalid UTF-8 byte sequences and improves embedded CLIXML sequences.
+- ssh - connection options were incorrectly templated during ``reset_connection`` tasks (https://github.com/ansible/ansible/pull/84238).
+
+v2.18.1
+=======
+
+Release Summary
+---------------
+
+| Release Date: 2024-12-02
+| `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
+
+Minor Changes
+-------------
+
+- ansible-test - When detection of the current container network fails, a warning is now issued and execution continues. This simplifies usage in cases where the current container cannot be inspected, such as when running in GitHub Codespaces.
+
+Security Fixes
+--------------
+
+- Templating will not prefer AnsibleUnsafe when a variable is referenced via hostvars - CVE-2024-11079
+
+Bugfixes
+--------
+
+- Fix returning 'unreachable' for the overall task result. This prevents false positives when a looped task has unignored unreachable items (https://github.com/ansible/ansible/issues/84019).
+- ansible-test - Fix traceback that occurs after an interactive command fails.
+- dnf5 - fix installing a package using ``state=latest`` when a binary of the same name as the package is already installed (https://github.com/ansible/ansible/issues/84259)
+- dnf5 - matching on a binary can be achieved only by specifying a full path (https://github.com/ansible/ansible/issues/84334)
+- runas become - Fix up become logic to still get the SYSTEM token with the most privileges when running as SYSTEM.
+
+v2.18.0
+=======
+
+Release Summary
+---------------
+
+| Release Date: 2024-11-04
+| `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
+
+Minor Changes
+-------------
+
+- Add ``gid_min``, ``gid_max`` to the group plugin to overwrite the defaults provided by the ``/etc/login.defs`` file (https://github.com/ansible/ansible/pull/81770).
+- Add ``python3.13`` to the default ``INTERPRETER_PYTHON_FALLBACK`` list.
+- Add ``uid_min``, ``uid_max`` to the user plugin to overwrite the defaults provided by the ``/etc/login.defs`` file (https://github.com/ansible/ansible/pull/81770).
+- Add a new meta task ``end_role`` (https://github.com/ansible/ansible/issues/22286)
+- Add a new mount_facts module to support gathering information about mounts that are excluded by default fact gathering.
+- Introducing COLOR_INCLUDED parameter. This can set a specific color for "included" events.
+- Removed the shell ``environment`` config entry as this is already covered by the play/task directives documentation and the value itself is not used in the shell plugins. This should remove any confusion around how people set the environment for a task.
+- Suppress cryptography deprecation warnings for Blowfish and TripleDES when the ``paramiko`` Python module is installed.
+- The minimum supported Python version on targets is now Python 3.8.
+- ``ansible-galaxy collection publish`` - add configuration options for the initial poll interval and the exponential when checking the import status of a collection, since the default is relatively slow.
+- ansible-config has new 'validate' option to find mispelled/forgein configurations in ini file or environment variables.
+- ansible-doc - show examples in role entrypoint argument specs (https://github.com/ansible/ansible/pull/82671).
+- ansible-galaxy - Handle authentication errors and token expiration
+- ansible-test - Add Ubuntu 24.04 remote.
+- ansible-test - Add support for Python 3.13.
+- ansible-test - An ``ansible_core.egg-info`` directory is no longer generated when running tests.
+- ansible-test - Connection options can be set for ansible-test managed remote Windows instances.
+- ansible-test - Default to Python 3.13 in the ``base`` and ``default`` containers.
+- ansible-test - Disable the ``deprecated-`` prefixed ``pylint`` rules as their results vary by Python version.
+- ansible-test - Improve container runtime probe error handling. When unexpected probe output is encountered, an error with more useful debugging information is provided.
+- ansible-test - Improve the error message shown when an unknown ``--remote`` or ``--docker`` option is given.
+- ansible-test - Remove Python 2.7 compatibility imports.
+- ansible-test - Removed the ``vyos/1.1.8`` network remote as it is no longer functional.
+- ansible-test - Replace Alpine 3.19 container and remote with Alpine 3.20.
+- ansible-test - Replace Fedora 39 container and remote with Fedora 40.
+- ansible-test - Replace FreeBSD 14.0 remote with FreeBSD 14.1.
+- ansible-test - Replace RHEL 9.3 remote with RHEL 9.4.
+- ansible-test - Replace Ubuntu 20.04 container with Ubuntu 24.04 container.
+- ansible-test - The ``empty-init`` sanity test no longer applies to ``module_utils`` packages.
+- ansible-test - Update ``ansible-test-utility-container`` to version 3.1.0.
+- ansible-test - Update ``base`` and ``default`` containers to omit Python 3.7.
+- ansible-test - Update ``coverage`` to version 7.6.1.
+- ansible-test - Update ``http-test-container`` to version 3.0.0.
+- ansible-test - Update ``nios-test-container`` to version 5.0.0.
+- ansible-test - Update ``pylint`` sanity test to use version 3.3.1.
+- ansible-test - Update ``pypi-test-container`` to version 3.2.0.
+- ansible-test - Update the ``base`` and ``default`` containers.
+- ansible-test - Updated the frozen requirements for all sanity tests.
+- ansible-test - Upgrade ``pip`` used in ansible-test managed virtual environments from version 24.0 to 24.2.
+- ansible-test - Virtual environments created by ansible-test no longer include the ``wheel`` or ``setuptools`` packages.
+- ansible-test - update HTTP test container to 3.2.0 (https://github.com/ansible/ansible/pull/83469).
+- ansible.log now also shows log severity field
+- distribution.py - Added SL-Micro in Suse OS Family. (https://github.com/ansible/ansible/pull/83541)
+- dnf - minor internal changes in how the errors from the dnf API are handled; rely solely on the exceptions rather than inspecting text embedded in them
+- dnf - remove legacy code for unsupported dnf versions
+- dnf5 - implement ``enable_plugin`` and ``disable_plugin`` options
+- fact gathering - Gather /proc/sysinfo facts on s390 Linux on Z
+- facts - add systemd version and features
+- find - change the datatype of ``elements`` to ``path`` in option ``paths`` (https://github.com/ansible/ansible/pull/83575).
+- ini lookup - add new ``interpolation`` option (https://github.com/ansible/ansible/issues/83755)
+- isidentifier - remove unwanted Python 2 specific code.
+- loop_control - add a break_when option to to break out of a task loop early based on Jinja2 expressions (https://github.com/ansible/ansible/issues/83442).
+- package_facts module now supports using aliases for supported package managers, for example managers=yum or managers=dnf will resolve to using the underlying rpm.
+- plugins, deprecations and warnings concerning configuration are now displayed to the user, technical issue that prevented 'de-duplication' have been resolved.
+- psrp - Remove connection plugin extras vars lookup. This should have no affect on existing users as all options have been documented.
+- remove extraneous selinux import (https://github.com/ansible/ansible/issues/83657).
+- replace random with secrets library.
+- rpm_key - allow validation of gpg key with a subkey fingerprint
+- rpm_key - enable gpg validation that requires presence of multiple fingerprints
+- service_mgr - add support for dinit service manager (https://github.com/ansible/ansible/pull/83489).
+- task timeout now returns timedout key with frame/code that was in execution when the timeout is triggered.
+- timedout test for checking if a task result represents a 'timed out' task.
+- unarchive - Remove Python 2.7 compatibility imports.
+- validate-modules sanity test - detect if names of an option (option name + aliases) do not match between argument spec and documentation (https://github.com/ansible/ansible/issues/83598, https://github.com/ansible/ansible/pull/83599).
+- validate-modules sanity test - reject option/aliases names that are identical up to casing but belong to different options (https://github.com/ansible/ansible/pull/83530).
+- vaulted_file test filter added, to test if the provided path is an 'Ansible vaulted' file
+- yum_repository - add ``excludepkgs`` alias to the ``exclude`` option.
+
+Breaking Changes / Porting Guide
+--------------------------------
+
+- Stopped wrapping all commands sent over SSH on a Windows target with a ``powershell.exe`` executable. This results in one less process being started on each command for Windows to improve efficiency, simplify the code, and make ``raw`` an actual raw command run with the default shell configured on the Windows sshd settings. This should have no affect on most tasks except for ``raw`` which now is not guaranteed to always be running in a PowerShell shell and from having the console output codepage set to UTF-8. To avoid this issue either swap to using ``ansible.windows.win_command``, ``ansible.windows.win_shell``, ``ansible.windows.win_powershell`` or manually wrap the raw command with the shell commands needed to set the output console encoding.
+- persistent connection plugins - The ``ANSIBLE_CONNECTION_PATH`` config option no longer has any effect.
+
+Deprecated Features
+-------------------
+
+- Deprecate ``ansible.module_utils.basic.AnsibleModule.safe_eval`` and ``ansible.module_utils.common.safe_eval`` as they are no longer used.
+- persistent connection plugins - The ``ANSIBLE_CONNECTION_PATH`` config option no longer has any effect, and will be removed in a future release.
+- yum_repository - deprecate ``async`` option as it has been removed in RHEL 8 and will be removed in ansible-core 2.22.
+- yum_repository - the following options are deprecated: ``deltarpm_metadata_percentage``, ``gpgcakey``, ``http_caching``, ``keepalive``, ``metadata_expire_filter``, ``mirrorlist_expire``, ``protect``, ``ssl_check_cert_permissions``, ``ui_repoid_vars`` as they have no effect for dnf as an underlying package manager. The options will be removed in ansible-core 2.22.
+
+Removed Features (previously deprecated)
+----------------------------------------
+
+- Play - removed deprecated ``ROLE_CACHE`` property in favor of ``role_cache``.
+- Remove deprecated `VariableManager._get_delegated_vars` method (https://github.com/ansible/ansible/issues/82950)
+- Removed Python 3.10 as a supported version on the controller. Python 3.11 or newer is required.
+- Removed support for setting the ``vars`` keyword to lists of dictionaries. It is now required to be a single dictionary.
+- loader - remove deprecated non-inclusive words (https://github.com/ansible/ansible/issues/82947).
+- paramiko_ssh - removed deprecated ssh_args from the paramiko_ssh connection plugin (https://github.com/ansible/ansible/issues/82939).
+- paramiko_ssh - removed deprecated ssh_common_args from the paramiko_ssh connection plugin (https://github.com/ansible/ansible/issues/82940).
+- paramiko_ssh - removed deprecated ssh_extra_args from the paramiko_ssh connection plugin (https://github.com/ansible/ansible/issues/82941).
+- play_context - remove deprecated PlayContext.verbosity property (https://github.com/ansible/ansible/issues/82945).
+- utils/listify - remove deprecated 'loader' argument from listify_lookup_plugin_terms API (https://github.com/ansible/ansible/issues/82949).
+
+Security Fixes
+--------------
+
+- include_vars action - Ensure that result masking is correctly requested when vault-encrypted files are read. (CVE-2024-8775)
+- task result processing - Ensure that action-sourced result masking (``_ansible_no_log=True``) is preserved. (CVE-2024-8775)
+- user action won't allow ssh-keygen, chown and chmod to run on existing ssh public key file, avoiding traversal on existing symlinks (CVE-2024-9902).
+
+Bugfixes
+--------
+
+- -> runas become - Generate new token for the SYSTEM token to use for become. This should result in the full SYSTEM token being used and problems starting the process that fails with ``The process creation has been blocked``.
+- Add a version ceiling constraint for pypsrp to avoid potential breaking changes in the 1.0.0 release.
+- Add descriptions for ``ansible-galaxy install --help` and ``ansible-galaxy role|collection install --help``.
+- Avoid truncating floats when casting into int, as it can lead to truncation and unexpected results. 0.99999 will be 0, not 1.
+- COLOR_SKIP will not alter "included" events color display anymore.
+- Callbacks now correctly get the resolved connection plugin name as the connection used.
+- Darwin - add unit tests for Darwin hardware fact gathering.
+- Errors now preserve stacked error messages even when YAML is involved.
+- Fix ``SemanticVersion.parse()`` to store the version string so that ``__repr__`` reports it instead of ``None`` (https://github.com/ansible/ansible/pull/83831).
+- Fix a traceback when an environment variable contains certain special characters (https://github.com/ansible/ansible/issues/83498)
+- Fix an issue when setting a plugin name from an unsafe source resulted in ``ValueError: unmarshallable object`` (https://github.com/ansible/ansible/issues/82708)
+- Fix an issue where registered variable was not available for templating in ``loop_control.label`` on skipped looped tasks (https://github.com/ansible/ansible/issues/83619)
+- Fix disabling SSL verification when installing collections and roles from git repositories. If ``--ignore-certs`` isn't provided, the value for the ``GALAXY_IGNORE_CERTS`` configuration option will be used (https://github.com/ansible/ansible/issues/83326).
+- Fix for ``meta`` tasks breaking host/fork affinity with ``host_pinned`` strategy (https://github.com/ansible/ansible/issues/83294)
+- Fix handlers not being executed in lockstep using the linear strategy in some cases (https://github.com/ansible/ansible/issues/82307)
+- Fix rapid memory usage growth when notifying handlers using the ``listen`` keyword (https://github.com/ansible/ansible/issues/83392)
+- Fix the task attribute ``resolved_action`` to show the FQCN instead of ``None`` when ``action`` or ``local_action`` is used in the playbook.
+- Fix using ``module_defaults`` with ``local_action``/``action`` (https://github.com/ansible/ansible/issues/81905).
+- Fix using the current task's directory for looking up relative paths within roles (https://github.com/ansible/ansible/issues/82695).
+- Improve performance on large inventories by reducing the number of implicit meta tasks.
+- Remove deprecated config options DEFAULT_FACT_PATH, DEFAULT_GATHER_SUBSET, and DEFAULT_GATHER_TIMEOUT in favor of setting ``fact_path``, ``gather_subset`` and ``gather_timeout`` as ``module_defaults`` for ``ansible.builtin.setup``.
+  These will apply to both the ``gather_facts`` play keyword, and any ``ansible.builtin.setup`` tasks.
+  To configure these options only for the ``gather_facts`` keyword, set these options as play keywords also.
+- Set LANGUAGE environment variable is set to a non-English locale (https://github.com/ansible/ansible/issues/83608).
+- Use the requested error message in the ansible.module_utils.facts.timeout timeout function instead of hardcoding one.
+- ``ansible-galaxy install --help`` - Fix the usage text and document that the requirements file passed to ``-r`` can include collections and roles.
+- ``ansible-galaxy role install`` - update the default timeout to download archive URLs from 20 seconds to 60 (https://github.com/ansible/ansible/issues/83521).
+- ``end_host`` - fix incorrect return code when executing ``end_host`` in the ``rescue`` section (https://github.com/ansible/ansible/issues/83447)
+- ``package``/``dnf`` action plugins - provide the reason behind the failure to gather the ``ansible_pkg_mgr`` fact to identify the package backend
+- addressed issue of trailing text been ignored, non-ASCII characters are parsed, enhance white space handling and fixed overly permissive issue of human_to_bytes filter(https://github.com/ansible/ansible/issues/82075)
+- ansible-config will now properly template defaults before dumping them.
+- ansible-doc - fixed "inicates" typo in output
+- ansible-doc - format top-level descriptions with multiple paragraphs as multiple paragraphs, instead of concatenating them (https://github.com/ansible/ansible/pull/83155).
+- ansible-doc - handle no_fail condition for role.
+- ansible-doc - make colors configurable.
+- ansible-galaxy collection install - remove old installation info when installing collections (https://github.com/ansible/ansible/issues/83182).
+- ansible-galaxy role install - fix symlinks (https://github.com/ansible/ansible/issues/82702, https://github.com/ansible/ansible/issues/81965).
+- ansible-test - Enable the ``sys.unraisablehook`` work-around for the ``pylint`` sanity test on Python 3.11. Previously the work-around was only enabled for Python 3.12 and later. However, the same issue has been discovered on Python 3.11.
+- ansible-test - The ``pylint`` sanity test now includes the controller/target context of files when grouping them. This allows the ``--py-version`` option to be passed to ``pylint`` to indicate the minimum supported Python version for each test context, preventing ``pylint`` from defaulting to the Python version used to invoke the test.
+- ansible-test action-plugin-docs - Fix to check for sidecar documentation for action plugins
+- ansible_managed restored it's 'templatability' by ensuring the possible injection routes are cut off earlier in the process.
+- apt - report changed=True when some packages are being removed (https://github.com/ansible/ansible/issues/46314).
+- apt_* - add more info messages raised while updating apt cache (https://github.com/ansible/ansible/issues/77941).
+- assemble - update argument_spec with 'decrypt' option which is required by action plugin (https://github.com/ansible/ansible/issues/80840).
+- atomic_move - fix using the setgid bit on the parent directory when creating files (https://github.com/ansible/ansible/issues/46742, https://github.com/ansible/ansible/issues/67177).
+- config, restored the ability to set module compression via a variable
+- connection plugins using the 'extras' option feature would need variables to match the plugin's loaded name, sometimes requiring fqcn, which is not the same as the documented/declared/expected variables. Now we fall back to the 'basename' of the fqcn, but plugin authors can still set the expected value directly.
+- copy - mtime/atime not updated. Fix now update mtime/atime(https://github.com/ansible/ansible/issues/83013)
+- csvfile lookup - give an error when no search term is provided using modern config syntax (https://github.com/ansible/ansible/issues/83689).
+- debconf - fix normalization of value representation for boolean vtypes in new packages (https://github.com/ansible/ansible/issues/83594)
+- debconf - set empty password values (https://github.com/ansible/ansible/issues/83214).
+- delay keyword is now a float, matching the underlying 'time' API and user expectations.
+- display - warn user about empty log filepath (https://github.com/ansible/ansible/issues/79959).
+- display now does a better job of mapping warnings/errors to the proper log severity when using ansible.log. We still use color as a fallback mapping (now prioritiezed by severity) but mostly rely on it beind directly set by warnning/errors calls.
+- distro package - update the distro package version from 1.8.0 to 1.9.0  (https://github.com/ansible/ansible/issues/82935)
+- dnf - Ensure that we are handling DownloadError properly in the dnf module
+- dnf - Substitute variables in DNF cache path (https://github.com/ansible/ansible/pull/80094).
+- dnf - fix an issue where two packages of the same ``evr`` but different arch failed to install (https://github.com/ansible/ansible/issues/83406)
+- dnf - honor installroot for ``cachedir``, ``logdir`` and ``persistdir``
+- dnf - perform variable substitutions in ``logdir`` and ``persistdir``
+- dnf, dnf5 - fix for installing a set of packages by specifying them using a wildcard character (https://github.com/ansible/ansible/issues/83373)
+- dnf5 - fix traceback when ``enable_plugins``/``disable_plugins`` is used on ``python3-libdnf5`` versions that do not support this functionality
+- dnf5 - re-introduce the ``state: installed`` alias to ``state: present`` (https://github.com/ansible/ansible/issues/83960)
+- dnf5 - replace removed API calls
+- ensure we have logger before we log when we have increased verbosity.
+- facts - `support_discard` now returns `0` if either `discard_granularity` or `discard_max_hw_bytes` is zero; otherwise it returns the value of `discard_granularity`, as before (https://github.com/ansible/ansible/pull/83480).
+- facts - add a generic detection for VMware in product name.
+- facts - add facts about x86_64 flags to detect microarchitecture (https://github.com/ansible/ansible/issues/83331).
+- facts - skip if distribution file path is directory, instead of raising error (https://github.com/ansible/ansible/issues/84006).
+- fetch - add error message when using ``dest`` with a trailing slash that becomes a local directory - https://github.com/ansible/ansible/issues/82878
+- file - retrieve the link's full path when hard linking a soft link with follow (https://github.com/ansible/ansible/issues/33911).
+- fixed the issue of creating user directory using tilde(~) always reported "changed".(https://github.com/ansible/ansible/issues/82490)
+- fixed unit test test_borken_cowsay to address mock not been properly applied when existing unix system already have cowsay installed.
+- freebsd - refactor dmidecode fact gathering code for simplicity.
+- freebsd - update disk and slices regex for fact gathering (https://github.com/ansible/ansible/pull/82081).
+- get_url - Verify checksum using tmpsrc, not dest (https://github.com/ansible/ansible/pull/64092)
+- git - check if git version is available or not before using it for comparison (https://github.com/ansible/ansible/issues/72321).
+- include_tasks - Display location when attempting to load a task list where ``include_*`` did not specify any value - https://github.com/ansible/ansible/issues/83874
+- known_hosts - the returned module invocation now accurately reflects the module arguments.
+- linear strategy now provides a properly templated task name to the v2_runner_on_started callback event.
+- linear strategy: fix handlers included via ``include_tasks`` handler to be executed in lockstep (https://github.com/ansible/ansible/issues/83019)
+- linux - remove extraneous get_bin_path API call.
+- local - handle error while parsing values in ini files (https://github.com/ansible/ansible/issues/82717).
+- lookup - Fixed examples of csv lookup plugin (https://github.com/ansible/ansible/issues/83031).
+- module_defaults - do not display action/module deprecation warnings when using an action_group that contains a deprecated plugin (https://github.com/ansible/ansible/issues/83490).
+- module_utils atomic_move (used by most file based modules), now correctly handles permission copy and setting mtime correctly across all paths
+- package_facts - apk fix when cache is empty (https://github.com/ansible/ansible/issues/83126).
+- package_facts - no longer fails silently when the selected package manager is unable to list packages.
+- package_facts - returns the correct warning when package listing fails.
+- persistent connection plugins - The correct Ansible persistent connection helper is now always used. Previously, the wrong script could be used, depending on the value of the ``PATH`` environment variable. As a result, users were sometimes required to set ``ANSIBLE_CONNECTION_PATH`` to use the correct script.
+- powershell - Implement more robust deletion mechanism for C# code compilation temporary files. This should avoid scenarios where the underlying temporary directory may be temporarily locked by antivirus tools or other IO problems. A failure to delete one of these temporary directories will result in a warning rather than an outright failure.
+- powershell - Improve CLIXML decoding to decode all control characters and unicode characters that are encoded as surrogate pairs.
+- psrp - Fix bug when attempting to fetch a file path that contains special glob characters like ``[]``
+- replace - Updated before/after example (https://github.com/ansible/ansible/issues/83390).
+- runtime-metadata sanity test - do not crash on deprecations if ``galaxy.yml`` contains an empty ``version`` field (https://github.com/ansible/ansible/pull/83831).
+- service - fix order of CLI arguments on FreeBSD (https://github.com/ansible/ansible/pull/81377).
+- service_facts - don't crash if OpenBSD rcctl variable contains '=' character (https://github.com/ansible/ansible/issues/83457)
+- service_facts will now detect failed services more accurately across systemd implementations.
+- setup module (fact gathering), added fallbcak code path to handle mount fact gathering in linux when threading is not available
+- setup/gather_facts will skip missing ``sysctl`` instead of being a fatal error (https://github.com/ansible/ansible/pull/81297).
+- shell plugin - properly quote all needed components of shell commands (https://github.com/ansible/ansible/issues/82535)
+- ssh - Fix bug when attempting to fetch a file path with characters that should be quoted when using the ``piped`` transfer method
+- support the countme option when using yum_repository
+- systemd - extend systemctl is-enabled check to handle "enabled-runtime" (https://github.com/ansible/ansible/pull/77754).
+- systemd facts - handle AttributeError raised while gathering facts on non-systemd hosts.
+- systemd_service - handle mask operation failure (https://github.com/ansible/ansible/issues/81649).
+- templating hostvars under native jinja will not cause serialization errors anymore.
+- the raw arguments error now just displays the short names of modules instead of every possible variation
+- unarchive - Better handling of files with an invalid timestamp in zip file (https://github.com/ansible/ansible/issues/81092).
+- unarchive - trigger change when size and content differ when other properties are unchanged (https://github.com/ansible/ansible/pull/83454).
+- unsafe data - Address an incompatibility when iterating or getting a single index from ``AnsibleUnsafeBytes``
+- unsafe data - Address an incompatibility with ``AnsibleUnsafeText`` and ``AnsibleUnsafeBytes`` when pickling with ``protocol=0``
+- unsafe data - Enable directly using ``AnsibleUnsafeText`` with Python ``pathlib`` (https://github.com/ansible/ansible/issues/82414)
+- uri - deprecate 'yes' and 'no' value for 'follow_redirects' parameter.
+- user action will now require O(force) to overwrite the public part of an ssh key when generating ssh keys, as was already the case for the private part.
+- user module now avoids changing ownership of files symlinked in provided home dir skeleton
+- vault - handle vault password file value when it is directory (https://github.com/ansible/ansible/issues/42960).
+- vault.is_encrypted_file is now optimized to be called in runtime and not for being called in tests
+- vault_encrypted test documentation, name and examples have been fixed, other parts were clarified
+- winrm - Add retry after exceeding commands per user quota that can occur in loops and action plugins running multiple commands.
+
+Known Issues
+------------
+
+- ansible-test - When using ansible-test containers with Podman on a Ubuntu 24.04 host, ansible-test must be run as a non-root user to avoid permission issues caused by AppArmor.
+- ansible-test - When using the Fedora 40 container with Podman on a Ubuntu 24.04 host, the ``unix-chkpwd`` AppArmor profile must be disabled on the host to allow SSH connections to the container.
+
+New Plugins
+-----------
+
+Test
+~~~~
+
+- timedout - did the task time out
+- vaulted_file - Is this file an encrypted vault
+
+New Modules
+-----------
+
+Lib
+~~~
+
+ansible.modules
+^^^^^^^^^^^^^^^
+
+- mount_facts - Retrieve mount information.
--- a/changelogs/changelog.yaml
+++ b/changelogs/changelog.yaml
@ -1,2 +1,771 @@
 ancestor: 2.17.0
-releases: {}
+releases:
+  2.18.0:
+    changes:
+      release_summary: '| Release Date: 2024-11-04
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
+
+        '
+    codename: Fool in the Rain
+    fragments:
+    - 2.18.0_summary.yaml
+    release_date: '2024-11-04'
+  2.18.0b1:
+    changes:
+      breaking_changes:
+      - Stopped wrapping all commands sent over SSH on a Windows target with a ``powershell.exe``
+        executable. This results in one less process being started on each command
+        for Windows to improve efficiency, simplify the code, and make ``raw`` an
+        actual raw command run with the default shell configured on the Windows sshd
+        settings. This should have no affect on most tasks except for ``raw`` which
+        now is not guaranteed to always be running in a PowerShell shell and from
+        having the console output codepage set to UTF-8. To avoid this issue either
+        swap to using ``ansible.windows.win_command``, ``ansible.windows.win_shell``,
+        ``ansible.windows.win_powershell`` or manually wrap the raw command with the
+        shell commands needed to set the output console encoding.
+      - persistent connection plugins - The ``ANSIBLE_CONNECTION_PATH`` config option
+        no longer has any effect.
+      bugfixes:
+      - -> runas become - Generate new token for the SYSTEM token to use for become.
+        This should result in the full SYSTEM token being used and problems starting
+        the process that fails with ``The process creation has been blocked``.
+      - Add a version ceiling constraint for pypsrp to avoid potential breaking changes
+        in the 1.0.0 release.
+      - Add descriptions for ``ansible-galaxy install --help` and ``ansible-galaxy
+        role|collection install --help``.
+      - Avoid truncating floats when casting into int, as it can lead to truncation
+        and unexpected results. 0.99999 will be 0, not 1.
+      - COLOR_SKIP will not alter "included" events color display anymore.
+      - Callbacks now correctly get the resolved connection plugin name as the connection
+        used.
+      - Darwin - add unit tests for Darwin hardware fact gathering.
+      - Fix ``SemanticVersion.parse()`` to store the version string so that ``__repr__``
+        reports it instead of ``None`` (https://github.com/ansible/ansible/pull/83831).
+      - Fix a traceback when an environment variable contains certain special characters
+        (https://github.com/ansible/ansible/issues/83498)
+      - 'Fix an issue when setting a plugin name from an unsafe source resulted in
+        ``ValueError: unmarshallable object`` (https://github.com/ansible/ansible/issues/82708)'
+      - Fix an issue where registered variable was not available for templating in
+        ``loop_control.label`` on skipped looped tasks (https://github.com/ansible/ansible/issues/83619)
+      - Fix for ``meta`` tasks breaking host/fork affinity with ``host_pinned`` strategy
+        (https://github.com/ansible/ansible/issues/83294)
+      - Fix handlers not being executed in lockstep using the linear strategy in some
+        cases (https://github.com/ansible/ansible/issues/82307)
+      - Fix rapid memory usage growth when notifying handlers using the ``listen``
+        keyword (https://github.com/ansible/ansible/issues/83392)
+      - Fix the task attribute ``resolved_action`` to show the FQCN instead of ``None``
+        when ``action`` or ``local_action`` is used in the playbook.
+      - Fix using ``module_defaults`` with ``local_action``/``action`` (https://github.com/ansible/ansible/issues/81905).
+      - Fix using the current task's directory for looking up relative paths within
+        roles (https://github.com/ansible/ansible/issues/82695).
+      - 'Remove deprecated config options DEFAULT_FACT_PATH, DEFAULT_GATHER_SUBSET,
+        and DEFAULT_GATHER_TIMEOUT in favor of setting ``fact_path``, ``gather_subset``
+        and ``gather_timeout`` as ``module_defaults`` for ``ansible.builtin.setup``.
+
+        These will apply to both the ``gather_facts`` play keyword, and any ``ansible.builtin.setup``
+        tasks.
+
+        To configure these options only for the ``gather_facts`` keyword, set these
+        options as play keywords also.'
+      - Set LANGUAGE environment variable is set to a non-English locale (https://github.com/ansible/ansible/issues/83608).
+      - '``ansible-galaxy install --help`` - Fix the usage text and document that
+        the requirements file passed to ``-r`` can include collections and roles.'
+      - '``ansible-galaxy role install`` - update the default timeout to download
+        archive URLs from 20 seconds to 60 (https://github.com/ansible/ansible/issues/83521).'
+      - '``end_host`` - fix incorrect return code when executing ``end_host`` in the
+        ``rescue`` section (https://github.com/ansible/ansible/issues/83447)'
+      - addressed issue of trailing text been ignored, non-ASCII characters are parsed,
+        enhance white space handling and fixed overly permissive issue of human_to_bytes
+        filter(https://github.com/ansible/ansible/issues/82075)
+      - ansible-config will now properly template defaults before dumping them.
+      - ansible-doc - fixed "inicates" typo in output
+      - ansible-doc - format top-level descriptions with multiple paragraphs as multiple
+        paragraphs, instead of concatenating them (https://github.com/ansible/ansible/pull/83155).
+      - ansible-doc - handle no_fail condition for role.
+      - ansible-doc - make colors configurable.
+      - ansible-galaxy collection install - remove old installation info when installing
+        collections (https://github.com/ansible/ansible/issues/83182).
+      - ansible-galaxy role install - fix symlinks (https://github.com/ansible/ansible/issues/82702,
+        https://github.com/ansible/ansible/issues/81965).
+      - ansible-test - The ``pylint`` sanity test now includes the controller/target
+        context of files when grouping them. This allows the ``--py-version`` option
+        to be passed to ``pylint`` to indicate the minimum supported Python version
+        for each test context, preventing ``pylint`` from defaulting to the Python
+        version used to invoke the test.
+      - ansible-test action-plugin-docs - Fix to check for sidecar documentation for
+        action plugins
+      - ansible_managed restored it's 'templatability' by ensuring the possible injection
+        routes are cut off earlier in the process.
+      - apt - report changed=True when some packages are being removed (https://github.com/ansible/ansible/issues/46314).
+      - apt_* - add more info messages raised while updating apt cache (https://github.com/ansible/ansible/issues/77941).
+      - assemble - update argument_spec with 'decrypt' option which is required by
+        action plugin (https://github.com/ansible/ansible/issues/80840).
+      - atomic_move - fix using the setgid bit on the parent directory when creating
+        files (https://github.com/ansible/ansible/issues/46742, https://github.com/ansible/ansible/issues/67177).
+      - config, restored the ability to set module compression via a variable
+      - connection plugins using the 'extras' option feature would need variables
+        to match the plugin's loaded name, sometimes requiring fqcn, which is not
+        the same as the documented/declared/expected variables. Now we fall back to
+        the 'basename' of the fqcn, but plugin authors can still set the expected
+        value directly.
+      - copy - mtime/atime not updated. Fix now update mtime/atime(https://github.com/ansible/ansible/issues/83013)
+      - csvfile lookup - give an error when no search term is provided using modern
+        config syntax (https://github.com/ansible/ansible/issues/83689).
+      - debconf - fix normalization of value representation for boolean vtypes in
+        new packages (https://github.com/ansible/ansible/issues/83594)
+      - delay keyword is now a float, matching the underlying 'time' API and user
+        expectations.
+      - display - warn user about empty log filepath (https://github.com/ansible/ansible/issues/79959).
+      - display now does a better job of mapping warnings/errors to the proper log
+        severity when using ansible.log. We still use color as a fallback mapping
+        (now prioritiezed by severity) but mostly rely on it beind directly set by
+        warnning/errors calls.
+      - distro package - update the distro package version from 1.8.0 to 1.9.0  (https://github.com/ansible/ansible/issues/82935)
+      - dnf - Ensure that we are handling DownloadError properly in the dnf module
+      - dnf - Substitute variables in DNF cache path (https://github.com/ansible/ansible/pull/80094).
+      - dnf - fix an issue where two packages of the same ``evr`` but different arch
+        failed to install (https://github.com/ansible/ansible/issues/83406)
+      - dnf - honor installroot for ``cachedir``, ``logdir`` and ``persistdir``
+      - dnf - perform variable substitutions in ``logdir`` and ``persistdir``
+      - dnf, dnf5 - fix for installing a set of packages by specifying them using
+        a wildcard character (https://github.com/ansible/ansible/issues/83373)
+      - 'dnf5 - re-introduce the ``state: installed`` alias to ``state: present``
+        (https://github.com/ansible/ansible/issues/83960)'
+      - dnf5 - replace removed API calls
+      - ensure we have logger before we log when we have increased verbosity.
+      - facts - `support_discard` now returns `0` if either `discard_granularity`
+        or `discard_max_hw_bytes` is zero; otherwise it returns the value of `discard_granularity`,
+        as before (https://github.com/ansible/ansible/pull/83480).
+      - facts - add a generic detection for VMware in product name.
+      - facts - add facts about x86_64 flags to detect microarchitecture (https://github.com/ansible/ansible/issues/83331).
+      - fetch - add error message when using ``dest`` with a trailing slash that becomes
+        a local directory - https://github.com/ansible/ansible/issues/82878
+      - file - retrieve the link's full path when hard linking a soft link with follow
+        (https://github.com/ansible/ansible/issues/33911).
+      - fixed the issue of creating user directory using tilde(~) always reported
+        "changed".(https://github.com/ansible/ansible/issues/82490)
+      - fixed unit test test_borken_cowsay to address mock not been properly applied
+        when existing unix system already have cowsay installed.
+      - freebsd - refactor dmidecode fact gathering code for simplicity.
+      - freebsd - update disk and slices regex for fact gathering (https://github.com/ansible/ansible/pull/82081).
+      - get_url - Verify checksum using tmpsrc, not dest (https://github.com/ansible/ansible/pull/64092)
+      - git - check if git version is available or not before using it for comparison
+        (https://github.com/ansible/ansible/issues/72321).
+      - include_tasks - Display location when attempting to load a task list where
+        ``include_*`` did not specify any value - https://github.com/ansible/ansible/issues/83874
+      - known_hosts - the returned module invocation now accurately reflects the module
+        arguments.
+      - linear strategy now provides a properly templated task name to the v2_runner_on_started
+        callback event.
+      - 'linear strategy: fix handlers included via ``include_tasks`` handler to be
+        executed in lockstep (https://github.com/ansible/ansible/issues/83019)'
+      - linux - remove extraneous get_bin_path API call.
+      - local - handle error while parsing values in ini files (https://github.com/ansible/ansible/issues/82717).
+      - lookup - Fixed examples of csv lookup plugin (https://github.com/ansible/ansible/issues/83031).
+      - module_defaults - do not display action/module deprecation warnings when using
+        an action_group that contains a deprecated plugin (https://github.com/ansible/ansible/issues/83490).
+      - module_utils atomic_move (used by most file based modules), now correctly
+        handles permission copy and setting mtime correctly across all paths
+      - package_facts - apk fix when cache is empty (https://github.com/ansible/ansible/issues/83126).
+      - package_facts - no longer fails silently when the selected package manager
+        is unable to list packages.
+      - package_facts - returns the correct warning when package listing fails.
+      - persistent connection plugins - The correct Ansible persistent connection
+        helper is now always used. Previously, the wrong script could be used, depending
+        on the value of the ``PATH`` environment variable. As a result, users were
+        sometimes required to set ``ANSIBLE_CONNECTION_PATH`` to use the correct script.
+      - powershell - Implement more robust deletion mechanism for C# code compilation
+        temporary files. This should avoid scenarios where the underlying temporary
+        directory may be temporarily locked by antivirus tools or other IO problems.
+        A failure to delete one of these temporary directories will result in a warning
+        rather than an outright failure.
+      - powershell - Improve CLIXML decoding to decode all control characters and
+        unicode characters that are encoded as surrogate pairs.
+      - psrp - Fix bug when attempting to fetch a file path that contains special
+        glob characters like ``[]``
+      - replace - Updated before/after example (https://github.com/ansible/ansible/issues/83390).
+      - runtime-metadata sanity test - do not crash on deprecations if ``galaxy.yml``
+        contains an empty ``version`` field (https://github.com/ansible/ansible/pull/83831).
+      - service - fix order of CLI arguments on FreeBSD (https://github.com/ansible/ansible/pull/81377).
+      - service_facts - don't crash if OpenBSD rcctl variable contains '=' character
+        (https://github.com/ansible/ansible/issues/83457)
+      - service_facts will now detect failed services more accurately across systemd
+        implementations.
+      - setup module (fact gathering), added fallbcak code path to handle mount fact
+        gathering in linux when threading is not available
+      - setup/gather_facts will skip missing ``sysctl`` instead of being a fatal error
+        (https://github.com/ansible/ansible/pull/81297).
+      - shell plugin - properly quote all needed components of shell commands (https://github.com/ansible/ansible/issues/82535)
+      - ssh - Fix bug when attempting to fetch a file path with characters that should
+        be quoted when using the ``piped`` transfer method
+      - support the countme option when using yum_repository
+      - systemd - extend systemctl is-enabled check to handle "enabled-runtime" (https://github.com/ansible/ansible/pull/77754).
+      - systemd facts - handle AttributeError raised while gathering facts on non-systemd
+        hosts.
+      - systemd_service - handle mask operation failure (https://github.com/ansible/ansible/issues/81649).
+      - templating hostvars under native jinja will not cause serialization errors
+        anymore.
+      - the raw arguments error now just displays the short names of modules instead
+        of every possible variation
+      - unarchive - Better handling of files with an invalid timestamp in zip file
+        (https://github.com/ansible/ansible/issues/81092).
+      - unarchive - trigger change when size and content differ when other properties
+        are unchanged (https://github.com/ansible/ansible/pull/83454).
+      - unsafe data - Address an incompatibility when iterating or getting a single
+        index from ``AnsibleUnsafeBytes``
+      - unsafe data - Address an incompatibility with ``AnsibleUnsafeText`` and ``AnsibleUnsafeBytes``
+        when pickling with ``protocol=0``
+      - unsafe data - Enable directly using ``AnsibleUnsafeText`` with Python ``pathlib``
+        (https://github.com/ansible/ansible/issues/82414)
+      - uri - deprecate 'yes' and 'no' value for 'follow_redirects' parameter.
+      - vault - handle vault password file value when it is directory (https://github.com/ansible/ansible/issues/42960).
+      - vault.is_encrypted_file is now optimized to be called in runtime and not for
+        being called in tests
+      - vault_encrypted test documentation, name and examples have been fixed, other
+        parts were clarified
+      - winrm - Add retry after exceeding commands per user quota that can occur in
+        loops and action plugins running multiple commands.
+      deprecated_features:
+      - Deprecate ``ansible.module_utils.basic.AnsibleModule.safe_eval`` and ``ansible.module_utils.common.safe_eval``
+        as they are no longer used.
+      - persistent connection plugins - The ``ANSIBLE_CONNECTION_PATH`` config option
+        no longer has any effect, and will be removed in a future release.
+      - yum_repository - deprecate ``async`` option as it has been removed in RHEL
+        8 and will be removed in ansible-core 2.22.
+      - 'yum_repository - the following options are deprecated: ``deltarpm_metadata_percentage``,
+        ``gpgcakey``, ``http_caching``, ``keepalive``, ``metadata_expire_filter``,
+        ``mirrorlist_expire``, ``protect``, ``ssl_check_cert_permissions``, ``ui_repoid_vars``
+        as they have no effect for dnf as an underlying package manager. The options
+        will be removed in ansible-core 2.22.'
+      known_issues:
+      - ansible-test - When using ansible-test containers with Podman on a Ubuntu
+        24.04 host, ansible-test must be run as a non-root user to avoid permission
+        issues caused by AppArmor.
+      - ansible-test - When using the Fedora 40 container with Podman on a Ubuntu
+        24.04 host, the ``unix-chkpwd`` AppArmor profile must be disabled on the host
+        to allow SSH connections to the container.
+      minor_changes:
+      - Add ``gid_min``, ``gid_max`` to the group plugin to overwrite the defaults
+        provided by the ``/etc/login.defs`` file (https://github.com/ansible/ansible/pull/81770).
+      - Add ``python3.13`` to the default ``INTERPRETER_PYTHON_FALLBACK`` list.
+      - Add ``uid_min``, ``uid_max`` to the user plugin to overwrite the defaults
+        provided by the ``/etc/login.defs`` file (https://github.com/ansible/ansible/pull/81770).
+      - Add a new meta task ``end_role`` (https://github.com/ansible/ansible/issues/22286)
+      - Add a new mount_facts module to support gathering information about mounts
+        that are excluded by default fact gathering.
+      - Introducing COLOR_INCLUDED parameter. This can set a specific color for "included"
+        events.
+      - Removed the shell ``environment`` config entry as this is already covered
+        by the play/task directives documentation and the value itself is not used
+        in the shell plugins. This should remove any confusion around how people set
+        the environment for a task.
+      - Suppress cryptography deprecation warnings for Blowfish and TripleDES when
+        the ``paramiko`` Python module is installed.
+      - The minimum supported Python version on targets is now Python 3.8.
+      - '``ansible-galaxy collection publish`` - add configuration options for the
+        initial poll interval and the exponential when checking the import status
+        of a collection, since the default is relatively slow.'
+      - ansible-config has new 'validate' option to find mispelled/forgein configurations
+        in ini file or environment variables.
+      - ansible-doc - show examples in role entrypoint argument specs (https://github.com/ansible/ansible/pull/82671).
+      - ansible-galaxy - Handle authentication errors and token expiration
+      - ansible-test - Add Ubuntu 24.04 remote.
+      - ansible-test - Add support for Python 3.13.
+      - ansible-test - An ``ansible_core.egg-info`` directory is no longer generated
+        when running tests.
+      - ansible-test - Connection options can be set for ansible-test managed remote
+        Windows instances.
+      - ansible-test - Improve the error message shown when an unknown ``--remote``
+        or ``--docker`` option is given.
+      - ansible-test - Remove Python 2.7 compatibility imports.
+      - ansible-test - Removed the ``vyos/1.1.8`` network remote as it is no longer
+        functional.
+      - ansible-test - Replace Alpine 3.19 container and remote with Alpine 3.20.
+      - ansible-test - Replace Fedora 39 container and remote with Fedora 40.
+      - ansible-test - Replace FreeBSD 14.0 remote with FreeBSD 14.1.
+      - ansible-test - Replace RHEL 9.3 remote with RHEL 9.4.
+      - ansible-test - Replace Ubuntu 20.04 container with Ubuntu 24.04 container.
+      - ansible-test - The ``empty-init`` sanity test no longer applies to ``module_utils``
+        packages.
+      - ansible-test - Update ``ansible-test-utility-container`` to version 3.1.0.
+      - ansible-test - Update ``base`` and ``default`` containers to omit Python 3.7.
+      - ansible-test - Update ``coverage`` to version 7.6.1.
+      - ansible-test - Update ``http-test-container`` to version 3.0.0.
+      - ansible-test - Update ``nios-test-container`` to version 5.0.0.
+      - ansible-test - Update ``pypi-test-container`` to version 3.2.0.
+      - ansible-test - Updated the frozen requirements for all sanity tests.
+      - ansible-test - Upgrade ``pip`` used in ansible-test managed virtual environments
+        from version 24.0 to 24.2.
+      - ansible-test - Virtual environments created by ansible-test no longer include
+        the ``wheel`` or ``setuptools`` packages.
+      - ansible-test - update HTTP test container to 3.2.0 (https://github.com/ansible/ansible/pull/83469).
+      - ansible.log now also shows log severity field
+      - distribution.py - Added SL-Micro in Suse OS Family. (https://github.com/ansible/ansible/pull/83541)
+      - dnf - minor internal changes in how the errors from the dnf API are handled;
+        rely solely on the exceptions rather than inspecting text embedded in them
+      - dnf - remove legacy code for unsupported dnf versions
+      - dnf5 - implement ``enable_plugin`` and ``disable_plugin`` options
+      - fact gathering - Gather /proc/sysinfo facts on s390 Linux on Z
+      - facts - add systemd version and features
+      - find - change the datatype of ``elements`` to ``path`` in option ``paths``
+        (https://github.com/ansible/ansible/pull/83575).
+      - ini lookup - add new ``interpolation`` option (https://github.com/ansible/ansible/issues/83755)
+      - isidentifier - remove unwanted Python 2 specific code.
+      - loop_control - add a break_when option to to break out of a task loop early
+        based on Jinja2 expressions (https://github.com/ansible/ansible/issues/83442).
+      - package_facts module now supports using aliases for supported package managers,
+        for example managers=yum or managers=dnf will resolve to using the underlying
+        rpm.
+      - plugins, deprecations and warnings concerning configuration are now displayed
+        to the user, technical issue that prevented 'de-duplication' have been resolved.
+      - psrp - Remove connection plugin extras vars lookup. This should have no affect
+        on existing users as all options have been documented.
+      - remove extraneous selinux import (https://github.com/ansible/ansible/issues/83657).
+      - replace random with secrets library.
+      - rpm_key - allow validation of gpg key with a subkey fingerprint
+      - rpm_key - enable gpg validation that requires presence of multiple fingerprints
+      - service_mgr - add support for dinit service manager (https://github.com/ansible/ansible/pull/83489).
+      - task timeout now returns timedout key with frame/code that was in execution
+        when the timeout is triggered.
+      - timedout test for checking if a task result represents a 'timed out' task.
+      - unarchive - Remove Python 2.7 compatibility imports.
+      - validate-modules sanity test - detect if names of an option (option name +
+        aliases) do not match between argument spec and documentation (https://github.com/ansible/ansible/issues/83598,
+        https://github.com/ansible/ansible/pull/83599).
+      - validate-modules sanity test - reject option/aliases names that are identical
+        up to casing but belong to different options (https://github.com/ansible/ansible/pull/83530).
+      - vaulted_file test filter added, to test if the provided path is an 'Ansible
+        vaulted' file
+      - yum_repository - add ``excludepkgs`` alias to the ``exclude`` option.
+      release_summary: '| Release Date: 2024-09-24
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
+
+        '
+      removed_features:
+      - Play - removed deprecated ``ROLE_CACHE`` property in favor of ``role_cache``.
+      - Remove deprecated `VariableManager._get_delegated_vars` method (https://github.com/ansible/ansible/issues/82950)
+      - Removed Python 3.10 as a supported version on the controller. Python 3.11
+        or newer is required.
+      - Removed support for setting the ``vars`` keyword to lists of dictionaries.
+        It is now required to be a single dictionary.
+      - loader - remove deprecated non-inclusive words (https://github.com/ansible/ansible/issues/82947).
+      - paramiko_ssh - removed deprecated ssh_args from the paramiko_ssh connection
+        plugin (https://github.com/ansible/ansible/issues/82939).
+      - paramiko_ssh - removed deprecated ssh_common_args from the paramiko_ssh connection
+        plugin (https://github.com/ansible/ansible/issues/82940).
+      - paramiko_ssh - removed deprecated ssh_extra_args from the paramiko_ssh connection
+        plugin (https://github.com/ansible/ansible/issues/82941).
+      - play_context - remove deprecated PlayContext.verbosity property (https://github.com/ansible/ansible/issues/82945).
+      - utils/listify - remove deprecated 'loader' argument from listify_lookup_plugin_terms
+        API (https://github.com/ansible/ansible/issues/82949).
+    codename: Fool in the Rain
+    fragments:
+    - 2.18.0b1_summary.yaml
+    - 42960_vault_password.yml
+    - 46314.yml
+    - 46742-atomic_move-fix-setgid.yml
+    - 62151-loop_control-until.yml
+    - 64092-get_url_verify_tmpsrc_checksum.yml
+    - 72321_git.yml
+    - 81770-add-uid-guid-minmax-keys.yml
+    - 82075.yml
+    - 82307-handlers-lockstep-linear-fix.yml
+    - 82490_creating_user_dir_using_tilde_always_reports_changed.yml
+    - 82535-properly-quote-shell.yml
+    - 82671-ansible-doc-role-examples.yml
+    - 82708-unsafe-plugin-name-error.yml
+    - 82831_countme_yum_repository.yml
+    - 82878-fetch-dest-is-dir.yml
+    - 82941.yml
+    - 82946.yml
+    - 82947.yml
+    - 83019-linear-handlers-lockstep-fix.yml
+    - 83031.yml
+    - 83155-ansible-doc-paragraphs.yml
+    - 83235-copy-module-update-mtime.yml
+    - 83294-meta-host_pinned-affinity.yml
+    - 83327.yml
+    - 83331.yml
+    - 83373-dnf5-wildcard.yml
+    - 83392-fix-memory-issues-handlers.yml
+    - 83406-dnf-fix-arch-cmp.yml
+    - 83447-end_host-rescue-rc.yml
+    - 83457-service_facts-openbsd-dont-crash-in-equals.yml
+    - 83469-http-test-container.yml
+    - 83480-fix-support-discard.yml
+    - 83498-command-tb-env.yml
+    - 83508_mount_facts.yml
+    - 83530-validate-modules-casing.yml
+    - 83540-update_disto_version.yml
+    - 83541-add-sl-micro-suse-family.yaml
+    - 83575-fix-sanity-ignore-for-find.yml
+    - 83599-validate-modules-aliases.yml
+    - 83601-debconf-normalize-bools.yml
+    - 83619-loop-label-register.yml
+    - 83716-enable-subkey-fingerprint-validation-in-rpm-key-module.yml
+    - 83755-ini-new-interpolation-option.yml
+    - 83803-collection-import-poll-interval.yml
+    - 83831-runtime-metadata-fix.yml
+    - 83874-include-parse-error-location.yml
+    - 83960-dnf5-state-installed-fix.yml
+    - PowerShell-AddType-temp.yml
+    - action-plugin-docs-sidecar.yml
+    - add_systemd_facts.yml
+    - ansible-config-validate.yml
+    - ansible-doc-color.yml
+    - ansible-doc-inicate.yml
+    - ansible-doc.yml
+    - ansible-drop-python-3.7.yml
+    - ansible-galaxy-install-archive-url-timeout.yml
+    - ansible-galaxy-install-help.yml
+    - ansible-galaxy-role-install-symlink.yml
+    - ansible-test-container-update.yml
+    - ansible-test-coverage-update.yml
+    - ansible-test-error-message-improvement.yml
+    - ansible-test-http-test-container-update.yml
+    - ansible-test-nios-container.yml
+    - ansible-test-no-egg-info.yml
+    - ansible-test-platform-updates.yml
+    - ansible-test-pylint-py-version.yml
+    - ansible-test-pypi-test-container-update.yml
+    - ansible-test-sanity-empty-init.yml
+    - ansible-test-sanity-test-requirements.yml
+    - ansible-test-utility-container-update.yml
+    - ansible-test-venv-bootstrap.yml
+    - ansible-test-vyos.yml
+    - ansible-test-windows-remote.yml
+    - ansible_connection_path.yml
+    - ansible_managed_restore.yml
+    - apk_package_facts.yml
+    - apt_cache.yml
+    - assemble.yml
+    - atomic_update_perms_time.yml
+    - become-runas-system.yml
+    - cleanup-outdated-galaxy-install-info.yml
+    - colors_for_included_events.yml
+    - config_init_fix.yml
+    - correct-callback-fqcn-old-style-action-invocation.yml
+    - correct_connection_callback.yml
+    - darwin_facts.yml
+    - delay_type.yml
+    - deprecate-safe-evals.yml
+    - dinit.yml
+    - display_fix_log_severity.yml
+    - dnf-exceptions-vs-text.yml
+    - dnf-installroot-substitutions.yml
+    - dnf-remove-legacy-code.yml
+    - dnf5-api-breaks.yml
+    - dnf5-enable-disable-plugins.yml
+    - dnf_cache_path.yml
+    - dnf_handle_downloaderror.yml
+    - dwim_is_role_fix_task_relative.yml
+    - empty_log_path.yml
+    - enabled_runtime.yml
+    - end_role.yml
+    - extras_fix.yml
+    - fetch-filename.yml
+    - file_hardlink.yml
+    - fix-inconsistent-csvfile-missing-search-error.yml
+    - fix-module-defaults-deprecations.yml
+    - fix_floating_ints.yml
+    - fix_log_verbosity.yml
+    - freebsd_disk_regex.yml
+    - freebsd_facts_refactor.yml
+    - freebsd_service.yml
+    - galaxy-reauth-error-handling.yml
+    - gather-s390-sysinfo.yml
+    - gather_facts_single.yml
+    - getoffmylawn.yml
+    - hostvars_fix.yml
+    - isidentifier.yml
+    - known_hosts_module_args.yml
+    - language.yml
+    - linear_started_name.yml
+    - linux_network_get.yml
+    - listify.yml
+    - local_facts_d.yml
+    - mask_me.yml
+    - mc_fix.yml
+    - package_facts_aliases.yml
+    - package_facts_warnings.yml
+    - powershell-clixml.yml
+    - psrp-extras.yml
+    - psrp-version-req.yml
+    - python-3.13.yml
+    - raw_clean_msg.yml
+    - remove-deprecated-gather-facts-config.yml
+    - remove-deprecated-get_delegated_vars.yml
+    - remove-deprecated-role-cache.yml
+    - remove-deprecated-vars-syntax.yml
+    - remove-python-2-compat.yml
+    - remove-python3.10-controller-support.yml
+    - replace_regex.yml
+    - secrets.yml
+    - selinux_import.yml
+    - service_facts_systemd_fix.yml
+    - shell-environment.yml
+    - ssh-windows.yml
+    - suppress-paramiko-warnings.yml
+    - sysctl_fact_fix.yml
+    - systemd_facts.yml
+    - timedout_test.yml
+    - timeout_show_frame.yml
+    - unarchive_differs.yml
+    - unarchive_timestamp.yml
+    - unsafe-fixes-2.yml
+    - unsafe-intern.yml
+    - uri_follow_redirect.yml
+    - v2.18.0-initial-commit.yaml
+    - vaulted_file_and_fixes.yml
+    - vmware_facts.yml
+    - winrm-quota.yml
+    - yum_repository.yml
+    modules:
+    - description: Retrieve mount information.
+      name: mount_facts
+      namespace: lib.ansible.modules
+    plugins:
+      test:
+      - description: did the task time out
+        name: timedout
+        namespace: null
+      - description: Is this file an encrypted vault
+        name: vaulted_file
+        namespace: null
+    release_date: '2024-09-24'
+  2.18.0rc1:
+    changes:
+      bugfixes:
+      - Errors now preserve stacked error messages even when YAML is involved.
+      - Fix disabling SSL verification when installing collections and roles from
+        git repositories. If ``--ignore-certs`` isn't provided, the value for the
+        ``GALAXY_IGNORE_CERTS`` configuration option will be used (https://github.com/ansible/ansible/issues/83326).
+      - Improve performance on large inventories by reducing the number of implicit
+        meta tasks.
+      - Use the requested error message in the ansible.module_utils.facts.timeout
+        timeout function instead of hardcoding one.
+      - '``package``/``dnf`` action plugins - provide the reason behind the failure
+        to gather the ``ansible_pkg_mgr`` fact to identify the package backend'
+      - ansible-test - Enable the ``sys.unraisablehook`` work-around for the ``pylint``
+        sanity test on Python 3.11. Previously the work-around was only enabled for
+        Python 3.12 and later. However, the same issue has been discovered on Python
+        3.11.
+      - debconf - set empty password values (https://github.com/ansible/ansible/issues/83214).
+      - dnf5 - fix traceback when ``enable_plugins``/``disable_plugins`` is used on
+        ``python3-libdnf5`` versions that do not support this functionality
+      - facts - skip if distribution file path is directory, instead of raising error
+        (https://github.com/ansible/ansible/issues/84006).
+      - user module now avoids changing ownership of files symlinked in provided home
+        dir skeleton
+      minor_changes:
+      - ansible-test - Default to Python 3.13 in the ``base`` and ``default`` containers.
+      - ansible-test - Disable the ``deprecated-`` prefixed ``pylint`` rules as their
+        results vary by Python version.
+      - ansible-test - Improve container runtime probe error handling. When unexpected
+        probe output is encountered, an error with more useful debugging information
+        is provided.
+      - ansible-test - Update ``pylint`` sanity test to use version 3.3.1.
+      - ansible-test - Update the ``base`` and ``default`` containers.
+      release_summary: '| Release Date: 2024-10-14
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
+
+        '
+    codename: Fool in the Rain
+    fragments:
+    - 2.18.0rc1_summary.yaml
+    - ansible-test-probe-error-handling.yml
+    - ansible-test-pylint-fix.yml
+    - ansible-test-update.yml
+    - debconf_empty_password.yml
+    - dnf5-plugins-compat.yml
+    - fix-ansible-galaxy-ignore-certs.yml
+    - fix-module-utils-facts-timeout.yml
+    - fix_errors.yml
+    - os_family.yml
+    - package-dnf-action-plugins-facts-fail-msg.yml
+    - skip-implicit-flush_handlers-no-notify.yml
+    - user_action_fix.yml
+    release_date: '2024-10-14'
+  2.18.0rc2:
+    changes:
+      bugfixes:
+      - user action will now require O(force) to overwrite the public part of an ssh
+        key when generating ssh keys, as was already the case for the private part.
+      release_summary: '| Release Date: 2024-10-29
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
+
+        '
+      security_fixes:
+      - include_vars action - Ensure that result masking is correctly requested when
+        vault-encrypted files are read. (CVE-2024-8775)
+      - task result processing - Ensure that action-sourced result masking (``_ansible_no_log=True``)
+        is preserved. (CVE-2024-8775)
+      - user action won't allow ssh-keygen, chown and chmod to run on existing ssh
+        public key file, avoiding traversal on existing symlinks (CVE-2024-9902).
+    codename: Fool in the Rain
+    fragments:
+    - 2.18.0rc2_summary.yaml
+    - cve-2024-8775.yml
+    - user_ssh_fix.yml
+    release_date: '2024-10-29'
+  2.18.1:
+    changes:
+      release_summary: '| Release Date: 2024-12-02
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
+
+        '
+    codename: Fool in the Rain
+    fragments:
+    - 2.18.1_summary.yaml
+    release_date: '2024-12-02'
+  2.18.1rc1:
+    changes:
+      bugfixes:
+      - Fix returning 'unreachable' for the overall task result. This prevents false
+        positives when a looped task has unignored unreachable items (https://github.com/ansible/ansible/issues/84019).
+      - ansible-test - Fix traceback that occurs after an interactive command fails.
+      - dnf5 - fix installing a package using ``state=latest`` when a binary of the
+        same name as the package is already installed (https://github.com/ansible/ansible/issues/84259)
+      - dnf5 - matching on a binary can be achieved only by specifying a full path
+        (https://github.com/ansible/ansible/issues/84334)
+      - runas become - Fix up become logic to still get the SYSTEM token with the
+        most privileges when running as SYSTEM.
+      minor_changes:
+      - ansible-test - When detection of the current container network fails, a warning
+        is now issued and execution continues. This simplifies usage in cases where
+        the current container cannot be inspected, such as when running in GitHub
+        Codespaces.
+      release_summary: '| Release Date: 2024-11-25
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
+
+        '
+      security_fixes:
+      - Templating will not prefer AnsibleUnsafe when a variable is referenced via
+        hostvars - CVE-2024-11079
+    codename: Fool in the Rain
+    fragments:
+    - 2.18.1rc1_summary.yaml
+    - 84019-ignore_unreachable-loop.yml
+    - 84259-dnf5-latest-fix.yml
+    - 84334-dnf5-consolidate-settings.yml
+    - ansible-test-fix-command-traceback.yml
+    - ansible-test-network-detection.yml
+    - become-runas-system-deux.yml
+    - unsafe_hostvars_fix.yml
+    release_date: '2024-11-25'
+  2.18.2:
+    changes:
+      release_summary: '| Release Date: 2025-01-27
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
+
+        '
+    codename: Fool in the Rain
+    fragments:
+    - 2.18.2_summary.yaml
+    release_date: '2025-01-27'
+  2.18.2rc1:
+    changes:
+      bugfixes:
+      - Ansible will now also warn when reserved keywords are set via a module (set_fact,
+        include_vars, etc).
+      - Ansible.Basic - Fix ``required_if`` check when the option value to check is
+        unset or set to null.
+      - Use consistent multiprocessing context for action write locks
+      - ansible-test - Fix up coverage reporting to properly translate the temporary
+        path of integration test modules to the expected static test module path.
+      - ansible-vault will now correctly handle `--prompt`, previously it would issue
+        an error about stdin if no 2nd argument was passed
+      - copy action now prevents user from setting internal options.
+      - gather_facts action now defaults to `ansible.legacy.setup` if `smart` was
+        set, no network OS was found and no other alias for `setup` was present.
+      - gather_facts action will now issues errors and warnings as appropriate if
+        a network OS is detected but no facts modules are defined for it.
+      - ssh - Improve the logic for parsing CLIXML data in stderr when working with
+        Windows host. This fixes issues when the raw stderr contains invalid UTF-8
+        byte sequences and improves embedded CLIXML sequences.
+      - ssh - connection options were incorrectly templated during ``reset_connection``
+        tasks (https://github.com/ansible/ansible/pull/84238).
+      release_summary: '| Release Date: 2025-01-20
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
+
+        '
+    codename: Fool in the Rain
+    fragments:
+    - 2.18.2rc1_summary.yaml
+    - 84238-fix-reset_connection-ssh_executable-templated.yml
+    - Ansible.Basic-required_if-null.yml
+    - ansible-test-coverage-test-files.yml
+    - copy_validate_input.yml
+    - gather_facts_netos_fixes.yml
+    - macos-correct-lock.yml
+    - reserved_module_chekc.yml
+    - ssh-clixml.yml
+    - vault_cli_fix.yml
+    release_date: '2025-01-20'
+  2.18.3:
+    changes:
+      release_summary: '| Release Date: 2025-02-24
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
+
+        '
+    codename: Fool in the Rain
+    fragments:
+    - 2.18.3_summary.yaml
+    release_date: '2025-02-24'
+  2.18.3rc1:
+    changes:
+      bugfixes:
+      - include_vars - fixed erroneous warning if an unreserved variable name contains
+        a single character that matches a reserved variable. (https://github.com/ansible/ansible/issues/84623)
+      - linear strategy - fix executing ``end_role`` meta tasks for each host, instead
+        of handling these as implicit run_once tasks (https://github.com/ansible/ansible/issues/84660).
+      minor_changes:
+      - ansible-test - Automatically retry HTTP GET/PUT/DELETE requests on exceptions.
+      - ansible-test - Use Python's ``urllib`` instead of ``curl`` for HTTP requests.
+      release_summary: '| Release Date: 2025-02-17
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
+
+        '
+    codename: Fool in the Rain
+    fragments:
+    - 2.18.3rc1_summary.yaml
+    - 84660-fix-meta-end_role-linear-strategy.yml
+    - ansible-test-curl.yml
+    - fix-include_vars-reserved-warning.yml
+    release_date: '2025-02-17'
+  2.18.4rc1:
+    changes:
+      bugfixes:
+      - Windows - add support for running on system where WDAC is in audit mode with
+        ``Dynamic Code Security`` enabled.
+      - dnf5 - fix ``is_installed`` check for packages that are not installed but
+        listed as provided by an installed package (https://github.com/ansible/ansible/issues/84578)
+      - dnf5 - libdnf5 - use ``conf.pkg_gpgcheck`` instead of deprecated ``conf.gpgcheck``
+        which is used only as a fallback
+      - facts - gather pagesize and calculate respective values depending upon architecture
+        (https://github.com/ansible/ansible/issues/84773).
+      - module respawn - limit to supported Python versions
+      release_summary: '| Release Date: 2025-03-17
+
+        | `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
+
+        '
+    codename: Fool in the Rain
+    fragments:
+    - 2.18.4rc1_summary.yaml
+    - 84578-dnf5-is_installed-provides.yml
+    - darwin_pagesize.yml
+    - dnf5-remove-usage-deprecated-option.yml
+    - respawn-min-python.yml
+    - win-wdac-audit.yml
+    release_date: '2025-03-17'
--- a/changelogs/fragments/2.18.0_summary.yaml
+++ b/changelogs/fragments/2.18.0_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2024-11-04
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
--- a/changelogs/fragments/2.18.0b1_summary.yaml
+++ b/changelogs/fragments/2.18.0b1_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2024-09-24
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
--- a/changelogs/fragments/2.18.0rc1_summary.yaml
+++ b/changelogs/fragments/2.18.0rc1_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2024-10-14
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
--- a/changelogs/fragments/2.18.0rc2_summary.yaml
+++ b/changelogs/fragments/2.18.0rc2_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2024-10-29
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
--- a/changelogs/fragments/2.18.1_summary.yaml
+++ b/changelogs/fragments/2.18.1_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2024-12-02
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
--- a/changelogs/fragments/2.18.1rc1_summary.yaml
+++ b/changelogs/fragments/2.18.1rc1_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2024-11-25
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
--- a/changelogs/fragments/2.18.2_summary.yaml
+++ b/changelogs/fragments/2.18.2_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2025-01-27
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
--- a/changelogs/fragments/2.18.2rc1_summary.yaml
+++ b/changelogs/fragments/2.18.2rc1_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2025-01-20
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
--- a/changelogs/fragments/2.18.3_summary.yaml
+++ b/changelogs/fragments/2.18.3_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2025-02-24
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
--- a/changelogs/fragments/2.18.3rc1_summary.yaml
+++ b/changelogs/fragments/2.18.3rc1_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2025-02-17
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
--- a/changelogs/fragments/2.18.4rc1_summary.yaml
+++ b/changelogs/fragments/2.18.4rc1_summary.yaml
@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2025-03-17
+   | `Porting Guide <https://docs.ansible.com/ansible-core/2.18/porting_guides/porting_guide_core_2.18.html>`__
--- a/changelogs/fragments/82708-unsafe-plugin-name-error.yml
+++ b/changelogs/fragments/82708-unsafe-plugin-name-error.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - "Fix an issue when setting a plugin name from an unsafe source resulted in ``ValueError: unmarshallable object`` (https://github.com/ansible/ansible/issues/82708)"
--- a/changelogs/fragments/84019-ignore_unreachable-loop.yml
+++ b/changelogs/fragments/84019-ignore_unreachable-loop.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - Fix returning 'unreachable' for the overall task result. This prevents false positives when a looped task has unignored unreachable items (https://github.com/ansible/ansible/issues/84019).
--- a/changelogs/fragments/84238-fix-reset_connection-ssh_executable-templated.yml
+++ b/changelogs/fragments/84238-fix-reset_connection-ssh_executable-templated.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - ssh - connection options were incorrectly templated during ``reset_connection`` tasks (https://github.com/ansible/ansible/pull/84238).
--- a/changelogs/fragments/84259-dnf5-latest-fix.yml
+++ b/changelogs/fragments/84259-dnf5-latest-fix.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - "dnf5 - fix installing a package using ``state=latest`` when a binary of the same name as the package is already installed (https://github.com/ansible/ansible/issues/84259)"
--- a/changelogs/fragments/84334-dnf5-consolidate-settings.yml
+++ b/changelogs/fragments/84334-dnf5-consolidate-settings.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - dnf5 - matching on a binary can be achieved only by specifying a full path (https://github.com/ansible/ansible/issues/84334)
--- a/changelogs/fragments/84578-dnf5-is_installed-provides.yml
+++ b/changelogs/fragments/84578-dnf5-is_installed-provides.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - "dnf5 - fix ``is_installed`` check for packages that are not installed but listed as provided by an installed package (https://github.com/ansible/ansible/issues/84578)"
--- a/changelogs/fragments/84660-fix-meta-end_role-linear-strategy.yml
+++ b/changelogs/fragments/84660-fix-meta-end_role-linear-strategy.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - linear strategy - fix executing ``end_role`` meta tasks for each host, instead of handling these as implicit run_once tasks (https://github.com/ansible/ansible/issues/84660).
--- a/changelogs/fragments/Ansible.Basic-required_if-null.yml
+++ b/changelogs/fragments/Ansible.Basic-required_if-null.yml
@ -0,0 +1,3 @@
+bugfixes:
+  - >-
+    Ansible.Basic - Fix ``required_if`` check when the option value to check is unset or set to null.
--- a/changelogs/fragments/ansible-test-coverage-test-files.yml
+++ b/changelogs/fragments/ansible-test-coverage-test-files.yml
@ -0,0 +1,4 @@
+bugfixes:
+  - >-
+    ansible-test - Fix up coverage reporting to properly translate the temporary path of integration test modules to
+    the expected static test module path.
--- a/changelogs/fragments/ansible-test-curl.yml
+++ b/changelogs/fragments/ansible-test-curl.yml
@ -0,0 +1,3 @@
+minor_changes:
+  - ansible-test - Use Python's ``urllib`` instead of ``curl`` for HTTP requests.
+  - ansible-test - Automatically retry HTTP GET/PUT/DELETE requests on exceptions.
--- a/changelogs/fragments/ansible-test-fix-command-traceback.yml
+++ b/changelogs/fragments/ansible-test-fix-command-traceback.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-test - Fix traceback that occurs after an interactive command fails.
--- a/changelogs/fragments/ansible-test-network-detection.yml
+++ b/changelogs/fragments/ansible-test-network-detection.yml
@ -0,0 +1,3 @@
+minor_changes:
+  - ansible-test - When detection of the current container network fails, a warning is now issued and execution continues.
+    This simplifies usage in cases where the current container cannot be inspected, such as when running in GitHub Codespaces.
--- a/changelogs/fragments/ansible-test-probe-error-handling.yml
+++ b/changelogs/fragments/ansible-test-probe-error-handling.yml
@ -0,0 +1,3 @@
+minor_changes:
+  - ansible-test - Improve container runtime probe error handling.
+    When unexpected probe output is encountered, an error with more useful debugging information is provided.
--- a/changelogs/fragments/ansible-test-pylint-fix.yml
+++ b/changelogs/fragments/ansible-test-pylint-fix.yml
@ -0,0 +1,4 @@
+bugfixes:
+  - ansible-test - Enable the ``sys.unraisablehook`` work-around for the ``pylint`` sanity test on Python 3.11.
+    Previously the work-around was only enabled for Python 3.12 and later.
+    However, the same issue has been discovered on Python 3.11.
--- a/changelogs/fragments/ansible-test-update.yml
+++ b/changelogs/fragments/ansible-test-update.yml
@ -0,0 +1,5 @@
+minor_changes:
+  - ansible-test - Update ``pylint`` sanity test to use version 3.3.1.
+  - ansible-test - Default to Python 3.13 in the ``base`` and ``default`` containers.
+  - ansible-test - Disable the ``deprecated-`` prefixed ``pylint`` rules as their results vary by Python version.
+  - ansible-test - Update the ``base`` and ``default`` containers.
--- a/changelogs/fragments/become-runas-system-deux.yml
+++ b/changelogs/fragments/become-runas-system-deux.yml
@ -0,0 +1,3 @@
+bugfixes:
+  - >-
+    runas become - Fix up become logic to still get the SYSTEM token with the most privileges when running as SYSTEM.
--- a/changelogs/fragments/copy_validate_input.yml
+++ b/changelogs/fragments/copy_validate_input.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - copy action now prevents user from setting internal options.
--- a/changelogs/fragments/cve-2024-8775.yml
+++ b/changelogs/fragments/cve-2024-8775.yml
@ -0,0 +1,5 @@
+security_fixes:
+  - task result processing - Ensure that action-sourced result masking (``_ansible_no_log=True``)
+    is preserved. (CVE-2024-8775)
+  - include_vars action - Ensure that result masking is correctly requested when vault-encrypted
+    files are read. (CVE-2024-8775)
--- a/changelogs/fragments/darwin_pagesize.yml
+++ b/changelogs/fragments/darwin_pagesize.yml
@ -0,0 +1,3 @@
+---
+bugfixes:
+  - facts - gather pagesize and calculate respective values depending upon architecture (https://github.com/ansible/ansible/issues/84773).
--- a/changelogs/fragments/debconf_empty_password.yml
+++ b/changelogs/fragments/debconf_empty_password.yml
@ -0,0 +1,3 @@
+---
+bugfixes:
+  - debconf - set empty password values (https://github.com/ansible/ansible/issues/83214).
--- a/changelogs/fragments/dnf5-plugins-compat.yml
+++ b/changelogs/fragments/dnf5-plugins-compat.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - "dnf5 - fix traceback when ``enable_plugins``/``disable_plugins`` is used on ``python3-libdnf5`` versions that do not support this functionality"
--- a/changelogs/fragments/dnf5-remove-usage-deprecated-option.yml
+++ b/changelogs/fragments/dnf5-remove-usage-deprecated-option.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - dnf5 - libdnf5 - use ``conf.pkg_gpgcheck`` instead of deprecated ``conf.gpgcheck`` which is used only as a fallback
--- a/changelogs/fragments/fix-ansible-galaxy-ignore-certs.yml
+++ b/changelogs/fragments/fix-ansible-galaxy-ignore-certs.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - Fix disabling SSL verification when installing collections and roles from git repositories. If ``--ignore-certs`` isn't provided, the value for the ``GALAXY_IGNORE_CERTS`` configuration option will be used (https://github.com/ansible/ansible/issues/83326).
--- a/changelogs/fragments/fix-include_vars-reserved-warning.yml
+++ b/changelogs/fragments/fix-include_vars-reserved-warning.yml
@ -0,0 +1,5 @@
+bugfixes:
+- >-
+  include_vars - fixed erroneous warning if an unreserved variable name
+  contains a single character that matches a reserved variable.
+  (https://github.com/ansible/ansible/issues/84623)
--- a/changelogs/fragments/fix-module-utils-facts-timeout.yml
+++ b/changelogs/fragments/fix-module-utils-facts-timeout.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - Use the requested error message in the ansible.module_utils.facts.timeout timeout function instead of hardcoding one.
--- a/changelogs/fragments/fix_errors.yml
+++ b/changelogs/fragments/fix_errors.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - Errors now preserve stacked error messages even when YAML is involved.
--- a/changelogs/fragments/gather_facts_netos_fixes.yml
+++ b/changelogs/fragments/gather_facts_netos_fixes.yml
@ -0,0 +1,3 @@
+bugfixes:
+  - gather_facts action will now issues errors and warnings as appropriate if a network OS is detected but no facts modules are defined for it.
+  - gather_facts action now defaults to `ansible.legacy.setup` if `smart` was set, no network OS was found and no other alias for `setup` was present.
--- a/changelogs/fragments/macos-correct-lock.yml
+++ b/changelogs/fragments/macos-correct-lock.yml
@ -0,0 +1,2 @@
+bugfixes:
+- Use consistent multiprocessing context for action write locks
--- a/changelogs/fragments/os_family.yml
+++ b/changelogs/fragments/os_family.yml
@ -0,0 +1,3 @@
+---
+bugfixes:
+  - facts - skip if distribution file path is directory, instead of raising error (https://github.com/ansible/ansible/issues/84006).
--- a/changelogs/fragments/package-dnf-action-plugins-facts-fail-msg.yml
+++ b/changelogs/fragments/package-dnf-action-plugins-facts-fail-msg.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - "``package``/``dnf`` action plugins - provide the reason behind the failure to gather the ``ansible_pkg_mgr`` fact to identify the package backend"
--- a/changelogs/fragments/reserved_module_chekc.yml
+++ b/changelogs/fragments/reserved_module_chekc.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - Ansible will now also warn when reserved keywords are set via a module (set_fact, include_vars, etc).
--- a/changelogs/fragments/respawn-min-python.yml
+++ b/changelogs/fragments/respawn-min-python.yml
@ -0,0 +1,2 @@
+bugfixes:
+- module respawn - limit to supported Python versions
--- a/changelogs/fragments/skip-implicit-flush_handlers-no-notify.yml
+++ b/changelogs/fragments/skip-implicit-flush_handlers-no-notify.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - "Improve performance on large inventories by reducing the number of implicit meta tasks."
--- a/changelogs/fragments/ssh-clixml.yml
+++ b/changelogs/fragments/ssh-clixml.yml
@ -0,0 +1,4 @@
+bugfixes:
+  - >-
+    ssh - Improve the logic for parsing CLIXML data in stderr when working with Windows host. This fixes issues when
+    the raw stderr contains invalid UTF-8 byte sequences and improves embedded CLIXML sequences.
--- a/changelogs/fragments/unsafe-fixes-2.yml
+++ b/changelogs/fragments/unsafe-fixes-2.yml
@ -0,0 +1,3 @@
+bugfixes:
+- unsafe data - Address an incompatibility with ``AnsibleUnsafeText`` and ``AnsibleUnsafeBytes`` when pickling with ``protocol=0``
+- unsafe data - Address an incompatibility when iterating or getting a single index from ``AnsibleUnsafeBytes``
--- a/changelogs/fragments/unsafe-intern.yml
+++ b/changelogs/fragments/unsafe-intern.yml
@ -0,0 +1,3 @@
+bugfixes:
+- unsafe data - Enable directly using ``AnsibleUnsafeText`` with Python ``pathlib``
+  (https://github.com/ansible/ansible/issues/82414)
--- a/changelogs/fragments/unsafe_hostvars_fix.yml
+++ b/changelogs/fragments/unsafe_hostvars_fix.yml
@ -0,0 +1,2 @@
+security_fixes:
+  - Templating will not prefer AnsibleUnsafe when a variable is referenced via hostvars - CVE-2024-11079
--- a/changelogs/fragments/user_action_fix.yml
+++ b/changelogs/fragments/user_action_fix.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - user module now avoids changing ownership of files symlinked in provided home dir skeleton
--- a/changelogs/fragments/user_ssh_fix.yml
+++ b/changelogs/fragments/user_ssh_fix.yml
@ -0,0 +1,4 @@
+bugfixes:
+  - user action will now require O(force) to overwrite the public part of an ssh key when generating ssh keys, as was already the case for the private part.
+security_fixes:
+  - user action won't allow ssh-keygen, chown and chmod to run on existing ssh public key file, avoiding traversal on existing symlinks (CVE-2024-9902).
--- a/changelogs/fragments/vault_cli_fix.yml
+++ b/changelogs/fragments/vault_cli_fix.yml
@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-vault will now correctly handle `--prompt`, previously it would issue an error about stdin if no 2nd argument was passed
--- a/changelogs/fragments/win-wdac-audit.yml
+++ b/changelogs/fragments/win-wdac-audit.yml
@ -0,0 +1,4 @@
+bugfixes:
+  - >-
+    Windows - add support for running on system where WDAC is in audit mode with
+    ``Dynamic Code Security`` enabled.
--- a/lib/ansible/cli/vault.py
+++ b/lib/ansible/cli/vault.py
@ -138,11 +138,12 @@ class VaultCLI(CLI):
            raise AnsibleOptionsError("At most one input file may be used with the --output option")

        if options.action == 'encrypt_string':
-            if '-' in options.args or not options.args or options.encrypt_string_stdin_name:
+            if '-' in options.args or options.encrypt_string_stdin_name or (not options.args and not options.encrypt_string_prompt):
+                # prompting from stdin and reading from stdin are mutually exclusive, if stdin is still provided, it is ignored
                self.encrypt_string_read_stdin = True

-            # TODO: prompting from stdin and reading from stdin seem mutually exclusive, but verify that.
            if options.encrypt_string_prompt and self.encrypt_string_read_stdin:
+                # should only trigger if prompt + either - or encrypt string stdin name were provided
                raise AnsibleOptionsError('The --prompt option is not supported if also reading input from stdin')

        return options
--- a/lib/ansible/errors/init.py
+++ b/lib/ansible/errors/init.py
@ -66,14 +66,18 @@ class AnsibleError(Exception):
        from ansible.parsing.yaml.objects import AnsibleBaseYAMLObject

        message = [self._message]
+
+        # Add from previous exceptions
+        if self.orig_exc:
+            message.append('. %s' % to_native(self.orig_exc))
+
+        # Add from yaml to give specific file/line no
        if isinstance(self.obj, AnsibleBaseYAMLObject):
            extended_error = self._get_extended_error()
            if extended_error and not self._suppress_extended_error:
                message.append(
                    '\n\n%s' % to_native(extended_error)
                )
-        elif self.orig_exc:
-            message.append('. %s' % to_native(self.orig_exc))

        return ''.join(message)

--- a/lib/ansible/executor/action_write_locks.py
+++ b/lib/ansible/executor/action_write_locks.py
@ -19,7 +19,7 @@ from __future__ import annotations

 import multiprocessing.synchronize

-from multiprocessing import Lock
+from ansible.utils.multiprocessing import context as multiprocessing_context

 from ansible.module_utils.facts.system.pkg_mgr import PKG_MGRS

@ -32,7 +32,7 @@ if 'action_write_locks' not in globals():
    # Below is a Lock for use when we weren't expecting a named module.  It gets used when an action
    # plugin invokes a module whose name does not match with the action's name.  Slightly less
    # efficient as all processes with unexpected module names will wait on this lock
-    action_write_locks[None] = Lock()
+    action_write_locks[None] = multiprocessing_context.Lock()

    # These plugins are known to be called directly by action plugins with names differing from the
    # action plugin name.  We precreate them here as an optimization.
@ -41,4 +41,4 @@ if 'action_write_locks' not in globals():

    mods.update(('copy', 'file', 'setup', 'slurp', 'stat'))
    for mod_name in mods:
-        action_write_locks[mod_name] = Lock()
+        action_write_locks[mod_name] = multiprocessing_context.Lock()
--- a/lib/ansible/executor/play_iterator.py
+++ b/lib/ansible/executor/play_iterator.py
@ -447,6 +447,24 @@ class PlayIterator:

            # if something above set the task, break out of the loop now
            if task:
+                # skip implicit flush_handlers if there are no handlers notified
+                if (
+                    task.implicit
+                    and task.action in C._ACTION_META
+                    and task.args.get('_raw_params', None) == 'flush_handlers'
+                    and (
+                        # the state store in the `state` variable could be a nested state,
+                        # notifications are always stored in the top level state, get it here
+                        not self.get_state_for_host(host.name).handler_notifications
+                        # in case handlers notifying other handlers, the notifications are not
+                        # saved in `handler_notifications` and handlers are notified directly
+                        # to prevent duplicate handler runs, so check whether any handler
+                        # is notified
+                        and all(not h.notified_hosts for h in self.handlers)
+                    )
+                ):
+                    continue
+
                break

        return (state, task)
--- a/lib/ansible/executor/task_executor.py
+++ b/lib/ansible/executor/task_executor.py
@ -150,6 +150,7 @@ class TaskExecutor:
                        if 'unreachable' in item and item['unreachable']:
                            item_ignore_unreachable = item.pop('_ansible_ignore_unreachable')
                            if not res.get('unreachable'):
+                                res['unreachable'] = True
                                self._task.ignore_unreachable = item_ignore_unreachable
                            elif self._task.ignore_unreachable and not item_ignore_unreachable:
                                self._task.ignore_unreachable = item_ignore_unreachable
@ -684,8 +685,8 @@ class TaskExecutor:
                self._handler.cleanup()
            display.debug("handler run complete")

-            # preserve no log
-            result["_ansible_no_log"] = no_log
+            # propagate no log to result- the action can set this, so only overwrite it with the task's value if missing or falsey
+            result["_ansible_no_log"] = bool(no_log or result.get('_ansible_no_log', False))

            if self._task.action not in C._ACTION_WITH_CLEAN_FACTS:
                result = wrap_var(result)
@ -1072,18 +1073,6 @@ class TaskExecutor:
        option_vars = C.config.get_plugin_vars('connection', self._connection._load_name)
        varnames.extend(option_vars)

-        # create dict of 'templated vars'
-        options = {'_extras': {}}
-        for k in option_vars:
-            if k in variables:
-                options[k] = templar.template(variables[k])
-
-        # add extras if plugin supports them
-        if getattr(self._connection, 'allow_extras', False):
-            for k in variables:
-                if k.startswith('ansible_%s_' % self._connection.extras_prefix) and k not in options:
-                    options['_extras'][k] = templar.template(variables[k])
-
        task_keys = self._task.dump_attrs()

        # The task_keys 'timeout' attr is the task's timeout, not the connection timeout.
@ -1101,7 +1090,8 @@ class TaskExecutor:
        del task_keys['retries']

        # set options with 'templated vars' specific to this plugin and dependent ones
-        self._connection.set_options(task_keys=task_keys, var_options=options)
+        var_options = self._connection._resolve_option_variables(variables, templar)
+        self._connection.set_options(task_keys=task_keys, var_options=var_options)
        varnames.extend(self._set_plugin_options('shell', variables, templar, task_keys))

        if self._connection.become is not None:
--- a/lib/ansible/executor/task_queue_manager.py
+++ b/lib/ansible/executor/task_queue_manager.py
@ -39,7 +39,6 @@ from ansible.plugins.loader import callback_loader, strategy_loader, module_load
 from ansible.plugins.callback import CallbackBase
 from ansible.template import Templar
 from ansible.vars.hostvars import HostVars
-from ansible.vars.reserved import warn_if_reserved
 from ansible.utils.display import Display
 from ansible.utils.lock import lock_decorator
 from ansible.utils.multiprocessing import context as multiprocessing_context
@ -282,7 +281,6 @@ class TaskQueueManager:

        all_vars = self._variable_manager.get_vars(play=play)
        templar = Templar(loader=self._loader, variables=all_vars)
-        warn_if_reserved(all_vars, templar.environment.globals.keys())

        new_play = play.copy()
        new_play.post_validate(templar)
--- a/lib/ansible/galaxy/collection/concrete_artifact_manager.py
+++ b/lib/ansible/galaxy/collection/concrete_artifact_manager.py
@ -10,6 +10,7 @@ import os
 import tarfile
 import subprocess
 import typing as t
+import yaml

 from contextlib import contextmanager
 from hashlib import sha256
@ -24,6 +25,7 @@ if t.TYPE_CHECKING:
    )
    from ansible.galaxy.token import GalaxyToken

+from ansible import context
 from ansible.errors import AnsibleError
 from ansible.galaxy import get_collections_galaxy_meta_info
 from ansible.galaxy.api import should_retry_error
@ -38,7 +40,7 @@ from ansible.module_utils.urls import open_url
 from ansible.utils.display import Display
 from ansible.utils.sentinel import Sentinel

-import yaml
+import ansible.constants as C


 display = Display()
@ -425,11 +427,14 @@ def _extract_collection_from_git(repo_url, coll_ver, b_path):

    # Perform a shallow clone if simply cloning HEAD
    if version == 'HEAD':
-        git_clone_cmd = git_executable, 'clone', '--depth=1', git_url, to_text(b_checkout_path)
+        git_clone_cmd = [git_executable, 'clone', '--depth=1', git_url, to_text(b_checkout_path)]
    else:
-        git_clone_cmd = git_executable, 'clone', git_url, to_text(b_checkout_path)
+        git_clone_cmd = [git_executable, 'clone', git_url, to_text(b_checkout_path)]
    # FIXME: '--branch', version

+    if context.CLIARGS['ignore_certs'] or C.GALAXY_IGNORE_CERTS:
+        git_clone_cmd.extend(['-c', 'http.sslVerify=false'])
+
    try:
        subprocess.check_call(git_clone_cmd)
    except subprocess.CalledProcessError as proc_err:
--- a/lib/ansible/module_utils/common/json.py
+++ b/lib/ansible/module_utils/common/json.py
@ -28,7 +28,7 @@ def _preprocess_unsafe_encode(value):
    Used in ``AnsibleJSONEncoder.iterencode``
    """
    if _is_unsafe(value):
-        value = {'__ansible_unsafe': to_text(value, errors='surrogate_or_strict', nonstring='strict')}
+        value = {'__ansible_unsafe': to_text(value._strip_unsafe(), errors='surrogate_or_strict', nonstring='strict')}
    elif is_sequence(value):
        value = [_preprocess_unsafe_encode(v) for v in value]
    elif isinstance(value, Mapping):
@ -61,7 +61,7 @@ class AnsibleJSONEncoder(json.JSONEncoder):
                value = {'__ansible_vault': to_text(o._ciphertext, errors='surrogate_or_strict', nonstring='strict')}
        elif getattr(o, '__UNSAFE__', False):
            # unsafe object, this will never be triggered, see ``AnsibleJSONEncoder.iterencode``
-            value = {'__ansible_unsafe': to_text(o, errors='surrogate_or_strict', nonstring='strict')}
+            value = {'__ansible_unsafe': to_text(o._strip_unsafe(), errors='surrogate_or_strict', nonstring='strict')}
        elif isinstance(o, Mapping):
            # hostvars and other objects
            value = dict(o)
--- a/lib/ansible/module_utils/common/respawn.py
+++ b/lib/ansible/module_utils/common/respawn.py
@ -4,11 +4,14 @@
 from __future__ import annotations

 import os
+import pathlib
 import subprocess
 import sys

 from ansible.module_utils.common.text.converters import to_bytes

+_ANSIBLE_PARENT_PATH = pathlib.Path(__file__).parents[3]
+

 def has_respawned():
    return hasattr(sys.modules['__main__'], '_respawned')
@ -54,11 +57,20 @@ def probe_interpreters_for_module(interpreter_paths, module_name):
    be returned (or ``None`` if probing fails for all supplied paths).
    :arg module_name: fully-qualified Python module name to probe for (eg, ``selinux``)
    """
+    PYTHONPATH = os.getenv('PYTHONPATH', '')
+    env = os.environ | {'PYTHONPATH': f'{_ANSIBLE_PARENT_PATH}:{PYTHONPATH}'.rstrip(': ')}
    for interpreter_path in interpreter_paths:
        if not os.path.exists(interpreter_path):
            continue
        try:
-            rc = subprocess.call([interpreter_path, '-c', 'import {0}'.format(module_name)])
+            rc = subprocess.call(
+                [
+                    interpreter_path,
+                    '-c',
+                    f'import {module_name}, ansible.module_utils.basic',
+                ],
+                env=env,
+            )
            if rc == 0:
                return interpreter_path
        except Exception:
--- a/lib/ansible/module_utils/common/text/converters.py
+++ b/lib/ansible/module_utils/common/text/converters.py
@ -267,14 +267,11 @@ def _json_encode_fallback(obj):


 def jsonify(data, **kwargs):
-    # After 2.18, we should remove this loop, and hardcode to utf-8 in alignment with requiring utf-8 module responses
-    for encoding in ("utf-8", "latin-1"):
-        try:
-            new_data = container_to_text(data, encoding=encoding)
-        except UnicodeDecodeError:
-            continue
-        return json.dumps(new_data, default=_json_encode_fallback, **kwargs)
-    raise UnicodeError('Invalid unicode encoding encountered')
+    try:
+        new_data = container_to_text(data, encoding='utf-8')
+    except UnicodeDecodeError:
+        raise UnicodeError('Invalid unicode encoding encountered')
+    return json.dumps(new_data, default=_json_encode_fallback, **kwargs)


 def container_to_bytes(d, encoding='utf-8', errors='surrogate_or_strict'):
--- a/lib/ansible/module_utils/csharp/Ansible.AccessToken.cs
+++ b/lib/ansible/module_utils/csharp/Ansible.AccessToken.cs
@ -339,19 +339,47 @@ namespace Ansible.AccessToken
        public static IEnumerable<SafeNativeHandle> EnumerateUserTokens(SecurityIdentifier sid,
            TokenAccessLevels access = TokenAccessLevels.Query)
        {
+            return EnumerateUserTokens(sid, access, (p, h) => true);
+        }
+
+        public static IEnumerable<SafeNativeHandle> EnumerateUserTokens(
+            SecurityIdentifier sid,
+            TokenAccessLevels access,
+            Func<System.Diagnostics.Process, SafeNativeHandle, bool> processFilter)
+        {
+            // We always need the Query access level so we can query the TokenUser
+            access |= TokenAccessLevels.Query;
+
            foreach (System.Diagnostics.Process process in System.Diagnostics.Process.GetProcesses())
            {
-                // We always need the Query access level so we can query the TokenUser
                using (process)
-                using (SafeNativeHandle hToken = TryOpenAccessToken(process, access | TokenAccessLevels.Query))
+                using (SafeNativeHandle processHandle = NativeMethods.OpenProcess(ProcessAccessFlags.QueryInformation, false, (UInt32)process.Id))
                {
-                    if (hToken == null)
+                    if (processHandle.IsInvalid)
+                    {
                        continue;
+                    }

-                    if (!sid.Equals(GetTokenUser(hToken)))
+                    if (!processFilter(process, processHandle))
+                    {
                        continue;
+                    }

-                    yield return hToken;
+                    SafeNativeHandle accessToken;
+                    if (!NativeMethods.OpenProcessToken(processHandle, access, out accessToken))
+                    {
+                        continue;
+                    }
+
+                    using (accessToken)
+                    {
+                        if (!sid.Equals(GetTokenUser(accessToken)))
+                        {
+                            continue;
+                        }
+
+                        yield return accessToken;
+                    }
                }
            }
        }
@ -440,18 +468,5 @@ namespace Ansible.AccessToken
            for (int i = 0; i < array.Length; i++, ptrOffset = IntPtr.Add(ptrOffset, Marshal.SizeOf(typeof(T))))
                array[i] = (T)Marshal.PtrToStructure(ptrOffset, typeof(T));
        }
-
-        private static SafeNativeHandle TryOpenAccessToken(System.Diagnostics.Process process, TokenAccessLevels access)
-        {
-            try
-            {
-                using (SafeNativeHandle hProcess = OpenProcess(process.Id, ProcessAccessFlags.QueryInformation, false))
-                    return OpenProcessToken(hProcess, access);
-            }
-            catch (Win32Exception)
-            {
-                return null;
-            }
-        }
    }
 }
--- a/lib/ansible/module_utils/csharp/Ansible.Basic.cs
+++ b/lib/ansible/module_utils/csharp/Ansible.Basic.cs
@ -1210,7 +1210,7 @@ namespace Ansible.Basic
                object val = requiredCheck[1];
                IList requirements = (IList)requiredCheck[2];

-                if (ParseStr(param[key]) != ParseStr(val))
+                if (param[key] == null || ParseStr(param[key]) != ParseStr(val))
                    continue;

                string term = "all";
--- a/lib/ansible/module_utils/csharp/Ansible.Become.cs
+++ b/lib/ansible/module_utils/csharp/Ansible.Become.cs
@ -93,10 +93,21 @@ namespace Ansible.Become
            CachedRemoteInteractive,
            CachedUnlock
        }
+
+        [Flags]
+        public enum ProcessChildProcessPolicyFlags
+        {
+            None = 0x0,
+            NoChildProcessCreation = 0x1,
+            AuditNoChildProcessCreation = 0x2,
+            AllowSecureProcessCreation = 0x4,
+        }
    }

    internal class NativeMethods
    {
+        public const int ProcessChildProcessPolicy = 13;
+
        [DllImport("advapi32.dll", SetLastError = true)]
        public static extern bool AllocateLocallyUniqueId(
            out Luid Luid);
@ -116,6 +127,13 @@ namespace Ansible.Become
        [DllImport("kernel32.dll")]
        public static extern UInt32 GetCurrentThreadId();

+        [DllImport("kernel32.dll", SetLastError = true)]
+        public static extern bool GetProcessMitigationPolicy(
+            SafeNativeHandle hProcess,
+            int MitigationPolicy,
+            ref NativeHelpers.ProcessChildProcessPolicyFlags lpBuffer,
+            IntPtr dwLength);
+
        [DllImport("user32.dll", SetLastError = true)]
        public static extern NoopSafeHandle GetProcessWindowStation();

@ -217,6 +235,7 @@ namespace Ansible.Become
        };
        private static int WINDOWS_STATION_ALL_ACCESS = 0x000F037F;
        private static int DESKTOP_RIGHTS_ALL_ACCESS = 0x000F01FF;
+        private static bool _getProcessMitigationPolicySupported = true;

        public static Result CreateProcessAsUser(string username, string password, string command)
        {
@ -333,12 +352,13 @@ namespace Ansible.Become
                // Grant access to the current Windows Station and Desktop to the become user
                GrantAccessToWindowStationAndDesktop(account);

-                // Try and impersonate a SYSTEM token. We need the SeTcbPrivilege for
-                //  - LogonUser for a service SID
-                //  - S4U logon
-                //  - Token elevation
+                // Try and impersonate a SYSTEM token, we need a SYSTEM token to either become a well known service
+                // account or have administrative rights on the become access token.
+                // If we ultimately are becoming the SYSTEM account we want the token with the most privileges available.
+                // https://github.com/ansible/ansible/issues/71453
+                bool usedForProcess = becomeSid == "S-1-5-18";
                systemToken = GetPrimaryTokenForUser(new SecurityIdentifier("S-1-5-18"),
-                    new List<string>() { "SeTcbPrivilege" });
+                    new List<string>() { "SeTcbPrivilege" }, usedForProcess);
                if (systemToken != null)
                {
                    try
@ -356,9 +376,11 @@ namespace Ansible.Become

            try
            {
+                if (becomeSid == "S-1-5-18")
+                    userTokens.Add(systemToken);
                // Cannot use String.IsEmptyOrNull() as an empty string is an account that doesn't have a pass.
                // We only use S4U if no password was defined or it was null
-                if (!SERVICE_SIDS.Contains(becomeSid) && password == null && logonType != LogonType.NewCredentials)
+                else if (!SERVICE_SIDS.Contains(becomeSid) && password == null && logonType != LogonType.NewCredentials)
                {
                    // If no password was specified, try and duplicate an existing token for that user or use S4U to
                    // generate one without network credentials
@ -381,11 +403,6 @@ namespace Ansible.Become
                    string domain = null;
                    switch (becomeSid)
                    {
-                        case "S-1-5-18":
-                            logonType = LogonType.Service;
-                            domain = "NT AUTHORITY";
-                            username = "SYSTEM";
-                            break;
                        case "S-1-5-19":
                            logonType = LogonType.Service;
                            domain = "NT AUTHORITY";
@ -427,8 +444,10 @@ namespace Ansible.Become
            return userTokens;
        }

-        private static SafeNativeHandle GetPrimaryTokenForUser(SecurityIdentifier sid,
-            List<string> requiredPrivileges = null)
+        private static SafeNativeHandle GetPrimaryTokenForUser(
+            SecurityIdentifier sid,
+            List<string> requiredPrivileges = null,
+            bool usedForProcess = false)
        {
            // According to CreateProcessWithTokenW we require a token with
            //  TOKEN_QUERY, TOKEN_DUPLICATE and TOKEN_ASSIGN_PRIMARY
@ -438,7 +457,19 @@ namespace Ansible.Become
                TokenAccessLevels.AssignPrimary |
                TokenAccessLevels.Impersonate;

-            foreach (SafeNativeHandle hToken in TokenUtil.EnumerateUserTokens(sid, dwAccess))
+            SafeNativeHandle userToken = null;
+            int privilegeCount = 0;
+
+            // If we are using this token for the process, we need to check the
+            // process mitigation policy allows child processes to be created.
+            var processFilter = usedForProcess
+                ? (Func<System.Diagnostics.Process, SafeNativeHandle, bool>)((p, t) =>
+                {
+                    return GetProcessChildProcessPolicyFlags(t) == NativeHelpers.ProcessChildProcessPolicyFlags.None;
+                })
+                : ((p, t) => true);
+
+            foreach (SafeNativeHandle hToken in TokenUtil.EnumerateUserTokens(sid, dwAccess, processFilter))
            {
                // Filter out any Network logon tokens, using become with that is useless when S4U
                // can give us a Batch logon
@ -448,6 +479,10 @@ namespace Ansible.Become

                List<string> actualPrivileges = TokenUtil.GetTokenPrivileges(hToken).Select(x => x.Name).ToList();

+                // If the token has less or the same number of privileges than the current token, skip it.
+                if (usedForProcess && privilegeCount >= actualPrivileges.Count)
+                    continue;
+
                // Check that the required privileges are on the token
                if (requiredPrivileges != null)
                {
@ -459,16 +494,22 @@ namespace Ansible.Become
                // Duplicate the token to convert it to a primary token with the access level required.
                try
                {
-                    return TokenUtil.DuplicateToken(hToken, TokenAccessLevels.MaximumAllowed,
+                    userToken = TokenUtil.DuplicateToken(hToken, TokenAccessLevels.MaximumAllowed,
                        SecurityImpersonationLevel.Anonymous, TokenType.Primary);
+                    privilegeCount = actualPrivileges.Count;
                }
                catch (Process.Win32Exception)
                {
                    continue;
                }
+
+                // If we don't care about getting the token with the most privileges, escape the loop as we already
+                // have a token.
+                if (!usedForProcess)
+                    break;
            }

-            return null;
+            return userToken;
        }

        private static SafeNativeHandle GetS4UTokenForUser(SecurityIdentifier sid, LogonType logonType)
@ -581,6 +622,35 @@ namespace Ansible.Become
                return null;
        }

+        private static NativeHelpers.ProcessChildProcessPolicyFlags GetProcessChildProcessPolicyFlags(SafeNativeHandle processHandle)
+        {
+            // Because this is only used to check the policy, we ignore any
+            // errors and pretend that the policy is None.
+            NativeHelpers.ProcessChildProcessPolicyFlags policy = NativeHelpers.ProcessChildProcessPolicyFlags.None;
+
+            if (_getProcessMitigationPolicySupported)
+            {
+                try
+                {
+                    if (NativeMethods.GetProcessMitigationPolicy(
+                        processHandle,
+                        NativeMethods.ProcessChildProcessPolicy,
+                        ref policy,
+                        (IntPtr)4))
+                    {
+                        return policy;
+                    }
+                }
+                catch (EntryPointNotFoundException)
+                {
+                    // If the function is not available, we won't try to call it again
+                    _getProcessMitigationPolicySupported = false;
+                }
+            }
+
+            return policy;
+        }
+
        private static NativeHelpers.SECURITY_LOGON_TYPE GetTokenLogonType(SafeNativeHandle hToken)
        {
            TokenStatistics stats = TokenUtil.GetTokenStatistics(hToken);
@ -637,4 +707,4 @@ namespace Ansible.Become
            { }
        }
    }
-}
+}
--- a/lib/ansible/module_utils/facts/ansible_collector.py
+++ b/lib/ansible/module_utils/facts/ansible_collector.py
@ -113,7 +113,13 @@ class CollectorMetaDataCollector(collector.BaseFactCollector):
        self.module_setup = module_setup

    def collect(self, module=None, collected_facts=None):
+        # NOTE: deprecate/remove once DT lands
+        # we can return this data, but should not be top level key
        meta_facts = {'gather_subset': self.gather_subset}
+
+        # NOTE: this is just a boolean indicator that 'facts were gathered'
+        # and should be moved to the 'gather_facts' action plugin
+        # probably revised to handle modules/subsets combos
        if self.module_setup:
            meta_facts['module_setup'] = self.module_setup
        return meta_facts
--- a/lib/ansible/module_utils/facts/hardware/darwin.py
+++ b/lib/ansible/module_utils/facts/hardware/darwin.py
@ -94,6 +94,8 @@ class DarwinHardware(Hardware):

        total_used = 0
        page_size = 4096
+        if 'hw.pagesize' in self.sysctl:
+            page_size = int(self.sysctl['hw.pagesize'])

        vm_stat_command = self.module.get_bin_path('vm_stat')
        if vm_stat_command is None:
--- a/lib/ansible/module_utils/facts/system/distribution.py
+++ b/lib/ansible/module_utils/facts/system/distribution.py
@ -30,7 +30,7 @@ def get_uname(module, flags=('-v')):

 def _file_exists(path, allow_empty=False):
    # not finding the file, exit early
-    if not os.path.exists(path):
+    if not os.path.isfile(path):
        return False

    # if just the path needs to exists (ie, it can be empty) we are done
--- a/lib/ansible/module_utils/facts/timeout.py
+++ b/lib/ansible/module_utils/facts/timeout.py
@ -48,7 +48,7 @@ def timeout(seconds=None, error_message="Timer expired"):
                return res.get(timeout_value)
            except multiprocessing.TimeoutError:
                # This is an ansible.module_utils.common.facts.timeout.TimeoutError
-                raise TimeoutError('Timer expired after %s seconds' % timeout_value)
+                raise TimeoutError(f'{error_message} after {timeout_value} seconds')
            finally:
                pool.terminate()

--- a/lib/ansible/module_utils/powershell/Ansible.ModuleUtils.AddType.psm1
+++ b/lib/ansible/module_utils/powershell/Ansible.ModuleUtils.AddType.psm1
@ -312,7 +312,7 @@ Function Add-CSharpType {
        # fatal error.
        # https://github.com/ansible-collections/ansible.windows/issues/598
        $ignore_warnings = [System.Collections.ArrayList]@('1610')
-        $compile_units = [System.Collections.Generic.List`1[System.CodeDom.CodeSnippetCompileUnit]]@()
+        $compile_units = [System.Collections.Generic.List`1[string]]@()
        foreach ($reference in $References) {
            # scan through code and add any assemblies that match
            # //AssemblyReference -Name ... [-CLR Framework]
@ -346,7 +346,7 @@ Function Add-CSharpType {
                }
                $ignore_warnings.Add($warning_id) > $null
            }
-            $compile_units.Add((New-Object -TypeName System.CodeDom.CodeSnippetCompileUnit -ArgumentList $reference)) > $null
+            $compile_units.Add($reference) > $null

            $type_matches = $type_pattern.Matches($reference)
            foreach ($match in $type_matches) {
@ -381,7 +381,10 @@ Function Add-CSharpType {

            $null = New-Item -Path $temp_path -ItemType Directory -Force
            try {
-                $compile = $provider.CompileAssemblyFromDom($compile_parameters, $compile_units)
+                # FromSource is important, it will create the .cs files with
+                # the required extended attribute for the source to be trusted
+                # when using WDAC.
+                $compile = $provider.CompileAssemblyFromSource($compile_parameters, $compile_units)
            }
            finally {
                # Try to delete the temp path, if this fails and we are running
--- a/lib/ansible/modules/command.py
+++ b/lib/ansible/modules/command.py
@ -15,12 +15,11 @@ version_added: historical
 description:
     - The M(ansible.builtin.command) module takes the command name followed by a list of space-delimited arguments.
     - The given command will be executed on all selected nodes.
-     - The command(s) will not be
-       processed through the shell, so variables like C($HOSTNAME) and operations
-       like C("*"), C("<"), C(">"), C("|"), C(";") and C("&") will not work.
+     - The command(s) will not be processed through the shell, so operations like C("*"), C("<"), C(">"), C("|"), C(";") and C("&") will not work.
+       Also, environment variables are resolved via Python, not shell, see O(expand_argument_vars) and are left unchanged if not matched.
       Use the M(ansible.builtin.shell) module if you need these features.
-     - To create C(command) tasks that are easier to read than the ones using space-delimited
-       arguments, pass parameters using the C(args) L(task keyword,https://docs.ansible.com/ansible/latest/reference_appendices/playbooks_keywords.html#task)
+     - To create C(command) tasks that are easier to read than the ones using space-delimited arguments,
+       pass parameters using the C(args) L(task keyword,https://docs.ansible.com/ansible/latest/reference_appendices/playbooks_keywords.html#task)
       or use O(cmd) parameter.
     - Either a free form command or O(cmd) parameter is required, see the examples.
     - For Windows targets, use the M(ansible.windows.win_command) module instead.
@ -41,8 +40,8 @@ attributes:
 options:
  expand_argument_vars:
    description:
-      - Expands the arguments that are variables, for example C($HOME) will be expanded before being passed to the
-        command to run.
+      - Expands the arguments that are variables, for example C($HOME) will be expanded before being passed to the command to run.
+      - If a variable is not matched, it is left unchanged, unlike shell substitution which would remove it.
      - Set to V(false) to disable expansion and treat the value as a literal argument.
    type: bool
    default: true
--- a/lib/ansible/modules/debconf.py
+++ b/lib/ansible/modules/debconf.py
@ -134,21 +134,24 @@ def get_password_value(module, pkg, question, vtype):
    cmd = [getsel]
    rc, out, err = module.run_command(cmd)
    if rc != 0:
-        module.fail_json(msg="Failed to get the value '%s' from '%s'" % (question, pkg))
+        module.fail_json(msg=f"Failed to get the value '{question}' from '{pkg}': {err}")

-    desired_line = None
    for line in out.split("\n"):
-        if line.startswith(pkg):
-            desired_line = line
-            break
+        if not line.startswith(pkg):
+            continue

-    if not desired_line:
-        module.fail_json(msg="Failed to find the value '%s' from '%s'" % (question, pkg))
-
-    (dpkg, dquestion, dvtype, dvalue) = desired_line.split()
-    if dquestion == question and dvtype == vtype:
-        return dvalue
-    return ''
+        # line is a collection of tab separated values
+        fields = line.split('\t')
+        if len(fields) <= 3:
+            # No password found, return a blank password
+            return ''
+        try:
+            if fields[1] == question and fields[2] == vtype:
+                # If correct question and question type found, return password value
+                return fields[3]
+        except IndexError:
+            # Fail safe
+            return ''


 def get_selections(module, pkg):
--- a/lib/ansible/modules/dnf5.py
+++ b/lib/ansible/modules/dnf5.py
@ -358,6 +358,31 @@ libdnf5 = None

 def is_installed(base, spec):
    settings = libdnf5.base.ResolveSpecSettings()
+    try:
+        settings.set_group_with_name(True)
+        # Disable checking whether SPEC is a binary -> `/usr/(s)bin/<SPEC>`,
+        # this prevents scenarios like the following:
+        #   * the `sssd-common` package is installed and provides `/usr/sbin/sssd` binary
+        #   * the `sssd` package is NOT installed
+        #   * due to `set_with_binaries(True)` being default `is_installed(base, "sssd")` would "unexpectedly" return True
+        # If users wish to target the `sssd` binary they can by specifying the full path `name=/usr/sbin/sssd` explicitly
+        # due to settings.set_with_filenames(True) being default.
+        settings.set_with_binaries(False)
+        # Disable checking whether SPEC is provided by an installed package.
+        # Consider following real scenario from the rpmfusion repo:
+        #   * the `ffmpeg-libs` package is installed and provides `libavcodec-freeworld`
+        #   * but `libavcodec-freeworld` is NOT installed (???)
+        #   * due to `set_with_provides(True)` being default `is_installed(base, "libavcodec-freeworld")`
+        #     would  "unexpectedly" return True
+        # We disable provides only for this `is_installed` check, for actual installation we leave the default
+        # setting to mirror the dnf cmdline behavior.
+        settings.set_with_provides(False)
+    except AttributeError:
+        # dnf5 < 5.2.0.0
+        settings.group_with_name = True
+        settings.with_binaries = False
+        settings.with_provides = False
+
    installed_query = libdnf5.rpm.PackageQuery(base)
    installed_query.filter_installed()
    match, nevra = installed_query.resolve_pkg_spec(spec, settings, True)
@ -451,7 +476,15 @@ class Dnf5Module(YumDnf):

    def fail_on_non_existing_plugins(self, base):
        # https://github.com/rpm-software-management/dnf5/issues/1460
-        plugin_names = [p.get_name() for p in base.get_plugins_info()]
+        try:
+            plugin_names = [p.get_name() for p in base.get_plugins_info()]
+        except AttributeError:
+            # plugins functionality requires python3-libdnf5 5.2.0.0+
+            # silently ignore here, the module will fail later when
+            # base.enable_disable_plugins is attempted to be used if
+            # user specifies enable_plugin/disable_plugin
+            return
+
        msg = []
        if enable_unmatched := set(self.enable_plugin).difference(plugin_names):
            msg.append(
@ -544,7 +577,14 @@ class Dnf5Module(YumDnf):
        elif self.best is not None:
            conf.best = self.best
        conf.install_weak_deps = self.install_weak_deps
-        conf.gpgcheck = not self.disable_gpg_check
+        try:
+            # raises AttributeError only on getter if not available
+            conf.pkg_gpgcheck   # pylint: disable=pointless-statement
+        except AttributeError:
+            # dnf5 < 5.2.7.0
+            conf.gpgcheck = not self.disable_gpg_check
+        else:
+            conf.pkg_gpgcheck = not self.disable_gpg_check
        conf.localpkg_gpgcheck = not self.disable_gpg_check
        conf.sslverify = self.sslverify
        conf.clean_requirements_on_remove = self.autoremove
@ -638,9 +678,12 @@ class Dnf5Module(YumDnf):
        settings = libdnf5.base.GoalJobSettings()
        try:
            settings.set_group_with_name(True)
+            settings.set_with_binaries(False)
        except AttributeError:
            # dnf5 < 5.2.0.0
            settings.group_with_name = True
+            settings.with_binaries = False
+
        if self.bugfix or self.security:
            advisory_query = libdnf5.advisory.AdvisoryQuery(base)
            types = []
--- a/lib/ansible/modules/meta.py
+++ b/lib/ansible/modules/meta.py
@ -22,8 +22,13 @@ options:
          points to implicitly trigger handler runs (after pre/post tasks, the final role execution, and the main tasks section of your plays).
        - V(refresh_inventory) (added in Ansible 2.0) forces the reload of the inventory, which in the case of dynamic inventory scripts means they will be
          re-executed. If the dynamic inventory script is using a cache, Ansible cannot know this and has no way of refreshing it (you can disable the cache
-          or, if available for your specific inventory datasource (e.g. aws), you can use the an inventory plugin instead of an inventory script).
-          This is mainly useful when additional hosts are created and users wish to use them instead of using the M(ansible.builtin.add_host) module.
+          or, if available for your specific inventory datasource (for example P(amazon.aws.aws_ec2#inventory)), you can use the an inventory plugin instead
+          of an inventory script). This is mainly useful when additional hosts are created and users wish to use them instead of using the
+          M(ansible.builtin.add_host) module.
+        - Note that neither V(refresh_inventory) nor the M(ansible.builtin.add_host) add hosts to the hosts the current play iterates over.
+          However, if needed, you can explicitly delegate tasks to new hosts with C(delegate_to). Generally,
+          C(delegate_to) can be used against hosts regardless of whether they are in the inventory or not, as long as
+          the value supplied is sufficient for the connection plugin to access the host.
        - V(noop) (added in Ansible 2.0) This literally does 'nothing'. It is mainly used internally and not recommended for general use.
        - V(clear_facts) (added in Ansible 2.1) causes the gathered facts for the hosts specified in the play's list of hosts to be cleared,
          including the fact cache.
--- a/lib/ansible/modules/user.py
+++ b/lib/ansible/modules/user.py
@ -1220,9 +1220,11 @@ class User(object):
        overwrite = None
        try:
            ssh_key_file = self.get_ssh_key_path()
+            pub_file = f'{ssh_key_file}.pub'
        except Exception as e:
            return (1, '', to_native(e))
        ssh_dir = os.path.dirname(ssh_key_file)
+
        if not os.path.exists(ssh_dir):
            if self.module.check_mode:
                return (0, '', '')
@ -1231,12 +1233,23 @@ class User(object):
                os.chown(ssh_dir, info[2], info[3])
            except OSError as e:
                return (1, '', 'Failed to create %s: %s' % (ssh_dir, to_native(e)))
+
        if os.path.exists(ssh_key_file):
            if self.force:
-                # ssh-keygen doesn't support overwriting the key interactively, so send 'y' to confirm
+                self.module.warn(f'Overwriting existing ssh key private file "{ssh_key_file}"')
                overwrite = 'y'
            else:
+                self.module.warn(f'Found existing ssh key private file "{ssh_key_file}", no force, so skipping ssh-keygen generation')
                return (None, 'Key already exists, use "force: yes" to overwrite', '')
+
+        if os.path.exists(pub_file):
+            if self.force:
+                self.module.warn(f'Overwriting existing ssh key public file "{pub_file}"')
+                os.unlink(pub_file)
+            else:
+                self.module.warn(f'Found existing ssh key public file "{pub_file}", no force, so skipping ssh-keygen generation')
+                return (None, 'Public key already exists, use "force: yes" to overwrite', '')
+
        cmd = [self.module.get_bin_path('ssh-keygen', True)]
        cmd.append('-t')
        cmd.append(self.ssh_type)
@ -1303,7 +1316,7 @@ class User(object):
            # If the keys were successfully created, we should be able
            # to tweak ownership.
            os.chown(ssh_key_file, info[2], info[3])
-            os.chown('%s.pub' % ssh_key_file, info[2], info[3])
+            os.chown(pub_file, info[2], info[3])
        return (rc, out, err)

    def ssh_key_fingerprint(self):
@ -1375,7 +1388,9 @@ class User(object):
                for d in dirs:
                    os.chown(os.path.join(root, d), uid, gid)
                for f in files:
-                    os.chown(os.path.join(root, f), uid, gid)
+                    full_path = os.path.join(root, f)
+                    if not os.path.islink(full_path):
+                        os.chown(full_path, uid, gid)
        except OSError as e:
            self.module.exit_json(failed=True, msg="%s" % to_native(e))

--- a/lib/ansible/parsing/yaml/dumper.py
+++ b/lib/ansible/parsing/yaml/dumper.py
@ -22,7 +22,7 @@ import yaml
 from ansible.module_utils.six import text_type, binary_type
 from ansible.module_utils.common.yaml import SafeDumper
 from ansible.parsing.yaml.objects import AnsibleUnicode, AnsibleSequence, AnsibleMapping, AnsibleVaultEncryptedUnicode
-from ansible.utils.unsafe_proxy import AnsibleUnsafeText, AnsibleUnsafeBytes, NativeJinjaUnsafeText, NativeJinjaText
+from ansible.utils.unsafe_proxy import AnsibleUnsafeText, AnsibleUnsafeBytes, NativeJinjaUnsafeText, NativeJinjaText, _is_unsafe
 from ansible.template import AnsibleUndefined
 from ansible.vars.hostvars import HostVars, HostVarsVars
 from ansible.vars.manager import VarsWithSources
@ -45,10 +45,14 @@ def represent_vault_encrypted_unicode(self, data):


 def represent_unicode(self, data):
+    if _is_unsafe(data):
+        data = data._strip_unsafe()
    return yaml.representer.SafeRepresenter.represent_str(self, text_type(data))


 def represent_binary(self, data):
+    if _is_unsafe(data):
+        data = data._strip_unsafe()
    return yaml.representer.SafeRepresenter.represent_binary(self, binary_type(data))


--- a/lib/ansible/playbook/base.py
+++ b/lib/ansible/playbook/base.py
@ -19,7 +19,7 @@ from ansible import context
 from ansible.errors import AnsibleError, AnsibleParserError, AnsibleUndefinedVariable, AnsibleAssertionError
 from ansible.module_utils.six import string_types
 from ansible.module_utils.parsing.convert_bool import boolean
-from ansible.module_utils.common.text.converters import to_text, to_native
+from ansible.module_utils.common.text.converters import to_text
 from ansible.parsing.dataloader import DataLoader
 from ansible.playbook.attribute import Attribute, FieldAttribute, ConnectionFieldAttribute, NonInheritableFieldAttribute
 from ansible.plugins.loader import module_loader, action_loader
@ -567,14 +567,14 @@ class FieldAttributeBase:
                setattr(self, name, value)
            except (TypeError, ValueError) as e:
                value = getattr(self, name)
-                raise AnsibleParserError("the field '%s' has an invalid value (%s), and could not be converted to %s. "
-                                         "The error was: %s" % (name, value, attribute.isa, e), obj=self.get_ds(), orig_exc=e)
+                raise AnsibleParserError(f"the field '{name}' has an invalid value ({value!r}), and could not be converted to {attribute.isa}.",
+                                         obj=self.get_ds(), orig_exc=e)
            except (AnsibleUndefinedVariable, UndefinedError) as e:
                if templar._fail_on_undefined_errors and name != 'name':
                    if name == 'args':
-                        msg = "The task includes an option with an undefined variable. The error was: %s" % (to_native(e))
+                        msg = "The task includes an option with an undefined variable."
                    else:
-                        msg = "The field '%s' has an invalid value, which includes an undefined variable. The error was: %s" % (name, to_native(e))
+                        msg = f"The field '{name}' has an invalid value, which includes an undefined variable."
                    raise AnsibleParserError(msg, obj=self.get_ds(), orig_exc=e)

        self._finalized = True
--- a/lib/ansible/playbook/conditional.py
+++ b/lib/ansible/playbook/conditional.py
@ -19,7 +19,7 @@ from __future__ import annotations

 import typing as t

-from ansible.errors import AnsibleError, AnsibleUndefinedVariable
+from ansible.errors import AnsibleError, AnsibleUndefinedVariable, AnsibleTemplateError
 from ansible.module_utils.common.text.converters import to_native
 from ansible.playbook.attribute import FieldAttribute
 from ansible.template import Templar
@ -100,14 +100,14 @@ class Conditional:
                    return False

            # If the result of the first-pass template render (to resolve inline templates) is marked unsafe,
-            # explicitly disable lookups on the final pass to prevent evaluation of untrusted content in the
-            # constructed template.
-            disable_lookups = hasattr(conditional, '__UNSAFE__')
+            # explicitly fail since the next templating operation would never evaluate
+            if hasattr(conditional, '__UNSAFE__'):
+                raise AnsibleTemplateError('Conditional is marked as unsafe, and cannot be evaluated.')

            # NOTE The spaces around True and False are intentional to short-circuit literal_eval for
            #      jinja2_native=False and avoid its expensive calls.
            return templar.template(
                "{%% if %s %%} True {%% else %%} False {%% endif %%}" % conditional,
-                disable_lookups=disable_lookups).strip() == "True"
+            ).strip() == "True"
        except AnsibleUndefinedVariable as e:
            raise AnsibleUndefinedVariable("error while evaluating conditional (%s): %s" % (original, e))
--- a/lib/ansible/playbook/play.py
+++ b/lib/ansible/playbook/play.py
@ -31,7 +31,6 @@ from ansible.playbook.helpers import load_list_of_blocks, load_list_of_roles
 from ansible.playbook.role import Role
 from ansible.playbook.task import Task
 from ansible.playbook.taggable import Taggable
-from ansible.vars.manager import preprocess_vars
 from ansible.utils.display import Display

 display = Display()
@ -227,6 +226,9 @@ class Play(Base, Taggable, CollectionSearch):
        return self.roles

    def _load_vars_prompt(self, attr, ds):
+        # avoid circular dep
+        from ansible.vars.manager import preprocess_vars
+
        new_ds = preprocess_vars(ds)
        vars_prompts = []
        if new_ds is not None:
--- a/lib/ansible/playbook/task.py
+++ b/lib/ansible/playbook/task.py
@ -293,6 +293,30 @@ class Task(Base, Conditional, Taggable, CollectionSearch, Notifiable, Delegatabl

        super(Task, self).post_validate(templar)

+    def _post_validate_args(self, attr, value, templar):
+        # smuggle an untemplated copy of the task args for actions that need more control over the templating of their
+        # input (eg, debug's var/msg, assert's "that" conditional expressions)
+        self.untemplated_args = value
+
+        # now recursively template the args dict
+        args = templar.template(value)
+
+        # FIXME: could we just nuke this entirely and/or wrap it up in ModuleArgsParser or something?
+        if '_variable_params' in args:
+            variable_params = args.pop('_variable_params')
+            if isinstance(variable_params, dict):
+                if C.INJECT_FACTS_AS_VARS:
+                    display.warning("Using a variable for a task's 'args' is unsafe in some situations "
+                                    "(see https://docs.ansible.com/ansible/devel/reference_appendices/faq.html#argsplat-unsafe)")
+                variable_params.update(args)
+                args = variable_params
+            else:
+                # if we didn't get a dict, it means there's garbage remaining after k=v parsing, just give up
+                # see https://github.com/ansible/ansible/issues/79862
+                raise AnsibleError(f"invalid or malformed argument: '{variable_params}'")
+
+        return args
+
    def _post_validate_loop(self, attr, value, templar):
        '''
        Override post validation for the loop field, which is templated
--- a/lib/ansible/plugins/action/init.py
+++ b/lib/ansible/plugins/action/init.py
@ -1226,15 +1226,13 @@ class ActionBase(ABC):
                try:
                    _validate_utf8_json(data)
                except UnicodeEncodeError:
-                    # When removing this, also remove the loop and latin-1 from ansible.module_utils.common.text.converters.jsonify
-                    display.deprecated(
+                    raise ValueError(
                        f'Module "{self._task.resolved_action or self._task.action}" returned non UTF-8 data in '
-                        'the JSON response. This will become an error in the future',
-                        version='2.18',
+                        'the JSON response.',
                    )

            data['_ansible_parsed'] = True
-        except ValueError:
+        except ValueError as e:
            # not valid json, lets try to capture error
            data = dict(failed=True, _ansible_parsed=False)
            data['module_stdout'] = res.get('stdout', u'')
@ -1248,7 +1246,7 @@ class ActionBase(ABC):
                data['exception'] = data['module_stdout']

            # The default
-            data['msg'] = "MODULE FAILURE"
+            data['msg'] = f"MODULE FAILURE: {e}"

            # try to figure out if we are missing interpreter
            if self._used_interpreter is not None:
--- a/lib/ansible/plugins/action/assert.py
+++ b/lib/ansible/plugins/action/assert.py
@ -63,8 +63,29 @@ class ActionModule(ActionBase):

        quiet = boolean(self._task.args.get('quiet', False), strict=False)

+        # directly access 'that' via untemplated args from the task so we can intelligently trust embedded
+        # templates and preserve the original inputs/locations for better messaging on assert failures and
+        # errors.
+        # FIXME: even in devel, things like `that: item` don't always work properly (truthy string value
+        # is not really an embedded expression)
+        # we could fix that by doing direct var lookups on the inputs
+        # FIXME: some form of this code should probably be shared between debug, assert, and
+        # Task.post_validate, since they
+        # have a lot of overlapping needs
+        try:
+            thats = self._task.untemplated_args['that']
+        except KeyError:
+            # in the case of "we got our entire args dict from a template", we can just consult the
+            # post-templated dict (the damage has likely already been done for embedded templates anyway)
+            thats = self._task.args['that']
+
+        # FIXME: this is a case where we only want to resolve indirections, NOT recurse containers
+        # (and even then, the leaf-most expression being wrapped is at least suboptimal
+        # (since its expression will be "eaten").
+        if isinstance(thats, str):
+            thats = self._templar.template(thats)
+
        # make sure the 'that' items are a list
-        thats = self._task.args['that']
        if not isinstance(thats, list):
            thats = [thats]

--- a/lib/ansible/plugins/action/copy.py
+++ b/lib/ansible/plugins/action/copy.py
@ -26,7 +26,7 @@ import tempfile
 import traceback

 from ansible import constants as C
-from ansible.errors import AnsibleError, AnsibleFileNotFound
+from ansible.errors import AnsibleError, AnsibleActionFail, AnsibleFileNotFound
 from ansible.module_utils.basic import FILE_COMMON_ARGUMENTS
 from ansible.module_utils.common.text.converters import to_bytes, to_native, to_text
 from ansible.module_utils.parsing.convert_bool import boolean
@ -414,6 +414,11 @@ class ActionModule(ActionBase):
        result = super(ActionModule, self).run(tmp, task_vars)
        del tmp  # tmp no longer has any effect

+        # ensure user is not setting internal parameters
+        for internal in ('_original_basename', '_diff_peek'):
+            if self._task.args.get(internal, None) is not None:
+                raise AnsibleActionFail(f'Invalid parameter specified: "{internal}"')
+
        source = self._task.args.get('src', None)
        content = self._task.args.get('content', None)
        dest = self._task.args.get('dest', None)
--- a/lib/ansible/plugins/action/dnf.py
+++ b/lib/ansible/plugins/action/dnf.py
@ -41,6 +41,13 @@ class ActionModule(ActionBase):
            facts = self._execute_module(
                module_name="ansible.legacy.setup", module_args=dict(filter="ansible_pkg_mgr", gather_subset="!all"),
                task_vars=task_vars)
+
+            if facts.get("failed", False):
+                raise AnsibleActionFail(
+                    f"Failed to fetch ansible_pkg_mgr to determine the dnf action backend: {facts.get('msg')}",
+                    result=facts,
+                )
+
            display.debug("Facts %s" % facts)
            module = facts.get("ansible_facts", {}).get("ansible_pkg_mgr", "auto")
            if (not self._task.delegate_to or self._task.delegate_facts) and module != 'auto':
@ -75,9 +82,4 @@ class ActionModule(ActionBase):
                result.update(self._execute_module(
                    module_name=module, module_args=new_module_args, task_vars=task_vars, wrap_async=self._task.async_val))

-        # Cleanup
-        if not self._task.async_val:
-            # remove a temporary path we created
-            self._remove_tmp_path(self._connection._shell.tmpdir)
-
        return result
--- a/lib/ansible/plugins/action/gather_facts.py
+++ b/lib/ansible/plugins/action/gather_facts.py
@ -8,6 +8,7 @@ import time
 import typing as t

 from ansible import constants as C
+from ansible.errors import AnsibleActionFail
 from ansible.executor.module_common import get_action_args_with_defaults
 from ansible.module_utils.parsing.convert_bool import boolean
 from ansible.plugins.action import ActionBase
@ -61,6 +62,7 @@ class ActionModule(ActionBase):
        return mod_args

    def _combine_task_result(self, result: dict[str, t.Any], task_result: dict[str, t.Any]) -> dict[str, t.Any]:
+        """ builds the final result to return """
        filtered_res = {
            'ansible_facts': task_result.get('ansible_facts', {}),
            'warnings': task_result.get('warnings', []),
@ -70,6 +72,33 @@ class ActionModule(ActionBase):
        # on conflict the last plugin processed wins, but try to do deep merge and append to lists.
        return merge_hash(result, filtered_res, list_merge='append_rp')

+    def _handle_smart(self, modules: list, task_vars: dict[str, t.Any]):
+        """ Updates the module list when 'smart' is used, lookup network os mappings or use setup, warn when things seem inconsistent """
+
+        if 'smart' not in modules:
+            return
+
+        modules.pop(modules.index('smart'))  # remove as this will cause 'module not found' errors
+        network_os = self._task.args.get('network_os', task_vars.get('ansible_network_os', task_vars.get('ansible_facts', {}).get('network_os')))
+
+        if network_os:
+
+            connection_map = C.config.get_config_value('CONNECTION_FACTS_MODULES', variables=task_vars)
+            if network_os in connection_map:
+                modules.append(connection_map[network_os])
+            elif not modules:
+                raise AnsibleActionFail(f"No fact modules available and we could not find a fact module for your network OS ({network_os}), "
+                                        "try setting one via the `FACTS_MODULES` configuration.")
+
+            if set(modules).intersection(set(C._ACTION_SETUP)):
+                # most don't realize how setup works with networking connection plugins (forced_local)
+                self._display.warning("Detected 'setup' module and a network OS is set, the output when running it will reflect 'localhost'"
+                                      " and not the target when a netwoking connection plugin is used.")
+
+        elif not set(modules).difference(set(C._ACTION_SETUP)):
+            # no network OS and setup not in list, add setup by default since 'smart'
+            modules.append('ansible.legacy.setup')
+
    def run(self, tmp: t.Optional[str] = None, task_vars: t.Optional[dict[str, t.Any]] = None) -> dict[str, t.Any]:

        result = super(ActionModule, self).run(tmp, task_vars)
@ -77,13 +106,9 @@ class ActionModule(ActionBase):

        # copy the value with list() so we don't mutate the config
        modules = list(C.config.get_config_value('FACTS_MODULES', variables=task_vars))
+        self._handle_smart(modules, task_vars)

        parallel = task_vars.pop('ansible_facts_parallel', self._task.args.pop('parallel', None))
-        if 'smart' in modules:
-            connection_map = C.config.get_config_value('CONNECTION_FACTS_MODULES', variables=task_vars)
-            network_os = self._task.args.get('network_os', task_vars.get('ansible_network_os', task_vars.get('ansible_facts', {}).get('network_os')))
-            modules.extend([connection_map.get(network_os or self._connection.ansible_name, 'ansible.legacy.setup')])
-            modules.pop(modules.index('smart'))

        failed = {}
        skipped = {}
--- a/lib/ansible/plugins/action/include_vars.py
+++ b/lib/ansible/plugins/action/include_vars.py
@ -237,7 +237,8 @@ class ActionModule(ActionBase):
            b_data, show_content = self._loader._get_file_contents(filename)
            data = to_text(b_data, errors='surrogate_or_strict')

-            self.show_content = show_content
+            self.show_content &= show_content  # mask all results if any file was encrypted
+
            data = self._loader.load(data, file_name=filename, show_content=show_content)
            if not data:
                data = dict()
--- a/lib/ansible/plugins/action/package.py
+++ b/lib/ansible/plugins/action/package.py
@ -68,6 +68,11 @@ class ActionModule(ActionBase):
                            module_args=dict(filter='ansible_pkg_mgr', gather_subset='!all'),
                            task_vars=task_vars,
                        )
+                        if facts.get("failed", False):
+                            raise AnsibleActionFail(
+                                f"Failed to fetch ansible_pkg_mgr to determine the package action backend: {facts.get('msg')}",
+                                result=facts,
+                            )
                        pmgr = 'ansible_pkg_mgr'

                    try:
@ -103,9 +108,5 @@ class ActionModule(ActionBase):

        except AnsibleAction as e:
            result.update(e.result)
-        finally:
-            if not self._task.async_val:
-                # remove a temporary path we created
-                self._remove_tmp_path(self._connection._shell.tmpdir)

        return result
--- a/lib/ansible/plugins/callback/init.py
+++ b/lib/ansible/plugins/callback/init.py
@ -37,7 +37,7 @@ from ansible.parsing.yaml.objects import AnsibleUnicode
 from ansible.plugins import AnsiblePlugin
 from ansible.utils.color import stringc
 from ansible.utils.display import Display
-from ansible.utils.unsafe_proxy import AnsibleUnsafeText, NativeJinjaUnsafeText
+from ansible.utils.unsafe_proxy import AnsibleUnsafeText, NativeJinjaUnsafeText, _is_unsafe
 from ansible.vars.clean import strip_internal_keys, module_response_deepcopy

 import yaml
@ -115,6 +115,8 @@ def _munge_data_for_lossy_yaml(scalar):

 def _pretty_represent_str(self, data):
    """Uses block style for multi-line strings"""
+    if _is_unsafe(data):
+        data = data._strip_unsafe()
    data = text_type(data)
    if _should_use_block(data):
        style = '|'
--- a/lib/ansible/plugins/connection/init.py
+++ b/lib/ansible/plugins/connection/init.py
@ -285,6 +285,27 @@ class ConnectionBase(AnsiblePlugin):
                display.debug('Set connection var {0} to {1}'.format(varname, value))
                variables[varname] = value

+    def _resolve_option_variables(self, variables, templar):
+        """
+        Return a dict of variable -> templated value, for any variables that
+        that match options registered by this plugin.
+        """
+        # create dict of 'templated vars'
+        var_options = {
+            '_extras': {},
+        }
+        for var_name in C.config.get_plugin_vars('connection', self._load_name):
+            if var_name in variables:
+                var_options[var_name] = templar.template(variables[var_name])
+
+        # add extras if plugin supports them
+        if getattr(self, 'allow_extras', False):
+            for var_name in variables:
+                if var_name.startswith(f'ansible_{self.extras_prefix}_') and var_name not in var_options:
+                    var_options['_extras'][var_name] = templar.template(variables[var_name])
+
+        return var_options
+

 class NetworkConnectionBase(ConnectionBase):
    """
--- a/lib/ansible/plugins/connection/ssh.py
+++ b/lib/ansible/plugins/connection/ssh.py
@ -389,7 +389,7 @@ from ansible.errors import (
 from ansible.module_utils.six import PY3, text_type, binary_type
 from ansible.module_utils.common.text.converters import to_bytes, to_native, to_text
 from ansible.plugins.connection import ConnectionBase, BUFSIZE
-from ansible.plugins.shell.powershell import _parse_clixml
+from ansible.plugins.shell.powershell import _replace_stderr_clixml
 from ansible.utils.display import Display
 from ansible.utils.path import unfrackpath, makedirs_safe

@ -1327,8 +1327,8 @@ class Connection(ConnectionBase):
        (returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)

        # When running on Windows, stderr may contain CLIXML encoded output
-        if getattr(self._shell, "_IS_WINDOWS", False) and stderr.startswith(b"#< CLIXML"):
-            stderr = _parse_clixml(stderr)
+        if getattr(self._shell, "_IS_WINDOWS", False):
+            stderr = _replace_stderr_clixml(stderr)

        return (returncode, stdout, stderr)

--- a/lib/ansible/plugins/filter/core.py
+++ b/lib/ansible/plugins/filter/core.py
@ -35,6 +35,7 @@ from ansible.utils.display import Display
 from ansible.utils.encrypt import do_encrypt, PASSLIB_AVAILABLE
 from ansible.utils.hashing import md5s, checksum_s
 from ansible.utils.unicode import unicode_wrap
+from ansible.utils.unsafe_proxy import _is_unsafe
 from ansible.utils.vars import merge_hash

 display = Display()
@ -219,6 +220,8 @@ def from_yaml(data):
        # The ``text_type`` call here strips any custom
        # string wrapper class, so that CSafeLoader can
        # read the data
+        if _is_unsafe(data):
+            data = data._strip_unsafe()
        return yaml_load(text_type(to_text(data, errors='surrogate_or_strict')))
    return data

@ -228,6 +231,8 @@ def from_yaml_all(data):
        # The ``text_type`` call here strips any custom
        # string wrapper class, so that CSafeLoader can
        # read the data
+        if _is_unsafe(data):
+            data = data._strip_unsafe()
        return yaml_load_all(text_type(to_text(data, errors='surrogate_or_strict')))
    return data

--- a/lib/ansible/plugins/filter/regex_search.yml
+++ b/lib/ansible/plugins/filter/regex_search.yml
@ -8,6 +8,9 @@ DOCUMENTATION:
    - Maps to Python's C(re.search).
    - 'The substring matched by the group is accessible via the symbolic group name or
      the ``\{number}`` special sequence. See examples section.'
+    - The return for no match will be C(None) in most cases, depending on whether it is used with other filters/tests or not.
+      It also depends on the Jinja version used and whether native is enabled.
+    - "For a more complete explanation see U(https://docs.ansible.com/ansible-core/devel/reference_appendices/faq.html#why-does-the-regex-search-filter-return-none-instead-of-an-empty-string)."
  positional: _input, _regex
  options:
    _input:
@ -52,5 +55,5 @@ EXAMPLES: |

 RETURN:
  _value:
-    description: Matched string or empty string if no match.
+    description: Matched string or if no match a C(None) or an empty string (see notes)
    type: str
--- a/lib/ansible/plugins/filter/unique.yml
+++ b/lib/ansible/plugins/filter/unique.yml
@ -10,6 +10,13 @@ DOCUMENTATION:
      description: A list.
      type: list
      required: true
+    case_sensitive:
+      description: Whether to consider case when comparing elements.
+      default: false
+      type: bool
+    attribute:
+      description: Filter objects with unique values for this attribute.
+      type: str
  seealso:
    - plugin_type: filter
      plugin: ansible.builtin.difference
@ -24,6 +31,27 @@ EXAMPLES: |
  # list1: [1, 2, 5, 1, 3, 4, 10]
  {{ list1 | unique }}
  # => [1, 2, 5, 3, 4, 10]
+
+  # return case sensitive unique elements
+  {{ ['a', 'A', 'a'] | unique(case_sensitive='true') }}
+  # => ['a', 'A']
+
+  # return case insensitive unique elements
+  {{ ['b', 'B', 'b'] | unique() }}
+  # => ['b']
+
+  # return unique elements of list based on attribute
+  # => [{"age": 12, "name": "a" }, { "age": 14, "name": "b"}]
+  - debug:
+      msg: "{{ sample | unique(attribute='age') }}"
+    vars:
+      sample:
+        - name: a
+          age: 12
+        - name: b
+          age: 14
+        - name: c
+          age: 14
 RETURN:
  _value:
    description: A list with unique elements, also known as a set.
--- a/lib/ansible/plugins/loader.py
+++ b/lib/ansible/plugins/loader.py
@ -34,6 +34,7 @@ from ansible.utils.collection_loader import AnsibleCollectionConfig, AnsibleColl
 from ansible.utils.collection_loader._collection_finder import _AnsibleCollectionFinder, _get_collection_metadata
 from ansible.utils.display import Display
 from ansible.utils.plugin_docs import add_fragments
+from ansible.utils.unsafe_proxy import _is_unsafe

 # TODO: take the packaging dep, or vendor SpecifierSet?

@ -861,6 +862,17 @@ class PluginLoader:

    def get_with_context(self, name, *args, **kwargs):
        ''' instantiates a plugin of the given name using arguments '''
+        if _is_unsafe(name):
+            # Objects constructed using the name wrapped as unsafe remain
+            # (correctly) unsafe. Using such unsafe objects in places
+            # where underlying types (builtin string in this case) are
+            # expected can cause problems.
+            # One such case is importlib.abc.Loader.exec_module failing
+            # with "ValueError: unmarshallable object" because the module
+            # object is created with the __path__ attribute being wrapped
+            # as unsafe which isn't marshallable.
+            # Manually removing the unsafe wrapper prevents such issues.
+            name = name._strip_unsafe()

        found_in_cache = True
        class_only = kwargs.pop('class_only', False)
--- a/lib/ansible/plugins/lookup/first_found.py
+++ b/lib/ansible/plugins/lookup/first_found.py
@ -138,7 +138,6 @@ RETURN = """
    elements: path
 """
 import os
-import re

 from collections.abc import Mapping, Sequence

@ -150,10 +149,22 @@ from ansible.plugins.lookup import LookupBase
 from ansible.utils.path import unfrackpath


+def _splitter(value, chars):
+    chars = set(chars)
+    v = ''
+    for c in value:
+        if c in chars:
+            yield v
+            v = ''
+            continue
+        v += c
+    yield v
+
+
 def _split_on(terms, spliters=','):
    termlist = []
    if isinstance(terms, string_types):
-        termlist = re.split(r'[%s]' % ''.join(map(re.escape, spliters)), terms)
+        termlist = list(_splitter(terms, spliters))
    else:
        # added since options will already listify
        for t in terms:
--- a/lib/ansible/plugins/lookup/varnames.py
+++ b/lib/ansible/plugins/lookup/varnames.py
@ -13,11 +13,14 @@ DOCUMENTATION = """
      _terms:
        description: List of Python regex patterns to search for in variable names.
        required: True
+    seealso:
+        - plugin_type: lookup
+          plugin: ansible.builtin.vars
 """

 EXAMPLES = """
 - name: List variables that start with qz_
-  ansible.builtin.debug: msg="{{ lookup('ansible.builtin.varnames', '^qz_.+')}}"
+  ansible.builtin.debug: msg="{{ lookup('ansible.builtin.varnames', '^qz_.+') }}"
  vars:
    qz_1: hello
    qz_2: world
@ -25,13 +28,16 @@ EXAMPLES = """
    qz_: "I won't show either"

 - name: Show all variables
-  ansible.builtin.debug: msg="{{ lookup('ansible.builtin.varnames', '.+')}}"
+  ansible.builtin.debug: msg="{{ lookup('ansible.builtin.varnames', '.+') }}"

 - name: Show variables with 'hosts' in their names
-  ansible.builtin.debug: msg="{{ lookup('ansible.builtin.varnames', 'hosts')}}"
+  ansible.builtin.debug: msg="{{ q('varnames', 'hosts') }}"

 - name: Find several related variables that end specific way
-  ansible.builtin.debug: msg="{{ lookup('ansible.builtin.varnames', '.+_zone$', '.+_location$') }}"
+  ansible.builtin.debug: msg="{{ query('ansible.builtin.varnames', '.+_zone$', '.+_location$') }}"
+
+- name: display values from variables found via varnames (note "*" is used to dereference the list to a 'list of arguments')
+  debug: msg="{{ lookup('vars', *lookup('varnames', 'ansible_play_.+')) }}"

 """

--- a/lib/ansible/plugins/lookup/vars.py
+++ b/lib/ansible/plugins/lookup/vars.py
@ -17,6 +17,10 @@ DOCUMENTATION = """
        description:
            - What to return if a variable is undefined.
            - If no default is set, it will result in an error if any of the variables is undefined.
+    seealso:
+    - plugin_type: lookup
+      plugin: ansible.builtin.varnames
+
 """

 EXAMPLES = """
@ -27,20 +31,23 @@ EXAMPLES = """
    myvar: ename

 - name: Show default empty since i dont have 'variablnotename'
-  ansible.builtin.debug: msg="{{ lookup('ansible.builtin.vars', 'variabl' + myvar, default='')}}"
+  ansible.builtin.debug: msg="{{ lookup('ansible.builtin.vars', 'variabl' + myvar, default='') }}"
  vars:
    variablename: hello
    myvar: notename

 - name: Produce an error since i dont have 'variablnotename'
-  ansible.builtin.debug: msg="{{ lookup('ansible.builtin.vars', 'variabl' + myvar)}}"
+  ansible.builtin.debug: msg="{{ q('vars', 'variabl' + myvar) }}"
  ignore_errors: True
  vars:
    variablename: hello
    myvar: notename

 - name: find several related variables
-  ansible.builtin.debug: msg="{{ lookup('ansible.builtin.vars', 'ansible_play_hosts', 'ansible_play_batch', 'ansible_play_hosts_all') }}"
+  ansible.builtin.debug: msg="{{ query('ansible.builtin.vars', 'ansible_play_hosts', 'ansible_play_batch', 'ansible_play_hosts_all') }}"
+
+- name: show values from variables found via varnames (note "*" is used to dereference the list to a 'list of arguments')
+  debug: msg="{{ q('vars', *q('varnames', 'ansible_play_.+')) }}"

 - name: Access nested variables
  ansible.builtin.debug: msg="{{ lookup('ansible.builtin.vars', 'variabl' + myvar).sub_var }}"
--- a/lib/ansible/plugins/shell/powershell.py
+++ b/lib/ansible/plugins/shell/powershell.py
@ -26,13 +26,85 @@ from ansible.module_utils.common.text.converters import to_bytes, to_text
 from ansible.plugins.shell import ShellBase

 # This is weird, we are matching on byte sequences that match the utf-16-be
-# matches for '_x(a-fA-F0-9){4}_'. The \x00 and {8} will match the hex sequence
-# when it is encoded as utf-16-be.
-_STRING_DESERIAL_FIND = re.compile(rb"\x00_\x00x([\x00(a-fA-F0-9)]{8})\x00_")
+# matches for '_x(a-fA-F0-9){4}_'. The \x00 and {4} will match the hex sequence
+# when it is encoded as utf-16-be byte sequence.
+_STRING_DESERIAL_FIND = re.compile(rb"\x00_\x00x((?:\x00[a-fA-F0-9]){4})\x00_")

 _common_args = ['PowerShell', '-NoProfile', '-NonInteractive', '-ExecutionPolicy', 'Unrestricted']


+def _replace_stderr_clixml(stderr: bytes) -> bytes:
+    """Replace CLIXML with stderr data.
+
+    Tries to replace an embedded CLIXML string with the actual stderr data. If
+    it fails to parse the CLIXML data, it will return the original data. This
+    will replace any line inside the stderr string that contains a valid CLIXML
+    sequence.
+
+    :param bytes stderr: The stderr to try and decode.
+
+    :returns: The stderr with the decoded CLIXML data or the original data.
+    """
+    clixml_header = b"#< CLIXML\r\n"
+
+    if stderr.find(clixml_header) == -1:
+        return stderr
+
+    lines: list[bytes] = []
+    is_clixml = False
+    for line in stderr.splitlines(True):
+        if is_clixml:
+            is_clixml = False
+
+            # If the line does not contain the closing CLIXML tag, we just
+            # add the found header line and this line without trying to parse.
+            end_idx = line.find(b"</Objs>")
+            if end_idx == -1:
+                lines.append(clixml_header)
+                lines.append(line)
+                continue
+
+            clixml = line[: end_idx + 7]
+            remaining = line[end_idx + 7 :]
+
+            # While we expect the stderr to be UTF-8 encoded, we fallback to
+            # the most common "ANSI" codepage used by Windows cp437 if it is
+            # not valid UTF-8.
+            try:
+                clixml.decode("utf-8")
+            except UnicodeDecodeError:
+                # cp427 can decode any sequence and once we have the string, we
+                # can encode any cp427 chars to UTF-8.
+                clixml_text = clixml.decode("cp437")
+                clixml = clixml_text.encode("utf-8")
+
+            try:
+                decoded_clixml = _parse_clixml(clixml)
+                lines.append(decoded_clixml)
+                if remaining:
+                    lines.append(remaining)
+
+            except Exception:
+                # Any errors and we just add the original CLIXML header and
+                # line back in.
+                lines.append(clixml_header)
+                lines.append(line)
+
+        elif line == clixml_header:
+            # The next line should contain the full CLIXML data.
+            is_clixml = True
+
+        else:
+            lines.append(line)
+
+    # This should never happen but if there was a CLIXML header without a newline
+    # following it, we need to add it back.
+    if is_clixml:
+        lines.append(clixml_header)
+
+    return b"".join(lines)
+
+
 def _parse_clixml(data: bytes, stream: str = "Error") -> bytes:
    """
    Takes a byte string like '#< CLIXML\r\n<Objs...' and extracts the stream
--- a/Show More
+++ b/Show More