* doc - Dynamically document jinja builtins
This change has `ansible-doc` dynamically generate the documentation for
any Jinja builtin filter and test plugins. These dynamic stubs will
point to the official Jinja documentation pages for more information.
* Fix sanity issues
* Add tests
* Update Jinja builtin doc gen
Co-authored-by: Matt Clay <matt@mystile.com>
---------
Co-authored-by: Matt Davis <nitzmahone@redhat.com>
Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit 8f2622c39f)
* Clean up TE error handling, wrap sigalrm handler
* Preserve error detail on AnsibleAction and Connection exceptions.
* Remove multiple layers of unreachable or redundant error handling.
* Wrap manual alarm signal/timeout handling into a context manager, add tests.
Co-authored-by: Matt Clay <matt@mystile.com>
* update error message check in test
* update test timeout message assertions
---------
Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit cbcefc53a3)
async_status and async_wrrapper now use booleans instead of 0/1 for 'thruthyness'
gather_facts also updated
Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit d41a3430b7)
Fixes the logic when running a module through App Control when the
module is not trusted to run in Full Language Mode. This ensures
coverage will still run as expected and that the trust verification only
happens in the wrappers that actually run/prepare the code.
Also expands on a comment to clarify why only that branch is set to set
the internal file encoding to UTF-8.
(cherry picked from commit 17cee7a982)
* Add support for Windows App Control/WDAC
Adds preview support for Windows App Control, formerly known as WDAC.
This is a tech preview feature and is designed to test out improvements
needed in future versions of Ansible.
* Use psd1 and parse it through the Ast to avoid any unexpected execution results
* Add tests for various manifest permutations
* Ignore test shebang failure
* Apply suggestions from code review
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
* Use more flexible test expectations
* Add type annotations for shell functions
---------
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
(cherry picked from commit 75f7b2267d)
* ansible-test - Improve deprecated checking type inference
Also disabled the ``bad-super-call`` pylint rule due to false positives.
* Add type comment support
* Try without using register_transform
(cherry picked from commit e82be177cd)
For calling setup_test_user, which adds a user to the system (and thus requiring
root):
- become
- become_su
- become_sudo
- slurp
And also:
- apt, which installs/removes system packages
- deb822_repository, which adds/removes apt repos
- hardware_facts calls losetup, which requires root
- package_facts, which installs/removes system packages
Co-authored-by: Lee Garrett <lgarrett@rocketjump.eu>
(cherry picked from commit ebd022ee42)
Fix the logic for a constructable inventory `keyed_groups` entry to use
the `default_value` if the ``key` expression result is `None` or `omit`
and not just an empty string. This bug was introduced with the changes
in data tagging and goes back to the original behaviour.
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
(cherry picked from commit fe2d9e316a)
* remove dead config comment noise
* update `list` typed config defaults to be lists
* fix tag preservation/propagation in config
* numerous other ensure_type bugfixes
* 100% unit test coverage of ensure_type
* emit warnings on template_default failures
* fix unhandled exception in convert_bool on unhashable inputs
Co-authored-by: Matt Clay <matt@mystile.com>
---------
Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit d33bedc48f)
* See changelog fragment for most changes.
* Defer early config warnings until display is functioning, eliminating related fallback display logic.
* Added more type annotations and docstrings.
* ansible-test - pylint sanity for deprecations improved.
* Refactored inline legacy resolutions in PluginLoader.
Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit ff6998f2b9)
* Implement TaskResult backward compatibility for callbacks
* general API cleanup
* misc deprecations
Co-authored-by: Matt Clay <matt@mystile.com>
* fix v2_on_any deprecation exclusion for base
---------
Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit 03181ac87b)
* Fix test_range_templating on 32-bit architectures
32-bit archtectures like i386, armel, armhf will fail with the error
ansible._internal._templating._errors.AnsibleTemplatePluginRuntimeError: The
filter plugin 'ansible.builtin.random' failed: Python int too large to convert
to C ssize_t
So just pick sys.maxsize (2**31 - 1) so it works on 32 bit machines.
---------
Co-authored-by: Lee Garrett <lgarrett@rocketjump.eu>
Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit 5f6aef95ac)
* Use `_UNSET` instead of allowing `ellipsis`
* Fix deprecation warning pre-check
* Deprecation warnings from modules can now be disabled.
* Deprecation warnings from modules get the "can be disabled" notice.
* Include help text in pre-display fatal errors
* Simplify lookup warning/debug messaging
* Fix return type of `timedout` test plugin
* Use `object` for `_UNSET`
* Remove obsolete `convert_data` tests
* Remove unnecessary template from test
* Improve legacy YAML objects backward compat
* Fix templar backward compat for None overrides
(cherry picked from commit 6cc97447aa)
* Added support for URL-safe decoding into b64decode
* Added support for URL-safe encoding into b64encode
Fixes: #84147
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
* Add ssh-agent launching, and ssh-agent python client
* Move things around, is this better??
* docs
* postpone creating dir after bin lookup
* fix method name
* changelog ssh agent
* address reviews
* fix typing
* do not redefine public_key
* typing
* more typing
* Catch OSError when starting ssh agent
* likely copy pasted old code
* var type fix
* why is this needed?
ci_complete
* ignoring the change for now
* write out pub key file atomically
* defensive timeout for the socket
* _populate_agent docstring
* do not allow setting these in config
* check expected length before slicing blobs
* test all key types
* remove lock/unlock functionality
* docstring
* private _ssh_agent
* .
* launch agent in cli and ansible_ssh_*
* additional info for ssh-agent comment
* Add tests for remove and remove_all
* comment on os.rename
* hopefully mitigate agent startup/delays problems
* exceptions
* unused import
* fix sanity
* perf
---------
Co-authored-by: Matt Martz <matt@sivel.net>
* Provide warning to user about the potential risk involved in
embedding username and password in git URL while checking out the source.
Fixes: #84557
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
* Use shallow copy of os.environ to update PYTHONPATH value
instead of using '|' operator
Fixes: #84954
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
Moved check to connection as it should be the final decider
Added property to become plugins to indicate support
Also removed hardcoded su exception
Added tty detection logic for ssh (pipelining won't work if tty is needed or forced)
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* Git allows embedding username and password in repo URL for
https authentication. This may lead to exposing the user sensitive
information to logs and unautheticated users.
Adding no_log will partially solve this.
* Added documentation warning user about URL embedded with username
and password.
* Added logic to remember user sensitive information for later sanitization
Fixes: #84557
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
* Clean up interpreter discovery
- Deprecated `auto_legacy` and `auto_legacy_silent`
- Removed obsolete platform fallback config and logic
- Replaced unit tests with integration tests
- Increased test coverage
Fixes the coverage collection for Windows and async tasks. This ensures
the async task still has access to the PSHost so that it can access the
runspace debugger tasks on the host.
* Support download_url without a scheme:host. Fixes#84213
* Support properly rebuilding pagination links. #84765
* Don't permit non absolute pagination links
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
Updates the Windows exec runner in preparation for the WDAC changes.
This new process is designed to improve the way modules are run by
Windows and expose common functionality to run PowerShell code in a
common environment. It also includes futher changes to improve the error
handling to make it easier to see where an error occurred in the running
code.
* Fix apt and dpkg_selections tests
The tests now support aarch64.
They also have reduced dependencies on URLs hosted on third-party servers.
* Skip tests on unsupported platforms
* Use same package version for both tests
* Fix multi-arch support for Ubuntu 22.04
* Improve performance of including tasks into the play
PlayIterator.add_tasks is used to insert tasks from an include into the
play for particular host. It makes a copy of the current block including
the tasks within the block and inserts the new tasks from the include
into the copied block. But there is no need to make copies of tasks
within the block, what we want is a "shallow" copy of the block. This
PR changes that to copy the block excluding the tasks within.
On a contrived playbook with 50 include_role tasks, each role has 1 task,
running on 10 hosts the running time is reduced from ~55s to ~44s in my
environment.
ci_complete
* Add changelog
* skip 'no such file or directory' error code while files and
directories and report them.
Fixes: #84873
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
* uri: form location correctly from relative redirect
Previously, the original URL would be combined with the relative location incorrectly, especially for URL of any complexity.
Add simple tests demonstrating the problem that fail without the fix
* fix pylint error, import the method similar to other uri methods
* add changelog fragment
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
Refactor the async wrapper and watchdog scripts for Windows. This
attempts to avoid WMI on connection plugins that allow breaking away
from a job like winrm and ssh as an optimisation and changes how WMI is
used so that we can get the error details on a failed process more
easily.
These changes are being made also in preparation for the WDAC
implementation that requires this new execution model where input needs
to be provided through stdin.
This deletes the fallback that would use `--venv` when `--docker` is
unavailable in `ansible-test`. But this option is present in all
supported versions of Ansible as it was added in v2.12 which is EOL
already.
PR #84819
* do not mention subprocess.Popen, it is an implementation detail
* use "shell injection vulnerability" to describe the problem that is
well-known, or easily searched for on $search_engine and do not link
to a 3rd-party site that is also specific to a tool for Python
This patch marks the `ansible.module_utils.compat.datetime` module as deprecated, including `UTC`, `utcfromtimestamp()` and `utcnow` shims that it provides, scheduling its removal for v2.21.
It also replaces any uses of the compatibility helpers with non-deprecated calls to CPython stdlib.
PR #81874
Co-authored-by: Matt Clay <matt@mystile.com>
Prior to this change, when the dependency resolver started looping over multiple versions of the same collection due to backtracking, it might take a lot of time to consider and disregard tens or hundreds of versions. But to the end-user, it looks like “nothing is happening, the program is *stuck*”. Even worse if such a time-consuming backtracking hits multiple collections and it “hangs” for longer cumulative period of time.
This patch improves the perceived responsiveness by printing out informational messages with the current status whenever the backtracking for a collection happens for the first, the eighth and the thirteenth times. The last message also reminds them that they can interrupt the process and attempt to adjust the constraints.
In debug mode, it also shows what caused conflicts leading up to candidate rejections. But this is only available with `resolvelib >= 0.9.0`.
The improvement is heavily inspired by https://github.com/pypa/pip/commit/9731131.
PR #81709
Co-Authored-By: Jordan Borean <jborean93@gmail.com>
Prevents `Group state for \"customenvgroup\" not found` error which may
or may not be a regression in dnf5. Just name groups/envs uniquely to
workaround the issue.
Removes the use of pexpect in the winrm connection plugin and rely on
just subprocess. In the past pexpect was used for macOS compatibility so
that it could handle the TTY prompt but after testing it seems like
subprocess with `start_new_session=True` is enough to get it reading
from stdin on all platforms. This simplifies the code as there's no
longer an optional library changing how things are called and will work
out of the box.
Fix up bug when attempting to run any module on a Windows host that has
been configured with WDAC and Dynamic Code Security in audit mode. This
does not enable WDAC support with signed scripts so Ansible will still
not pass the audit events but it no longer fails to run.
* Simplify conditional for service_facts integration test
This test requires systemd, so just test for that. Also fixes running this on
Debian testing/unstable releases, where ansible_distribution_version is "n/a".
* Clean up after service_facts integration test
* Simplify set_fact in interpreter_discovery_python test
Those vars are always set, either to something discovered, or "n/a". There are
no evaluations against the value "unknown".
* interpreter_discovery_python: Fix Debian test
Debian doesn't set VERSION_ID in /etc/os-release on pre-releases, see
https://bugs.debian.org/1008735
ansible 2.17 onwards does not support python 2.7 anyway.
* fix deb822_repository integration test failure on Debian
* Fix integration test hostname for Debian
* Fix integration test mount_facts
Some VMs might only have a single mount point, so they only have a single UUID.
* Add package deps for integration test subversion
/usr/bin/htpasswd is shipped in apache2-utils and needed by the main playbook.
* Fix integration test "service" on Debian sid
Debian sid does not set ansible_distribution_version, as such any tests assuming
it's a numeric value will fail. Since this is ancient test code that predates
ansible_service_mgr, remove the error-prone heuristic and rely on that var
instead.
* Fix service integration tests not running on *BSD
---------
Co-authored-by: Lee Garrett <lgarrett@rocketjump.eu>
* Ensure implicit flush_handlers have a parent block
To avoid getting tracebacks when calling methods like ``get_play()`` on
them.
* task needs to be copied
* copy only when necessary
No the file name that caused the error will be apparent
---------
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
Co-authored-by: Abhijeet Kasurde <Akasurde@redhat.com>
* Fixed various become-related issues in `local` connection plugin.
* Fixed various issues in `sudo` and `su` become plugins.
* Added unit and integration test coverage.
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Davis <nitzmahone@redhat.com>
* Link to amazon.aws.aws_ec2 inventory plugin.
* Mention that refresh_inventory does not update the selected hosts for a play.
* Address comments by bcoca.
* Apply suggestions from code review
---------
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
* test: enable user test for alpine
* Disable user home update tests
* Disable some more tests which are not applicable for Alpine
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Previously, requirement version specs starting with `!=` were
incorrectly considered as pinned release requests because the
comparison was being made against a one-char string while the
operator is two-char. This patch changes the check to test against `!`
which is enough to detect this case.
Previously, the integration test depended on luck. `setuptools` used
to be bundled in Python stdlib's `ensurepip`. Python 3.12 and newer no
longer include it. This test imports `pkg_resources` that is a part of
`setuptools`, meaning that it'll run out of luck at some point, under
newer Python runtimes.
This test does not seem to be useful to us so instead of fixing it, we
remove it from the repo [[1]].
Supersedes #84681.
[1]: https://github.com/ansible/ansible/pull/84681#discussion_r1945525317
* rc-status commands returns unwanted lines with service names
and their status. Skip such lines while parsing service names
Fixes: #84512
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* apt-key module updates due to debian removal
Still kept for now for backwards compat, but removing from testing when not present
And adding more explicit mesasges to errors to point to new module
* added docs and error msg
* clog
* aslkdfj
* no docs to document doc changes
* also add warning to apt_repository
* clog on apt_repo too
* fix string concat
* Apply suggestions from code review
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
---------
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Improves the logic for parsing CLIXML values in the stderr returned by
SSH. This fixes encoding problems by having a fallback in case the
output is not valid UTF-8. It also can now extract embedded CLIXML
sequences in all of stderr rather than just at the start.
When a user doesn't exist and user module is used to create the user and the
homedir, adduser is called which parses HOME_MODE from /etc/login.defs, and when
not set calculates the mode from UMASK from the same file.
When a user already exists without homedir, and the user module is used to add a
home dir, it incorrectly ignores HOME_MODE, resulting in a world-readable home
dir when UMASK is not set. This is for example the case in Debian trixie and
later, and likely Ubuntu 25.04 and later.
Signed-off-by: Lee Garrett <lgarrett@rocketjump.eu>
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Correct the test that expects an error when using ansible-vault to write against
a non-writeable dir. Skip the test as root, as root can always write.
Co-authored-by: Lee Garrett <lgarrett@rocketjump.eu>
* fix[doc.py]: path will be undefined and a direct reference will throw an UnboundLocalError.
If none of the files in files exists, path will be undefined and a direct reference will throw an UnboundLocalError.
Repair function file parameter type annotation is inaccurate
* Update changelogs/fragments/fix-cli-doc-path_undefined.yaml
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
---------
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
It is rare that we need to provide information like "before and after
the change". In majority of cases we just ignore "Additional
information" when submitting PRs. We can just put the needed information
into the summary if needed and remove the section from the template,
simplifying it.
Fixes:
Traceback (most recent call last):
File "/home/shertel/ansible/lib/ansible/plugins/vars/host_group_vars.py", line 139, in get_vars
FOUND[key] = found_files = loader.find_vars_files(opath, entity_name)
^^^
UnboundLocalError: cannot access local variable 'key' where it is not associated with a value
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/shertel/ansible/lib/ansible/plugins/vars/host_group_vars.py", line 151, in get_vars
raise AnsibleParserError(to_native(e))
ansible.errors.AnsibleParserError: cannot access local variable 'key' where it is not associated with a value
Also remove redundant check from tqm
Now covers module output (set_fact/include_vars)
Includes play objects at any stage (tasks that error were not covered)
Added tests, moved them to role structure
Some versions and architectures come with `ca_root_nss` pre-installed.
However, at least FreeBSD 13.4 on aarch64 does not.
This change ensures the certificates will always be installed.
* ansible-test - Replace Fedora 40 with 41
Also update other distro containers to remove unnecessary packages:
* apache2
* subversion
* ruby
* Fix subversion test on Ubuntu
Also remove obsolete vars files from subversion test.
* Skip dnf test on Fedora 41+
The dnf5 test is all that is needed.
* Support dnf5 for the package test
* Extend unix-chkpwd work-around for Fedora
The test has been updated to use a custom type which does not support pickling,
instead of relying on Jinja's `Undefined` type. As of Jinja 3.1.5 that type now
supports pickle, which breaks the original implementation of the test.
* Added docstrings to V2 methods in the CallbackBase Class (4 & 5 of 27)
* Made corrections as requested by webknjaz.
* Cleaned up whitespace issues.
* Corrections to customization note for review by webknjaz.
* Added rtype to return in docstrings.
* Simplified docstrings.
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Sviatoslav Sydorenko <webknjaz@redhat.com>
gather_facts, fix network_os and smart logic and defaults
setup will be default for smart only if network_os is not set, now you get warnings and errors when missing a valid facts module for a network os
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
* simplify redundancy with AnsibleModule set_*_if_different methods
* simplify copying a source directory to a dest directory without modifying behavior
* Allow user to control the trimming of blocks while templating
* Added tests
* Updated documentation and example
Fixes: #75962
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Clamp zip timestamps to representible values when unpacking zip files on
platforms that use 32-bit time_t (e.g. Debian i386). This is a
non-issue in practice (in 2024), but should allow the test suite to pass
on Debian i386.
We use a round value of 2038-01-01 00:00:00 for simplicity, and to avoid
running into timezone offsets closer to the actual limit.
MR #81520 introduced sanity-checking tests that used dates not
representable with a 32-bit time_t.
* ssh: Test reset_connection with templated ansible_ssh_executable
Add failing test to confirm subsequent fixes are necessary & sufficient.
* ssh: Fix reset_connection with templated ansible_ssh_executable
Signed-off-by: Alex Willmer <alex@moreati.org.uk>
Open the double quotes at the beginning of the entire `failed_when` value.
Individual conditions work well, yet fail each time they are combined with a logic `or` (as per the example).
Double quoting the entire string solved the problem.
This prevents "[WARNING]: Reset is not implemented for this connection" when
using wait_for_connection with localhost or other local hosts.
It's arguable (from a consistency/correctness perspective) that
`ansible.plugins.connection.local.Connection.reset()` should call
`Connection.close()`. I went for a no-op on the basis of "if it aint broke
don't fix it", and erred on the side of keeping existing semantics. However
either option would be fine with me.
* Fix callback_default integration test for non-root users
This test compares the test output to previously collected output. However, this
previously assumed that the tests are run as root, even though needs/root in
aliases is not set.
So instead parameterize the output, and use sed to replace the value that
diverge when running it as different users.
* integration tests: Ensure temp file removal doesn't fail as non-root user
* Don't fail when ansible-galaxy integration test runs as non-root
* Update integration test callback list
---------
Co-authored-by: Lee Garrett <lgarrett@rocketjump.eu>
Fedora 41 no longer has python3-rpm installed by default either,
so package_facts blows up on Fedora 41 hosts unless you make sure
python3-rpm is installed first. Not sure we can do a lot about
this besides extending this note.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Fixes the coverage path translation for modules located in integration
test paths. Instead of trying to match by the unique temporary path name
that the module is executed as, the reporting tool will translate it to
the static path that the module is actually located under.
* integrity tests: Tag (destructive) root tests as such
- apt_key needs root to touch the apt key database
- debconf needs root to change debconf values of system packages
- gathering writes to /etc/ansible/*, writeable only to root
- group creates system groups
- noexec mounts/umounts a ramdisk
- systemd requires root to start/stop services
Mark all except noexec as "destructive" as they change the state of the system.
* integration test cron requires root, as it calls setup_cron
* integration test dpkg_selection runs dpkg as root
* integration test facts_linux_network requires root
It adds/removes IP addresses from network interfaces, requiring root for that.
* integration test package requires root
installs/removes system packages
* Integration test service requires root
Creates/starts/stops/removes systemd services
* integration test user requires root to create users
* integration tests using setup_test_user require root
---------
Co-authored-by: Lee Garrett <lgarrett@rocketjump.eu>
When detection of the current container network fails, a warning is now issued and execution continues.
This simplifies usage in cases where the current container cannot be inspected, such as when running in GitHub Codespaces.
Fixes the logic when attempting to become the SYSTEM user using the
runas plugin. It was incorrectly assumed that calling LogonUser with the
SYSTEM username would produce a new token with all the privileges but
instead it creates a copy of the existing token. This reverts the logic
back to the original process and adds in new logic to avoid any tokens
that are restricted from creating new processes.
though the previous docs were the 'intended' outcome,
the current behaviour has been active for a long time and should
not be removed due to backward compatibility issues.
One thing we can do going forward is deprecate substitution enabled by default.
* Update resolvelib upperbound to 2.0.0
Minor SemVer releases should not contain breaking changes
* Test the latest minor release and reduce number of resolvelib versions tested for efficiency
* Update the documentation for check_required_by
* Fix return value for check_required_by (now returns empty list on success)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Improve error handling for create-bulk-issues.py
* Add support for setting assignee
* Add example YAML to feature --help output.
* Add additional help message for token issues.
Several tasks within the modification_time.yml and state_link.yml task
lists have explicitly enabled diff_mode because these tests previously
assumed a diff attribute would always be returned from the file module.
While showing the deprecation message, mention the collection name
from which the module is removed.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* fixes for CVE-2024-8775
* propagate truthy `_ansible_no_log` in action result (previously superseded by task-calculated value)
* always mask entire `include_vars` action result if any file loaded had a false `show_content` flag (previously used only the flag value from the last file loaded)
* update no_log tests for CVE-2024-8775
* include validation of _ansible_no_log preservation when set by actions
* replace static values with dynamic for increased robustness to logging/display/callback changes (but still using grep counts :( )
* changelog
* use ternary, coerce to bool explicitly
variable_manager unit tests are unreachable. These tests
are already covered in integration tests.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
In ``Handler.notify_host`` we ensure that ``Handler.notified_hosts`` can contain
particular host at most once. Therefore for removing a host it should be
faster to use ``list.remove`` which removes the first item in the list,
as opposed to using list comprehension removing all such items.
* Update triple single quotes to triple double quotes
This change was fully automated.
The updated Python files have been verified to tokenize the same as the originals, except for the expected change in quoting of strings, which were verified through literal_eval.
* Manual conversion of docstring quotes
This cleans up the implementation of the pylint sanity test and enables the docstyle extension rule `bad-docstring-quotes` for tests.
The rule will be enabled for the rest of ansible-core once automated cleanup has been performed on existing docstrings.
* Added support for testing unit tests with mypy.
* Added support for ignoring individual mypy error codes.
* Added missing assert on unit tests and marked xfail.
* Added type hints for some unit tests.
* Added ignores for unit tests not passing mypy.
* Fixed incorrect autouse argument in unit test fixtures.
* Fixed minor issues causing problems with mypy in unit tests.
* Fix installing collections|roles from git repos with GALAXY_IGNORE_CERTS
* Fix installing collections from git repos with --ignore-certs
* Update unit test
* Add test case
This greatly reduces run time on large inventories since meta tasks are
executed in the main process sequentially and just executing them is expensive.
This change avoids running the following implicit meta tasks:
* ``flush_handlers`` on hosts where no handlers are notified
* ``noop`` for the linear strategy's lockstep, instead hosts that are
not executing the current task are just not part of the current host loop
A playbook consiting of two simple plays both running on ~6000 hosts
runs in:
devel: 37s
this PR: 1.3s
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <wk.cvs.github@sydorenko.org.ua>
Since we don't really care about the type we don't have to
query for it and just retrieve the filename value.
Unfortunately we cannot use module_utils.urls.get_response_filename
as we don't have the response object, so just utilize
email.message.Message to parse the filename
instead of manually doing the work ourselves.
Fixes: #83690
* INVENTORY_IGNORE_EXTS stop ignoring ini
Originally added to avoid configuration files for inventory scripts
now clashes with the much more common ini inventory files.
* add a note to script plugin
* Update lib/ansible/plugins/inventory/script.py
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
* facts: Skip path if the distribution path is directory
Skip path if the distribution path is directory instead of file.
Handle exception raised while handling distribution path.
Fixes: #84006
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Review requests
---------
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
The following Python output proves that the second member of the resulting list should be " a", not "a":
```txt
$ python3
Python 3.12.3 (main, Sep 11 2024, 14:17:37) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> string = "jojo is, a"
>>> string.split(',')
['jojo is', ' a']
```
* Add a mount_facts module capable of gathering mounts skipped by default
fact gathering
* By default, collect mount facts from standard locations including
/etc/mtab, /proc/mounts, /etc/fstab, /etc/mnttab, /etc/vfstab, and on AIX,
/etc/filesystems.
When no file-based source for the current mounts can be found
(like /proc/mounts), the module falls back to using mount as a source.
This allows BSD and AIX to collect the existing mounts by default, without
causing Linux hosts to use both /proc/mounts and mount output.
* Non-standard locations and "mount" can be configured as a sources.
* Support returning an aggregate list of mount points in addition to first
found.
When there are multiple mounts for the same mount point in an
individual source, a warning is given if the include_aggregate_mounts
option is not configured.
* Add options to filter on fstypes and devices (supporting UNIX shell
wildcards).
* Support configuring a timeout and timeout behavior to make it easier
to use the module as a default facts module without risking a hang.
* Include the source and line(s) corresponding to a mount for easier
debugging.
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
* add descriptions for `ansible-galaxy install` and `ansible-galaxy role|collection install`
* fix the usage for installing roles and collections together and include collections in the description for -r
Closes#81159
Co-authored-by: Alan Rominger <arominge@redhat.com>
Co-authored-by: Sandra McCann <samccann@redhat.com>
* update communication details in README
* update comms details in contributing
* update comms details in issue templates
* add link to contributors guide
* point to devel for comms
Instead of re-using the token used in impersonation, this change will
create a new token for the SYSTEM account as returned by LogonUser. The
benefits of this is that the token will contain the full privileges for
the SYSTEM account rather than potentially one that has restricted
privileges we used during impersonation. It should also help avoid
problems on Windows that fails on status 0x0000016F when the
impersonated token during become was from a process that is restricted
from creating sub processes.
* Fact gathering fix 'no shm' branhc
* Use concurrent.futures instead of multiprocessing
This entirely avoids the need for fallback logic since the concurrent.futures thread pool does not depend on `/dev/shm`.
Co-authored-by: Matt Clay <matt@mystile.com>
Use the changelog sanity test requirements instead of the package-data sanity test requirements.
This enables removal of most package-data sanity test requirements, as they are no longer used by the test itself.
The additional requirements were being maintained only to provide pinned requirements for building the changelog during a release.
It's been discouraged for the past decade. And CPython actually ships
with pip nowadays, that is bundled within the built-in `ensurepip`
stdlib module.
* add a loop_control break_when directive to break out of a loop after any item
* remove loop var as normal exit would
* example usage:
- name: generate a random password up to 10 times, until it matches the policy
set_fact:
password: "{{ lookup('password', '/dev/null', chars=character_set, length=length) }}"
loop: "{{ range(0, 10) }}"
loop_control:
break_when:
- password is match(password_policy)
Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
* release.py - Add missing setuptools arg to prepare
This allows the prepare command to accept the `--no-setuptools` argument.
It also fixes a traceback when using the `prepare` command.
* Use a more accurate type hint
* Add DaemonThreadPoolExecutor impl
* Provide a simple parallel execution method with the ability to abandon timed-out operations that won't block threadpool/process shutdown, and without a dependency on /dev/shm (as multiprocessing Thread/Process pools have).
* Create module_utils/_internal to ensure that this is clearly not supported for public consumption.
* psrp - Remove extras lookups
Removed the extras variable lookups for the psrp connection plugin. All
valid options are already documented and the extras functionality is
slated to be deprecated at a future point in time. This should have
affect on existing user's playbooks.
* Fix up sanity tests and add explicit boolean conversion test
Adds the datastore details to the parser error when attempting to
include tasks that contain include_tasks without a filename set. This
change will now display the exact location of the include_tasks that
failed like any normal syntax error.
Expands the test matrix used for testing on Windows to cover the three
connection plugins we support for all the tasks. This change also
changes how raw commands are run over SSH to avoid starting a
`powershell.exe` process that was uneeded in the majority of cases used
in Ansible. This simplifies our code a bit more by removing extra
Windows specific actions in the ssh plugin and improves the efficiency
when running tasks.
* Rely on dnf.base.remove, no special handling isn't needed,
let the dnf internals figure out what is needed to be done.
This is more in line with what dnf cli does.
* "already installed" in Exception (if it is even a thing) should be
caught by special exceptions like MarkingError or CompsError. This
appears to be a historic check that is no longer needed.
Supersedes: #83295
* Fix up raw_params for ansible.windows modules
Fixes up the logic for detecting if using ansible.windows.win_command or
ansible.windows.win_shell with _raw_params. These two modules are
special in that they can be referenced in 4 different ways but the
ansible.windows collection specific prefix needs to be manually added to
the list.
* Fix up sanity issue
Currently we match the load name, which can be an fqcn, but most users expect the 'naked' name
Now plugins can declare that name by setting _extras_prefix property or fallback to 'non fqcn' if no extras prefix
* ssh and psrp - Support more complex chars in fetch_file
Fixes the psrp and ssh (with piped) fetch function to work with paths
that contains glob like characters in the path. For Windows this was
needed when using paths that contain `[]` in the path. For ssh this was
a problem with FreeBSD when using the piped transfer method with similar
characters.
Also tidies up the psrp logic to not inject the paths and buffer size
in the script but pass it as an object through an argument/parameter.
* Fix sanity check
* fix creating file in directory with setgid bit
* add a test using the copy module's content option to create a file in a directory with setgid bit
Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
* Enable validation of subkeys in rpm key module
A gpg subkey may change while the primary key remains the same. Due to
this behavior, there are situations where validation of the primary gpg
key fingerprint is not sufficient because the desired target is actually
the gpg subkey. This change allows the user to validate against either
the fingerprint of the primary gpg key or its subkey.
Signed-off-by: Kellin <kellin@retromud.org>
* Improve tests, add multi-fingerprint
- Improve tests to cover all cases
- add multi fingerprint validation
Signed-off-by: Kellin <kellin@retromud.org>
* package-data - Test min/max setuptools version
* Fix multi-version abstraction
* Convert mypy test to script based test
* Fix f-string in pymarkdown test
* Sanity test fixes
add caplevel to display to pass through
also reverse dict order as 'last update wins'
added tests ... and also log severity to log
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
When releases are prepared, the upper bound on setuptools in pyproject.toml will be automatically updated
to the latest version available on PyPI. This version will then be tested by the package-data sanity test
during the release process and will be used to build the release.
This change ensures that a released version of ansible-core can be built in the future if a new setuptools
release includes breaking changes that would prevent building a functional package. If a downstream package
maintainer requires a newer setuptools version than the upper bound permits, they can patch pyproject.toml
as needed. Since ansible-core releases support specific Python versions, lack of support for new setuptools
releases will have no effect on support for future Python versions.
The marker is removed in ansible-test managed environments, but the apt test restores it
by installing/upgrading packages. To avoid breaking later tests, the marker needs to be
removed again.
ci_complete
* winrm - quota retry handling
Add a retry attempt when receiving ERROR_WSMAN_QUOTA_MAX_OPERATIONS when
starting a command. This can occur when running a loop with multiple
iterations or an action plugin that runs multiple commands.
* Update pywinrm constraint for test
* Add verbose hint and mark test as destructive
This change simplifies construction and footprint of testing rpm
repository created by rpmfluff:
* all packages default to noarch
* only when necessary build package for a specific architecture(s)
* do not build several repositories each for specific arch
* remove duplicate "incompatible arch" test
* skip_broken_and_nobest: move testing packages from an external repo
into our dummy repo for transparency
* remove compatibility code from create_repo.py for versions no longer
needed
* remove support for old OS versions from setup_rpm_repo
* simplify representation of an rpm in create_repo.py to allow easier
changes
Notes
* there is one more external testing repo used at
https://ci-files.testing.ansible.com/test/integration/targets/setup_rpm_repo/repo-with-updateinfo
Removes the docs for the environment keyword in the shell base plugins
as they are a no-op in the plugins themselves. The environment value is
provided by the action base which gets it from the action base on the
task.environment value. This should avoid confusion around how its being
set and removes some code that is not used at all.
While upgrade process removes a package, module should
report changed=True instead of changed=False
Fixes: #46314
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* lift code that normalizes value type for boolean vtype to cover both
branches of conditional.
* remove obsolete and incomplete conversion of type in set_selection.
Fixes: #83594
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
Previous change overlooked 'uncommenting' the variable entry as a way to update this to keep the functionality.
Co-authored-by: Glandos <bugs-github@antipoul.fr>
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
* dnf: follow-up on is_newer_installed arches fix
* fix for the non package object case
* prevent early bailout in dnf _is_newer_version_installed
* non-installed available arches would fail out of the check early
---------
Co-authored-by: Matt Davis <nitzmahone@redhat.com>
* Quality-of-life improvements for release tool
- Default devel releases to b1
- Default non-devel releases to rc1
- Default to release announcement to console
- Avoid auto-links in GH release annoucements for file sizes
Using this dictionary to store the return values results in
the return values showing up in the returned
`invocation['module_args']`, which is confusing. It also causes all
module arguments to be returned, which is preserved by this change but
should ideally be removed in the future.
* Reject option/alias names equal up to casing belonging to different options.
* Update test/lib/ansible_test/_util/controller/sanity/validate-modules/validate_modules/main.py
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
previouslly we recorded but did not show to avoid spam
since we could not dedup from forks, that was already
fixed in another PR so now we can show/display them.
Also:
* funcitonalize deprecation msg construct from docs
* reuse formatting func in cli
* normalize alternatives: most of the code used intended plural
but some and most data/tests used the singular
* update schemas and tests
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
* Replace references to archived ansible/ansible-examples repository
* Update examples to reference ansible repository
* Include HTTPS among SSH references in examples
Previously, `support_discard` simply returned the value of
`/sys/block/{device}/queue/discard_granularity`. When its value is `0`, then
the block device doesn't support discards; _however_, it being greater than
zero doesn't necessarily mean that the block device _does_ support discards.
But another indication that a block device doesn't support discards is
`/sys/block/{device}/queue/discard_max_hw_bytes` being equal to `0` (with the
same caveat as above). So if either of those are `0`, set `support_discard` to
zero, otherwise set it to the value of `discard_granularity` for backwards
compatibility.
Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
Identical branches in conditional node: 'elif name != permanent_hostname' and 'else'. First bruch is not nessesary
Co-authored-by: mar.s <marchenkov.s.d@gmail.com>
* Enable Ubuntu 24.04 group 6 in CI
* Disable rootfull Podman on Ubuntu
* Disable unix-chkpwd AppArmor profile on Ubuntu for Fedora 40 tests
* Document AppArmor and rootfull issues
Previously, if the checksum of the downloaded file did not match the
specified checksum, the *destination* file was removed. This possibly
leaves the system that is being provisioned in an invalid state.
Instead, the checksum should be calculated on the temporary file only.
If there's a mismatch, delete the *temporary* file, not the destination
file.
This requires checking the checksum before moving the file.
The timezone support module was used only for changing the timezone in the user module integration tests.
Changing the timezone for the tests is unecessarily complex for the purpose of asserting proper parsing of user expiration times.
When vault password file env variable is set to blank,
this value is converted to CWD and passed for further
processing.
Check if ANSIBLE_VAULT_PASSWORD_FILE is not a directory.
Fixes: #42960
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
also make the YAML booleanization the same as spec/JSON/module function
previous 'aproximation' was missing several options.
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Improves the Add-Type temporary directory handler to include a retry
mechanism and not fail on an error. Deleting a temporary file used in
compilation is not a critical error and should improve the reliability
of Ansible on Windows hosts.
* Fix task.resolved_action for callbacks when playbooks use action or local_action
* Fix using module_defaults with 'action' and 'local_action' task FA and add a test case
Fixes#81905
If user provides checksum URL which requires basic auth,
then module parameter username and password is used to
download the checksum file.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Correct a code comment in the PlaybookExecutor class that explains
the host failure condition under which the PBE breaks out of the
run. This comment previously stated that the maximum failure
percentage was checked, but that logic has been refactored into
the linear strategy plugin.
* Disable cipher suite integration tests
When the client and server both support TLS 1.3, cipher suite selection is effectively a no-op.
Python provides no way to disable TLS 1.3 cipher suites, and no way to prefer user-selected cipher suites.
Since TLS 1.3 includes a mandatory cipher suite, this makes it impossible to induce a cipher suite related error when using TLS 1.3.
* Update multipart form-data integration test
* Fix netrc integration tests
* Update HTTP test host substring
* ansible-test - Update http-test-container
* Documentation on the timeout keyword
* wording changes
* keyword description wording edit
* removed whitespace at the end of lines.
* Changed to more accurate wording
---------
Co-authored-by: Niko Mlynarčik <536617@mail.muni.cz>
* remove no longer needed tasks
* remove workarounds
* consolidate conditionals
* remove distros/versions from conditionals that are no longer present
in CI
* Add version ceiling for pypsrp
Add a version ceiling for the pypsrp requirements. This ensures that the
future v1.0.0 release won't impact existing users who install the
library using the version range specified.
* Use constraints file
* Add changelog for this change
* decrypt option is used by assemble action plugin.
Add this parameter to remove failure raised by
validate-modules:nonexistent-parameter-documented
Fixes: #80840
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Fix installing roles containing symlinks
Fix sanitizing tarfile symlinks relative to the link directory instead of the archive
For example:
role
├── handlers
│ └── utils.yml -> ../tasks/utils/suite.yml
The link ../tasks/utils/suite.yml will resolve to a path outside of the link's directory, but within the role
role/handlers/../tasks/utils/suite.yml
the resolved path relative to the role is tasks/utils/suite.yml, but if the symlink is set to that value, tarfile would extract it from role/handlers/tasks/utils/suite.yml
* Replace overly forgiving test case with tests for a symlink in a subdirectory of the archive and a symlink in the archive dir when these are not equivalent.
* Build test case from role files to make it easier to add test cases
Fixes#82702Fixes#81965Fixes#82051
This fixes "Arch dependent binaries in noarch package" error cause by
including files created by make_elf function in noarch packages. While the
error only manifests itself on EL 7 and 8 it is better to use files
suitable for noarch packages to prevent the error potentially
re-occuring in the future.
* test: remove ansible-examples.git repo
* To speed up git tests remove reference to ansible-examples.git
Fixes: #81327
* Make CI green
Signed-off-by: Ansible Test Runner <noreply@example.com>
Co-authored-by: Ansible Test Runner <noreply@example.com>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
In #80094 support for var substitution for cachedir was added but there
are more options that should be supported. Using an API for
prepend_installroot which should be done anyway provide that feature
so use that. In addition, perform the operation once all substitutes
are in place (releasever as well).
In general the existence of a runtime-enabled unit should not prevent
a persistent enable being set.
Specifically this handles the case where there is an entry in fstab
for a mount point (which is retained to allow manual mount/umount to
take place) and yet a systemd mount unit needs to be deployed to
handle other unit options. There will be a generator-created unit file
which shows the unit as enabled-runtime and the persistent enable of
the mount unit will fail.
Additionally improve the comments and modify the code to use rsplit()
and the "in" notation since "systemctl is-enabled" is documented to
return specific values in the cases of interest.
---------
Signed-off-by: David Greaves <david@dgreaves.com>
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
The cache directory can be specified with variables that are expanded by DNF, for example,
```
cachedir=/var/cache/yum/$basearch/$releasever
```
But the `dnf` module would use that path literally, instead of replacing
`$basearch` and `$releasever` with their values.
This commit ensures that variables in `cachedir` are properly substituted.
Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
Instead, set module_defaults at the play level, which will apply to the
gather_facts keyword as well as any
ansible.builtin.setup/ansible.builtin.gather_facts tasks.
- hosts: all
gather_facts: yes
module_defaults:
ansible.builtin.setup:
fact_path: /path/to/facts.d/
gather_subset: '!all,!min,local'
gather_timeout: 20
tasks:
# also applies to:
- ansible.builtin.setup:
- ansible.builtin.gather_facts:
If you wish to only apply these to the gather_facts keyword, set as play keywords.
Add explicit error when the calculated dest path for fetch becomes a
local directory. The existing behaviour will not be checked unlike when
the path did not end with a trailing slash.
* add a note for user to not to include extraneous single or double
quotes while specifying package name with a specific version.
Fixes: #82763
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
allow_duplicates is not part of the role uniqueness, so the value on the cached role may not match the current role.
* remove the allow_duplicates check from Role.has_run() which operates on the deduplicated role
* check the current role's allow_duplicates value in the strategy
Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
* Allow role name prefix for handler task listen topics
For example,
- name: handler name
debug:
listen: topic1
can be notified using `topic1`, `role : topic1` if the handler is in a
standalone or collection role, and `ns.col.role: topic1` if the role is
in a collection, the same way handler names work.
changelog
* fix changelog and tests
* Add prefix to `origin` when configuration variables come from ini files
Fixes ansible#82387
This change was suggested by @bcoca in
https://github.com/ansible/ansible/pull/82388#discussion_r1424235728 and
https://github.com/ansible/ansible/pull/82388#discussion_r1424249732
When configuration variables come from an ini file, their `origin` is
now set to `ini: <file>`. Similarly, once supported, YAML configuration
files will have their `origin` as `yaml: <file>`.
Consequently, since unquoting configuration strings should happen if and
only if they come from an ini file, this condition boils down to testing
whether their `origin` starts with `ini:`.
* Do not add prefix to `origin` but explicitly pass `origin_ftype`
So as not to rely on a specific format of the `origin` string,
as suggested by @sivel in
https://github.com/ansible/ansible/pull/82388#issuecomment-1881714871
When using winrm over HTTP with message encryption enabled and a task
has timed out the connection plugin will fail to cleanup the WinRM
command. This will change that exception into a warning as a timeout is
already an exception event and a failure to clean the operation should
not override the timeout error shown.
* Allow check mode async task disabling check_mode
Allows running an async task with check_mode: False when the playbook is
being run in check mode.
* Add check_mode attribute to internal cleanup task
* install recommended packages while installing deb files and
install_recommends is set to true.
Fixes: #29726
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* use copystat to copy as many attributes as possible before os.rename
update unit test mocks for updated method of attribute preservation
add integration test for lineinfile case
remove erroneous `- meta: end_play` from lineinfile test suite
* add keep_dest_attrs parameter to control whether src attributes are
copied initially, and for existing destinations, whether the src is
updated using the dest before being renamed
consolidate with copy unsetting extended attrs
ci_complete
Enable file cache for vaulted host_vars_files vars plugin
* fixes#81994
* Changed cache arg from bool to str to allow 'vaulted' only file cache
* removed unused used var
---------
Co-authored-by: Steffen Oschatz <so@ypsilon.net>
Until ansible-core 2.12 the facts cache file created by this module, have permission set as 644 which allows the other users to read the cache, since ansible-core 2.13, we create the temporary file, but we do not set the permission after renaming the temporary file. Adding the line to set the permission to allow other users/groups to read this file.
* added integration test for issue 82611 regarding discrepency between apt-get clean and ansible.builtin.apt: clean
* fixed new line issue
* Implementation of bug fix for 82611
Fixed discrepancy in behaviour between apt-get clean and its equivalent ansible.builtin.apt: clean=True
Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
---------
Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
* added integration test for issue 82655 regarding ansible_loop_var and ansible_index_var inside included yml
* ensure correct handling of ansible_loop_var and ansible_index_var inside ansible builtin include_tasks Co-authored-by: Matt <matt@sivel.net>
* added changelog
* fixed new line issue
* Handle error raised when argument validation with elements=int
and value is not within choices
Fixes: #82776
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
If the connection plugin fails to write the data to run to stdin we will
only attempt to get the output with one operation attempt. If this times
out we will consider the command to have failed and raise an exception
instead of forever attempting to get the output.
Additionally, this patch takes care of installing GPG within the
`ansible-galaxy-collection` test when running under macOS 14 and higher.
PR #82697
ci_complete
allow extra vars when templating j2 files in the skeleton, for example:
ansible-galaxy init --role-skeleton /path/to/skeleton --extra-vars @/path/to/vars_file.yml newrole
ansible-galaxy init --extra-vars "min_ansible_version=2.17.0" newrole
Extend the wordings in warning if the reserved keyword _ansible_
is used as a module parameter.
Fixes: #82514
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Jordan Borean <jborean93@gmail.com>
The ``validate-modules`` sanity test no longer attempts to process files with unrecognized extensions as Python.
Integration tests have been added to verify Python-specific checks do not apply to these files.
The `invalid-extension` and `missing-gplv3-license` checks still apply to these files. This may change in the future.
* Allow subdirectories with import_role _from options
Add tests that tasks_from is restricted to the role
Note that a task like:
- import_role:
name: role
tasks_from: tasks/entrypoint.yml
will now load tasks from "{{ role_path }}/tasks/tasks/entrypoint.yml"
instead of "{{ role_path }}/tasks/entrypoint.yml". This change in
behavior matches include_role.
* better test case (filename doesn't match one in tasks/)
Fixes#82584
best/nobest options are one of the options whose default values are set
by an OS distribution. For example in our CI, both Fedora and RHEL set
the best option to different default values. As such we should defer to
the distributions for the default value and not change it by default but
if users wish to change it they can do so explicitly.
Currently the dnf module sets the nobest option inconsistenly and not for
all cases. This patch fixes that to reflect the behavior described
above. In addition adding the best option for both dnf and dnf5 modules
since the best option is prefer to nobest in dnf while in dnf5 nobest is
completely removed in favor of best.
Fixes#82616
* prettify ansibile-doc output
delimiters when no color
avoid triggering color mode for existing tests
all use _format to observe nocolor
more v more info
imporoved conditional display
updated version on -v
normalize role errors
expand role data, dedupe code, fix formatting on warning
fix bug with galaxy info
role list improvements:
lists all roles (even w/o meta/argspec)
specs now indented under role
role only listed once
Updated tests to reflect format changes
Co-authored-by: Felix Fontein <felix@fontein.de>
Jinja may generate an invalid Python source code from a template. Trying
to compile such source code into a Python code object results in
SyntaxError being thrown. An example of such a template is providing the
same keyword argument into a lookup twice, resulting in:
`SyntaxError: keyword argument repeated`.
Since `jinja2.exceptions.TemplateSyntaxError` does not cover such a
case, as it is not a Jinja parsing error, we need to catch SyntaxError
explicitly ourselves.
Fixes#82606
* Fix loading legacy vars plugins when the plugin loader cache is reset
* Remove extra cache layer by ensuring vars plugin names are cached (stateless or not) so that the plugin loader cache can double as the load order
Prior to this commit, it was impossible to use a module like dnf with a
URL that contains a username with an @ such as an email address
username, because:
dnf:
name: https://foo@example.com:bar@example.com/some.rpm
Would cause netloc parsing to fail. However, the following:
dnf:
name: https://foo%40example.com:bar@example.com/some.rpm
Would also fail because ansible would *not* URL-decode the credentials,
causing the following to be base64 encoded in the Authorization header:
Zm9vJTQwZXhhbXBsZS5jb206YmFyCg==
Which decodes to:
foo%40example.com:foo
Which is *not* the authorized username, and as such, *won't* pass basic
auth.
With this commit, Ansible's url lib behaves like curl, chromium, wget,
etc, and encodes the above to:
Zm9vQGV4YW1wbGUuY29tOmJhcgo=
Which decodes to:
foo@example.com:bar
Which will actually pass the HTTP Basic Auth, and is the same behaviour
that you will find ie. with:
curl -vvI https://foo%40bar:test@example.com 2>&1 |grep Auth | awk '{ print $4 }'
This moves handling of callbacks and play recap stats from
_load_included_file to individual strategies so include_role tasks are
accounted for, not just include_tasks.
Fixes#77336
* uri: Two tests that demonstrate missing handling of the "force" parameter
Add unit and integration tests that demonstrate that the uri module is not
handling the "force" parameter.
The unit test demonstrates that when "force" is present in the module parameters,
it is not being passed through to fetch_url().
The integration test demonstrates that "force" does not disable caching as
documented, and calls with a "dest" parameter that points to an existing file
can result in a "304 Not Modified" response.
* uri: Handle the "force" parameter properly
The uri module documents a "force" parameter that can be used to disable caching.
The module accepted the parameter but didn't pass it through to the fetch_url() method
which implements the logic to handle setting the appropriate headers for disabling
caching. This change passes the "force" parameter through as expected, allowing caching
to be disabled when requested by the module caller.
* when doing a 'contains' search, determine the encoding of the files to be searched
* set default encoding to None for backwards compatibility
* changelog, error handling, tests added
* add sanity ignore.txt for non-utf-8 test
This patch removes an import fallback that was only executed under
Python 2. Now that we don't run tests against that runtime, it
generates an uncovered line. Dropping it will slightly increase the
coverage metric as a side effect.
* default svn URL to 127.0.0.1 for subversion integration tests
* svn client gives up before trying IPv4 addresses when localhost->(::1,127.0.0.1) in dual-stack envs (eg podman, most real hosts)
* svn client also requires legacy CN match on cert
* IPv6 works, but setup playbook would need a bunch more templating exceptions to conditionally manage `[::1]`
* explain IPv4 defaults
The timing of the async tasks was a little unpredictable, meaning that
sometimes we would get an unexpected number of v2_runner_on_async_poll
callbacks, and fail the test. This change fixes the issue by increasing
the poll interval to 2 seconds and the sleep duration to 3 seconds, such
that on a reasonably responsive system we will poll twice per task, with
the sleep ending in the middle of the two polls.
The include_me.yml file does not exist in this integration test. It has
been added.
The remote_tmp_dir.path expression is invalid - the setup_remote_tmp_dir
role uses set_fact to set remote_tmp_dir to remote_tmp_dir.path.
The integration tests run with ANSIBLE_HOST_PATTERN_MISMATCH=error,
meaning that the final play was never reached. Set
ANSIBLE_HOST_PATTERN_MISMATCH=warning to continue past the play and
trigger the v2_playbook_on_no_hosts_matched callback.
When ansible-test installs requirements, it now instructs pip to allow installs on externally managed environments as defined by PEP 668.
This only occurs in ephemeral environments managed by ansible-test, such as containers,
or when the `--requirements` option is used.
* Remove role name conversion based on whether the repo name starts with
ansible-role
This was added in 2.3 to match the Galaxy ui behavior of truncating the
'ansible-role-' prefix automatically, but the new backend requires an
alternate name to be provided or defined in the ``galaxy_info``
metadata.
Roles that were imported using the ansible-role-$name convention will
need to use ``--role-name`` or add ``role_name`` to the ``galaxy_info``
dictionary in ``meta/main.yml``.
changelog
* Update changelogs/fragments/fix-default-ansible-galaxy-role-import-name.yml
* Update changelogs/fragments/fix-default-ansible-galaxy-role-import-name.yml
Co-authored-by: flowerysong <junk+github@flowerysong.com>
---------
Co-authored-by: flowerysong <junk+github@flowerysong.com>
* deb822_repository: handle idempotency
Sort the parameters in order to handle idempotency
Fixes: #82454
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Deprecate `required` param in get_bin_path
* The parameter `required` in process.get_bin_path API
is deprecated. Will be removed in 2.21
Fixes: #82464
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* Add the task info for tombstoned plugins
* Fix deprecation for 'include' by removing it from BUILTIN_TASKS which skip the plugin loader lookup
* changelog
remove obsolete unit test using 'include'
* Update changelogs/fragments/improve-tombstone-error.yml
* move incorrect, unused 'others' option
this was to document 'file' options are usable .. but we already
import the file fragment to document those options.
* removed now bad ignore
* Various fixes to roles
- static property is now properly set
- role_names and other magic vars now have full list
- role public/private var loading is now done when adding to play.roles instead of on each var query
- added tests
Co-authored-by: Felix Fontein <felix@fontein.de>
* plugin config, ensure set_option correctly casts
until now we relied on plugin author getting the right type, now
the config system itself will process as it would for set_options
* lookups, make file searching use better is_role
The dwim function will internally try by detecting tasks/main['','.yml','.yaml]
but this is far from optimial, the existince of role path in vars is much better
indicator that we can use to pass a hint
* updated test to avoid main.yml
* Add helptext to ansible-inventory's positional arg
This explanation is already present in the documentation and
should make the usage more clear when looking at the --help output
* fix indentation
* change description of positional arg
It has no effect on --list and the docs now reflect that
* Run svn integration test locally with TLS
This patch uses a `trustme` to make an ephemeral CA, and server, and
client TLS artifacts for testing. These are integrated into the Apache
web server via it's `mod_ssl`.
Resolves#82207
* Replace GitHub SVN HTTPS URL w/ localhost over TLS
This change gets rid of the need to use GitHub, which is just about to
drop support for SVN [[1]]. Moreover, it eliminates the need to use
external network for any SVN commands in the test.
[1]: https://github.blog/2023-01-20-sunsetting-subversion-support/
Set the tarfile attribute to a normalized value from unfrackpath instead
of validating path parts and omiting potentially invald parts
Allow tarfile paths/links containing '..', '$', '~' as long as the
normalized realpath is in the tarfile's role directory
* Amend doc of name= parameter to indicate a list of packages can be used
One of the examples here shows package installing a list of packages, but the name= parameter did not reflect this possibility.
* add dependency to underlying package manager
* apt_repository: Do not convert symlink repo to a normal file
* module manually writes to repo files. If the file is a symlink
module used to re-write and convert a symlink to a normal file.
This fix tracks the symlink and keeps the changes intact.
Fixes: #49809
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* ValueError exception is raised when Null Character is sent
to syslog.syslog with Python 3.12.
* Handle this error gracefully instead of stacktrace
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Enable use of vendored copy of distutils by removing
the environment variable SETUPTOOLS_USE_DISTUTILS
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
This replaces the regex based check previously used within the validate-modules sanity test.
NOTE: The pylint check overlaps with a similar check for pep8.
Both are used, since the pep8 based check is more nuanced than the one in pylint.
For example, allowing `is` checks, but not `==` comparisons.
If the pylint check is ignored, the pep8 check would still be relevant.
* Remove unused argument in call to build_vault_ids
* Remove obsolete build_vault_ids tests
* Change tests to account for argument removal
* Remove redundant test
TLS 1.3 adds a different method it can use to request a client
certificate after the handshake but Python does not allow this by
default. This commit sets the attribute needed to enable this scenario
when using client certificates on Python 3.8+, 3.7.1+.
* Fix locale related parsing error in git.py
This fixes the issue found at https://github.com/ansible/ansible/issues/77213 which got introduced by changing over to "best parseable locale" approach, but missing out the requirement of locales other than `L` having to have `LANGUAGE` specified as well. For further details, check this post within the issue discussion: https://github.com/ansible/ansible/issues/77213#issuecomment-1446919617
* Add changelog
Create 81931-locale-related-parsing-error-git.yml
- Update `data2_vaulted_string_with_id` to match the documented plaintext.
- Add a comment explaining how `data2_vaulted_string_with_id` was derived.
- Add assertions for unvaulted values to ensure they match their plaintext.
- Add round-trip tests for vault+unvault when no salt is used.
Python 3.10 is the minimum version on the controller, and it requires openssl 1.1.1 or later.
As a result, there's no need to check the openssl version any longer.
Adds an option that can have an action plugin tell the module to ignore
options that do not fit its arg spec. This is to enable support for core
running modules that exist outside of the collection that may not be new
enough to support some of the options supplied to it.
* Clean up hacking/ansible-profile script
Also rename the script so it has a `.py` extension.
* Remove test-module symlink
Use the test-module.py script instead.
* Reword the error message when the module fails to parse parameters
in JSON format
* misc typo fixes
Fixes: #81188
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Improve host_group_vars efficiency:
* normalize the basedir with `os.path.realpath()` once and cache it
* cache missing paths/files
* reduce the calls to `isinstance`
Add a couple more general improvements in vars/plugins.py get_vars_from_path():
* call `PluginLoader.all()` once for vars plugins and reload specific
plugins subsequently
* don't reload legacy/builtin vars plugins that are not enabled
Add a test for host_group_vars and legacy plugin loading
Co-authored-by: Matt Davis <mrd@redhat.com>
* changelog
* Add a new is_stateless attribute to the vars plugin baseclass
update integration tests to be quieter and use the same test pattern
Fix deprecation and adjust test that didn't catch the issue (deprecation only occured when the value was False)
move realpath cache to host_group_vars (do not smuggle call state as instance data)
refactor under a single 'if cache:' statement
Call os.path.isdir instead of always calling os.path.exists first. Just call os.path.exists to differentiate between missing and non-directory.
remove call to super(VarsModule, self).get_vars()
use the entity name as the cache key instead of variable location
Remove isinstance checks and use a class attribute just in case any plugins are subclassing Host/Group
Replace startswith by checking index 0 of the name instead, since host/group names are required
* rename is_stateless to cache_instance to make it more clear what it does
* add plugin instance cache using the path to plugin loader
reduce loading stage option if a new instance isn't created
don't require a known subdir on PluginLoader instantiation for backwards
compatibility
rename attribute again
contain reading from/initializing cached instances to a plugin loader method
* Deprecate v2 vars plugins
* Refactor to use the cache in existing plugin loader methods
Rename the attribute again
Refactor host_group_vars with requested changes
Make changelog a bugfixes fragment
Add a deprecation fragment for v2 vars plugins.
Add type hints
* unbreak group_vars
* Apply suggestions from code review
* misc tweaks
* always cache instance by both requested and resolved FQ name
* add lru_cache to stage calculation to avoid repeated config consultation
* handle KeyError from missing stage option
---------
Co-authored-by: Matt Davis <mrd@redhat.com>
When templating tags (which happens outside of standard `post_validate`) we
need to template each object in the inheritance chain and set the templated
values on those objects individually. That way when `task.tags` is called the
`extend` functionality properly picks up the templated values of all
parents into one flatten list.
Fixes#81053
revert to previous behavior to push vars to play at compile time
add `public` parameter to allow per import control of exporting (vs just the global config)
Co-authored-by: tchernomax <maxime.deroucy@gmail.com>
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
Provide more information about plugin usage which needs to be
used instead of the deprecated plugin in the deprecation message.
Fixes: #80561
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* The module systemd is renamed to systemd_service to maintain
the scope of the module. Mention this in the module description.
* Misc typo fixes.
Fixes: #80917
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Test heuristic_log_sanitize. See #81689
* Add note about what this test is doing
* grammar
Co-authored-by: Matt Clay <matt@mystile.com>
---------
Co-authored-by: Matt Clay <matt@mystile.com>
While creating a symlink, we apply fs attributes changes
to the destination. But in case of a non-existent destination, we warn
the user. This warning message can be suppressed by setting follow=false.
Add a note of this behavior in the `follow` parameter.
Fixes: #50543
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* ansible-galaxy fix scm dependency error
also changed usage of 'virtual colleciton' to actual type
avoid error by filtering out virtual collections that dont have
expected properties
simplified as per webknjaz
* Update lib/ansible/galaxy/collection/__init__.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Make sure paths are correct when building collection files manifest
This commit makes sure the path of the files part of
the collection build manifest are correct.
This commit uses os.path.commonprefix instead of
dealing with strings.
Signed-off-by: Carlos Camacho <ccamacho@redhat.com>
Bugfix Pull Request
Fixes: #81618
* Revert the change note type to `minor_changes`
* Clarify the change note with user-oriented details
---------
Signed-off-by: Carlos Camacho <ccamacho@redhat.com>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
PR #81606.
Prior to this patch, when `--pre` CLI flag was not passed, the
dependency resolver would treat concrete collection dependency
candidates (Git repositories, subdirs, tarball URLs, or local dirs or
files etc) as not meeting the requirements.
This patch makes it so pre-releases in any concrete artifact
references, and the ones being specifically pinned dependencies or
user requests, met anywhere in the dependency tree, are allowed
unconditionally.
This is achieved by moving the pre-release check from
`is_satisfied_by()` to the `find_matches()` hook, following the
Pip's example.
As a bonus, this change also fixes the situation when a collection
pre-releases weren't considered if it didn't have any stable releases.
This now works even if `--pre` wasn't requested explicitly.
Finally, this patch partially reverts commit
6f4b4c345b, except for the tests. And it
also improves the `--pre` hint warning to explain that it mostly
affects Galaxy/Automation Hub-hosted collection releases.
Ref #73416
Ref #79112Fixes#79168Fixes#80048Resolves#81605
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
Please see the [Community Guide](https://docs.ansible.com/ansible/latest/community/index.html) for information on how to ask questions on the [mailing lists](https://docs.ansible.com/ansible/latest/community/communication.html#mailing-list-information) and IRC.
If you have questions about anything related to Ansible, get in touch with us!
See [Communicating with the Ansible community](https://docs.ansible.com/ansible/devel/community/communication.html) to find out how.
The GitHub issue tracker is not the best place for questions for various reasons, but both IRC and the mailing list are very helpful places for those things, as the community page explains best.
The [Community Guide](https://docs.ansible.com/ansible/devel/community/index.html) also explains how to contribute
and interact with the project, including how to submit bug reports and code to Ansible.
Please note that the GitHub issue tracker is not the best place to ask questions for several reasons.
You'll get more helpful, and quicker, responses in the forum.
## CONTRIBUTING ?
By contributing you agree that these contributions are your own (or approved by your employer) and you grant a full, complete, irrevocable copyright license to all users and developers of the project, present and future, pursuant to the license of the project. You can also read the same [CLA](https://docs.ansible.com/ansible/latest/community/contributor_license_agreement.html) on the Ansible docsite.
By contributing to this project you agree to the [Developer Certificate of Origin (DCO)](#developer-certificate-of-origin-dco).
Please review the [Community Guide](https://docs.ansible.com/ansible/latest/community/index.html) for more information on contributing to Ansible.
The Ansible project is licensed under the [GPL-3.0](COPYING) or later. Some portions of the code fall under other licenses as noted in individual files.
The Ansible project accepts contributions through GitHub pull requests.
Please review the [Community Guide](https://docs.ansible.com/ansible/devel/community/index.html) for more information on contributing to Ansible.
## BUG TO REPORT ?
First and foremost, also check the [Community Guide](https://docs.ansible.com/ansible/latest/community/index.html).
First and foremost, also check the [Community Guide](https://docs.ansible.com/ansible/devel/community/index.html).
You can report bugs or make enhancement requests at the [Ansible GitHub issue page](http://github.com/ansible/ansible/issues/new/choose) by filling out the issue template that will be presented.
You can report bugs or make enhancement requests at
the [Ansible GitHub issue page](http://github.com/ansible/ansible/issues/new/choose) by filling out the issue template that will be presented.
Also please make sure you are testing on the latest released version of Ansible or the development branch; see the [Installation Guide](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html) for details.
Also please make sure you are testing on the latest released version of Ansible or the development branch.
See the [Installation Guide](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html) for details.
Thanks!
## DEVELOPER CERTIFICATE OF ORIGIN (DCO)
This document was created by the Linux Kernel community and is a simple statement that you, as a contributor, have the legal right to make the contribution.
```text
Developer Certificate of Origin
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
[](https://docs.ansible.com/ansible/latest/community/code_of_conduct.html)
[](https://docs.ansible.com/ansible/devel/community/code_of_conduct.html)
[](https://bestpractices.coreinfrastructure.org/projects/2372)
@ -40,21 +40,33 @@ features and fixes, directly. Although it is reasonably stable, you are more lik
breaking changes when running the `devel` branch. We recommend getting involved
in the Ansible community if you want to run the `devel` branch.
## Get Involved
## Communication
* Read [Community Information](https://docs.ansible.com/ansible/latest/community) for all
Join the Ansible forum to ask questions, get help, and interact with the
community.
* [Get Help](https://forum.ansible.com/c/help/6): Find help or share your Ansible knowledge to help others.
Use tags to filter and subscribe to posts, such as the following:
* Posts tagged with [ansible](https://forum.ansible.com/tag/ansible)
* Posts tagged with [ansible-core](https://forum.ansible.com/tag/ansible-core)
* Posts tagged with [playbook](https://forum.ansible.com/tag/playbook)
* [Social Spaces](https://forum.ansible.com/c/chat/4): Meet and interact with fellow enthusiasts.
* [News & Announcements](https://forum.ansible.com/c/news/5): Track project-wide announcements including social events.
* [Bullhorn newsletter](https://docs.ansible.com/ansible/devel/community/communication.html#the-bullhorn): Get release announcements and important changes.
For more ways to get in touch, see [Communicating with the Ansible community](https://docs.ansible.com/ansible/devel/community/communication.html).
## Contribute to Ansible
* Check out the [Contributor's Guide](./.github/CONTRIBUTING.md).
* Read [Community Information](https://docs.ansible.com/ansible/devel/community) for all
kinds of ways to contribute to and interact with the project,
including mailing list information and how to submit bug reports and
code to Ansible.
* Join a [Working Group](https://github.com/ansible/community/wiki),
an organized community devoted to a specific technology domain or platform.
including how to submit bug reports and code to Ansible.
* Submit a proposed code update through a pull request to the `devel` branch.
* Talk to us before making larger changes
to avoid duplicate efforts. This not only helps everyone
know what is going on, but it also helps save time and effort if we decide
some changes are needed.
* For a list of email lists, IRC channels and Working Groups, see the
@ -67,7 +79,7 @@ We document our Coding Guidelines in the [Developer Guide](https://docs.ansible.
* The `devel` branch corresponds to the release actively under development.
* The `stable-2.X` branches correspond to stable releases.
* Create a branch based on `devel` and set up a [dev environment](https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_general.html#common-environment-setup) if you want to open a PR.
* Create a branch based on `devel` and set up a [dev environment](https://docs.ansible.com/ansible/devel/dev_guide/developing_modules_general.html#common-environment-setup) if you want to open a PR.
* See the [Ansible release and maintenance](https://docs.ansible.com/ansible/devel/reference_appendices/release_and_maintenance.html) page for information about active branches.
- Improved SUSE distribution detection in distribution.py by parsing VARIANT_ID from /etc/os-release for identifying SLES_SAP and SL-Micro. Falls back to /etc/products.d/baseproduct symlink for older systems.
- Remove unnecessary shebang from the ``hostname`` module.
- Use ``importlib.metadata.version()`` to detect Jinja version as jinja2.__version__ is deprecated and will be removed in Jinja 3.3.
- ansible-doc - Return dynamic stub when reporting on Jinja filters and tests not explicitly documented in Ansible
- ansible-test - Add RHEL 10.0 as a remote platform for testing.
- apt_repository - remove Python 2 support
- csvfile lookup - remove Python 2 compat
- display - Add ``help_text`` and ``obj`` to ``Display.error_as_warning``.
- display - Replace Windows newlines (``\r\n``) in display output with Unix newlines (``\n``). This ensures proper display of strings sourced from Windows hosts in environments which treat ``\r`` as ``\n``, such as Azure Pipelines.
- facts - add "Linode" for Linux VM in virtual facts
- module_utils - Add optional ``help_text`` argument to ``AnsibleModule.warn``.
- ssh agent - Added ``SSH_AGENT_EXECUTABLE`` config to allow override of ssh-agent.
- ssh connection plugin - Added ``verbosity`` config to decouple SSH debug output verbosity from Ansible verbosity. Previously, the Ansible verbosity value was always applied to the SSH client command-line, leading to excessively verbose output. Set the ``ANSIBLE_SSH_VERBOSITY`` envvar or ``ansible_ssh_verbosity`` Ansible variable to a positive integer to increase SSH client verbosity.
- task timeout - Specifying a timeout greater than 100,000,000 now results in an error.
- windows - Added support for ``#AnsibleRequires -Wrapper`` to request a PowerShell module be run through the execution wrapper scripts without any module utils specified.
- windows - Added support for running signed modules and scripts with a Windows host protected by Windows App Control/WDAC. This is a tech preview and the interface may be subject to change.
- windows - Script modules will preserve UTF-8 encoding when executing the script.
Deprecated Features
-------------------
- The ``ShellModule.checksum`` method is now deprecated and will be removed in ansible-core 2.23. Use ``ActionBase._execute_remote_stat()`` instead.
- The ``ansible.module_utils.common.collections.count()`` function is deprecated and will be removed in ansible-core 2.23. Use ``collections.Counter()`` from the Python standard library instead.
-``ansible.compat.importlib_resources`` is deprecated and will be removed in ansible-core 2.23. Use ``importlib.resources`` from the Python standard library instead.
Bugfixes
--------
- Core Jinja test plugins - Builtin test plugins now always return ``bool`` to avoid spurious deprecation warnings for some malformed inputs.
- ansible-test - Disabled the ``bad-super-call`` pylint rule due to false positives.
- ansible-test - Fix incorrect handling of options with optional args (e.g. ``--color``), when followed by other options which are omitted during arg filtering (e.g. ``--docker``). Previously it was possible for non-option arguments to be incorrectly omitted in these cases. (https://github.com/ansible/ansible/issues/85173)
- ansible-test - Improve type inference for pylint deprecated checks to accommodate some type annotations.
- async_status module - The ``started`` and ``finished`` return values are now ``True`` or ``False`` instead of ``1`` or ``0``.
- constructed inventory - Use the ``default_value`` or ``trailing_separator`` in a ``keyed_groups`` entry if the expression result of ``key`` is ``None`` and not just an empty string.
- dnf5 - handle all libdnf5 specific exceptions (https://github.com/ansible/ansible/issues/84634)
- error handling - Error details and tracebacks from connection and built-in action exceptions are preserved. Previously, much of the detail was lost or mixed into the error message.
- from_yaml_all filter - `None` and empty string inputs now always return an empty list. Previously, `None` was returned in Jinja native mode and empty list in classic mode.
- local connection plugin - The command-line used to create subprocesses is now always ``str`` to avoid issues with debuggers and profilers.
- ssh agent - Fixed several potential startup hangs for badly-behaved or overloaded ssh agents.
- task timeout - Specifying a negative task timeout now results in an error.
- facts - add "CloudStack KVM Hypervisor" for Linux VM in virtual facts (https://github.com/ansible/ansible/issues/85089).
- modules - use ``AnsibleModule.warn`` instead of passing ``warnings`` to ``exit_json`` or ``fail_json`` which is deprecated.
Bugfixes
--------
- ansible-test - Updated the ``pylint`` sanity test to skip some deprecation validation checks when all arguments are dynamic.
- config - Preserve or apply Origin tag to values returned by config.
- config - Prevented fatal errors when ``MODULE_IGNORE_EXTS`` configuration was set.
- config - Templating failures on config defaults now issue a warning. Previously, failures silently returned an unrendered and untrusted template to the caller.
- config - ``ensure_type`` correctly propagates trust and other tags on returned values.
- config - ``ensure_type`` now converts mappings to ``dict`` when requested, instead of returning the mapping.
- config - ``ensure_type`` now converts sequences to ``list`` when requested, instead of returning the sequence.
- config - ``ensure_type`` now correctly errors when ``pathlist`` or ``pathspec`` types encounter non-string list items.
- config - ``ensure_type`` now reports an error when ``bytes`` are provided for any known ``value_type``. Previously, the behavior was undefined, but often resulted in an unhandled exception or incorrect return type.
- config - ``ensure_type`` with expected type ``int`` now properly converts ``True`` and ``False`` values to ``int``. Previously, these values were silently returned unmodified.
- convert_bool.boolean API conversion function - Unhashable values passed to ``boolean`` behave like other non-boolean convertible values, returning False or raising ``TypeError`` depending on the value of ``strict``. Previously, unhashable values always raised ``ValueError`` due to an invalid set membership check.
- dnf5 - when ``bugfix`` and/or ``security`` is specified, skip packages that do not have any such updates, even for new versions of libdnf5 where this functionality changed and it is considered failure
- plugin loader - Apply template trust to strings loaded from plugin configuration definitions and doc fragments.
- template action - Template files where the entire file's output renders as ``None`` are no longer emitted as the string "None", but instead render to an empty file as in previous releases.
- ansible-config will now show internal, but not test configuration entries. This allows for debugging but still denoting the configurations as internal use only (_ prefix).
- ansible-test - Improved ``pylint`` checks for Ansible-specific deprecation functions.
- ansible-test - Use the ``-t`` option to set the stop timeout when stopping a container. This avoids use of the ``--time`` option which was deprecated in Docker v28.0.
- collection metadata - The collection loader now parses scalar values from ``meta/runtime.yml`` as strings. This avoids issues caused by unquoted values such as versions or dates being parsed as types other than strings.
- deprecation warnings - Deprecation warning APIs automatically capture the identity of the deprecating plugin. The ``collection_name`` argument is only required to correctly attribute deprecations that occur in module_utils or other non-plugin code.
- deprecation warnings - Improved deprecation messages to more clearly indicate the affected content, including plugin name when available.
- deprecations - Collection name strings not of the form ``ns.coll`` passed to deprecation API functions will result in an error.
- deprecations - Removed support for specifying deprecation dates as a ``datetime.date``, which was included in an earlier 2.19 pre-release.
- deprecations - Some argument names to ``deprecate_value`` for consistency with existing APIs. An earlier 2.19 pre-release included a ``removal_`` prefix on the ``date`` and ``version`` arguments.
- modules - The ``AnsibleModule.deprecate`` function no longer sends deprecation messages to the target host's logging system.
Deprecated Features
-------------------
- Passing a ``warnings` or ``deprecations`` key to ``exit_json`` or ``fail_json`` is deprecated. Use ``AnsibleModule.warn`` or ``AnsibleModule.deprecate`` instead.
- plugins - Accessing plugins with ``_``-prefixed filenames without the ``_`` prefix is deprecated.
Bugfixes
--------
- Ansible will now ensure predictable permissions on remote artifacts, until now it only ensured executable and relied on system masks for the rest.
- dnf5 - avoid generating excessive transaction entries in the dnf5 history (https://github.com/ansible/ansible/issues/85046)
- Jinja plugins - Jinja builtin filter and test plugins are now accessible via their fully-qualified names ``ansible.builtin.{name}``.
- Task Execution / Forks - Forks no longer inherit stdio from the parent ``ansible-playbook`` process. ``stdout``, ``stderr``, and ``stdin`` within a worker are detached from the terminal, and non-functional. All needs to access stdio from a fork for controller side plugins requires use of ``Display``.
- ansible-test - Packages beneath ``module_utils`` can now contain ``__init__.py`` files.
- variables - The type system underlying Ansible's variable storage has been significantly overhauled and formalized. Attempts to store unsupported Python object types in variables now more consistently yields early warnings or errors.
- variables - To support new Ansible features, many variable objects are now represented by subclasses of their respective native Python types. In most cases, they behave indistinguishably from their original types, but some Python libraries do not handle builtin object subclasses properly. Custom plugins that interact with such libraries may require changes to convert and pass the native types.
Minor Changes
-------------
- Added a -vvvvv log message indicating when a host fails to produce output within the timeout period.
- AnsibleModule.uri - Add option ``multipart_encoding`` for ``form-multipart`` files in body to change default base64 encoding for files
- INVENTORY_IGNORE_EXTS config, removed ``ini`` from the default list, inventory scripts using a corresponding .ini configuration are rare now and inventory.ini files are more common. Those that need to ignore the ini files for inventory scripts can still add it to configuration.
- Jinja plugins - Plugins can declare support for undefined values.
- Jinja2 version 3.1.0 or later is now required on the controller.
- Move ``follow_redirects`` parameter to module_utils so external modules can reuse it.
- PlayIterator - do not return tasks from already executed roles so specific strategy plugins do not have to do the filtering of such tasks themselves
- SSH Escalation-related -vvv log messages now include the associated host information.
- Windows - Add support for Windows Server 2025 to Ansible and as an ``ansible-test`` remote target - https://github.com/ansible/ansible/issues/84229
- Windows - refactor the async implementation to better handle errors during bootstrapping and avoid WMI when possible.
-``ansible-galaxy collection install`` — the collection dependency resolver now prints out conflicts it hits during dependency resolution when it's taking too long and it ends up backtracking a lot. It also displays suggestions on how to help it compute the result more quickly.
- ansible-test - Automatically retry HTTP GET/PUT/DELETE requests on exceptions.
- ansible-test - Default to Python 3.13 in the ``base`` and ``default`` containers.
- ansible-test - Disable the ``deprecated-`` prefixed ``pylint`` rules as their results vary by Python version.
- ansible-test - Disable the ``pep8`` sanity test rules ``E701`` and ``E704`` to improve compatibility with ``black``.
- ansible-test - Improve container runtime probe error handling. When unexpected probe output is encountered, an error with more useful debugging information is provided.
- ansible-test - Replace container Alpine 3.20 with 3.21.
- ansible-test - Replace container Fedora 40 with 41.
- ansible-test - Replace remote Alpine 3.20 with 3.21.
- ansible-test - Replace remote Fedora 40 with 41.
- ansible-test - Replace remote FreeBSD 13.3 with 13.5.
- ansible-test - Replace remote FreeBSD 14.1 with 14.2.
- ansible-test - Replace remote RHEL 9.4 with 9.5.
- ansible-test - Show a more user-friendly error message when a ``runme.sh`` script is not executable.
- ansible-test - The ``yamllint`` sanity test now enforces string values for the ``!vault`` tag.
- ansible-test - Update ``nios-test-container`` to version 7.0.0.
- ansible-test - Update ``pylint`` sanity test to use version 3.3.1.
- ansible-test - Update sanity test requirements to latest available versions.
- ansible-test - Update the HTTP test container.
- ansible-test - Update the PyPI test container.
- ansible-test - Update the ``base`` and ``default`` containers.
- ansible-test - Update the utility container.
- ansible-test - Use Python's ``urllib`` instead of ``curl`` for HTTP requests.
- ansible-test - When detection of the current container network fails, a warning is now issued and execution continues. This simplifies usage in cases where the current container cannot be inspected, such as when running in GitHub Codespaces.
- ansible-test acme test container - bump `version to 2.3.0 <https://github.com/ansible/acme-test-container/releases/tag/2.3.0>`__ to include newer versions of Pebble, dependencies, and runtimes. This adds support for ACME profiles, ``dns-account-01`` support, and some smaller improvements (https://github.com/ansible/ansible/pull/84547).
- apt_key module - add notes to docs and errors to point at the CLI tool deprecation by Debian and alternatives
- apt_repository module - add notes to errors to point at the CLI tool deprecation by Debian and alternatives
- become plugins get new property 'pipelining' to show support or lack there of for the feature.
- callback plugins - add has_option() to CallbackBase to match other functions overloaded from AnsiblePlugin
- callback plugins - fix get_options() for CallbackBase
- copy - fix sanity test failures (https://github.com/ansible/ansible/pull/83643).
- copy - parameter ``local_follow`` was incorrectly documented as having default value ``True`` (https://github.com/ansible/ansible/pull/83643).
- cron - Provide additional error information while writing cron file (https://github.com/ansible/ansible/issues/83223).
- csvfile - let the config system do the typecasting (https://github.com/ansible/ansible/pull/82263).
- display - Deduplication of warning and error messages considers the full content of the message (including source and traceback contexts, if enabled). This may result in fewer messages being omitted.
- distribution - Added openSUSE MicroOS to Suse OS family (#84685).
- docs - add collection name in message from which the module is being deprecated (https://github.com/ansible/ansible/issues/84116).
- env lookup - The error message generated for a missing environment variable when ``default`` is an undefined value (e.g. ``undef('something')``) will contain the hint from that undefined value, except when the undefined value is the default of ``undef()`` with no arguments. Previously, any existing undefined hint would be ignored.
- file - enable file module to disable diff_mode (https://github.com/ansible/ansible/issues/80817).
- file - make code more readable and simple.
- filter - add support for URL-safe encoding and decoding in b64encode and b64decode (https://github.com/ansible/ansible/issues/84147).
- find - add a checksum_algorithm parameter to specify which type of checksum the module will return
- from_json filter - The filter accepts a ``profile`` argument, which defaults to ``tagless``.
- handlers - Templated handler names with syntax errors, or that resolve to ``omit`` are now skipped like handlers with undefined variables in their name.
- improved error message for yaml parsing errors in plugin documentation
- local connection plugin - A new ``become_strip_preamble`` config option (default True) was added; disable to preserve diagnostic ``become`` output in task results.
- local connection plugin - A new ``become_success_timeout`` operation-wide timeout config (default 10s) was added for ``become``.
- local connection plugin - When a ``become`` plugin's ``prompt`` value is a non-string after the ``check_password_prompt`` callback has completed, no prompt stripping will occur on stderr.
- lookup_template - add an option to trim blocks while templating (https://github.com/ansible/ansible/issues/75962).
- module - set ipv4 and ipv6 rules simultaneously in iptables module (https://github.com/ansible/ansible/issues/84404).
- module_utils - Add ``NoReturn`` type annotations to functions which never return.
- modules - PowerShell modules can now receive ``datetime.date``, ``datetime.time`` and ``datetime.datetime`` values as ISO 8601 strings.
- modules - PowerShell modules can now receive strings sourced from inline vault-encrypted strings.
- modules - Unhandled exceptions during Python module execution are now returned as structured data from the target. This allows the new traceback handling to be applied to exceptions raised on targets.
- pipelining logic has mostly moved to connection plugins so they can decide/override settings.
- plugin error handling - When raising exceptions in an exception handler, be sure to use ``raise ... from`` as appropriate. This supersedes the use of the ``AnsibleError`` arg ``orig_exc`` to represent the cause. Specifying ``orig_exc`` as the cause is still permitted. Failure to use ``raise ... from`` when ``orig_exc`` is set will result in a warning. Additionally, if the two cause exceptions do not match, a warning will be issued.
- removed harcoding of su plugin as it now works with pipelining.
- runtime-metadata sanity test - improve validation of ``action_groups`` (https://github.com/ansible/ansible/pull/83965).
- service_facts module got freebsd support added.
- ssh connection plugin - Support ``SSH_ASKPASS`` mechanism to provide passwords, making it the default, but still offering an explicit choice to use ``sshpass`` (https://github.com/ansible/ansible/pull/83936)
- ssh connection plugin now overrides pipelining when a tty is requested.
- ssh-agent - ``ansible``, ``ansible-playbook`` and ``ansible-console`` are capable of spawning or reusing an ssh-agent, allowing plugins to interact with the ssh-agent. Additionally a pure python ssh-agent client has been added, enabling easy interaction with the agent. The ssh connection plugin contains new functionality via ``ansible_ssh_private_key`` and ``ansible_ssh_private_key_passphrase``, for loading an SSH private key into the agent from a variable.
- templating - Access to an undefined variable from inside a lookup, filter, or test (which raises MarkerError) no longer ends processing of the current template. The triggering undefined value is returned as the result of the offending plugin invocation, and the template continues to execute.
- templating - Embedding ``range()`` values in containers such as lists will result in an error on use. Previously the value would be converted to a string representing the range parameters, such as ``range(0, 3)``.
- templating - Handling of omitted values is now a first-class feature of the template engine, and is usable in all Ansible Jinja template contexts. Any template that resolves to ``omit`` is automatically removed from its parent container during templating.
- templating - Template evaluation is lazier than in previous versions. Template expressions which resolve only portions of a data structure no longer result in the entire structure being templated.
- templating - Templating errors now provide more information about both the location and context of the error, especially for deeply-nested and/or indirected templating scenarios.
- templating - Unified ``omit`` behavior now requires that plugins calling ``Templar.template()`` handle cases where the entire template result is omitted, by catching the ``AnsibleValueOmittedError`` that is raised. Previously, this condition caused a randomly-generated string marker to appear in the template result.
- templating - Variables of type ``set`` and ``tuple`` are now converted to ``list`` when exiting the final pass of templating.
- to_json / to_nice_json filters - The filters accept a ``profile`` argument, which defaults to ``tagless``.
- troubleshooting - Tracebacks can be collected and displayed for most errors, warnings, and deprecation warnings (including those generated by modules). Tracebacks are no longer enabled with ``-vvv``; the behavior is directly configurable via the ``DISPLAY_TRACEBACK`` config option. Module tracebacks passed to ``fail_json`` via the ``exception`` kwarg will not be included in the task result unless error tracebacks are configured.
- undef jinja function - The ``undef`` jinja function now raises an error if a non-string hint is given. Attempting to use an undefined hint also results in an error, ensuring incorrect use of the function can be distinguished from the function's normal behavior.
- validate-modules sanity test - make sure that ``module`` and ``plugin````seealso`` entries use FQCNs (https://github.com/ansible/ansible/pull/84325).
- vault - improved vault filter documentation by adding missing example content for dump_template_data.j2, refining examples for clarity, and ensuring variable consistency (https://github.com/ansible/ansible/issues/83583).
- warnings - All warnings (including deprecation warnings) issued during a task's execution are now accessible via the ``warnings`` and ``deprecations`` keys on the task result.
- when the ``dict`` lookup is given a non-dict argument, show the value of the argument and its type in the error message.
- windows - add hard minimum limit for PowerShell to 5.1. Ansible dropped support for older versions of PowerShell in the 2.16 release but this reqirement is now enforced at runtime.
- windows - refactor windows exec runner to improve efficiency and add better error reporting on failures.
- winrm - Remove need for pexpect on macOS hosts when using ``kinit`` to retrieve the Kerberos TGT. By default the code will now only use the builtin ``subprocess`` library which should handle issues with select and a high fd count and also simplify the code.
Breaking Changes / Porting Guide
--------------------------------
- Support for the ``toml`` library has been removed from TOML inventory parsing and dumping. Use ``tomli`` for parsing on Python 3.10. Python 3.11 and later have built-in support for parsing. Use ``tomli-w`` to support outputting inventory in TOML format.
- assert - The ``quiet`` argument must be a commonly-accepted boolean value. Previously, unrecognized values were silently treated as False.
- conditionals - Conditional expressions that result in non-boolean values are now an error by default. Such results often indicate unintentional use of templates where they are not supported, resulting in a conditional that is always true. When this option is enabled, conditional expressions which are a literal ``None`` or empty string will evaluate as true, for backwards compatibility. The error can be temporarily changed to a deprecation warning by enabling the ``ALLOW_BROKEN_CONDITIONALS`` config option.
- first_found lookup - When specifying ``files`` or ``paths`` as a templated list containing undefined values, the undefined list elements will be discarded with a warning. Previously, the entire list would be discarded without any warning.
- internals - The ``AnsibleLoader`` and ``AnsibleDumper`` classes for working with YAML are now factory functions and cannot be extended.
- internals - The ``ansible.utils.native_jinja`` Python module has been removed.
- inventory - Invalid variable names provided by inventories result in an inventory parse failure. This behavior is now consistent with other variable name usages throughout Ansible.
- lookup plugins - Lookup plugins called as `with_(lookup)` will no longer have the `_subdir` attribute set.
- lookup plugins - ``terms`` will always be passed to ``run`` as the first positional arg, where previously it was sometimes passed as a keyword arg when using ``with_`` syntax.
- loops - Omit placeholders no longer leak between loop item templating and task templating. Previously, ``omit`` placeholders could remain embedded in loop items after templating and be used as an ``omit`` for task templating. Now, values resolving to ``omit`` are dropped immediately when loop items are templated. To turn missing values into an ``omit`` for task templating, use ``| default(omit)``. This solution is backward-compatible with previous versions of ansible-core.
- modules - Ansible modules using ``sys.excepthook`` must use a standard ``try/except`` instead.
- plugins - Any plugin that sources or creates templates must properly tag them as trusted.
- plugins - Custom Jinja plugins that accept undefined top-level arguments must opt in to receiving them.
- plugins - Custom Jinja plugins that use ``environment.getitem`` to retrieve undefined values will now trigger a ``MarkerError`` exception. This exception must be handled to allow the plugin to return a ``Marker``, or the plugin must opt-in to accepting ``Marker`` values.
- public API - The ``ansible.vars.fact_cache.FactCache`` wrapper has been removed.
- serialization of ``omit`` sentinel - Serialization of variables containing ``omit`` sentinels (e.g., by the ``to_json`` and ``to_yaml`` filters or ``ansible-inventory``) will fail if the variable has not completed templating. Previously, serialization succeeded with placeholder strings emitted in the serialized output.
- set_fact - The string values "yes", "no", "true" and "false" were previously converted (ignoring case) to boolean values when not using Jinja2 native mode. Since Jinja2 native mode is always used, this conversion no longer occurs. When boolean values are required, native boolean syntax should be used where variables are defined, such as in YAML. When native boolean syntax is not an option, the ``bool`` filter can be used to parse string values into booleans.
- template lookup - The ``convert_data`` option is deprecated and no longer has any effect. Use the ``from_json`` filter on the lookup result instead.
- templating - Access to ``_`` prefixed attributes and methods, and methods with known side effects, is no longer permitted. In cases where a matching mapping key is present, the associated value will be returned instead of an error. This increases template environment isolation and ensures more consistent behavior between the ``.`` and ``[]`` operators.
- templating - Conditionals and lookups which use embedded inline templates in Jinja string constants now display a warning. These templates should be converted to their expression equivalent.
- templating - Many Jinja plugins (filters, lookups, tests) and methods previously silently ignored undefined inputs, which often masked subtle errors. Passing an undefined argument to a Jinja plugin or method that does not declare undefined support now results in an undefined value.
- templating - Templates are always rendered in Jinja2 native mode. As a result, non-string values are no longer automatically converted to strings.
- templating - Templates resulting in ``None`` are no longer automatically converted to an empty string.
- templating - Templates with embedded inline templates that were not contained within a Jinja string constant now result in an error, as support for multi-pass templating was removed for security reasons. In most cases, such templates can be easily rewritten to avoid the use of embedded inline templates.
- templating - The ``allow_unsafe_lookups`` option no longer has any effect. Lookup plugins are responsible for tagging strings containing templates to allow evaluation as a template.
- templating - The result of the ``range()`` global function cannot be returned from a template- it should always be passed to a filter (e.g., ``random``). Previously, range objects returned from an intermediate template were always converted to a list, which is inconsistent with inline consumption of range objects.
- templating - ``#jinja2:`` overrides in templates with invalid override names or types are now templating errors.
Deprecated Features
-------------------
- CLI - The ``--inventory-file`` option alias is deprecated. Use the ``-i`` or ``--inventory`` option instead.
- Stategy Plugins - Use of strategy plugins not provided in ``ansible.builtin`` are deprecated and do not carry any backwards compatibility guarantees going forward. A future release will remove the ability to use external strategy plugins. No alternative for third party strategy plugins is currently planned.
-``ansible.module_utils.compat.datetime`` - The datetime compatibility shims are now deprecated. They are scheduled to be removed in ``ansible-core`` v2.21. This includes ``UTC``, ``utcfromtimestamp()`` and ``utcnow`` importable from said module (https://github.com/ansible/ansible/pull/81874).
- bool filter - Support for coercing unrecognized input values (including None) has been deprecated. Consult the filter documentation for acceptable values, or consider use of the ``truthy`` and ``falsy`` tests.
- cache plugins - The `ansible.plugins.cache.base` Python module is deprecated. Use `ansible.plugins.cache` instead.
- callback plugins - The `v2_on_any` callback method is deprecated. Use specific callback methods instead.
- callback plugins - The v1 callback API (callback methods not prefixed with `v2_`) is deprecated. Use `v2_` prefixed methods instead.
- conditionals - Conditionals using Jinja templating delimiters (e.g., ``{{``, ``{%``) should be rewritten as expressions without delimiters, unless the entire conditional value is a single template that resolves to a trusted string expression. This is useful for dynamic indirection of conditional expressions, but is limited to trusted literal string expressions.
- config - The ``ACTION_WARNINGS`` config has no effect. It previously disabled command warnings, which have since been removed.
- config - The ``DEFAULT_JINJA2_NATIVE`` option has no effect. Jinja2 native mode is now the default and only option.
- config - The ``DEFAULT_NULL_REPRESENTATION`` option has no effect. Null values are no longer automatically converted to another value during templating of single variable references.
- display - The ``Display.get_deprecation_message`` method has been deprecated. Call ``Display.deprecated`` to display a deprecation message, or call it with ``removed=True`` to raise an ``AnsibleError``.
- file loading - Loading text files with ``DataLoader`` containing data that cannot be decoded under the expected encoding is deprecated. In most cases the encoding must be UTF-8, although some plugins allow choosing a different encoding. Previously, invalid data was silently wrapped in Unicode surrogate escape sequences, often resulting in later errors or other data corruption.
- first_found lookup - Splitting of file paths on ``,;:`` is deprecated. Pass a list of paths instead. The ``split`` method on strings can be used to split variables into a list as needed.
- interpreter discovery - The ``auto_legacy`` and ``auto_legacy_silent`` options for ``INTERPRETER_PYTHON`` are deprecated. Use ``auto`` or ``auto_silent`` options instead, as they have the same effect.
- oneline callback - The ``oneline`` callback and its associated ad-hoc CLI args (``-o``, ``--one-line``) are deprecated.
- paramiko - The paramiko connection plugin has been deprecated with planned removal in 2.21.
- playbook variables - The ``play_hosts`` variable has been deprecated, use ``ansible_play_batch`` instead.
- plugin error handling - The ``AnsibleError`` constructor arg ``suppress_extended_error`` is deprecated. Using ``suppress_extended_error=True`` has the same effect as ``show_content=False``.
- template lookup - The jinja2_native option is no longer used in the Ansible Core code base. Jinja2 native mode is now the default and only option.
- templating - Support for enabling Jinja2 extensions (not plugins) has been deprecated.
- templating - The ``ansible_managed`` variable available for certain templating scenarios, such as the ``template`` action and ``template`` lookup has been deprecated. Define and use a custom variable instead of relying on ``ansible_managed``.
- templating - The ``disable_lookups`` option has no effect, since plugins must be updated to apply trust before any templating can be performed.
- to_yaml/to_nice_yaml filters - Implicit YAML dumping of vaulted value ciphertext is deprecated. Set `dump_vault_tags` to explicitly specify the desired behavior.
- tree callback - The ``tree`` callback and its associated ad-hoc CLI args (``-t``, ``--tree``) are deprecated.
Removed Features (previously deprecated)
----------------------------------------
- Remove deprecated plural form of collection path (https://github.com/ansible/ansible/pull/84156).
- encrypt - passing unsupported passlib hashtype now raises AnsibleFilterError.
- manager - remove deprecated include_delegate_to parameter from get_vars API.
- modules - Modules returning non-UTF8 strings now result in an error. The ``MODULE_STRICT_UTF8_RESPONSE`` setting can be used to disable this check.
- removed deprecated pycompat24 and compat.importlib.
- selector - remove deprecated compat.selector related files (https://github.com/ansible/ansible/pull/84155).
- windows - removed common module functions ``ConvertFrom-AnsibleJson``, ``Format-AnsibleException`` from Windows modules as they are not used and add uneeded complexity to the code.
Security Fixes
--------------
- include_vars action - Ensure that result masking is correctly requested when vault-encrypted files are read. (CVE-2024-8775)
- task result processing - Ensure that action-sourced result masking (``_ansible_no_log=True``) is preserved. (CVE-2024-8775)
- templating - Ansible's template engine no longer processes Jinja templates in strings unless they are marked as coming from a trusted source. Untrusted strings containing Jinja template markers are ignored with a warning. Examples of trusted sources include playbooks, vars files, and many inventory sources. Examples of untrusted sources include module results and facts. Plugins which have not been updated to preserve trust while manipulating strings may inadvertently cause them to lose their trusted status.
- templating - Changes to conditional expression handling removed numerous instances of insecure multi-pass templating (which could result in execution of untrusted template expressions).
- user action won't allow ssh-keygen, chown and chmod to run on existing ssh public key file, avoiding traversal on existing symlinks (CVE-2024-9902).
Bugfixes
--------
- Ansible will now also warn when reserved keywords are set via a module (set_fact, include_vars, etc).
- Ansible.Basic - Fix ``required_if`` check when the option value to check is unset or set to null.
- Correctly return ``False`` when using the ``filter`` and ``test`` Jinja tests on plugin names which are not filters or tests, respectively. (resolves issue https://github.com/ansible/ansible/issues/82084)
- Do not run implicit ``flush_handlers`` meta tasks when the whole play is excluded from the run due to tags specified.
- Errors now preserve stacked error messages even when YAML is involved.
- Fix a display.debug statement with the wrong param in _get_diff_data() method
- Fix disabling SSL verification when installing collections and roles from git repositories. If ``--ignore-certs`` isn't provided, the value for the ``GALAXY_IGNORE_CERTS`` configuration option will be used (https://github.com/ansible/ansible/issues/83326).
- Fix ipv6 pattern bug in lib/ansible/parsing/utils/addresses.py (https://github.com/ansible/ansible/issues/84237)
- Fix returning 'unreachable' for the overall task result. This prevents false positives when a looped task has unignored unreachable items (https://github.com/ansible/ansible/issues/84019).
- Implicit ``meta: flush_handlers`` tasks now have a parent block to prevent potential tracebacks when calling methods like ``get_play()`` on them internally.
- Improve performance on large inventories by reducing the number of implicit meta tasks.
- Jinja plugins - Errors raised will always be derived from ``AnsibleTemplatePluginError``.
- Optimize the way tasks from within ``include_tasks``/``include_role`` are inserted into the play.
- Time out waiting on become is an unreachable error (https://github.com/ansible/ansible/issues/84468)
- Use consistent multiprocessing context for action write locks
- Use the requested error message in the ansible.module_utils.facts.timeout timeout function instead of hardcoding one.
- Windows - add support for running on system where WDAC is in audit mode with ``Dynamic Code Security`` enabled.
- YAML parsing - The `!unsafe` tag no longer coerces non-string scalars to strings.
-``ansible-galaxy`` — the collection dependency resolver now treats version specifiers starting with ``!=`` as unpinned.
-``package``/``dnf`` action plugins - provide the reason behind the failure to gather the ``ansible_pkg_mgr`` fact to identify the package backend
- action plugins - Action plugins that raise unhandled exceptions no longer terminate playbook loops. Previously, exceptions raised by an action plugin caused abnormal loop termination and loss of loop iteration results.
- ansible-config - format galaxy server configs while dumping in JSON format (https://github.com/ansible/ansible/issues/84840).
- ansible-doc - If none of the files in files exists, path will be undefined and a direct reference will throw an UnboundLocalError (https://github.com/ansible/ansible/pull/84464).
- ansible-galaxy - Small adjustments to URL building for ``download_url`` and relative redirects.
- ansible-pull change detection will now work independently of callback or result format settings.
- ansible-test - Enable the ``sys.unraisablehook`` work-around for the ``pylint`` sanity test on Python 3.11. Previously the work-around was only enabled for Python 3.12 and later. However, the same issue has been discovered on Python 3.11.
- ansible-test - Ensure CA certificates are installed on managed FreeBSD instances.
- ansible-test - Fix support for PowerShell module_util imports with the ``-Optional`` flag.
- ansible-test - Fix support for detecting PowerShell modules importing module utils with the newer ``#AnsibleRequires`` format.
- ansible-test - Fix traceback that occurs after an interactive command fails.
- ansible-test - Fix up coverage reporting to properly translate the temporary path of integration test modules to the expected static test module path.
- ansible-test - Fixed traceback when handling certain YAML errors in the ``yamllint`` sanity test.
- ansible-test - Managed macOS instances now use the ``sudo_chdir`` option for the ``sudo`` become plugin to avoid permission errors when dropping privileges.
- ansible-vault will now correctly handle `--prompt`, previously it would issue an error about stdin if no 2nd argument was passed
- ansible_uptime_second - added ansible_uptime_seconds fact support for AIX (https://github.com/ansible/ansible/pull/84321).
- apt_key module - prevent tests from running when apt-key was removed
- build - Pin ``wheel`` in ``pyproject.toml`` to ensure compatibility with supported ``setuptools`` versions.
- config - various fixes to config lookup plugin (https://github.com/ansible/ansible/pull/84398).
- copy - refactor copy module for simplicity.
- copy action now prevents user from setting internal options.
- debconf - set empty password values (https://github.com/ansible/ansible/issues/83214).
- debug - hide loop vars in debug var display (https://github.com/ansible/ansible/issues/65856).
- default callback - Error context is now shown for failing tasks that use the ``debug`` action.
- display - The ``Display.deprecated`` method once again properly handles the ``removed=True`` argument (https://github.com/ansible/ansible/issues/82358).
- distro - add support for Linux Mint Debian Edition (LMDE) (https://github.com/ansible/ansible/issues/84934).
- distro - detect Debian as os_family for LMDE 6 (https://github.com/ansible/ansible/issues/84934).
- dnf5 - Handle forwarded exceptions from dnf5-5.2.13 where a generic ``RuntimeError`` was previously raised
- dnf5 - fix ``is_installed`` check for packages that are not installed but listed as provided by an installed package (https://github.com/ansible/ansible/issues/84578)
- dnf5 - fix installing a package using ``state=latest`` when a binary of the same name as the package is already installed (https://github.com/ansible/ansible/issues/84259)
- dnf5 - fix traceback when ``enable_plugins``/``disable_plugins`` is used on ``python3-libdnf5`` versions that do not support this functionality
- dnf5 - libdnf5 - use ``conf.pkg_gpgcheck`` instead of deprecated ``conf.gpgcheck`` which is used only as a fallback
- dnf5 - matching on a binary can be achieved only by specifying a full path (https://github.com/ansible/ansible/issues/84334)
- facts - gather pagesize and calculate respective values depending upon architecture (https://github.com/ansible/ansible/issues/84773).
- facts - skip if distribution file path is directory, instead of raising error (https://github.com/ansible/ansible/issues/84006).
- find - skip ENOENT error code while recursively enumerating files. find module will now be tolerant to race conditions that remove files or directories from the target it is currently inspecting. (https://github.com/ansible/ansible/issues/84873).
- first_found lookup - Corrected return value documentation to reflect None (not empty string) for no files found.
- gather_facts action now defaults to `ansible.legacy.setup` if `smart` was set, no network OS was found and no other alias for `setup` was present.
- gather_facts action will now issues errors and warnings as appropriate if a network OS is detected but no facts modules are defined for it.
- gather_facts action, will now add setup when 'smart' appears with other modules in the FACTS_MODULES setting (#84750).
- get_url - add support for BSD-style checksum digest file (https://github.com/ansible/ansible/issues/84476).
- get_url - fix honoring ``filename`` from the ``content-disposition`` header even when the type is ``inline`` (https://github.com/ansible/ansible/issues/83690)
- host_group_vars - fixed defining the 'key' variable if the get_vars method is called with cache=False (https://github.com/ansible/ansible/issues/84384)
- include_vars - fix including previously undefined hash variables with hash_behaviour merge (https://github.com/ansible/ansible/issues/84295).
- iptables - Allows the wait parameter to be used with iptables chain creation (https://github.com/ansible/ansible/issues/84490)
- linear strategy - fix executing ``end_role`` meta tasks for each host, instead of handling these as implicit run_once tasks (https://github.com/ansible/ansible/issues/84660).
- local connection plugin - Become timeout errors now include all received data. Previously, the most recently-received data was discarded.
- local connection plugin - Ensure ``become`` success validation always occurs, even when an active plugin does not set ``prompt``.
- local connection plugin - Fixed cases where the internal ``BECOME-SUCCESS`` message appeared in task output.
- local connection plugin - Fixed hang or spurious failure when data arrived concurrently on stdout and stderr during a successful ``become`` operation validation.
- local connection plugin - Fixed hang when a become plugin expects a prompt but a password was not provided.
- local connection plugin - Fixed hang when an active become plugin incorrectly signals lack of prompt.
- local connection plugin - Fixed hang when an internal become read timeout expired before the password prompt was written.
- local connection plugin - Fixed hang when only one of stdout or stderr was closed by the ``become_exe`` subprocess.
- local connection plugin - Fixed long timeout/hang for ``become`` plugins that repeat their prompt on failure (e.g., ``sudo``, some ``su`` implementations).
- local connection plugin - Fixed silent ignore of ``become`` failures and loss of task output when data arrived concurrently on stdout and stderr during ``become`` operation validation.
- local connection plugin - Fixed task output header truncation when post-become data arrived before ``become`` operation validation had completed.
- lookup plugins - The ``terms`` arg to the ``run`` method is now always a list. Previously, there were cases where a non-list could be received.
- module arg templating - When using a templated raw task arg and a templated ``args`` keyword, args are now merged. Previously use of templated raw task args silently ignored all values from the templated ``args`` keyword.
- module defaults - Module defaults are no longer templated unless they are used by a task that does not override them. Previously, all module defaults for all modules were templated for every task.
- module respawn - limit to supported Python versions
- package_facts module when using 'auto' will return the first package manager found that provides an output, instead of just the first one, as this can be foreign and not have any packages.
- psrp - Improve stderr parsing when running raw commands that emit error records or stderr lines.
- regex_search filter - Corrected return value documentation to reflect None (not empty string) for no match.
- respawn - use copy of env variables to update existing PYTHONPATH value (https://github.com/ansible/ansible/issues/84954).
- runas become - Fix up become logic to still get the SYSTEM token with the most privileges when running as SYSTEM.
- sequence lookup - sequence query/lookups without positional arguments now return a valid list if their kwargs comprise a valid sequence expression (https://github.com/ansible/ansible/issues/82921).
- service_facts - skip lines which does not contain service names in openrc output (https://github.com/ansible/ansible/issues/84512).
- ssh - Improve the logic for parsing CLIXML data in stderr when working with Windows host. This fixes issues when the raw stderr contains invalid UTF-8 byte sequences and improves embedded CLIXML sequences.
- ssh - connection options were incorrectly templated during ``reset_connection`` tasks (https://github.com/ansible/ansible/pull/84238).
- stability - Fixed silent process failure on unhandled IOError/OSError under ``linear`` strategy.
- su become plugin - Ensure generated regex from ``prompt_l10n`` config values is properly escaped.
- su become plugin - Ensure that password prompts are correctly detected in the presence of leading output. Previously, this case resulted in a timeout or hang.
- su become plugin - Ensure that trailing colon is expected on all ``prompt_l10n`` config values.
- sudo become plugin - The `sudo_chdir` config option allows the current directory to be set to the specified value before executing sudo to avoid permission errors when dropping privileges.
- sunos - remove hard coding of virtinfo command in facts gathering code (https://github.com/ansible/ansible/pull/84357).
- to_yaml/to_nice_yaml filters - Eliminated possibility of keyword arg collisions with internally-set defaults.
- unarchive - Clamp timestamps from beyond y2038 to representible values when unpacking zip files on platforms that use 32-bit time_t (e.g. Debian i386).
- uri - Form location correctly when the server returns a relative redirect (https://github.com/ansible/ansible/issues/84540)
- uri - Handle HTTP exceptions raised while reading the content (https://github.com/ansible/ansible/issues/83794).
- uri - mark ``url`` as required (https://github.com/ansible/ansible/pull/83642).
- user - Create Buildroot subclass as alias to Busybox (https://github.com/ansible/ansible/issues/83665).
- user - Set timeout for passphrase interaction.
- user - Update prompt for SSH key passphrase (https://github.com/ansible/ansible/issues/84484).
- user - Use higher precedence HOME_MODE as UMASK for path provided (https://github.com/ansible/ansible/pull/84482).
- user action will now require O(force) to overwrite the public part of an ssh key when generating ssh keys, as was already the case for the private part.
- user module now avoids changing ownership of files symlinked in provided home dir skeleton
- vars lookup - The ``default`` substitution only applies when trying to look up a variable which is not defined. If the variable is defined, but templates to an undefined value, the ``default`` substitution will not apply. Use the ``default`` filter to coerce those values instead.
- wait_for_connection - a warning was displayed if any hosts used a local connection (https://github.com/ansible/ansible/issues/84419)
- tasks - the ``retries`` keyword can be specified without ``until`` in which case the task is retried until it succeeds but at most ``retries`` times (https://github.com/ansible/ansible/issues/20802)
- modules/user.py - Add check for valid directory when creating new user homedir (allows /dev/null as skeleton) (https://github.com/ansible/ansible/issues/75063)
- module/role argument spec - validate the type for options that are None when the option is required or has a non-None default (https://github.com/ansible/ansible/issues/79656).
- iptables - remove default rule creation when creating iptables chain to be more similar to the command line utility (https://github.com/ansible/ansible/issues/80256).
- "validate-modules sanity test - replace semantic markup parsing and validating code with the code from `antsibull-docs-parser 0.2.0 <https://github.com/ansible-community/antsibull-docs-parser/releases/tag/0.2.0>`__ (https://github.com/ansible/ansible/pull/80406)."
- The ``hostname`` module now also updates both current and permanent hostname on OpenBSD. Before it only updated the permanent hostname (https://github.com/ansible/ansible/issues/80520).
- "Properly disable ``jinja2_native`` in the template module when jinja2 override is used in the template (https://github.com/ansible/ansible/issues/80605)"
- unarchive - fix unarchiving sources that are copied to the remote node using a relative temporory directory path (https://github.com/ansible/ansible/issues/80710).
- handlers - the ``listen`` keyword can affect only one handler with the same name, the last one defined as it is a case with the ``notify`` keyword (https://github.com/ansible/ansible/issues/81013)
- "paramiko_ssh, psrp, and ssh connection plugins - ensure that all values for options that should be strings are actually converted to strings (https://github.com/ansible/ansible/pull/81029)."
- "ansible-console - fix filtering by collection names when a collection search path was set (https://github.com/ansible/ansible/pull/81450)."
minor_changes:
- "ansible-doc - allow to filter listing of collections and metadata dump by more than one collection (https://github.com/ansible/ansible/pull/81450)."
- ansible-galaxy - used to crash out with a "Errno 20 Not a directory" error when extracting files from a role when hitting a file with an illegal name (https://github.com/ansible/ansible/pull/81553). Now it gives a warning identifying the culprit file and the rule violation (e.g., ``my$class.jar`` has a ``$`` in the name) before crashing out, giving the user a chance to remove the invalid file and try again. (https://github.com/ansible/ansible/pull/81555).
known_issues:
- ansible-galaxy - dies in the middle of installing a role when that role contains Java inner classes (files with $ in the file name). This is by design, to exclude temporary or backup files. (https://github.com/ansible/ansible/pull/81553).
- Replace uses of ``configparser.ConfigParser.readfp()`` which was removed in Python 3.12 with ``configparser.ConfigParser.read_file()`` (https://github.com/ansible/ansible/issues/81656)
- get_url - fix honoring ``filename`` from the ``content-disposition`` header even when the type is ``inline`` (https://github.com/ansible/ansible/issues/83690)
- ssh connection plugin - Support ``SSH_ASKPASS`` mechanism to provide passwords, making it the default, but still offering an explicit choice to use ``sshpass``
(https://github.com/ansible/ansible/pull/83936)
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.