* Do not use YAML parsing/serialization for .deps files.
* Prevent crash.
(cherry picked from commit 9545f2e0de)
Co-authored-by: Felix Fontein <felix@fontein.de>
* Don't assume column index for netmask and broadcast. Fixes#79117
* fix typo
(cherry picked from commit f53dbf9)
Co-authored-by: Matt Martz <matt@sivel.net>
* [stable-2.14] Ensure that we do not squash keywords in validate (#79049)
* Ensure that we do not squash keywords in validate. Fixes#79021
* become_user: nobody should only apply to the test tasks, not the setup_test_user role
* Update how become_user is specified
* Add test to ensure keyword inheritance is working for become
* Add clog frag
* Cache fattributes to prevent re-calculation
* ci_complete
* Remove unnecessary getattr.
(cherry picked from commit 420564c5bc)
Co-authored-by: Matt Martz <matt@sivel.net>
* Remove unneeded merge conflict changes
* omit keyword should reset to context (#78917)
* omit keyword should reset to context
ensure we use context/inheritance when calculating value,
using default only when context is unavailable.
fixes#75692
(cherry picked from commit 9650ddb11c)
* fixes to FA inheritance (#78990)
finalized applies to all field attributes
fix getting parent value
also remove unused/needed extend/prepend signature
moar testing
(cherry picked from commit ff6e4da36a)
* setup role needs it's own info
* Use python re to parse service output instead of grep. Fixes#78541
* Add clog frag
* Add an extra guard to abort if rc is 4, and /etc/init.d is missing
(cherry picked from commit 4458128)
Co-authored-by: Matt Martz <matt@sivel.net>
* Do not crash templating when filter/test name is not a valid Ansible plugin name.
* Store and re-raise KeyError if there was one.
Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
(cherry picked from commit 6d0aeac1e1)
* Bump antsibull-docs version to 1.6.0 to fully support filter docs.
* known_good_reqs.txt no longer exists.
* Bump version to 1.6.1.
(cherry picked from commit d93883645c)
Co-authored-by: Felix Fontein <felix@fontein.de>
* iss#72389: Adds note and polishes docs/docsite/rst/tips_tricks/sample_setup.rst (#78858)
Co-authored-by: Jaroslav Klech <jklech@redhat.com>
(cherry picked from commit 964e2bc6fe)
* Fixes typo in rst format (#78884)
.. hint:: requires a blank line above and below.
(cherry picked from commit 71adb02142)
* Update yum.py (#78829)
##### SUMMARY
The entire yum command should be highlighted, not only its options.
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
(cherry picked from commit f8f76c5886)
* Add Ansible community 7.x porting guide (#78916)
(cherry picked from commit 0884a7cf0e)
* Fixed spelling (#78911)
(cherry picked from commit ae66d9a1f4)
* Add message about viewing chat history when on IRC (#78834)
(cherry picked from commit 0343ae46d2)
Co-authored-by: JaroslavKlech <klechh@gmail.com>
Co-authored-by: Anwesha Das <anwesha@das.community>
Co-authored-by: Olivier Clavel <olivier.clavel@gmail.com>
Co-authored-by: Christian Adams <chadams@redhat.com>
Co-authored-by: Joel McGuire <jlmcguire4@gmail.com>
Co-authored-by: Greg "Gwmngilfen" Sutcliffe <github@emeraldreverie.org>
* Fix known_hosts changed status when removing non-existing key (#78748)
* Ensure no change is reported when removing missing key for host with other keys
Co-authored-by: Matt Martz <matt@sivel.net>
* Rename changelog fragment for fix known hosts changed status cherry pick
* Rename changelog according to new PR number
Co-authored-by: Matt Martz <matt@sivel.net>
- Allow disabled, unsupported, unstable and destructive integration test targets to be selected using their respective prefixes.
- Allow unstable tests to run when targeted changes are made and the ``--allow-unstable-changed`` option is specified (resolves https://github.com/ansible/ansible/issues/74213).
(cherry picked from commit d3d7785472)
Co-authored-by: Matt Clay <matt@mystile.com>
- Update sanity test requirements to fully support Python 3.11.
- Make Python 3.11 the default in the base and default test containers.
(cherry picked from commit 4d25233ece)
* Add --offline option to 'ansible-galaxy collection install' to prevent querying distribution servers
This allows installing/upgrading individual tarfiles to have dependency resolution.
Previously needed to be done manually with --no-deps or else all collections and dependencies needed to be included in the requirements.
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Add OSMC to Debian OS_FAMILY_MAP
- os_family fact of the Debian-based OSMC distribution was not detected correctly
* tweak changelog
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
* new _fqcn attribute to plugin objects
* unbreak plugins in subdirs
* Fix inadventent changes to _load_name and use existing vars
* add plugin aliases and name property, and replace plugin._load_name where incorrect
* Fix listing plugin names
Fix listing legacy and builtin together
test deprecated plugin documentation
fix doc extensions
remove sometimes inaccurate _load_name handling from plugin.name
* Add tests for REJECT_EXTS and doc extensions
Fix unpredictable collection redirects so non-fqcns in the redirect list are guaranteed to be legacy (instead of determined by the collections keyword)
Move aliases and name properties to _update_object so all plugin types, including doc fragments, can use them
* make legacy plugin names internally consistent
* rename attributes to ansible_name and ansible_aliases
This fixes bug which prevents identification of broken playbook when
passing multiple playbooks as arguments, something that is common
especially with `--syntax-check`.
Affects: ansible-lint
* Recover unix load average into an ansible_loadavg fact
The average load is available out of the box with os.getloadavg().
It adds useful context to the existing ansible_memory_mb and ansible_mounts
facts that provide size and utilization or memory and disk space.
* Add an integration test
* Add remote platform symlinks.
These were already supported by ansible-test.
This change adds the symlinks for use in the CI test matrix.
* Remove obsolete AIX test symlink.
AIX was already removed from ansible-test.
* Fix distro fact handling for Flatcar
The existence of the file /etc/flatcar/update.conf depends on
bootstrap configuration typically provided by the user. For that
reason this file is unsuitable for determining distro facts for
Flatcar Container Linux.
The distribution_release fact is meaningless in the case of Flatcar
since Flatcar doesn't have named releases. The distribution_version
fact, however, IS meaningful and should contain a number such as
"3139.2.0".
- Use /etc/os-release instead of /etc/flatcar/update.conf.
- Drop the distribution_release fact.
- Set the distribution_version fact.
- Update distro test fixture for Flatcar
- Generate the fixture using gen_distribution_version_testcase.py.
- Override result.distribution and result.os_family manually as the
generator script gives wrong values.
- Use a recent Flatcar version.
Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
* Limit Galaxy API calls during ansible-galaxy collection dependency resolution when possible
Installing a tarfile with a dependency from a Galaxy server (e.g. dependencies: {'ns.coll': '>=1.0.0'}) does not get the available versions of the dependency from the galaxy server if a sufficient version is already installed.
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* refactor and remove redundant code in documentation
allow location and building api to be more accessible
fix issues with displaying ansible.legacy and ansible.builtin
ensure we don't x2 process tokens (some modules reference them also) fixes#77764
move to constants vs hardcoded
more informative errors and comments
now have actual filter/test plugins, which expose the filter/test functions
moved filter/test loading/finding logic into jinja2pluginloader, removed dupe implementations
added tests for case in which we unique by basename when listing
Update lib/ansible/utils/plugin_docs.py
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* Add a warning for collections that are attempting to be autoloaded to no effect
* Deprecate REQUIRES_WHITELIST and add support for REQUIRES_ENABLED so the docs are accurate
* Fix documentation
* add more vars plugin tests
* Simplify code and add a FIXME for another bug
* fix precedence
* Make setting the class attr at all a warning, even if it's True
* Add fun parsing for _load_name
* include _load_name in messages
This specific integration test gets stuck periodically causing the
Galaxy jobs to be killed on timeout wasting an hour of runtime. The
module that gets stuck waiting on Pulp is an in-test one, called
`setup_collections`. When it works, the task is complete in around 70
seconds but when it doesn't, it just freezes the whole play.
This patch attempts to make it fail faster by putting a reasonable
timeout value of 2 minutes.
On BusyBox systems such as Alpine, chattr on a tmpfs fails with a status of 0 and output only on stderr.
This change updates the test to not assume output on stdout.
Sometimes pytest errors out with an `ImportError` during its tests
collection stage when a Python package/directory containing the test
module does not have an `__init__.py` in it. This is being observed
under Python 3.9 and higher.
The patch provides a workaround for this problem but does not address
the root cause which is currently unknown.
Ref:
https://github.com/ansible/ansible/pull/78585#issuecomment-1220885431
* add required file to collection skeleton
This file is required to be able to upload a collection.
It is present in
https://github.com/ansible-collections/collection_template/blob/main/meta/runtime.yml
but that does not get used by default.
Without this, if you use the "ansible-galaxy collection init" command
and you try and publish that collection without adding this file, you
get the error:
"ERROR! Galaxy import process failed: 'requires_ansible' in
meta/runtime.yml is mandatory, but no meta/runtime.yml found (Code:
UNKNOWN)"
Also updates relevant test and adds a changelog fragment
* yum_repository: Do not set default value for async
The `async` repository option is deprecated in RHEL 8, but Ansible sets
```ini
async = 1
```
even when it's omitted from the module options, which causes `dnf` to complain
about an unknown configuration option.
This commit removes the default value from the `async` parameter, which means
it won't be added to the repository file if omitted from the module parameters.
Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
* Document the specific versions with the default True and the option deprecation for RHEL 8
Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
Replace get_persistent_connection_options with get_options
Remove special case for network sub_plugin in _set_plugin_options
Try to avoid mock connection pretending to be persistent
Rename variables->options to reflect what they actually are
Gather options for ssh_type_conn on network_cli
Drop reliance on sub_plugin["type"]
* Relocate venv-pythons.py script.
* Split out unsupported-directory test.
* Split out sanity-import test.
* Split out sanity-validate-modules test.
* Split out units test.
* Split out integration test.
* Split out units-constraints test.
* Split out integration-constraints test.
* Split out coverage test.
* Split out sanity test.
* Split out git test.
* Update test groups.
* ansible-config init fixes
now handles --disabled correctly for 'vars' format
also does not display internal config entries anymore
fixes#78438
note plugins have _prefixed settings that are public, this should change
* Don't actually update the cache in check mode
Add tests for updating the cache in check mode
* Don't mark packages as manually installed in check mode and add a test for it
* changelog
* Update test/integration/targets/apt/tasks/apt.yml
* fix test
* update attributes of files that are symlinks
* update attributes of files that are hard links
* fix default state in documentation
* remove unnecessary suppression
* add to changelog
* ansible-test - Avoid use of deprecated type hints.
PEP 585 deprecated many container types in the `typing` module in favor of the actual types, which support subscripting as of Python 3.9.
Conversion of `t.Type` was skipped since PyCharm does not currently recognize it.
* ansible-test - Fix `t` and `c` imports/shadowing.
* Simple regex replace of multi-line function arg annotations on the first line.
* Manually fix up ArgumentParser type annotations.
* Manual type hint conversions.
* Manual conversion of function type hints.
* Remove unnecessary type hints on for statements.
* Simple regex replace of multi-line function arg annotations.
* Simple regex replace of multi-line function arg annotations with default values.
* Simple regex replace of multi-line function arg return annotations.
* Simple regex replace of assignment annotations.
* `context/target` tests must be in groups 1 - 2.
* `context/controller` tests must be in groups 3 - 5.
This makes it easier to efficiently organize groups and balance test runtimes.
* ansible-test - Use more native type hints.
Simple search and replace to switch from comments to native type hints for return types of functions with no arguments.
* ansible-test - Use more native type hints.
Conversion of simple single-line function annotation type comments to native type hints.
* ansible-test - Use more native type hints.
Conversion of single-line function annotation type comments with default values to native type hints.
* ansible-test - Use more native type hints.
Manual conversion of type annotation comments for functions which have pylint directives.
Now that core requires UTF-8 filesystem encoding, ansible-test does as well.
Additionally, the `en_US.UTF-8` or `C.UTF-8` encoding must be available.
Previously the `en_US.UTF-8` encoding was requested, but its availability was never verified.
The fallback to `C.UTF-8` maintains UTF-8 encoding while allowing more flexibility in the running environment.
* Rethread pr/70185 through the dependency resolver
Hang optional metadata toggle on the ConcreteArtifactsManager instead of threading it through whole list codepath
Don't error while listing collections if a collection's metadata is missing keys required for building a collection.
Give an informative warning if metadata has been badly formatted.
Co-authored-by: Sam Doran <sdoran@redhat.com>
The current language pack fails to install on Ubuntu 22.04.
However, since the langauge only needs to be non-English, changing it preserves the test functionality.
The code block is rendered verbatim. This currently results in `\*` being shown, which is incorrect (i.e., not parsed as a wildcard pattern).
Removing the superfluous `\` fixes the rendering.
This PR fixes all instances of this issue on that page.
* Report OpenSuSE >= 15 as opensuse
Make distro.id() report newer versions of OpenSuSE (at least >=15) also report
as opensuse. They report themselves as opensuse-leap.
* Add a test
* Fix KeyError for ansible-galaxy when caching paginated responses from v3
* changelog
* generate responses in loop for test
Co-authored-by: Matt Martz <matt@sivel.net>
* wait_for - Read file and perform comparisons using bytes to avoid decode errors. Fixes#78214
* Write non-ascii via script instead of static file
* Use contexlib.closing to support py27 context manager
* Use executable from task, instead of shebang
* Update encoded bytes to utf16
* Don't set become_flags in become integration test.
Setting the flags in the test overrides the default value in the become plugins, which was not intentional.
* Remove hack for broken implicit tilde support.
It was only broken because the tests were overriding the default become flags for sudo.
* Simplify assertions.
`group` and `setup_cron` integration tests currently install `libuser` and `faketime`
packages respectively, from the non-default edge/testing repositories. It is no
longer necessary to do so for the former so this patch drops the repo option. As
for the latter, this patch includes a note explaining when the same could be done
for it too.
This patch also adds a note on installing `faketime` on Alpine in CI.
Co-Authored-By: Matt Clay <matt@mystile.com>
* Add setup_test_user integration target.
This integration target creates a temporary test user and removes
it when the current integration target finishes. This user is
configured with a password so it can be used with become, even
when the current user is unprivileged.
* Fix slurp integration test.
* Fix unarchive integration test.
* Fix module_utils integration test.
- `processor_count` was erroneously set to the number of cores
- `processor_cores` was erroneously set to the number of threads per core
- `processor_vcpus` and `processor_threads_per_core` were not set
- `processor` was a string, while it's supposed to be a list
Before:
```
"ansible_processor": "PowerPC_POWER7",
"ansible_processor_cores": 4,
"ansible_processor_count": 12,
```
After:
```
"ansible_processor": [
"PowerPC_POWER7"
],
"ansible_processor_cores": 12,
"ansible_processor_count": 1,
"ansible_processor_threads_per_core": 4,
"ansible_processor_vcpus": 48,
```
Also add a unit test.
Co-authored-by: Baptiste Jonglez <git@bitsofnetworks.org>
* listify_lookup_plugin_terms deprecate dataloader
deprecated useless dataloader pass to function
also removed from callers in core
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
I'm not aware of a way to easily get vault secrets decoded on the
ansible-connection side without sending the vault secrets over the
connection in the same way, so just decode them for transport.
* Fix workding for :ref|term: subsitution
now matches 'seealso' and makes more sense:
```
delay:
applies_to:
- Task
description: Number of seconds to delay between retries. This setting is only used
in combination with `until`.
```
vs
```
delay:
applies_to:
- Task
description: Number of seconds to delay between retries. This setting is only used
in combination with website for `until`.
```
* updated unit tests
* match see also
* more sanity
* Update dpkg_selections.py to include an syntax example showing how to remove package holds
* Fix pep8 issue: dpkg_selections.py
Missed some trailing spaces when adding example text.
* Add context to inventory plugins documentation
* Show how to add the inventory plugin to the `enable_plugins` configuration item - not just replace it.
* Show adding tags to `keyed_groups` without specifying the separator.
* Show adding more complex `groups` and `compose` statements, including using the ternary filter.
* As I added an additional group, I've updated the resulting `ansible-inventory` output accordingly.
* Move undefined check from concat to finalize
In the classic Jinja2's Environment str() is called on the return value of the
finalize method to potentially trigger the undefined error. That is not
the case in NativeEnvironment where string conversion of the return value is
not desired. We workaround that by checking for Undefined in all of our concat
functions. It seems simpler to do it earlier in the finalize method(s) instead.
As a side-effect it fixes an undefined variable detection in imported templates.
Fixes#78156
ci_complete
* Fix sanity
* ...
* sigh
* [dnf] Fix skip_broken, add test coverage
Change:
- skip_broken was set in config but not actually used in calls to
base.install()
- added a lot of test cases with specialized repo
- got rid of external (docker repo) nobest test cases since the
specialized repo works well for those too
- Slight cleanup and adding comments in dnf module
Test Plan:
- ci_complete
Tickets:
- Fixes#73072
Original-author: Rick Elrod <rick@elrod.me>
* Use a better test for checking results list
ci_complete
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Rick Elrod <rick@elrod.me>
* Relocate update-ignore.py for easier re-use.
* Add script to ease collection testing.
* Skip ignore rewrite if file does not exist.
* Add integration test for the shebang sanity test.
* Fix ansible-test-no-tty integration test.
Previously the test only verified a TTY was not used if a TTY already existed.
This prevented the test from verifying behavior when run in CI.
Now the test creates a PTY before invoking ansible-test.
* Clean up ansible-test-docker integration test.
##### SUMMARY
The sample value given for the 'mode' parameter is shown without quotes, but the data type is string. If you actually try to use an unquoted numeric string for this value you're in for a nasty surprise! I added quotation marks to the sample value.
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
* ansible-galaxy configurable timeouts
- also fixed issues with precedence,
so --ignore-certs now overrides config
- made galaxy_timeout generic setting,
if set, it becomes default for server configs,
but now specific servers can override
- updated tests or added notes (some tests ignore/override precedence)
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* Use ctypes to access shadow, instead of spwd which is deprecated
* Remove sanity ignore
* cast to str to satisfy mypy
* Add typing.cast polyfill
* Appease mypy redefine
* Add clog frag
* Remove .travis.yml files from galaxy init skeletons
These files have existed for many years but in light of the recent
circumstances around Travis, we should not include them by default in
the galaxy init skeletons.
* Add breaking_changes changelog fragment with recommendation to use a custom skeleton if the .travis.yml file is still needed.
Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
* Fix 'Permission denied' in user module while generating SSH keys
Fix#78017
Use try/except for spwd usage to prevent "Permission denied".
Signed-off-by: Sagi Shnaidman <sshnaidm@redhat.com>
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
* facts: Add uptime fact for NetBSD
This is copied from the current OpenBSD facts code.
Tested on the GCC compile farm: https://cfarm.tetaneutral.net/
* Added changelog fragment
Co-authored-by: Baptiste Jonglez <git@bitsofnetworks.org>
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* Add configurable option for paramiko banner_ssh timeout
This PR adds a configurable option for paramiko connection
to control the value of banner timeout in paramiko library.
The banner timeout value can be set in ansible.cfg file as
The default value of banner timeout is set to 30 seconds in
paramiko connection plugin.
```
[paramiko_connection]
banner_timeout = 30
```
or using enviornment variable
```
export ANSIBLE_PARAMIKO_BANNER_TIMEOUT=30
```
* Fix CI failure
* Fix review comments
* Fix review comment
Add a new handler class ZipZArchive to use unzip -Z as an alternative to zipinfo
Run 'unzip -Z' in can_handle_archive so we fall back to the next handler if it's not available (failing in is_unarchived is too late)
* Add a test for unzip -Z when zipinfo is not available
* Update test for missing binary altogether by removing /usr/bin from the PATH
* ansible-galaxy - support resolvelib versions >= 0.5.3, <= 0.8.1
Test incompatibilities are removed for resolvelib >= 0.6.0
Test against the latest 0.8.x version and fix requirements
* Fix tests - use a venv for testing the range of resolvelib versions
* Update temporary hardcoded fallback for ansible-test
* Update hardcoded upperbound for sanity tests
* Make error check more flexible
checksum can also accept a checksum only file (no filename beside the checksum).
fixes#54390
Co-authored-by: Baptiste Mille-Mathias <baptiste.millemathias@gmail.com>
* Add RHEL 9.0 to CI.
* Restrict network manager inspection to RHEL8
* Skip module tests when astream_name is undefined, undefine it for RHEL9 until 9.1
* Remove redundant test.
Co-authored-by: Matt Martz <matt@sivel.net>
* Add find test to assert patterns with regex
* Add test to assert that nested includes with relative path work
* Add test to assert symmetric_difference supports unhashable types
* ci_complete ci_coverage
* Add test to assert JSON values are converted to Python
* Add test to assert how unexpected module failures behave
* Add test to verify remote_is_local
* ci_complete ci_coverage
* Get real path of local tmp
* ci_complete ci_coverage
* Expanded symmetric_difference tests
* ci_complete ci_coverage
* I don't even know
* ci_complete ci_coverage
* Revert mathstuff changes
* ci_complete ci_coverage
* apt: fix virtual package install version detection
Change 4a62c4e3e4 introduced version
matching in installation.
The problem stems from
if version_installable or version:
pkg_list.append("'%s=%s'" % (name, version_installable or version))
When the package is a virtual-package, package_status() is returning
the "version_installable" of the package *satisfying* the
virtual-package; but then this is trying to install the
virtual-package with this version pin.
For example, "yaml-mode" is a virtual package satisifed by
"elpa-yaml-mode" (currently 0.0.14-1) and trying to install it fails
with
$ usr/bin/apt-get -y ... install 'yaml-mode=0.0.14-1'
... failed: E: Version '0.0.14-1' for 'yaml-mode' was not found ...
In the case of a virtual-package with nothing installed to satisfy it,
we should just return blank values to allow apt-get to do it's thing.
The tests are updated to install and remove this package.
Fixes: #76779
* Fix traceback when a supported version of resolvelib is not installed
Try to read the supported version range from the package distribution info and fall back to a hardcoded lowerbound/upperbound (>=0.5.3,<0.6.0).
* Add tests for unsupported resolvelib versions
* Resolve remaining import sanity test issues.
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Martz <matt@sivel.net>
* If there is a platform specific handler, prefer the resolved module over the resolved action when loading module_defaults
Add a toggle for action plugins to prefer the resolved module when loading module_defaults
Allow moving away from modules intercepted as actions pattern
Fixes#77059
* winrm, psrps added missing var entry
this handles issue with the default being set to inventory_hostname
but defaults not being templated implicitly
fixes#77841
* Add intentional test coverage for lib/ansible/plugins/inventory/__init__.py
* Add intentional (integration) test coverage for lib/ansible/module_utils/common/network.py
* Add ansible_release info test from https://github.com/ansible/ansible/pull/74673
ci_complete
ci_coverage
Co-authored-by: Rick Elrod <rick@elrod.me>
* ansible-test - Add a Ubuntu 22.04 container.
* ansible-test - Add a Fedora 36 container.
* ansible-test - Update distro containers.
* Fix dnf test on Fedora 36.
* Work around scp test issues.
* Add new templating target, test splitter
* Add test for host:port parsing in parse_address via add_host
* Test already notified listening handler
* Add test for gathering bridge network facts
* Add veth to bridge
* ci_complete ci_coverage
The requirements for virtualenv and coverage are now kept in a requirements file for easier container builds.
The test-constraints sanity test has been updated to make sure the requirements file is kept up-to-date.
* rename systemd module to services only
disambiguates what it handles since systemd is now much more
that a service manager, but the module is specific to services
* ansible-test - Add shell --export option.
* ansible-test - Support cmd args for shell command.
Also allow shell to be used without a valid layout if no delegation is required.
* ansible-test - Improve stderr/stdout consistency.
By default all output goes to stdout only, with the exception of a fatal error.
When using any of the following, all output defaults to stderr instead:
* sanity with the `--lint` option -- sanity messages to stdout
* coverage analyze -- output to stdout if the output file is `/dev/stdout`
* shell -- shell output to stdout
This fixes issues two main issues:
* Unpredictable output order when using both info and error/warning messages.
* Mixing of lint/command/shell output with bootstrapping messages on stdout.
* ansible-test - Add changelog fragment.
##### SUMMARY
The docs mentions explicitly that you can run the tests against certain files which might not relay that it works on folders too!
Let's make this clear :)
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
* make the pre-flight sniffing more robust to different failure conditions (was failing on Ubuntu 22.04 VMs)
* remove skip aliases (the test needs to function everywhere to assert that the selinux facts bits behave properly when it's N/A)
* Run code-smell sanity tests in UTF-8 Mode.
* Update subprocess use in sanity test programs.
* Use raw_command instead of run_command with always=True set.
* Add more capture=True usage.
* Don't expose stdin to subprocesses.
* Capture more output. Warn on retry.
* Add more captures.
* Capture coverage cli output.
* Capture windows and network host checks.
* Be explicit about interactive usage.
* Use a shell for non-captured, non-interactive subprocesses.
* Add integration test to assert no TTY.
* Add unit test to assert no TTY.
* Require blocking stdin/stdout/stderr.
* Use subprocess.run in ansible-core sanity tests.
* Remove unused arg.
* Be explicit with subprocess.run check=False.
* Add changelog.
* Use a Python subprocess instead of a shell.
* Use InternalError instead of Exception.
* Require capture argument.
* Check for invalid raw_command arguments.
* Removed pointless communicate=True usage.
* Relocate stdout w/o capture check.
* Use threads instead of a subprocess for IO.
* Expand ansible-doc to tests/filters and fix existing issues
enable filter/test docs if in single file or companion yaml
add docs for several filters/tests plugins
allow .yml companion for docs for other plugins, must be colocated
verify plugins are valid (not modules, cannot)
fix 'per collection' filtering
limit old style deprecation (_ prefix) to builtin/legacy
start move to pathlib for saner path handling
moved some funcitons, kept backwards compat shims with deprecation notice
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Sandra McCann <samccann@redhat.com>
* trim down to use sanity test docs requirements file
* add requirements back
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update docs/docsite/rst/community/documentation_contributions.rst
* Prevent losing unsafe from lookups
This patch fixes a bug which under certain conditions results in data
returned from lookups not being marked as unsafe.
Each time Templar.do_template is invoked a new AnsibleContext is
created and stored effectively at two places:
1) as an instance variable in templar_obj.cur_context
2) as a local variable called new_context in do_template method of Templar
Due to custom functionality in Ansible's Context that allows for nested
templating it is possible that during resolving variable's value
template/do_template method is called recursively again, again creating
a new context. At that point the problem manifests itself because as
mentioned in 1) above the context is overwriten on the templar object
which means that any subsequent calls to _lookup will use the new
context to mark it as unsafe which is now different to the local
new_context which is used for testing for unsafe property.
The solution to the problem appears to be to restore the original
context inside do_template and also to eliminate the local variable
new_context to prevent problems in the future.
It appears that we don't have a better way of storing the context other
than as some form of global variable and so this appears to be the
"best" solution possible at this point. Hopefully data tagging will be
the solution here.
For more examples see unit and integration tests included in this patch.
Fixes#77535
* Run code-smell sanity tests in UTF-8 Mode.
* Update subprocess use in sanity test programs.
* Use raw_command instead of run_command with always=True set.
* Add more capture=True usage.
* Don't expose stdin to subprocesses.
* Capture more output. Warn on retry.
* Add more captures.
* Capture coverage cli output.
* Capture windows and network host checks.
* Be explicit about interactive usage.
* Use a shell for non-captured, non-interactive subprocesses.
* Add integration test to assert no TTY.
* Add unit test to assert no TTY.
* Require blocking stdin/stdout/stderr.
* Use subprocess.run in ansible-core sanity tests.
* Remove unused arg.
* Be explicit with subprocess.run check=False.
* Add changelog.
* git fix docs and wrapper script
fixes#77582
now env var is set to wrapper or full command depending on version
as was the intent of previous PR
added ref to git commit from git for why/how we used the env vars
* handle key_file
When looking up the `no_log` setting for a parameter that is an alias in
`AnsibleModule._log_invocation()`, the alias value will always be an
empty dictionary since `self.aliases` on the `AnsibleModule` instance is
never updated after initialization. Since the `no_log` setting is on the
canonical parameter not the alias, an incorrect warning is issued if the
parameter matches `PASSWORD_MATCH`.
This PR returns the aliases dictionary as an attribute of the
`ValidationResult` and updates the `aliases` attribute on the
`AnsibleModule` instance.
* Revert "Revert "Config, ensure templating happens at functions (#77483)""
This reverts commit 94c9106153.
* removed update configdata, which is unused
* removed test for action we don't perform anymore
* removed unused configdata
move away from special templating in constants.py
simplified main get, unified paths to get info
* Let native jinja deal with eval for type
* make constants available for templating defaults
* ensure we use config file
* ansible-connection verboistery
for cli, just use normal parser creation
this also adds --help, but that seems fine
also some error cleanup
Co-authored-by: Nathaniel Case <this.is@nathanielca.se>
* replace hardcoded '~/.ansible' to C.ANSIBLE_HOME
* rename previously existing env ANSIBLE_HOME in env-setup script
* modify cache dir monkeypatching in galaxy api unit tests
* update "version_added" to 2.14 for ANSIBLE_HOME
* fix description of collections with proper use of ANSIBLE_HOME
Co-authored-by: htol <github@h-tol.net>
Co-authored-by: stefanwascoding <stefan@syntaxhelden.de>
* Proper error on missing jinja2 override separator
Properly catch the case when no (or a wrong) separator is used in a
jinja2 override, and return a useful error message to the user.
* Support colons in jinja2 override value
By limiting the split to 1, any colons in the value are preserved and
passed on.
##### SUMMARY
Use of the meta/runtime.yml file to specify the ansible-core version for collections
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
* Add script to handle more deprecations.
This script currently supports deprecations from the following sanity tests:
* deprecated-config
* update-bundled
* Ignore script in package-data test.
* Fall back to implicit namespace.name from the path if the metadata is invalid
* Test listing a collection with null namespace/name/version fields in its galaxy.yml
* Add option --no-fail-on-errors to return errors for ansible-doc --metadata-dump in JSON result instead of failing.
* Adjust changelog fragment.
* Add basic tests.
* Support ignoring of certificates for ansible-galaxy during SCM cloning
* Add integration tests installing a role from an untrusted repository
Test installing the role without --ignore-certs fails
Test installing the role with --ignore-certs is successful
* Add find_spec and exec_module to RestrictedModuleLoader
* Fix getting new loader with correct path
Fix pep8 errors
* Use convert_ansible_name_to_absolute_paths instead of the loader path
* Apply suggestions from code review
* Fix type hints and test them in CI
* Fix error message for ansible.module_utils.basic if it's missing
Add mypy ignored missing imports for controller sanity tests
* Add mypy attr-defined ignore entries for python 3.8, 3.9, 3.10 for vendored six
Add mypy attr-defined ignore for python 2.7 in lib/ansible/utils/collection_loader/_collection_finder.py
* Just test controller python versions to simplify ignoring mypy errors
* Add a toggle to control the number of signatures required to verify the authenticity of a collection
* Make the default number of required valid signatures 1
* Add option to make signature verification strict and fail if there are no valid signatures (e.g. "+1")
* Use a regex to validate --required-valid-signature-count
* Add a toggle to limit the gpg status codes that are considered a failure
* Update documentation and changelog
* Add unit and integration tests for the new options
* Fixes#77146
Fix using user-provided signatures when running 'ansible-galaxy collection verify ns.coll --offline'
Add a test for a user-provided signature when running ansible-galaxy collection verify with --offline
Fix displaying overall gpg failure without extra verbosity
Add a test for displaying gpg failure without verbosity
Improve documentation to be more clear that signature verification only currently applies to collections directly sourced from Galaxy servers
For IPv6 addresses, Ansible already provides the prefix length for IP
addresses in the `prefix` fact. This patch adjusts the facts for IPv4
addresses to also contain the prefix length in the prefix fact. This
makes it easier to use the facts consistently when the CIDR notation is
needed.
Signed-off-by: Till Maas <opensource@till.name>
While getting hostname from container, podman command
fails to return JSON so wrap exception and return
hostname as 'None'
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Add more sanity tests on schema level (so they also work for plugins).
* Fix various issues the sanity test reported.
* Add changelog fragment.
* Fix function name.
Describe difference between targeting a single host using -i vs. --limit. In particular, pointing out the behavior difference that -i will not rad inventory variables.
* Let validate-modules also validate plugins.
* Support 'option' in 'cli'.
* Use DOCUMENTABLE_PLUGINS instead of UNDOCUMENTED_PLUGIN_TYPES.
* Support 'keyword', clean up error codes.
* Call settings.process_errors only once; remove __version__.
* Add changelog fragment.
* Fix ansible console to respect ANSIBLE_STDOUT_CALLBACK
This makes the ansible-console able to customize the stdout callback.
Ansible console currently uses statically minimal callback.
* Add changelog
* Move the fragment changelog in the right directory
`with_first_found` only takes a list of things, it's not like `tags`
where it can take a thing or a list of things.
Inspired by locally discovered breakage and then finding
https://github.com/ansible/ansible/issues/77136 where someone was also
doing this and getting into trouble.
* Remove unnecessary PyCharm ignores.
* Ignore intentional undefined attribute usage.
* Add missing type hints. Fix existing type hints.
* Fix docstrings and comments.
* Use function to register completion handler.
* Pass strings to display functions.
* Fix CompositeAction handling of dest argument.
* Use consistent types in expressions/assignments.
* Use custom function to keep linters happy.
* Add missing raise for custom exception.
* Clean up key/value type handling in cloud plugins.
* Use dataclass instead of dict for results.
* Add custom type_guard function to check lists.
* Ignore return type that can't be checked (yet).
* Avoid changing types on local variables.
Also, bump the default. A number of people have reported problems with
this 20 second timeout for a couple of years on
https://github.com/ansible/galaxy/issues/2302
It is mentioned in the issue that resolving dns can sometimes take up to
30 seconds, and that this timeout includes dns resolution.
Includes a changelog message
The current usage only works if `python` is Python 2.x since the syntax is invalid on Python 3.x.
When running on Python 3.x it would either be missing, or if Python 2.x was also present, incorrect.
* apply owner/group permissions to top folder
* remove unused var unarchive30
* fix permissions for top folders if the tarball include multiple top folders
* added test for top folder ownership
The use of `--venv-system-site-packages` was an optimization to use the `coverage` package pre-installed in the AZP test container.
However, now that the venv is bootstrapped by ansible-test that optimization no longer makes sense, since other downloads are already taking place.
ci_coverage
ci_complete
* fix facts delegation loop overwrite
partial revert of change to allow facts to be present in each loop iteration
was not needed in final results as result processing alreayd had the disctiontion
and ended up breaking the assumptions in the calling code.
fixes#76676
* play_context, compensate for existing plugins
some connection plugins are not fully using the correct configuration,
but this was previously hidden from them as play_context was providing
the info instead, now play_context provides the 'correct' info, but hitting
these bad configurations.
* ansible-galaxy collection install|verify:
- Support verifying the origin of the MANIFEST.json when the Galaxy server has provided signatures.
- Allow supplemental signatures to use during verification on the CLI/requirements file.
* ansible-galaxy collection install:
- Support disabling signature verification. This silences the warning provided by ansible-galaxy if the Galaxy server provided signatures it cannot use because no keyring is configured.
- Store Galaxy server metadata alongside installed collections for provenance. This is used by 'ansible-galaxy collection verify --offline'.
* Add unit tests for method that gets signatures from a Galaxy server
* Add integration tests for user-provided signature sources
- Test CLI option combinations
- Test installing collections with valid/invalid signature sources
- Test disabling GPG verification when installing collections
- Test verifying collections with valid/invalid signature sources
* Make signature verification advisory-by-default if signatures are provided by the Galaxy server
- Make the default keyring None
- Warn if the keyring is None but the Galaxy server provided signatures
- Error if the keyring is None but the user supplied signatures
- Error if the keyring is not None but is invalid
* changelog
* add ansible-galaxy user documentation for new options
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
Co-authored-by: Sandra McCann <samccann@redhat.com>
Co-authored-by: Andy Mott <amott@redhat.com>
Co-authored-by: John R Barker <john@johnrbarker.com>
* Return rc=0 on success.
Error handling in playbooks generally expects `rc` to be set to 0 when a module has not failed. Playbook authors should not have to check for the existence of `rc` first.
* Use single definition and added changelog
* Fix up tests with new return value
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* add DebianStrategy tests
* ensure hostname can be changed by using become
* use Systemd strat for debian and Base for generic.
* add test to ensure all strategies are available
Signed-off-by: Thomas Sjögren <konstruktoid@users.noreply.github.com>
avoid polluting vars with incorrect settings
simplify variables and templars
- variables, original, only updated with final results, reset are copies of this
- tempvars used for preliminary templating
- cvars used for connection/shell/become plugins, delegation aware
- vars_copy ignore tempvars updates and use connection plugin to get
finalized version per loop item/delegation, also used to store temp results
till we are ready to update 'variables'
- fine tune nolog just cause we are here
- also fix inventory_hostname_short for IP addresses
git module now uses env vars exclusively
- updated docs to clarify usage
- now env vars append instead of overwrite to allow existing custom setups to keep working
fixes#38104, #64673, #64674
- added note for hostkeychecking more securely
fixes#69846
- keep script cause old versions still choke on env
- env var cannot hold more than 'command' for older versions
- all ssh_opts in one place
ssh plugin, use 'correct' information source in all cases
* still fallback to pc
* added inventory to new test
* undef var can still show as parser error on pc
now task_exectuer has a more accurate error handling
Installation of `gcc` is failing while trying to install dependencies:
```
[MIRROR] libxcrypt-devel-4.4.18-3.el9.x86_64.rpm: Interrupted by header callback: Server reports Content-Length: 32565 but expected size is: 33101
```
Azure Pipelines only supports publishing a single code coverage file.
This limits us to reporting on either Python or PowerShell.
It also prevents us from reporting coverage per test command without further aggregation.
Most of our users are only aware of the reports published to codecov.io.
* ansible-test - Use relative paths in junit output.
Also fix a traceback in the junit callback during automatic fact gathering.
* ansible-test - Handle out-of-tree JUnit paths.
* minor refactor in other options by pushing common code into functions
* consolidate coll_filter
* more normalizing loader
* dont pass plugin_loader, its global import
* Also dump roles and collections
* adjusted tests to new err msg
* disable namespace filter (unused)
* ansible-test - Clarify need for empty __init__.py
* ansible-test - Update code-smell boilerplate.
* Update code-smell boilerplate for core.
* Update future boilerplate test for ansible-test.
All ansible-test code (except for targets) and core-specific sanity tests now use the same boilerplate.
The test also checks for unwanted `__future__` and `metaclass` boilerplate.
* Relocate target tools to the correct directory.
Several tools used on target Python versions were incorrectly placed in the controller directory.
* Make include_role/include_tasks work with any_errors_fatal v2
Fixes#50897
Co-authored-by: Matt Martz <matt@sivel.net>
* Add failed to results in free strategy too
* Fix
* Avoid duplicating results
* ci_complete
Co-authored-by: Matt Martz <matt@sivel.net>
* ansible-test - Improve help for unsupported cwd.
* The `--help` option is now available when an unsupported cwd is in use.
* The `--help` output now shows the same instructions about cwd as would be shown in error messages if the cwd is unsupported.
* Add `--version` support to show the ansible-core version.
* The explanation about cwd usage has been improved to explain more clearly what is required.
Resolves https://github.com/ansible/ansible/issues/64523
Resolves https://github.com/ansible/ansible/issues/67551
* group2 - switch to setup_remote_tmp_dir instead of using output_dir
* output_file needs to be in the tmp dir
* Use comparison that should work with macos
* remove unused dep
* allow inputting 0 for password_expire_{min|max}
0 is meaningful for min days (any time)
0 is technically valid for max_days
* add test for setting both min and max expiry
* [0] return result of execute_command from set_password_expire*
* [1] better return for set_password_expire
* [2] handle returns from set_password_expire*
* only set password expiry if user exists
* collect return-handling code
* combine password min and max into one execution
* handle case where spwd is not present like on macOS and FreeBSD
Co-authored-by: Sam Doran <sdoran@redhat.com>
* config, moved 'choices' from wall of text, to keys
* updated to ensure all pythons handle the 2 modes correctly
* Update lib/ansible/config/base.yml
Co-authored-by: Sandra McCann <samccann@redhat.com>
* go back to keyword being explicit for plugin configs instead of using option name as implicit match
* switch spec to singular, no changes needed since no one used
* Don't send full task with async callback events. Fixes#76729
* Use args for async_status task, instead of k=v
* Make sure we send back the async task attrs for polling
* Add clog frag
* load is a staticmethod
* include_vars initialize failed
if source dir is given, but not present and traversal is empty you
can end up trying to access failed w/o it ever being defined.
also future proof for more corner cases in decision tree
dont display _terms or _intput on only changed
those always change and it expected for the plugins that support them
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
- Avoids false positives on become strings being echoed back
by ssh cli itself
- added test for debug lines
- also simplified some of existing test code
modules with python were always normalized to /usr/bin/python,
while other interpreters could have specific versions.
* now shebang is always constructed by get_shebang and args are preserved
* only update shebang if interpreter changed
* updated test expectation
* added python shebang test
* Update dnf tests to reflect new behavior.
Previously dnf would report there was nothing to do when trying to install a package from the cache when it was not present.
A recent update to dnf has changed this behavior to match yum, resulting in a failure instead.
* Allow dnf to fail or report no changes.
* ansible-galaxy - fix the --ignore-certs flag for the implicit galaxy server
* changelog
* Add a test without the server config
* Fix respecting --ignore-certs for individual --server URLs also
* Update changelogs/fragments/76735-ansible-galaxy-fix-ignore-certs.yaml
This adds the ability to specify a package version using >=. This will ensure the package is at the specified version or above.
* If the package is not installed, the latest version will be installed.
* If the package is installed and less than the specified version, it will be upgraded.
* If the package is installed and greater than or equal to the specified version, it will be left alone.
The version selection is handled by Apt itself, so things like the system policy, pinning, etc, are considered.
* Add additional tests for check_mode with shell/command/script
* update check_mode documentation: if a module/action is not skipped in check mode and accurately reflects whether a change is made to the remote support should be 'full'
* Make reporting skipped in check mode mutually exclusive with 'changed: True'
* Add missing documented attributes
* Fix tests to expect skipped=True and changed=True do not occur together
* Fix script check_mode support documentation
* Fix earlier changelog
* document platforms attribute
* Use tasks's check mode since the value from PlayContext does not reflect loop items
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
'dscl -list' returns 0 even if the user does not exists. This
leads to errorenous condition in user module.
Using 'dscl -read UniqueID' can return if user exists or not.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Ensure the version is valid for directories and collections in git repos before installing
Fix the error message for invalid semantic versions
* Make requested changes
* Add a test case for unhandled ValueError exception
* Add changelog
* Update lib/ansible/galaxy/collection/galaxy_api_proxy.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Reword error message
Include link to learn how to compose a semver version
* Move version validation into the caller, find_matches
* Add tests for more invalid version types
* Remove unused import
Fix raising unexpected error
* Update lib/ansible/galaxy/collection/__init__.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Update lib/ansible/galaxy/dependency_resolution/providers.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Update lib/ansible/galaxy/dependency_resolution/providers.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* fixes#28977
* Include the original error message when pick_handler fails
* Add a test to unarchive a tar file with an invalid extra option
* add a changelog
* Disable the _distutils_hack in newer setuptools. Doesn't fix the underlying issue of the venv finding the _distutils_hack of a setuptools that is not its own.
ci_complete
* re-throw blanket
* check if service is indirect status / add integration test
* Update changelogs/fragments/76453-indirect-systemd-status.yml
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
* simplify indirect check / remove integration test restriction on rhel9
* add dummy indirect service for integration tests
* unarchive: fix non-english locales
For GNU Gettext, the LANGUAGE environment variable takes precedence over LANG or LC_ALL. On systems where LANGUAGE was set to a non-english locale, the output of the tar command therefore not understood and the module failed silently ("changed": false, but the archive was not extracted).
* add tests
* changelog
* check finder type before passing path
ci_complete
* Reduce nesting
* Test find_module does not cause a traceback with Python 3 FileFinder
* Update lib/ansible/utils/collection_loader/_collection_finder.py
Filter out the /usr/bin/pkg because on Altlinux it is actually the
perl-Package (not Solaris package manager).
Since the pkg5 takes precedence over apt, this workaround is
required to select the suitable package manager on Altlinux.
* reset defaults for json filters
these had change on unification of json parsing/dumping
now they behave like before, but are still controllable by user.
* Implement find_spec and exec_module to remove reliance on deprecated methods in the collection loader
ci_complete
* Move module execution to exec_module
Remove extra sys.modules handling
Use default module initialization by returning None from loader.create_module
Refactor
ci_complete
* Remove ansible-test's copy of the collection loader
ci_complete
* Fix metaclass for Python 2.x
ci_complete
* Fix Py2/Py3 syntax compatibility
* Refactor
ci_complete
* update collection_loader comments
ci_complete
* simplify find_module
ci_complete
* Fix Py2 compatibility - don't get loader from nonexistent spec
Remove unnecessary PY3 checking
* Refactor common code in load_module and exec_module
ci_complete
* tidy diff
ci_complete
* Include collection_loader in target paths for 'compile' sanity test
* add changelog
* Add "return None" instead of doing it implicitly
Remove get_filename
short-circuit exec_module if it's a redirect
ci_complete
Ref #44741
Ref ansible/proposals#140
* Replace select filter with a more portable thing
* Add context
This is needed for split controller/remote
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Add changelog fragment and integration tests
* Break up function signature into multiple lines
It passed the 160 character width. Change it to multi-line for easier reading
and better diffs in the future.
* Remove unused import
* Update version added for new feature
* Move changelog fragment to correct location
Co-authored-by: Thomas Stähle <thomas.staehle@check24.de>
Co-authored-by: Sam Doran <sdoran@redhat.com>
* align requirements files to more recent versions
* bump up to latest antsibull
* update sphinx in rstcheck requirements
* update to dependency versions that sphinx 4.2 pulls in
* use same no-cap spelling
* Add script to freeze sanity requirements.
* Declare sanity test requirements and freeze
* Use pinned requirements for import.plugin test.
* Expand scope of import test for ansible-core.
* Add ignores for galaxy import errors.
* Update test-constraints sanity test.
* ansible-test - Add support for RHEL 8.5 instances.
* Clear libdnf checksum cache from unsigned package in dnf integration test.
Co-authored-by: Matt Martz <matt@sivel.net>
* First pass of adding yaml result format output to default callback
* Add clog frag
* flow style always false
* Further normalization of results across distros
* no lossy, now pretty
* Fix env var in runme.sh
* Rename variable to better self document
* include NativeJinjaUnsafeText
* Linting cleanup
* Add tests specific to the prior yaml callback
* Make data munging more exact, following the checks in libyaml/pyyaml
* Remove unused import
* Extend comment
* more correct and some comments
* Fix consistency of tab vs space in this section of the file
* Add str representer
* be cooler
* faster character filtering
* Clarify None as a sentinel for default behavior
* Flip filtering logic to avoid CRLF issues with 'space break' filtering
* Py3 change and comment about the use of __call__
Co-authored-by: Matt Davis <mrd@redhat.com>
- On Python 3, decode bytes to str
- Tolerate whitespace around the line.
- Report a missing HOSTNAME line as an error.
- Don't clobber line separators when setting a new hostname.
- Add some tests for this strategy to verify correct operation.
* `podman inspect` falls back to a same-named image if the named container is not present; since eg `http-test-container` is both the name of the image and container, it wasn't working properly under podman in many instances. Switching to `docker|podman container inspect` limits the query to containers only for both podman and docker, allowing the support container detection/creation to work properly.
This task was previously added in https://github.com/ansible/ansible/pull/25243 as a work-around for a setuptools bug.
The pinned version does not work with Python 3.10, and the task should no longer be needed.
* Hashing is not encrypting
All encryption systems have a formal inverse function to decrypt
A hash is a one way function without inverse by definition
* Update docs/docsite/rst/user_guide/playbooks_filters.rst
Co-authored-by: Sandra McCann <samccann@redhat.com>
* Move RHEL pip package install into function.
* Add RHEL 9 support to bootstrap script.
* ansible-test - Add RHEL 9 support.
* Add install and uninstall instructions for RHEL9 to paramiko tests
* Skip incidental_inventory_docker_swarm on RHEL 9.
* Update dnf tests for rhel9
* Fix prepare_http_tests on RHEL 9.
* fix RHEL9 Python discovery tests to match distro map
* Skip subversion test on RHEL 9.0b.
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Matt Davis <mrd@redhat.com>
* Skip recursive suboption validation if sub_parameters is not a dictionary
* Ensure sub parameter elements is a sequence to prevent iterating over string characters and causing duplicate error messages for the same param
* Try to find any valid vault id before raising an exception
* Apply the same logic during vault password file loading
* Also. Improve the warning msg and tests
When we introduced an include parameter to the unarchive module, we
inadvertenly flipped the exclusion logic. This flip meant that the
unarchive module started rejecting files that should be extracted.
This commit flips the bad logic and adds some tests that will make
sure things do not go bad again.
Clarify that the `requires_ansible` field in meta/runtime.yml refers to the version of Ansible Core (ansible-core), not any package called "ansible" e.g. https://pypi.org/project/ansible/
* Use NativeEnvironment for all templating
ci_complete
* Keep Templar.copy_with_new_env for backwards compat
* Mention that AnsibleUndefined.__repr__ changed in the porting guide
* Templar.copy_with_new_env backwards compat
* ci_complete
The scripts now use `--list-targets -v` instead of `--explain` to evaluate changes.
This is faster and does not trigger parsing of a non-existent inventory file.
* Overhaul ansible-test integration tests.
* ansible-test - Fix import test pyyaml usage.
* ansible-test - Remove unused import.
* ansible-test - Fix traceback when pip is unavailable.
* ansible-test - Fix typo in port forwarding message.
* ansible-test - Fix controller logic in requirements install.
* Fix unit tests in ansible-test integration test.
Unit tests are now run for available Python versions which
provide `virtualenv` (Python 2.x) or `venv` (Python 3.x).
* Don't use output_dir in user tests
* Move blockinfile tests from using output_dir to depending on setup_remote_tmp_dir
* Don't use output_dir in git tests
* Don't use output_dir in uri tests
* Update network_resources.rst
I am not sure a fully qualified URL should be here... but teach me great documentation gods
* Change platform index external link to internal ref
Co-authored-by: Aine Riordan <44700011+ariordan-redhat@users.noreply.github.com>
Unofficial packaging should be maintained externally since
it is not tested in CI or used in the official release process.
Maintainers of unofficial packages are better equipped with the necessary
expertise and testing resources to support alternative packaging.
Prepend the task name to the warning "The loop variable 'item' is
already in use", so that you can get some context even if stdout and
stderr go to separate places.
- The containers are now based on Ubuntu 20.04 instead of Ubuntu 18.04.
- The default Python version used for the containers is now Python 3.10.
- The included version of PowerShell was upgraded to version 7.1.4.
- The included version of shellcheck was upgraded to version 0.7.0.
ci_complete
* removes AWS scenario guide, moving to collection
* first attept to replace TOC entries
* not sure what I did, but saving it
* updates TOCs to reflect new location of aws guide
* reinstates original page as a stub
* adds links to new location for AWS guide, updates header
Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
* Removed translatable words from code blocks
##### SUMMARY
Removed translatable words from codeblocks as per #59449
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
* Maintaining intend as well as clearity
* Preserving "save_as" as the key
* showing equivalence and keeping same context
* Bump antsibull version to get support for attributes and CLI options.
* Include fix for attributes.
* Include fix for rst_epilog collision and for #75799.
* docs - add some info on plugin option source precedence
##### SUMMARY
Add some clarity on precedence, especially from sources of the same type.
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
* update RST formatting
* Update docs/docsite/rst/dev_guide/developing_plugins.rst
Apply review suggestions
Co-authored-by: Aine Riordan <44700011+ariordan-redhat@users.noreply.github.com>
Co-authored-by: Aine Riordan <44700011+ariordan-redhat@users.noreply.github.com>
Resolves https://github.com/ansible/ansible/issues/75320
The option `--prime-containers` was chosen over `--docker-pull-only` to match the recently added `--prime-venvs` option for sanity tests.
It would also fit well with a future `--prime-requirements` option for pre-installing requirements for unit and integration tests.
* ansible-test - Create injector scripts at runtime.
* Set bootstrap.sh shebang at runtime.
* Remove shebang and execute bit from importer.
* Update shebang sanity test.
* Preserve line numbers.
* Use the task dependency chain to compile the role instead of the parent role's _parents list
* keep existing role inheritance, but limit it to roles in the current task dependency chain
* Test getting role parents recursively by using the current task dependency chain
* changelog
* download role from api response location
* include changelog fragment
* add unit test for role download url
Co-authored-by: Sam Doran <sdoran@redhat.com>
* ansible-galaxy: Clone git collections using shallow clones
This ensures the collection obtained via git url is a result of a
shallow git clone (git clone --depth=1). The git history of the
collection is not used by ansible, and as such, cloning the entire
history of the repo is unnecessary.
Signed-off-by: Tomas Babej <tomas@tbabej.com>
* ansible-galaxy: Only perform shallow clones on non-versioned git urls
In general, the version can be anything we can checkout (branch, tag,
but also a commit hash). In particular for commit hashes we cannot
perform a shallow clone.
Err on the safe side and only perform shallow clones for non-versioned
git urls (cloning HEAD).
* galaxy-install: Make shallow cloning compatible with older Python versions
Signed-off-by: Tomas Babej <tomas@tbabej.com>
* Pass args as a tuple
Add tests for the git command created from different repo and version combinations
* changelog
Co-authored-by: Tomas Babej <tomas@tbabej.com>
* add tests for fail filter
also tests that fail does not block inspectability
* add fail filter
fallback message is a bit clunky,
since you can't invoke a filter without specifying an input.
That is, "{{ fail }}" doesn't work,
so you have to do "{{ None | fail }}"
* document 'fail' filter
* add changelog fragment
* fail filter uses default message on Undefined or emptystring
makes it slightly easier to use the default message:
```diff
- "{{ None | fail }}"
+ "{{ '' | fail }}"
```
and the user sees a slightly more relevant message
if the message itself is undefined:
```diff
- The error was: {{ failmsg | fail }}: 'failmsg' is undefined
+ The error was: {{ failmsg | fail }}: Mandatory variable has not been overridden
```
* rebuild as the builtin `Undefined`
* harmonise `hint` parameter for make_undefined with jinja
* use code block for documentation item
[ref](https://github.com/ansible/ansible/pull/75435#discussion_r707661035)
* rename to `undef` to expose less Python into the Jinja
[ref](https://github.com/ansible/ansible/pull/75435#pullrequestreview-757799031)
* explicitly instantiate undefined value now that it's possible
see I knew we would break something with reflection
* preserve test coverage of undefined variable
Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
Change:
- Remove only user-facing use of ANSIBLE_ASYNC_DIR.
- Remove two comments saying to change things that, apparently, we
aren't going to change...
Test Plan:
- ci_complete
Tickets:
- Fixes#74139
- Fixes#74138
- Refs #74226
Signed-off-by: Rick Elrod <rick@elrod.me>
* Enable ansible-galaxy to specify client id override with Keycloak Token
* Specify ability to provide override of client_id
* Test client_id can be configured for individual servers
* Add issue link to changelog
* Document client_id as a config option and add an example
Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
* Fix test usage of `which python`.
Also use `command -v` instead of `which` where needed.
* Fix testing_formatter.sh file checking.
The index is longer expected to exist when generating it.
The generation script is run with `set -eux`.
* apt module: add option to allow package downgrades
* Add new option to module so users don't have to force downgrades which
is insecure and dangerous
* Add integration tests similar to upgrade integration tests
* Changelog
* Update changelog fragment
* Update changelogs/fragments/74852-apt-allow-downgrade.yaml
Co-authored-by: Amin Vakil <info@aminvakil.com>
* Update lib/ansible/modules/apt.py
Co-authored-by: Amin Vakil <info@aminvakil.com>
* Update lib/ansible/modules/apt.py
Co-authored-by: Amin Vakil <info@aminvakil.com>
Co-authored-by: Amin Vakil <info@aminvakil.com>
* ensure dump produces json
* clog and tests
* remove library additions since they are not usable
* avoid any/all dirs
* ensure we dont use local dir as playbookdir
* If an exception occurs when getting a collection's metadata, continue to the next in the server list.
* Warn for unknown exceptions when finding versions of a collection
* Test that an invalid server is no longer fatal if a subsequent server has the collection
* Fix server for verify tests - compare checksums against the server from which it was installed
* Add tests for verify and fix that code path to mirror install/download behavior for server errors
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* pip - Use pip from the current Python interpreter.
If `executable` and `virtualenv` were not specified, and
the `pip` Python module is available for the current interpreter,
use that `pip` module instead of searching for a `pip` command.
* Add comment about needing `__main__` to run `pip`.
* Fix unit test.
* Add porting guide entry.
* Update changelog to match porting guide description.
ci_complete
* Make individual dirs that should exist instead of using shutil.copytree
* Test build_ignore when installing collections in git repos
* changelog
* Fix assertion
Fix git directory
* Fix nested content while building the collection dir
Test installing a collection from a git repo and artifact have the same result
Refactor to use variables
* Refs #75478 - On the communicating page, add Matrix links to first 4 chat spaces, improve introduction, update working groups with Matrix and IRC locations, add a link to the community FAQ, add localised community translations, and remove Lightbulb room, the GitHub project has been archived
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
removing out of date example ansible.cfg, show user how to generate from ansible-config
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
* Support templating _from_files with --extra-vars for imported roles
* Add tests for templating the filenames for import_role, import_playbook, and import_tasks
* Add documentation
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
* Test a passlib wrapped algo with a password lookup
* Fix error when passlib is used with a wrapped algo
The exception was:
An unhandled exception occurred while running the lookup plugin 'password'.
Error was a <class 'TypeError'>, original message: issubclass() arg 1 must be a class
and can be reproduced using the following command:
ansible localhost -i localhost, -mdebug -amsg="{{ lookup('password', '/dev/null encrypt=ldap_sha512_crypt') }}"
The concerned algo are: bsd_nthash, django_argon2, django_bcrypt, ldap_bcrypt,
ldap_bsdi_crypt, ldap_des_crypt, ldap_hex_md5, ldap_hex_sha1, ldap_md5_crypt,
ldap_pbkdf2_sha1, ldap_pbkdf2_sha256, ldap_pbkdf2_sha512, ldap_sha1_crypt,
ldap_sha256_crypt, ldap_sha512_crypt, roundup_plaintext
* allow vars_files to use dwim (vars/) and vaults
* also fixed bug with temp file cleanup, some pythons dont like it when you alter loop
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
Rather than adding further complexity to the regex, preprocess the output to
remove any empty lines. Now the only purpose of the regex is to fix
wrapped lines.
Fixes#70949
* avoid getting delegated vars w/o inventory host
fixes#75512
In the case of imports, we don't have a host, so getting host vars for
the delegated host makes no sense and should be avoided.
* also avoid error on vars_files with per host vars
* test
* testing given case
* oops
* add 'file options' for become and connection pass
* implemented getting passwords from file or script
* added config entry
* fixed env var name and noted executable behaviuor
Co-authored-by: Rick Elrod <rick@elrod.me>
##### SUMMARY
When I used this example it complains with:
ERROR! The tasks/main.yml file for role 'myrole' must contain a list of tasks
The error appears to be in 'ansible/roles/myrole/tasks/main.yml': line 2, column 1, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
---
tasks:
^ here
Removing "tasks:" fixes it.
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
* Add new comment attribute to template plugin
Add comment_start_string and comment_end_string attribute to template
plugin
Co-authored-by: Hossein Zolfi <h.zolfi@inside.sahab.ir>
Some job names contain '/' which causes the download job to fail.
Example job name: `Post-job: Checkout ansible/ansible@refs/pull/75445/merge to ansible.log`
* user - Remove unused code.
* Replace deprecated abstractproperty decorator.
* Fix __all__ to be a tuple.
* Use a generator in subelements lookup.
* Use from import in basic.py
* Add changelog fragment.
* Fix selinux unit test.
* ansible-test - Fix use of abstractproperty
* ansible-test - Use dict.items() where possible.
* ansible-test - Remove unused code.
* ansible-test - Cleanup issues reported by pylint.
* ansible-test - Use dict.items() where possible.
* ansible-test - Use generator.
* Improve error for InconsistentCandidate exceptions
* Add test case for installing a collection with an inconsistent version
* Add test case for installing a collection that has a dependency with an inconsistent version
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Sviatoslav Sydorenko <webknjaz@redhat.com>
* Add basic schema for attributes.
* Support attributes for version_added_collection adding mechanism.
* Add changelog fragment.
* Support new attribute properties from #74331.
* Require additional properties strictly.
* Fix a bug with the dnf module not using all components of a package name when filtering to determine if it's installed
* changelog
* Simplify splitting on the last '.'
* Update lib/ansible/modules/dnf.py
Push /usr/bin/python to almost the bottom of the barrel. This makes the strategy to prefer
specific versions of the "mystery meat" version.
* Change INTERPRETER_PYTHON default to 'auto'
Change description to match change in behavior.
Change deprecation message to a warning.
* Update docs
* Add porting guide entry
* Update unit tests
* Update integration test
* Allow INTERPRETER_PYTHON_FALLBACK to be configure using a variable
* Prefer platform-python above other Python 2 interpreters
* Add Python 3.10 to the list of interpreters
* Make undefined variables in handler names non-fatal if the handler is not used
* If the handler has no way to be notified (i.e. the name can't be templated and the handler has no listen topics), display a warning
* Add tests for variables in handler names
* changelog
* Fix when evaluation on Native Jinja and Python 3.10
* Add unit test
* Add explaining comment
* Enable jinja2_native before tests
Co-Authored-By: Matt Martz <matt@sivel.net>
* Sanity
* Return native template module instead of modifying globals
Co-authored-by: Matt Martz <matt@sivel.net>
* Make sure AnsibleModule positional arguments are validated.
* Extract ANSIBLE_MODULE_CONSTURCTOR_ARGS with inspection.
* Remove no longer necessary return value.
* Fix PR #.
* argument_spec might not have been specified, as in community.general.xenserver_facts.
* Fix typo.
test/integration/targets/any_errors_fatal/18602.yml is not run via
runme.sh and is testing the same as
test/integration/targets/any_errors_fatal/always_block.yml which is
actually run. The former file is safe to remove.
* Add packaging to requirement of ansible-test
Fix#75353
After requires_ansible field was added as mandatory to runtime.yml
file, ansible-test fails to check this field if it doesn't have
packaging module.
[1] https://github.com/ansible/galaxy-importer/pull/124
* updates Tower page to talk about RHAAP
* updates scenario guide Tower references
* updates Tower references to use AWX and/or RHAAP
* more scenario guides fixes
* fix CI failures
* Update docs/docsite/rst/porting_guides/porting_guide_2.8.rst
* removes tower link
* for 2.8, still mention Tower along with AWX and RHAAP
* aws guides should be updated in the collection, where they now exist
Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
* Update network user guide to point to ansible.utils.cli_parse module as ansible.netcommon.cli_parse is deprecated from ansible.netcommon 2.0.0 release
* modules moved to use best_parsable_locale
* fixed invocations
* better better
* also module_utils
* converted to function as per fb
* patch testt
* whitespace
* set host_key_checking check to False, rather than if not (which captures False and None)
* add host_key_checking default to ssh.py / update documentation
* Canonicalize module_defaults actions and action_groups pre-fork and cache them on the play
* Call get_action_args_with_defaults with the resolved FQCN plugin and don't pass the redirect list
* Add validation for action_group metadata and a toggle to disable the warnings
* Handle groups recursively referring to each other
* Remove special-casing for non-fqcn actions in module_defaults groups
* Error for actions and groups in module_defaults that can't be resolved
* Error for fully templated module_defaults
* Add integration tests for action_groups
* Changelog
This is useful for testing the effects on docs of changing what
collections are in a version of ansible. ansible-build-data can be
cloned locally, modified, and then build-ansible.py docs-build can be
told to use the local checkout instead of the canonical one on github.
* env_setup - Prefer Python3
Prefer the 'python3' executable over 'python'.
* Add shebang for syntax detection
While the script isn't executable and we recommend sourcing it, having a shebang allows
variuos text editors to detect that it is a shell script and apply proper syntax highlighting.
While implementing basic Ansible module testing I realised that the `self.module` used in some places is not actually a thing, and I changed to `my_module` to make it work as I understand it was expected.
Since the task and connection both have the same 'retries' keyword, the task default
would override the connection value.
Do not pass 'retries' from the task to the connection options.
* Set ssh_connection retries default value back to 0
It was 0 before the move to config and was changed to 3 by accident.
This version relies on an external lib implementing lexers and styles.
That lib pulls in proper Pygments version so this patch also drops it
from direct requirements.
This is a follow-up of previous years' 1a11cec and c8315bf. It deals
with links which at that point presumably either were not present or
did not support https://
* module_common - handle cache directory creation collision
Occasionally multiple workers can try to create the cache directory if it exists. Catch the exception
and handle it.
* Just re-raise if it doesn't exist rather than trying again
* Use exists_ok rather than handling an exception
* Remove unused import and unused variable
* Go back to try/except but with OSError
Since we do not have split controller remote, this needs to be able to run in Python 2 in order
to be merged currently.
Since we moved yum_repository_test_repo.baseurl to file:/// format,
using urlsplit filter on it returned an empty string. Using an empty
string as a left operand of `in` was then always evaluated to True.
This was discovered with native jinja being on where urlsplit returns
None which results in a hard fail as None cannot be a left
operand of `in <string>`.
* The resolved_action is the formatted version of the final plugin in the PluginLoadContext's redirect_list
* Collection plugins are represented as FQCN
* Legacy plugins are represented with only the plugin name
* Add tests
* Changelog
* Fix struture of cli/doc.py snippet code.
A couple releases ago, cli/doc.py was modified to mostly conform to the
data processing pipeline steps. format_plugin_doc() was the biggest
exception in that refactor. When the snippet code was made generic
instead of being only for modules, the new code should have conformed to
the data processing pipeline too.
* Move the decision to output a snippet to the run() method alongside
the decision to output a listing versus plugin_docs.
* Move the test for invalid plugin_types to the run() method as it
affects all snippets in this run, not just a single snippet. (-t can
only be specified once)
* Rename get_snippet_text() to format_snippet() as:
* This is the data formatting step
* The format_snippet() name matches with its conceptual sibling,
format_plugin_doc().
* Use ValueError inside of format_snippet() to flag unrecoverable errors
formatting a single snippet.
* Emit a warning when format_snippet() raises ValueError and continue to
the next snippet.
* If the yaml(?) or toml inventory plugin is specified for snippet output,
raise ValueError() so that the user sees a warning instead of simply
seeing blank output.
* Do not modify arguments passed into format_snippet(). This is the
formatting step so data should not be modified.
* Change _do_yaml_snippet() and _do_lookup_snippet() to operate side
effect free.
* Fix raising of exceptions when formatting requred options for snippets.
* Unrelated: Use to_text() instead of to_native when calling
display.warning(). to_native() is used for raising exceptions. Not
for display methods.
* Add a changelog
specially for when you have parameters in unicode but need
to scrape responses, C is still the fallback
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
* Require antsibull 0.34.0.
* Remove Makefile comment about the devel docs building only the core
(formerly base) docs; the behavior was updated in ccbfdec334
Currently 'make coredocs' builds core-only docs; 'make webdocs' builds the full docs even on the devel
branch
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
* removes upper bound on sphinx version
* updates versions of docs build dependencies, adds known good requirements file
* adds instructions for using known_good_reqs file
Can now specify plugin/plugin type for list and dump actions
New 'init' action to create usable config sources
Co-authored-by: Felix Fontein <felix@fontein.de>
* Change tests to pytest-style tests
* Add tests for invalid hosts
* Validate host inputs
- check for empty values
- check for None
- check for values that are not a sequence and are not strings
- add unit tests
* Move play name setting to get_name() and out of load()
* Add _validate_hosts() method
By defining this method, it gets called automatically by FieldAttributeBase.validate().
* Improve the error handling code
Rather than multiple return paths, have a single return and set the message based
on the type of failure.
* Add another test for non-specific failures
* Reorganize tests so failure tests are in one tasks file
* Remove os.stat() call and add changelog
* Remove use of _remote_checksum from fetch module
* Add deprecation message displayed during runtime
* Increase test coverage for fetch
* Add tests covering the use of stat from the fetch module
This required creating an unpriveleged user account and connecting as that user remotely since
it is not possible to create a file that the root user cannot stat.
* Use fact caching to persist remote tmp dir across playbook runs
* Add variables to setup_remote_tmp test role to allow caching of the remote temp dir fact
and preventing removal of the remote_tmp_dir
* subversion - set LC_ALL for accurate command output parsing
When LC_ALL is not set, the output language of commands will differ based on locale. There
is a lot of history of trying to fix this. See the following pull requests:
https://github.com/ansible/ansible-modules-core/pull/4358https://github.com/ansible/ansible-modules-core/pull/4358
This patch attempts to fix this my setting LC_ALL to a UTF-8 locale. Setting LC_ALL to C reintroduces this bug https://github.com/ansible/ansible-modules-core/issues/4178.
I'm sure there are some problems I am not seeing with setting this to en_US.UTF-8, but that is
the only way I could find to fix this bug without reintriducing the bug mentioned above.
* Rather than setting locale, just check for matches before trying to get groups
This is a pragmatic solution to avoid the stack trace since setting the locale correctly
to ensure message parsing is accurate is problematic.
* Improve regexps for finding revision and URL
Since moving to distro, it is possible to return this information for all platforms, not just Linux.
Also return version information for all platfrom not just Linux.
Update unit tests.
Remove some duplicate unit tests though I think there are more to remove.
* Fix docstring formatting
* Minor docstring changes
* Mock distro.id for Solaris service test
* Update comment
Test for the required binaries in the can_handle_archive() method and fail there. This
prevents failures for missing binaries unrelated to the archive type.
* Update missing zip binary message to match tar message
* Update unit tests
* Add integration tests
* Define packages based on the system rather than ignoring failures
* minor service_mgr facts fixes
handle case in which ps command fails or returns empty
updated tests since it now does keep trying to detect after ps fails
- remove need for module (at least for posix side)
- adds retry with backoff on fetching file, since
race is bigger since we don't spend time on module
- now gives more info on fail
- also made actionfail/skip handle results if given
* rebased with upstream
* removed extra usetty as it wasnt needed, style changes, added var option setable by inventory for pkcs11
* update pkcs11_provider version_added
* Update lib/ansible/plugins/connection/ssh.py
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* Update lib/ansible/plugins/connection/ssh.py
Correct logic for a password being required for pkcs11_provider
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* style nit fixes for checking pkcs11_provider is set
* fixed duplication when using password_prompt with pkcs11_provider
* added changelog fragment
* added changelog fragment
Co-authored-by: David Whiteside <david.whiteside@nrel.gov>
Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
Co-authored-by: Jordan Borean <jborean93@gmail.com>
base._update_security_filters is a private attribute of DNF used
as performance optimization. Modification or even call from outside
of DNF is against all recommendation including PEP8.
* Improve compatibility with all DNF versions
* Add changelog fragment for dnf security change
* Add resiliency to linux hw fact gathering
Now traps unexpected exceptions on mounts and continues
gathiering other info.
Also gives more info on why mount info gathering failed.
more info if debugging
Changed confusing wording. People new to ansible, like myself, have a hard time grocking what is a playbook and what is a task. I changed the name of the playbook to make it clearer.
* Due to the takeover of freenode we're moving to a different irc network.
* Our channels updated to point at the same channel name on libera.chat
* Some links went to webchat.freenode.net. At this time, libera.chat
doesn't point you to an official webchat client so I changed these to
https://libera.chat. (kiwi irc does work with libera.chat so that
could be another option).
* In general, I used the name irc.libera.net for link names and
https://libera.chat for link targets. This is because the irc service
is hosted on irc.libera.chat but the project web server is hosted on
libera.chat. (This appears to also be true for freenode but we were
using http://irc.freenode.net which doesn't seem to work. Oops).
* Removed http://irc.freenode.net from the linkcheck exceptions.
linkcheck was actually correct to flag that as invalid (should have
been http://frenode.net instead).
* Looks like hte important people in #yaml are now in libera.chat
* Link to where contributors should get help
Add a link target and then link to where contributors should get support
for developing groups of modules.
* Update docs/docsite/rst/dev_guide/developing_modules_in_groups.rst
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: John R Barker <john@johnrbarker.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
* Fix module-specific defaults in the gather_facts, package, and service action plugins.
* Handle ansible.legacy actions better in get_action_args_with_defaults
* Add tests for each action plugin
* Changelog
Fixes#72918
Change:
- Make strategies behave consistently and return the empty string
instead of "UNKNOWN" (or "temporarystub") for the "before" value if
the permanent hostname file does not exist or could not be read.
- Switch to `with open()` instead of annoying exception handling code
(which was wrong and leaked file handles in several places). This
drops Python 2.4 support for this module.
- Updated porting guide since users could be relying on these former,
inconsistent values.
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Hostname strategies' get_*() methods should never write to the
filesystem. They are used in check_mode by default to determine if
there is any work to be done.
Test Plan:
- New unit tests to ensure that (at least when in check_mode) the get
methods don't ever call write.
Tickets:
- Fixes#66432
Signed-off-by: Rick Elrod <rick@elrod.me>
- AWS has moved to Zuul CI
- Kubernetes collection has been renamed
- OpenShift/OKD collection was missing
- ansible.windows is on Zuul and Azure Pipelines
- ansible.posix is on Zuul
Signed-off-by: Jill Rouleau <jill.rouleau@bespokess.com>
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
It is not clear that you can publish without configuring an ansible.cfg. The `--token` can be displayed with the `ansible-galaxy collection publish -h`
* Don't mutate os.environ in AnsibleModule.run_command, make a copy, and pass to Popen. Fixes#74783
* Simplify code a bit
* More simple
* Address some other potentially non threadsafe operations
* Add if around umask
* Address unit test assumptions
* Add clog frag
* yaml syntax issue
service module acts as proxy module for all service manager modules,
just like `package` module.
This is helpful in heterogeneous environments to manage services.
Fixes: #74507
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* docs: Update Python 2 doc links
Update links from Python 2 to Python 3
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* use docs.python.com/3/ everywhere, except onethat should remain 2.6
* refer to python 3 in module docs and comments
* format two python docs links as list
* updates links in unwanted.py test file
* per matt clay, this should link to python 2
Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
When an empty value is provided, no `version` attribute will exist on the `LooseVersion` or
`StrictVersion` object. We catch and handle this, but it's not immediatebly clear that an
AttributeError means an empty value was provided.
Specifically handle the case where value or version are empty and add more
helpful error messages.
Add integration tests.
* Remove PyCrypto from setup.py and packaging script
* Remove mention of pycrpto from installation docs
* Remove PyCrypto from vault
* Remove pycryto constraint and unit test requirement
* Remove PyCrypto tests from unit tests
* Add docs and fix warning message
* Remove section about cryptography library in Ansible Vault docs
The URLs to the Jinja website have moved, seemingly because of the renaming from `master` to `main` (hence from `master` to `latest` in the doc). This commit fixes that.
This should allow the same script to be used for two different scenarios:
- ansible-core across multiple versions (not all of which support `--docker` for coverage reports)
- collections (which should always use the `devel` branch for coverage reports)
* For long-running playbooks, discourage users from using
`ansible_date_time`. Use `pipe` or `now` as an alternative.
* Review comments
Fixes: #22561
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
* Only remove crontabs if they are empty
* Add integration test to ensure system cron tab doesn't get removed. Increase cron integration tests separation.
* Also detect crontab which only contains whitespace as empty.
* cron integration test: Adjust system crontab path to be distribution specific.
Co-authored-by: Fabian Klemp <fabian.klemp@elara-gmbh.de>
* Update to six 1.16.0
* Address linting issues
* Remove six find_spex/exec_module warning filters
* Remove unnecessary comment about Py2.6, 2.13 will not support Py2.6, and we're bumping this for 2.12
* ci_complete
* Add changelog fragment
* Add constraint for MarkupSafe
MarkupSafe >= 2.0.0 requires Python >= 3.6.0. Add a constraint for older Python versions
and fix the `groupby_filter` test.
* Fix template_jinja2_latest test.
* patch filter decorators on newer Jinja2
* Jinja2 >= 3.0 renames several filter decorators used by Ansible itself, as well as by filters in collections. This patch ensures that the old names are usable within Ansible and by collections without warnings or errors.
* Ignore docs-build issues.
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Davis <mrd@redhat.com>
While logging, journal.send accepts module parameters.
If module parameters similar to arguments in journal.send,
rename the parameter names before sending to journal.send
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* fix: action _fixup_perms2 macos +a remote_paths in list() as it can be tuple
in `lib/ansible/plugin/action/__init__.py`'s `_fixup_perms2`,
`remote_paths` can be a list or tuple. however, the macos
specific attempt to use chmod +a attempts to concatenate
`remote_paths` with a list, which will fail if it is a tuple.
wrapping `remote_paths` in `list()` fixes this error.
* Update changelogs/fragments/74613-actionfixup_perms2_macos_remote_paths_ensure_list.yml
Co-authored-by: Rick Elrod <rick@elrod.me>
* support separate role argspec file in ansible-doc
* support separate role argspec file in ansible-core
* support both .yml and .yaml extensions on argspec file in ansible-doc
* fix filename building bug and rename some argspec files to test variations
* use yaml extensions from constants
* add superfluous meta/main.yml files to tests
* Update lib/ansible/cli/doc.py
Co-authored-by: Sam Doran <sdoran@redhat.com>
* update docs
* ci_complete
* add changelog and allow for main.yml variations
* add collection role testing
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Update Python versions for controller-only tests.
Both `ansible-galaxy` and `ansible-test` integration tests only run on the controller.
* Use a venv to make requirements available.
* Get available collection versions with page_size=100 for v2 and limit=100 for v3
* Update unit tests for larger page sizes
* Add a generic retry decorator in module_utils/api.py that accepts an Iterable of delays and a callable to determine if an exception inheriting from Exception should be retried
* Use the new decorator to handle Galaxy API rate limiting
* Add unit tests for new retry decorator
* Preserve the decorated function's metadata with functools.wraps
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Always use create_homedir when we are asked to create a home directory
in the User class. Don't use the -m and -k parameters from
useradd / luseradd as they behave differently with respect to
preexisting home directories. Instead always specify -M to ensure
that useradd / luseradd do not try to create the home directory.
This does not change potential different behaviours in child classes
of the User class.
Consider the new umask option from #73821 in create_homedir as well as
we do not let luseradd / useradd create the home directory any longer.
From the sha512sum man page:
... The default mode is to print a line with checksum, a character indicating type ('*' for binary, ' ' for text), and name for each FILE.
The distutils module is not shipped with SUNWPython on Solaris.
It's in the SUNWPython-devel package. Do not use LooseVersion.
Fixes: #74488
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* removal of deprecations and minor fix
name or cron_file is required.
reboot was removed.
updated docs to clarify usage.
now using cron_file=/etc/crontab is an error
fixes#37355
* warn show pathing issues when wallking through provided paths
moved issues from msg to actual warnings AND a specific return field
fixes#25314
Co-authored-by: Sam Doran <sdoran@redhat.com>
* pause - ensure control characters are always set appropriately
On some systems, curses.tigetstr() returns None, which does not work as a control character.
* Add unit tests
* Sort imports
* Skip on older Python
This is an action plugin and only runs on the controller, so no need to test of Python 2. Making
the import hackery work on Python 2 would required some more work which I am not sure is
worth it since we are moving away from Python 2 support on the controller.
* Make the tests work on Python 2 and 3
References to Shippable were changed to Azure Pipelines.
Also remove rebalance.py as it does not work with Azure Pipelines due to the required data not being present.
* Fix section for inventory cache prefix
* Redirect general config inventory cache options to inventory plugin documentation
* Update the template to fix the docs-build error
* removes 'default' entries for config settings with default: ''
* single backticks => italics, double backticks => code
* use 'Default: ~' to match other entries that have no default setting
Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
Change:
- After we output the newline in help text, we also added a space. This
caused either a space to show up before the shell prompt (in shells
like bash), or " %" and a newline before the next prompt on shells
like zsh that automatically force commands to end with a newline.
- This change removes the extra space, so that peace can be restored in
the minds of many.
- This only showed up when running commands with no arguments or invalid
arguments.
Test Plan:
- Ran `ansible`, `ansible -h`, `ansible-playbook`, `ansible-galaxy`, and
`ansible-galaxy -h`.
Tickets:
- Introduced in #69458
Signed-off-by: Rick Elrod <rick@elrod.me>
* dont specify default for port, allow ssh/config
also added general note on how defaults work.
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Sam Doran <sdoran@redhat.com>
For unknown reasons, these tests started failing recently because the results
from testhost11 are being returned before testhost10, but only when coverage is
enabled.
Setting the initial host sleep time to 0 fixes this failure.
ci_coverage
* Fix get_available_python_versions calls.
* Make run_playbook vars optional.
* Use ansible_pipelining in inventory.
* Fix type hint.
* Fix order of conditional evaluation.
* Remove unused ibmi platform.
* Add changelog fragment.
Change:
- Allow overriding python version as env var for Makefile
- Including hacking directory in git export, for docs build
Test Plan:
- Tested as part of recent downstream work
Co-authored-by: Yanis Guenane <yguenane@gmail.com>
* Fix BaseFileCacheModule#keys to respect prefix
Change:
- Previously BaseFileCacheModule#keys would return keys with the cache
prefix. These keys are impossible to retrieve from the cache without
removing the prefix or using the cache without a prefix.
Now it removes the prefix from the key and only returns keys that
share the same prefix as the cache.
Test Plan:
- New unit tests
* Add explicit BaseFileCacheModule#keys test
Test that files that do not match the cache prefix are ignored.
Test that the prefix is removed from the cache key.
Change:
- Added Fedora 34 container image to docker.txt
- Dropped Fedora 32 from CI
- Added Fedora 34 to CI
- Updated Fedora 32 and 33 containers for ssh-related fixes
- Move cron fix from 698eae3f3d into
cron_setup and make it more generic; it can affect modern distros too.
Test Plan:
- ci_complete
Signed-off-by: Rick Elrod <rick@elrod.me>
Due to the recent security incident, use our own copy hosted in S3
to mitigate future risk from running an arbitrary script downloaded from a
remote and untrtusted server.
* dont rely on vars, task already gives us info
* ensure we always display delegation in host label
* also added parens with ansible_host to show target host vs resolved host
* delegating to self is not delegating
* delegated vars restoration for backwards compat
* tests need mock task with delegate_to
* correctly parse device from string
* check for command presence before running them
* check for command presence and return code for solaris and aix as well
* add changelog
Improve the documentation for regex_search by adding a few more
realistic examples such as showing how to extract only parts of a
string by using regex groups.
Extend regex_findall and regex_replace by an example that includes
the multiline and ignorecase parameters.
Previously it was hosted on bintray, but that service is shutting down on May 1. Using the new
repository also required using a newer version of RabbitMQ.
Change:
- Instead of returning the `str` type, return the value that was
calculated.
Test Plan:
- New unit tests.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Fix world readable setting
now uses shell option in all cases, the old constant still exists
as 'documentation' but it not settable.
also fix the docsite link in warnings/errors
* fix typose#
* fix comment#
* added deprecated to config
* missing :
* import cleanup
* Update default containers to 3.4.0.
The 3.4.0 containers use Python 3.6 (the system Python) for `/usr/bin/python3`.
Python 3.9 continues to be the default Python version selected by `ansible-test` for these containers.
* Fix shebang on build-ansible.py.
Using `python` instead of `python3` allows `ansible-test` python interception and requirements install to function.
Fixes#74255
* Fix call to 'unique(case_sensitive=False)' triggering error when falling back to Ansible's version which **is** case-sensitive
* Test multiple situations of 'unique' filter errors with fallback not handling specific parameters
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Rick Elrod <rick@elrod.me>
Changed FAQ examples to conform with the Jinja documentation:
If both values on either side of a plus/+ are numbers, they will be added whereas using "~" will convert all operands into strings and then concatenate them. Closes#73799.
Homebrew's default install location for macOS on ARM is /opt/homebrew.
Source: https://docs.brew.sh/FAQ
On a Mac M1 (Apple Silicon), homebrew will be installed at
/opt/homebrew/bin/brew.
The unarchive integration test installs pinned packages via pip,
the uri test installs them from system packages. This explodes.
PR lets unarchive uninstall packages after test ran.
* Add yaml utils file and use it
* Linting fix and missing import
* Abstract a few more details
* Parser imports
* Don't use CSafeDumper in AnsibleDumper
* Move and use convert_yaml_objects_to_native when libyaml is present
* yaml_load and yaml_dump, instead of safe_*
* re-use HAS_LIBYAML from utils.yaml
* add changelog fragment
* Address recent changes
* Use representer instead of recursive type converter
* Restore needed import
* move yaml utils to module_utils
* Properly guard imports
* Update from_yaml(_all)? to handle text wrappers with CSafeLoader
* Use yaml utils for legacy_collection_loader
* Add HAS_YAML, and ignore pylint issue
* oops
* GPL->BSD
* Map Debian 8 to Python 2
If Python 3 is installed on Debian 8 Ansible cannot run, as the version
is too old (3.4)
* Add integration test for python interpreter discovery on Debian 8
* fix test issue on Debian 9, add changelog
* un"fix" not broken test :D
Co-authored-by: Fabian Klemp <fabian.klemp@elara-gmbh.de>
Co-authored-by: Matt Davis <mrd@redhat.com>
* Add ArgumentSpecValidator to docs
* Improve docs for ArgumentSpecValidator
* Document removal of private methods
* Update module_utils documentation
- Add docs for argument spec classes as well as validation and parameters files.
- preserve the order in the source for errors.py
- document DEFAULT_TYPE_VALIDATORS so it can be referenced elsewhere
- fix automodule directive for validation.py
* Update docs in arg_spec and paremeters
- This improves the generated documentation.
* Document breaking changes in porting guide.
* Update formatting in porting guide and add a Deprecated section
* Fine tune module_utils documentation
* Move instance docstring to the __init__ method
Remove optional description since it fails the sanity test and I am not 100% it is valid anyway.
* Remoe incorrect parameter from docstring
This was changed a while ago but wasn't removed from the docstring.
* Use attr rather than attribute
The py:attribute: domain only exists in newer Sphinx >= 3.1.
* Improve documentation on exceptions
* Final pass
- use args/kwargs instead of param
- fix formatting errors that didn't display examples correctly
- format TypeErrors so they are referenced as classes
- specify complex types
Document that users must pass `allow_unsafe=True` as an option in the lookup to allow templating, with a note about security implications.
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
Co-authored-by: Sandra McCann <samccann@redhat.com>
* Add umask option to user module
* Fail on setting both umask and local: True
* Add integration test
* Add changelog
* Run integration tests only if HOME_MODE is not set
* Run integration tests only on Linux
Co-authored-by: Matt Clay <matt@mystile.com>
* fix module schema
not out of date hardcoded list anymore, uses 'current + 4' to
set valid deprecation targets
Co-authored-by: Matt Clay <matt@mystile.com>
Change:
- This works around an issue that causes the cron test to fail sometimes
on el6.
Test Plan:
- ansible-test integration cron --docker centos6
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Newer Solaris drops setfacl. Add a fallback for its chmod ACL syntax.
Test Plan:
- New units
Tickets:
- Fixes#74282
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Regression introduced in #70785
- When macOS chmod ACL syntax is used, Solaris-derived chmods return
with a status of 5. This is also used for our sshpass handling,
because sshpass will return 5 on auth failure. This means on Solaris,
we incorrectly assume auth failure when we reach this branch of logic
and try to run chmod with macOS syntax.
- We now wrap this specific use of chmod in an exception handler that
looks for AnsibleAuthenticationFailure and skips over it. This adds
another authentication attempt (something we normally avoid to prevent
account lockout), but seems better than the regression of not allowing
other fallbacks to be used.
- Without this patch, if setfacl fails on Solaris (and sshpass is used),
we do not try common_remote_group or world-readable tmpdir fallbacks.
Test Plan:
- New unit
Signed-off-by: Rick Elrod <rick@elrod.me>
- Unit tests for `modules` and `module_utils` are now limited to importing only `ansible.module_utils` from the `ansible` module.
- Unit tests other than `modules` and `module_utils` are now run only on Python versions supported by the controller (Python 3.8+).
- Unit tests are now run in separate contexts (`controller`, `modules`, `module_utils`), each using separate invocations of `pytest`.
* Handle netconf plugin ncclient import error when running in FIPS mode
* While running in FIPS mode importing ncclient result in
InternalError raised by cryptography
* Refer https://github.com/ansible/ansible/pull/65477
- Improve code reuse.
- Add missing type hints, fix existing ones and convert them to PEP 484 style.
- Add missing imports and clean up existing ones.
- Add missing docstrings and clean up existing ones.
This simplifies rendering the hostname (or hostname+delegated host) in
the default callback module, and reduces code duplication
I've chosen not move where in each handler the host label is rendered,
in case subsequent operations has side effects. However I'm happy to
change that if considered safe.
I've chosen not to change the formatting operator used (%), to avoid
changes in rendering that might result.
Signed-off-by: Alex Willmer <alex@moreati.org.uk>
Change:
- We were passing a directory name directly to re.compile().
If the directory isn't valid regex (or is) this can have odd side
effects, such as crashing.
- Fix a few other similar cases, but less likely to be a real issue.
Test Plan:
- New test
Signed-off-by: Rick Elrod <rick@elrod.me>
* Catch errors getting filters, and fail
* Add changelog
* Switch to warnings instead of errors, to match other plugin loader behavior
* Add tests
* Handle collections
* deprecated include
Update lib/ansible/modules/_include.py
updated version numbers in schema check (real fix in separate PR)
Co-authored-by: flowerysong <junk+github@flowerysong.com>
Co-authored-by: Matt Clay <matt@mystile.com>
* all lookups to support config system
- added get_options to get full dict with all opts
- fixed tests to match new error messages
- kept inline string k=v parsing methods for backwards compat
- placeholder depredation for inline string k=v parsing
- updated tests and examples to also show new way
- refactored and added comments to most custom k=v parsing
- added missing docs for template_vars to template
- normalized error messages and exception types
- fixed constants default
- better details value errors
Co-authored-by: Felix Fontein <felix@fontein.de>
* various fixes to command
- Updated splitter to allow for all expected args in ad-hoc
- Ensure we always return the returns we promissed to always return (i.e stderr/stdout)
- Updated docs to clarify creates/removes precdence in checking
- Removed abspath from chdir to allow reporting to handle symlinks correctly
- Corrected tests to new output messages
Co-authored-by: Felix Fontein <felix@fontein.de>
* Support omitting the trailing separator when a dictionary key's value is an empty string
* Support a default value when the value used in the group name is an empty string
* Add tests
* change log
Change:
- When a "distro" package exists in PYTHONPATH but isn't what we expect,
fall back to our own vendored one and use it. This prevents a
traceback if someone has some random thing that provides "distro" but
isn't actually the "distro" library we need.
Test Plan:
- new tests
Tickets:
- Fixes#74228
Signed-off-by: Rick Elrod <rick@elrod.me>
* nuke playbook test file
Signed-off-by: Rick Elrod <rick@elrod.me>
* test fixes
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Tests for advanced_host_list; it is now at 100% coverage.
- There was a typo (host vs hostnames) where when the host pattern
failed to parse, instead of treating the name as a literal as
intended, it would trigger UnboundLocalError. This didn't fatal
as there's a global "Exception" handler below, but it did lead to
an ugly error and incorrect behavior.
Test Plan:
- New tests
- Local experimenting
Signed-off-by: Rick Elrod <rick@elrod.me>
* changelog
Signed-off-by: Rick Elrod <rick@elrod.me>
* fix tests
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Mostly increase error coverage for various conditions in play.py
- Also fix a string in an error, where get_name() was called before
self.name was read in, so get_name() was always ''.
Test Plan:
- new tests
Signed-off-by: Rick Elrod <rick@elrod.me>
* Fix regex for py2 and py3
Signed-off-by: Rick Elrod <rick@elrod.me>
* py2 hates me
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Variables used in ``when`` conditionals are no longer parsed and
attempted to be converted to booleans. All non-empty strings are
considered true (empty strings, false).
Test Plan:
- Updated existing tests
- Added a bunch of new tests with various kinds of truthy/falsy
values.
Tickets:
- Fixes#74134
Signed-off-by: Rick Elrod <rick@elrod.me>
use load_name of valid plugin to find config, will handle the renamed loaded plugin that collections do:
`ansible_collection.<ns>.<collname>.plugins.<type>.<name>`
Change:
- Previously when the same package name was installed twice under
different architectures, we only reported it once in changes.updated.
- This was the result of using a dict internally and keying on package
name alone.
- This change still keys on package name but turns the values into lists
which can contain multiple packages per name.
Test Plan:
- Added a lot of tests around multi-arch support
- Added some tests around virtual provides as well
Tickets:
- Fixes#73284
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Remove two deprecated features
- We now error if a playbook is an empty list instead of just skipping
- We now error if using 'include' instead of 'import_playbook'
Test Plan:
- Added new tests for new errors
Tickets:
- Fixes#74133
Signed-off-by: Rick Elrod <rick@elrod.me>
* sanity & changelog
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Require name always
- Drop 'reboot' parameter in favor of 'special_time: reboot'
Test Plan:
- CI
Tickets:
- fixes#74132
Signed-off-by: Rick Elrod <rick@elrod.me>
The `a/` and `b/` prefixes can be disabled in the `git diff` output by setting
`diff.noprefix` to `true`. The output is still a valid diff, but `ansible-test` would
raise an exception since without the prefixes, it thought the diff line was invalid.
tar -C deb-build/$${DIST} -xvf dist/$(NAME)-$(VERSION).tar.gz ;\
cp -a packaging/debian deb-build/$${DIST}/$(NAME)-$(VERSION)/ ;\
sed -ie "s|%VERSION%|$(DEB_VERSION)|g;s|%RELEASE%|$(DEB_RELEASE)|;s|%DIST%|$${DIST}|g;s|%DATE%|$(DEB_DATE)|g" deb-build/$${DIST}/$(NAME)-$(VERSION)/debian/changelog ;\
- ansible-galaxy CLI - ``collection verify`` command now exits with a non-zero exit code on verification failure
- ansible-galaxy CLI - ``collection verify`` command now supports a ``--offline`` option for local-only verification
Bugfixes
--------
- Correctly set template_path and template_fullpath for usage in template lookup and action plugins.
- Try to avoid kernel 'blocking' state on reading files while fact gathering.
- apt - fix policy_rc_d parameter throwing an exception when restoring original file (https://github.com/ansible/ansible/issues/66211)
- argument spec validation - fix behavior of ``apply_defaults=True`` when an empty dictionary is specified for such an option (https://github.com/ansible/ansible/pull/74029).
- pause - do not accept enter to continue when a timeout is set (https://github.com/ansible/ansible/issues/73948)
- setup module, fix error handling on bad subset given
- wait_for module, move missing socket into function to get proper comparrison in time.
- AnsibleModule - use ``ArgumentSpecValidator`` class for validating argument spec and remove private methods related to argument spec validation. Any modules using private methods should now use the ``ArgumentSpecValidator`` class or the appropriate validation function.
Minor Changes
-------------
- Callbacks - Migrate more places in the ``TaskExecutor`` to sending callbacks directly over the queue, instead of sending them as ``TaskResult`` and short circuiting in the Strategy to send the callback. This enables closer to real time callbacks of retries and loop results (https://github.com/ansible/ansible/issues/73899)
- setup - fix distribution facts for Older Amazon Linux with ``/etc/os-release`` (https://github.com/ansible/ansible/issues/73946).
Bugfixes
--------
- Fix adding unrelated candidate names to the plugin loader redirect list.
- Strategy - When building the task in the Strategy from the Worker, ensure it is properly marked as finalized and squashed. Addresses an issue with ``ansible_failed_task``. (https://github.com/ansible/ansible/issues/57399)
- ansible-pull - Run all playbooks that when multiple are supplied via the command line (https://github.com/ansible/ansible/issues/72708)
- find module, fix default pattern when use_regex is true.
- ansible-test - The generated ``resource_prefix`` variable now meets the host name syntax requirements specified in RFC 1123 and RFC 952. The value used for local tests now places the random number before the hostname component, rather than after. If the resulting value is too long, it will be truncated.
- ansible-test validate-modules - option names that seem to indicate they contain secret information that should be marked ``no_log=True`` are now flagged in the validate-modules sanity test. False positives can be marked by explicitly setting ``no_log=False`` for these options in the argument spec. Please note that many false positives are expected; the assumption is that it is by far better to have false positives than false negatives (https://github.com/ansible/ansible/pull/73508).
- distribution - add facts about Amazon Linux Distribution facts (https://github.com/ansible/ansible/issues/73742).
- module payload builder - module_utils imports in any nested block (eg, ``try``, ``if``) are treated as optional during module payload builds; this allows modules to implement runtime fallback behavior for module_utils that do not exist in older versions of Ansible.
Bugfixes
--------
- A handler defined within a role will now search handlers subdir for included tasks (issue https://github.com/ansible/ansible/issues/71222).
- ALLOW_WORLD_READABLE_TMP, switched to 'moved' message as 'deprecation' is misleading since config settings still work w/o needing change.
- Automatically remove async cache files for polled async tasks that have completed (issue https://github.com/ansible/ansible/issues/73206).
- Deal with failures when sorting JSON and you have incompatible key types.
- Setup virtualization_facts - add RHV and oVirt type. This change will fully work for VMs in clusters at cluster level 4.4 or newer (https://github.com/ansible/ansible/pull/72876).
- [set_fact] Corrected and expanded documentation as well as now raise errors that were previously ignored.
- ansible-test - ensure unit test paths for connection and inventory plugins are correctly identified for collections (https://github.com/ansible/ansible/issues/73876).
- connection/ssh, ensure parameters come from correct source get_option, so functionality matches docs.
- connection/ssh, fix reset to use same parameters to check if socket exists as actually used, was hardcoded to default string construction previouslly.
- j2 plugin loader clarified comments, made note with better fqcn detection.
- notify keyword is not ignored anymore on import_tasks, also able to apply to blocks now.
- restrict module valid JSON parsed output to objects as lists are not valid responses.
- setup, don't give up on all local facts gathering if one script file fails.
- su become plugin, ensure correct type for localization option.
- A collection can be reinstalled with new version requirements without using the ``--force`` flag. The collection's dependencies will also be updated if necessary with the new requirements. Use ``--upgrade`` to force transitive dependency updates.
- Declared ``resolvelib >= 0.5.3, < 0.6.0`` a direct dependency of
- It became possible to install Ansible Collections from local folders and namespaces folder similar to SCM structure with multiple collections.
- It became possible to upgrade Ansible collections from Galaxy servers using the ``--upgrade`` option with ``ansible-galaxy collection install``.
- Support for role argument specification validation at role execution time. When a role contains an argument spec, an implicit validation task is inserted at the start of role execution.
- add ``ArgumentSpecValidator`` class for validating parameters against an argument spec outside of ``AnsibleModule`` (https://github.com/ansible/ansible/pull/73335)
Minor Changes
-------------
- Add ``--format`` CLI option to ``ansible-galaxy collection list`` which allows for ``human`` (default), ``yaml``, or ``json``. (https://github.com/ansible/ansible/pull/73474)
- Add an example for using var in with_sequence (https://github.com/ansible/ansible/issues/68836).
- Add new rolespec_validate option to the import/include_role modules do allow disabling of the implicit role arg validation task on a per-role basis.
- Add option to pass extra vars to ansible-inventory
- Add path of collection location in Ansible CLI version info.
- Add standard Python 2/3 compatibility boilerplate to setup script, module_utils and docs_fragments which were missing them.
- Add support for `argument_specs` data in role metadata.
- Add support for datetime.date object type in module result (https://github.com/ansible/ansible/issues/70583).
- Add which conditional is being evaluated at each step when debugging.
- Add yum/dnf version comparison documentation for package install
- Added NO_COLOR environment var to ansible color configuration, allowing it to integrate with existing convention.
- Added name of aliases in user error (https://github.com/ansible/ansible/issues/58752).
- Added support for GSSAPI/Kerberos authentication with ``urls.py`` that is used by ``uri`` and ``get_url``.
- Added support for specify custom credentials for GSSAPI authentication.
- Allow an attribute to be passed to the min and max filters with Jinja 2.10+
- Allow for the skipped filter to be used on a registered looped task results. (https://github.com/ansible/ansible/issues/16949)
- Allow inventory plugins access to extra vars by default
- Allow unsafe_writes to be set on target via env var, for those targets that need a blanket setting.
- Also added extra vars cli option to console CLI.
- AnsiballZ - Improve performance of ``ModuleDepFinder`` by using faster lookups and reducing the object types that are walked while looking for ``import`` statements. (https://github.com/ansible/ansible/pull/70475)
- CLI - Specify jinja version in ``--version`` output
- CLI - Specify whether PyYAML includes libyaml support in version output
- CLI version displays clarified as core version
- Collection routing: Cisco NSO content from community.network migrated to cisco.nso (https://github.com/ansible/ansible/pull/73046).
- Collection routing: DellEMC content from community.general migrated to dellemc.openmanage (https://github.com/ansible/ansible/pull/73046).
- Collection routing: FortiOS content from community.network migrated to community.fortios (https://github.com/ansible/ansible/pull/73046).
- Collection routing: Google content from community.general migrated to community.google (https://github.com/ansible/ansible/pull/73046).
- Collection routing: Hashi Vault content from community.general migrated to community.hashi_vault (https://github.com/ansible/ansible/pull/73046).
- Collection routing: Hetzner Robot content from community.general migrated to community.hrobot (https://github.com/ansible/ansible/pull/73046).
- Collection routing: KubeVirt content from community.general migrated to community.kubevirt (https://github.com/ansible/ansible/pull/73046).
- Collection routing: OC content from community.general migrated to community.okd (https://github.com/ansible/ansible/pull/73046).
- Collection routing: PostgreSQL content from community.general migrated to community.postgresql (https://github.com/ansible/ansible/pull/73046).
- Collection routing: RouterOS content from community.network migrated to community.routeros (https://github.com/ansible/ansible/pull/73046).
- Collection routing: docker content from community.general migrated to community.docker (https://github.com/ansible/ansible/pull/73046).
- Controller - Add warning for Ansible 2.11 when running a Python version older than Python 3.8 to inform users that 2.12 will only support Python 3.8 and newer on the controller. Starting with Ansible 2.11, the project will only be packaged for Python 3.8 and newer.
- Discourage the use of 'hexdigits' in password lookup, as it distorts expected entropy.
- Enable extra vars for inventory plugin options
- Errors - Ensure that errors passed with ``orig_exc`` include the context of that exception (https://github.com/ansible/ansible/issues/68605)
- Filters - Add new ``split`` filter for splitting strings
- Fixed ansible-doc to not substitute for words followed by parenthesis. For instance, ``IBM(International Business Machines)`` will no longer be substituted with a link to a non-existent module. https://github.com/ansible/ansible/pull/71070
- Force the template module to use non-native Jinja2 (https://github.com/ansible/ansible/issues/46169)
- Internal config entries will not be documented, to mark an entry as internal it must start with `_`.
- Interpreter Discovery - Add Python 3.8 and Python 3.9 to the fallback list
- Minor code cleanup in plugin loader.
- Module API - libselinux-python is no longer required for basic module API selinux operations (affects core modules assemble, blockinfile, copy, cron, file, get_url, lineinfile, setup, replace, unarchive, uri, user, yum_repository)
- Module API - new module_respawn API allows modules that need to run under a specific Python interpreter to respawn in place under that interpreter
- Module iptables multiport destination support added (https://github.com/ansible/ansible/pull/72928)
- Module iptables set/ipset support added (https://github.com/ansible/ansible/pull/72984)
- New 'timeout' feature added to adhoc and console CLIs, corresponding to the recent 'timeout' task keyword.
- New virtualization facts, ``virtualization_tech_guest`` and ``virtualization_tech_host`` now allow for conveying when a system is a host or guest of multiple virtualization technologies.
- Now 'choices' keyword in config definitions also restricts valid values for the entry.
- Refactored ``ansible-galaxy collection [download|install|list|verify]`` CLI subcommands with the public interface kept intact.
- Restructured _fixup_perms2() in ansible.plugins.action to make it more linear
- Shadow prompt input to ansible-vault encrypt-string unless the ``--show-input`` flag is set
- Switch to hashlib.sha256() for ansible-test to allow for FIPs mode.
- TOML inventory plugin is no longer in preview status
- Templar - reduce the complexity of ``Templar._lookup`` (https://github.com/ansible/ansible/pull/73277)
- The ``csvfile`` lookup plugin now uses ``parse_kv()`` internally. As a result, multi-word search keys can now be passed.
- The ``csvfile`` lookup plugin's documentation has been fixed; it erroneously said that the delimiter could be ``t`` which was never true. We now accept ``\t``, however, and the error in the documentation has been fixed to note that.
- The constructed inventory plugin has new option to force using vars plugins on previouslly processed inventory sources.
- The find module is now more specific about the reasons it skips candidate files.
- The logging functionality in module_utils.basic now returns a nicer error when it falls back to syslog but ends up getting a TypeError thrown back.
- The new dependency resolver prefers ``MANIFEST.json`` over ``galaxy.yml`` if it exists in the target directory.
- The plugin loader now keeps track of the collection where a plugin was resolved to, in particular whether the plugin was loaded from ansible-core's internal paths (``ansible.builtin``) or from user-supplied paths (no collection name).
- Toggle allowing usage of extra_vars in compose
- When connecting as an unprivileged user, and becoming an unprivileged user, we now fall back to also trying ``chmod +a`` which works on macOS and makes use of ACLs.
- allow tree callback plugin to be configurable, for use with playbooks.
- ansible-doc - In Windows setup steps, ``ExecutionPolicy`` should be restored to default value ``RemoteSigned`` (https://github.com/ansible/ansible/pull/72993).
- ansible-doc - provide ``has_action`` field in JSON output for modules. That information is currently only available in the text view (https://github.com/ansible/ansible/pull/72359).
- ansible-doc has new option to show keyword documentation.
- ansible-doc will now format, ``L()``, ``R()``, and ``HORIZONTALLINE`` in plugin docs just as the website docs do. https://github.com/ansible/ansible/pull/71070
- ansible-galaxy - Added caching mechanisms when retrieving collection info to speed up installs and downloads
- ansible-galaxy - Change the output verbosity level of the download message from 3 to 0 (https://github.com/ansible/ansible/issues/70010)
- ansible-galaxy - Ensure ``get_collection_versions`` returns an empty list when a collection does not exist for consistency across API versions.
- ansible-galaxy - find any collection dependencies in the globally configured Galaxy servers and not just the server the parent collection is from.
- ansible-test - A warning is no longer emitted when a ``pip*`` or ``python*`` binary is found without a matching couterpart.
- ansible-test - Add ``macos/10.15`` as a supported value for the ``--remote`` option.
- ansible-test - Add a ``--docker-network`` option to choose the network for running containers when using the ``--docker`` option.
- ansible-test - Add support for running tests on Fedora 33 (https://github.com/ansible/ansible/pull/72861).
- ansible-test - Added Ubuntu 20.04 LTS image to the default completion list
- ansible-test - Added a ``--export`` option to the ``ansible-test coverage combine`` command to facilitate multi-stage aggregation of coverage in CI pipelines.
- ansible-test - Added the ``-remote rhel/7.9`` option to run tests on RHEL 7.9
- ansible-test - Allow custom ``--remote-stage`` options for development and testing.
- ansible-test - CentOS 8 container is now 8.2.2004 (https://github.com/ansible/distro-test-containers/pull/45).
- ansible-test - Changed the internal name of the custom plugin used to identify use of unwanted imports and functions.
- ansible-test - Cleaned up code to resolve warnings and errors reported by PyCharm.
- ansible-test - Code cleanup in the ``import`` sanity test.
- ansible-test - Code cleanup in the internal logic for code coverage collection of PowerShell modules.
- ansible-test - Collections can now specify pip constraints for unit and integration test requirements using ``tests/unit/constraints.txt`` and ``tests/integration/constraints.txt`` respectively.
- ansible-test - Containers used with the ``--remote`` option have been updated to version 1.29.0 to include the latest Ansible requirements.
- ansible-test - Files used to track remote instances no longer have a region suffix.
- ansible-test - Fix container hostname/IP discovery for the ``acme`` test plugin.
- ansible-test - FreeBSD 11.4 and 12.2 provisioning can now be used with the ``--python 3.8`` option.
- ansible-test - FreeBSD instances provisioned with ``--remote`` now make ``libyaml`` available for use with PyYAML installation.
- ansible-test - Generation of an ``egg-info`` directory, if needed, is now done after installing test dependencies and before running tests. When running from an installed version of ``ansible-test`` a temporary directory is used to avoid permissions issues. Previously it was done before installing test dependencies and adjacent to the installed directory.
- ansible-test - Implemented CloudStack test container selection by ENV variable `ANSIBLE_CLOUDSTACK_CONTAINER` with a default to `quay.io/ansible/cloudstack-test-container:1.4.0`.
- ansible-test - Improved handling of minimum Python version requirements for sanity tests. Supported versions are now included in warning messages displayed when tests are skipped.
- ansible-test - More sanity test requirements have been pinned to specific versions to provide consistent test results.
- ansible-test - Most sanity test specific ``pip`` constraints are now used only when running sanity tests. This should reduce conflicts with ``pip`` requirements and constraints when testing collections.
- ansible-test - Most sanity tests are now skipped on Python 3.5 and earlier with a warning. Previously this was done for Python 2.7 and earlier.
- ansible-test - Now supports freebsd/11.4 remote (https://github.com/ansible/ansible/issues/48782).
- ansible-test - Now supports freebsd/12.2 remote (https://github.com/ansible/ansible/issues/72366).
- ansible-test - OpenSuse container now uses Leap 15.2 (https://github.com/ansible/distro-test-containers/pull/48).
- ansible-test - Pin the ``virtualenv`` version used for ``--remote`` pip installs to the latest version supported by Python 2.x, which is version 16.7.10.
- ansible-test - Provisioning of RHEL instances now includes installation of pinned versions of ``packaging`` and ``pyparsing`` to match the downstream vendored versions.
- ansible-test - RHEL 8.2+ provisioning can now be used with the ``--python 3.8`` option, taking advantage of the Python 3.8 AppStream.
- ansible-test - Raise the number of bytes scanned by ansible-test to determine if a file is binary to 4096.
- ansible-test - Refactor code for installing ``cryptography`` to allow re-use in the future.
- ansible-test - Refactor code to remove unused logic for obsolete support of multiple provisioning endpoints.
- ansible-test - Remove ``pytest < 6.0.0`` constraint for managed installations on Python 3.x now that pytest 6 is supported.
- ansible-test - Remove em dash from the Pytest configuration file in order to be readable on systems where preferred encoding is ASCII. (https://github.com/ansible/ansible/issues/71739)
- ansible-test - Removed ``pip`` constraints related to integration tests that have been moved to collections. This should reduce conflicts with ``pip`` requirements and constraints when testing collections.
- ansible-test - Removed the obsolete ``--remote-aws-region`` provisioning option.
- ansible-test - Removed the obsolete ``tower`` test plugin for testing Tower modules.
- ansible-test - Removed unused provisioning code and cleaned up remote provider management logic.
- ansible-test - Rename internal functions to match associated constant names that were previously updated.
- ansible-test - Reorganize internal ``pylint`` configuration files for easier comparison and maintenance.
- ansible-test - Report the correct line number in the ``yamllint`` sanity test when reporting ``libyaml`` parse errors in module documentation.
- ansible-test - Request remote resources by provider name for all provider types.
- ansible-test - Show a warning when the obsolete ``--remote-aws-region`` option is used.
- ansible-test - Silence ``pip`` warnings about Python 3.5 being EOL when installing requirements.
- ansible-test - Support custom remote endpoints with the ``--remote-endpoint`` option.
- ansible-test - The ``--remote`` option no longer pre-installs the ``virtualenv`` module on Python 3.x instances. The Python built-in ``venv`` module should be used instead.
- ansible-test - The ``default`` container for both collections and core have been updated to versions 2.11.0 and 1.9.0 respectively.
- ansible-test - The ``pylint`` sanity test is now skipped with a warning on Python 3.9 due to unresolved upstream regressions.
- ansible-test - The ``pylint`` sanity test is now supported on Python 3.8.
- ansible-test - The ``rstcheck`` sanity test is no longer used for collections, but continues to be used for ansible-core.
- ansible-test - Ubuntu containers as well as ``default-test-container`` and ``ansible-base-test-container`` are now slightly smaller due to apt cleanup (https://github.com/ansible/distro-test-containers/pull/46).
- ansible-test - Update ``pylint`` and its dependencies to the latest available versions to support Python 3.9.
- ansible-test - Update built-in service endpoints for the ``--remote`` option.
- ansible-test - Updated the default test containers to version 3.1.0.
- ansible-test - Upgrade ansible-runner version used in compatibility tests, remove some tasks that were only needed with older versions, and skip in python2 because ansible-runner is soon dropping it.
- ansible-test - Use new endpoint for Parallels based instances with the ``--remote`` option.
- ansible-test - ``default-test-container`` and ``ansible-base-test-container`` now use Python 3.9.0 instead of 3.9.0rc1.
- ansible-test - add https endpoint for ansible-test
- ansible-test - add macOS 11.1 as a remote target (https://github.com/ansible/ansible/pull/72622)
- ansible-test - add the collection plugin directories ``plugin_utils`` and ``sub_plugins`` to list of plugin types. This ensures such plugins are tested for the ``import`` sanity test (https://github.com/ansible/ansible/pull/73599).
- ansible-test - centos6 end of life - container image updated to point to vault base repository (https://github.com/ansible/distro-test-containers/pull/54)
- ansible-test - centos6 image now has multiple fallback yum repositories for CentOS Vault.
- ansible-test - default container now uses default-test-container 2.7.0 and ansible-base-test-container 1.6.0. This brings in Python 3.9.0rc1 for testing.
- ansible-test - now makes a better attempt to support podman when calling ``docker images`` and asking for JSON format.
- ansible-test - python-cryptography is now bounded at <3.2, as 3.2 drops support for OpenSSL 1.0.2 upon which some of our CI infrastructure still depends.
- ansible-test - remote macOS instances no longer install ``virtualenv`` during provisioning
- ansible-test - the ACME test container was updated, it now supports external account creation and has a basic OCSP responder (https://github.com/ansible/ansible/pull/71097, https://github.com/ansible/acme-test-container/releases/tag/2.0.0).
- ansible-test - the ``import`` sanity test now also tries to import all non-module and non-module_utils Python files in ``lib/ansible/`` resp. ``plugins/`` (https://github.com/ansible/ansible/pull/72497).
- ansible-test - virtualenv helper scripts now prefer ``venv`` on Python 3 over ``virtualenv``
- ansible-test Now supports RHEL 8.3
- ansible-test pylint - ensure that removal collection version numbers conform to the semantic versioning specification at https://semver.org/ (https://github.com/ansible/ansible/pull/71679).
- ansible-test runtime-metadata - compare deprecation and tombstone versions to the current version to ensure that they are correct (https://github.com/ansible/ansible/pull/72625).
- ansible-test runtime-metadata - ensure that removal collection version numbers conform to the semantic versioning specification at https://semver.org/ (https://github.com/ansible/ansible/pull/71679).
- ansible-test runtime-metadata - ensure that the tombstone removal date is not in the future (https://github.com/ansible/ansible/pull/72625).
- ansible-test runtime-metadata - validate removal version numbers, and check removal dates more strictly (https://github.com/ansible/ansible/pull/71679).
- ansible-test validate-modules - ensure that removal collection version numbers and version_added collection version numbers conform to the semantic versioning specification at https://semver.org/ (https://github.com/ansible/ansible/pull/71679).
- ansible-test validate-modules - no longer assume that ``default`` for ``type=bool`` options is ``false``, as the default is ``none`` and for some modules, ``none`` and ``false`` mean different things (https://github.com/ansible/ansible/issues/69561).
- ansible-test validate-modules - validate removal version numbers (https://github.com/ansible/ansible/pull/71679).
- ansible.utils.encrypt now returns `AnsibleError` instead of crypt.crypt's `OSError` on Python 3.9
- apt - module now works under any supported Python interpreter
- apt_repository - module now works under any supported Python interpreter
- callback plugins - ``meta`` tasks now get sent to ``v2_playbook_on_task_start``. Explicit tasks are always sent. Plugins can opt in to receiving implicit ones.
- conditionals - change the default of CONDITIONAL_BARE_VARS to False (https://github.com/ansible/ansible/issues/70682).
- config - more types are now automatically coerced to string when ``type: str`` is used and the value is parsed as a different type
- constructed - Add a toggle to allow the separator to be omitted if no prefix has been provided.
- constructed inventory plugin - Sanitize group names created from the ``groups`` option silently.
- create ``get_type_validator`` standalone function and move that functionality out of ``AnsibleModule`` (https://github.com/ansible/ansible/pull/72667)
- create ``get_unsupported_parameters`` validation function (https://github.com/ansible/ansible/pull/72447/files)
- debconf - add a note about no_log=True since module might expose sensitive information to logs (https://github.com/ansible/ansible/issues/32386).
- default callback - add ``show_task_path_on_failure`` option to display file and line number of tasks only on failed tasks when running at normal verbosity level (https://github.com/ansible/ansible/issues/64625)
- default callback - task name is now shown for ``include_tasks`` when using the ``free`` strategy (https://github.com/ansible/ansible/issues/71277).
- default callback - task name is now shown for ``include_tasks`` when using the ``linear`` strategy with ``ANSIBLE_DISPLAY_SKIPPED_HOSTS=0``.
- default_callback - moving 'check_mode_markers' documentation in default_callback doc_fragment (https://github.com/ansible-collections/community.general/issues/565).
- distribution - add support for DragonFly distribution (https://github.com/ansible/ansible/issues/43739).
- distribution - added distribution fact and hostname support for Parrot OS (https://github.com/ansible/ansible/pull/69158).
- distribution - handle NetBSD OS Family (https://github.com/ansible/ansible/issues/43739).
- distribution facts - ``distribution_release`` is now ``"Stream"`` on CentOS Stream (https://github.com/ansible/ansible/issues/73027).
- dnf - When ``state: absent``, package names are now matched similarly to how the ``dnf`` CLI matches them (https://github.com/ansible/ansible/issues/72809).
- dnf - module now works under any supported Python interpreter
- dnf - now shows specific package changes (installations/removals) under ``results`` in check_mode. (https://github.com/ansible/ansible/issues/66132)
- facts - ``/dev/kvm`` is now consulted in Linux virtualization facts, and the host is considered a KVM host if this file exists and none of the pre-existing checks matched.
- facts - add new fact ``date_time['tz_dst']``, which returns the daylight saving timezone (https://github.com/ansible/ansible/issues/69004).
- facts - add uptime to openbsd
- find module - Now has a ``read_whole_file`` boolean parameter which allows for reading the whole file and doing an ``re.search()`` regex evaluation on it when searching using the ``contains`` option. This allows (for example) for ensuring the very end of the file matches a pattern.
- galaxy - add documentation about galaxy parameters in examples/ansible.cfg (https://github.com/ansible/ansible/issues/68402).
- galaxy - handle token as dict while loading from yaml file (https://github.com/ansible/ansible/issues/70887).
- get_url - allow checksum urls to point to file:// resources, moving scheme test to function
- get_url - handle same SHA sum for checksum file (https://github.com/ansible/ansible/issues/71420).
- hash filter - fail when unsupported hash type is passed as an argument (https://github.com/ansible/ansible/issues/70258)
- inventory cache - do not show a warning when the cache file does not (yet) exist.
- iptables - add a note about ipv6-icmp in protocol parameter (https://github.com/ansible/ansible/issues/70905).
- iptables - fixed get_chain_policy API (https://github.com/ansible/ansible/issues/68612).
- iptables - reorder comment postition to be at the end (https://github.com/ansible/ansible/issues/71444).
- lineinfile - add search_string parameter for non-regexp searching (https://github.com/ansible/ansible/issues/70470)
- linux facts - Add additional check to ensure 'container' virtual fact gets added to guest_tech when appropriate (https://github.com/ansible/ansible/pull/71385)
- meta - now include a ``skip_reason`` when skipped (https://github.com/ansible/ansible/pull/71355).
- module_utils - ``get_file_attributes()`` now takes an optional ``include_version`` boolean parameter. When ``True`` (default), the file's version/generation number is included in the result (but requires ``lsattr -v`` to work on the target platform).
- now !unsafe works on all types of data, not just strings, even recursively for mappings and sequences.
- package_facts - module support for apt and rpm now works under any supported Python interpreter
- pipe lookup - update docs for Popen with shell=True usages (https://github.com/ansible/ansible/issues/70159).
- plugin examples - Allow non-YAML examples, so that examples for plugins like the INI and TOML inventory plugins can be directly represented (https://github.com/ansible/ansible/pull/71184)
- plugin option validation - now the option type ``dict``/``dictionary`` is also validated by the config manager (https://github.com/ansible/ansible/pull/71928).
- reboot - add ``reboot_command`` parameter to allow specifying the command used to reboot the system (https://github.com/ansible/ansible/issues/51359)
- remove ``excommunicate`` debug command from AnsiballZ
- unarchive - add ``include`` parameter to allow extracting specific files from an archive (https://github.com/ansible/ansible/pull/40522)
- update sphinx to 2.1.2 and rstcheck to 3.3.1 for building documentation.
- uri - add ``ca_path`` argument to allow specification of a CA certificate (https://github.com/ansible/ansible/pull/71979).
- user - add new parameters ``password_expire_max`` and ``password_expire_min`` for controlling password expiration (https://github.com/ansible/ansible/issues/68775)
- varnames lookup plugin - Fixed grammar error in exception message when the plugin is given a non-string term.
- vault - Provide better error for single value encrypted values to indicate the file, line, and column of the errant vault (https://github.com/ansible/ansible/issues/72276)
- version test - Add semantic version functionality
- virtual facts - containerd cgroup is now recognized as container tech (https://github.com/ansible/ansible/issues/66304).
- virtualization facts - Detect ``vdsmd`` in addition to ``vdsm`` when trying to detect RHEV hosts. (https://github.com/ansible/ansible/issues/66147)
- winrm - Added ``ansible_winrm_kinit_args`` that can be used to control the args that are sent to the ``kinit`` call for Kerberos authentication.
- yum - module now works under any supported Python interpreter
- yum_repository - added boolean option module_hotfixes which allows to enable functionality for dnf.
Breaking Changes / Porting Guide
--------------------------------
- Made SCM collections be reinstalled regardless of ``--force`` being present.
- NetBSD virtualization facts (specifically ``ansible_virtualization_type``) now returns a more accurate value by checking the value of the ``machdep.hypervisor````sysctl`` key. This change is breaking because in some cases previously, we would erroneously report ``xen`` even when the target is not running on Xen. This prevents that behavior in most cases. (https://github.com/ansible/ansible/issues/69352)
- Replaced the in-tree dependency resolver with an external implementation that pip >= 20.3 uses now by default — ``resolvelib``. (https://github.com/ansible/ansible/issues/71784)
- The ``meta`` module now supports tags for user-defined tasks. Internal ``meta`` tasks continue to always run. (https://github.com/ansible/ansible/issues/64558)
- ansible-galaxy login command has been removed (see https://github.com/ansible/ansible/issues/71560)
Deprecated Features
-------------------
- Starting in 2.14, shell and command modules will no longer have the option to warn and suggest modules in lieu of commands. The ``warn`` parameter to these modules is now deprecated and defaults to ``False``. Similarly, the ``COMMAND_WARNINGS`` configuration option is also deprecated and defaults to ``False``. These will be removed and their presence will become an error in 2.14.
- apt_key - the paramater ``key`` does not have any effect, has been deprecated and will be removed in ansible-core version 2.14 (https://github.com/ansible/ansible/pull/70319).
- psrp - Set the minimum version of ``pypsrp`` to ``0.4.0``.
Removed Features (previously deprecated)
----------------------------------------
- Removed `SharedPluginLoaderObj` class from ansible.plugins.strategy. It was deprecated in favor of using the standard plugin loader.
- Removed `_get_item()` alias from callback plugin base class which had been deprecated in favor of `_get_item_label()`.
- The "user" parameter was previously deprecated and is now removed in favor of "scope"
- The deprecated ``ansible.constants.BECOME_METHODS`` has been removed.
- The deprecated ``ansible.constants.get_config()`` has been removed.
- The deprecated ``ansible.constants.mk_boolean()`` has been removed.
-`with_*` loops are no longer optimized for modules whose `name` parameters can take lists (mostly package managers). Use `name` instead of looping over individual names with `with_items` and friends.
Security Fixes
--------------
-**security issue** - Mask default and fallback values for ``no_log`` module options (CVE-2021-20228)
-**security issue** - copy - Redact the value of the no_log 'content' parameter in the result's invocation.module_args in check mode. Previously when used with check mode and with '-vvv', the module would not censor the content if a change would be made to the destination path. (CVE-2020-14332)
- Sanitize no_log values from any response keys that might be returned from the uri module (CVE-2020-14330).
- dnf - Previously, regardless of the ``disable_gpg_check`` option, packages were not GPG validated. They are now. (CVE-2020-14365)
Bugfixes
--------
- ANSIBLE_COLLECTIONS_PATHS - remove deprecation so that users of Ansible 2.9 and 2.10+ can use the same var when specifying a collection path without a warning.
- Added unsafe_writes test.
- Address compat with rpmfluff-0.6 for integration tests
- Address the deprecation of the use of stdlib distutils in packaging. It's a short-term hotfix for the problem (https://github.com/ansible/ansible/issues/70456, https://github.com/pypa/setuptools/issues/2230, https://github.com/pypa/setuptools/commit/bd110264)
- Adjust various hard-coded action names to also include their ``ansible.builtin.`` and ``ansible.legacy.`` prefixed version (https://github.com/ansible/ansible/issues/71817, https://github.com/ansible/ansible/issues/71818, https://github.com/ansible/ansible/pull/71824).
- Allow TypeErrors on Undefined variables in filters to be handled or deferred when processing for loops.
- Allow `~` to be present in file names in galaxy roles (https://github.com/ansible/ansible/issues/72966)
- Always mention the name of the deprecated or tombstoned plugin in routing deprecation/tombstone messages (https://github.com/ansible/ansible/pull/73059).
- Ansible output now uses stdout to determine column width instead of stdin
- AnsibleModule - added arg ``ignore_invalid_cwd`` to ``AnsibleModule.run_command()``, to control its behaviour when ``cwd`` is invalid. (https://github.com/ansible/ansible/pull/72390)
- Apply ``_wrap_native_text`` only for builtin filters specified in STRING_TYPE_FILTERS.
- Be smarter about collection paths ending with ansible_collections, emulating a-galaxy behaviour. Issue 72628
- CLI - Restore git information in version output when running from source
- Collection callbacks were ignoring options and rules for stdout and adhoc cases.
- Collections - Ensure ``action_loader.get`` is called with ``collection_list`` to properly find collections when ``collections:`` search is specified (https://github.com/ansible/ansible/issues/72170)
- Command module now returns stdout & stderr if executable is missing or an unknown error occurs
- ConfigManager - Normalize ConfigParser between Python2 and Python3 to for handling comments (https://github.com/ansible/ansible/issues/73709)
- Continue execution when 'flatten' filter when it hits a None/null value as part of the list.
- Correct the inventory source error parse handling, specifically make the config INVENTORY_ANY_UNPARSED_IS_FAILED work as expected.
- Display - Use wcswidth to calculate printable width of a text string (https://github.com/ansible/ansible/issues/63105)
- Enabled unsafe_writes for get_url which was ignoring the paramter.
- Ensure Ansible's unique filter preserves order (https://github.com/ansible/ansible/issues/63417)
- Ensure if a traceback halts ``strategy.run`` that we still attempt to clean up (https://github.com/ansible/ansible/issues/23958)
- Ensure password passed in by -k is used on delegated hosts that do not have ansible_password set
- Ensure the correct options are used when ssh executables are used that don't match ssh executable names.
- Facts collection - get serial number of NVMe device without sg_inq (https://github.com/ansible/ansible/issues/66663).
- Fix --list-tasks format `role_name : task_name` when task name contains the role name. (https://github.com/ansible/ansible/issues/72505)
- Fix ``RecursionError`` when templating large vars structures (https://github.com/ansible/ansible/issues/71920)
- Fix ``delegate_facts: true`` when ``ansible_python_interpreter`` is not set. (https://github.com/ansible/ansible/issues/70168)
- Fix an exit code for a non-failing playbook (https://github.com/ansible/ansible/issues/71306)
- Fix ansible-galaxy collection list to show collections in site-packages (https://github.com/ansible/ansible/issues/70147)
- Fix bytestring vs string comparison in module_utils.basic.is_special_selinux_path() so that special-cased filesystems which don't support SELinux context attributes still allow files to be manipulated on them. (https://github.com/ansible/ansible/issues/70244)
- Fix execution of the meta tasks 'clear_facts', 'clear_host_errors', 'end_play', 'end_host', and 'reset_connection' when the CLI flag '--flush-cache' is provided.
- Fix fileglob bug where it could return different results for different order of parameters (https://github.com/ansible/ansible/issues/72873).
- Fix incorrect msg in the results dict in loops
- Fix incorrect re-run of roles with tags (https://github.com/ansible/ansible/issues/69848)
- Fix incorrect variable scoping when using ``import with context`` in Jinja2 templates. (https://github.com/ansible/ansible/issues/72615)
- Fix jsonfile cache plugin option '_uri' to be a type path instead of a string. (https://github.com/ansible/ansible/issues/38002)
- Fix notifying handlers via `role_name : handler_name` when handler name contains the role name. (https://github.com/ansible/ansible/issues/70582)
- Fix parsing of values when using empty string as a key (https://github.com/ansible/ansible/issues/57132)
- Fix statistics reporting when rescue block contains another block (issue https://github.com/ansible/ansible/issues/61253).
- Fix to previous deprecation change (#70504) which caused command warning deprecation to show in all cases, even when not specified by the user.
- Fixed TypeError instancemethod expecting at least 2 arguments for apt_repository(issue https://github.com/ansible/ansible/issues/69308, PR https://github.com/ansible/ansible/pull/69463)
- Fixed issue when `netstat` is either missing or doesn't have execution permissions leading to incorrect command being executed.
- Fixes ``ansible-galaxy role info`` to support multiple roles on the command line (https://github.com/ansible/ansible/pull/70148)
- Fixes ansible-test traceback when plugin author is not a string or a list of strings (https://github.com/ansible/ansible/pull/70507)
- Handle more varnames that can create conflicts, expand a function in general, handle jinja2 globals in particular (https://github.com/ansible/ansible/issues/41955).
- INTERPRETER_PYTHON_DISTRO_MAP - prefer ``/usr/libexec/platform-python`` on ``oraclelinux 8`` when other pythons are present.
- Improve Ansible config deprecations to show the source of the deprecation (ansible-core). Also remove space before a comma in config deprecations (https://github.com/ansible/ansible/pull/72697).
- Improved/fixed regular expressions in ``validate-modules/validate_modules/schema.py`` and ``utils/collection_loader/_collection_finder.py`` (https://github.com/ansible/ansible/pull/73577).
- Includes - Explicitly get the include task, and not assume it is the parent (https://github.com/ansible/ansible/issues/65710)
- InventoryManager - Fix unhandled exception when given limit file was actually a directory.
- InventoryManager - Fix unhandled exception when inventory directory was empty or contained empty subdirectories (https://github.com/ansible/ansible/issues/73658).
- JSON Encoder - Ensure we treat single vault encrypted values as strings (https://github.com/ansible/ansible/issues/70784)
- Lookup user by UID in password database if login name is not found (https://github.com/ansible/ansible/issues/17029)
- Pass expression in angle-bracket notation as filename argument to a ``compile()`` built-in function, so that Python debuggers do not try to parse it as filename.
- Pass the connection's timeout to connection plugins instead of the task's timeout.
- Provide more information in AnsibleUndefinedVariable (https://github.com/ansible/ansible/issues/55152)
- Python module_utils finder - refactor logic to eliminate many corner cases, remove recursion, fix base module_utils redirections
- Remove an embedded function from RoleMixin and add tests for it (https://github.com/ansible/ansible/pull/72754).
- Remove the warning displayed when validating the arg spec of a role with dependencies and add it to the documentation.
- Restore the ability for changed_when/failed_when to function with group_by (#70844).
- Restored unsafe_writes functionality which was being skipped.
- Restructured pipelining settings to be at the connection plugins leaving base config as global and for backwards compatiblity.
- SSH plugin - Improve error message when ssh client is not found on the host
- Skip invalid collection names when listing in ansible-doc instead of throwing exception. Issue#72257
- Skip literal_eval for string filters results in native jinja. (https://github.com/ansible/ansible/issues/70831)
- Stop adding the connection variables to the output results
- Suppress warning when user directory used in --playbook-dir option with ansible-inventory command (https://github.com/ansible/ansible/issues/65262).
- TOML inventory - Ensure we register dump functions for ``AnsibleUnsafe`` to support dumping unsafe values. Note that the TOML format has no functionality to mark that the data is unsafe for re-consumption. (https://github.com/ansible/ansible/issues/71307)
- Terminal plugins - add "\e[m" to the list of ANSI sequences stripped from device output
- The ``docker`` and ``k8s`` action groups / module default groups now also support the moved modules in `community.docker <https://galaxy.ansible.com/community/docker>`_, `community.kubevirt <https://github.com/ansible-collections/community.kubevirt>`_, `community.okd <https://galaxy.ansible.com/community/okd>`_, and `kubernetes.core <https://galaxy.ansible.com/kubernetes/core>`_ (https://github.com/ansible/ansible/pull/72428).
- The ``flush()`` method of ``CachePluginAdjudicator`` now calls the plugin's ``flush()`` method instead of iterating over the keys that the adjudicator knows about and deleting those from the cache. (https://github.com/ansible/ansible/issues/68770)
- The `ansible_become` value was not being treated as a boolean value when set in an INI format inventory file (fixes bug https://github.com/ansible/ansible/issues/70476).
- The machine-readable changelog ``changelogs/changelog.yaml`` is now contained in the release.
- Updated docs and added warning on max_fail_percentage and free strategy usage. fixes issue 16666.
- VariableManager - Add the 'vars' key before getting delegated variables (https://github.com/ansible/ansible/issues/71092).
- Vault - Allow single vault encrypted values to be used directly as module parameters. (https://github.com/ansible/ansible/issues/68275)
- account for bug in Python 2.6 that occurs during interpreter shutdown to avoid stack trace
- action plugins - change all action/module delegations to use FQ names while allowing overrides (https://github.com/ansible/ansible/issues/69788)
- add AlmaLinux to fact gathering (https://github.com/ansible/ansible/pull/73458)
- add constraints file for ``ansible_runner`` test since an update to ``psutil`` is now causing test failures
- add magic/connection vars updates from delegated host info.
- add support for alpine linux 'apk' package manager in package_facts
- allow become method 'su' to work on 'local' connection by allocating a fake tty.
- ansible-console - Ctrl+C (in a task) abort current task, and put you back on prompt (this behavior doesn't change) (ditto)
- ansible-console - Ctrl+C (on prompt) used to exit the shell, unlike most shells, it should just reset the current line (ie. abort it and spawn a new prompt) (https://github.com/ansible/ansible/issues/68529)
- ansible-console - Ctrl+D (on prompt) now exit the shell, this is the expected behavior in a shell (cf bash, sh, zsh, ipython, ...) (ditto)
- ansible-console - add more documentation, specifically on various commands[1] (https://github.com/ansible/ansible/issues/72195)
- ansible-console - fixes few strings' typos
- ansible-console - remove useless and poorly formatted comment section (replaced with [1])
- ansible-doc - account for an empty ``meta/main.yml`` file when displaying role information (https://github.com/ansible/ansible/pull/73590)
- ansible-doc - collection name for plugin top-level deprecation was not inserted when deprecating by version (https://github.com/ansible/ansible/pull/70344).
- ansible-doc - improve error message in text formatter when ``description`` is missing for a (sub-)option or a return value or its ``contains`` (https://github.com/ansible/ansible/pull/70046).
- ansible-doc - improve man page formatting to avoid problems when YAML anchors are used (https://github.com/ansible/ansible/pull/70045).
- ansible-doc - include the collection name in the text output (https://github.com/ansible/ansible/pull/70401).
- ansible-doc - plugin option deprecations now also get ``collection_name`` added (https://github.com/ansible/ansible/pull/71735).
- ansible-doc - properly show plugin name when ``name:`` is used instead of ``<plugin_type>:`` (https://github.com/ansible/ansible/pull/71966).
- ansible-galaxy - Cache the responses for available collection versions after getting all pages. (https://github.com/ansible/ansible/issues/73071)
- ansible-galaxy - Instead of assuming the first defined server is galaxy, filter based on the servers that support the v1 API, and return the first of those (https://github.com/ansible/ansible/issues/65440)
- ansible-galaxy - Use ``sys.exit`` instead of ``exit`` when reporting an error for the removed login command.
- ansible-galaxy - correct ``collections-path`` command line argument (https://github.com/ansible/ansible/issues/73127)
- ansible-galaxy - fixed galaxy role init command (https://github.com/ansible/ansible/issues/71977).
- ansible-galaxy collection download - fix downloading tar.gz files and collections in git repositories (https://github.com/ansible/ansible/issues/70429)
- ansible-galaxy collection install - fix fallback mechanism if the AH server did not have the collection requested - https://github.com/ansible/ansible/issues/70940
- ansible-galaxy download - fix bug when downloading a collection in a SCM subdirectory
- ansible-test - Add ``pytest < 6.0.0`` constraint for managed installations on Python 3.x to avoid issues with relative imports.
- ansible-test - Always connect additional Docker containers to the network used by the current container (if any).
- ansible-test - Always map ``/var/run/docker.sock`` into test containers created by the ``--docker`` option if the docker host is not ``localhost``.
- ansible-test - Attempt to detect the Docker hostname instead of assuming ``localhost``.
- ansible-test - Avoid using ``/tmp`` to resolve occasional failures starting tests with the ``--docker`` option.
- ansible-test - Change classification using ``--changed`` now consistently handles common configuration files for supported CI providers.
- ansible-test - Change detection now properly resolves relative imports instead of treating them as absolute imports.
- ansible-test - Correctly detect changes in a GitHub pull request when running on Azure Pipelines.
- ansible-test - Correctly detect running in a Docker container on Azure Pipelines.
- ansible-test - Do not try to validate PowerShell modules ``setup.ps1``, ``slurp.ps1``, and ``async_status.ps1``
- ansible-test - Prefer container IP at ``.NetworkSettings.Networks.{NetworkName}.IPAddress`` over ``.NetworkSettings.IPAddress``.
- ansible-test - Running tests using an installed version of ``ansible-test`` against one Python version from another no longer fails due to a missing ``egg-info`` directory. This could occur when testing plugins which import ``pkg_resources``.
- ansible-test - Running tests using an installed version of ``ansible-test`` no longer generates an error attempting to create an ``egg-info`` directory when an existing one is not found in the expected location. This could occur if the existing ``egg-info`` directory included a Python version specifier in the name.
- ansible-test - Skip installing requirements if they are already installed.
- ansible-test - Symbolic links are no longer used to inject ``python`` into the environment, since they do not work reliably in all cases. Instead, the existing Python based exec wrapper is always used.
- ansible-test - Temporarily limit ``cryptography`` to versions before 3.4 to enable tests to function.
- ansible-test - The ``--raw`` option for ``ansible-test shell --remote`` now uses ``sh`` for the shell instead of ``bash``, which may not be present.
- ansible-test - The ``--remote`` option has been updated for Python 2.7 to work around breaking changes in the newly released ``get-pip.py`` bootstrapper.
- ansible-test - The ``--remote`` option has been updated to use a versioned ``get-pip.py`` bootstrapper to avoid issues with future releases.
- ansible-test - The ``ansible-doc`` sanity test now works for ``netconf`` plugins.
- ansible-test - The ``changelog`` sanity test has been updated to ensure ``rstcheck`` does not load the ``sphinx`` module.
- ansible-test - The ``cs`` and ``openshift`` test plugins now search for containers on the current network instead of assuming the ``bridge`` network.
- ansible-test - The ``resource_prefix`` variable provided to tests running on Azure Pipelines is now converted to lowercase to match other CI providers.
- ansible-test - Unified SSH key management for all instances created with the ``--remote`` or ``--docker`` options.
- ansible-test - Using the ``--remote`` option on Azure Pipelines now works from a job running in a container.
- ansible-test - ``cryptography`` is now limited to versions prior to 3.2 only when an incompatible OpenSSL version (earlier than 1.1.0) is detected
- ansible-test - add constraint for ``cffi`` to prevent failure on systems with older versions of ``gcc`` (https://foss.heptapod.net/pypy/cffi/-/issues/480)
- ansible-test - convert target paths to unicode on Python 2 to avoid ``UnicodeDecodeError`` (https://github.com/ansible/ansible/issues/68398, https://github.com/ansible/ansible/pull/72623).
- ansible-test - improve classification of changes to ``.gitignore``, ``COPYING``, ``LICENSE``, ``Makefile``, and all files ending with one of ``.in`, ``.md`, ``.rst``, ``.toml``, ``.txt`` in the collection root directory (https://github.com/ansible/ansible/pull/72353).
- ansible-test - integration and unit test change detection now works for filter, lookup and test plugins
- ansible-test now always uses the ``--python`` option for ``virtualenv`` to select the correct interpreter when creating environments with the ``--venv`` option
- ansible-test sanity changelog test - bump dependency on antsibull-changelog to 0.9.0 so that `fragments that add new plugins or objects <https://github.com/ansible-community/antsibull-changelog/blob/main/docs/changelogs.rst#adding-new-roles-playbooks-test-and-filter-plugins>`_ will not fail validation (https://github.com/ansible/ansible/pull/73428).
- ansible-test units - fixed collection location code to work under pytest >= 6.0.0
- ansible-test validate-modules - ``version_added`` on module level was not validated for modules in collections (https://github.com/ansible/ansible/pull/70869).
- ansible-test validate-modules - return correct error codes ``option-invalid-version-added`` resp. ``return-invalid-version-added`` instead of the wrong error ``deprecation-either-date-or-version`` when an invalid value of ``version_added`` is specified for an option or a return value (https://github.com/ansible/ansible/pull/70869).
- ansible-test validate-modules - when a module uses ``add_file_common_args=True`` and does not use a keyword argument for ``argument_spec`` in ``AnsibleModule()``, the common file arguments were not considered added during validation (https://github.com/ansible/ansible/pull/72334).
- ansible_pkg_mgr fact - now correctly returns ``atomic_container`` when run on "RHEL for Edge" images and Fedora/RHEL/CentOS Atomic Host (https://github.com/ansible/ansible/issues/73084).
- api - time.clock is removed in Python 3.8, add backward compatible code (https://github.com/ansible/ansible/issues/70649).
- apt - add ``fail_on_autoremove`` param to apt module to avoid unintended package removals (https://github.com/ansible/ansible/issues/63231)
- apt - include exception message from apt python library in error output
- apt_key - Specifying ``file`` as mutually exclusive with ``data``, ``keyserver``, ``url`` (https://github.com/ansible/ansible/pull/70492).
- assemble - fix decrypt argument in the module (https://github.com/ansible/ansible/issues/65450).
- async - Fix Python 3 interpreter parsing from module by comparing with bytes (https://github.com/ansible/ansible/issues/70690)
- async_wrapper - Fix race condition when ``~/.ansible_async`` folder tries to be created by multiple async tasks at the same time - https://github.com/ansible/ansible/issues/59306
- avoid possible errors accessing os.environ by not assuming existance of variables.
- basic - handle exceptions for default selectors in Python 2.7 (https://github.com/ansible/ansible/issues/71704).
- basic - use PollSelector implementation when DefaultSelector fails (https://github.com/ansible/ansible/issues/70238).
- bcrypt hashing - Ensure we repair the salt, to avoid warnings (https://github.com/ansible/ansible/issues/36129)
- blockinfile - properly insert a block at the end of a file that does not have a trailing newline character (https://github.com/ansible/ansible/issues/72055)
- blockinfile now returns name of backup file when this option is used.
- clarified changed status to reflect existing rule that had never been written down.
- collection metadata - ensure collection loader uses libyaml/CSafeLoader to parse collection metadata if available
- cron - cron file should not be empty after adding var (https://github.com/ansible/ansible/pull/71207)
- cron - encode and decode crontab files in UTF-8 explicitly to allow non-ascii chars in cron filepath and job (https://github.com/ansible/ansible/issues/69492)
- default callback - Ensure that the ``host_pinned`` strategy is not treated as lockstep (https://github.com/ansible/ansible/issues/73364)
- delegate_to - Ensure that calculating ``delegate_to`` vars with a loop uses the correct context to correctly evaluate the loop (https://github.com/ansible/ansible/issues/37132)
- display correct error information when an error exists in the last line of the file (https://github.com/ansible/ansible/issues/16456)
- distribution - add support for Pardus Linux distribution (https://github.com/ansible/ansible/issues/71636).
- distribution facts - Allow ``distribution_major_version`` and ``distribution_version`` to work for RC and PRERELEASE versions of FreeBSD (and derived distributions) (https://github.com/ansible/ansible/issues/72331).
- dnf - fix filtering to avoid dependncy conflicts (https://github.com/ansible/ansible/issues/72316)
- dnf - it is now possible to specify both ``security: true`` and ``bugfix: true`` to install updates of both types. Previously, only security would get installed if both were true. (https://github.com/ansible/ansible/issues/70854)
- ensure 'local' connection always has the correct default user for actions to consume.
- ensure delegated vars can resolve hostvars object and access vars from hostvars[inventory_hostname].
- ensure we don't clobber role vars data when getting an empty file
- expect - Operate pexpect with bytes to avoid potential encoding issues (https://github.com/ansible/ansible/issues/29351)
- facts - account for Slackware OS with ``+`` in the name (https://github.com/ansible/ansible/issues/38760)
- facts - fix distribution fact for SLES4SAP (https://github.com/ansible/ansible/pull/71559).
- facts - fix incorrect UTC timestamp in ``iso8601_micro`` and ``iso8601``
- facts - properly report virtualization facts for Linux guests running on bhyve (https://github.com/ansible/ansible/issues/73167)
- file - prevent link src from being rewritten when src is not specified explicitly (https://github.com/ansible/ansible/issues/65448)
- file - the module should warn in check_mode when path an owner/group don't exist (https://github.com/ansible/ansible/issues/67307).
- find module - Don't treat empty excludes as a match (https://github.com/ansible/ansible/issues/70640)
- find module - Stop traversing directories past the requested depth. (https://github.com/ansible/ansible/issues/73627)
- fix issue with inventory_hostname and delegated host vars mixing on connection settings.
- fortimanager httpapi plugin - fix redirect to point to the ``fortinet.fortimanager`` collection (https://github.com/ansible/ansible/pull/71073).
- galaxy - handle plus sign in user token appearing in role url (https://github.com/ansible/ansible/issues/45475).
- get_sysctl now handles multiline values and does not die silently anymore.
- get_url - skip checksum during ``--check`` (https://github.com/ansible/ansible/issues/61369).
- git - Only pass ``--raw`` flag to git verify commands (verify-tag, verify-commit) when ``gpg_whitelist`` is in use. Otherwise don't pass it so that non-whitelist GPG validation still works on older Git versions. (https://github.com/ansible/ansible/issues/64469)
- gluster modules - fix redirect to point to the ``gluster.gluster`` collection (https://github.com/ansible/ansible/pull/71240).
- hostname - Fixed an issue where the hostname on the alinux could not be set.
- hostname - add Almalinux support (https://github.com/ansible/ansible/pull/73619)
- hostname - add macOS support (https://github.com/ansible/ansible/pull/54439)
- if the ``type`` for a module parameter in the argument spec is callable, do not pass ``kwargs`` to avoid errors (https://github.com/ansible/ansible/issues/70017)
- import_playbook - change warning about extra parameters to deprecation (https://github.com/ansible/ansible/issues/72745)
- improve deprecation message when using bare variable (https://github.com/ansible/ansible/pull/70687)
- inventory - pass the vars dictionary to combine_vars instead of an individual key's value (https://github.com/ansible/ansible/issues/72975).
- inventory plugins - Let plugins define the sanitization method for the constructed ``groups`` feature.
- inventory_hostnames - Use ``InventoryManager`` instead of trying to replicate its behavior (https://github.com/ansible/ansible/issues/17268)
- is_string/vault - Ensure the is_string helper properly identifies AnsibleVaultEncryptedUnicode as a string (https://github.com/ansible/ansible/pull/71609)
- lineinfile - fix not subscriptable error in exception handling around file creation
- linux network facts - get the correct value for broadcast address (https://github.com/ansible/ansible/issues/64384)
- package - use list of built in package managers from facts rather than creating a new list
- paramiko connection plugin - Ensure we only reset the connection when one has been previously established (https://github.com/ansible/ansible/issues/65812)
- password hashing - Ensure we validate salts against allowed characters and length when using ``crypt`` (https://github.com/ansible/ansible/issues/71107)
- password lookup - Try to automatically generate salts using known salt sizes (https://github.com/ansible/ansible/issues/53750)
- pause - Fix indefinite hang when using a pause task on a background process (https://github.com/ansible/ansible/issues/32142)
- pause - catch additional error on setting up curses (https://github.com/ansible/ansible/pull/73588).
- pause - do not warn when running in the background if a timeout is provided (https://github.com/ansible/ansible/issues/73042)
- pause - handle exception when there is no stdout (https://github.com/ansible/ansible/pull/47851)
- powershell - fix escaping of strings that broken modules like fetch when dealing with special chars - https://github.com/ansible/ansible/issues/62781
- powershell - fix the CLIXML parser when it contains nested CLIXML objects - https://github.com/ansible/ansible/issues/69550
- powershell - remove getting the PowerShell version from the env var ``POWERSHELL_VERSION``. This feature never worked properly and can cause conflicts with other libraries that use this var
- psrp - Fix hang when copying an empty file to the remote target
- psrp - Use native PSRP mechanism when copying files to support custom endpoints
- quote filter - normalize how ``None`` is handled, to match Python3 behavior (https://github.com/ansible/ansible/issues/32174)
- reboot - Add support for the runit init system, used on Void Linux, that does not support the normal Linux syntax.
- remove contradictory recomendation from template docs. https://github.com/ansible/ansible/issues/63484
- remove redundant remote_user setting in play_context for local as plugin already does it, also removes fork/thread issue from use of pwd library.
- reset logging level to INFO due to CVE-2019-14846.
- runas - create a new token when running as ``SYSTEM`` to ensure it has the full privileges assigned to that account
- service - Fix for the BSD rcconf code using a Python 2 specific string replace function
- set_mode_if_different - handle symlink if it is inside a directory with sticky bit set (https://github.com/ansible/ansible/pull/45198)
- several fixes to make apt_key better at identifying needs for change and also to avoid changes in check_mode.
- shell - fix quoting of mkdir command in creation of remote_tmp in order to allow spaces and other special characters (https://github.com/ansible/ansible/issues/69577).
- splunk httpapi plugin - switch from splunk.enterprise_security to splunk.es in runtime.yml to reflect upstream change of Collection Name
- ssh connection plugin - use ``get_option()`` rather than ``_play_context`` to ensure ``ANSBILE_SSH_ARGS`` are applied properly (https://github.com/ansible/ansible/issues/70437)
- stat - handle colons in filename while parsing the mimetype output (https://github.com/ansible/ansible/issues/70256).
- strftime filter - Input epoch is allowed to be a float (https://github.com/ansible/ansible/issues/71257)
- systemd - account for templated unit files using ``@`` when searching for the unit file (https://github.com/ansible/ansible/pull/72347#issuecomment-730626228)
- systemd - fixed chroot usage on new versions of systemd, that broke because of upstream changes in systemctl output
- systemd - follow up fix to https://github.com/ansible/ansible/issues/72338 to use ``list-unit-files`` rather than ``list-units`` in order to show all units files on the system.
- systemd - made the systemd module work correctly when the SYSTEMD_OFFLINE environment variable is set
- systemd - preserve the full unit name when using a templated service and ``systemd`` failed to parse dbus due to a known bug in ``systemd`` (https://github.com/ansible/ansible/pull/72985)
- systemd - work around bug with ``systemd`` 245 and 5.8 kernel that does not correctly report service state (https://github.com/ansible/ansible/issues/71528)
- task parsing - strip spaces from action name when using ``action: foo bar=baz`` form. (https://github.com/ansible/ansible/issues/62136)
- templating - fix error message for ``x in y`` when y is undefined (https://github.com/ansible/ansible/issues/70984)
- the unvault lookup plugin returned a byte string. Now returns a real string.
- to_text(stdout) before json.loads in psrp.Connection.put_file in case stdout is bytes
- unarchive - ``zip`` unarchive no longer errors on RHEL/CentOS 6 and old Fedora when attempting to use a numeric gid (https://github.com/ansible/ansible/issues/71903).
- unarchive - check ``fut_gid`` against ``run_gid`` in addition to supplemental groups (https://github.com/ansible/ansible/issues/49284)
- undeprecate hash_merge setting and add more docs clarifying its use and why not to use it.
- uri - ``status_code`` elements are type ``int``
- url lookup - make sure that options supplied in ansible.cfg are actually used (https://github.com/ansible/ansible/pull/71736).
- url lookup - set default user agent to ``ansible-httpget`` (https://github.com/ansible/ansible/pull/72324)
- urls - Close filedescriptor of certificate chain tempfile to prevent stale filedescriptor leakage (https://github.com/ansible/ansible/pull/71825).
- user - AnsibleModule.run_command returns a tuple of return code, stdout and stderr. The module main function of the user module expects user.create_user to return a tuple of return code, stdout and stderr. Fix the locations where stdout and stderr got reversed.
- user - Local users with an expiry date cannot be created as the ``luseradd`` / ``lusermod`` commands do not support the ``-e`` option. Set the expiry time in this case via ``lchage`` after the user was created / modified. (https://github.com/ansible/ansible/issues/71942)
- user - do the right thing when ``password_lock=True`` and ``password`` are used together (https://github.com/ansible/ansible/issues/72992)
- user - don't create home directory and missing parents when create_home == false (https://github.com/ansible/ansible/pull/70600).
- validate-modules - do not raise an ``AttributeError`` if a value is assigned to a module attribute in a try/except block.
- vault - Support reading raw binary data from stdin under python3
- virtual facts - kubevirt is now identified as "KubeVirt" and with a "guest" role instead of "kvm" and "host" role (https://github.com/ansible/ansible/issues/72001).
- wait_for - catch and ignore errors when getting active connections with psutil (https://github.com/ansible/ansible/issues/72322)
- win setup - Fix redirection path for the windows setup module
- windows async - use full path when calling PowerShell to reduce reliance on environment vars being correct - https://github.com/ansible/ansible/issues/70655
- winrm - preserve winrm forensic data on put_file failures
- yamllint - do not raise an ``AttributeError`` if a value is assigned to a module attribute at the top of the module.
Known Issues
------------
- ansible-test - The ``pylint`` sanity test no longer correctly detects "bad" variable names for non-constants. See https://github.com/PyCQA/pylint/issues/3701 for additional details.
New Modules
-----------
- validate_argument_spec - Validate role argument specs.
- BSD network facts - Do not assume column indexes, look for ``netmask`` and ``broadcast`` for determining the correct columns when parsing ``inet`` line (https://github.com/ansible/ansible/issues/79117)
- ansible-config limit shorthand format to assigned values
- ansible-test - Update the ``pylint`` sanity test to use version 2.15.4.
- Do not crash when templating an expression with a test or filter that is not a valid Ansible filter name (https://github.com/ansible/ansible/issues/78912, https://github.com/ansible/ansible/pull/78913).
- handlers - fix an issue where the ``flush_handlers`` meta task could not be used with FQCN: ``ansible.builtin.meta`` (https://github.com/ansible/ansible/issues/79023)
- keyword inheritance - Ensure that we do not squash keywords in validate (https://github.com/ansible/ansible/issues/79021)
- omit on keywords was resetting to default value, ignoring inheritance.
- service_facts - Use python re to parse service output instead of grep (https://github.com/ansible/ansible/issues/78541)
- ansible-test validate-modules - Added support for validating module documentation stored in a sidecar file alongside the module (``{module}.yml`` or ``{module}.yaml``). Previously these files were ignored and documentation had to be placed in ``{module}.py``.
- apt_repository will use the trust repo directories in order of preference (more appropriate to less) as they exist on the target.
Breaking Changes / Porting Guide
--------------------------------
- ansible-test validate-modules - Removed the ``missing-python-doc`` error code in validate modules, ``missing-documentation`` is used instead for missing PowerShell module documentation.
Bugfixes
--------
- Fix reusing a connection in a task loop that uses a redirected or aliased name - https://github.com/ansible/ansible/issues/78425
- Fix setting become activation in a task loop - https://github.com/ansible/ansible/issues/78425
- apt module should not traceback on invalid type given as package. issue 78663.
- known_hosts - do not return changed status when a non-existing key is removed (https://github.com/ansible/ansible/issues/78598)
- plugin loader, fix detection for existing configuration before initializing for a plugin
- Move handler processing into new ``PlayIterator`` phase to use the configured strategy (https://github.com/ansible/ansible/issues/65067)
- ansible - At startup the filesystem encoding and locale are checked to verify they are UTF-8. If not, the process exits with an error reporting the errant encoding.
- ansible - Increase minimum Python requirement to Python 3.9 for CLI utilities and controller code
- ansible-test - At startup the filesystem encoding is checked to verify it is UTF-8. If not, the process exits with an error reporting the errant encoding.
- ansible-test - At startup the locale is configured as ``en_US.UTF-8``, with a fallback to ``C.UTF-8``. If neither encoding is available the process exits with an error. If the fallback is used, a warning is displayed. In previous versions the ``en_US.UTF-8`` locale was always requested. However, no startup checking was performed to verify the locale was successfully configured.
Minor Changes
-------------
- Add a new "INVENTORY_UNPARSED_WARNING" flag add to hide the "No inventory was parsed, only implicit localhost is available" warning
- Add an 'action_plugin' field for modules in runtime.yml plugin_routing.
This fixes module_defaults by supporting modules-as-redirected-actions
without redirecting module_defaults entries to the common action.
.. code: yaml
plugin_routing:
action:
facts:
redirect: ns.coll.eos
command:
redirect: ns.coll.eos
modules:
facts:
redirect: ns.coll.eos_facts
command:
redirect: ns.coll.eos_command
With the runtime.yml above for ns.coll, a task such as
will end up with defaults for eos_facts and eos_command
since both modules redirect to the same action.
To select an action plugin for a module without merging
module_defaults, define an action_plugin field for the resolved
module in the runtime.yml.
.. code: yaml
plugin_routing:
modules:
facts:
redirect: ns.coll.eos_facts
action_plugin: ns.coll.eos
command:
redirect: ns.coll.eos_command
action_plugin: ns.coll.eos
The action_plugin field can be a redirected action plugin, as
it is resolved normally.
Using the modified runtime.yml, the example task will only use
the ns.coll.eos_facts defaults.
- Add support for parsing ``-a`` module options as JSON and not just key=value arguments - https://github.com/ansible/ansible/issues/78112
- Added Kylin Linux Advanced Server OS in RedHat OS Family.
- Allow ``when`` conditionals to be used on ``flush_handlers`` (https://github.com/ansible/ansible/issues/77616)
- Allow meta tasks to be used as handlers.
- Display - The display class will now proxy calls to Display.display via the queue from forks/workers to be handled by the parent process for actual display. This reduces some reliance on the fork start method and improves reliability of displaying messages.
- Jinja version test - Add pep440 version_type for version test. (https://github.com/ansible/ansible/issues/78288)
- Loops - Add new ``loop_control.extended_allitems`` to allow users to disable tracking all loop items for each loop (https://github.com/ansible/ansible/issues/75216)
- NetBSD - Add uptime_seconds fact
- Provide a `utc` option for strftime to show time in UTC rather than local time
- Raise a proper error when ``include_role`` or ``import_role`` is used as a handler.
- Remove the ``AnsibleContext.resolve`` method as its override is not necessary. Furthermore the ability to override the ``resolve`` method was deprecated in Jinja 3.0.0 and removed in Jinja 3.1.0.
- Utilize @classmethod and @property together to form classproperty (Python 3.9) to access field attributes of a class
-``LoopControl`` is now templated through standard ``post_validate`` method (https://github.com/ansible/ansible/pull/75715)
-``ansible-galaxy collection install`` - add an ``--offline`` option to prevent querying distribution servers (https://github.com/ansible/ansible/issues/77443).
- ansible - Add support for Python 3.11 to Python interpreter discovery.
- ansible - At startup the stdin/stdout/stderr file handles are checked to verify they are using blocking IO. If not, the process exits with an error reporting which file handle(s) are using non-blocking IO.
- ansible-config adds JSON and YAML output formats for list and dump actions.
- ansible-connection now supports verbosity directly on cli
- ansible-console added 'collections' command to match playbook keyword.
- ansible-doc - remove some of the manual formatting, and use YAML more uniformly. This in particular means that ``true`` and ``false`` are used for boolean values, instead of ``True`` and ``False`` (https://github.com/ansible/ansible/pull/78668).
- ansible-galaxy - Support resolvelib versions 0.6.x, 0.7.x, and 0.8.x. The full range of supported versions is now >= 0.5.3, < 0.9.0.
- ansible-galaxy now supports a user defined timeout, instead of existing hardcoded 60s (now the default).
- ansible-test - Add FreeBSD 13.1 remote support.
- ansible-test - Add RHEL 9.0 remote support.
- ansible-test - Add support for Python 3.11.
- ansible-test - Add support for RHEL 8.6 remotes.
- ansible-test - Add support for Ubuntu VMs using the ``--remote`` option.
- ansible-test - Add support for exporting inventory with ``ansible-test shell --export {path}``.
- ansible-test - Add support for multi-arch remotes.
- ansible-test - Add support for provisioning Alpine 3.16 remote instances.
- ansible-test - Add support for provisioning Fedora 36 remote instances.
- ansible-test - Add support for provisioning Ubuntu 20.04 remote instances.
- ansible-test - Add support for provisioning remotes which require ``doas`` for become.
- ansible-test - Add support for running non-interactive commands with ``ansible-test shell``.
- ansible-test - Alpine remotes now use ``sudo`` for tests, using ``doas`` only for bootstrapping.
- ansible-test - An improved error message is shown when the download of a pip bootstrap script fails. The download now uses ``urllib2`` instead of ``urllib`` on Python 2.
- ansible-test - Avoid using the ``mock_use_standalone_module`` setting for unit tests running on Python 3.8 or later.
- ansible-test - Become support for remote instance provisioning is no longer tied to a fixed list of platforms.
- ansible-test - Blocking mode is now enforced for stdin, stdout and stderr. If any of these are non-blocking then ansible-test will exit during startup with an error.
- ansible-test - Distribution specific test containers are now multi-arch, supporting both x86_64 and aarch64.
- ansible-test - Distribution specific test containers no longer contain a ``/etc/ansible/hosts`` file.
- ansible-test - Enable loading of ``coverage`` data files created by older supported ansible-test releases.
- ansible-test - Fedora 36 has been added as a test container.
- ansible-test - FreeBSD remotes now use ``sudo`` for tests, using ``su`` only for bootstrapping.
- ansible-test - Improve consistency of output messages by using stdout or stderr for most output, but not both.
- ansible-test - Remote Alpine instances now have the ``acl`` package installed.
- ansible-test - Remote Fedora instances now have the ``acl`` package installed.
- ansible-test - Remote FreeBSD instances now have ACLs enabled on the root filesystem.
- ansible-test - Remote Ubuntu instances now have the ``acl`` package installed.
- ansible-test - Remove Ubuntu 18.04 test container.
- ansible-test - Remove support for Python 2.7 on provisioned FreeBSD instances.
- ansible-test - Remove support for Python 3.8 on the controller.
- ansible-test - Remove the ``opensuse15py2`` container.
- ansible-test - Support multiple pinned versions of the ``coverage`` module. The version used now depends on the Python version in use.
- ansible-test - Test containers have been updated to remove the ``VOLUME`` instruction.
- ansible-test - The Alpine 3 test container has been updated to Alpine 3.16.0.
- ansible-test - The ``http-test-container`` container is now multi-arch, supporting both x86_64 and aarch64.
- ansible-test - The ``pypi-test-container`` container is now multi-arch, supporting both x86_64 and aarch64.
- ansible-test - The ``shell`` command can be used outside a collection if no controller delegation is required.
- ansible-test - The openSUSE test container has been updated to openSUSE Leap 15.4.
- ansible-test - Ubuntu 22.04 has been added as a test container.
- ansible-test - Update pinned sanity test requirements for all tests.
- ansible-test - Update the ``base`` container to 3.4.0.
- ansible-test - Update the ``default`` containers to 6.6.0.
- apt_repository remove dependency on apt-key and use gpg + /usr/share/keyrings directly instead
- blockinfile - The presence of the multiline flag (?m) in the regular expression for insertafter opr insertbefore controls whether the match is done line by line or with multiple lines (https://github.com/ansible/ansible/pull/75090).
- calls to listify_lookup_plugin_terms in core do not pass in loader/dataloader anymore.
- collections - ``ansible-galaxy collection build`` can now utilize ``MANIFEST.in`` style directives from ``galaxy.yml`` instead of ``build_ignore`` effectively inverting the logic from include by default, to exclude by default. (https://github.com/ansible/ansible/pull/78422)
- config manager, move templating into main query function in config instead of constants
- config manager, remove updates to configdata as it is mostly unused
- configuration entry INTERPRETER_PYTHON_DISTRO_MAP is now 'private' and won't show up in normal configuration queries and docs, since it is not 'settable' this avoids user confusion.
- distribution - add distribution_minor_version for Debian Distro (https://github.com/ansible/ansible/issues/74481).
- documentation construction now gives more information on error.
- facts - add OSMC to Debian os_family mapping
- get_url - permit to pass to parameter ``checksum`` an URL pointing to a file containing only a checksum (https://github.com/ansible/ansible/issues/54390).
- new tests url, uri and urn will verify string as such, but they don't check existance of the resource
- plugin loader - add ansible_name and ansible_aliases attributes to plugin objects/classes.
- systemd is now systemd_service to better reflect the scope of the module, systemd is kept as an alias for backwards compatibility.
- templating - removed internal template cache
- uri - cleanup write_file method, remove overkill safety checks and report any exception, change shutilcopyfile to use module.atomic_move
- urls - Add support to specify SSL/TLS ciphers to use during a request (https://github.com/ansible/ansible/issues/78633)
- validate-modules - Allow ``type: raw`` on a module return type definition for values that have a dynamic type
- version output now includes the path to the python executable that Ansible is running under
- yum_repository - do not give the ``async`` parameter a default value anymore, since this option is deprecated in RHEL 8. This means that ``async = 1`` won't be added to repository files if omitted, but it can still be set explicitly if needed.
Breaking Changes / Porting Guide
--------------------------------
- Allow for lazy evaluation of Jinja2 expressions (https://github.com/ansible/ansible/issues/56017)
- The default ansible-galaxy role skeletons no longer contain .travis.yml files. You can configure ansible-galaxy to use a custom role skeleton that contains a .travis.yml file to continue using Galaxy's integration with Travis CI.
- ansible - At startup the filesystem encoding and locale are checked to verify they are UTF-8. If not, the process exits with an error reporting the errant encoding.
- ansible - Increase minimum Python requirement to Python 3.9 for CLI utilities and controller code
- ansible-test - At startup the filesystem encoding is checked to verify it is UTF-8. If not, the process exits with an error reporting the errant encoding.
- ansible-test - At startup the locale is configured as ``en_US.UTF-8``, with a fallback to ``C.UTF-8``. If neither encoding is available the process exits with an error. If the fallback is used, a warning is displayed. In previous versions the ``en_US.UTF-8`` locale was always requested. However, no startup checking was performed to verify the locale was successfully configured.
- strategy plugins - Make ``ignore_unreachable`` to increase ``ignored`` and ``ok`` and counter, not ``skipped`` and ``unreachable``. (https://github.com/ansible/ansible/issues/77690)
Deprecated Features
-------------------
- Deprecate ability of lookup plugins to return arbitrary data. Lookup plugins must return lists, failing to do so will be an error in 2.18. (https://github.com/ansible/ansible/issues/77788)
- Encryption - Deprecate use of the Python crypt module due to it's impending removal from Python 3.13
- PlayContext.verbosity is deprecated and will be removed in 2.18. Use ansible.utils.display.Display().verbosity as the single source of truth.
-``DEFAULT_FACT_PATH``, ``DEFAULT_GATHER_SUBSET`` and ``DEFAULT_GATHER_TIMEOUT`` are deprecated and will be removed in 2.18. Use ``module_defaults`` keyword instead.
-``PlayIterator`` - deprecate ``cache_block_tasks`` and ``get_original_task`` which are noop and unused.
-``Templar`` - deprecate ``shared_loader_obj`` option which is unused. ``ansible.plugins.loader`` is used directly instead.
- listify_lookup_plugin_terms, deprecate 'loader/dataloader' parameter as it not used.
- vars plugins - determining whether or not to run ansible.legacy vars plugins with the class attribute REQUIRES_WHITELIST is deprecated, set REQUIRES_ENABLED instead.
Removed Features (previously deprecated)
----------------------------------------
- PlayIterator - remove deprecated ``PlayIterator.ITERATING_*`` and ``PlayIterator.FAILED_*``
- Ensure syntax check errors include playbook filenames
- Ensure the correct ``environment_class`` is set on ``AnsibleJ2Template``
- Error for collection redirects that do not use fully qualified collection names, as the redirect would be determined by the ``collections`` keyword.
- Fix PluginLoader to mimic Python import machinery by adding module to sys.modules before exec
- Fix ``-vv`` output for meta tasks to not have an empty message when skipped, print the skip reason instead. (https://github.com/ansible/ansible/issues/77315)
- Fix an issue where ``ansible_play_hosts`` and ``ansible_play_batch`` were not properly updated when a failure occured in an explicit block inside the rescue section (https://github.com/ansible/ansible/issues/78612)
- Fix dnf module documentation to indicate that comparison operators for package version require spaces around them (https://github.com/ansible/ansible/issues/78295)
- Fix for linear strategy when tasks were executed in incorrect order or even removed from execution. (https://github.com/ansible/ansible/issues/64611, https://github.com/ansible/ansible/issues/64999, https://github.com/ansible/ansible/issues/72725, https://github.com/ansible/ansible/issues/72781)
- Fix for network_cli not getting all relevant connection options
- Fix handlers execution with ``serial`` in the ``linear`` strategy (https://github.com/ansible/ansible/issues/54991)
- Fix potential, but unlikely, cases of variable use before definition.
- Fix traceback when installing a collection from a git repository and git is not installed (https://github.com/ansible/ansible/issues/77479).
- GALAXY_IGNORE_CERTS reworked to allow each server entry to override
- More gracefully handle separator errors in jinja2 template overrides (https://github.com/ansible/ansible/pull/77495).
- Move undefined check from concat to finalize (https://github.com/ansible/ansible/issues/78156)
- Prevent losing unsafe on results returned from lookups (https://github.com/ansible/ansible/issues/77535)
- Propagate ``ansible_failed_task`` and ``ansible_failed_result`` to an outer rescue (https://github.com/ansible/ansible/issues/43191)
- Properly execute rescue section when an include task fails in all loop iterations (https://github.com/ansible/ansible/issues/23161)
- Properly send a skipped message when a list in a ``loop`` is empty and comes from a template (https://github.com/ansible/ansible/issues/77934)
- Support colons in jinja2 template override values (https://github.com/ansible/ansible/pull/77495).
-``ansible-galaxy`` - remove extra server api call during dependency resolution for requirements and dependencies that are already satisfied (https://github.com/ansible/ansible/issues/77443).
-`ansible-config init -f vars` will now use shorthand format
- action plugins now pass cannonical info to modules instead of 'temporary' info from play_context
- ansible - Exclude Python 2.6 from Python interpreter discovery.
- ansible-config dump - Only display plugin type headers when plugin options are changed if --only-changed is specified.
- ansible-configi init should now skip internal reserved config entries
- ansible-connection - decrypt vaulted parameters before sending over the socket, as vault secrets are not available on the other side.
- ansible-console - Renamed the first argument of ``ConsoleCLI.default`` from ``arg`` to ``line`` to match the first argument of the same method on the base class ``Cmd``.
- ansible-console commands now all have a help entry.
- ansible-console fixed to load modules via fqcn, short names and handle redirects.
- ansible-console now shows installed collection modules.
- ansible-doc - fix listing plugins.
- ansible-doc will not add 'website for' in ":ref:" substitutions as it made them confusing.
- ansible-doc will not again warn and skip when missing docs, always show the doc file (for edit on github) and match legacy plugins.
- ansible-doc will not traceback when legacy plugins don't have docs nor adjacent file with docs
- ansible-doc will now also display until as an 'implicit' templating keyword.
- ansible-doc will now not display version_added_collection under same conditions it does not display version_added.
- ansible-galaxy - Fix detection of ``--role-file`` in arguments for implicit role invocation (https://github.com/ansible/ansible/issues/78204)
- ansible-galaxy - Fix exit codes for role search and delete (https://github.com/ansible/ansible/issues/78516)
- ansible-galaxy - Fix loading boolean server options so False doesn't become a truthy string (https://github.com/ansible/ansible/issues/77416).
- ansible-galaxy - Fix reinitializing the whole collection directory with ``ansible-galaxy collection init ns.coll --force``. Now directories and files that are not included in the collection skeleton will be removed.
- ansible-galaxy - Fix unhandled traceback if a role's dependencies in meta/main.yml or meta/requirements.yml are not lists.
- ansible-galaxy - do not require mandatory keys in the ``galaxy.yml`` of source collections when listing them (https://github.com/ansible/ansible/issues/70180).
- ansible-galaxy - fix installing collections that have dependencies in the metadata set to null instead of an empty dictionary (https://github.com/ansible/ansible/issues/77560).
- ansible-galaxy - fix listing collections that contains metadata but the namespace or name are not strings.
- ansible-galaxy - fix missing meta/runtime.yml in default galaxy skeleton used for ansible-galaxy collection init
- ansible-galaxy - fix setting the cache for paginated responses from Galaxy NG/AH (https://github.com/ansible/ansible/issues/77911).
- ansible-galaxy - handle unsupported versions of resolvelib gracefully.
- ansible-galaxy --ignore-certs now has proper precedence over configuration
- ansible-test - Allow disabled, unsupported, unstable and destructive integration test targets to be selected using their respective prefixes.
- ansible-test - Allow unstable tests to run when targeted changes are made and the ``--allow-unstable-changed`` option is specified (resolves https://github.com/ansible/ansible/issues/74213).
- ansible-test - Always remove containers after failing to create/run them. This avoids leaving behind created containers when using podman.
- ansible-test - Correctly detect when running as the ``root`` user (UID 0) on the origin host. The result of the detection was incorrectly being inverted.
- ansible-test - Delegation for commands which generate output for programmatic consumption no longer redirect all output to stdout. The affected commands and options are ``shell``, ``sanity --lint``, ``sanity --list-tests``, ``integration --list-targets``, ``coverage analyze``
- ansible-test - Delegation now properly handles arguments given after ``--`` on the command line.
- ansible-test - Don't fail if network cannot be disconnected (https://github.com/ansible/ansible/pull/77472)
- ansible-test - Fix bootstrapping of Python 3.9 on Ubuntu 20.04 remotes.
- ansible-test - Fix change detection for ansible-test's own integration tests.
- ansible-test - Fix internal validation of remote completion configuration.
- ansible-test - Fix skipping of tests marked ``needs/python`` on the origin host.
- ansible-test - Fix skipping of tests marked ``needs/root`` on the origin host.
- ansible-test - Prevent ``--target-`` prefixed options for the ``shell`` command from being combined with legacy environment options.
- ansible-test - Sanity test output with the ``--lint`` option is no longer mixed in with bootstrapping output.
- ansible-test - Subprocesses are now isolated from the stdin, stdout and stderr of ansible-test. This avoids issues with subprocesses tampering with the file descriptors, such as SSH making them non-blocking. As a result of this change, subprocess output from unit and integration tests on stderr now go to stdout.
- ansible-test - Subprocesses no longer have access to the TTY ansible-test is connected to, if any. This maintains consistent behavior between local testing and CI systems, which typically do not provide a TTY. Tests which require a TTY should use pexpect or another mechanism to create a PTY.
- ansible-test - Temporary executables are now verified as executable after creation. Without this check, path injected scripts may not be found, typically on systems with ``/tmp`` mounted using the "noexec" option. This can manifest as a missing Python interpreter, or use of the wrong Python interpreter, as well as other error conditions.
- ansible-test - Test configuration for collections is now parsed only once, prior to delegation. Fixes issue: https://github.com/ansible/ansible/issues/78334
- ansible-test - Test containers are now run with the ``--tmpfs`` option for ``/tmp``, ``/run`` and ``/run/lock``. This allows use of containers built without the ``VOLUME`` instruction. Additionally, containers with those volumes defined no longer create anonymous volumes for them. This avoids leaving behind volumes on the container host after the container is stopped and deleted.
- ansible-test - The ``shell`` command no longer redirects all output to stdout when running a provided command. Any command output written to stderr will be mixed with the stderr output from ansible-test.
- ansible-test - The ``shell`` command no longer requests a TTY when using delegation unless an interactive shell is being used. An interactive shell is the default behavior when no command is given to pass to the shell.
- ansible-test - ansible-doc sanity test - Correctly determine the fully-qualified collection name for plugins in subdirectories, resolving https://github.com/ansible/ansible/issues/78490.
- ansible-test - validate-modules - Documentation-only modules, used for documenting actions, are now allowed to have docstrings (https://github.com/ansible/ansible/issues/77972).
- ansible-test compile sanity test - do not crash if a column could not be determined for an error (https://github.com/ansible/ansible/pull/77465).
- apt - Fix module failure when a package is not installed and only_upgrade=True. Skip that package and check the remaining requested packages for upgrades. (https://github.com/ansible/ansible/issues/78762)
- apt - don't actually update the cache in check mode with update_cache=true.
- apt - don't mark existing packages as manually installed in check mode (https://github.com/ansible/ansible/issues/66413).
- apt - fix package selection to include /etc/apt/preferences(.d) (https://github.com/ansible/ansible/issues/77969)
- apt module now correctly handles virtual packages.
- arg_spec - Fix incorrect ``no_log`` warning when a parameter alias is used (https://github.com/ansible/ansible/pull/77576)
- callback plugins - do not crash when ``exception`` passed from a module is not a string (https://github.com/ansible/ansible/issues/75726, https://github.com/ansible/ansible/pull/77781).
- cli now emits clearer error on no hosts selected
- config, ensure that pulling values from configmanager are templated if possible.
- display itself should be single source of 'verbosity' level to the engine.
- dnf - Condense a few internal boolean returns.
- dnf - The ``nobest`` option now also works for ``state=latest``.
- dnf - The ``skip_broken`` option is now used in installs (https://github.com/ansible/ansible/issues/73072).
- dnf - fix output parsing on systems with ``LANGUAGE`` set to a language other than English (https://github.com/ansible/ansible/issues/78193)
- facts - fix IP address discovery for specific interface names (https://github.com/ansible/ansible/issues/77792).
- facts - fix processor facts on AIX: correctly detect number of cores and threads, turn ``processor`` into a list (https://github.com/ansible/ansible/pull/78223).
- fetch_file - Ensure we only use the filename when calculating a tempfile, and do not incude the query string (https://github.com/ansible/ansible/issues/29680)
- file - setting attributes of symbolic links or files that are hard linked no longer fails when the link target is unspecified (https://github.com/ansible/ansible/issues/76142).
- file backed cache plugins now handle concurrent access by making atomic updates to the files.
- git module fix docs and proper use of ssh wrapper script and GIT_SSH_COMMAND depending on version.
- if a config setting prevents running ansible it should at least show it's "origin".
- include module - add docs url to include deprecation message (https://github.com/ansible/ansible/issues/76684).
- items2dict - Handle error if an item is not a dictionary or is missing the required keys (https://github.com/ansible/ansible/issues/70337).
- local facts - if a local fact in the facts directory cannot be stated, store an error message as the fact value and emit a warning just as if just as if the facts execution has failed. The stat can fail e.g. on dangling symlinks.
- module_utils - Make distro.id() report newer versions of OpenSuSE (at least >=15) also report as ``opensuse``. They report themselves as ``opensuse-leap``.
- module_utils.service - daemonize - Avoid modifying the list of file descriptors while iterating over it.
- null_representation config entry changed to 'raw' as it must allow 'none/null' and empty string.
- paramiko - Add a new option to allow paramiko >= 2.9 to easily work with all devices now that rsa-sha2 support was added to paramiko, which prevented communication with numerous platforms. (https://github.com/ansible/ansible/issues/76737)
- paramiko - Add back support for ``ssh_args``, ``ssh_common_args``, and ``ssh_extra_args`` for parsing the ``ProxyCommand`` (https://github.com/ansible/ansible/issues/78750)
- password lookup does not ignore k=v arguments anymore.
- pause module will now report proper 'echo' vs always being true.
- pip - fix cases where resolution of pip Python module fails when importlib.util has not already been imported
- plugin loader will now load config data for plugin by name instead of by file to avoid issues with the same file being loaded under different names (fqcn + short name).
- plugin loader, now when skipping a plugin due to an abstract method error we provide that in 'verbose' mode instead of totally obscuring the error. The current implementation assumed only the base classes would trigger this and failed to consider 'in development' plugins.
- prevent lusermod from using group name instead of group id (https://github.com/ansible/ansible/pull/77914)
- prevent type annotation shim failures from causing runtime failures (https://github.com/ansible/ansible/pull/77860)
- psrp connection now handles default to inventory_hostname correctly.
- roles, fixed issue with roles loading paths not contained in the role itself when using the `_from` options.
- setup - Adds a default value to ``lvm_facts`` when lvm or lvm2 is not installed on linux (https://github.com/ansible/ansible/issues/17393)
- shell plugins now give a more user friendly error when fed the wrong type of data.
- template module/lookup - fix ``convert_data`` option that was effectively always set to True for Jinja macros (https://github.com/ansible/ansible/issues/78141)
- unarchive - if unzip is available but zipinfo is not, use unzip -Z instead of zipinfo (https://github.com/ansible/ansible/issues/76959).
- uri - properly use uri parameter use_proxy (https://github.com/ansible/ansible/issues/58632)
- uri module - failed status when Authentication Bearer used with netrc, because Basic authentication was by default. Fix now allows to ignore netrc by changing use_netrc=False (https://github.com/ansible/ansible/issues/74397).
- urls - Guard imports of ``urllib3`` by catching ``Exception`` instead of ``ImportError`` to prevent exceptions in the import process of optional dependencies from preventing use of ``urls.py`` (https://github.com/ansible/ansible/issues/78648)
- user - Fix error "Permission denied" in user module while generating SSH keys (https://github.com/ansible/ansible/issues/78017).
- user - fix creating a local user if the user group already exists (https://github.com/ansible/ansible/pull/75042)
- user module - Replace uses of the deprecated ``spwd`` python module with ctypes (https://github.com/ansible/ansible/pull/78050)
- validate-modules - fix validating version_added for new options.
- variablemanager, more efficient read of vars files
- vault secrets file now executes in the correct context when it is a symlink (not resolved to canonical file).
- wait_for - Read file and perform comparisons using bytes to avoid decode errors (https://github.com/ansible/ansible/issues/78214)
- winrm - Ensure ``kinit`` is run with the same ``PATH`` env var as the Ansible process
- winrm connection now handles default to inventory_hostname correctly.
- yaml inventory plugin - fix the error message for non-string hostnames (https://github.com/ansible/ansible/issues/77519).
- yum - fix traceback when ``releasever`` is specified with ``latest`` (https://github.com/ansible/ansible/issues/78058)
- find module - Now has a ``read_whole_file`` boolean parameter which allows for reading the whole file and doing an ``re.search()`` regex evaluation on it when searching using the ``contains`` option. This allows (for example) for ensuring the very end of the file matches a pattern.
- git - Only pass ``--raw`` flag to git verify commands (verify-tag, verify-commit) when ``gpg_whitelist`` is in use. Otherwise don't pass it so that non-whitelist GPG validation still works on older Git versions. (https://github.com/ansible/ansible/issues/64469)
- The ``meta`` module now supports tags for user-defined tasks. Internal ``meta`` tasks continue to always run. (https://github.com/ansible/ansible/issues/64558)
- The ``flush()`` method of ``CachePluginAdjudicator`` now calls the plugin's ``flush()`` method instead of iterating over the keys that the adjudicator knows about and deleting those from the cache. (https://github.com/ansible/ansible/issues/68770)
- NetBSD virtualization facts (specifically ``ansible_virtualization_type``) now returns a more accurate value by checking the value of the ``machdep.hypervisor`` ``sysctl`` key. This change is breaking because in some cases previously, we would erroneously report ``xen`` even when the target is not running on Xen. This prevents that behavior in most cases. (https://github.com/ansible/ansible/issues/69352)
- Fixed TypeError instancemethod expecting at least 2 arguments for apt_repository(issue https://github.com/ansible/ansible/issues/69308, PR https://github.com/ansible/ansible/pull/69463)
- user - add new parameters ``password_expire_max`` and ``password_expire_min`` for controlling password expiration (https://github.com/ansible/ansible/issues/68775)
- shell - fix quoting of mkdir command in creation of remote_tmp in order to allow spaces and other special characters (https://github.com/ansible/ansible/issues/69577).
- "ansible-doc - improve error message in text formatter when ``description`` is missing for a (sub-)option or a return value or its ``contains`` (https://github.com/ansible/ansible/pull/70046)."
- "Fix ``delegate_facts: true`` when ``ansible_python_interpreter`` is not set. (https://github.com/ansible/ansible/issues/70168)"
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.