* Log `runme.sh` execution in integration tests (#79263)
* Log `runme.sh` execution in integration tests
This patch adds `set -x` where it's missing in the integration tests.
It also enables `pipefail` in `runme.sh` scripts that use pipes.
* Add a change note for PR #79263
(cherry picked from commit 6674c43edd)
* Delete a bad PR #79263 change note
(cherry picked from commit 9cfb3f73e8)
as chmod/mode is not the only thing we attempt and Ubuntu not shipping acl
in newer versions can lead to some confusion
fixes#79146
(cherry picked from commit 0f18ddca9f)
* Don't assume column index for netmask and broadcast. Fixes#79117
* fix typo
(cherry picked from commit f53dbf9)
Co-authored-by: Matt Martz <matt@sivel.net>
* Use python re to parse service output instead of grep. Fixes#78541
* Add clog frag
* Add an extra guard to abort if rc is 4, and /etc/init.d is missing
(cherry picked from commit 4458128)
Co-authored-by: Matt Martz <matt@sivel.net>
* ansible-galaxy install - fix unnecessary api check when installing a role from git repo (#79090)
* delay server api evaluation until a GalaxyRole needs to make an api call for info, list, and install
(cherry picked from commit cb2e434dd2)
* fix type for older python
* Fix isinstance check (#79159)
Use GalaxyAPI for isinstance check instead of RoleDistributionServer, since the latter is defined in __main__ sometimes (when running integration tests or ansible-galaxy from source) and importing from ansible.cli.galaxy won't reference the same object.
(cherry picked from commit 89d682464b)
* apt - fix failure when package is not installed and only_upgrade=True (#78781)
* apt - fix module failure when package is not installed and only_upgrade is True
* changelog
(cherry picked from commit 4b45b4b09d)
* apt - fix module short-circuiting when a package is not installed and only_upgrade is True
(cherry picked from commit 14f46845f9)
* Limit Galaxy API calls during ansible-galaxy collection dependency resolution when possible
Installing a tarfile with a dependency from a Galaxy server (e.g. dependencies: {'ns.coll': '>=1.0.0'}) does not get the available versions of the dependency from the galaxy server if a sufficient version is already installed.
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
(cherry picked from commit 41b62f7db4)
* Fix known_hosts changed status when removing non-existing key (#78748)
* Ensure no change is reported when removing missing key for host with other keys
Co-authored-by: Matt Martz <matt@sivel.net>
* Add changelog for fix known hosts wrong changed status cherry pick
* Remove old changelog for fix known hosts wrong changed status
* Rename changelog according to PR number
Co-authored-by: Matt Martz <matt@sivel.net>
- Allow disabled, unsupported, unstable and destructive integration test targets to be selected using their respective prefixes.
- Allow unstable tests to run when targeted changes are made and the ``--allow-unstable-changed`` option is specified (resolves https://github.com/ansible/ansible/issues/74213).
(cherry picked from commit d3d7785472)
Co-authored-by: Matt Clay <matt@mystile.com>
* wait_for - Read file and perform comparisons using bytes to avoid decode errors. Fixes#78214
* Write non-ascii via script instead of static file
* Use contexlib.closing to support py27 context manager
* Use executable from task, instead of shebang
* Update encoded bytes to utf16
(cherry picked from commit 9d4ced1)
Co-authored-by: Matt Martz <matt@sivel.net>
Sometimes pytest errors out with an `ImportError` during its tests
collection stage when a Python package/directory containing the test
module does not have an `__init__.py` in it. This is being observed
under Python 3.9 and higher.
The patch provides a workaround for this problem but does not address
the root cause which is currently unknown.
Ref:
https://github.com/ansible/ansible/pull/78585#issuecomment-1220885431
(cherry picked from commit beb70daf14)
* Fix --role-file arg detection. Fixes#78204
* Do not traceback, give better error
* Add coverage for compound shortopts to match -r. Fixes#78491
(cherry picked from commit 86298b7)
Co-authored-by: Matt Martz <matt@sivel.net>
This specific integration test gets stuck periodically causing the
Galaxy jobs to be killed on timeout wasting an hour of runtime. The
module that gets stuck waiting on Pulp is an in-test one, called
`setup_collections`. When it works, the task is complete in around 70
seconds but when it doesn't, it just freezes the whole play.
This patch attempts to make it fail faster by putting a reasonable
timeout value of 2 minutes.
(cherry picked from commit f1c56e988d)
Replace get_persistent_connection_options with get_options
Remove special case for network sub_plugin in _set_plugin_options
Try to avoid mock connection pretending to be persistent
Rename variables->options to reflect what they actually are
Gather options for ssh_type_conn on network_cli
Drop reliance on sub_plugin["type"]
(cherry picked from commit bf1ef5a1f3)
* Don't actually update the cache in check mode
Add tests for updating the cache in check mode
* Don't mark packages as manually installed in check mode and add a test for it
* changelog
* Update test/integration/targets/apt/tasks/apt.yml
* fix test
(cherry picked from commit 3add5e9321)
On BusyBox systems such as Alpine, chattr on a tmpfs fails with a status of 0 and output only on stderr.
This change updates the test to not assume output on stdout.
(cherry picked from commit 2e536c0afb)
* Supply some descriptions for settings used in ansible.cfg (#78536)
(cherry picked from commit 4604ea38dd)
* Targeting a small typo correction (#78531)
(cherry picked from commit b6f42547f4)
* clarify collection signature not available from Community Galaxy (#78386)
(cherry picked from commit 7ae1eabaa4)
* update FAQ with not about libyaml (#78483)
(cherry picked from commit 33ee5d4c1b)
* add note to use package_facts module (#78484)
(cherry picked from commit 8dc097989b)
* Fix ansible-docs: shipping ansible in wheel format #78485 (#78524)
(cherry picked from commit 7f69629fa7)
* update link for RH customers in docs banner (#78563)
(cherry picked from commit 2fdaee143a)
Co-authored-by: Andrew Latham <lathama@users.noreply.github.com>
Co-authored-by: Ian Wienand <iwienand@redhat.com>
Co-authored-by: Arnold Kamanzi <kamstan6@gmail.com>
* update the documentation for the _collection init_ command (#78404)
(cherry picked from commit 69c7e3f4a3)
* fix configuration name in porting guide 2.12 (#78472)
(cherry picked from commit 82246df5b7)
* docs: misc fixes in The Ansible Development Cycle (#78513)
(cherry picked from commit 0d3523df8f)
Co-authored-by: Alex <aizquier@redhat.com>
Co-authored-by: Akira Yokochi <akira6592@gmail.com>
* fix broken link (#78434)
(cherry picked from commit eb46ab1399)
* Fix escaped asterisk in code block demonstrating wildcards (#78375)
The code block is rendered verbatim. This currently results in `\*` being shown, which is incorrect (i.e., not parsed as a wildcard pattern).
Removing the superfluous `\` fixes the rendering.
This PR fixes all instances of this issue on that page.
(cherry picked from commit 9bf5da8f75)
* add example to import a playbook from collection (#78443)
(cherry picked from commit eb462edb16)
* docs: Fix control node reference links (#78424)
(cherry picked from commit f5cac6bc5d)
* Update galaxy_server_list (#78398)
(cherry picked from commit d92f6da592)
* Improve grammar (#78391)
(cherry picked from commit ddd8b8bfd9)
Co-authored-by: Tom Janson <priv.tom.janson@gmail.com>
Co-authored-by: Sebastian Gumprich <rndmh3ro@users.noreply.github.com>
Co-authored-by: Ian Wienand <iwienand@redhat.com>
Co-authored-by: Tabah Baridule <dulemartins07@gmail.com>
Co-authored-by: Geoffrey van Wyk <geoffrey@vanwyk.biz>
* Add basic docs on how to create a collection docsite with antsibull-docs. (#78387)
(cherry picked from commit 733a767948)
* Fix typo: missing 'install'. (#78448)
(cherry picked from commit 2218b63aef)
* Rethread pr/70185 through the dependency resolver
Hang optional metadata toggle on the ConcreteArtifactsManager instead of threading it through whole list codepath
Don't error while listing collections if a collection's metadata is missing keys required for building a collection.
Give an informative warning if metadata has been badly formatted.
Co-authored-by: Sam Doran <sdoran@redhat.com>
(cherry picked from commit 05608b20e8)
* Fall back to implicit namespace.name from the path if the metadata is invalid
* Test listing a collection with null namespace/name/version fields in its galaxy.yml
(cherry picked from commit f8ec660e62)
I'm not aware of a way to easily get vault secrets decoded on the
ansible-connection side without sending the vault secrets over the
connection in the same way, so just decode them for transport.
(cherry picked from commit fff14d7c1d)
* Add missing colon to a snippet in complex data doc (#78349)
(cherry picked from commit e4087baa83)
* update the out-of-date ansible-test sanity ignores (#78342)
(cherry picked from commit 0f688f07af)
* Update information on the Intro_patterns.rst (#78192)
(cherry picked from commit 85bc2bcfcd)
* add note about permission change (#78128)
(cherry picked from commit b6ec06ab25)
Co-authored-by: Sviatoslav Sydorenko <wk@sydorenko.org.ua>
Co-authored-by: Muhammad Rafly Andrianza <luphieanza@gmail.com>
Co-authored-by: Tabah Baridule <dulemartins07@gmail.com>
* Relocate update-ignore.py for easier re-use.
* Add script to ease collection testing.
* Skip ignore rewrite if file does not exist.
* Add integration test for the shebang sanity test.
* Fix ansible-test-no-tty integration test.
Previously the test only verified a TTY was not used if a TTY already existed.
This prevented the test from verifying behavior when run in CI.
Now the test creates a PTY before invoking ansible-test.
* Clean up ansible-test-docker integration test.
(cherry picked from commit f70cc2fb7e)
Co-authored-by: Matt Clay <matt@mystile.com>
`group` and `setup_cron` integration tests currently install `libuser` and `faketime`
packages respectively, from the non-default edge/testing repositories. It is no
longer necessary to do so for the former so this patch drops the repo option. As
for the latter, this patch includes a note explaining when the same could be done
for it too.
This patch also adds a note on installing `faketime` on Alpine in CI.
(cherry picked from commit 619a1fa7e2)
* Documentation: Update japanese translation
* Replace MySQL by Ansible where it was mistakenly translated
Co-authored-by: Sandra McCann <samccann@redhat.com>
* Pr 72292- Add context to inventory plugins (#78070)
* Add context to inventory plugins documentation
* Show how to add the inventory plugin to the `enable_plugins` configuration item - not just replace it.
* Show adding tags to `keyed_groups` without specifying the separator.
* Show adding more complex `groups` and `compose` statements, including using the ternary filter.
* As I added an additional group, I've updated the resulting `ansible-inventory` output accordingly.
(cherry picked from commit a77fc2e822)
* Update dpkg_selections.py (#78177)
* Update dpkg_selections.py to include an syntax example showing how to remove package holds
* Fix pep8 issue: dpkg_selections.py
Missed some trailing spaces when adding example text.
(cherry picked from commit 489b7f1859)
* fix ternary problem (#78186)
(cherry picked from commit 7ec8916097)
* documentation: become plugins: correct pluralization in description (#78196)
fixes#78195
(cherry picked from commit e10851d495)
* Update python_3_support.rst (#78239)
Fix invalid characters warning when using example group names. (Replace dashes in example group name with underscores.)
(cherry picked from commit be88250a83)
* Adding applicable licensing requirements (#78162)
(cherry picked from commit 6ddccc1604)
* Document choice to use Ansible Project in copyright lines (#78164)
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
(cherry picked from commit c9a341902b)
* stat only returns pw_name/gr_name if those can be looked up (#78183)
(cherry picked from commit 221af7640d)
* example showing how to deal with keys w/o apt_key (#78206)
(cherry picked from commit 0590ce065c)
* Migrate vmware.vmware_rest Dev Guide to collection (#78255)
(cherry picked from commit 048adac5b7)
* Updated release_and_maintenance.rst (#78272)
(cherry picked from commit 358a8ad7af)
Co-authored-by: Micah Beasley <MBfromOK@users.noreply.github.com>
Co-authored-by: Alex Willmer <alex@moreati.org.uk>
Co-authored-by: Nathan Hyde <nathanhyde@gmail.com>
Co-authored-by: Desmond Obisi <51109125+DesmondSanctity@users.noreply.github.com>
Co-authored-by: Evgeni Golov <evgeni@golov.de>
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Mario Lenz <m@riolenz.de>
Co-authored-by: Ompragash Viswanathan <21008429+Ompragash@users.noreply.github.com>
* Fix workding for :ref|term: subsitution
now matches 'seealso' and makes more sense:
```
delay:
applies_to:
- Task
description: Number of seconds to delay between retries. This setting is only used
in combination with `until`.
```
vs
```
delay:
applies_to:
- Task
description: Number of seconds to delay between retries. This setting is only used
in combination with website for `until`.
```
* updated unit tests
* match see also
* more sanity
(cherry picked from commit d26801e994)
* Fix 'Permission denied' in user module while generating SSH keys
Fix#78017
Use try/except for spwd usage to prevent "Permission denied".
Signed-off-by: Sagi Shnaidman <sshnaidm@redhat.com>
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit 30a923fb5c)
In the classic Jinja2's Environment str() is called on the return value of the
finalize method to potentially trigger the undefined error. That is not
the case in NativeEnvironment where string conversion of the return value is
not desired. We workaround that by checking for Undefined in all of our concat
functions. It seems simpler to do it earlier in the finalize method(s) instead.
As a side-effect it fixes an undefined variable detection in imported templates.
Fixes#78156
ci_complete
(cherry picked from commit 17d52c8d64)
* fixed hyperlinks to galaxy.ansible for issue #78104 (#78152)
(cherry picked from commit 4bd7e50612)
* Fix documentation with misleading data type (#78157)
##### SUMMARY
The sample value given for the 'mode' parameter is shown without quotes, but the data type is string. If you actually try to use an unquoted numeric string for this value you're in for a nasty surprise! I added quotation marks to the sample value.
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
(cherry picked from commit 4c9385dab7)
* Update README.md (#78139)
Update Python version requirement to >= 3.8.
(cherry picked from commit 58b42abede)
* elaborate on import/include conditional example (#78138)
(cherry picked from commit 7ec84c511f)
* Update vault.rst (#78140)
(cherry picked from commit 84105e39f4)
Co-authored-by: Lewis Brogan <ilewisbrogan@gmail.com>
Co-authored-by: mlevens-hw <55262896+mlevens-hw@users.noreply.github.com>
Co-authored-by: Diana S. Cardona <89112552+Flecha21@users.noreply.github.com>
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
Co-authored-by: jlownie <jlownie@hotmail.com>
* ansible-galaxy - support resolvelib versions >= 0.5.3, <= 0.8.1
Test incompatibilities are removed for resolvelib >= 0.6.0
Test against the latest 0.8.x version and fix requirements
* Fix tests - use a venv for testing the range of resolvelib versions
* Update temporary hardcoded fallback for ansible-test
* Update hardcoded upperbound for sanity tests
* Make error check more flexible
(cherry picked from commit 143e7fb45e)
* Fix ansible-galaxy traceback when unexpected version of resolvelib is installed (#77630)
* Fix traceback when a supported version of resolvelib is not installed
Try to read the supported version range from the package distribution info and fall back to a hardcoded lowerbound/upperbound (>=0.5.3,<0.6.0).
* Add tests for unsupported resolvelib versions
* Resolve remaining import sanity test issues.
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Martz <matt@sivel.net>
(cherry picked from commit 82f3a57bee)
* Fix boolean condition so that ansible-galaxy collection install works when a valid resolvelib is installed. (#77906)
(cherry picked from commit 6fbc8bd2bc)
* ansible-galaxy - ensure variable is defined for any error when getting the ansible-core distribution (#77993)
(cherry picked from commit db335498d0)
Co-authored-by: Felix Fontein <felix@fontein.de>
* `importlib.util` appears to be lazily imported and is sometimes unavailable as an attribute of `importlib` without an explicit import
(cherry picked from commit 6e78425f8d)
* apt: fix virtual package install version detection
Change 4a62c4e3e4 introduced version
matching in installation.
The problem stems from
if version_installable or version:
pkg_list.append("'%s=%s'" % (name, version_installable or version))
When the package is a virtual-package, package_status() is returning
the "version_installable" of the package *satisfying* the
virtual-package; but then this is trying to install the
virtual-package with this version pin.
For example, "yaml-mode" is a virtual package satisifed by
"elpa-yaml-mode" (currently 0.0.14-1) and trying to install it fails
with
$ usr/bin/apt-get -y ... install 'yaml-mode=0.0.14-1'
... failed: E: Version '0.0.14-1' for 'yaml-mode' was not found ...
In the case of a virtual-package with nothing installed to satisfy it,
we should just return blank values to allow apt-get to do it's thing.
The tests are updated to install and remove this package.
Fixes: #76779
(cherry picked from commit e4c0bbf885)
Co-authored-by: Ian Wienand <iwienand@redhat.com>
* If there is a platform specific handler, prefer the resolved module over the resolved action when loading module_defaults
Add a toggle for action plugins to prefer the resolved module when loading module_defaults
Allow moving away from modules intercepted as actions pattern
Fixes#77059
(cherry picked from commit 621e782ed0)
* winrm, psrps added missing var entry
this handles issue with the default being set to inventory_hostname
but defaults not being templated implicitly
fixes#77841
(cherry picked from commit eecbaee7f4)
* make the pre-flight sniffing more robust to different failure conditions (was failing on Ubuntu 22.04 VMs)
* remove skip aliases (the test needs to function everywhere to assert that the selinux facts bits behave properly when it's N/A)
(cherry picked from commit 4ff10fcd31)
* Fix 'ansible-config dump --only-changed -t all' to only display headers if plugin options are changed
* changelog
* add a test
(cherry picked from commit 1214b63f4f)
* ansible-test - Backport `InternalError`
NOTE: This is a partial backport, including only one new class.
(cherry picked from commit b960641759)
* ansible-test - Fix subprocess management. (#77641)
* Run code-smell sanity tests in UTF-8 Mode.
* Update subprocess use in sanity test programs.
* Use raw_command instead of run_command with always=True set.
* Add more capture=True usage.
* Don't expose stdin to subprocesses.
* Capture more output. Warn on retry.
* Add more captures.
* Capture coverage cli output.
* Capture windows and network host checks.
* Be explicit about interactive usage.
* Use a shell for non-captured, non-interactive subprocesses.
* Add integration test to assert no TTY.
* Add unit test to assert no TTY.
* Require blocking stdin/stdout/stderr.
* Use subprocess.run in ansible-core sanity tests.
* Remove unused arg.
* Be explicit with subprocess.run check=False.
* Add changelog.
* Use a Python subprocess instead of a shell.
* Use InternalError instead of Exception.
* Require capture argument.
* Check for invalid raw_command arguments.
* Removed pointless communicate=True usage.
* Relocate stdout w/o capture check.
* Use threads instead of a subprocess for IO.
(cherry picked from commit 5c2d830dea)
* ansible-test - Add support for remote Ubuntu VMs.
(cherry picked from commit 6513453310)
* ansible-test - Fix remote completion validation.
(cherry picked from commit e2200e8dfc)
* ansible-test - Add multi-arch remote support.
(cherry picked from commit 2cc74b04c4)
* ansible-test - Enhance the shell command. (#77734)
* ansible-test - Add shell --export option.
* ansible-test - Support cmd args for shell command.
Also allow shell to be used without a valid layout if no delegation is required.
* ansible-test - Improve stderr/stdout consistency.
By default all output goes to stdout only, with the exception of a fatal error.
When using any of the following, all output defaults to stderr instead:
* sanity with the `--lint` option -- sanity messages to stdout
* coverage analyze -- output to stdout if the output file is `/dev/stdout`
* shell -- shell output to stdout
This fixes issues two main issues:
* Unpredictable output order when using both info and error/warning messages.
* Mixing of lint/command/shell output with bootstrapping messages on stdout.
* ansible-test - Add changelog fragment.
(cherry picked from commit fe349a1ccd)
* ansible-test - Fix remote args restriction.
The platform-specific and global fallbacks were not working with the `--remote` option.
This regression was introduced by https://github.com/ansible/ansible/pull/77711
(cherry picked from commit 76ead1e768)
* docs: update porting guide for ansible 6.0.0b1 (#77913)
(cherry picked from commit 7c89562580)
* docs: add missing notes in porting guide for 6.0.0b1 (#77926)
There was an issue with the changelog in community.network 4.0.0, it's
been fixed in 4.0.1 and now properly included.
(cherry picked from commit 9f7956ba30)
* Remove platform dev guides from core docs (#77915)
(cherry picked from commit f3c684564c)
* Migrate community.vmware Dev Guide to collection (#77723)
(cherry picked from commit b69eb28ed2)
* Issue #77681 revamp getting started guide (#77897)
(cherry picked from commit 89e4fb71e6)
* ssh retry docs addition (#77930)
(cherry picked from commit 15750aec52)
* fix: update button text to "Load token" (#77925)
(cherry picked from commit 16e0d4a121)
* fix bad merge conflict resolution
Co-authored-by: David Moreau Simard <moi@dmsimard.com>
Co-authored-by: Mario Lenz <m@riolenz.de>
Co-authored-by: Don Naro <dnaro@redhat.com>
Co-authored-by: Dan Kolepp <10145457+dkolepp@users.noreply.github.com>
* ansible-test - Add RHEL 9.0 remote support.
* Add RHEL 9.0 to CI. (#77853)
* Add RHEL 9.0 to CI.
* Restrict network manager inspection to RHEL8
* Skip module tests when astream_name is undefined, undefine it for RHEL9 until 9.1
* Remove redundant test.
Co-authored-by: Matt Martz <matt@sivel.net>
(cherry picked from commit 43d650f924)
* Include test fix from https://github.com/ansible/ansible/pull/77847
* stub out ovirt dev guide with pointer to new location (#77751)
(cherry picked from commit 5e22fe21fd)
* Docsite/communication.rst: fix formatting (#77760)
(cherry picked from commit 97e574fe6e)
* Updated the playbook section in this docs (#77747)
(cherry picked from commit 42086c14a3)
* 2.13 GA has been bumped up a week to release on May 16 (#77772)
(cherry picked from commit 75bc630565)
* Fix plugin dev examples (#77759)
* Fix plugin dev examples
* Update docs/docsite/rst/dev_guide/developing_plugins.rst
Co-authored-by: Felix Fontein <felix@fontein.de>
(cherry picked from commit 9923e92d97)
* docs: Update ansible-core release date in Ansible roadmap (#77775)
(cherry picked from commit 235f2598b1)
* revamp basic concepts docs (#77748)
(cherry picked from commit 8e91403412)
* core-2.13 - update release table (#77780)
(cherry picked from commit 6230244537)
* clarify Ansible EOL details (#77778)
* clarify Ansible EOL details: update docs/docsite/rst/reference_appendices/release_and_maintenance.rst
Co-authored-by: Felix Fontein <felix@fontein.de>
(cherry picked from commit 654fff05e6)
* Update url to quickstart video. (#77779)
(cherry picked from commit f75bb09a8b)
Co-authored-by: Andrew Klychkov <aklychko@redhat.com>
Co-authored-by: Desmond Obisi <51109125+DesmondSanctity@users.noreply.github.com>
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Mario Lenz <m@riolenz.de>
Co-authored-by: David Moreau Simard <moi@dmsimard.com>
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Tabah Baridule <dulemartins07@gmail.com>
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
* docs: Update porting guide for ansible 5.7.0 (#77646)
(cherry picked from commit 709ae7c050)
* remove import_play examples from include module (#77597)
(cherry picked from commit d321fa3f15)
* trim down to use sanity test docs requirements file (#77612)
* trim down to use sanity test docs requirements file
* add requirements back
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update docs/docsite/rst/community/documentation_contributions.rst
(cherry picked from commit a65fbfad5b)
* Typo fix (#77648)
(cherry picked from commit cf5f0690f0)
* Update guide_vultr.rst (#77665)
(cherry picked from commit f6fb402d1f)
* add select distro installs to docs (#77567)
(cherry picked from commit 096393a221)
* community_steering_committee.rst: add new member (#77589)
(cherry picked from commit 3cbfee7852)
* clarify latest release (#77577)
* clarify latest release
* fix tables
(cherry picked from commit f633b62cbc)
* typo in doc (#77672)
(cherry picked from commit c71faec595)
Co-authored-by: David Moreau Simard <moi@dmsimard.com>
Co-authored-by: deosrc <deosrc@g-de.co.uk>
Co-authored-by: Jan-Piet Mens <jp@mens.de>
Co-authored-by: Andrew Klychkov <aklychko@redhat.com>
Co-authored-by: Alexei Znamensky <103110+russoz@users.noreply.github.com>
* Prevent losing unsafe from lookups
This patch fixes a bug which under certain conditions results in data
returned from lookups not being marked as unsafe.
Each time Templar.do_template is invoked a new AnsibleContext is
created and stored effectively at two places:
1) as an instance variable in templar_obj.cur_context
2) as a local variable called new_context in do_template method of Templar
Due to custom functionality in Ansible's Context that allows for nested
templating it is possible that during resolving variable's value
template/do_template method is called recursively again, again creating
a new context. At that point the problem manifests itself because as
mentioned in 1) above the context is overwriten on the templar object
which means that any subsequent calls to _lookup will use the new
context to mark it as unsafe which is now different to the local
new_context which is used for testing for unsafe property.
The solution to the problem appears to be to restore the original
context inside do_template and also to eliminate the local variable
new_context to prevent problems in the future.
It appears that we don't have a better way of storing the context other
than as some form of global variable and so this appears to be the
"best" solution possible at this point. Hopefully data tagging will be
the solution here.
For more examples see unit and integration tests included in this patch.
Fixes#77535
(cherry picked from commit 3980eb8c09)
When looking up the `no_log` setting for a parameter that is an alias in
`AnsibleModule._log_invocation()`, the alias value will always be an
empty dictionary since `self.aliases` on the `AnsibleModule` instance is
never updated after initialization. Since the `no_log` setting is on the
canonical parameter not the alias, an incorrect warning is issued if the
parameter matches `PASSWORD_MATCH`.
This PR returns the aliases dictionary as an attribute of the
`ValidationResult` and updates the `aliases` attribute on the
`AnsibleModule` instance.
(cherry picked from commit 1b947eaf92)
Co-authored-by: Sam Doran <github@samdoran.com>
* git fix docs and wrapper script
fixes#77582
now env var is set to wrapper or full command depending on version
as was the intent of previous PR
added ref to git commit from git for why/how we used the env vars
* handle key_file
(cherry picked from commit d06d99cbb6)
* add links.yml to collection structure docs (#77415)
* add links.yml to collection structure docs
* fix link
* Update docs/docsite/rst/dev_guide/developing_collections_structure.rst
(cherry picked from commit 8caa6efaab)
* Example is a single play, not a playbook
(cherry picked from commit bf8e186c68)
* Fix list formatting (#77354)
(cherry picked from commit e918cfa588)
* Most Matrix channels are ansible.com and not .im (#77352)
(cherry picked from commit 6557a60d08)
* Docs cheatsheet - skeleton and ansible-playbook (#76655)
(cherry picked from commit 21addac5da)
Co-authored-by: rwhector <richard@walnut.gen.nz>
Co-authored-by: Christian Adamini <christian.adamini@invision.de>
Co-authored-by: bluikko <14869000+bluikko@users.noreply.github.com>
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
* [stable-2.13] Catch ImportError when pyyaml doesn't have libyaml extension (#77434)
(cherry picked from commit e3aa73c)
Co-authored-by: Matt Martz <matt@sivel.net>
* Add test to assert that missing libyaml doesn't result in an error. Fixes#77437
(cherry picked from commit 2797dc644a)
* Add option --no-fail-on-errors to return errors for ansible-doc --metadata-dump in JSON result instead of failing.
* Adjust changelog fragment.
* Add basic tests.
* Support ignoring of certificates for ansible-galaxy during SCM cloning
* Add integration tests installing a role from an untrusted repository
Test installing the role without --ignore-certs fails
Test installing the role with --ignore-certs is successful
* Add find_spec and exec_module to RestrictedModuleLoader
* Fix getting new loader with correct path
Fix pep8 errors
* Use convert_ansible_name_to_absolute_paths instead of the loader path
* Apply suggestions from code review
* Fix type hints and test them in CI
* Fix error message for ansible.module_utils.basic if it's missing
Add mypy ignored missing imports for controller sanity tests
* Add mypy attr-defined ignore entries for python 3.8, 3.9, 3.10 for vendored six
Add mypy attr-defined ignore for python 2.7 in lib/ansible/utils/collection_loader/_collection_finder.py
* Just test controller python versions to simplify ignoring mypy errors
* Add a toggle to control the number of signatures required to verify the authenticity of a collection
* Make the default number of required valid signatures 1
* Add option to make signature verification strict and fail if there are no valid signatures (e.g. "+1")
* Use a regex to validate --required-valid-signature-count
* Add a toggle to limit the gpg status codes that are considered a failure
* Update documentation and changelog
* Add unit and integration tests for the new options
* Fixes#77146
Fix using user-provided signatures when running 'ansible-galaxy collection verify ns.coll --offline'
Add a test for a user-provided signature when running ansible-galaxy collection verify with --offline
Fix displaying overall gpg failure without extra verbosity
Add a test for displaying gpg failure without verbosity
Improve documentation to be more clear that signature verification only currently applies to collections directly sourced from Galaxy servers
For IPv6 addresses, Ansible already provides the prefix length for IP
addresses in the `prefix` fact. This patch adjusts the facts for IPv4
addresses to also contain the prefix length in the prefix fact. This
makes it easier to use the facts consistently when the CIDR notation is
needed.
Signed-off-by: Till Maas <opensource@till.name>
While getting hostname from container, podman command
fails to return JSON so wrap exception and return
hostname as 'None'
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Add more sanity tests on schema level (so they also work for plugins).
* Fix various issues the sanity test reported.
* Add changelog fragment.
* Fix function name.
Describe difference between targeting a single host using -i vs. --limit. In particular, pointing out the behavior difference that -i will not rad inventory variables.
* Let validate-modules also validate plugins.
* Support 'option' in 'cli'.
* Use DOCUMENTABLE_PLUGINS instead of UNDOCUMENTED_PLUGIN_TYPES.
* Support 'keyword', clean up error codes.
* Call settings.process_errors only once; remove __version__.
* Add changelog fragment.
* Fix ansible console to respect ANSIBLE_STDOUT_CALLBACK
This makes the ansible-console able to customize the stdout callback.
Ansible console currently uses statically minimal callback.
* Add changelog
* Move the fragment changelog in the right directory
`with_first_found` only takes a list of things, it's not like `tags`
where it can take a thing or a list of things.
Inspired by locally discovered breakage and then finding
https://github.com/ansible/ansible/issues/77136 where someone was also
doing this and getting into trouble.
* Remove unnecessary PyCharm ignores.
* Ignore intentional undefined attribute usage.
* Add missing type hints. Fix existing type hints.
* Fix docstrings and comments.
* Use function to register completion handler.
* Pass strings to display functions.
* Fix CompositeAction handling of dest argument.
* Use consistent types in expressions/assignments.
* Use custom function to keep linters happy.
* Add missing raise for custom exception.
* Clean up key/value type handling in cloud plugins.
* Use dataclass instead of dict for results.
* Add custom type_guard function to check lists.
* Ignore return type that can't be checked (yet).
* Avoid changing types on local variables.
Also, bump the default. A number of people have reported problems with
this 20 second timeout for a couple of years on
https://github.com/ansible/galaxy/issues/2302
It is mentioned in the issue that resolving dns can sometimes take up to
30 seconds, and that this timeout includes dns resolution.
Includes a changelog message
The current usage only works if `python` is Python 2.x since the syntax is invalid on Python 3.x.
When running on Python 3.x it would either be missing, or if Python 2.x was also present, incorrect.
* apply owner/group permissions to top folder
* remove unused var unarchive30
* fix permissions for top folders if the tarball include multiple top folders
* added test for top folder ownership
The use of `--venv-system-site-packages` was an optimization to use the `coverage` package pre-installed in the AZP test container.
However, now that the venv is bootstrapped by ansible-test that optimization no longer makes sense, since other downloads are already taking place.
ci_coverage
ci_complete
* fix facts delegation loop overwrite
partial revert of change to allow facts to be present in each loop iteration
was not needed in final results as result processing alreayd had the disctiontion
and ended up breaking the assumptions in the calling code.
fixes#76676
* play_context, compensate for existing plugins
some connection plugins are not fully using the correct configuration,
but this was previously hidden from them as play_context was providing
the info instead, now play_context provides the 'correct' info, but hitting
these bad configurations.
* ansible-galaxy collection install|verify:
- Support verifying the origin of the MANIFEST.json when the Galaxy server has provided signatures.
- Allow supplemental signatures to use during verification on the CLI/requirements file.
* ansible-galaxy collection install:
- Support disabling signature verification. This silences the warning provided by ansible-galaxy if the Galaxy server provided signatures it cannot use because no keyring is configured.
- Store Galaxy server metadata alongside installed collections for provenance. This is used by 'ansible-galaxy collection verify --offline'.
* Add unit tests for method that gets signatures from a Galaxy server
* Add integration tests for user-provided signature sources
- Test CLI option combinations
- Test installing collections with valid/invalid signature sources
- Test disabling GPG verification when installing collections
- Test verifying collections with valid/invalid signature sources
* Make signature verification advisory-by-default if signatures are provided by the Galaxy server
- Make the default keyring None
- Warn if the keyring is None but the Galaxy server provided signatures
- Error if the keyring is None but the user supplied signatures
- Error if the keyring is not None but is invalid
* changelog
* add ansible-galaxy user documentation for new options
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
Co-authored-by: Sandra McCann <samccann@redhat.com>
Co-authored-by: Andy Mott <amott@redhat.com>
Co-authored-by: John R Barker <john@johnrbarker.com>
* Return rc=0 on success.
Error handling in playbooks generally expects `rc` to be set to 0 when a module has not failed. Playbook authors should not have to check for the existence of `rc` first.
* Use single definition and added changelog
* Fix up tests with new return value
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* add DebianStrategy tests
* ensure hostname can be changed by using become
* use Systemd strat for debian and Base for generic.
* add test to ensure all strategies are available
Signed-off-by: Thomas Sjögren <konstruktoid@users.noreply.github.com>
avoid polluting vars with incorrect settings
simplify variables and templars
- variables, original, only updated with final results, reset are copies of this
- tempvars used for preliminary templating
- cvars used for connection/shell/become plugins, delegation aware
- vars_copy ignore tempvars updates and use connection plugin to get
finalized version per loop item/delegation, also used to store temp results
till we are ready to update 'variables'
- fine tune nolog just cause we are here
- also fix inventory_hostname_short for IP addresses
git module now uses env vars exclusively
- updated docs to clarify usage
- now env vars append instead of overwrite to allow existing custom setups to keep working
fixes#38104, #64673, #64674
- added note for hostkeychecking more securely
fixes#69846
- keep script cause old versions still choke on env
- env var cannot hold more than 'command' for older versions
- all ssh_opts in one place
ssh plugin, use 'correct' information source in all cases
* still fallback to pc
* added inventory to new test
* undef var can still show as parser error on pc
now task_exectuer has a more accurate error handling
Installation of `gcc` is failing while trying to install dependencies:
```
[MIRROR] libxcrypt-devel-4.4.18-3.el9.x86_64.rpm: Interrupted by header callback: Server reports Content-Length: 32565 but expected size is: 33101
```
Azure Pipelines only supports publishing a single code coverage file.
This limits us to reporting on either Python or PowerShell.
It also prevents us from reporting coverage per test command without further aggregation.
Most of our users are only aware of the reports published to codecov.io.
* ansible-test - Use relative paths in junit output.
Also fix a traceback in the junit callback during automatic fact gathering.
* ansible-test - Handle out-of-tree JUnit paths.
* minor refactor in other options by pushing common code into functions
* consolidate coll_filter
* more normalizing loader
* dont pass plugin_loader, its global import
* Also dump roles and collections
* adjusted tests to new err msg
* disable namespace filter (unused)
* ansible-test - Clarify need for empty __init__.py
* ansible-test - Update code-smell boilerplate.
* Update code-smell boilerplate for core.
* Update future boilerplate test for ansible-test.
All ansible-test code (except for targets) and core-specific sanity tests now use the same boilerplate.
The test also checks for unwanted `__future__` and `metaclass` boilerplate.
* Relocate target tools to the correct directory.
Several tools used on target Python versions were incorrectly placed in the controller directory.
* Make include_role/include_tasks work with any_errors_fatal v2
Fixes#50897
Co-authored-by: Matt Martz <matt@sivel.net>
* Add failed to results in free strategy too
* Fix
* Avoid duplicating results
* ci_complete
Co-authored-by: Matt Martz <matt@sivel.net>
* ansible-test - Improve help for unsupported cwd.
* The `--help` option is now available when an unsupported cwd is in use.
* The `--help` output now shows the same instructions about cwd as would be shown in error messages if the cwd is unsupported.
* Add `--version` support to show the ansible-core version.
* The explanation about cwd usage has been improved to explain more clearly what is required.
Resolves https://github.com/ansible/ansible/issues/64523
Resolves https://github.com/ansible/ansible/issues/67551
* group2 - switch to setup_remote_tmp_dir instead of using output_dir
* output_file needs to be in the tmp dir
* Use comparison that should work with macos
* remove unused dep
* allow inputting 0 for password_expire_{min|max}
0 is meaningful for min days (any time)
0 is technically valid for max_days
* add test for setting both min and max expiry
* [0] return result of execute_command from set_password_expire*
* [1] better return for set_password_expire
* [2] handle returns from set_password_expire*
* only set password expiry if user exists
* collect return-handling code
* combine password min and max into one execution
* handle case where spwd is not present like on macOS and FreeBSD
Co-authored-by: Sam Doran <sdoran@redhat.com>
* config, moved 'choices' from wall of text, to keys
* updated to ensure all pythons handle the 2 modes correctly
* Update lib/ansible/config/base.yml
Co-authored-by: Sandra McCann <samccann@redhat.com>
* go back to keyword being explicit for plugin configs instead of using option name as implicit match
* switch spec to singular, no changes needed since no one used
* Don't send full task with async callback events. Fixes#76729
* Use args for async_status task, instead of k=v
* Make sure we send back the async task attrs for polling
* Add clog frag
* load is a staticmethod
* include_vars initialize failed
if source dir is given, but not present and traversal is empty you
can end up trying to access failed w/o it ever being defined.
also future proof for more corner cases in decision tree
dont display _terms or _intput on only changed
those always change and it expected for the plugins that support them
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
- Avoids false positives on become strings being echoed back
by ssh cli itself
- added test for debug lines
- also simplified some of existing test code
modules with python were always normalized to /usr/bin/python,
while other interpreters could have specific versions.
* now shebang is always constructed by get_shebang and args are preserved
* only update shebang if interpreter changed
* updated test expectation
* added python shebang test
* Update dnf tests to reflect new behavior.
Previously dnf would report there was nothing to do when trying to install a package from the cache when it was not present.
A recent update to dnf has changed this behavior to match yum, resulting in a failure instead.
* Allow dnf to fail or report no changes.
* ansible-galaxy - fix the --ignore-certs flag for the implicit galaxy server
* changelog
* Add a test without the server config
* Fix respecting --ignore-certs for individual --server URLs also
* Update changelogs/fragments/76735-ansible-galaxy-fix-ignore-certs.yaml
This adds the ability to specify a package version using >=. This will ensure the package is at the specified version or above.
* If the package is not installed, the latest version will be installed.
* If the package is installed and less than the specified version, it will be upgraded.
* If the package is installed and greater than or equal to the specified version, it will be left alone.
The version selection is handled by Apt itself, so things like the system policy, pinning, etc, are considered.
* Add additional tests for check_mode with shell/command/script
* update check_mode documentation: if a module/action is not skipped in check mode and accurately reflects whether a change is made to the remote support should be 'full'
* Make reporting skipped in check mode mutually exclusive with 'changed: True'
* Add missing documented attributes
* Fix tests to expect skipped=True and changed=True do not occur together
* Fix script check_mode support documentation
* Fix earlier changelog
* document platforms attribute
* Use tasks's check mode since the value from PlayContext does not reflect loop items
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
'dscl -list' returns 0 even if the user does not exists. This
leads to errorenous condition in user module.
Using 'dscl -read UniqueID' can return if user exists or not.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Ensure the version is valid for directories and collections in git repos before installing
Fix the error message for invalid semantic versions
* Make requested changes
* Add a test case for unhandled ValueError exception
* Add changelog
* Update lib/ansible/galaxy/collection/galaxy_api_proxy.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Reword error message
Include link to learn how to compose a semver version
* Move version validation into the caller, find_matches
* Add tests for more invalid version types
* Remove unused import
Fix raising unexpected error
* Update lib/ansible/galaxy/collection/__init__.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Update lib/ansible/galaxy/dependency_resolution/providers.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Update lib/ansible/galaxy/dependency_resolution/providers.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* fixes#28977
* Include the original error message when pick_handler fails
* Add a test to unarchive a tar file with an invalid extra option
* add a changelog
* Disable the _distutils_hack in newer setuptools. Doesn't fix the underlying issue of the venv finding the _distutils_hack of a setuptools that is not its own.
ci_complete
* re-throw blanket
* check if service is indirect status / add integration test
* Update changelogs/fragments/76453-indirect-systemd-status.yml
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
* simplify indirect check / remove integration test restriction on rhel9
* add dummy indirect service for integration tests
* unarchive: fix non-english locales
For GNU Gettext, the LANGUAGE environment variable takes precedence over LANG or LC_ALL. On systems where LANGUAGE was set to a non-english locale, the output of the tar command therefore not understood and the module failed silently ("changed": false, but the archive was not extracted).
* add tests
* changelog
* check finder type before passing path
ci_complete
* Reduce nesting
* Test find_module does not cause a traceback with Python 3 FileFinder
* Update lib/ansible/utils/collection_loader/_collection_finder.py
Filter out the /usr/bin/pkg because on Altlinux it is actually the
perl-Package (not Solaris package manager).
Since the pkg5 takes precedence over apt, this workaround is
required to select the suitable package manager on Altlinux.
* reset defaults for json filters
these had change on unification of json parsing/dumping
now they behave like before, but are still controllable by user.
* Implement find_spec and exec_module to remove reliance on deprecated methods in the collection loader
ci_complete
* Move module execution to exec_module
Remove extra sys.modules handling
Use default module initialization by returning None from loader.create_module
Refactor
ci_complete
* Remove ansible-test's copy of the collection loader
ci_complete
* Fix metaclass for Python 2.x
ci_complete
* Fix Py2/Py3 syntax compatibility
* Refactor
ci_complete
* update collection_loader comments
ci_complete
* simplify find_module
ci_complete
* Fix Py2 compatibility - don't get loader from nonexistent spec
Remove unnecessary PY3 checking
* Refactor common code in load_module and exec_module
ci_complete
* tidy diff
ci_complete
* Include collection_loader in target paths for 'compile' sanity test
* add changelog
* Add "return None" instead of doing it implicitly
Remove get_filename
short-circuit exec_module if it's a redirect
ci_complete
Ref #44741
Ref ansible/proposals#140
* Replace select filter with a more portable thing
* Add context
This is needed for split controller/remote
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Add changelog fragment and integration tests
* Break up function signature into multiple lines
It passed the 160 character width. Change it to multi-line for easier reading
and better diffs in the future.
* Remove unused import
* Update version added for new feature
* Move changelog fragment to correct location
Co-authored-by: Thomas Stähle <thomas.staehle@check24.de>
Co-authored-by: Sam Doran <sdoran@redhat.com>
* align requirements files to more recent versions
* bump up to latest antsibull
* update sphinx in rstcheck requirements
* update to dependency versions that sphinx 4.2 pulls in
* use same no-cap spelling
* Add script to freeze sanity requirements.
* Declare sanity test requirements and freeze
* Use pinned requirements for import.plugin test.
* Expand scope of import test for ansible-core.
* Add ignores for galaxy import errors.
* Update test-constraints sanity test.
* ansible-test - Add support for RHEL 8.5 instances.
* Clear libdnf checksum cache from unsigned package in dnf integration test.
Co-authored-by: Matt Martz <matt@sivel.net>
* First pass of adding yaml result format output to default callback
* Add clog frag
* flow style always false
* Further normalization of results across distros
* no lossy, now pretty
* Fix env var in runme.sh
* Rename variable to better self document
* include NativeJinjaUnsafeText
* Linting cleanup
* Add tests specific to the prior yaml callback
* Make data munging more exact, following the checks in libyaml/pyyaml
* Remove unused import
* Extend comment
* more correct and some comments
* Fix consistency of tab vs space in this section of the file
* Add str representer
* be cooler
* faster character filtering
* Clarify None as a sentinel for default behavior
* Flip filtering logic to avoid CRLF issues with 'space break' filtering
* Py3 change and comment about the use of __call__
Co-authored-by: Matt Davis <mrd@redhat.com>
- On Python 3, decode bytes to str
- Tolerate whitespace around the line.
- Report a missing HOSTNAME line as an error.
- Don't clobber line separators when setting a new hostname.
- Add some tests for this strategy to verify correct operation.
* `podman inspect` falls back to a same-named image if the named container is not present; since eg `http-test-container` is both the name of the image and container, it wasn't working properly under podman in many instances. Switching to `docker|podman container inspect` limits the query to containers only for both podman and docker, allowing the support container detection/creation to work properly.
This task was previously added in https://github.com/ansible/ansible/pull/25243 as a work-around for a setuptools bug.
The pinned version does not work with Python 3.10, and the task should no longer be needed.
* Hashing is not encrypting
All encryption systems have a formal inverse function to decrypt
A hash is a one way function without inverse by definition
* Update docs/docsite/rst/user_guide/playbooks_filters.rst
Co-authored-by: Sandra McCann <samccann@redhat.com>
* Move RHEL pip package install into function.
* Add RHEL 9 support to bootstrap script.
* ansible-test - Add RHEL 9 support.
* Add install and uninstall instructions for RHEL9 to paramiko tests
* Skip incidental_inventory_docker_swarm on RHEL 9.
* Update dnf tests for rhel9
* Fix prepare_http_tests on RHEL 9.
* fix RHEL9 Python discovery tests to match distro map
* Skip subversion test on RHEL 9.0b.
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Matt Davis <mrd@redhat.com>
* Skip recursive suboption validation if sub_parameters is not a dictionary
* Ensure sub parameter elements is a sequence to prevent iterating over string characters and causing duplicate error messages for the same param
* Try to find any valid vault id before raising an exception
* Apply the same logic during vault password file loading
* Also. Improve the warning msg and tests
When we introduced an include parameter to the unarchive module, we
inadvertenly flipped the exclusion logic. This flip meant that the
unarchive module started rejecting files that should be extracted.
This commit flips the bad logic and adds some tests that will make
sure things do not go bad again.
Clarify that the `requires_ansible` field in meta/runtime.yml refers to the version of Ansible Core (ansible-core), not any package called "ansible" e.g. https://pypi.org/project/ansible/
* Use NativeEnvironment for all templating
ci_complete
* Keep Templar.copy_with_new_env for backwards compat
* Mention that AnsibleUndefined.__repr__ changed in the porting guide
* Templar.copy_with_new_env backwards compat
* ci_complete
The scripts now use `--list-targets -v` instead of `--explain` to evaluate changes.
This is faster and does not trigger parsing of a non-existent inventory file.
* Overhaul ansible-test integration tests.
* ansible-test - Fix import test pyyaml usage.
* ansible-test - Remove unused import.
* ansible-test - Fix traceback when pip is unavailable.
* ansible-test - Fix typo in port forwarding message.
* ansible-test - Fix controller logic in requirements install.
* Fix unit tests in ansible-test integration test.
Unit tests are now run for available Python versions which
provide `virtualenv` (Python 2.x) or `venv` (Python 3.x).
* Don't use output_dir in user tests
* Move blockinfile tests from using output_dir to depending on setup_remote_tmp_dir
* Don't use output_dir in git tests
* Don't use output_dir in uri tests
* Update network_resources.rst
I am not sure a fully qualified URL should be here... but teach me great documentation gods
* Change platform index external link to internal ref
Co-authored-by: Aine Riordan <44700011+ariordan-redhat@users.noreply.github.com>
Unofficial packaging should be maintained externally since
it is not tested in CI or used in the official release process.
Maintainers of unofficial packages are better equipped with the necessary
expertise and testing resources to support alternative packaging.
Prepend the task name to the warning "The loop variable 'item' is
already in use", so that you can get some context even if stdout and
stderr go to separate places.
- The containers are now based on Ubuntu 20.04 instead of Ubuntu 18.04.
- The default Python version used for the containers is now Python 3.10.
- The included version of PowerShell was upgraded to version 7.1.4.
- The included version of shellcheck was upgraded to version 0.7.0.
ci_complete
* removes AWS scenario guide, moving to collection
* first attept to replace TOC entries
* not sure what I did, but saving it
* updates TOCs to reflect new location of aws guide
* reinstates original page as a stub
* adds links to new location for AWS guide, updates header
Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
* Removed translatable words from code blocks
##### SUMMARY
Removed translatable words from codeblocks as per #59449
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
* Maintaining intend as well as clearity
* Preserving "save_as" as the key
* showing equivalence and keeping same context
* Bump antsibull version to get support for attributes and CLI options.
* Include fix for attributes.
* Include fix for rst_epilog collision and for #75799.
* docs - add some info on plugin option source precedence
##### SUMMARY
Add some clarity on precedence, especially from sources of the same type.
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
* update RST formatting
* Update docs/docsite/rst/dev_guide/developing_plugins.rst
Apply review suggestions
Co-authored-by: Aine Riordan <44700011+ariordan-redhat@users.noreply.github.com>
Co-authored-by: Aine Riordan <44700011+ariordan-redhat@users.noreply.github.com>
Resolves https://github.com/ansible/ansible/issues/75320
The option `--prime-containers` was chosen over `--docker-pull-only` to match the recently added `--prime-venvs` option for sanity tests.
It would also fit well with a future `--prime-requirements` option for pre-installing requirements for unit and integration tests.
* ansible-test - Create injector scripts at runtime.
* Set bootstrap.sh shebang at runtime.
* Remove shebang and execute bit from importer.
* Update shebang sanity test.
* Preserve line numbers.
* Use the task dependency chain to compile the role instead of the parent role's _parents list
* keep existing role inheritance, but limit it to roles in the current task dependency chain
* Test getting role parents recursively by using the current task dependency chain
* changelog
* download role from api response location
* include changelog fragment
* add unit test for role download url
Co-authored-by: Sam Doran <sdoran@redhat.com>
* ansible-galaxy: Clone git collections using shallow clones
This ensures the collection obtained via git url is a result of a
shallow git clone (git clone --depth=1). The git history of the
collection is not used by ansible, and as such, cloning the entire
history of the repo is unnecessary.
Signed-off-by: Tomas Babej <tomas@tbabej.com>
* ansible-galaxy: Only perform shallow clones on non-versioned git urls
In general, the version can be anything we can checkout (branch, tag,
but also a commit hash). In particular for commit hashes we cannot
perform a shallow clone.
Err on the safe side and only perform shallow clones for non-versioned
git urls (cloning HEAD).
* galaxy-install: Make shallow cloning compatible with older Python versions
Signed-off-by: Tomas Babej <tomas@tbabej.com>
* Pass args as a tuple
Add tests for the git command created from different repo and version combinations
* changelog
Co-authored-by: Tomas Babej <tomas@tbabej.com>
* add tests for fail filter
also tests that fail does not block inspectability
* add fail filter
fallback message is a bit clunky,
since you can't invoke a filter without specifying an input.
That is, "{{ fail }}" doesn't work,
so you have to do "{{ None | fail }}"
* document 'fail' filter
* add changelog fragment
* fail filter uses default message on Undefined or emptystring
makes it slightly easier to use the default message:
```diff
- "{{ None | fail }}"
+ "{{ '' | fail }}"
```
and the user sees a slightly more relevant message
if the message itself is undefined:
```diff
- The error was: {{ failmsg | fail }}: 'failmsg' is undefined
+ The error was: {{ failmsg | fail }}: Mandatory variable has not been overridden
```
* rebuild as the builtin `Undefined`
* harmonise `hint` parameter for make_undefined with jinja
* use code block for documentation item
[ref](https://github.com/ansible/ansible/pull/75435#discussion_r707661035)
* rename to `undef` to expose less Python into the Jinja
[ref](https://github.com/ansible/ansible/pull/75435#pullrequestreview-757799031)
* explicitly instantiate undefined value now that it's possible
see I knew we would break something with reflection
* preserve test coverage of undefined variable
Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
Change:
- Remove only user-facing use of ANSIBLE_ASYNC_DIR.
- Remove two comments saying to change things that, apparently, we
aren't going to change...
Test Plan:
- ci_complete
Tickets:
- Fixes#74139
- Fixes#74138
- Refs #74226
Signed-off-by: Rick Elrod <rick@elrod.me>
* Enable ansible-galaxy to specify client id override with Keycloak Token
* Specify ability to provide override of client_id
* Test client_id can be configured for individual servers
* Add issue link to changelog
* Document client_id as a config option and add an example
Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
* Fix test usage of `which python`.
Also use `command -v` instead of `which` where needed.
* Fix testing_formatter.sh file checking.
The index is longer expected to exist when generating it.
The generation script is run with `set -eux`.
* apt module: add option to allow package downgrades
* Add new option to module so users don't have to force downgrades which
is insecure and dangerous
* Add integration tests similar to upgrade integration tests
* Changelog
* Update changelog fragment
* Update changelogs/fragments/74852-apt-allow-downgrade.yaml
Co-authored-by: Amin Vakil <info@aminvakil.com>
* Update lib/ansible/modules/apt.py
Co-authored-by: Amin Vakil <info@aminvakil.com>
* Update lib/ansible/modules/apt.py
Co-authored-by: Amin Vakil <info@aminvakil.com>
Co-authored-by: Amin Vakil <info@aminvakil.com>
* ensure dump produces json
* clog and tests
* remove library additions since they are not usable
* avoid any/all dirs
* ensure we dont use local dir as playbookdir
* If an exception occurs when getting a collection's metadata, continue to the next in the server list.
* Warn for unknown exceptions when finding versions of a collection
* Test that an invalid server is no longer fatal if a subsequent server has the collection
* Fix server for verify tests - compare checksums against the server from which it was installed
* Add tests for verify and fix that code path to mirror install/download behavior for server errors
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* pip - Use pip from the current Python interpreter.
If `executable` and `virtualenv` were not specified, and
the `pip` Python module is available for the current interpreter,
use that `pip` module instead of searching for a `pip` command.
* Add comment about needing `__main__` to run `pip`.
* Fix unit test.
* Add porting guide entry.
* Update changelog to match porting guide description.
ci_complete
* Make individual dirs that should exist instead of using shutil.copytree
* Test build_ignore when installing collections in git repos
* changelog
* Fix assertion
Fix git directory
* Fix nested content while building the collection dir
Test installing a collection from a git repo and artifact have the same result
Refactor to use variables
* Refs #75478 - On the communicating page, add Matrix links to first 4 chat spaces, improve introduction, update working groups with Matrix and IRC locations, add a link to the community FAQ, add localised community translations, and remove Lightbulb room, the GitHub project has been archived
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
removing out of date example ansible.cfg, show user how to generate from ansible-config
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
* Support templating _from_files with --extra-vars for imported roles
* Add tests for templating the filenames for import_role, import_playbook, and import_tasks
* Add documentation
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
* Test a passlib wrapped algo with a password lookup
* Fix error when passlib is used with a wrapped algo
The exception was:
An unhandled exception occurred while running the lookup plugin 'password'.
Error was a <class 'TypeError'>, original message: issubclass() arg 1 must be a class
and can be reproduced using the following command:
ansible localhost -i localhost, -mdebug -amsg="{{ lookup('password', '/dev/null encrypt=ldap_sha512_crypt') }}"
The concerned algo are: bsd_nthash, django_argon2, django_bcrypt, ldap_bcrypt,
ldap_bsdi_crypt, ldap_des_crypt, ldap_hex_md5, ldap_hex_sha1, ldap_md5_crypt,
ldap_pbkdf2_sha1, ldap_pbkdf2_sha256, ldap_pbkdf2_sha512, ldap_sha1_crypt,
ldap_sha256_crypt, ldap_sha512_crypt, roundup_plaintext
* allow vars_files to use dwim (vars/) and vaults
* also fixed bug with temp file cleanup, some pythons dont like it when you alter loop
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
Rather than adding further complexity to the regex, preprocess the output to
remove any empty lines. Now the only purpose of the regex is to fix
wrapped lines.
Fixes#70949
* avoid getting delegated vars w/o inventory host
fixes#75512
In the case of imports, we don't have a host, so getting host vars for
the delegated host makes no sense and should be avoided.
* also avoid error on vars_files with per host vars
* test
* testing given case
* oops
* add 'file options' for become and connection pass
* implemented getting passwords from file or script
* added config entry
* fixed env var name and noted executable behaviuor
Co-authored-by: Rick Elrod <rick@elrod.me>
##### SUMMARY
When I used this example it complains with:
ERROR! The tasks/main.yml file for role 'myrole' must contain a list of tasks
The error appears to be in 'ansible/roles/myrole/tasks/main.yml': line 2, column 1, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
---
tasks:
^ here
Removing "tasks:" fixes it.
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
* Add new comment attribute to template plugin
Add comment_start_string and comment_end_string attribute to template
plugin
Co-authored-by: Hossein Zolfi <h.zolfi@inside.sahab.ir>
Some job names contain '/' which causes the download job to fail.
Example job name: `Post-job: Checkout ansible/ansible@refs/pull/75445/merge to ansible.log`
* user - Remove unused code.
* Replace deprecated abstractproperty decorator.
* Fix __all__ to be a tuple.
* Use a generator in subelements lookup.
* Use from import in basic.py
* Add changelog fragment.
* Fix selinux unit test.
* ansible-test - Fix use of abstractproperty
* ansible-test - Use dict.items() where possible.
* ansible-test - Remove unused code.
* ansible-test - Cleanup issues reported by pylint.
* ansible-test - Use dict.items() where possible.
* ansible-test - Use generator.
* Improve error for InconsistentCandidate exceptions
* Add test case for installing a collection with an inconsistent version
* Add test case for installing a collection that has a dependency with an inconsistent version
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Sviatoslav Sydorenko <webknjaz@redhat.com>
* Add basic schema for attributes.
* Support attributes for version_added_collection adding mechanism.
* Add changelog fragment.
* Support new attribute properties from #74331.
* Require additional properties strictly.
* Fix a bug with the dnf module not using all components of a package name when filtering to determine if it's installed
* changelog
* Simplify splitting on the last '.'
* Update lib/ansible/modules/dnf.py
Push /usr/bin/python to almost the bottom of the barrel. This makes the strategy to prefer
specific versions of the "mystery meat" version.
* Change INTERPRETER_PYTHON default to 'auto'
Change description to match change in behavior.
Change deprecation message to a warning.
* Update docs
* Add porting guide entry
* Update unit tests
* Update integration test
* Allow INTERPRETER_PYTHON_FALLBACK to be configure using a variable
* Prefer platform-python above other Python 2 interpreters
* Add Python 3.10 to the list of interpreters
* Make undefined variables in handler names non-fatal if the handler is not used
* If the handler has no way to be notified (i.e. the name can't be templated and the handler has no listen topics), display a warning
* Add tests for variables in handler names
* changelog
* Fix when evaluation on Native Jinja and Python 3.10
* Add unit test
* Add explaining comment
* Enable jinja2_native before tests
Co-Authored-By: Matt Martz <matt@sivel.net>
* Sanity
* Return native template module instead of modifying globals
Co-authored-by: Matt Martz <matt@sivel.net>
* Make sure AnsibleModule positional arguments are validated.
* Extract ANSIBLE_MODULE_CONSTURCTOR_ARGS with inspection.
* Remove no longer necessary return value.
* Fix PR #.
* argument_spec might not have been specified, as in community.general.xenserver_facts.
* Fix typo.
test/integration/targets/any_errors_fatal/18602.yml is not run via
runme.sh and is testing the same as
test/integration/targets/any_errors_fatal/always_block.yml which is
actually run. The former file is safe to remove.
* Add packaging to requirement of ansible-test
Fix#75353
After requires_ansible field was added as mandatory to runtime.yml
file, ansible-test fails to check this field if it doesn't have
packaging module.
[1] https://github.com/ansible/galaxy-importer/pull/124
* updates Tower page to talk about RHAAP
* updates scenario guide Tower references
* updates Tower references to use AWX and/or RHAAP
* more scenario guides fixes
* fix CI failures
* Update docs/docsite/rst/porting_guides/porting_guide_2.8.rst
* removes tower link
* for 2.8, still mention Tower along with AWX and RHAAP
* aws guides should be updated in the collection, where they now exist
Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
* Update network user guide to point to ansible.utils.cli_parse module as ansible.netcommon.cli_parse is deprecated from ansible.netcommon 2.0.0 release
* modules moved to use best_parsable_locale
* fixed invocations
* better better
* also module_utils
* converted to function as per fb
* patch testt
* whitespace
* set host_key_checking check to False, rather than if not (which captures False and None)
* add host_key_checking default to ssh.py / update documentation
* Canonicalize module_defaults actions and action_groups pre-fork and cache them on the play
* Call get_action_args_with_defaults with the resolved FQCN plugin and don't pass the redirect list
* Add validation for action_group metadata and a toggle to disable the warnings
* Handle groups recursively referring to each other
* Remove special-casing for non-fqcn actions in module_defaults groups
* Error for actions and groups in module_defaults that can't be resolved
* Error for fully templated module_defaults
* Add integration tests for action_groups
* Changelog
This is useful for testing the effects on docs of changing what
collections are in a version of ansible. ansible-build-data can be
cloned locally, modified, and then build-ansible.py docs-build can be
told to use the local checkout instead of the canonical one on github.
* env_setup - Prefer Python3
Prefer the 'python3' executable over 'python'.
* Add shebang for syntax detection
While the script isn't executable and we recommend sourcing it, having a shebang allows
variuos text editors to detect that it is a shell script and apply proper syntax highlighting.
While implementing basic Ansible module testing I realised that the `self.module` used in some places is not actually a thing, and I changed to `my_module` to make it work as I understand it was expected.
Since the task and connection both have the same 'retries' keyword, the task default
would override the connection value.
Do not pass 'retries' from the task to the connection options.
* Set ssh_connection retries default value back to 0
It was 0 before the move to config and was changed to 3 by accident.
This version relies on an external lib implementing lexers and styles.
That lib pulls in proper Pygments version so this patch also drops it
from direct requirements.
This is a follow-up of previous years' 1a11cec and c8315bf. It deals
with links which at that point presumably either were not present or
did not support https://
* module_common - handle cache directory creation collision
Occasionally multiple workers can try to create the cache directory if it exists. Catch the exception
and handle it.
* Just re-raise if it doesn't exist rather than trying again
* Use exists_ok rather than handling an exception
* Remove unused import and unused variable
* Go back to try/except but with OSError
Since we do not have split controller remote, this needs to be able to run in Python 2 in order
to be merged currently.
Since we moved yum_repository_test_repo.baseurl to file:/// format,
using urlsplit filter on it returned an empty string. Using an empty
string as a left operand of `in` was then always evaluated to True.
This was discovered with native jinja being on where urlsplit returns
None which results in a hard fail as None cannot be a left
operand of `in <string>`.
* The resolved_action is the formatted version of the final plugin in the PluginLoadContext's redirect_list
* Collection plugins are represented as FQCN
* Legacy plugins are represented with only the plugin name
* Add tests
* Changelog
* Fix struture of cli/doc.py snippet code.
A couple releases ago, cli/doc.py was modified to mostly conform to the
data processing pipeline steps. format_plugin_doc() was the biggest
exception in that refactor. When the snippet code was made generic
instead of being only for modules, the new code should have conformed to
the data processing pipeline too.
* Move the decision to output a snippet to the run() method alongside
the decision to output a listing versus plugin_docs.
* Move the test for invalid plugin_types to the run() method as it
affects all snippets in this run, not just a single snippet. (-t can
only be specified once)
* Rename get_snippet_text() to format_snippet() as:
* This is the data formatting step
* The format_snippet() name matches with its conceptual sibling,
format_plugin_doc().
* Use ValueError inside of format_snippet() to flag unrecoverable errors
formatting a single snippet.
* Emit a warning when format_snippet() raises ValueError and continue to
the next snippet.
* If the yaml(?) or toml inventory plugin is specified for snippet output,
raise ValueError() so that the user sees a warning instead of simply
seeing blank output.
* Do not modify arguments passed into format_snippet(). This is the
formatting step so data should not be modified.
* Change _do_yaml_snippet() and _do_lookup_snippet() to operate side
effect free.
* Fix raising of exceptions when formatting requred options for snippets.
* Unrelated: Use to_text() instead of to_native when calling
display.warning(). to_native() is used for raising exceptions. Not
for display methods.
* Add a changelog
specially for when you have parameters in unicode but need
to scrape responses, C is still the fallback
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
* Require antsibull 0.34.0.
* Remove Makefile comment about the devel docs building only the core
(formerly base) docs; the behavior was updated in ccbfdec334
Currently 'make coredocs' builds core-only docs; 'make webdocs' builds the full docs even on the devel
branch
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
* removes upper bound on sphinx version
* updates versions of docs build dependencies, adds known good requirements file
* adds instructions for using known_good_reqs file
Can now specify plugin/plugin type for list and dump actions
New 'init' action to create usable config sources
Co-authored-by: Felix Fontein <felix@fontein.de>
* Change tests to pytest-style tests
* Add tests for invalid hosts
* Validate host inputs
- check for empty values
- check for None
- check for values that are not a sequence and are not strings
- add unit tests
* Move play name setting to get_name() and out of load()
* Add _validate_hosts() method
By defining this method, it gets called automatically by FieldAttributeBase.validate().
* Improve the error handling code
Rather than multiple return paths, have a single return and set the message based
on the type of failure.
* Add another test for non-specific failures
* Reorganize tests so failure tests are in one tasks file
* Remove os.stat() call and add changelog
* Remove use of _remote_checksum from fetch module
* Add deprecation message displayed during runtime
* Increase test coverage for fetch
* Add tests covering the use of stat from the fetch module
This required creating an unpriveleged user account and connecting as that user remotely since
it is not possible to create a file that the root user cannot stat.
* Use fact caching to persist remote tmp dir across playbook runs
* Add variables to setup_remote_tmp test role to allow caching of the remote temp dir fact
and preventing removal of the remote_tmp_dir
* subversion - set LC_ALL for accurate command output parsing
When LC_ALL is not set, the output language of commands will differ based on locale. There
is a lot of history of trying to fix this. See the following pull requests:
https://github.com/ansible/ansible-modules-core/pull/4358https://github.com/ansible/ansible-modules-core/pull/4358
This patch attempts to fix this my setting LC_ALL to a UTF-8 locale. Setting LC_ALL to C reintroduces this bug https://github.com/ansible/ansible-modules-core/issues/4178.
I'm sure there are some problems I am not seeing with setting this to en_US.UTF-8, but that is
the only way I could find to fix this bug without reintriducing the bug mentioned above.
* Rather than setting locale, just check for matches before trying to get groups
This is a pragmatic solution to avoid the stack trace since setting the locale correctly
to ensure message parsing is accurate is problematic.
* Improve regexps for finding revision and URL
Since moving to distro, it is possible to return this information for all platforms, not just Linux.
Also return version information for all platfrom not just Linux.
Update unit tests.
Remove some duplicate unit tests though I think there are more to remove.
* Fix docstring formatting
* Minor docstring changes
* Mock distro.id for Solaris service test
* Update comment
Test for the required binaries in the can_handle_archive() method and fail there. This
prevents failures for missing binaries unrelated to the archive type.
* Update missing zip binary message to match tar message
* Update unit tests
* Add integration tests
* Define packages based on the system rather than ignoring failures
* minor service_mgr facts fixes
handle case in which ps command fails or returns empty
updated tests since it now does keep trying to detect after ps fails
- remove need for module (at least for posix side)
- adds retry with backoff on fetching file, since
race is bigger since we don't spend time on module
- now gives more info on fail
- also made actionfail/skip handle results if given
* rebased with upstream
* removed extra usetty as it wasnt needed, style changes, added var option setable by inventory for pkcs11
* update pkcs11_provider version_added
* Update lib/ansible/plugins/connection/ssh.py
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* Update lib/ansible/plugins/connection/ssh.py
Correct logic for a password being required for pkcs11_provider
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* style nit fixes for checking pkcs11_provider is set
* fixed duplication when using password_prompt with pkcs11_provider
* added changelog fragment
* added changelog fragment
Co-authored-by: David Whiteside <david.whiteside@nrel.gov>
Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
Co-authored-by: Jordan Borean <jborean93@gmail.com>
base._update_security_filters is a private attribute of DNF used
as performance optimization. Modification or even call from outside
of DNF is against all recommendation including PEP8.
* Improve compatibility with all DNF versions
* Add changelog fragment for dnf security change
* Add resiliency to linux hw fact gathering
Now traps unexpected exceptions on mounts and continues
gathiering other info.
Also gives more info on why mount info gathering failed.
more info if debugging
Changed confusing wording. People new to ansible, like myself, have a hard time grocking what is a playbook and what is a task. I changed the name of the playbook to make it clearer.
* Due to the takeover of freenode we're moving to a different irc network.
* Our channels updated to point at the same channel name on libera.chat
* Some links went to webchat.freenode.net. At this time, libera.chat
doesn't point you to an official webchat client so I changed these to
https://libera.chat. (kiwi irc does work with libera.chat so that
could be another option).
* In general, I used the name irc.libera.net for link names and
https://libera.chat for link targets. This is because the irc service
is hosted on irc.libera.chat but the project web server is hosted on
libera.chat. (This appears to also be true for freenode but we were
using http://irc.freenode.net which doesn't seem to work. Oops).
* Removed http://irc.freenode.net from the linkcheck exceptions.
linkcheck was actually correct to flag that as invalid (should have
been http://frenode.net instead).
* Looks like hte important people in #yaml are now in libera.chat
* Link to where contributors should get help
Add a link target and then link to where contributors should get support
for developing groups of modules.
* Update docs/docsite/rst/dev_guide/developing_modules_in_groups.rst
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: John R Barker <john@johnrbarker.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
* Fix module-specific defaults in the gather_facts, package, and service action plugins.
* Handle ansible.legacy actions better in get_action_args_with_defaults
* Add tests for each action plugin
* Changelog
Fixes#72918
Change:
- Make strategies behave consistently and return the empty string
instead of "UNKNOWN" (or "temporarystub") for the "before" value if
the permanent hostname file does not exist or could not be read.
- Switch to `with open()` instead of annoying exception handling code
(which was wrong and leaked file handles in several places). This
drops Python 2.4 support for this module.
- Updated porting guide since users could be relying on these former,
inconsistent values.
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Hostname strategies' get_*() methods should never write to the
filesystem. They are used in check_mode by default to determine if
there is any work to be done.
Test Plan:
- New unit tests to ensure that (at least when in check_mode) the get
methods don't ever call write.
Tickets:
- Fixes#66432
Signed-off-by: Rick Elrod <rick@elrod.me>
- AWS has moved to Zuul CI
- Kubernetes collection has been renamed
- OpenShift/OKD collection was missing
- ansible.windows is on Zuul and Azure Pipelines
- ansible.posix is on Zuul
Signed-off-by: Jill Rouleau <jill.rouleau@bespokess.com>
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
It is not clear that you can publish without configuring an ansible.cfg. The `--token` can be displayed with the `ansible-galaxy collection publish -h`
* Don't mutate os.environ in AnsibleModule.run_command, make a copy, and pass to Popen. Fixes#74783
* Simplify code a bit
* More simple
* Address some other potentially non threadsafe operations
* Add if around umask
* Address unit test assumptions
* Add clog frag
* yaml syntax issue
service module acts as proxy module for all service manager modules,
just like `package` module.
This is helpful in heterogeneous environments to manage services.
Fixes: #74507
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* docs: Update Python 2 doc links
Update links from Python 2 to Python 3
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* use docs.python.com/3/ everywhere, except onethat should remain 2.6
* refer to python 3 in module docs and comments
* format two python docs links as list
* updates links in unwanted.py test file
* per matt clay, this should link to python 2
Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
When an empty value is provided, no `version` attribute will exist on the `LooseVersion` or
`StrictVersion` object. We catch and handle this, but it's not immediatebly clear that an
AttributeError means an empty value was provided.
Specifically handle the case where value or version are empty and add more
helpful error messages.
Add integration tests.
* Remove PyCrypto from setup.py and packaging script
* Remove mention of pycrpto from installation docs
* Remove PyCrypto from vault
* Remove pycryto constraint and unit test requirement
* Remove PyCrypto tests from unit tests
* Add docs and fix warning message
* Remove section about cryptography library in Ansible Vault docs
The URLs to the Jinja website have moved, seemingly because of the renaming from `master` to `main` (hence from `master` to `latest` in the doc). This commit fixes that.
This should allow the same script to be used for two different scenarios:
- ansible-core across multiple versions (not all of which support `--docker` for coverage reports)
- collections (which should always use the `devel` branch for coverage reports)
* For long-running playbooks, discourage users from using
`ansible_date_time`. Use `pipe` or `now` as an alternative.
* Review comments
Fixes: #22561
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
* Only remove crontabs if they are empty
* Add integration test to ensure system cron tab doesn't get removed. Increase cron integration tests separation.
* Also detect crontab which only contains whitespace as empty.
* cron integration test: Adjust system crontab path to be distribution specific.
Co-authored-by: Fabian Klemp <fabian.klemp@elara-gmbh.de>
* Update to six 1.16.0
* Address linting issues
* Remove six find_spex/exec_module warning filters
* Remove unnecessary comment about Py2.6, 2.13 will not support Py2.6, and we're bumping this for 2.12
* ci_complete
* Add changelog fragment
* Add constraint for MarkupSafe
MarkupSafe >= 2.0.0 requires Python >= 3.6.0. Add a constraint for older Python versions
and fix the `groupby_filter` test.
* Fix template_jinja2_latest test.
* patch filter decorators on newer Jinja2
* Jinja2 >= 3.0 renames several filter decorators used by Ansible itself, as well as by filters in collections. This patch ensures that the old names are usable within Ansible and by collections without warnings or errors.
* Ignore docs-build issues.
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Davis <mrd@redhat.com>
While logging, journal.send accepts module parameters.
If module parameters similar to arguments in journal.send,
rename the parameter names before sending to journal.send
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* fix: action _fixup_perms2 macos +a remote_paths in list() as it can be tuple
in `lib/ansible/plugin/action/__init__.py`'s `_fixup_perms2`,
`remote_paths` can be a list or tuple. however, the macos
specific attempt to use chmod +a attempts to concatenate
`remote_paths` with a list, which will fail if it is a tuple.
wrapping `remote_paths` in `list()` fixes this error.
* Update changelogs/fragments/74613-actionfixup_perms2_macos_remote_paths_ensure_list.yml
Co-authored-by: Rick Elrod <rick@elrod.me>
* support separate role argspec file in ansible-doc
* support separate role argspec file in ansible-core
* support both .yml and .yaml extensions on argspec file in ansible-doc
* fix filename building bug and rename some argspec files to test variations
* use yaml extensions from constants
* add superfluous meta/main.yml files to tests
* Update lib/ansible/cli/doc.py
Co-authored-by: Sam Doran <sdoran@redhat.com>
* update docs
* ci_complete
* add changelog and allow for main.yml variations
* add collection role testing
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Update Python versions for controller-only tests.
Both `ansible-galaxy` and `ansible-test` integration tests only run on the controller.
* Use a venv to make requirements available.
* Get available collection versions with page_size=100 for v2 and limit=100 for v3
* Update unit tests for larger page sizes
* Add a generic retry decorator in module_utils/api.py that accepts an Iterable of delays and a callable to determine if an exception inheriting from Exception should be retried
* Use the new decorator to handle Galaxy API rate limiting
* Add unit tests for new retry decorator
* Preserve the decorated function's metadata with functools.wraps
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Always use create_homedir when we are asked to create a home directory
in the User class. Don't use the -m and -k parameters from
useradd / luseradd as they behave differently with respect to
preexisting home directories. Instead always specify -M to ensure
that useradd / luseradd do not try to create the home directory.
This does not change potential different behaviours in child classes
of the User class.
Consider the new umask option from #73821 in create_homedir as well as
we do not let luseradd / useradd create the home directory any longer.
From the sha512sum man page:
... The default mode is to print a line with checksum, a character indicating type ('*' for binary, ' ' for text), and name for each FILE.
The distutils module is not shipped with SUNWPython on Solaris.
It's in the SUNWPython-devel package. Do not use LooseVersion.
Fixes: #74488
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* removal of deprecations and minor fix
name or cron_file is required.
reboot was removed.
updated docs to clarify usage.
now using cron_file=/etc/crontab is an error
fixes#37355
* warn show pathing issues when wallking through provided paths
moved issues from msg to actual warnings AND a specific return field
fixes#25314
Co-authored-by: Sam Doran <sdoran@redhat.com>
* pause - ensure control characters are always set appropriately
On some systems, curses.tigetstr() returns None, which does not work as a control character.
* Add unit tests
* Sort imports
* Skip on older Python
This is an action plugin and only runs on the controller, so no need to test of Python 2. Making
the import hackery work on Python 2 would required some more work which I am not sure is
worth it since we are moving away from Python 2 support on the controller.
* Make the tests work on Python 2 and 3
References to Shippable were changed to Azure Pipelines.
Also remove rebalance.py as it does not work with Azure Pipelines due to the required data not being present.
* Fix section for inventory cache prefix
* Redirect general config inventory cache options to inventory plugin documentation
* Update the template to fix the docs-build error
* removes 'default' entries for config settings with default: ''
* single backticks => italics, double backticks => code
* use 'Default: ~' to match other entries that have no default setting
Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
Change:
- After we output the newline in help text, we also added a space. This
caused either a space to show up before the shell prompt (in shells
like bash), or " %" and a newline before the next prompt on shells
like zsh that automatically force commands to end with a newline.
- This change removes the extra space, so that peace can be restored in
the minds of many.
- This only showed up when running commands with no arguments or invalid
arguments.
Test Plan:
- Ran `ansible`, `ansible -h`, `ansible-playbook`, `ansible-galaxy`, and
`ansible-galaxy -h`.
Tickets:
- Introduced in #69458
Signed-off-by: Rick Elrod <rick@elrod.me>
* dont specify default for port, allow ssh/config
also added general note on how defaults work.
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Sam Doran <sdoran@redhat.com>
For unknown reasons, these tests started failing recently because the results
from testhost11 are being returned before testhost10, but only when coverage is
enabled.
Setting the initial host sleep time to 0 fixes this failure.
ci_coverage
* Fix get_available_python_versions calls.
* Make run_playbook vars optional.
* Use ansible_pipelining in inventory.
* Fix type hint.
* Fix order of conditional evaluation.
* Remove unused ibmi platform.
* Add changelog fragment.
Change:
- Allow overriding python version as env var for Makefile
- Including hacking directory in git export, for docs build
Test Plan:
- Tested as part of recent downstream work
Co-authored-by: Yanis Guenane <yguenane@gmail.com>
* Fix BaseFileCacheModule#keys to respect prefix
Change:
- Previously BaseFileCacheModule#keys would return keys with the cache
prefix. These keys are impossible to retrieve from the cache without
removing the prefix or using the cache without a prefix.
Now it removes the prefix from the key and only returns keys that
share the same prefix as the cache.
Test Plan:
- New unit tests
* Add explicit BaseFileCacheModule#keys test
Test that files that do not match the cache prefix are ignored.
Test that the prefix is removed from the cache key.
Change:
- Added Fedora 34 container image to docker.txt
- Dropped Fedora 32 from CI
- Added Fedora 34 to CI
- Updated Fedora 32 and 33 containers for ssh-related fixes
- Move cron fix from 698eae3f3d into
cron_setup and make it more generic; it can affect modern distros too.
Test Plan:
- ci_complete
Signed-off-by: Rick Elrod <rick@elrod.me>
Due to the recent security incident, use our own copy hosted in S3
to mitigate future risk from running an arbitrary script downloaded from a
remote and untrtusted server.
* dont rely on vars, task already gives us info
* ensure we always display delegation in host label
* also added parens with ansible_host to show target host vs resolved host
* delegating to self is not delegating
* delegated vars restoration for backwards compat
* tests need mock task with delegate_to
* correctly parse device from string
* check for command presence before running them
* check for command presence and return code for solaris and aix as well
* add changelog
Improve the documentation for regex_search by adding a few more
realistic examples such as showing how to extract only parts of a
string by using regex groups.
Extend regex_findall and regex_replace by an example that includes
the multiline and ignorecase parameters.
Previously it was hosted on bintray, but that service is shutting down on May 1. Using the new
repository also required using a newer version of RabbitMQ.
Change:
- Instead of returning the `str` type, return the value that was
calculated.
Test Plan:
- New unit tests.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Fix world readable setting
now uses shell option in all cases, the old constant still exists
as 'documentation' but it not settable.
also fix the docsite link in warnings/errors
* fix typose#
* fix comment#
* added deprecated to config
* missing :
* import cleanup
* Update default containers to 3.4.0.
The 3.4.0 containers use Python 3.6 (the system Python) for `/usr/bin/python3`.
Python 3.9 continues to be the default Python version selected by `ansible-test` for these containers.
* Fix shebang on build-ansible.py.
Using `python` instead of `python3` allows `ansible-test` python interception and requirements install to function.
Fixes#74255
* Fix call to 'unique(case_sensitive=False)' triggering error when falling back to Ansible's version which **is** case-sensitive
* Test multiple situations of 'unique' filter errors with fallback not handling specific parameters
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Rick Elrod <rick@elrod.me>
Changed FAQ examples to conform with the Jinja documentation:
If both values on either side of a plus/+ are numbers, they will be added whereas using "~" will convert all operands into strings and then concatenate them. Closes#73799.
Homebrew's default install location for macOS on ARM is /opt/homebrew.
Source: https://docs.brew.sh/FAQ
On a Mac M1 (Apple Silicon), homebrew will be installed at
/opt/homebrew/bin/brew.
The unarchive integration test installs pinned packages via pip,
the uri test installs them from system packages. This explodes.
PR lets unarchive uninstall packages after test ran.
* Add yaml utils file and use it
* Linting fix and missing import
* Abstract a few more details
* Parser imports
* Don't use CSafeDumper in AnsibleDumper
* Move and use convert_yaml_objects_to_native when libyaml is present
* yaml_load and yaml_dump, instead of safe_*
* re-use HAS_LIBYAML from utils.yaml
* add changelog fragment
* Address recent changes
* Use representer instead of recursive type converter
* Restore needed import
* move yaml utils to module_utils
* Properly guard imports
* Update from_yaml(_all)? to handle text wrappers with CSafeLoader
* Use yaml utils for legacy_collection_loader
* Add HAS_YAML, and ignore pylint issue
* oops
* GPL->BSD
* Map Debian 8 to Python 2
If Python 3 is installed on Debian 8 Ansible cannot run, as the version
is too old (3.4)
* Add integration test for python interpreter discovery on Debian 8
* fix test issue on Debian 9, add changelog
* un"fix" not broken test :D
Co-authored-by: Fabian Klemp <fabian.klemp@elara-gmbh.de>
Co-authored-by: Matt Davis <mrd@redhat.com>
* Add ArgumentSpecValidator to docs
* Improve docs for ArgumentSpecValidator
* Document removal of private methods
* Update module_utils documentation
- Add docs for argument spec classes as well as validation and parameters files.
- preserve the order in the source for errors.py
- document DEFAULT_TYPE_VALIDATORS so it can be referenced elsewhere
- fix automodule directive for validation.py
* Update docs in arg_spec and paremeters
- This improves the generated documentation.
* Document breaking changes in porting guide.
* Update formatting in porting guide and add a Deprecated section
* Fine tune module_utils documentation
* Move instance docstring to the __init__ method
Remove optional description since it fails the sanity test and I am not 100% it is valid anyway.
* Remoe incorrect parameter from docstring
This was changed a while ago but wasn't removed from the docstring.
* Use attr rather than attribute
The py:attribute: domain only exists in newer Sphinx >= 3.1.
* Improve documentation on exceptions
* Final pass
- use args/kwargs instead of param
- fix formatting errors that didn't display examples correctly
- format TypeErrors so they are referenced as classes
- specify complex types
Document that users must pass `allow_unsafe=True` as an option in the lookup to allow templating, with a note about security implications.
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
Co-authored-by: Sandra McCann <samccann@redhat.com>
* Add umask option to user module
* Fail on setting both umask and local: True
* Add integration test
* Add changelog
* Run integration tests only if HOME_MODE is not set
* Run integration tests only on Linux
Co-authored-by: Matt Clay <matt@mystile.com>
* fix module schema
not out of date hardcoded list anymore, uses 'current + 4' to
set valid deprecation targets
Co-authored-by: Matt Clay <matt@mystile.com>
Change:
- This works around an issue that causes the cron test to fail sometimes
on el6.
Test Plan:
- ansible-test integration cron --docker centos6
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Newer Solaris drops setfacl. Add a fallback for its chmod ACL syntax.
Test Plan:
- New units
Tickets:
- Fixes#74282
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Regression introduced in #70785
- When macOS chmod ACL syntax is used, Solaris-derived chmods return
with a status of 5. This is also used for our sshpass handling,
because sshpass will return 5 on auth failure. This means on Solaris,
we incorrectly assume auth failure when we reach this branch of logic
and try to run chmod with macOS syntax.
- We now wrap this specific use of chmod in an exception handler that
looks for AnsibleAuthenticationFailure and skips over it. This adds
another authentication attempt (something we normally avoid to prevent
account lockout), but seems better than the regression of not allowing
other fallbacks to be used.
- Without this patch, if setfacl fails on Solaris (and sshpass is used),
we do not try common_remote_group or world-readable tmpdir fallbacks.
Test Plan:
- New unit
Signed-off-by: Rick Elrod <rick@elrod.me>
- Unit tests for `modules` and `module_utils` are now limited to importing only `ansible.module_utils` from the `ansible` module.
- Unit tests other than `modules` and `module_utils` are now run only on Python versions supported by the controller (Python 3.8+).
- Unit tests are now run in separate contexts (`controller`, `modules`, `module_utils`), each using separate invocations of `pytest`.
* Handle netconf plugin ncclient import error when running in FIPS mode
* While running in FIPS mode importing ncclient result in
InternalError raised by cryptography
* Refer https://github.com/ansible/ansible/pull/65477
- Improve code reuse.
- Add missing type hints, fix existing ones and convert them to PEP 484 style.
- Add missing imports and clean up existing ones.
- Add missing docstrings and clean up existing ones.
This simplifies rendering the hostname (or hostname+delegated host) in
the default callback module, and reduces code duplication
I've chosen not move where in each handler the host label is rendered,
in case subsequent operations has side effects. However I'm happy to
change that if considered safe.
I've chosen not to change the formatting operator used (%), to avoid
changes in rendering that might result.
Signed-off-by: Alex Willmer <alex@moreati.org.uk>
Change:
- We were passing a directory name directly to re.compile().
If the directory isn't valid regex (or is) this can have odd side
effects, such as crashing.
- Fix a few other similar cases, but less likely to be a real issue.
Test Plan:
- New test
Signed-off-by: Rick Elrod <rick@elrod.me>
* Catch errors getting filters, and fail
* Add changelog
* Switch to warnings instead of errors, to match other plugin loader behavior
* Add tests
* Handle collections
* deprecated include
Update lib/ansible/modules/_include.py
updated version numbers in schema check (real fix in separate PR)
Co-authored-by: flowerysong <junk+github@flowerysong.com>
Co-authored-by: Matt Clay <matt@mystile.com>
* all lookups to support config system
- added get_options to get full dict with all opts
- fixed tests to match new error messages
- kept inline string k=v parsing methods for backwards compat
- placeholder depredation for inline string k=v parsing
- updated tests and examples to also show new way
- refactored and added comments to most custom k=v parsing
- added missing docs for template_vars to template
- normalized error messages and exception types
- fixed constants default
- better details value errors
Co-authored-by: Felix Fontein <felix@fontein.de>
* various fixes to command
- Updated splitter to allow for all expected args in ad-hoc
- Ensure we always return the returns we promissed to always return (i.e stderr/stdout)
- Updated docs to clarify creates/removes precdence in checking
- Removed abspath from chdir to allow reporting to handle symlinks correctly
- Corrected tests to new output messages
Co-authored-by: Felix Fontein <felix@fontein.de>
* Support omitting the trailing separator when a dictionary key's value is an empty string
* Support a default value when the value used in the group name is an empty string
* Add tests
* change log
Change:
- When a "distro" package exists in PYTHONPATH but isn't what we expect,
fall back to our own vendored one and use it. This prevents a
traceback if someone has some random thing that provides "distro" but
isn't actually the "distro" library we need.
Test Plan:
- new tests
Tickets:
- Fixes#74228
Signed-off-by: Rick Elrod <rick@elrod.me>
* nuke playbook test file
Signed-off-by: Rick Elrod <rick@elrod.me>
* test fixes
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Tests for advanced_host_list; it is now at 100% coverage.
- There was a typo (host vs hostnames) where when the host pattern
failed to parse, instead of treating the name as a literal as
intended, it would trigger UnboundLocalError. This didn't fatal
as there's a global "Exception" handler below, but it did lead to
an ugly error and incorrect behavior.
Test Plan:
- New tests
- Local experimenting
Signed-off-by: Rick Elrod <rick@elrod.me>
* changelog
Signed-off-by: Rick Elrod <rick@elrod.me>
* fix tests
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Mostly increase error coverage for various conditions in play.py
- Also fix a string in an error, where get_name() was called before
self.name was read in, so get_name() was always ''.
Test Plan:
- new tests
Signed-off-by: Rick Elrod <rick@elrod.me>
* Fix regex for py2 and py3
Signed-off-by: Rick Elrod <rick@elrod.me>
* py2 hates me
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Variables used in ``when`` conditionals are no longer parsed and
attempted to be converted to booleans. All non-empty strings are
considered true (empty strings, false).
Test Plan:
- Updated existing tests
- Added a bunch of new tests with various kinds of truthy/falsy
values.
Tickets:
- Fixes#74134
Signed-off-by: Rick Elrod <rick@elrod.me>
use load_name of valid plugin to find config, will handle the renamed loaded plugin that collections do:
`ansible_collection.<ns>.<collname>.plugins.<type>.<name>`
Change:
- Previously when the same package name was installed twice under
different architectures, we only reported it once in changes.updated.
- This was the result of using a dict internally and keying on package
name alone.
- This change still keys on package name but turns the values into lists
which can contain multiple packages per name.
Test Plan:
- Added a lot of tests around multi-arch support
- Added some tests around virtual provides as well
Tickets:
- Fixes#73284
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Remove two deprecated features
- We now error if a playbook is an empty list instead of just skipping
- We now error if using 'include' instead of 'import_playbook'
Test Plan:
- Added new tests for new errors
Tickets:
- Fixes#74133
Signed-off-by: Rick Elrod <rick@elrod.me>
* sanity & changelog
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Require name always
- Drop 'reboot' parameter in favor of 'special_time: reboot'
Test Plan:
- CI
Tickets:
- fixes#74132
Signed-off-by: Rick Elrod <rick@elrod.me>
The `a/` and `b/` prefixes can be disabled in the `git diff` output by setting
`diff.noprefix` to `true`. The output is still a valid diff, but `ansible-test` would
raise an exception since without the prefixes, it thought the diff line was invalid.
* --offline allows in-place verify for installed collections with manifests
* manifest hash, collection name, version, and path are now always displayed
* test updates
##### SUMMARY
<!--- Your description here -->
On the main freebsd change, and no the version: py37-ansible
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
* Added prerequisites to install for CentOS as well so CentOS users don't have to look for alternatives for each Ubuntu package.
Signed-off-by: mm667937 <marketa.muzikova@broadcom.com>
Signed-off-by: mm667937 <marketa.muzikova@broadcom.com>
The GPG key in getfedore.org/static/fedora.gpg changed and caused the test to fail. Update
to using the new key ID and save the GPG file in our S3 to prevent spontaneous changes/breakage.
* Begin using ArgumentSpecValidator in AnsibleModule
* Add check parameters to ArgumentSpecValidator
Add additional parameters for specifying required and mutually exclusive parameters.
Add code to the .validate() method that runs these additional checks.
* Make errors related to unsupported parameters match existing behavior
Update the punctuation in the message slightly to make it more readable.
Add a property to ArgumentSpecValidator to hold valid parameter names.
* Set default values after performining checks
* FIx sanity test failure
* Use correct parameters when checking sub options
* Use a dict when iterating over check functions
Referencing by key names makes things a bit more readable IMO.
* Fix bug in comparison for sub options evaluation
* Add options_context to check functions
This allows the parent parameter to be added the the error message if a validation
error occurs in a sub option.
* Fix bug in apply_defaults behavior of sub spec validation
* Accept options_conext in get_unsupported_parameters()
If options_context is supplied, a tuple of parent key names of unsupported parameter will be
created. This allows the full "path" to the unsupported parameter to be reported.
* Build path to the unsupported parameter for error messages.
* Remove unused import
* Update recursive finder test
* Skip if running in check mode
This was done in the _check_arguments() method. That was moved to a function that has no
way of calling fail_json(), so it must be done outside of validation.
This is a silght change in behavior, but I believe the correct one.
Previously, only unsupported parameters would cause a failure. All other checks would not be executed
if the modlue did not support check mode. This would hide validation failures in check mode.
* The great purge
Remove all methods related to argument spec validation from AnsibleModule
* Keep _name and kind in the caller and out of the validator
This seems a bit awkward since this means the caller could end up with {name} and {kind} in
the error message if they don't run the messages through the .format() method
with name and kind parameters.
* Double moustaches work
I wasn't sure if they get stripped or not. Looks like they do. Neat trick.
* Add changelog
* Update unsupported parameter test
The error message changed to include name and kind.
* Remove unused import
* Add better documentation for ArgumentSpecValidator class
* Fix example
* Few more docs fixes
* Mark required and mutually exclusive attributes as private
* Mark validate functions as private
* Reorganize functions in validation.py
* Remove unused imports in basic.py related to argument spec validation
* Create errors is module_utils
We have errors in lib/ansible/errors/ but those cannot be used by modules.
* Update recursive finder test
* Move errors to file rather than __init__.py
* Change ArgumentSpecValidator.validate() interface
Raise AnsibleValidationErrorMultiple on validation error which contains all AnsibleValidationError
exceptions for validation failures.
Return the validated parameters if validation is successful rather than True/False.
Update docs and tests.
* Get attribute in loop so that the attribute name can also be used as a parameter
* Shorten line
* Update calling code in AnsibleModule for new validator interface
* Update calling code in validate_argument_spec based in new validation interface
* Base custom exception class off of Exception
* Call the __init__ method of the base Exception class to populate args
* Ensure no_log values are always updated
* Make custom exceptions more hierarchical
This redefines AnsibleError from lib/ansible/errors with a different signature since that cannot
be used by modules. This may be a bad idea. Maybe lib/ansible/errors should be moved to
module_utils, or AnsibleError defined in this commit should use the same signature as the original.
* Just go back to basing off Exception
* Return ValidationResult object on successful validation
Create a ValidationResult class.
Return a ValidationResult from ArgumentSpecValidator.validate() when validation is successful.
Update class and method docs.
Update unit tests based on interface change.
* Make it easier to get error objects from AnsibleValidationResultMultiple
This makes the interface cleaner when getting individual error objects contained in a single
AnsibleValidationResultMultiple instance.
* Define custom exception for each type of validation failure
These errors indicate where a validation error occured. Currently they are empty but could
contain specific data for each exception type in the future.
* Update tests based on (yet another) interface change
* Mark several more functions as private
These are all doing rather "internal" things. The ArgumentSpecValidator class is the preferred
public interface.
* Move warnings and deprecations to result object
Rather than calling deprecate() and warn() directly, store them on the result object so the
caller can decide what to do with them.
* Use subclass for module arg spec validation
The subclass uses global warning and deprecations feature
* Fix up docs
* Remove legal_inputs munging from _handle_aliases()
This is done in AnsibleModule by the _set_internal_properties() method. It only makes sense
to do that for an AnsibleModule instance (it should update the parameters before performing
validation) and shouldn't be done by the validator.
Create a private function just for getting legal inputs since that is done in a couple of places.
It may make sense store that on the ValidationResult object.
* Increase test coverage
* Remove unnecessary conditional
ci_complete
* Mark warnings and deprecations as private in the ValidationResult
They can be made public once we come up with a way to make them more generally useful,
probably by creating cusom objects to store the data in more structure way.
* Mark valid_parameter_names as private and populate it during initialization
* Use a global for storing the list of additonal checks to perform
This list is used by the main validate method as well as the sub spec validation.
When using "use_regex: yes" and setting an excludes: without
specifying a pattern: the existing code passes the file-glob '*' to
the regex matcher. This results in an internal invalid-regex
exception being thrown.
This maintains the old semantics of a default match-all for pattern:
but switches the default to '.*' when use_regex is specified.
The code made sense as-is before excludes: was added (2.5). In that
case, it made no sense to set use_regex but *not* set a pattern.
However, with excludes: it now makes sense to only want to exclude a
given regex but not specify a specific matching pattern.
Closes: #50067
* moved change to new location
added changelog
* Update lib/ansible/modules/find.py
Co-authored-by: Ian Wienand <iwienand@redhat.com>
* Fix a bug adding unrelated candidates to the plugin loader redirect_list
* Add tests for the redirect list
* test redirect list for builtin module
* test redirect list for redirected builtin module
* test redirect list for collection module
* test redirect list for redirected collection module
* test redirect list for legacy module
* changelog
Ansible can gather distribution facts for older Amazon Linux
with /etc/os-release data.
Fixes: #73946
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Update developing_inventory.rst
hashes/dictionaries in JSON (JavaScript Object Notation) are represented by what JSON calls "objects", so use that established terminology instead of "hash", "dictionary", or "hash/dictionary" etc.
* Update docs/docsite/rst/dev_guide/developing_inventory.rst
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* ansible-pull: run all playbooks when multiple are supplied
* add test for ansible-pull with multiple playbooks supplied from cli
* add changelog fragment
This helps users to find the right ressources more easily and prevents them from clicking the first matching link only to be redirected to a subsection on the samepage from there
Co-authored-by: Jens Heinrich <github.com/JensHeinrich>
* Catch more potential errors (and increase false-positive rate).
* Flag some false-positives in lib/ansible/modules/ with no_log=False.
Co-authored-by: John Barker <john@johnrbarker.com>
* add optional module_utils import support
Treat core and collections module_utils imports nested within any Python block statement (eg, `try`, `if`) as optional. This allows Ansible modules to implement runtime fallback behavior for missing module_utils (eg from a newer version of ansible-core), where previously, the module payload builder would always fail when unable to locate a module_util (regardless of any runtime behavior the module may implement).
* sanity test fixes
ci_complete
* Documentation: Lay the ground for i18n work.
* Translating both ansible-core and ansible docs requires msgcat
(a program from the gettext package). If msgcat is not present,
only the ansible-core index file's strings will be extracted for translation.
If this is a problem, a short hacking script could be written to take the
place of msgcat (merging the various index pot files into a single pot file).
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
* finish migrating ssh plugin to config system
fixes#72739fixes#57220
* fix connection detection in reset
* correct options for connection meta reset
Co-authored-by: David Shrewsbury <Shrews@users.noreply.github.com>
* Convert markdown issue templates into issue forms
* Limit allowed issues to the form based ones
* Title-case the form field labels
Co-authored-by: John R Barker <john@johnrbarker.com>
* Remove CoC checkboxes from the forms
Co-authored-by: John R Barker <john@johnrbarker.com>
The file module changes existing sym links from relative to absolute
if the src is not stated in the tasks since it uses `os.path.realpath`
to fetch the link source and not `os.readlink`. Changed that.
* Add more scenarios to basic valid testing
* Update invalid tests
* Fix test for Python 2
* Condense data
* Add tests for missing required and invalid-elements
* Update aliases tests
* Add invalid scenarios for aliases
* Add tests for _add_error() method
* Fix sanity test failure
* galaxy: restore left hand slicing in assignment
Fix 'ansible-galaxy role init --role-skeleton=role-skeleton' when the role skeleton
contains an ignored directory.
The issue was because the 'dirs' variable was changed to reference a different list,
but needs to be mutated instead to stop os.walk from traversing ignored directories.
Fixes: #71977
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Change:
- Fix regression: unhandled exception when given inventory directory
is empty or contains empty subdirectories.
- Fix unhandled exception when limit file is actually a directory
instead of a file.
- Fix inventory tests which previously could never fail due to missing
`set -e`. Fixed up tests that failed after `set -e` was added. Added
several tests.
Test Plan:
- New tests
- Fixed existing tests which previously could never fail
Tickets:
- Fixes#73658
Signed-off-by: Rick Elrod <rick@elrod.me>
Adds in the caveat that Python's string modules have different constants depending on the major version (2.x vs 3.x), but preserves the fact that a fair amount of them are common. Other minor docs touchups also included.
* excludes scenario guides from core docs, splits porting guides and roadmaps, symlinks indices to create index.html pages, and adds .gitignore entries for conf.py and the toplevel index.rst files generated by the docs build
This solution builds three types of docs:
* ansible-2.10 and earlier: all the docs. Handle this via `make webdocs
ANSIBLE_VERSION=2.10`
* ansible-3 and later: a subset of the docs for the ansible package.
Handle this via `make webdocs ANSIBLE_VERSION=3` (change the
ANSIBLE_VERSION to match the version being built for.
* ansible-core: a subset of the docs for the ansible-core package.
Handle this via `make coredocs`.
* `make webdocs` now always builds all the collection docs
* Use `make coredocs` to limit it to core plugins only
* The user specifies the desired version. If no ANSIBLE_VERSION is specified, build plugins for the latest release of ansible
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
Co-authored-by: Matt Clay <matt@mystile.com>
* Manage the in-memory cache in _call_galaxy but let the caller set the file cache after getting paginated results
* Add a test for caching successful and not caching unsuccessful paginated results
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Initial import of modified version of alikins' code
* Add unit testing for new Role methods
* Fix validate_arg_spec module for sanity test. Add test_include_role_fails.yml integration test from orig PR.
* Add testing of suboptions
* Use new ArgumentSpecValidator class instead of AnsibleModule
* fix for roles with no tasks, use FQ name of new plugin
* Add role dep warning
Allow constructed to optionally use vars plugin data
* mostly for those looking to leverage group_vars/ and host_vars/
* limited to already processed sources
Ensure `yamllint`'s `check_assignment()` correctly ignore the
attribute assignment. Those don't have any `.id` attribute and will
trigger an `AttributeError` exception.
See: https://github.com/ansible/ansible/pull/73322
Only add data when there is data to add
also avoid clobbering existing data with empty file
fixes#45843
* remove redundant code, update comments
* fix mock dataloader, original does not return None
* added test
* module compat for py3.8+ controller
* replaced internal usages of selinux bindings with internal ctypes binding (allows basic selinux operations from any Python interpreter), plus tests
* added new respawn_module API to allow modules to import Python packages that are only available under a well-known interpreter, plus tests
* added respawn logic to modules that need Python libs from a specific system interpreter (apt, apt_repository, dnf, yum)
minimize internal HAVE_SELINUX usage
spurious junk
pep8
* pylint fixes
* add RHEL8 Python 3.8 testing
* more pylint
* import sanity
* unit tests
* changelog update
* fix a bunch of stuff
* tweak changelog
* fix setup_rpm_repo on EL8
* misc sanity/test fixes
* misc feedback tweaks
* fix import fallback in test module
* fix selinux MU test
* fix dnf tests to avoid python-dependent test packages
* add trailing LFs to aliases
* fix yum tests to avoid test package with Python deps
* hack create_repo for EL6 to create noarch package
* Upgrade pylint and deps in ansible-test.
* Enable pylint on Python 3.9.
* Update pylint config.
* Add ignore for vendored six.
* Add ignores for support plugins.
* Fix issue reported by pylint.
When running in verbosity <2, display the file and line number for tasks that fail. This provides
useful information without having to run at increased verbosity.
* Move _print_task_path to CallbackBase class
* Add integration tests
* Add color parameter to _print_task_path()
* Keep color output consistent for now
Currently the path is display with COLOR_DEBUG formatting with verbosity >= 2.
Instead of the color of the path changing based on verbosity level, just keep it at the
currently behavior of COLOR_DEBUG. Having the color of the same information change
based on verbosity level seems incorrect and makes visual parsing of the information
more difficult.
Co-authored-by: tahar.jegham <jeghamseifeddine@gmail.com>
* Docker content from community.general migrated to community.docker.
* Hetzner Robot content from community.general migrated to community.hrobot.
* RouterOS content from community.network migrated to community.routeros.
* PostgreSQL content from community.general migrated to community.postgresql.
* OC content from community.general migrated to community.okd.
* Cisco NSO content from community.network migrated to cisco.nso.
* Add changelog fragment.
* Google content from community.general migrated to community.google.
* Hashi Vault content from community.general migrated to community.hashi_vault.
* KubeVirt content from community.general migrated to community.kubevirt.
* FortiOS content from community.network migrated to community.fortios.
* Update changelog fragment.
* DellEMC content from community.general migrated to dellemc.openmanage.
* treat oracle same as rhel/centos
* fix misunderstood use of yaml anchors
* add changelog fragment
* Update changelogs/fragments/73498-INTERPRETER_PYTHON_DISTRO_MAP-Treat-oracle-same-as-rhel-centos.yml
Change:
- Other targets might remove rpm-build as they clean up after
themselves. Ensure that it's present in setup_rpm_repo because
rpmfluff needs it.
Test Plan:
- Local experimentation with yum_repository and mysql_db (the latter of
which depends on a handler which was removing rpm-build) on
stable-2.9.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Include all collections in single json object / yaml document
* Add tests
* For galaxy list yaml/json output, use dictionary of dictionaries instead of list
* Add tests for listing single collection in yaml / output format
* --output -> --format
* Add explicit test for listing collection in human format
* Fix bug where empty json object was emitted + add test
* add changelog fragment for #71979 (ca_path for uri)
* add integration tests for ca_path in the uri module
* return path of ca cert instead of its content
* connect to port 444 on self_signed_host
and use quay.io/ansible/http-test-container:1.3.0
* state that the certificate in ca_path is used for validation
* Fix regression introduced in b77abd0491 causing bug in inventory modules which break functionality in user setting use_contrib_script_compatible_sanitization parameter.
* Add changelog
Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
The community.kubernetes collection will be migrated / renamed to
kubernetes.core in the collection's 2.0 dev cycle. This should coincide
with the ansible-core 2.11 cycle. The collection is being released to
galaxy under both the community.k8s and k8s.core names today.
* introduce self-signed.ansible.http.tests
* forwarding of port 444
* forward port 8444 to port 444 on http test container
* Fix port forwarding for Windows under docker
* add changelog fragment
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* Ensure there is a single source of collection metadata
* Allow collection subdirs to be detected by a galaxy.yml or MANIFEST.json
* Add documentation about installing and downloading collection directories
* Add an example for downloading a git repository
* Update documented valid metadata sources for installing git repositories
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
* Improve setup_rpm_repo
- add handlers to remove repos
- add variable to control whethere or not repos are created
* Use local repo for all distros
* Change repo creation script to module
* validate_modules: fails with .id attribute not found
This patch addresses a problem in the `found_try_except_import` test.
This module tries to identify lines like:
`HAS_FOO = True`
In this case, the target (`HAS_FOO`) is of type `ast.Name` and has a
`id` attribute which provide the name.
In my case, I've a line that set a module attribute`. In this case, the
target (`module.var`) has the type `ast.Attribute` and no `id`
attribute. The code trigger an `AttributeError` exception.
This patch ensures we compare a `ast.Name`.
* Update test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/main.py
Change:
- Remove check that states that only Fedora can be an OSTree
distribution.
- This allows us to correctly return "atomic_container" as the pkg_mgr
fact for RHEL for Edge, Fedora/RHEL/CentOS Atomic Host, etc.
Test Plan:
- Created local RHEL for Edge image and tested against it.
- Tested against regular RHEL 8 and still got `dnf` as expected.
- Tested against RHEL 7 Atomic Host and got `atomic_container` now.
- New unit tests.
Tickets:
- Fixes#73084
Signed-off-by: Rick Elrod <rick@elrod.me>
ansible-galaxy currently behaves bad then a role to be installed
contains ~ or $ at any place in the path of a file in that role.
It extracts the parent directory of the offending path level as an
empty file. This explodes if that directory contains anything else.
Change this behaviour. `~` is now allowed allowed when it is
not a full level (Yes: `some~thing/`, no: `~/`). The code should
get refactoring in an other PR.
The `UserFactCollector` queries the user login name via
`getpass.getuser()` and looks up the corresponding entry
in the password database.
The login name may differ from the actual user name,
eg. if the `LOGNAME` env variable is set. The lookup
fails in this case. Added a fallback in this case that
tries to get the entry via the user ID.
* Add direct+transitive pre-release regression test
* Match user-requested transitive pre-releases
This change makes sure that in scenario when a user requests
collectionA-pre and collectionB that depends on collectionA,
collectionA-pre would actually match that collectionA requirement
while `--pre` is unset.
Co-authored-by: Jordan Borean <jborean93@gmail.com>
This warning was intended to help improve test environment configuration.
However, it has ended up producing mostly non-actionable warning noise instead.
In most situations a missing pip or python should result in test failures.
Reporting a missing pip also implies that it should be used by tests, which is not the case.
Tests should be invoking pip as a python module with the appropriate python interpreter instead.
Change detection and check mode fixes for apt_key
* allow apt-key to use proxies
* add note about deprecation of apt-key itself
* expanded error msgs
* show all keys
* fix short_format parsing
* added more return info and documented it
PR #72591
This change:
* Adds an artifacts manager that abstracts away extracting the
metadata from artifacts, downloading and caching them in a
temporary location.
* Adds `resolvelib` to direct ansible-core dependencies[0].
* Implements a `resolvelib`-based dependency resolver for
`collection` subcommands that replaces the legacy
in-house code.
This is a dependency resolution library that pip 20.3+ uses
by default. It's now integrated for use for the collection
dependency resolution in ansible-galaxy CLI.
* Refactors of the `ansible-galaxy collection` CLI.
In particular, it:
- reimplements most of the `download`, `install`, `list` and
`verify` subcommands from scratch;
- reuses helper bits previously moved out into external modules;
- replaces the old in-house resolver with a more clear
implementation based on the resolvelib library[0][1][2].
* Adds a multi Galaxy API proxy layer that abstracts accessing the
version and dependencies via API or local artifacts manager.
* Makes `GalaxyAPI` instances sortable.
* Adds string representation methods to `GalaxyAPI`.
* Adds dev representation to `GalaxyAPI`.
* Removes unnecessary integration and unit tests.
* Aligns the tests with the new expectations.
* Adds more tests, integration ones in particular.
[0]: https://pypi.org/p/resolvelib
[1]: https://github.com/sarugaku/resolvelib
[2]: https://pradyunsg.me/blog/2020/03/27/pip-resolver-testing
Co-Authored-By: Jordan Borean <jborean93@gmail.com>
Co-Authored-By: Matt Clay <matt@mystile.com>
Co-Authored-By: Sam Doran <sdoran@redhat.com>
Co-Authored-By: Sloane Hertel <shertel@redhat.com>
Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
Signed-Off-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
While getting policy name in get_chain_policy API,
module does not require any additional parameters except chain
Enabling flag in get_chain_policy API call fixes this.
Fixes: #68612
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Use correct ssh executable and options in all cases on connection plugin
* Also nicer naming/comments
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Ability to add local variables into AnsibleJ2Vars was added in
18a9eff11f to fix#6653. Local variables
are added using ``AnsibleJ2Vars.add_locals()`` method when creating a
new context - typically when including/importing a template with
context. For that use case local template variables created using
``set`` should override variables from higher contexts - either from the
play or any parent template, or both; Jinja behaves the same way.
Also removes AnsibleJ2Vars.extras instance variable which is not used.
Also adds missing test for #6653.
Fixes#72262Fixes#72615
ci_complete
The `command` module does not return stdout & sterr when calling
a non existing executable or an unknown exception arises. This fix
lets the module return empty byte strings in those cases.
Change:
- Add Ubuntu 20.04 to CI now that venv is default instead of virtualenv in ansible-test.
Test Plan:
- CI
Tickets:
- Fixes#69203
Signed-off-by: Rick Elrod <rick@elrod.me>
The init script for the test container writes additional lines to
the .bashrc of the user. This was done via a `cat` multiline
instruction, which is implemented internally by writing a
temporary file to TMPDIR (/tmp in this case) first. Docker fails
to provide /tmp just after creation, which results in a race
condition that rarely makes the init fail. Changed the `cat`
statement to multiple `echo`s.
Co-authored-by: Matt Clay <matt@mystile.com>
* Consolidate filters/tests handling into JinjaPluginIntercept
ci_complete
* Postpone loading all ansible plugins
* Do we need to create an overlay?
ci_complete
* Typo
ci_complete
* Add FIXME
* conditional.py: use public Environment.parse() method
* Remove remaining occurrences of shared_loader_obj being passed to Templar
* __UNROLLED__ not needed with this change anymore
* Incorrect rebase at some point?
##### SUMMARY
One of the parameters to the slaac filter was unquoted causing a crash:
```
template error while templating string: expected token 'end of print statement', got ':'. String: {{ fdcf:1894:23b5:d38c:0000:0000:0000:0000 | slaac('c2:31:b3:83:bf:2b') }}
```
This adds the missing quotes.
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
When calling compile(), the filename argument should be either a real file
name or a string. According to Python docs, suggested one is '<string>'.
Keep the current behaviour (encapsulate the actual expression),
but enclose it into angle brackets.
Signed-off-by: Oldřich Jedlička <oldium.pro@gmail.com>
* Fix YAML error message when error is at the end of the file
If a YAML file fails to load due to a syntax error in a file, or there is an error in the last line of a
file, PyYAML reports the last line number of the file as the index where the error occurred.
When reading the file lines, we use that index to the get the relevant line. If the index value is out
of range, the relevant line is lost for error reporting.
Subtract one from the index value to avoid the IndexError in this specific scenario. It is possible
to still get an IndexError, which will be handled as it is currently.
* Update existing tests and add new tests
##### SUMMARY
Added `[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12` to the Powershell scripts that use a file download. If not specified the script won't be allowed to download data from the mentioned URL's.
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
* Allow restricting config values to enumerated list
* dont document internal entries
* also ignore private defs for ansible-config
* remove invalid value from tests
* added porting entry
##### SUMMARY
<!--- Your description here -->
single_list must be transformed to a list of pairs before feeding it into dict()
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
When the pause module is run in the background and seconds parameter is provided,
do not warn.
* Add tests
* Fix existing tests
The test wasn't failing when it should have.
Change:
- On CentOS Stream, make distribution_release be "Stream"
- On CentOS Core, it continues to be "Core"
- Implement custom distribution file parser for CentOS, so we can look
for "CentOS Linux" and "CentOS Stream"
- Two new fixtures introduced (CentOS Linux 8.1 and CentOS Stream 8)
- Removed two dicts from `Distribution` class that were seemingly not
used anywhere.
Test Plan:
- ci_complete
- New test fixtures
Tickets:
- Fixes#73027
Signed-off-by: Rick Elrod <rick@elrod.me>
The test currently only expects the literal `ansible` followed
by a semver in the first output line of `ansible --version`.
When running from within a git checkout, additional information
like the currently checkout branch are attached, which lets
the test fail. This commit allows arbitrary information to
follow the semver.
`test_prepare_multipart` fails in non Debian environments since
Debian installations map the file ending `.key` to the MIME type
`application/pgp-keys`, which is not IANA conformant. This commit
explicitly sets the corresponding file type to
`application/octet-stream` and adjusts the expected serialized
result.
* Remove unused code in ansible-test.
* Remove obsolete endpoint logic from ansible-test.
* Remove obsolete region selection in ansible-test.
* Remove obsolete port logic in ansible-test.
* Clean up ansible-test remote providers.
Do the right thing on Linux when password lock and a password hash are provided by writing
out the password hash prepended by the appropriate lock string rather than using -U and -L.
This is the correct way to set and lock the account in one command.
On BSD, run separate commands as appropriate since locking and setting the password cannot
be done in a single action.
FreeBSD requires running several commands to get the account in the desired state. As a result,
the rc, output, and error from all commands need to be combined and evaluated so an accurate
and complete summary can be given at the end of module execution.
* Improve integration tests to cover this scenario.
* Break up user integration tests into smaller files
* Properly lock account when creating a new account and password is supplied
* Simplify rc collection in FreeBSD class
Since the _handle_lock() method was added, the rc would be set to None, which could make
task change reporting incorrect. My first attempt to solve this used a set and was a bit too
complicated. Simplify it my comparing the rc from _handle_lock() and the current value of rc.
* Improve the Linux password hash and locking behavior
If password lock and hash are provided, set the hash and lock the account by using a password
hash since -L cannot be used with -p.
* Ensure -U and -L are not combined with -p since they are mutually exclusive to usermod.
* Clarify password_lock behavior.
Adds a note pointing to the datetime documentation as to_datetime does not use the time library but instead uses datetime.
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
Tests should use the Python built-in ``venv`` module on Python 3 instead of the standalone ``virtualenv`` module.
On Python 2 the ``virtualenv`` module continues to be the only option.
The version installed is either the OS packaged version or the last release to support Python 2, which is version 16.7.10.
The old example failed with:
fatal: [localhost]: FAILED! => {"msg": "to count backwards make stride negative"}
I fixed this and placed a start parameter to consolidate with the rest of the examples provided.
Co-authored-by: Jorge Rua <jorua@redhat.com>
The standard Port for ansible winrm is set to 5986/HTTPS.
To be able to use Kerberos, you need to set ansible_port: 5985/HTTP otherwiese you get error message or timeout.
HTTPS is not needed for environments with Kerberos, since kerberos has server identification built in to the protocoll.
FYI: HTTP traffic will by default also be encrypted when using kerberos
Change:
- Internally, use dnf.subject.Subject#get_best_query for state: absent
- Add a bunch of tests for removing packages, given a bunch of different
pkg specs (nv, nvr, nvra, wildcard, etc.)
Test Plan:
- New tests
- Local experiments with DNF API via PDB.
Tickets:
- Fixes#72809
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Changes to make F33 work with current tests
- Add F33 to docker.txt
- Add F33 to AZP config
Test Plan:
- ci_complete
Signed-off-by: Rick Elrod <rick@elrod.me>
* Replace the use of the function string.replace() with the method str.replace()
because that's what works for both Python 2 and 3.
* Cleanup the unused string import.
Added a changelog fragment.
* The documentation for os.write() seems a bit iffy, but in Python 3 we
definitely cannot pass it a string and we need to encode it into
bytes.
The Python documentation at
https://docs.python.org/3/howto/pyporting.html#text-versus-binary-data
says that this code will work in Python 2 as well.
Co-authored-by: Lloyd Parkes <lloyd@must-have-coffee.gen.nz>
Change:
- The FreeBSD release can contain -RC or -PRERELEASE in addition to
-RELEASE, -STABLE, or -CURRENT.
Test Plan:
- Added new fixed from an RC version of TrueNAS which uses a -PRERELEASE
version of FreeBSD.
Tickets:
- Fixes#72331
Signed-off-by: Rick Elrod <rick@elrod.me>
* ExecutionPolicy default value is RemoteSigned
##### SUMMARY
In Windows setup steps, settings `ExecutionPolicy` is restored to value `Restricted`. But actually default value for Windows server is not `Restricted` but `RemoteSigned` ([source](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_execution_policies?view=powershell-7.1#default)).
I have got issue with `win_dsc` because `ExecutionPolicy` was `Restricted`.
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
* Add changelog fragment
* Apply suggestions from code review
Co-authored-by: Sandra McCann <samccann@redhat.com>
* Change example to use role params to represent the documented outcome. Fixes#68922
* Add notes about how vars vs role params affect variable scoping, and how lazy evaluation and role-deduplicating is affected. Fixes#72650
* link to term
* Add link to all module and plugin indexes.
* Use glob setting for toctree.
* Explicitly include new file.
* Move all_plugins one level up.
* Revert "Move all_plugins one level up."
This reverts commit bd47e82ead.
* Revert "Explicitly include new file."
This reverts commit 74ae50a734.
* Adjust .gitignore
* Revert "Revert "Explicitly include new file.""
This reverts commit f30d99c861.
* Fix docs make clean
Now that there's a static file in rst/collections, we have to be more
explicit about how we clean the generated files out of that directory.
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
* arg_spec - move type checking lookup method to a function
* Change get_wanted_type name and behavior
Change the name to get_validator to bette describe what it is doing.
Change the interface to always return a value. This lines up with the behavior of get_*
functions always returning something or None and check_* functions raising an
Exception if something went wrong during the check.
* Add param to check_type_str()
Not meant to be a long term fix, but gets tests passing. More work is needed to figure
out how to solve this cleanly.
* Remove private attribute mapping types to validator
Since the function that needs it has moved to parameters.py, there is no need to have it as
a attribute of AnsibleModule.
Update tests that were referencing the private attribute.
* Use private method for 'str' type
To avoid having to put the string conversion warning behavior in the check_type_str() method,
use the private _check_type_str() method for 'str' type.
Import CHECK_ARGUMENT_TYPES_DISPATCHER for backwards compalitibility and store it as
a private attribute.
Revert changes to support plugins that are referencing serf._CHECK_ARGUMENT_TYPES_DISPATCHER.
* Add changelog
* Change function name to better reflect its... function
* Change dict name to better reflect its contents
CHECK_ARGUMENT_TYPES_DISPATCHER --> DEFAULT_TYPE_VALIDATORS
* Fix changelog
* systemd - do not overwrite unit name when searching
PR #72702 introduced a bug that changed the unit name when splitting it up for the purpose
of searching for the unit. This only happens on unit file templates on systems that have a 5.8
or newer kernel and a version of systemd that does not contain a bugfix that causes systmed
to fail to parse dbus.
* Use facts rather than a manual probe to determine if systmed is present
* Remov unnecessary block
* Use vars files instead of set_fact
* Add tests for using a templated unit file
* Update changelog fragment
* Use template to get correct path to sleep binary
* import_playbook - change additional params to deprecation
I incorrectly recommended this be set as a warning when it should have been a deprecation.
* Fix deprecation sanity test to not required a collection name when not inside a collection
Also pin virtualenv to 16.7.10 for older Mac OS X systems. This was the version being installed
anway with the previous constraint (<20).
On systems with Python 3, now prefer venv over virtualenv. Test to see if venv is functional since
some systems have a non-functional venv installation (such as Debian).
* Increase fetch depth to reduce the frequency of full test runs for out-of-date PRs.
* Update scripts to match those which will be used in collections.
* Change role argspec file used by ansible-doc
This changes the file used for role argument specs from meta/argument_specs.yml
to meta/main.yml. The argument specs are expected to be in that file under the
top-level entry of `argument_spec`.
* Switch to argument_specs
* Update yum/dnf module docs to include version comp
Fixes#61234
Signed-off-by: Adam Miller <admiller@redhat.com>
* fix up docs syntax to sanity tests are happy
Signed-off-by: Adam Miller <admiller@redhat.com>
* ansible-galaxy collection list and verify now utilize collections in site-packages.
This is a short term fix for #70147. The long term fix needs to handle
install (but that discussion is also bound up in how upgrade is going to
work and where things can get installed so it's deferred for 2.11.)
* Add test for ansible-galaxy collection list with site-packages
* Fix sanity issue
Co-authored-by: David Moreau Simard <moi@dmsimard.com>
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* Test installing a dependency compatible with multiple collection requirements
Test downloading a collection without dependencies
* Move cleanup to the end of the test suite
On python3 sys.stdin is an encoded file object that does not support
reading raw binary data. Use the supplied buffer object to do so.
Signed-off-by: Sven Wegener <sven.wegener@inovex.de>
Co-authored-by: Sven Wegener <sven.wegener@inovex.de>
This should allow users to extract specific files from an archive as
desired.
Fixes#16130, #27081.
* Rebase and make a few minor changes
* Add changelog
* Improve tests
- move to separate tasks file
- change assertions to check for exactly one file
- use remote_tmp_dir for output dir
* Make exclude and include mutually exclusive
* Don't remove files needed by other tasks
* Fix sanity tests
* Improve feature documentation
* Skip tests that use map() on CentOS 6
* Use fnmatch on include for zip archives
This matches the behavior of exclude
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Remove space before comma in '... bla , use ...'
* 'why' is inserted in the middle of a sentence, between two commas.
* Make deprecations from base.yml show source ansible-core.
* Add changelog fragment.
* Improve some more 'why's.
* Add PR URL to fragment.
* file: add symlink is in a sticky directory tests
* file: handle symlink in a sticky directory
Co-Authored-By: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Add changelog and fix unit test
The builtins import was removed since it was unused, but it is now needed.
This is in reference to the issue: https://github.com/ansible/ansible/issues/41130 (Cannot decrypt with copy module when remote_src: yes)
Autodecryption of the vaulted files does not work when we want to copy files with remote_src=yes.
Co-authored-by: ziaurrehman72 <ziaurrheman72@yahoo.com>
The recently released version of cffi fails to install on systems with an older version of gcc. In
our case, this in the CentOS 6 test image. There is a fix but it has not yet been released.
https://foss.heptapod.net/pypy/cffi/-/issues/480
Fix the order of the arguments for ansible.netcommon.reduce_on_network. Pass the network as the argument and the list of IP addresses as the filter input.
* Move _syslog_facitily to __init__
No good reason it should not be set for each object
* Move internal property setting to private method
* Create check_arguments() function
* Remove unused import
* Rename function to better match its behavior
Change the behavior to return a set, either empty or populated, with unsupported keys.
Accept legal_inputs as optional which will not required calling handle_aliases before calling
get_unsupported_parameters().
* Add changelog
* Rework function behavior and documentation
I realized I missed the original intent of this method when moving it to a function. It
is meant to compared the parameter keys to legal inputs always, not compare
parameter keys to argument spec keys, even though the argument spec keys should
be a subset of legal inputs.
* Add tests
* Fix typo.
* Set internal properties when handling suboptions
We could not find reason for the docutils install so trying out removing it.
Also bumping to latest version of ansible-runner
Skip all python2 versions because next ansible-runner drops it
Co-authored-by: Elijah DeLee <kdelee@redhat.com>
* Support listing roles in text and JSON
* Change tests for unfrack'd playbook_dir var
These tests were using '/tmp' for testing the setting of the playbook_dir
var. Now that we unfrack that var, MacOS will change this to '/private/tmp'
causing the tests to fail. We can choose a path that does not exist (since
unfrack does not validate existence) so that we can guarantee unfracking
will not change the value.
##### SUMMARY
Make clear that `ansible_play_name` is the name attribute from the play and not the file name of the playbook.
improves #57361
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
Change:
- Initial set of changes for renaming to ansible-core
- Includes changelog fragment changes from base -> core
- Does NOT include docs changes
- Modifies detection stuff in setup.py to support ansible<2.9 and ansible-base
Test Plan:
- ci_complete
* Fix type of mode field in copy module
* Remove module type checking from ignore.txt sanity checks
* Remove mode type
Co-authored-by: Matt Martz <matt@sivel.net>
* Fix super annoying Python 2.6 multiprocessing.Queue stack trace in CI
A bug exists in Python 2.6 that sometimes raises an exception during interpreter shutdown. We
encounter this frequently in our CI since we run tests on CentOS 6 as the control node, which
has Python 2.6.6 with this bug.
This PR adds a very minor sleep only on Python 2.6 which gets around this issue. I did lot of testing
using a standalon script I found that easily duplicated the issue to find the minimum sleep value
needed to avoid this issue.
CPython issue: https://bugs.python.org/issue4106
Fix in CPython: https://hg.python.org/cpython/rev/d316315a8781
* Use correct attribute
* Consolidate logic for determining whether or not session is interactive
into a single function, is_interactive()
* Increase test coverage
I wasn't able to find a good way of simulating running a backgrounded test with CI since the
whole test is essentially run not in a TTY, which is similar enough to cause the new is_interactive()
function to always return false.
* add return documentation for unarchive, add fragment for changelog, apply suggestions from code review
Co-authored-by: Amin Vakil <info@aminvakil.com>
* Clarify collection paths in docs, fix rst syntax, rephrase docs about using collection from folder/local git
Co-authored-by: Felix Fontein <felix@fontein.de>
* Added caching mechanism for Galaxy API requests
* Add cache options and split up code
* Added unit tests
* Fix sanity test
* Use modified date and fix up caching for explicit servers
* Make sure credentials are not in cached server name
* Added test for getting updated cache version
* Changes from review
Change:
- Bump default, ansible-base, distro containers
- We do NOT add fedora33 yet, because it doesn't work right on Shippable
due to an old kernel. This will be added post-AZP.
Test Plan:
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
* Reorder comment postition
* Add comment unit test
* Fix unit test
* Fix unit test
* Add changelog
* Add paramaters which would be problematic without this fix
* Fix typo
* Fix unit test
* Fix unit test
* Emit warning when running on the controller with a Python older than 3.8
* Add spaces
Co-authored-by: Matt Clay <matt@mystile.com>
* and more spaces
Co-authored-by: Matt Clay <matt@mystile.com>
* s/Py/Python/
Co-authored-by: Matt Clay <matt@mystile.com>
* Add note to Control node requirements about Py3.8 requirement
* Add collection_name to deprecated call
* more spaces
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Clarify that we are only packaging for py3.8+
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Fix 'role_name : tast_name' notation if task contains role name
* Add tests for notifying handler names which contain the role name
Co-authored-by: Thomas Wouters <thomaswouters@gmail.com>
* Support docker and k8s action groups for moved modules in community.docker and community.kubevirt.
* Also support k8s action group for community.okd.
* Also add kubernetes.core.
* Adjust PR #.
* Fix changelog fragment.
* Remove community.okd.
* Revert "Remove community.okd."
This reverts commit 812b5aa6e2.
Change:
- Docs: Add note that security/bugfix apply to dependencies too, like
the dnf command.
- dnf: security/bugfix only makes sense for updates, so limit the
package query sack to available updates.
- tests: Limit tests to our known-good test packages, so that RHEL
packages marked security/bugfix without similarly marked dependencies
don't fail our tests.
Test Plan:
- Tested with `dnf upgrade-minimal --bugfix` and reproduced the same
error currently seen in CI, showing that we are consistent with what
dnf does.
Tickets:
- Likely fixes#72316
Signed-off-by: Rick Elrod <rick@elrod.me>
* Indent Jinja2 directives.
* Show version_added and deprecations for ini settings and environment variables.
* Make default conditional, add choices. Copied from #55474.
* Add Ansible variables. Copied from #55474.
* Turn ini keys and environment variables into lists. Copied from #55474.
* Improve formatting. Copied from #55474.
* hint in doc of git module to abort on missing http password
improves #69489
* fix syntax in doc of git module
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Matt Martz <matt@sivel.net>
* ansible-doc: export has_action when --json is used.
* Remove docuri and now_data, which were not used resp. ignored in format_plugin_doc and the functions it calls anyway.
* Add function _combine_plugin_doc.
* added timeout options to adhoc and console
* added test
* fix typosesz
* fix conflict
* task_timeout
* fix timeout option, added extra vars to console
* actually use right cli switch .. DUH!
* added timeout to include 'valid' but ignored keys
* fix default
* fixes per review
* fixes for collection playbooks
- add fqcn invocation, also handles extensions
- bring import_playbook into new normal
- avoid adding collection playbooks to adjacency
- deal with extensions or lack of em
- added tests
- fix bugs with multiple playbooks supplied
- unicode/bytes/native fixes
- fix modulenotfound for < py3.7
- tests
* Update galaxy server image used in CI
* Fix port and migration issue
* Add delete step, still need to deal with pagination
* Make cleanup more efficient
* Remove testing code
* Add DarwinStrategy class and integration tests
macOS has three seprate hostname params that need to be set. One of those params, LocalHostName, has more stringent requirements than the other two, which accept special characters and spaces. Create a method to scrub the hostname to ensure it works well with the system requirements.
* Update documentation
* Account for virtualization type returned on Azure Pipelines
* Do not be dependent on order of self.name_types
Use the scrubbed name when the name type is LocalHostName
A bug existed in systemd 245 that did not properly handle unknown kernel
capabilities gracefully. This resulted in incomplete output when querying
for the service status. It is possible to get service status by other means.
This PR works around this issue by getting service status using other commands
in the event of a failure due to this bug.
Change:
- Cryptography 3.2 drops support for OpenSSL 1.0.2. Some of our CI
infrastructure still uses this version (FreeBSD, namely). For now,
just add a constraint to use old cryptography.
Test Plan:
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
* GitHub is removing the underlying API used to implement the `login` command. Since the general consensus seems to be that relatively nobody currently uses this command (in favor of explicit token passing), support was simply removed for interactive login. If a future need arises, this command should be reimplemented via OAuth Device Auth Grants.
* login or role login commands now produce a fatal error with a descriptive message
* updated 2.10 and 2.11 porting guide entries
* remove dead code/config, update messages and porting guides
* VMware: scenario guide for vmware_tools connection plugin
Scenario doc to guide user about the usage of vmware_tools
using connection plugin.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* review comments
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Fix ansible-test docker container detection.
* Attach test containers to the correct network.
* Do not assume `localhost` for accesing Docker.
* Look for containers on current network.
* Always map /var/run/docker.sock into containers.
This fixes issues when using a remote Docker host.
* Support container IP lookup from networks list.
* Fix container network attachment.
* Remove redundant container detection messages.
* Limit DOCKER_HOST parsing to TCP.
* Restore docker socket existence check.
The check is skipped if the docker hostname is not localhost.
* Correct changelog entry.
* Fix the reversed order of return values
AnsibleModule.run_command returns a tuple of return code, stdout and stderr.
The module main function of the user module expects user.create_user to
return a tuple of return code, stdout and stderr.
Fix the locations where stdout and stderr got reversed.
* Fix another missed occurance
* Add changelog
* Fix further reversed order of return values occurrences
Followup to 72c3ba62c8:
Fix further reversed order of return values occurrences.
Change:
- containerd is now recognized as container tech
Test Plan:
- Tested with Docker inside Github Actions in a private repo:
"ansible_virtualization_role": "guest",
"ansible_virtualization_tech_guest": [
"containerd",
"container",
"VirtualPC"
],
"ansible_virtualization_tech_host": [],
"ansible_virtualization_type": "containerd",
Tickets:
- Fixes#66304 because this is what docker containers show up as
in Github Actions.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Fixing missing hosts entry under the group dict.
* Added missing 'vars:' key
* Reordering arguments for consistency with the format displayed by "ansible-playbook --help"
* Fixing contradictory text
* Ensure we call action_loader.get with collection_list. Fixes#72170
* Add tests and changelog
* Remove grep, do assertion in playbook. ci_complete
* Skip old jinja2 versions
* ci_complete
* dedupe
Change:
- In this test we end up upgrading dnf (and python3-dnf) so that we can
test its new logging behavior. However, the latest Fedora 32 dnf had a
packaging issue which caused it to not pull in the latest
python3-libdnf. This is fixed, but not synced out to mirrors yet.
Fixing it in this test will get CI passing again in the meanwhile.
Test Plan:
- CI
Tickets:
- https://bugzilla.redhat.com/show_bug.cgi?id=1887502
Signed-off-by: Rick Elrod <rick@elrod.me>
make collections whitelist follow normal flow
* fixes missing set_options call and adhoc and stdout processing rules
* avoid dupes
* fixed to handle redirects
* also updated tests with new and more accurate skip message
* fix callback tests for envs with cowsay installed
* lots MOAR comments on why the code is as it is, some todos to refactor in future
Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
beside the fact that naming the variable to specify the number of machines executed in *parallel* is named *serial* - the number of hosts chosen in this example is not optimal, either: 2x2 = 4; so - 3 hosts in 2 batches should clear things up a little
Change:
- When a plugin defines `type: str` on a parameter, treat more kinds of
input as a string instead of whatever it is parsed as.
Test Plan:
- New unit tests
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
* Add support for GSSAPI/Kerberos to urls.py
* Test out changes with the latest test container
* Get remote hosts working
* Fix up httptester_krb5_password reader
* Fix tests for opensuse and macOS
* Hopefully last lot of testing changes
* Dont do CBT on macOS
* Fixes from review
* Add explicit apt tests for fnmatch and update_cache
* Add explicit apt_key tests for fetching key directly from url
* ci_complete ci_coverage
* Remove repo only by repo
* ci_complete ci_coverage
* Add apt cache update after apt_repository to show that the cache doesn't update
* ci_complete ci_coverage
* Add systemd tests for enabling and disabling a service
* ci_complete ci_coverage
* Remove incidental_zabbix_host
* ci_complete ci_coverage
* Fix various sanity errors in plugins.
* Revert callback_type -> type transform.
* Undo paramiko_ssh 'connection' change, since this discrepancy is hardcoded in the plugin loader.
* Standardize on name and type (for callbacks).
* Fix existing author entries.
* Add 'Unknown (!UNKNOWN)' as author when author is missing.
* These are actually parsed as integers in the code.
* Revert "Add 'Unknown (!UNKNOWN)' as author when author is missing."
This reverts commit 29d1438aca.
* Add a test suite for module_utils.common.dict_transformations
* ci_complete
ci_coverage
Add a wait_for test using delegate_to
* Remove incidental_ec2_instance
* Remove unused test support modules
* Requested changes
ci_complete
ci_coverage
* Oops, put everything back to test coverage again
ci_complete
ci_coverage
* Remove incidental_ec2_instance tests and supporting modules
Change:
- No longer fail due to old Fedora/RHEL and our failure to try to cast
gids to integers before trying to pass them to getgrgid() before
trying to use them.
- Add tests for user/mode for various unarchive formats.
Test Plan:
- New integration tests, ran against centos6 container
Tickets:
- Fixes#71903
Change:
- Make KubeVirt be identified as such, and with a guest role.
- This is a breaking change, since playbooks could be relying on the
incorrect behavior previously.
Test Plan:
- Issue author
Tickets:
- Fixes#72001
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- podman > 2 && < 2.2 does not support "images --format {{json .}}"
- podman also now outputs images JSON differently than docker
- Work around both of the above.
Test Plan:
- Tested with podman 2.0.6 in Fedora 31.
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Change:
- Use `sysctl -n` for openbsd uptime information
- Allow `get_sysctl()` to account for multi-line sysctl settings
- Add unit tests for `get_sysctl()`
Test Plan:
- New unit tests
Tickets:
- Fixes#71968
- Refs #72025
- Refs #72067
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Brian Coca <brian.coca+git@gmail.com>
Co-authored-by: Brian Coca <bcoca@ansible.com>
The luseradd / lusermod commands do not support the -e option. Set
the expiry time in this case via lchage after the user was
created / modified.
Fixes: #71942
In Python3 math.floor returns an integer whereas Python2 returns a float.
Hence always convert the result of math.floor to an int to ensure that
lexpires is an integer.
Move local expires tests in a separate file and import the tasks to the
main.yml to keep main.yml smaller.
Module package is actually a proxy to a native package manager
modules (internally it delegates to the module that matches the
results of ansible_pkg_mgr).
Fixes#69371
b6b238a fixed the SLES4SAP detection, which was at this time ok.
Sadly Suse changed with SLES 15 the /etc/os-release file, so the above
change will no longer work.
This commit updates the SLES4SAP detection regarding
https://www.suse.com/support/kb/doc/?id=000019341.
The symlink realpath is matched with endswith, because in SLES 12+ the
link target is SLES_SAP.prod, but in SLES 11 the link target is
SUSE_SLES_SAP.prod.
* Validate removal versions.
* Validate that removal collection versions and version_added collection versions conform to semver spec.
* Validate removal version numbers in meta/runtime.yml.
* Stricter validation for isodates (f.ex. YYYY-M-D is not allowed).
* Improve error reporting.
* Validate removal collection versions.
Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
Change:
- pip packages should get removed after, not try to add them again
- Try removing containerd.io package too
Test Plan:
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
* Use default group of staff on macos to prevent sudo issues
* Install gnu-tar for macos in git and unarchive tests
* Enable timezone module to support py3 on macos
* If the virtualenv command is missing, try python -m virtualenv
* Install passlib for filter_core on macos
* Install paramiko via pip on macos for paramiko tests
* Normalize discovered python interpreter on macos
* Get pip tests passing, by ensuring we have wheel installed
* Create /etc/ansible for ca certs on mac, list lookup_url as destructive
* Fixups for CA certs
* Include macos
* Dynamically get cafile instead of hardcoding the path
* DOCS: updates intersphinx references for docs links
* TESTS: Raise the number of bytes scanned to determine if a file is binary. The newest ansible-2.10.inv file has its first null byte at position 2261. 4096 is still a cheap chunksize to read so it still makes sense to raise this.
Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
This change preserves how all the external imports refer to this code
while allowing us to start cutting the spaghetti into more easily
maintainable pieces.
This is a start of the upcoming refactoring effort destined to
eliminate tight coupling, implicit data manipulation, god objects,
abstraction leaks and other code smells.
Essentially, `ansible.galaxy.collection` is going to be a package that
holds parts of the collection management code spread across loosely
coupled modules.
PR: #71931
Change:
- The docker-ce.repo file for centos does not work on RHEL since it uses
$releasever and on RHEL that is, e.g., "7Server".
- Instead, set up the repo manually.
- Additionally, the docker centos8 repo no longer has old versions, so
we use the (only) version in the repo instead.
Test Plan:
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
* Let get_file_attributes() work without `lsattr -v`
Change:
- module_utils's get_file_attributes() expects `lsattr -v` to work, but
in some cases, it may not.
- The function now takes an optional include_version bool parameter,
which removes this expectation.
- Places where we call get_file_attributes() without using the 'version'
it returns, we now call it with include_version=False.
Test Plan:
- New unit tests
Signed-off-by: Rick Elrod <rick@elrod.me>
* Reorder test/sanity/ignore.txt to better see changes
* Remove extra empty line causing pass fail
* Apply suggestions made by Akasurde
* Minor fix in package_facts and remove two unnecessary ignores
* Fix subversion based on a suggestion made by felixfontein
* Apply suggestions made by felixfontein and Andersson007
* Fix subversion.py as suggested by felixfontein
* Minor reformatting in yum_repository description
* Reformat changelog
* Add key to apt_key, add deprecated changelog
* Add PR url to changelog
* Ignore paramater-type-not-in-doc in favour of adding key back to apt_key
* Fix apt_key
* Remove undocumented-paramater from apt_key ignore
* Ignore doc-choices-do-not-match-spec in package_facts
* Fix package_facts
* Fix filter option in setup module
Change:
- In some cases (always with free strategy, sometimes with linear), the
default callback would not show the task banner for include_tasks.
- This only affects the include_tasks task itself, not the tasks in the
included file.
Test Plan:
- Updated default callback tests
Tickets:
- Fixes#71277
Signed-off-by: Rick Elrod <rick@elrod.me>
The test is not supported when running in a container. It now recognizes both 'docker' and 'container' as virtualization types that should cause the test to be skipped.
A recent update to cffi that was yanked is still being installed on our
Mac OS X 10.11 test image since the version of pip there is very old and
does not ignore yanked packages.
Pin the version of pyOpenSSL and its dependencies to fix this and avoid
future spontaneous failures.
* [docs] add porting guide for DNF GPG validation
Change:
- This was a breaking change (security fix), but I neglected to add a
porting guide entry for it previously.
Tickets:
- Refs #71537
- Refs #71539
- Refs #71540
- Refs #71541
Signed-off-by: Rick Elrod <rick@elrod.me>
* changes from sivel
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Previously when `security: true` and `bugfix: true` were both given,
only security updates would get applied. Filters now accumulate so
that both get applied in this case.
Test Plan:
- New integration tests for both check_mode and not. These tests make
use of a contrived yum repository which is stored in S3.
Tickets:
- Fixes#70854
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Matt Martz <matt@sivel.net>
* fix coverage output from synthetic packages
* synthetic packages (eg, implicit collection packages without `__init__.py`) were always created at runtime with empty string source, which was compiled to a code object and exec'd during the package load. When run with code coverage, it created a bogus coverage entry (since the `__synthetic__`-suffixed `__file__` entry didn't exist on disk).
* modified collection loader `get_code` to preserve the distinction between `None` (eg synthetic package) and empty string (eg empty `__init__.py`) values from `get_source`, and to return `None` when the source is `None`. This allows the package loader to skip `exec`ing things that truly have no source file on disk, thus not creating bogus coverage entries, while preserving behavior and coverage reporting for empty package inits that actually exist.
* add unit test
On some systems (tested with official Debian Buster-based Python 3.6
docker image), setting the LC_ALL environment variable to en_US.UTF-8
will trigger Python into switching its preferred encoding to ASCII.
If any python process tries to read a non-ASCII file in this scenario,
it will terminate with an error.
And this is exactly what happens to pytest when it tries to load its
configuration that ansible-test supplies because the configuration
contains an em dash.
In order to bypass this issue, we replaced the em dash with a regular
dash and things started working again.
An alternative solution would be to replace the en_US.UTF-8 locale with
something safer, but unfortunately, the en_US.UTF-8 is probably as safe
as it gets.
The setup module can now filter out multiple pattern by providing a list
to the filter parameter instead of just a string. Single string sill
works. Previous behaviour remains.
(cherry picked from commit b5c36dac483fdd74d6c570d77cc8f3e396720366)
* Use InventoryManager for inventory_hostnames lookup. Fixes#17268
* Add test for using patterns
* s/it's/its/
* Allow bypassing parse_sources in InventoryManager
* Remove unneeded method call
In order to test this piece of functionality, we create a dedicated
collection that contains files that should be ignored when building
the collection tarball.
After the collection is built, we make sure the produced tarball does
not contain any files that should be ignored.
Change:
- By default the dnf API does not gpg-verify packages. This is a feature
that is executed in its CLI code. It never made it into Ansible's
usage of the API, so packages were previously not verified.
- This fixes CVE-2020-14365.
Test Plan:
- New integration tests
Signed-off-by: Rick Elrod <rick@elrod.me>
test_dict_undefined_values is a duplicate of test_dict_undefined_values_is_defined.
It seems originally test_dict_undefined_values was supposed
to do what test_dict_undefined_values_bare does so it is safe to remove.
Remove "mode: preserve" option from documentation in doc fragments "FILE_COMMON_ARGUMENTS", as it was incorrectly included in the documentation for the `lineinfile`, `unarchive` and other file-related modules.
The `copy` and `templates` modules documentation remains untouched and still contain "mode: preserve", as intended.
* Validate salt when using crypt. Respect salt_size in password lookup. Repair salt for bcrypt. Fixes#71107. Fixes#53750. Fixes#36129.
* Handle algorithms we don't know about, and make sure to return the salt
* Account for old passlib
* Add tests for salt constraints
* Add changelog fragment
* Add test for #36129
Some parameters for systemd are ignored, add a note about such parameters in documentation.
Fixes: #23144
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
nitzmahone's blog nicely explained why Windows is not supported
as Ansible controller. Link that in documentation so users can
read about it.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* added configurable dir for tree callback
- allows usage in playbooks
- also made errors more specific if bad dir is supplied
- ensure we expand paths
* docfix
This script is mostly unmaintained and relies on the deprecated and
unmaintained `boto` library. Featuring it prominently in the docs
leads to many new users using it instead of the supported `aws_ec2`
inventory plugin.
##### SUMMARY
Per #67913, when comparing dependencies, Ansible takes into account parameters, tags and the when clause in determining whether a role is a duplicate or not.
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
* quick fix to change main back to devel
* Update docs/docsite/rst/dev_guide/developing_collections.rst
Co-authored-by: Felix Fontein <felix@fontein.de>
Change:
- Now sends meta tasks to the task start callback
- Lets callback plugins opt-in to receiving implicit tasks
Test Plan:
- New integration tests
Tickets:
- Indirectly fixes#71007 by allowing custom callbacks with this data
Signed-off-by: Rick Elrod <rick@elrod.me>
ini-style inventory does not support Ansible Vault password.
This fixes network_best_practices_2.5 doc.
Fixes: #69039
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
skip_missing parameter in subelements lookup plugin is accepted from
inside the dictionary.
Fixes: #38182
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
"command -v" is the POSIX compliant way to get the path to a command.
It returns the path to the command if it exists and exits with a
non-zero status when it does not, without any output. "which" is not
POSIX and it can have various different effects. With Zsh, it says
"python not found" on stdout.
See:
- https://pubs.opengroup.org/onlinepubs/009695399/utilities/command.html
Signed-off-by: Vincent Bernat <vincent@bernat.ch>
* check run state of current block only
* Add changelog and test
* Add test for issue 29047
* Fix for both tests
* blerg
* Change test messages
* fix tests
* Add multi-level block in rescue test case
* Add recursive rescue check and multi-level test
* Should probably run the new test
* ci_complete
* Merge new tests
* ci_complete
* Don't do conflict check on sdist and egg_info. Fixes#71279
* Add changelog
* adjust changelog PR link
Co-authored-by: Rick Elrod <rick@elrod.me>
* add warning about bdist_wheel
Co-authored-by: Rick Elrod <rick@elrod.me>
This would be a partial solution for #69364 in that the SHASUMS file can be downloaded and gpg verified but then used from the downloaded location to verify the get_url's file.
* Make checksum url parsing more explicit
Use urlsplit to test if the checksum string has a (currently tested and) supported url scheme.
* Fix whitespace
* Changelog fragment
* Added tests
* Fix typo in test setup
The message generated by systemctl has been updated in 9321e23c40, which requires a corresponding change in the systemd module.
In addition, this fixes the module when the SYSTEMD_OFFLINE environment variable is set.
* POC for supporting callback events that come from the worker
* linting fixes. ci_complete
* fix up units. ci_complete
* Try moving the sentinel put higher. ci_complete
* safeguards. ci_complete
* Move queue killing to terminate
* LINTING. ci_complete
* Subclass Queue, to add helper send_callback method
* Just use _final_q instead of adding another queue and thread
* Revert a few changes
* Add helper for inserting a TaskResult into the _final_q
* Add changelog fragment
* Address rebase issue
* ci_complete
* Add test to assert async poll callback from fork
* Don't use full path
* ci_complete
* Use _results_lock as a context manager
* Add new generic lock decorator, and use it with send_callback
MacOSX seems to want bare `+` whereas GNU expr wants escaped `+` (`\+`)
to mean match one or more. Use `\{1,\}` instead which will match one or
more on both MaxOSX and GNU-using systems.
Fixes#71053
* [docs] 2.7 is EOL, add 2.10 which is almost out
- Remove 2.7 support from the maintenance schedule
- Add 2.10 which is in RC and will be out soon enough.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Update docs/docsite/rst/reference_appendices/release_and_maintenance.rst, fix table and separate ansible-base from ansible, fix rstcheck errors, clean up sections, explain the two packages
Co-authored-by: Sandra McCann <samccann@redhat.com>
Co-authored-by: Rick Elrod <rick@elrod.me>
* Revert "Change default file permissions so they are not world readable (#70221)"
This reverts commit 5260527c4a.
* Revert "Fix warning for new default permissions when mode is not specified (#70976)"
This reverts commit dc79528cc6.
ansible_play_batch lists the currently targeted host(s) in the serial/batch, while
ansible_play_hosts lists all the hosts which will be targeted by the play.
Password lookup can be used to generate random string
with desired length. This is useful in various scenarios.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Ensure -k is set to delegated hosts without a pass
* Fix up some broken tests
* Update task_executor.py
one possible fix, the other is updating winrm to normalize on 'password' like the other connection plugins
* Add alias for winrm and fix incorrect assumption
* Make sure aliases are used for keyword options
* Conditionally run test if sshpass is present, fix sanity
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
* Start of alpine testing
* More updates
* Add forgotten file
* remove debug
* Add alpine3
* equal
* group 4
* group 4
* group 5
* Try to decrease test length
* libuser only available in testing
* Remove debug
* Make loops target work on hosts without gnu date
* Enable alpine testing
* ci_complete
* Don't specify uid for creating test user
* ci_complete
* Re-sort docker completion
* use newer container image
* ci_complete
* fix indentation
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Clay <matt@mystile.com>
* The version comparison for determining what ansible package to build
docs against was comparing the version number for ansible-base but it
needed to check the version number for ansible instead
* add a comment about some bad logic than needs to be fixed after 2.10.0
The ansible collection repository correctly renamed their default branch from `master` to `main`, which has caused a number for broken urls. This PR fixes those urls.
Change:
- Allow systems to declare multiple virt techs. For example if a system
is both a docker container, but virtualized on KVM, show both. If a
system is set up to run virtualbox and KVM VMs, show both.
- This is done by introducing new facts keys:
- virtualization_tech_guest
- virtualization_tech_host
- Backwards compatibility is preserved by keeping track of the previous
return-points and refusing to update those keys after we would have
returned, but now returning them at the end, so that the new keys can
accumulate their data.
Test Plan:
- Local
- CI
Tickets:
- Refs #66304
- Refs #17151
- Refs #17058
- Probably others
Signed-off-by: Rick Elrod <rick@elrod.me>
* Fix tty_ify bugs and refactor
* Move tty_ify() and supporting attributes to the DocCLI class as that's
the only thing using it.
* Add unittest for the code.
* Fix a bug where the substitution macros can be detected when they are
a part of another word.
* Add support for L(), R(), and HORIZONTALLINE which were added to the
website docs many years ago.
* Update test/units/cli/test_doc.py
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Clay <matt@mystile.com>
* add jinja2 global to reserved names
also allow expansion by additional context provided from caller
fixes#41955
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Change:
- Use `chmod +a` in the fallback chain to allow MacOS to use ACLs to
allow an unprivileged user to become an unprivileged user.
Test Plan:
- CI, new tests
Tickets:
- Fixes#70648
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Adds some intentional coverage around PluginLoader for cases that
incidental_azure_rm_resource covered.
- Specifically, modules starting with an underscore, and starting with
an underscore but a symlink.
Test Plan:
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Add hardware_facts test target which manually sets up some LVM devices
and tests facts against them.
Test Plan:
- New integration tests
Tickets:
- Refs #71041 and #71042 both of which I discovered during this
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Add a parameter `read_whole_file` which allows for reading the whole
file when doing a `contains` regex search.
- This allows for (for example) matching a pattern at the very end of
a file.
Test Plan:
- New integration tests
Tickets:
- Fixes#63378
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Previously, `cache_timeout=0` was seen as falsey and not passed along
to cache plugins as an option. Now only "nulley" values are not sent.
Test Plan:
- New integration tests
Tickets:
- Fixes#70702
Signed-off-by: Rick Elrod <rick@elrod.me>
Changes to sanity and unit tests now trigger the ansible-test self-test integration tests.
No changelog entry since this only affects tests for ansible itself and not collections.
Change:
- Previously CachePluginAdjudicator#flush only removed entries from the
cache backend that it knew about by using them earlier. Now it calls
the underlying plugin's flush() method.
Test Plan:
- New unit tests
Tickets:
- Fixes#68770
Signed-off-by: Rick Elrod <rick@elrod.me>
There are links from the scenario guides to collections but collections
docs aren't built in testing or locally when we're on the devel branch.
Due to that we need to make sure those references resolve to the
production docsite. We can use intersphinx to make sure that happens.
* The test for binary files wasn't reading enough of the file.
Checking for null bytes in the first 1024 bytes failed to diagnose the
ansible_2_10.inv file as binary
* Strip spaces in module names in explicit actions
Change:
- When an action is called like "action: copy foo=bar", strip spaces
around the action name.
- This allows "action: copy foo=bar" to work as expected.
Test Plan:
- New integration tests
Tickets:
- Fixes#62136
Signed-off-by: Rick Elrod <rick@elrod.me>
Update the docs to guide users to use `ansible-runner` instead of using Python API directly. In many use cases, executing Ansible playbooks are sufficient. In those use cases, `ansible-runner` is easier and much stable to use comparing with Python API, but there is no mention of it.
* Incase of network integration test for connection local
test the paramiko_ssh auth_timeout is the value of timeout
under defaults section which is 10 seconds.
* For slower connection 10sec timout value result in authentication
timeout error hence increase the timeout value to 90 seconds
* Fix changelog link title.
* Rename Ansible 2.10 and 2.11 porting guides to Ansible-base porting guides.
* Add stub for automatically generated 2.10 porting guide.
* Move things that should not be in the ansible-base porting guide to the ansible porting guide.
* Apply changes to base porting guides.
* Add remark that ansible-base is mainly for developers.
* Ansible Base -> Ansible-base
* Fix link in base porting guide.
* Add generated porting guide.
* Use same header signs as antsibull-changelog's RST builder.
* Update generated porting guide.
Change:
- Fix a bug where rc/beta versions throw off the "is this an
ansible-base release"? check.
Test Plan:
- Used it for 2.10.0rc4
Signed-off-by: Rick Elrod <rick@elrod.me>
* document how to migrate between collections
* Apply suggestions from code review
Co-authored-by: John R Barker <john@johnrbarker.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Add integration tests for unarchiving as unprivileged user
Break tasks into separate files for easier reading and maintenance
Create a user by specifying a default group of 'staff' for macOS.
The user module does not actually remove the user directory on macOS,
so explicitly remove it.
Put the removal tasks in an always block to ensure they always run
Co-authored-by: Philip Douglass <philip.douglass@amadeus.com>
Co-authored-by: Sam Doran <sdoran@redhat.com>
The iso8601_micro and iso8601 facts incorrectly called now.utcnow(), resulting
in a new timestamp at the time it was called, not a conversion of the previously
stored timestamp.
Correct this by capturing the UTC timestamp once then calculating the local
time using the UTC offset of the current system.
* Use time.time() for getting the current time
* Convert from that stored epoch timestamp to local and UTC times
* Used existing timestamp for epoch time
* Add unit tests that validate the formate of the return value rather than an exact value since mocking time and timezone is non-trivial
Follow up to #70221
Related to #67794
CVE-2020-1736
When set_mode_if_different() is called with mode of 'None', ensure we issue
a warning about the change in default permissions.
Add integration tests to ensure the warning works properly.
* Fix tests
- actually use custom module 🤦♂️
- verify file permission on created files
- use remote_tmp_dir so we're ready for split controller
- improve test module so we can skip the call to set_fs_attributes_if_different()
- fix tests for CentOS 6
Change:
- Clarify that not passing `--tags` will cause `ansible_run_tags` to
default to `["all"]`.
- Add some extra coverage around `ansible_run_tags`
Test Plan:
- New integration and unit tests
Tickets:
- Fixes#69619
Signed-off-by: Rick Elrod <rick@elrod.me>
- Combine remote and local debugging instructions.
- Update the example code to match current AnsiballZ structure and behavior
- Change reference name and update references
- Clarify how PYTHON path is modified
- Also add note about other remote debugging tools.
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
Change:
- Allow older git to verify tags again
- Enable verification tests everywhere, even if most of them only work
on newer git. Some of them work on older git and they test the --raw
parameter.
Test Plan:
- Re-enabled subset of git tests
Tickets:
- Fixes#64469
Signed-off-by: Rick Elrod <rick@elrod.me>
* Fix verbosity for var_templating test
Display the subtests
* fix specificity and set ANSIBLE_BECOME_ALLOW_SAME_USER
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Matt Martz <matt@sivel.net>
Change:
- Previously, we only showed that something would have changed, not what
would have changed. This allows us to show what will chang as well.
Test Plan:
- Local RHEL8 VM
- New integration tests
Tickets:
- Fixes#66132
Signed-off-by: Rick Elrod <rick@elrod.me>
* Also validate top-level version_added.
* Fix error code.
* Produce same version_added validation error in schema than in code (and stop returning it twice).
* Return correct error codes for invalid version_added for options and return values.
* Add changelog.
* Fix forgotten closing braket.
* Accept 'historical' for some top-level version_added.
* dnf: Add nobest option
* dnf: Fix indent, add nobest specifically to dnf not yum
* Add changelog for dnf: add nobest option
* dnf: Add nobest to yumdnf module argument_spec
* dnf: remove nobest from module paramaters in yumdnf.py
* dnf: Add test for nobest option
* dnf: Cleanup packages in nobest test at last
* dnf: Cleanup manually added repos in nobest test at last
* dnf: Remove dnf-plugins-core as well in nobest test
* dnf: Change nobest release version to 2.11
* Change changelog number according to change in PR number
* Change changelog number according to change in PR number
Change:
- On Linux, there are situations where a host might be a KVM host but
not have the kernel module enabled (it might be compiled in instead).
In these cases, /dev/kvm will still exist, and rather than reporting
NA, we should report that the host is a KVM host.
Test Plan:
- Local
Signed-off-by: Rick Elrod <rick@elrod.me>
PR #70446: it's a follow-up for #70445.
It includes a merge of `examples/scripts/uptime.py` and a similar
code snippet from `docs/docsite/rst/dev_guide/developing_api.rst`.
This patch also changes the docs RST file to include contents of
the example file instead of holding a copy of a similar code.
* Change default file permissions so they are not world readable
CVE-2020-1736
Set the default permissions for files we create with atomic_move() to 0o0660. Track
which files we create that did not exist and warn if the module supports 'mode'
and it was not specified and the module did not call set_mode_if_different(). This allows the user to take action and specify a mode rather than using the defaults.
A code audit is needed to find all instances of modules that call atomic_move()
but do not call set_mode_if_different(). The findings need to be documented in
a changelog since we are not warning. Warning in those instances would be frustrating
to the user since they have no way to change the module code.
- use a set for storing list of created files
- just check the argument spac and params rather than using another property
- improve the warning message to include the default permissions
* ensure hostvars are available on delegation
* also inventory_hostname must point to current host and not delegated one
* fix get_connection since it was still mixing original host vars and delegated ones
* also return connection vars for delegation and non delegation alike
* add test to ensure we have expected usage when directly assigning for non delegated host
Some platform such as ESXi does not implement EpollSelector,
which is selected by DefaultSelector. Use SelectSelector which is
based upon 'select' implementation. This works perfectly with
a platform like VMware ESXi.
Fixes: #70238
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Change:
- Remove "New Module Pull Request" since we're not taking new modules in
ansible/ansible
- Add "Tests Pull Request" to use for things like adding coverage that
aren't bugfixes.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Encode/Decode files in UTF-8
* Use helper function in ansible
* Add an integration test
* Use emoji in test data.
* add changelog
* Also support non-ascii chars in filepath and add tests about this.
* Also use non-ascii chars in replaced text and ensure not to break cron syntax.
* rename self.existing to self.n_existing
* rename crontab.existing to crontab.n_existing
Change:
- Refactoring to make it harder to get wrong and easier to read.
- Generalize become_unprivileged tests and fix some that never worked
but also never failed.
Test Plan:
- CI, new units/integration tests
Signed-off-by: Rick Elrod <rick@elrod.me>
* Format using `` instead of `, add line breaks for long lines, rephrase or remove useless text.
Move some text.
* Add clearer version of OpenSSh is affected by SCP bug
* Review some pages using ansible doc writing guide.
* Guard against allowing ansible to ansible-base upgrades
* newline
* use alias
* Add an explicit line detailing this is a 1 time thing
* period
* Read __version__ and __author__ rather than import, update working, and add ability to skip conflict checks
* Remove commented code
* Re introduce removed changes from rebase
* Just use open
* Nuke unused import
* Fix missing quoting for remote_tmp in second mkdir of shell module. Issue #69577
* adding changelog
* fixing typo in changelog entry
* adding test case
Adding test case written by bmillemayhias.
* using $HOME instead of ~
* fixing commit measage
* Update 69578-shell-remote_tmp-quoting.yaml
Co-authored-by: Brian Kohles <me@briankohles.com>
debconf module exposes sensitive information to logs, console.
Add a note to user about using no_log=True to hide such
information from console.
Fixes: #32386
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
From what I have observed it is the block itself that doesn't support the name attribute rather than the tasks inside the block.
* Update docs/docsite/rst/user_guide/playbooks_blocks.rst
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
* Followup to #70610: fix redirects to wrong collections
* gcp module_utils is in community.general.
* dellos10_* modules are in dellemc_networking.os10, not dellemc.os10.
* intersight_facts was renamed to intersight_info in cisco.intersight in version 1.0.7.
* Fix more dellos10 redirects.
A couple of years ago Slackware -current began using a plus (“+”) at the end of the distribution version string to indicate a future version work-in-progress.
Rearrange distribution_files unit tests to easily support more tests
- add conftest with common fixtures
- use parametrize for testing multiple scenarios
* Add changelog
* Add unit tests for Slackware distribution parsing
* Use correct fixtures for Slackware
Data comes from /etc/slackware-version
Co-authored-by: Sam Doran <sdoran@redhat.com>
Co-authored-by: <Eduard Rozenberg <eduardr@pobox.com>>
* Build documentation for Ansible-2.10 (formerly known as ACD).
Builds plugin docs from collections whose source is on galaxy
The new command downloads collections from galaxy, then finds the
plugins inside of them to get the documentation for those plugins.
* Update the python syntax checks
* docs builds can now require python 3.6+.
* Move plugin formatter code out to an external tool, antsibull-docs.
Collection owners want to be able to extract docs for their own
websites as well.
* The jinja2 filters, tests, and other support code have moved to antsibull
* Remove document_plugins as that has now been integrated into antsibull-docs
* Cleanup and bugfix to other build script code:
* The Commands class needed to have its metaclass set for abstractmethod
to work correctly
* Fix lint issues in some command plugins
* Add the docs/docsite/rst/collections to .gitignore as
everything in that directory will be generated so we don't want any of
it saved in the git repository
* gitignore the build dir and remove edit docs link on module pages
* Add docs/rst/collections as a directory to remove on make clean
* Split the collections docs from the main docs
* remove version and edit on github
* remove version banner for just collections
* clarify examples need collection keyword defined
* Remove references to plugin documentation locations that no longer exist.
* Perhaps the pages in plugins/*.rst should be deprecated
altogether and their content moved?
* If not, perhaps we want to rephrase and link into the collection
documentation?
* Or perhaps we want to link to the plugins which are present in
collections/ansible/builtin?
* Remove PYTHONPATH from the build-ansible calls
One of the design goals of the build-ansible.py script was for it to
automatically set its library path to include the checkout of ansible
and the library of code to implement itself. Because it automatically
includes the checkout of ansible, we don't need to set PYTHONPATH in
the Makefile any longer.
* Create a command to only build ansible-base plugin docs
* When building docs for devel, only build the ansible-base docs for
now. This is because antsibull needs support for building a "devel
tree" of docs. This can be changed once that is implemented
* When building docs for the sanity tests, only build the ansible-base
plugin docs for now. Those are the docs which are in this repo so
that seems appropriate for now.
A recent updated to psutil, which is a dependency of ansible-runner, fails
to install on older versions of pip.
Commit with the breaking change:
135628639b
* Get m_u.facts.utils coverage up to 100%
Change:
- Add tests to 'gathering_facts' integration target to get
module_utils.facts.utils coverage up to 100%.
- This also clears incidental coverage from incidental_selinux.
Test Plan:
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
A little further down the page is another index, ansible_loop.index, which shares a similar description but is 1 indexed.
Its zero indexed twin has a 0 suffix.
``ansible_loop.index`` The current iteration of the loop. (1 indexed)
``ansible_loop.index0`` The current iteration of the loop. (0 indexed)
To remove ambiguity around the usage of index_var, explicitly mention that this variable is 0 indexed.
Change:
- Test async_wrapper when the module it runs has stderr output
Test Plan:
- CI
- Looked at coverage report and saw green for a few lines that weren't
previously green.
Signed-off-by: Rick Elrod <rick@elrod.me>
pipe lookup plugin uses Popen with shell=True intentionally.
This is considered a security issue if user input is not validated.
Updated docs to reflect this information for the user. Also, added
Bandit B602 documentation link for further reading.
Fixes: #70159
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Change:
- Add integration tests for various cases
- Fix wrong use of "its" in an exception thrown in varnames when it
throws an AnsibleError, given a term of the wrong type.
Test Plan:
- new tests, CI
Tickets:
- Fixes#70546
Signed-off-by: Rick Elrod <rick@elrod.me>
- ensure we preserve the typeerror part of the exception so loop defereed error handling
can postpone those caused by undefined variables until the when check is done.
- fix tests to comply with the 'new normal'
- human_to_bytes and others can issue TypeError not only on 'non string'
but also bad string that is not convertable.
Co-authored-by: Sloane Hertel <shertel@redhat.com>
Co-authored-by: Sloane Hertel <shertel@redhat.com>
Change:
- Use parse_kv() for parsing in the csvfile lookup plugin. This allows
us to handle multi-word search keys and filenames. Previously, the
plugin split on space and so none of these things worked as expected.
- Add integration tests for csvfile, testing a plethora of weird cases.
Test Plan:
- New integration tests, CI
Tickets:
- Fixes#70545
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Move hostnamectl check out of GenericStrategy because it was incorrect
for everything except the SystemdStrategy which is where it belongs.
- Add some initial tests for the hostname module, though we are limited
by the fact that we can't do much testing with it in containers.
Test Plan:
- new hostname integration tests
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Sam Doran <sdoran@redhat.com>
Change:
- Followup to #70504. We need to not pass the 'warn' parameter from the
action plugin either, unless it's True. Otherwise, even though it
defaults to false, we always show the deprecation.
Test Plan:
- Local
Signed-off-by: Rick Elrod <rick@elrod.me>
* speed up modify_module
* Remove debugging
* ci_complete
* Simplify generic_visit, alias to visit, eliminate some attr lookups
* ci_complete
* Add changelog fragment
* Fix ansible-test error in community.aws
* Add changelog entry for fix
* Change check from None to string_types
* Update changelogs/fragments/70507-validate-null-author.yaml
clarify wording "or a list of strings"
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/schema.py
clarify wording - single string or not specified valid
Co-authored-by: Felix Fontein <felix@fontein.de>
* Do not fail but return None when given outside list
Co-authored-by: Felix Fontein <felix@fontein.de>
Change:
- The command warnings feature which suggests that users use modules
instead of certain commands is now deprecated. Its `warn` paramater
and `COMMAND_WARNINGS` configuration options are also deprecated.
Their use will become an error in version 2.13.
Test Plan:
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Sam Doran <sdoran@redhat.com>
Change:
Our handling of NetBSD virtualization facts led to facts that were just
plain incorrect. One example is reporting Xen even when the system is
running on something completely different (like KVM).
As stated by the reporter of #69352, NetBSD has a better sysctl setting
to use for this information, machdep.hypervisor.
This PR does the following:
- Try to use machdep.hypervisor sysctl value if the other sysctl values
we check don't end up with enough information to be useful
- Only look for /dev/xencons and assume Xen if nothing else works
(Really this should probably return 'unknown' since the file exists on
non-Xen systems and is not very useful).
- Add a few more patterns (Xen matches and also Hyper-V) to
VirtualSysctlDetectionMixin#detect_virt_product.
This change is slightly breaking:
- If the first two attempts at using sysctl worked before,
(machdep.dmi.system-product and machdep.dmi.system-vendor), they will
continue to work.
- For cases when those values didn't work, previously the existence of
/dev/xencons was checked, and if found, we reported 'xen' (even on
non-Xen systems when the file existed). After this PR, we try the
machdep.hypervisor sysctl key before still falling back to
/dev/xencons. This means that in some cases, we might go from
(wrongly) saying "xen" to giving a more accurate value such as "kvm"
or "Hyper-V".
Test Plan:
- Tested with local NetBSD VM and got 'kvm' instead of 'xen' back.
Tickets:
- Fixes#69352
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- module_utils.basic.is_special_selinux_path() used a string ==
bytestring comparison which returned False and made Ansible think that
certain filesystems aren't, in fact, special-cased, when they should
be. Ensure both sides of the == are bytestrings.
Test Plan:
- Added `copy` integration tests for this case.
Tickets:
- Fixes#70244
Signed-off-by: Rick Elrod <rick@elrod.me>
* Add type for options in the sample module shown in Developing Modules, as this is required to have the tests being
green
* Remove duplicated strings: the same sentence is in "Python tips" and "Module security", keeping the
latter.
* ansible-doc man formatter: do not crash when description isn't there.
* Change to report a better error message when description is not there.
* Add test.
Previous version initialized the `TaskQueueManager` after calling
`Play.load()` while advertising a way to inject a custom library
location path. This caused the tasks loader not to find any custom
modules because it was triggered before the path was actually added
to the module loader.
This patch changes the order of the operations to ensure that the
customized `context.CLIARGS` actually influences things.
Resolves https://github.com/ansible/ansible/issues/69758.
If a local module has no documentation, the doc command will fail without any hints of what is wrong. Add another way to confirm the presence of a local module.
* Update docs/docsite/rst/dev_guide/developing_locally.rst
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
When user uses home directory in --playbook-dir option
of ansible-inventory command, it warns user about this.
This PR suppress the warning message for user's home directory usage
in ansible-inventory command.
Fixes: #65262
Signed-off-by: Satyajit Bulage <sbulage@redhat.com>
The ssh_args were sometimes not correctly applied to the connection
when using _play_context. Use get_option() instead to ensure the
correct ssh_args are always applied.
Change:
- Update bundled six to 1.13 (last with py2.6 support)
- Make it pass lint
- Fix check to allow skipping over compat __init__.py files we authored
- Fix check to allow files that can't be updated for some reason
Test Plan:
- ansible-test sanity --docker
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
* Test against galaxy_ng
* Switch container image
* Remove redundant |default
* Re-enable
* Update image
* Update wording
* Don't use pulp as the container name
* Use the first galaxy server supporting v1 for roles. Fixes#65440
* Add changelog fragment
* This is best effort, fall back to original behavior if something bad happens
* misc collection metadata fixes
* parse collection meta with libyaml if available
* require only Mapping for validation
* add explanatory text for _meta_yml_to_dict
* ignore custom pylint rule
* this code shouldn't import a bunch of stuff from ansible, since it's run under the import sanity test
Change:
- In certain situations, such as when the input string contains null
bytes (\0), syslog.syslog will throw a TypeError. Handle that and
fail_json instead.
Test Plan:
- New test
- ansible-test --docker centos[68] (for py2 and py3 respectively)
Tickets:
- Refs #70269
Signed-off-by: Rick Elrod <rick@elrod.me>
* ansible-test - do not validate blacklisted ps modules
* Update changelogs/fragments/validate-modules-ps-doc-blacklist.yaml
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Clay <matt@mystile.com>
CIFS can be mounted using backward slash as well in /etc/fstab like
\\Windows\share /data/ cifs credentials=/root/.creds 0 0
Handle this condition while gather mount information in Linux.
Fixes: #48813
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Update documentation as per sanity tests
* Added example about custom SSH port in example section
Fixes: #29236
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Ensure not to remove existing packages while installing apt packages.
* Make all lines shorter than 160 characters
* Allow removing packages only when upgrading.
* Add integration tests
Change:
- Fix a UnicodeDecodeError in executor.module_common that could get
triggered with -vvvvv.
Test Plan:
- `ansible-test integration --docker centos7 module_utils -vvvvv`
This would show the error previously, and no loner does after this
patch.
Signed-off-by: Rick Elrod <rick@elrod.me>
In some usecases, we want to be able to clone a single branch
of a repository, without using --depth (which implies --single-branch).
* Use branch name when available
- update description of parameter
- consolidate branch or tag checking for easy reuse
* Add changelog
* Use static task imports rather than dynamic includes
* Add integration tests for single_branch
* Account for older versions of git
* Minor tweak to warnings
Co-authored-by: Laurent Coustet <laurent.coustet@clarisys.fr>
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Update docs/docsite/rst/dev_guide/developing_collections.rst
* add steps to create changelogs, add sentence about not using the tool
* add note for rerunning the command
Co-authored-by: Felix Fontein <felix@fontein.de>
* [Bugfix] curses.setupterm() error
When run playbook in celery task, curses.setupterm() will be failed
```
File "<frozen importlib._bootstrap>", line 675, in _load
File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
File "/Users/guang/.virtualenvs/ansible/lib/python3.6/site-packages/ansible/plugins/action/pause.py", line 45, in <module>
curses.setupterm()
TypeError: argument must be an int, or have a fileno() method.
```
* Add changelog
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Add description of collections keyword
* Update based on feedback.
- Add link to become plugins.
- Add note about how the collections keyword works with roles.
Handle colon appearing in filename while parsing the mimetype and charset
using file command.
Fixes: #70256
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Change:
- On Python 3.9, `crypt.crypt` will throw instead of returning `None`
when the algorithm isn't supported. So we catch that and handle it
the same way we handled the algorithm not being supported on 3.8: by
throwing AnsibleError.
Test Plan:
- CI for <=3.8.
- Local for 3.9b3:
ansible -m debug -a "msg=\"{{ 'changeme' | password_hash('bcrypt') }}\"" localhost
Before:
localhost | FAILED! => {
"msg": "Unexpected failure during module execution.",
"stdout": ""
}
After:
localhost | FAILED! => {
"msg": "crypt.crypt does not support 'bcrypt' algorithm"
}
Tickets:
- Fixes#69930
Signed-off-by: Rick Elrod <rick@elrod.me>
The `packaging` and `pyparsing` packages are now installed by `ansible-test` during provisioning of RHEL instances to match the downstream vendored versions.
Change:
- Remove _get_item() alias as it has been deprecated
- Update tests
- Remove relevant sanity curtailment
- Add changelog
Test Plan:
CI, grep
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
Removes with_* loop squashing and tests for 2.11
Test Plan:
CI, and grepped for with_items in package manager integration targets.
There might be some test cases in collections which need to stop testing
this behavior.
Signed-off-by: Rick Elrod <rick@elrod.me>
Rather than looking through tests.json to find the data file, look for it explicitly
within a given target to avoid problems processing data in tests.json.
* Disable too-many-ancestors pylint check.
This check was only enabled for ansible-test code.
In some cases this check counts 1 more ancestor in python 3.7+ than under python 3.6, making the check inconsistent and unsuitable for use in ansible-test.
* Disable additional pylint checks:
- import-error
- no-name-in-module
These checks were only enabled for ansible-test code.
These checks provide inconsistent results since they are dependent on available imports, which vary between environments.
It may be practical to enable these checks in the future if changes are made to ensure a consistent test environment for pylint.
* Test galaxy cli against pulp
* linting fix
* Renames and small fixes
* Better handling for resetting pulp
* Clean up some things, add a comment
* I can't spell
* Bump fallaxy, use alternate pulp image
* Only reset pulp when we're are executing against pulp
* Update for updated pulp container
* Update some comments with correct URLs and typos
* Linting fix
* Pin pulp-fedora31 to a digest
* Address review comments for documentation
Provide all relevant information below, e.g. target OS versions,
network device firmware, etc.
placeholder:RHEL 8, CentOS Stream etc.
validations:
required:true
- type:textarea
attributes:
label:Steps to Reproduce
description:|
Describe exactly how to reproduce the problem, using a minimal test-case. It would *really* help us understand your problem if you could also pased any playbooks, configs and commands you used.
**HINT:** You can paste https://gist.github.com links for larger files.
value:|
<!--- Paste example playbooks or commands between quotes below -->
```yaml (paste below)
```
placeholder:|
1. Implement the following playbook:
```yaml
---
# ping.yml
- hosts: all
gather_facts: false
tasks:
- ping:
...
```
2. Then run `ANSIBLE_DEBUG=1 ansible-playbook ping.yml -vvvvv`
3. An error occurs.
validations:
required:true
- type:textarea
attributes:
label:Expected Results
description:>-
Describe what you expected to happen when running the steps above.
placeholder:>-
I expected X to happen because I assumed Y and was shocked
that it did not.
validations:
required:true
- type:textarea
attributes:
label:Actual Results
description:|
Describe what actually happened. If possible run with extra verbosity (`-vvvv`).
Paste verbatim command output and don't wrap it with tripple backticks — your
whole input will be turned into a code snippet automatically.
tar -C deb-build/$${DIST} -xvf dist/$(NAME)-$(VERSION).tar.gz ;\
cp -a packaging/debian deb-build/$${DIST}/$(NAME)-$(VERSION)/ ;\
sed -ie "s|%VERSION%|$(DEB_VERSION)|g;s|%RELEASE%|$(DEB_RELEASE)|;s|%DIST%|$${DIST}|g;s|%DATE%|$(DEB_DATE)|g" deb-build/$${DIST}/$(NAME)-$(VERSION)/debian/changelog ;\
We document our Coding Guidelines in the `Developer Guide <https://docs.ansible.com/ansible/devel/dev_guide/>`_. We particularly suggest you review:
*`Contributing your module to Ansible <https://docs.ansible.com/ansible/devel/dev_guide/developing_modules_checklist.html>`_
*`Conventions, tips and pitfalls <https://docs.ansible.com/ansible/devel/dev_guide/developing_modules_best_practices.html>`_
*`Conventions, tips, and pitfalls <https://docs.ansible.com/ansible/devel/dev_guide/developing_modules_best_practices.html>`_
Branch Info
===========
@ -72,7 +69,7 @@ Branch Info
* The ``devel`` branch corresponds to the release actively under development.
* The ``stable-2.X`` branches correspond to stable releases.
* Create a branch based on ``devel`` and set up a `dev environment <https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_general.html#common-environment-setup>`_ if you want to open a PR.
* See the `Ansible release and maintenance <https://docs.ansible.com/ansible/latest/reference_appendices/release_and_maintenance.html>`_ page for information about active branches.
* See the `Ansible release and maintenance <https://docs.ansible.com/ansible/devel/reference_appendices/release_and_maintenance.html>`_ page for information about active branches.
Roadmap
=======
@ -84,7 +81,7 @@ Authors
=======
Ansible was created by `Michael DeHaan <https://github.com/mpdehaan>`_
and has contributions from over 4700 users (and growing). Thanks everyone!
and has contributions from over 5000 users (and growing). Thanks everyone!
`Ansible <https://www.ansible.com>`_ is sponsored by `Red Hat, Inc.
<https://www.redhat.com>`_
@ -96,12 +93,12 @@ GNU General Public License v3.0 or later
- ansible-test - Improve consistency of version specific documentation links.
Bugfixes
--------
- BSD network facts - Do not assume column indexes, look for ``netmask`` and ``broadcast`` for determining the correct columns when parsing ``inet`` line (https://github.com/ansible/ansible/issues/79117)
- ansible-galaxy - make initial call to Galaxy server on-demand only when installing, getting info about, and listing roles.
- ansible-test - Add ``wheel < 0.38.0`` constraint for Python 3.6 and earlier.
- ansible-test - Fix broken documentation link for ``aws`` test plugin error messages.
- copy module will no longer move 'non files' set as src when remote_src=true.
- file lookup now handles missing files more gracefully.
- service_facts - Use python re to parse service output instead of grep (https://github.com/ansible/ansible/issues/78541)
- updated error messages to include 'acl' and not just mode changes when failing to set required permissions on remote.
-``ansible-galaxy`` - remove extra server api call during dependency resolution for requirements and dependencies that are already satisfied (https://github.com/ansible/ansible/issues/77443).
- ansible-test - Allow disabled, unsupported, unstable and destructive integration test targets to be selected using their respective prefixes.
- ansible-test - Allow unstable tests to run when targeted changes are made and the ``--allow-unstable-changed`` option is specified (resolves https://github.com/ansible/ansible/issues/74213).
- apt - Fix module failure when a package is not installed and only_upgrade=True. Skip that package and check the remaining requested packages for upgrades. (https://github.com/ansible/ansible/issues/78762)
- apt module should not traceback on invalid type given as package. issue 78663.
- known_hosts - do not return changed status when a non-existing key is removed (https://github.com/ansible/ansible/issues/78598)
- paramiko - Add back support for ``ssh_args``, ``ssh_common_args``, and ``ssh_extra_args`` for parsing the ``ProxyCommand`` (https://github.com/ansible/ansible/issues/78750)
- plugin loader, fix detection for existing configuration before initializing for a plugin
- Fix for network_cli not getting all relevant connection options
- ansible-galaxy - Fix detection of ``--role-file`` in arguments for implicit role invocation (https://github.com/ansible/ansible/issues/78204)
- ansible-galaxy - Fix exit codes for role search and delete (https://github.com/ansible/ansible/issues/78516)
- ansible-test - Fix change detection for ansible-test's own integration tests.
- ansible-test - ansible-doc sanity test - Correctly determine the fully-qualified collection name for plugins in subdirectories, resolving https://github.com/ansible/ansible/issues/78490.
- apt - don't actually update the cache in check mode with update_cache=true.
- apt - don't mark existing packages as manually installed in check mode (https://github.com/ansible/ansible/issues/66413).
- apt - fix package selection to include /etc/apt/preferences(.d) (https://github.com/ansible/ansible/issues/77969)
- urls - Guard imports of ``urllib3`` by catching ``Exception`` instead of ``ImportError`` to prevent exceptions in the import process of optional dependencies from preventing use of ``urls.py`` (https://github.com/ansible/ansible/issues/78648)
- wait_for - Read file and perform comparisons using bytes to avoid decode errors (https://github.com/ansible/ansible/issues/78214)
- Avoid 'unreachable' error when chmod on AIX has 255 as return code.
- Fix PluginLoader to mimic Python import machinery by adding module to sys.modules before exec
- Fix dnf module documentation to indicate that comparison operators for package version require spaces around them (https://github.com/ansible/ansible/issues/78295)
- ansible-connection - decrypt vaulted parameters before sending over the socket, as vault secrets are not available on the other side.
- ansible-galaxy - Fix reinitializing the whole collection directory with ``ansible-galaxy collection init ns.coll --force``. Now directories and files that are not included in the collection skeleton will be removed.
- ansible-galaxy - do not require mandatory keys in the ``galaxy.yml`` of source collections when listing them (https://github.com/ansible/ansible/issues/70180).
- ansible-galaxy - fix listing collections that contains metadata but the namespace or name are not strings.
- ansible-galaxy - fix setting the cache for paginated responses from Galaxy NG/AH (https://github.com/ansible/ansible/issues/77911).
- ansible-test - Delegation for commands which generate output for programmatic consumption no longer redirect all output to stdout. The affected commands and options are ``shell``, ``sanity --lint``, ``sanity --list-tests``, ``integration --list-targets``, ``coverage analyze``
- ansible-test - Delegation now properly handles arguments given after ``--`` on the command line.
- ansible-test - Test configuration for collections is now parsed only once, prior to delegation. Fixes issue: https://github.com/ansible/ansible/issues/78334
- ansible-test - The ``shell`` command no longer redirects all output to stdout when running a provided command. Any command output written to stderr will be mixed with the stderr output from ansible-test.
- ansible-test - The ``shell`` command no longer requests a TTY when using delegation unless an interactive shell is being used. An interactive shell is the default behavior when no command is given to pass to the shell.
- dnf - fix output parsing on systems with ``LANGUAGE`` set to a language other than English (https://github.com/ansible/ansible/issues/78193)
- if a config setting prevents running ansible it should at least show it's "origin".
- prevent type annotation shim failures from causing runtime failures (https://github.com/ansible/ansible/pull/77860)
- template module/lookup - fix ``convert_data`` option that was effectively always set to True for Jinja macros (https://github.com/ansible/ansible/issues/78141)
- uri - properly use uri parameter use_proxy (https://github.com/ansible/ansible/issues/58632)
- yum - fix traceback when ``releasever`` is specified with ``latest`` (https://github.com/ansible/ansible/issues/78058)
- ansible-test - An improved error message is shown when the download of a pip bootstrap script fails. The download now uses ``urllib2`` instead of ``urllib`` on Python 2.
Bugfixes
--------
- Move undefined check from concat to finalize (https://github.com/ansible/ansible/issues/78156)
- ansible-doc - no longer list module and plugin aliases that are created with symlinks (https://github.com/ansible/ansible/pull/78137).
- ansible-doc - when listing modules in collections, proceed recursively. This fixes module listing for community.general 5.x.y and community.network 4.x.y (https://github.com/ansible/ansible/pull/78137).
- ansible-doc will not add 'website for' in ":ref:" substitutions as it made them confusing.
- file backed cache plugins now handle concurrent access by making atomic updates to the files.
- password lookup does not ignore k=v arguments anymore.
- user - Fix error "Permission denied" in user module while generating SSH keys (https://github.com/ansible/ansible/issues/78017).
will end up with defaults for eos_facts and eos_command
since both modules redirect to the same action.
To select an action plugin for a module without merging
module_defaults, define an action_plugin field for the resolved
module in the runtime.yml.
.. code: yaml
plugin_routing:
modules:
facts:
redirect: ns.coll.eos_facts
action_plugin: ns.coll.eos
command:
redirect: ns.coll.eos_command
action_plugin: ns.coll.eos
The action_plugin field can be a redirected action plugin, as
it is resolved normally.
Using the modified runtime.yml, the example task will only use
the ns.coll.eos_facts defaults.
- ansible-galaxy - Support resolvelib versions 0.6.x, 0.7.x, and 0.8.x. The full range of supported versions is now >= 0.5.3, < 0.9.0.
- ansible-test - Add RHEL 9.0 remote support.
- ansible-test - Add support for Ubuntu VMs using the ``--remote`` option.
- ansible-test - Add support for exporting inventory with ``ansible-test shell --export {path}``.
- ansible-test - Add support for multi-arch remotes.
- ansible-test - Add support for running non-interactive commands with ``ansible-test shell``.
- ansible-test - Avoid using the ``mock_use_standalone_module`` setting for unit tests running on Python 3.8 or later.
- ansible-test - Blocking mode is now enforced for stdin, stdout and stderr. If any of these are non-blocking then ansible-test will exit during startup with an error.
- ansible-test - Improve consistency of output messages by using stdout or stderr for most output, but not both.
- ansible-test - The ``shell`` command can be used outside a collection if no controller delegation is required.
Bugfixes
--------
- Add PyYAML >= 5.1 as a dependency of ansible-core to be compatible with Python 3.8+.
- ansible-config dump - Only display plugin type headers when plugin options are changed if --only-changed is specified.
- ansible-galaxy - handle unsupported versions of resolvelib gracefully.
- ansible-test - Fix internal validation of remote completion configuration.
- ansible-test - Prevent ``--target-`` prefixed options for the ``shell`` command from being combined with legacy environment options.
- ansible-test - Sanity test output with the ``--lint`` option is no longer mixed in with bootstrapping output.
- ansible-test - Subprocesses are now isolated from the stdin, stdout and stderr of ansible-test. This avoids issues with subprocesses tampering with the file descriptors, such as SSH making them non-blocking. As a result of this change, subprocess output from unit and integration tests on stderr now go to stdout.
- ansible-test - Subprocesses no longer have access to the TTY ansible-test is connected to, if any. This maintains consistent behavior between local testing and CI systems, which typically do not provide a TTY. Tests which require a TTY should use pexpect or another mechanism to create a PTY.
- apt module now correctly handles virtual packages.
- plugin loader will now load config data for plugin by name instead of by file to avoid issues with the same file being loaded under different names (fqcn + short name).
- psrp connection now handles default to inventory_hostname correctly.
- winrm connection now handles default to inventory_hostname correctly.
- Jinja2 Controller Requirement - Jinja2 3.0.0 or newer is required for the control node (the machine that runs Ansible) (https://github.com/ansible/ansible/pull/75881)
- Templating - remove ``safe_eval`` in favor of using ``NativeEnvironment`` but utilizing ``literal_eval`` only in cases when ``safe_eval`` was used (https://github.com/ansible/ansible/pull/75587)
Minor Changes
-------------
- Action Plugins - Add helper method for argument spec validation, and extend to pause and async_wrapper
- Added AIX root CA certs folders - enhance the TLS support in ``uri`` task on AIX
- Added ``module_utils.compat.typing`` to facilitate type-hinting on all supported Python versions.
- Ansible.Basic - small changes to allow use in PowerShell modules running on non-Windows platforms (https://github.com/ansible/ansible/pull/76924).
- AnsibleModule.run_command() now has a toggle to allow caller to decide to handle exceptions from executing the command itself
- Attach concat function to an environment class (https://github.com/ansible/ansible/pull/76282)
- Clarify in a comment that unrolling an iterator in ``Templar._finalize`` is actually necessary. Also switch to using the ``_unroll_iterator`` decorator directly to deduplicate code in ``Templar._finalize``. (https://github.com/ansible/ansible/pull/76436)
- Installation - modernize our python installation, to reduce dynamic code in setup.py, and migrate what is feasible to setup.cfg. This will enable shipping wheels in the future.
- PlayIterator - introduce public methods to access ``PlayIterator._host_states`` (https://github.com/ansible/ansible/pull/74416)
- PlayIterator - use enums for Iterating and Failed states (https://github.com/ansible/ansible/pull/74511)
- Reduce number of iterations through PlayIterator (https://github.com/ansible/ansible/pull/74175)
- Remove more Python 2.x compatibility code from controller (https://github.com/ansible/ansible/pull/77320).
- Start of moving away from using Six, Python 2 and 3 compatibility library (https://github.com/ansible/ansible/pull/75863)
- The collection loader now reports a Python warning if an attempt is made to install the Ansible collection loader a second time. Previously this condition was reported using an Ansible warning.
-``ansible-galaxy collection [install|verify]`` - allow user-provided signature sources in addition to those from the Galaxy server. Each collection entry in a requirements file can specify a ``signatures`` key followed by a list of sources. Collection name(s) provided on the CLI can specify additional signature sources by using the ``--signatures`` CLI option. Signature sources should be URIs that can be opened with ``urllib.request.urlopen()``, such as "https://example.com/path/to/detached_signature.asc" or "file:///path/to/detached_signature.asc". The ``--keyring`` option must be specified if signature sources are provided.
-``ansible-galaxy collection [install|verify]`` - use gpg to verify the authenticity of the signed ``MANIFEST.json`` with ASCII armored detached signatures provided by the Galaxy server. The keyring (which is not managed by ``ansible-galaxy``) must be provided with the ``--keyring`` option to use signature verification. If no ``--keyring`` is specified and the collection to ``install|verify`` has associated detached signatures on the Galaxy server, a warning is provided.
-``ansible-galaxy collection install`` - Add a global configuration to modify the required number of signatures that must verify the authenticity of the collection. By default, the number of required successful signatures is 1. Set this option to ``all`` to require all signatures verify the collection. To ensure signature verification fails if there are no valid signatures, prepend the value with '+', such as ``+all`` or ``+1``.
-``ansible-galaxy collection install`` - Add a global ignore list for gpg signature errors. This can be used to ignore certain signatures when the number of required successful signatures is all. When the required number of successful signatures is a positive integer, the only effect this has is to display fewer errors to the user on failure (success is determined by having the minimum number of successful signatures, in which case all errors are disregarded).
-``ansible-galaxy collection install`` - Add a global toggle to turn off GPG signature verification.
-``ansible-galaxy collection install`` - Store Galaxy server metadata alongside installed collections for provenance. Signatures obtained from the Galaxy server can be used for offline verification with ``ansible-galaxy collection verify --offline``.
- ansible-console - Provide a way to customize the stdout callback
- ansible-doc metadata dump - add option ``--no-fail-on-errors`` which allows to not fail the ansible-doc invocation when errors happen during docs parsing or processing. Instead they are reported in the JSON result in an ``error`` key for the affected plugins (https://github.com/ansible/ansible/pull/77035).
- ansible-galaxy - the option to skip certificate verification now also applies when cloning via SCM (git/hg) (https://github.com/ansible/ansible/issues/41077)
- ansible-test - Accept new-style Python modules without a shebang.
- ansible-test - Add ``--version`` support to show the ansible-core version.
- ansible-test - Add support for ``rhel/8.5`` remote instances.
- ansible-test - Add support for remote testing of FreeBSD 12.3.
- ansible-test - Add support for running container tests with ``podman remote`` (https://github.com/ansible/ansible/pull/75753)
- ansible-test - Added the ``fedora35`` test container.
- ansible-test - Change the maximum number of open files in a test container from the default to ``10240``.
- ansible-test - Declare public dependencies of ansible-core and use to limit unguarded imports in plugins.
- ansible-test - Importing ``distutils`` now results in an error.
- ansible-test - Installation of ``cryptography`` is no longer version constrained when ``openssl`` 1.1.0 or later is installed.
- ansible-test - Miscellaneous code cleanup and type hint fixes.
- ansible-test - PowerShell in the ``base`` and ``default`` containers has been upgraded to version 7.1.4.
- ansible-test - Removed support for Sherlock from the Azure provisioning plugin.
- ansible-test - Removed used ``MarkupSafe`` constraint for Python 3.5 and earlier.
- ansible-test - Requirements for the plugin import test are now frozen.
- ansible-test - Shellcheck in the ``base`` and ``default`` containers has been upgraded to version 0.7.0.
- ansible-test - Stop early with an error if the current working directory contains an invalid collection namespace or name.
- ansible-test - The ``--help`` option is now available when an unsupported cwd is in use.
- ansible-test - The ``--help`` output now shows the same instructions about cwd as would be shown in error messages if the cwd is unsupported.
- ansible-test - The ``pip`` and ``wheel`` packages are removed from all sanity test virtual environments after installation completes to reduce their size. Previously they were only removed from the environments used for the ``import`` sanity test.
- ansible-test - The explanation about cwd usage has been improved to explain more clearly what is required.
- ansible-test - The hash for all managed sanity test virtual environments has changed. Containers that include ``ansible-test sanity --prime-venvs`` will need to be rebuilt to continue using primed virtual environments.
- ansible-test - Update ``base`` container to version 2.1.0.
- ansible-test - Update ``base`` container to version 2.2.0.
- ansible-test - Update ``default`` containers to version 5.2.0.
- ansible-test - Update ``default`` containers to version 5.4.0.
- ansible-test - Update ``default`` containers to version 5.5.0.
- ansible-test - Update ``default`` containers to version 5.6.2.
- ansible-test - Update ``default`` containers to version 5.7.0.
- ansible-test - Update ``default`` containers to version 5.8.0.
- ansible-test - Update ``default`` containers to version 5.9.0.
- ansible-test - Update ``pip`` used to bootstrap remote FreeBSD instances from version 20.3.4 to 21.3.1.
- ansible-test - Update sanity test requirements.
- ansible-test - Update the NIOS test plugin container to version 1.4.0.
- ansible-test - Update the ``alpine`` container to version 3.3.0. This updates the base image from 3.14.2 to 3.15.0, which includes support for installing binary wheels using pip.
- ansible-test - Update the ``base`` and ``default`` containers from Python 3.10.0rc2 to 3.10.0.
- ansible-test - Update the ``base`` and ``default`` containers from a Ubuntu 18.04 to Ubuntu 20.04 base image.
- ansible-test - Update the ``default`` containers to version 5.1.0.
- ansible-test - Update the ``galaxy`` test plugin to get its container from a copy on quay.io.
- ansible-test - Update the ``openshift`` test plugin to get its container from a copy on quay.io.
- ansible-test - Use Python 3.10 as the default Python version for the ``base`` and ``default`` containers.
- ansible-test - add macOS 12.0 as a remote target (https://github.com/ansible/ansible/pull/76328)
- ansible-test pslint - Added the `AvoidLongLines <https://github.com/PowerShell/PSScriptAnalyzer/blob/master/docs/Rules/AvoidLongLines.md>`_ rule set to a length of 160.
- ansible-test pslint - Added the `PlaceCloseBrace <https://github.com/PowerShell/PSScriptAnalyzer/blob/master/docs/Rules/PlaceCloseBrace.md>`_ rule set to enforce close braces on a newline.
- ansible-test pslint - Added the `PlaceOpenBrace <https://github.com/PowerShell/PSScriptAnalyzer/blob/master/docs/Rules/PlaceOpenBrace.md>`_ rule set to enforce open braces on the same line and a subsequent newline.
- ansible-test pslint - Added the `UseConsistentIndentation <https://github.com/PowerShell/PSScriptAnalyzer/blob/master/docs/Rules/UseConsistentIndentation.md>`_ rule to enforce indentation is done with 4 spaces.
- ansible-test pslint - Added the `UseConsistentWhitespace <https://github.com/PowerShell/PSScriptAnalyzer/blob/master/docs/Rules/UseConsistentWhitespace.md>`_ rule to enforce whitespace consistency in PowerShell.
- ansible-test pslint - Updated ``PowerShellScriptAnalyzer`` to 1.20.0
- ansible-test sanity validate-modules - the validate-modules sanity test now also checks the documentation of documentable plugin types (https://github.com/ansible/ansible/pull/71734).
- ansible-test validate-modules sanity test - add more schema checks to improve quality of plugin documentation (https://github.com/ansible/ansible/pull/77268).
- ansible-test validate_modules - allow ``choices`` for return values (https://github.com/ansible/ansible/pull/76009).
- apt - Add support for using ">=" in package version number matching.
- apt - Adds APT option ``--allow-change-held-packages`` as module parameter ``allow_change_held_packages`` to allow APT up- or downgrading a package which is on APTs hold list (https://github.com/ansible/ansible/issues/65325)
- auto inventory plugin will now give plugin loading information on verbose output
- callbacks - Add result serialization format options to ``_dump_results`` allowing plugins such as the ``default`` callback to emit ``YAML`` serialized task results in addition to ``JSON``
- dnf - add more specific error message for GPG validation (https://github.com/ansible/ansible/issues/76192)
- env lookup, add default option
- facts - report prefix length for IPv4 addresses in Linux network facts.
- get_parsable_locale now logs result when in debug mode.
- git - display the destination directory path in error msg when local_mods detects local modifications conflict so that users see the exact location
- iptables - add the ``chain_management`` parameter that controls iptables chain creation and deletion
- jinja2_native - keep same behavior on Python 3.10.
- junit callback - Add support for replacing the directory portion of out-of-tree relative task paths with a placeholder.
- k8s - scenario guides for kubernetes migrated to ``kubernetes.core`` collection.
- module_utils.distro - Add missing ``typing`` import from original code.
- package_facts - add pkg_info support for OpenBSD and NetBSD (https://github.com/ansible/ansible/pull/76580)
- services_facts - Add support for openrc (https://github.com/ansible/ansible/pull/76373).
- setting DEFAULT_FACT_PATH is being deprecated in favor of the generic module_defaults keyword
- uri - Avoid reading the response body when not needed
- uri - Eliminate multiple requests to determine the final URL for file naming with ``dest``
- validate-modules - do some basic validation on the ``M(...)``, ``U(...)``, ``L(..., ...)`` and ``R(..., ...)`` documentation markups (https://github.com/ansible/ansible/pull/76262).
- vmware - migrated vmware scenario guides to `community.vmware` repo.
- yum, dnf - add sslverify option to temporarily disable certificate validation for a repository
Breaking Changes / Porting Guide
--------------------------------
- Module Python Dependency - Drop support for Python 2.6 in module execution.
- Templating - it is no longer allowed to perform arithmetic and concatenation operations outside of the jinja template (https://github.com/ansible/ansible/pull/75587)
- The ``finalize`` method is no longer exposed in the globals for use in templating.
Deprecated Features
-------------------
- ansible-core - Remove support for Python 2.6.
- ansible-test - Remove support for Python 2.6.
- ssh connection plugin option scp_if_ssh in favor of ssh_transfer_method.
- cli - remove deprecated ability to set verbosity before the sub-command (https://github.com/ansible/ansible/issues/75823)
- copy - remove deprecated ``thirsty`` alias (https://github.com/ansible/ansible/issues/75824)
- psrp - Removed fallback on ``put_file`` with older ``pypsrp`` versions. Users must have at least ``pypsrp>=0.4.0``.
- url_argument_spec - remove deprecated ``thirsty`` alias for ``get_url`` and ``uri`` modules (https://github.com/ansible/ansible/issues/75825, https://github.com/ansible/ansible/issues/75826)
Security Fixes
--------------
- Do not include params in exception when a call to ``set_options`` fails. Additionally, block the exception that is returned from being displayed to stdout. (CVE-2021-3620)
Bugfixes
--------
- Add a YAML representer for ``NativeJinjaText``
- Add a YAML representer for ``NativeJinjaUnsafeText``
- AnsiballZ - Ensure we use the full python package in the module cache filename to avoid a case where ``collections:`` is used to execute a module via short name, where the short name duplicates another module from ``ansible.builtin`` or another collection that was executed previously.
- Ansible.ModuleUtils.LinkUtil - Ignore the ``LIB`` environment variable when loading the ``LinkUtil`` code
- Ansible.ModuleUtils.SID - Use user principal name as is for lookup in the ``Convert-ToSID`` function - https://github.com/ansible/ansible/issues/77316
- Detect package manager for Amazon Linux 2022 (AL2022) as dnf
- Ensure the correct ``environment_class`` is set on ``AnsibleJ2Template``
- Fix ``AttributeError`` when providing password file via ``--connection-password-file`` (https://github.com/ansible/ansible/issues/76530)
- Fix ``end_play`` to end the current play only (https://github.com/ansible/ansible/issues/76672)
- Fix executing includes in the always section in the free strategy (https://github.com/ansible/ansible/issues/75642)
- Fix for when templating empty template file resulted in file with string 'None' (https://github.com/ansible/ansible/issues/76610)
- Fix help message for the 'ansible-galaxy collection verify' positional argument. The positional argument must be a collection name (https://github.com/ansible/ansible/issues/76087).
- Fix module logging issue when using custom module on WSL2 (https://github.com/ansible/ansible/issues/76320)
- Fix task debugger to work with ``run_once`` using ``linear`` strategy (https://github.com/ansible/ansible/issues/76049)
- Fix traceback when installing a collection from a git repository and git is not installed (https://github.com/ansible/ansible/issues/77479).
- Interpreter Discovery - Fallback to OS family if the distro is not found in ``INTERPRETER_PYTHON_DISTRO_MAP`` (https://github.com/ansible/ansible/issues/75560)
- Interpreter discovery - Add ``RHEL`` to ``OS_FAMILY_MAP`` for correct family fallback for interpreter discovery (https://github.com/ansible/ansible/issues/77368)
- Make include_role/include_tasks work with any_errors_fatal (https://github.com/ansible/ansible/issues/50897)
- Parser errors from within includes should not be rescueable (https://github.com/ansible/ansible/issues/73657)
- Prevent losing unsafe on results returned from lookups (https://github.com/ansible/ansible/issues/77535)
- Templating - Ensure we catch exceptions when getting ``.filters`` and ``.tests`` attributes on their respective plugins and properly error, instead of aborting which results in no filters being added to the jinja2 environment
- Trigger an undefined error when an undefined variable is detected within a dictionary and/or list (https://github.com/ansible/ansible/pull/75587)
- _run_loop - Add the task name to the warning (https://github.com/ansible/ansible/issues/76011)
-``Templar.copy_with_new_env`` - set the ``finalize`` method of the new ``Templar`` object for the new environment (https://github.com/ansible/ansible/issues/76379)
- add_host/group_by: fix using changed_when in a loop (https://github.com/ansible/ansible/issues/71627)
- ansible - Exclude Python 2.6 from Python interpreter discovery.
- ansible-config avoid showing _terms and _input when --only-changed.
- ansible-galaxy - Fix using the '--ignore-certs' option when there is no server-specific configuration for the Galaxy server.
- ansible-galaxy - installing/downloading collections with invalid versions in git repositories and directories now gives a formatted error message (https://github.com/ansible/ansible/issues/76425, https://github.com/ansible/ansible/issues/75404).
- ansible-galaxy - when installing a role properly raise an error when inaccessible path is specified multiple times in roles_path (e.g. via environment variable and command line option) (https://github.com/ansible/ansible/issues/76316)
- ansible-galaxy collection build - Ignore any existing ``MANIFEST.json`` and ``FILES.json`` in the root directory when building a collection.
- ansible-galaxy collection verify - display files/directories not included in the FILES.json as modified content.
- ansible-galaxy publish - Fix warning and error detection in import messages to properly display them in Ansible
- ansible-pull handle case where hostname and nodename are empty
- ansible-test - Add default entry for Windows remotes to be used with unknown versions.
- ansible-test - All virtual environments managed by ansible-test are marked as usable after being bootstrapped, to avoid errors caused by use of incomplete environments. Previously this was only done for sanity tests. Existing environments from previous versions of ansible-test will be recreated on demand due to lacking the new marker.
- ansible-test - Automatic target requirements installation is now based on the target environment instead of the controller environment.
- ansible-test - Correctly detect when running as the ``root`` user (UID 0) on the origin host. The result of the detection was incorrectly being inverted.
- ansible-test - Don't fail if network cannot be disconnected (https://github.com/ansible/ansible/pull/77472)
- ansible-test - Fix Python real prefix detection when running in a ``venv`` virtual environment.
- ansible-test - Fix ``windows-integration`` and ``network-integration`` when used with the ``--docker`` option and user-provided inventory.
- ansible-test - Fix installation and usage of ``pyyaml`` requirement for the ``import`` sanity test for collections.
- ansible-test - Fix path to inventory file for ``windows-integration`` and ``network-integration`` commands for collections.
- ansible-test - Fix plugin loading.
- ansible-test - Fix skipping of tests marked ``needs/python`` on the origin host.
- ansible-test - Fix skipping of tests marked ``needs/root`` on the origin host.
- ansible-test - Fix the ``import`` sanity test to work properly when Ansible's built-in vendoring support is in use.
- ansible-test - Fix the ``validate-modules`` sanity test to avoid double-loading the collection loader and possibly failing on import of the ``packaging`` module.
- ansible-test - Fix traceback in ``import`` sanity test on Python 2.7 when ``pip`` is not available.
- ansible-test - Fix traceback in the ``validate-modules`` sanity test when testing an Ansible module without any callables.
- ansible-test - Fix traceback when running from an install and delegating execution to a different Python interpreter.
- ansible-test - Fix traceback when using the ``--all`` option with PowerShell code coverage.
- ansible-test - Fix type hints.
- ansible-test - Import ``yaml.cyaml.CParser`` instead of ``_yaml.CParser`` in the ``yamllint`` sanity test.
- ansible-test - Limit ``paramiko`` installation to versions before 2.9.0. This is required to maintain support for systems which do not support RSA SHA-2 algorithms.
- ansible-test - Pylint Deprecated Plugin - Use correct message symbols when date or version is not a float or str (https://github.com/ansible/ansible/issues/77085)
- ansible-test - Relocate constants to eliminate symlink.
- ansible-test - Replace the directory portion of out-of-tree paths in JUnit files from integration tests with the ``out-of-tree:`` prefix.
- ansible-test - Sanity tests run with the ``--requirements` option for Python 2.x now install ``virtualenv`` when it is missing or too old. Previously it was only installed if missing. Version 16.7.12 is now installed instead of the latest version.
- ansible-test - Set the ``pytest`` option ``--rootdir`` instead of letting it be auto-detected.
- ansible-test - Show an error message instead of a traceback when running outside of a supported directory.
- ansible-test - Target integration test requirements are now correctly installed for target environments running on the controller.
- ansible-test - The ``import`` sanity test no longer reports errors about ``packaging`` being missing when testing collections.
- ansible-test - Update distribution test containers to version 3.1.0.
- ansible-test - Update help links to reference ``ansible-core`` instead of ``ansible``.
- ansible-test - Update unit tests to use the ``--forked`` option instead of the deprecated ``--boxed`` option.
- ansible-test - Use https://ci-files.testing.ansible.com/ for instance bootstrapping instead of an S3 endpoint.
- ansible-test - Use relative paths in JUnit files generated during integration test runs.
- ansible-test - Use the correct variable to reference the client's SSH key when generating inventory.
- ansible-test - Use the legacy collection loader for ``import`` sanity tests on target-only Python versions.
- ansible-test - Virtual environments managed by ansible-test now use consistent versions of ``pip``, ``setuptools`` and ``wheel``. This avoids issues with virtual environments containing outdated or dysfunctional versions of these tools. The initial bootstrapping of ``pip`` is done by ansible-test from an HTTPS endpoint instead of creating the virtual environment with it already present.
- ansible-test - fix a typo in validate-modules.
- ansible-test - fixed support container failures (eg http-test-container) under podman
- ansible-test compile sanity test - do not crash if a column could not be determined for an error (https://github.com/ansible/ansible/pull/77465).
- ansible-test pslint - Fix error when encountering validation results that are highly nested - https://github.com/ansible/ansible/issues/74151
- ansible-vault encrypt_string - fix ``--output`` option to correctly write encrypted string into given file (https://github.com/ansible/ansible/issues/75101)
- ansible_facts.devices - Fix parsing of device serial number detected via sg_inq for rhel8 (https://github.com/ansible/ansible/issues/75420)
- apt - fails to deploy deb file to old debian systems using python-apt < 0.8.9 (https://github.com/ansible/ansible/issues/47277)
- arg_spec - Fix incorrect ``no_log`` warning when a parameter alias is used (https://github.com/ansible/ansible/pull/77576)
- async - Improve performance of sending async callback events by never sending the full task through the queue (https://github.com/ansible/ansible/issues/76729)
- catch the case that cowsay is broken which would lead to missing output
- cleaning facts will now only warn about the variable name and not post the content, which can be undesireable to disclose
- collection_loader - Implement 'find_spec' and 'exec_module' to override deprecated importlib methods 'find_module' and 'load_module' when applicable (https://github.com/ansible/ansible/issues/74660).
- correctly inherit vars from parent in block (https://github.com/ansible/ansible/issues/75286).
- default callback - Ensure we compare FQCN also in lockstep logic, to ensure using the FQCN of a strategy plugin triggers the correct behavior in the default callback plugin. (https://github.com/ansible/ansible/issues/76782)
- distribution - add EuroLinux to fact gathering (https://github.com/ansible/ansible/pull/76624).
- distribution - detect tencentos and gather correct facts on the distro.
- dnf - ensure releasever is passed into libdnf as string (https://github.com/ansible/ansible/issues/77010)
- extend timeout for ansible-galaxy when communicating with the galaxy server api, and apply it to all interactions with the api
- facts - add support for deepin distro information detection (https://github.com/ansible/ansible/issues/77286).
- first_found - fix to allow for spaces in file names (https://github.com/ansible/ansible/issues/77136)
- gather_facts - Fact gathering now continues even if it fails to read a file
- gather_facts action now handles the move of base connection plugin types into collections to add/prevent subset argument correctly
- gather_facts/setup will not fail anymore if capsh is present but not executable
- git module fix docs and proper use of ssh wrapper script and GIT_SSH_COMMAND depending on version.
- git module is more consistent and clearer about which ssh options are added to git calls.
- git module no longer uses wrapper script for ssh options.
- hacking - fix incorrect usage of deprecated fish-shell redirection operators that failed to honor ``--quiet`` flag when sourced (https://github.com/ansible/ansible/pull/77180).
- hostname - Do not require SystemdStrategy subclasses for every distro (https://github.com/ansible/ansible/issues/76792)
- hostname - Fix Debian strategy KeyError, use `SystemdStrategy` (https://github.com/ansible/ansible/issues/76124)
- hostname - Update the systemd strategy in the ``hostname`` module to not interfere with NetworkManager (https://github.com/ansible/ansible/issues/76958)
- hostname - add hostname support for openEuler distro (https://github.com/ansible/ansible/pull/76619).
- hostname - use ``file_get_content()`` to read the file containing the host name in the ``FileStrategy.get_permanent_hostname()`` method. This prevents a ``TypeError`` from being raised when the strategy is used (https://github.com/ansible/ansible/issues/77025).
- include_vars, properly initialize variable as there is corner case in which it can end up referenced and not defined
- inventory - parameterize ``disable_lookups`` in ``Constructable`` (https://github.com/ansible/ansible/issues/76769).
- inventory manager now respects --flush-cache
- junit callback - Fix traceback during automatic fact gathering when using relative paths.
- module_utils.common.yaml - The ``SafeLoader``, ``SafeDumper`` and ``Parser`` classes now fallback to ``object`` when ``yaml`` is not available. This fixes tracebacks when inheriting from these classes without requiring a ``HAS_YAML`` guard around class definitions.
- parameters - handle blank values when argument is a list (https://github.com/ansible/ansible/issues/77108).
- play_context now compensates for when a conneciton sets the default to inventory_hostname but skips adding it to the vars.
- playbook/strategy have more informative 'attribute' based errors for playbook objects and handlers.
- python modules (new type) will now again prefer the specific python stated in the module's shebang instead of hardcoding to /usr/bin/python.
- replace - Always return ``rc`` to ensure return values are consistent - https://github.com/ansible/ansible/pull/71963
- script - skip in check mode if the plugin cannot determine if a change will occur (i.e. neither `creates` or `removes` are provided).
- service - Fixed handling of sleep arguments during service restarts on AIX. (https://github.com/ansible/ansible/issues/76877)
- service - Fixed service restarts with arguments on AIX. (https://github.com/ansible/ansible/issues/76840)
- service_facts module will now give more meaningful warnings when it fails to gather data.
- set_fact/include_vars correctly handle delegation assignments within loops
- setup - detect docker container with check for ./dockerenv or ./dockinit (https://github.com/ansible/ansible/pull/74349).
- shell/command - only return changed as True if the task has not been skipped.
- shell/command - only skip in check mode if the options `creates` and `removes` are both None.
- ssh connection - properly quote controlpersist path given by user to avoid issues with spaces and other characters
- ssh connection avoid parsing ssh cli debug lines as they can match expected output at high verbosities.
- ssh connection now uses more correct host source as play_context can ignore loop/delegation variations.
- sudo become plugin, fix handling of non interactive flags, previous substitution was too naive
- systemd - check if service is alias so it gets enabled (https://github.com/ansible/ansible/issues/75538).
- systemd - check if service is indirect so it gets enabled (https://github.com/ansible/ansible/issues/76453).
- task_executor reverts the change to push facts into delegated vars on loop finalization as result managing code already handles this and was duplicating effort to wrong result.
- to_json/to_nice_json filters defaults back to unvaulting/no unsafe packing.
- unarchive - Fix zip archive file listing that caused issues with content postprocessing (https://github.com/ansible/ansible/issues/76067).
- unarchive - Make extraction work when the LANGUAGE environment variable is set to a non-English locale.
- unarchive - apply ``owner`` and ``group`` permissions to top folder (https://github.com/ansible/ansible/issues/35426)
- unarchive - include the original error when a handler cannot manage the archive (https://github.com/ansible/ansible/issues/28977).
- unarchive - the ``io_buffer_size`` option added in 2.12 was not accepted by the module (https://github.com/ansible/ansible/pull/77271).
- urls - Allow ``ca_path`` to point to a bundle containing multiple PEM certs (https://github.com/ansible/ansible/issues/75015)
- urls/uri - Address case where ``HTTPError`` isn't fully initialized due to the error, and is missing certain methods and attributes (https://github.com/ansible/ansible/issues/76386)
- user - allow ``password_expiry_min`` and ``password_expiry_min`` to be set to ``0`` (https://github.com/ansible/ansible/issues/75017)
- user - allow password min and max to be set at the same time (https://github.com/ansible/ansible/issues/75017)
- user - update logic to check if user exists or not in MacOS.
- validate_argument_spec - Skip suboption validation if the top level option is an invalid type (https://github.com/ansible/ansible/issues/75612).
- variablemanager, more efficient read of vars files
- vault - Warn instead of fail for missing vault IDs if at least one valid vault secret is found.
- winrm - Ensure ``kinit`` is run with the same ``PATH`` env var as the Ansible process
- yum - prevent storing unnecessary cache data by running `yum makecache fast` (https://github.com/ansible/ansible/issues/76336)
Known Issues
------------
- get_url - document ``check_mode`` correctly with unreliable changed status (https://github.com/ansible/ansible/issues/65687).
- ssh - connection plugin now supports a new variable ``sshpass_prompt`` which gets passed to ``sshpass`` allowing the user to set a custom substring to search for a password prompt (requires sshpass 1.06+)
- apt_repository - Implemented an exponential backoff behaviour when retrying to update the apt cache with new params ``update_cache_retry_max_delay`` and ``update_cache_retries`` to control the behavior.
- combine filter - ``[dict1, [dict2]] | combine`` now raise an error; previously ``combine`` had an undocumented behaviour where it was flattening the list before combining it (https://github.com/ansible/ansible/pull/57894#discussion_r339517518).
minor_changes:
- combine filter - now accept a ``list_merge`` argument which modifies its behaviour when the hashes to merge contain arrays/lists.
- CLI - the `ANSIBLE_PLAYBOOK_DIR` envvar or `playbook_dir` config can now substitute for the --playbook-dir arg on CLIs that support it (https://github.com/ansible/ansible/issues/59464)
- cron and cronvar - use get_bin_path utility to locate the default crontab executable instead of the hardcoded /usr/bin/crontab. (https://github.com/ansible/ansible/pull/59765)
- apt - Implemented an exponential backoff behaviour when retrying to update the cache with new params ``update_cache_retry_max_delay`` and ``update_cache_retries`` to control the behavior.
- Handle exception encountered while parsing the argument description in module when invoked via ansible-doc command (https://github.com/ansible/ansible/issues/60587).
- vars plugins - Support vars plugins in collections by adding the ability to whitelist plugins.
- host_group_vars plugin - Require whitelisting and whitelist by default.
- Add a global toggle to control when vars plugins are executed (per task by default for backward compatibility or after importing inventory).
- Add a per-plugin stage option to override the global toggle to control the execution of individual vars plugins (per task, after inventory, or both).
- script inventory plugin - The 'cache' option is deprecated and will be removed in 2.12. Its use has been removed from the plugin since it has never had any effect.
- "make ``no_log=False`` on a module option silence the ``no_log`` warning (https://github.com/ansible/ansible/issues/49465 https://github.com/ansible/ansible/issues/64656)"
- fix wrong command line length calculation in ``ansible-console`` when long command inputted
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.