* Improve the error handling code
Rather than multiple return paths, have a single return and set the message based
on the type of failure.
* Add another test for non-specific failures
* Reorganize tests so failure tests are in one tasks file
* Remove os.stat() call and add changelog.
(cherry picked from commit afe6eb574e)
Co-authored-by: Sam Doran <sdoran@redhat.com>
* module_common - handle cache directory creation collision
Occasionally multiple workers can try to create the cache directory if it exists. Catch the exception
and handle it.
* Just re-raise if it doesn't exist rather than trying again
* Use exists_ok rather than handling an exception
* Remove unused import and unused variable
* Go back to try/except but with OSError
Since we do not have split controller remote, this needs to be able to run in Python 2 in order
to be merged currently.
(cherry picked from commit 16a74b798c)
Co-authored-by: Sam Doran <sdoran@redhat.com>
ansible-test aws provider now creates and exposes a new tiny_prefix
variable to provide a shorter prefix for the AWS tests.
(cherry picked from commit 5b8fb4dcd3)
* Ensure we keep the original path as per cli
fixes#75126
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit 60c9f045cc)
Since the task and connection both have the same 'retries' keyword, the task default
would override the connection value.
Do not pass 'retries' from the task to the connection options.
* Set ssh_connection retries default value back to 0
It was 0 before the move to config and was changed to 3 by accident.
(cherry picked from commit a8de35e131)
Co-authored-by: Sam Doran <sdoran@redhat.com>
This version relies on an external lib implementing lexers and styles.
That lib pulls in proper Pygments version so this patch also drops it
from direct requirements.
(cherry picked from commit d75d9443fc)
Co-authored-by: Sviatoslav Sydorenko <webknjaz@redhat.com>
* Use `sphinx_ansible_theme` Sphinx theme in docs (#74318)
(cherry picked from commit 346c7a765d)
* 🔥 Drop unused `core.css` file
This is a forgotten leftover from #74318 that should've been removed
earlier.
(cherry picked from commit ec408a69f1)
* Update docs requirements list (#74956)
* removes upper bound on sphinx version
* updates versions of docs build dependencies, adds known good requirements file
* adds instructions for using known_good_reqs file
(cherry picked from commit 58f26388be)
Co-authored-by: Sviatoslav Sydorenko <webknjaz@redhat.com>
Co-authored-by: Sviatoslav Sydorenko <wk@sydorenko.org.ua>
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
* Use the module redirect_list when getting defaults for action plugins (#73864)
* Fix module-specific defaults in the gather_facts, package, and service action plugins.
* Handle ansible.legacy actions better in get_action_args_with_defaults
* Add tests for each action plugin
* Changelog
Fixes#72918
(cherry picked from commit 5640093f1c)
* Fix tests for < 3.8
* subversion - set LC_ALL for accurate command output parsing
When LC_ALL is not set, the output language of commands will differ based on locale. There
is a lot of history of trying to fix this. See the following pull requests:
https://github.com/ansible/ansible-modules-core/pull/4358https://github.com/ansible/ansible-modules-core/pull/4358
This patch attempts to fix this my setting LC_ALL to a UTF-8 locale. Setting LC_ALL to C reintroduces this bug https://github.com/ansible/ansible-modules-core/issues/4178.
I'm sure there are some problems I am not seeing with setting this to en_US.UTF-8, but that is
the only way I could find to fix this bug without reintriducing the bug mentioned above.
* Rather than setting locale, just check for matches before trying to get groups
This is a pragmatic solution to avoid the stack trace since setting the locale correctly
to ensure message parsing is accurate is problematic.
* Improve regexps for finding revision and URL
(cherry picked from commit a8cf0196f7)
Co-authored-by: Sam Doran <sdoran@redhat.com>
While mode is specified in check_mode, don't call lstat.
Since file may not present.
Fixes: #61185
(cherry picked from commit 7099657dd7)
Co-authored-by: Logistic Bot <logistic-bot@protonmail.com>
* Updates our channels to point at the same channel name on libera.chat
* Links that used to point to webchat.freenode.net now point to https://libera.chat
* Removes http://irc.freenode.net from the linkcheck exceptions.
* Updates docs/docsite/rst/dev_guide/developing_modules_in_groups.rst
(cherry picked from commit 80e7e1a17c)
Co-authored-by: John R Barker <john@johnrbarker.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
When an empty value is provided, no `version` attribute will exist on the `LooseVersion` or
`StrictVersion` object. We catch and handle this, but it's not immediatebly clear that an
AttributeError means an empty value was provided.
Specifically handle the case where value or version are empty and add more
helpful error messages.
Add integration tests.
(cherry picked from commit 71e33d2578)
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Localization: Import first set of translated files
(cherry picked from commit d45c2cec55)
* Ignore smart quotes in the dev guide translation.
The smart quotes are in the warning message to not use smart quotes.
(cherry picked from commit 3f694e9f02)
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
* adds banner for GalaxyNG survey (#74889)
(cherry picked from commit 62fc4ad8da)
* gives each banner a unique div ID (#74907)
(cherry picked from commit 33924d7e8d)
While using ``seed`` parameter of the 'random' filter, behavior changes between Python 2 and Python 3.
Added a note about this in the documentation.
Fixes: #63784
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Sandra McCann <samccann@redhat.com>
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
now uses shell option in all cases, the old constant still exists
as 'documentation' but it not settable.
also fix the docsite link in warnings/errors
import cleanup
(cherry picked from commit d8fa2b50d5)
* restores delegatd vars that were not passed
this will restore delegate display on callbacks using the vars
also moves to use delegate_to directly on the default callback
* clog
(cherry picked from commit cbc83545582d8d7a718140263c465129e0e535d9)
* Only remove crontabs if they are empty
(cherry picked from commit fefda12827)
* Add integration test to ensure system cron tab doesn't get removed. Increase cron integration tests separation.
(cherry picked from commit 1e37fa86b4)
* Also detect crontab which only contains whitespace as empty.
(cherry picked from commit 4b69c8f501)
* cron integration test: Adjust system crontab path to be distribution specific.
(cherry picked from commit 70be3730db)
* Add changelog fragment for #74497.
(cherry picked from commit c606b50a3d)
Co-authored-by: Fabian Klemp <fabian.klemp@elara-gmbh.de>
* Map Debian 8 to Python 2
If Python 3 is installed on Debian 8 Ansible cannot run, as the version
is too old (3.4)
* Add integration test for python interpreter discovery on Debian 8
* fix test issue on Debian 9, add changelog
* un"fix" not broken test :D
Co-authored-by: Fabian Klemp <fabian.klemp@elara-gmbh.de>
Co-authored-by: Matt Davis <mrd@redhat.com>
(cherry picked from commit 437a08eb6d)
Co-authored-by: elara-leitstellentechnik <elara-leitstellentechnik@users.noreply.github.com>
Homebrew's default install location for macOS on ARM is /opt/homebrew.
Source: https://docs.brew.sh/FAQ
On a Mac M1 (Apple Silicon), homebrew will be installed at
/opt/homebrew/bin/brew.
Change:
- Newer Solaris drops setfacl. Add a fallback for its chmod ACL syntax.
Test Plan:
- New units
Tickets:
- Fixes#74282
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Regression introduced in #70785
- When macOS chmod ACL syntax is used, Solaris-derived chmods return
with a status of 5. This is also used for our sshpass handling,
because sshpass will return 5 on auth failure. This means on Solaris,
we incorrectly assume auth failure when we reach this branch of logic
and try to run chmod with macOS syntax.
- We now wrap this specific use of chmod in an exception handler that
looks for AnsibleAuthenticationFailure and skips over it. This adds
another authentication attempt (something we normally avoid to prevent
account lockout), but seems better than the regression of not allowing
other fallbacks to be used.
- Without this patch, if setfacl fails on Solaris (and sshpass is used),
we do not try common_remote_group or world-readable tmpdir fallbacks.
Test Plan:
- New unit
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Allow overriding python version as env var for Makefile
- Including hacking directory in git export, for docs build
Test Plan:
- Tested as part of recent downstream work
Co-authored-by: Yanis Guenane <yguenane@gmail.com>
also added general note on how defaults work.
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Sam Doran <sdoran@redhat.com>
(cherry picked from commit 30912b6a47)
On some systems, curses.tigetstr() returns None, which does not work as a control character.
* Add unit tests
* Sort imports
* Skip on older Python
This is an action plugin and only runs on the controller, so no need to test of Python 2. Making
the import hackery work on Python 2 would required some more work which I am not sure is
worth it since we are moving away from Python 2 support on the controller.
* Make the tests work on Python 2 and 3
(cherry picked from commit 55b401a3e7)
Co-authored-by: Sam Doran <sdoran@redhat.com>
The distutils module is not shipped with SUNWPython on Solaris.
It's in the SUNWPython-devel package. Do not use LooseVersion.
Fixes: #74488
(cherry picked from commit 4d7dc15d4e)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Get available collection versions with page_size=100 for v2 and limit=100 for v3
* Update unit tests for larger page sizes
* Add a generic retry decorator in module_utils/api.py that accepts an Iterable of delays and a callable to determine if an exception inheriting from Exception should be retried
* Use the new decorator to handle Galaxy API rate limiting
* Add unit tests for new retry decorator
* Preserve the decorated function's metadata with functools.wraps
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
(cherry picked from commit ee725846f0)
* support separate role argspec file in ansible-core
* support both .yml and .yaml extensions on argspec file in ansible-doc
* fix filename building bug and rename some argspec files to test variations
* use yaml extensions from constants
* add superfluous meta/main.yml files to tests
* Update lib/ansible/cli/doc.py
* update docs
* add changelog and allow for main.yml variations
* add collection role testing
Co-authored-by: Sam Doran <sdoran@redhat.com>
(cherry picked from commit 8fb54885bf)
From the sha512sum man page:
... The default mode is to print a line with checksum, a character indicating type ('*' for binary, ' ' for text), and name for each FILE.
(cherry picked from commit 403a5d147d)
* Jinja intersphinx should point at 3.0; other updates
(cherry picked from commit 8c34eaa69f)
* Update AWS guides with new SDK and Python requirements (#74585)
The 2.0 collection releases will introduce new Python requirements
and a new SDK support policy
(cherry picked from commit fa15d35cb1)
* Add JC filter plugin example (#74563)
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit 91212b2008)
* Fix typo on roles documentation (#74696)
##### SUMMARY
Small fix on the roles directory structure documentation
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
(cherry picked from commit 25de07946c)
* intro: Add a note about current user (#74638)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit 39df263f1a)
Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
Co-authored-by: Jill R <4121322+jillr@users.noreply.github.com>
Co-authored-by: Kelly Brazil <kellyjonbrazil@gmail.com>
Co-authored-by: m0nt3cr1st0 <gonzalo.rubio.casas@gmail.com>
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
* Add constraint for MarkupSafe
MarkupSafe >= 2.0.0 requires Python >= 3.6.0. Add a constraint for older Python versions
and fix the `groupby_filter` test.
* Fix template_jinja2_latest test.
* patch filter decorators on newer Jinja2
* Jinja2 >= 3.0 renames several filter decorators used by Ansible itself, as well as by filters in collections. This patch ensures that the old names are usable within Ansible and by collections without warnings or errors.
* Ignore docs-build issues.
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Davis <mrd@redhat.com>.
(cherry picked from commit f99d024851)
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Change Shippable refs in docs to Azure Pipelines.
(cherry picked from commit 8bd599d048)
* Update cli_parsing.rst (#74555)
Bad indent in an example
(cherry picked from commit dcb807dea7)
* systemd doc update (#74442)
Document that systemd module manages units (timers, services, ...) and not only service. (#74442)
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
(cherry picked from commit 74647a1314)
* Update the module lifecycle document to also cover plugins and collections (#74413)
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit a6cc508822)
* Docs: communication.rst - add ansible-community to the IRC channel list (#74570)
(cherry picked from commit 5463cbb841)
* adhoc clarify help (#74616)
also added comments to code
(cherry picked from commit 7f7d3067e3)
* update docs copyright (#74643)
(cherry picked from commit 7195367635)
* docs: Add a note about `crypt` (#74640)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit 79e12ba98e)
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Joseph Jacobs <joej164@gmail.com>
Co-authored-by: Baptiste Mille-Mathias <baptiste.millemathias@gmail.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Andrew Klychkov <aklychko@redhat.com>
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
For unknown reasons, these tests started failing recently because the results
from testhost11 are being returned before testhost10, but only when coverage is
enabled.
Setting the initial host sleep time to 0 fixes this failure.
ci_coverage
(cherry picked from commit 728e43bc59)
Co-authored-by: Sam Doran <sdoran@redhat.com>
* AWS: update reference to auto-retrying ResourceNotFound (#72314)
AWS Dev Guidelines: update reference to auto-retrying ResourceNotFound
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit c387b318a6)
* docs: Improve documentation for regex_search,replace,findall (#74368)
Improve the documentation for regex_search by adding a few more
realistic examples such as showing how to extract only parts of a
string by using regex groups.
Extend regex_findall and regex_replace by an example that includes
the multiline and ignorecase parameters.
(cherry picked from commit f194108a26)
* Improve a documentation of the first_found lookup plugin (#73080)
Co-authored-by: Wojciech Ściesiński <wojciech.sciesinski@huuugegames.com>
(cherry picked from commit f40ab25909)
* Update the porting guide for a new ansible version (#74490)
(cherry picked from commit 112b175f17)
* setup: Add note and example of delegated facts (#74479)
* Added a note about 'delegated_facts'
* Added an example of 'delegated_facts'
Fixes: #67067
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit 7b03ebf939)
* command - add command name to warning (#74475)
(cherry picked from commit 9e61952cdc)
* update backporting instructions to reflect new latest stable branch, some edits (#74512)
Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
(cherry picked from commit b765d3799b)
Co-authored-by: Mark Chappell <mchappel@redhat.com>
Co-authored-by: Christian Fetzer <christian.fetzer@bmw.de>
Co-authored-by: Wojciech Sciesinski <it-praktyk@users.noreply.github.com>
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
Previously it was hosted on bintray, but that service is shutting down on May 1. Using the new
repository also required using a newer version of RabbitMQ.
(cherry picked from commit 62cba4a6ad)
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Add ArgumentSpecValidator to docs
* Improve docs for ArgumentSpecValidator
* Document removal of private methods
* Update module_utils documentation
- Add docs for argument spec classes as well as validation and parameters files.
- preserve the order in the source for errors.py
- document DEFAULT_TYPE_VALIDATORS so it can be referenced elsewhere
- fix automodule directive for validation.py
* Update docs in arg_spec and paremeters
- This improves the generated documentation.
* Document breaking changes in porting guide.
* Update formatting in porting guide and add a Deprecated section
* Fine tune module_utils documentation
* Move instance docstring to the __init__ method
Remove optional description since it fails the sanity test and I am not 100% it is valid anyway.
* Remove incorrect parameter from docstring
This was changed a while ago but wasn't removed from the docstring.
* Use attr rather than attribute
The py:attribute: domain only exists in newer Sphinx >= 3.1.
* Improve documentation on exceptions
* Final pass
- use args/kwargs instead of param
- fix formatting errors that didn't display examples correctly
- format TypeErrors so they are referenced as classes
- specify complex types
(cherry picked from commit 2cbfd1e350)
Co-authored-by: Sam Doran <sdoran@redhat.com>
Due to the recent security incident, use our own copy hosted in S3
to mitigate future risk from running an arbitrary script downloaded from a
remote and untrtusted server.
(cherry picked from commit aa12af1d34)
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Add description for COLLECTIONS_SCAN_SYS_PATH (#74351)
Fixes: #74275
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit 567361b124)
* lighten navigation background to make section labels easier to read for core docs (#74356)
* make section labels for /ansible-core/ docs easier to read, with black text and lighter gray background
(cherry picked from commit 6119fb0a9a)
* Correct splitext() description, and example (#74377)
`splitext()` returns a 2-tuple of strings, and the last element of the return value includes the `.`
(cherry picked from commit c295de661c)
* Using "~" instead of "+" for concatination (#74364)
Changed FAQ examples to conform with the Jinja documentation:
If both values on either side of a plus/+ are numbers, they will be added whereas using "~" will convert all operands into strings and then concatenate them. Closes#73799.
(cherry picked from commit e6a5245d60)
* Docs - Split Developing collections page, add info on optional module_utils (#74105)
*
(cherry picked from commit c90922ee36)
* Add weos4 network platform to documentation (#74088)
* Add weos4 network platform to documentation
* Fix small format issues
(cherry picked from commit 7ca5dede97)
* Fix typo in Makefile (#74396)
Fixed minor typo specfic -> specific
(cherry picked from commit 4880fee6ca)
* Update complex_data_manipulation.rst (#72509)
(cherry picked from commit c2985c491b)
* Update VMware library installation docs (#71219)
Depending upon OS/distro, please use pip/pip3.
(cherry picked from commit ddfc648d37)
* Update AWS dev guides to use collections utils and fragments (#72312)
(cherry picked from commit cf08c23b4f)
* Use is_boto3_error_code in 'standard' example (#72313)
Use is_boto3_error_code in 'standard' example rather than e.response['Error']['Code'] (#72313)
Co-authored-by: Sloane Hertel <shertel@redhat.com>
(cherry picked from commit 63afb33d86)
* Update Kubernetes collection name in docs (#74440)
(cherry picked from commit 8d499bbc83)
* Update argcomplete docs links on installation guide (#74410)
Link on installation docs is outdated. Switch to currently docs at: https://kislyuk.github.io/argcomplete/
(cherry picked from commit f97787ca74)
* fix spacing to fix header, reorg contributing page (#74421)
Co-authored-by: John R Barker <john@johnrbarker.com>
(cherry picked from commit 9d9b08bece)
* Update first_found documentation (#70502)
* import_tasks do not work with loop, use use include_tasks instead
* update documentation
(cherry picked from commit bacede7a2b)
* Product-related updates. (#74454)
(cherry picked from commit 34c9ed8a28)
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Sandra McCann <samccann@redhat.com>
Co-authored-by: Alex Willmer <alex@moreati.org.uk>
Co-authored-by: Hublerho <43293510+Hublerho@users.noreply.github.com>
Co-authored-by: Ernst Oudhof <17832702+ernst-s@users.noreply.github.com>
Co-authored-by: Hu Shuai <hus.fnst@cn.fujitsu.com>
Co-authored-by: dhx-mike-palandra <45608336+dhx-mike-palandra@users.noreply.github.com>
Co-authored-by: jakelevinez <31458570+jakelevinez@users.noreply.github.com>
Co-authored-by: Mark Chappell <mchappel@redhat.com>
Co-authored-by: Lidiane Taquehara <lidi.mayra@gmail.com>
Co-authored-by: Alex Domoradov <alex.hha@gmail.com>
Co-authored-by: Bill Nottingham <notting@redhat.com>
* Docs: clarify role docs - vars vs params, dependencies (#74104)
(cherry picked from commit 05662fd2ab)
* pin docutils to 0.16 until theme problem is fixed (#74193)
(cherry picked from commit 7443e9d659)
* Docs: Remove duplicate word (#74223)
The word "to" was erroneously used twice consecutively.
Signed-off-by: Matthew Cengia <mattcen@mattcen.com>
(cherry picked from commit c7473828c7)
* Change look and feel of the `ansible-core` docs (#74200)
(cherry picked from commit d7f826c987)
* fix see_also formatting (#74252)
(cherry picked from commit 270f109bb3)
* Update the porting guide for ansible package 4.0.0 alpha4 (#74280)
(cherry picked from commit ee38202fc0)
* Typo fix in help output (#74277)
Should still fit in 80 chars
(cherry picked from commit 8e5dc7306e)
* update intersphinx links for Ansible 3
(cherry picked from commit 481c036632)
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
Co-authored-by: Matthew Cengia <mattcen+github@mattcen.com>
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
Co-authored-by: Richlv <richlv@nakts.net>
For connection units tests, if the `__init__.py` file is modified, the trailing separator was
not added, making the target invalid.
(cherry picked from commit 7b79b69369)
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Update distro test containers to version 2.0.2.
* Update the default test containers.
(cherry picked from commit 459ea5a4a4)
Co-authored-by: Matt Clay <mclay@redhat.com>
* --offline allows in-place verify for installed collections with manifests
* manifest hash, collection name, version, and path are now always displayed
* test updates
##### SUMMARY
<!--- Your description here -->
On the main freebsd change, and no the version: py37-ansible
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
* Added prerequisites to install for CentOS as well so CentOS users don't have to look for alternatives for each Ubuntu package.
Signed-off-by: mm667937 <marketa.muzikova@broadcom.com>
Signed-off-by: mm667937 <marketa.muzikova@broadcom.com>
The GPG key in getfedore.org/static/fedora.gpg changed and caused the test to fail. Update
to using the new key ID and save the GPG file in our S3 to prevent spontaneous changes/breakage.
* Begin using ArgumentSpecValidator in AnsibleModule
* Add check parameters to ArgumentSpecValidator
Add additional parameters for specifying required and mutually exclusive parameters.
Add code to the .validate() method that runs these additional checks.
* Make errors related to unsupported parameters match existing behavior
Update the punctuation in the message slightly to make it more readable.
Add a property to ArgumentSpecValidator to hold valid parameter names.
* Set default values after performining checks
* FIx sanity test failure
* Use correct parameters when checking sub options
* Use a dict when iterating over check functions
Referencing by key names makes things a bit more readable IMO.
* Fix bug in comparison for sub options evaluation
* Add options_context to check functions
This allows the parent parameter to be added the the error message if a validation
error occurs in a sub option.
* Fix bug in apply_defaults behavior of sub spec validation
* Accept options_conext in get_unsupported_parameters()
If options_context is supplied, a tuple of parent key names of unsupported parameter will be
created. This allows the full "path" to the unsupported parameter to be reported.
* Build path to the unsupported parameter for error messages.
* Remove unused import
* Update recursive finder test
* Skip if running in check mode
This was done in the _check_arguments() method. That was moved to a function that has no
way of calling fail_json(), so it must be done outside of validation.
This is a silght change in behavior, but I believe the correct one.
Previously, only unsupported parameters would cause a failure. All other checks would not be executed
if the modlue did not support check mode. This would hide validation failures in check mode.
* The great purge
Remove all methods related to argument spec validation from AnsibleModule
* Keep _name and kind in the caller and out of the validator
This seems a bit awkward since this means the caller could end up with {name} and {kind} in
the error message if they don't run the messages through the .format() method
with name and kind parameters.
* Double moustaches work
I wasn't sure if they get stripped or not. Looks like they do. Neat trick.
* Add changelog
* Update unsupported parameter test
The error message changed to include name and kind.
* Remove unused import
* Add better documentation for ArgumentSpecValidator class
* Fix example
* Few more docs fixes
* Mark required and mutually exclusive attributes as private
* Mark validate functions as private
* Reorganize functions in validation.py
* Remove unused imports in basic.py related to argument spec validation
* Create errors is module_utils
We have errors in lib/ansible/errors/ but those cannot be used by modules.
* Update recursive finder test
* Move errors to file rather than __init__.py
* Change ArgumentSpecValidator.validate() interface
Raise AnsibleValidationErrorMultiple on validation error which contains all AnsibleValidationError
exceptions for validation failures.
Return the validated parameters if validation is successful rather than True/False.
Update docs and tests.
* Get attribute in loop so that the attribute name can also be used as a parameter
* Shorten line
* Update calling code in AnsibleModule for new validator interface
* Update calling code in validate_argument_spec based in new validation interface
* Base custom exception class off of Exception
* Call the __init__ method of the base Exception class to populate args
* Ensure no_log values are always updated
* Make custom exceptions more hierarchical
This redefines AnsibleError from lib/ansible/errors with a different signature since that cannot
be used by modules. This may be a bad idea. Maybe lib/ansible/errors should be moved to
module_utils, or AnsibleError defined in this commit should use the same signature as the original.
* Just go back to basing off Exception
* Return ValidationResult object on successful validation
Create a ValidationResult class.
Return a ValidationResult from ArgumentSpecValidator.validate() when validation is successful.
Update class and method docs.
Update unit tests based on interface change.
* Make it easier to get error objects from AnsibleValidationResultMultiple
This makes the interface cleaner when getting individual error objects contained in a single
AnsibleValidationResultMultiple instance.
* Define custom exception for each type of validation failure
These errors indicate where a validation error occured. Currently they are empty but could
contain specific data for each exception type in the future.
* Update tests based on (yet another) interface change
* Mark several more functions as private
These are all doing rather "internal" things. The ArgumentSpecValidator class is the preferred
public interface.
* Move warnings and deprecations to result object
Rather than calling deprecate() and warn() directly, store them on the result object so the
caller can decide what to do with them.
* Use subclass for module arg spec validation
The subclass uses global warning and deprecations feature
* Fix up docs
* Remove legal_inputs munging from _handle_aliases()
This is done in AnsibleModule by the _set_internal_properties() method. It only makes sense
to do that for an AnsibleModule instance (it should update the parameters before performing
validation) and shouldn't be done by the validator.
Create a private function just for getting legal inputs since that is done in a couple of places.
It may make sense store that on the ValidationResult object.
* Increase test coverage
* Remove unnecessary conditional
ci_complete
* Mark warnings and deprecations as private in the ValidationResult
They can be made public once we come up with a way to make them more generally useful,
probably by creating cusom objects to store the data in more structure way.
* Mark valid_parameter_names as private and populate it during initialization
* Use a global for storing the list of additonal checks to perform
This list is used by the main validate method as well as the sub spec validation.
When using "use_regex: yes" and setting an excludes: without
specifying a pattern: the existing code passes the file-glob '*' to
the regex matcher. This results in an internal invalid-regex
exception being thrown.
This maintains the old semantics of a default match-all for pattern:
but switches the default to '.*' when use_regex is specified.
The code made sense as-is before excludes: was added (2.5). In that
case, it made no sense to set use_regex but *not* set a pattern.
However, with excludes: it now makes sense to only want to exclude a
given regex but not specify a specific matching pattern.
Closes: #50067
* moved change to new location
added changelog
* Update lib/ansible/modules/find.py
Co-authored-by: Ian Wienand <iwienand@redhat.com>
* Fix a bug adding unrelated candidates to the plugin loader redirect_list
* Add tests for the redirect list
* test redirect list for builtin module
* test redirect list for redirected builtin module
* test redirect list for collection module
* test redirect list for redirected collection module
* test redirect list for legacy module
* changelog
Ansible can gather distribution facts for older Amazon Linux
with /etc/os-release data.
Fixes: #73946
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Update developing_inventory.rst
hashes/dictionaries in JSON (JavaScript Object Notation) are represented by what JSON calls "objects", so use that established terminology instead of "hash", "dictionary", or "hash/dictionary" etc.
* Update docs/docsite/rst/dev_guide/developing_inventory.rst
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* ansible-pull: run all playbooks when multiple are supplied
* add test for ansible-pull with multiple playbooks supplied from cli
* add changelog fragment
This helps users to find the right ressources more easily and prevents them from clicking the first matching link only to be redirected to a subsection on the samepage from there
Co-authored-by: Jens Heinrich <github.com/JensHeinrich>
* Catch more potential errors (and increase false-positive rate).
* Flag some false-positives in lib/ansible/modules/ with no_log=False.
Co-authored-by: John Barker <john@johnrbarker.com>
* add optional module_utils import support
Treat core and collections module_utils imports nested within any Python block statement (eg, `try`, `if`) as optional. This allows Ansible modules to implement runtime fallback behavior for missing module_utils (eg from a newer version of ansible-core), where previously, the module payload builder would always fail when unable to locate a module_util (regardless of any runtime behavior the module may implement).
* sanity test fixes
ci_complete
* Documentation: Lay the ground for i18n work.
* Translating both ansible-core and ansible docs requires msgcat
(a program from the gettext package). If msgcat is not present,
only the ansible-core index file's strings will be extracted for translation.
If this is a problem, a short hacking script could be written to take the
place of msgcat (merging the various index pot files into a single pot file).
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
* finish migrating ssh plugin to config system
fixes#72739fixes#57220
* fix connection detection in reset
* correct options for connection meta reset
Co-authored-by: David Shrewsbury <Shrews@users.noreply.github.com>
* Convert markdown issue templates into issue forms
* Limit allowed issues to the form based ones
* Title-case the form field labels
Co-authored-by: John R Barker <john@johnrbarker.com>
* Remove CoC checkboxes from the forms
Co-authored-by: John R Barker <john@johnrbarker.com>
The file module changes existing sym links from relative to absolute
if the src is not stated in the tasks since it uses `os.path.realpath`
to fetch the link source and not `os.readlink`. Changed that.
* Add more scenarios to basic valid testing
* Update invalid tests
* Fix test for Python 2
* Condense data
* Add tests for missing required and invalid-elements
* Update aliases tests
* Add invalid scenarios for aliases
* Add tests for _add_error() method
* Fix sanity test failure
* galaxy: restore left hand slicing in assignment
Fix 'ansible-galaxy role init --role-skeleton=role-skeleton' when the role skeleton
contains an ignored directory.
The issue was because the 'dirs' variable was changed to reference a different list,
but needs to be mutated instead to stop os.walk from traversing ignored directories.
Fixes: #71977
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Change:
- Fix regression: unhandled exception when given inventory directory
is empty or contains empty subdirectories.
- Fix unhandled exception when limit file is actually a directory
instead of a file.
- Fix inventory tests which previously could never fail due to missing
`set -e`. Fixed up tests that failed after `set -e` was added. Added
several tests.
Test Plan:
- New tests
- Fixed existing tests which previously could never fail
Tickets:
- Fixes#73658
Signed-off-by: Rick Elrod <rick@elrod.me>
Adds in the caveat that Python's string modules have different constants depending on the major version (2.x vs 3.x), but preserves the fact that a fair amount of them are common. Other minor docs touchups also included.
* excludes scenario guides from core docs, splits porting guides and roadmaps, symlinks indices to create index.html pages, and adds .gitignore entries for conf.py and the toplevel index.rst files generated by the docs build
This solution builds three types of docs:
* ansible-2.10 and earlier: all the docs. Handle this via `make webdocs
ANSIBLE_VERSION=2.10`
* ansible-3 and later: a subset of the docs for the ansible package.
Handle this via `make webdocs ANSIBLE_VERSION=3` (change the
ANSIBLE_VERSION to match the version being built for.
* ansible-core: a subset of the docs for the ansible-core package.
Handle this via `make coredocs`.
* `make webdocs` now always builds all the collection docs
* Use `make coredocs` to limit it to core plugins only
* The user specifies the desired version. If no ANSIBLE_VERSION is specified, build plugins for the latest release of ansible
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
Co-authored-by: Matt Clay <matt@mystile.com>
* Manage the in-memory cache in _call_galaxy but let the caller set the file cache after getting paginated results
* Add a test for caching successful and not caching unsuccessful paginated results
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Initial import of modified version of alikins' code
* Add unit testing for new Role methods
* Fix validate_arg_spec module for sanity test. Add test_include_role_fails.yml integration test from orig PR.
* Add testing of suboptions
* Use new ArgumentSpecValidator class instead of AnsibleModule
* fix for roles with no tasks, use FQ name of new plugin
* Add role dep warning
Allow constructed to optionally use vars plugin data
* mostly for those looking to leverage group_vars/ and host_vars/
* limited to already processed sources
Ensure `yamllint`'s `check_assignment()` correctly ignore the
attribute assignment. Those don't have any `.id` attribute and will
trigger an `AttributeError` exception.
See: https://github.com/ansible/ansible/pull/73322
Only add data when there is data to add
also avoid clobbering existing data with empty file
fixes#45843
* remove redundant code, update comments
* fix mock dataloader, original does not return None
* added test
* module compat for py3.8+ controller
* replaced internal usages of selinux bindings with internal ctypes binding (allows basic selinux operations from any Python interpreter), plus tests
* added new respawn_module API to allow modules to import Python packages that are only available under a well-known interpreter, plus tests
* added respawn logic to modules that need Python libs from a specific system interpreter (apt, apt_repository, dnf, yum)
minimize internal HAVE_SELINUX usage
spurious junk
pep8
* pylint fixes
* add RHEL8 Python 3.8 testing
* more pylint
* import sanity
* unit tests
* changelog update
* fix a bunch of stuff
* tweak changelog
* fix setup_rpm_repo on EL8
* misc sanity/test fixes
* misc feedback tweaks
* fix import fallback in test module
* fix selinux MU test
* fix dnf tests to avoid python-dependent test packages
* add trailing LFs to aliases
* fix yum tests to avoid test package with Python deps
* hack create_repo for EL6 to create noarch package
* Upgrade pylint and deps in ansible-test.
* Enable pylint on Python 3.9.
* Update pylint config.
* Add ignore for vendored six.
* Add ignores for support plugins.
* Fix issue reported by pylint.
When running in verbosity <2, display the file and line number for tasks that fail. This provides
useful information without having to run at increased verbosity.
* Move _print_task_path to CallbackBase class
* Add integration tests
* Add color parameter to _print_task_path()
* Keep color output consistent for now
Currently the path is display with COLOR_DEBUG formatting with verbosity >= 2.
Instead of the color of the path changing based on verbosity level, just keep it at the
currently behavior of COLOR_DEBUG. Having the color of the same information change
based on verbosity level seems incorrect and makes visual parsing of the information
more difficult.
Co-authored-by: tahar.jegham <jeghamseifeddine@gmail.com>
* Docker content from community.general migrated to community.docker.
* Hetzner Robot content from community.general migrated to community.hrobot.
* RouterOS content from community.network migrated to community.routeros.
* PostgreSQL content from community.general migrated to community.postgresql.
* OC content from community.general migrated to community.okd.
* Cisco NSO content from community.network migrated to cisco.nso.
* Add changelog fragment.
* Google content from community.general migrated to community.google.
* Hashi Vault content from community.general migrated to community.hashi_vault.
* KubeVirt content from community.general migrated to community.kubevirt.
* FortiOS content from community.network migrated to community.fortios.
* Update changelog fragment.
* DellEMC content from community.general migrated to dellemc.openmanage.
* treat oracle same as rhel/centos
* fix misunderstood use of yaml anchors
* add changelog fragment
* Update changelogs/fragments/73498-INTERPRETER_PYTHON_DISTRO_MAP-Treat-oracle-same-as-rhel-centos.yml
Change:
- Other targets might remove rpm-build as they clean up after
themselves. Ensure that it's present in setup_rpm_repo because
rpmfluff needs it.
Test Plan:
- Local experimentation with yum_repository and mysql_db (the latter of
which depends on a handler which was removing rpm-build) on
stable-2.9.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Include all collections in single json object / yaml document
* Add tests
* For galaxy list yaml/json output, use dictionary of dictionaries instead of list
* Add tests for listing single collection in yaml / output format
* --output -> --format
* Add explicit test for listing collection in human format
* Fix bug where empty json object was emitted + add test
* add changelog fragment for #71979 (ca_path for uri)
* add integration tests for ca_path in the uri module
* return path of ca cert instead of its content
* connect to port 444 on self_signed_host
and use quay.io/ansible/http-test-container:1.3.0
* state that the certificate in ca_path is used for validation
* Fix regression introduced in b77abd0491 causing bug in inventory modules which break functionality in user setting use_contrib_script_compatible_sanitization parameter.
* Add changelog
Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
The community.kubernetes collection will be migrated / renamed to
kubernetes.core in the collection's 2.0 dev cycle. This should coincide
with the ansible-core 2.11 cycle. The collection is being released to
galaxy under both the community.k8s and k8s.core names today.
* introduce self-signed.ansible.http.tests
* forwarding of port 444
* forward port 8444 to port 444 on http test container
* Fix port forwarding for Windows under docker
* add changelog fragment
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* Ensure there is a single source of collection metadata
* Allow collection subdirs to be detected by a galaxy.yml or MANIFEST.json
* Add documentation about installing and downloading collection directories
* Add an example for downloading a git repository
* Update documented valid metadata sources for installing git repositories
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
* Improve setup_rpm_repo
- add handlers to remove repos
- add variable to control whethere or not repos are created
* Use local repo for all distros
* Change repo creation script to module
* validate_modules: fails with .id attribute not found
This patch addresses a problem in the `found_try_except_import` test.
This module tries to identify lines like:
`HAS_FOO = True`
In this case, the target (`HAS_FOO`) is of type `ast.Name` and has a
`id` attribute which provide the name.
In my case, I've a line that set a module attribute`. In this case, the
target (`module.var`) has the type `ast.Attribute` and no `id`
attribute. The code trigger an `AttributeError` exception.
This patch ensures we compare a `ast.Name`.
* Update test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/main.py
Change:
- Remove check that states that only Fedora can be an OSTree
distribution.
- This allows us to correctly return "atomic_container" as the pkg_mgr
fact for RHEL for Edge, Fedora/RHEL/CentOS Atomic Host, etc.
Test Plan:
- Created local RHEL for Edge image and tested against it.
- Tested against regular RHEL 8 and still got `dnf` as expected.
- Tested against RHEL 7 Atomic Host and got `atomic_container` now.
- New unit tests.
Tickets:
- Fixes#73084
Signed-off-by: Rick Elrod <rick@elrod.me>
ansible-galaxy currently behaves bad then a role to be installed
contains ~ or $ at any place in the path of a file in that role.
It extracts the parent directory of the offending path level as an
empty file. This explodes if that directory contains anything else.
Change this behaviour. `~` is now allowed allowed when it is
not a full level (Yes: `some~thing/`, no: `~/`). The code should
get refactoring in an other PR.
The `UserFactCollector` queries the user login name via
`getpass.getuser()` and looks up the corresponding entry
in the password database.
The login name may differ from the actual user name,
eg. if the `LOGNAME` env variable is set. The lookup
fails in this case. Added a fallback in this case that
tries to get the entry via the user ID.
* Add direct+transitive pre-release regression test
* Match user-requested transitive pre-releases
This change makes sure that in scenario when a user requests
collectionA-pre and collectionB that depends on collectionA,
collectionA-pre would actually match that collectionA requirement
while `--pre` is unset.
Co-authored-by: Jordan Borean <jborean93@gmail.com>
This warning was intended to help improve test environment configuration.
However, it has ended up producing mostly non-actionable warning noise instead.
In most situations a missing pip or python should result in test failures.
Reporting a missing pip also implies that it should be used by tests, which is not the case.
Tests should be invoking pip as a python module with the appropriate python interpreter instead.
Change detection and check mode fixes for apt_key
* allow apt-key to use proxies
* add note about deprecation of apt-key itself
* expanded error msgs
* show all keys
* fix short_format parsing
* added more return info and documented it
PR #72591
This change:
* Adds an artifacts manager that abstracts away extracting the
metadata from artifacts, downloading and caching them in a
temporary location.
* Adds `resolvelib` to direct ansible-core dependencies[0].
* Implements a `resolvelib`-based dependency resolver for
`collection` subcommands that replaces the legacy
in-house code.
This is a dependency resolution library that pip 20.3+ uses
by default. It's now integrated for use for the collection
dependency resolution in ansible-galaxy CLI.
* Refactors of the `ansible-galaxy collection` CLI.
In particular, it:
- reimplements most of the `download`, `install`, `list` and
`verify` subcommands from scratch;
- reuses helper bits previously moved out into external modules;
- replaces the old in-house resolver with a more clear
implementation based on the resolvelib library[0][1][2].
* Adds a multi Galaxy API proxy layer that abstracts accessing the
version and dependencies via API or local artifacts manager.
* Makes `GalaxyAPI` instances sortable.
* Adds string representation methods to `GalaxyAPI`.
* Adds dev representation to `GalaxyAPI`.
* Removes unnecessary integration and unit tests.
* Aligns the tests with the new expectations.
* Adds more tests, integration ones in particular.
[0]: https://pypi.org/p/resolvelib
[1]: https://github.com/sarugaku/resolvelib
[2]: https://pradyunsg.me/blog/2020/03/27/pip-resolver-testing
Co-Authored-By: Jordan Borean <jborean93@gmail.com>
Co-Authored-By: Matt Clay <matt@mystile.com>
Co-Authored-By: Sam Doran <sdoran@redhat.com>
Co-Authored-By: Sloane Hertel <shertel@redhat.com>
Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
Signed-Off-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
While getting policy name in get_chain_policy API,
module does not require any additional parameters except chain
Enabling flag in get_chain_policy API call fixes this.
Fixes: #68612
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Use correct ssh executable and options in all cases on connection plugin
* Also nicer naming/comments
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Ability to add local variables into AnsibleJ2Vars was added in
18a9eff11f to fix#6653. Local variables
are added using ``AnsibleJ2Vars.add_locals()`` method when creating a
new context - typically when including/importing a template with
context. For that use case local template variables created using
``set`` should override variables from higher contexts - either from the
play or any parent template, or both; Jinja behaves the same way.
Also removes AnsibleJ2Vars.extras instance variable which is not used.
Also adds missing test for #6653.
Fixes#72262Fixes#72615
ci_complete
The `command` module does not return stdout & sterr when calling
a non existing executable or an unknown exception arises. This fix
lets the module return empty byte strings in those cases.
Change:
- Add Ubuntu 20.04 to CI now that venv is default instead of virtualenv in ansible-test.
Test Plan:
- CI
Tickets:
- Fixes#69203
Signed-off-by: Rick Elrod <rick@elrod.me>
The init script for the test container writes additional lines to
the .bashrc of the user. This was done via a `cat` multiline
instruction, which is implemented internally by writing a
temporary file to TMPDIR (/tmp in this case) first. Docker fails
to provide /tmp just after creation, which results in a race
condition that rarely makes the init fail. Changed the `cat`
statement to multiple `echo`s.
Co-authored-by: Matt Clay <matt@mystile.com>
* Consolidate filters/tests handling into JinjaPluginIntercept
ci_complete
* Postpone loading all ansible plugins
* Do we need to create an overlay?
ci_complete
* Typo
ci_complete
* Add FIXME
* conditional.py: use public Environment.parse() method
* Remove remaining occurrences of shared_loader_obj being passed to Templar
* __UNROLLED__ not needed with this change anymore
* Incorrect rebase at some point?
##### SUMMARY
One of the parameters to the slaac filter was unquoted causing a crash:
```
template error while templating string: expected token 'end of print statement', got ':'. String: {{ fdcf:1894:23b5:d38c:0000:0000:0000:0000 | slaac('c2:31:b3:83:bf:2b') }}
```
This adds the missing quotes.
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
When calling compile(), the filename argument should be either a real file
name or a string. According to Python docs, suggested one is '<string>'.
Keep the current behaviour (encapsulate the actual expression),
but enclose it into angle brackets.
Signed-off-by: Oldřich Jedlička <oldium.pro@gmail.com>
* Fix YAML error message when error is at the end of the file
If a YAML file fails to load due to a syntax error in a file, or there is an error in the last line of a
file, PyYAML reports the last line number of the file as the index where the error occurred.
When reading the file lines, we use that index to the get the relevant line. If the index value is out
of range, the relevant line is lost for error reporting.
Subtract one from the index value to avoid the IndexError in this specific scenario. It is possible
to still get an IndexError, which will be handled as it is currently.
* Update existing tests and add new tests
##### SUMMARY
Added `[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12` to the Powershell scripts that use a file download. If not specified the script won't be allowed to download data from the mentioned URL's.
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
* Allow restricting config values to enumerated list
* dont document internal entries
* also ignore private defs for ansible-config
* remove invalid value from tests
* added porting entry
##### SUMMARY
<!--- Your description here -->
single_list must be transformed to a list of pairs before feeding it into dict()
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
When the pause module is run in the background and seconds parameter is provided,
do not warn.
* Add tests
* Fix existing tests
The test wasn't failing when it should have.
Change:
- On CentOS Stream, make distribution_release be "Stream"
- On CentOS Core, it continues to be "Core"
- Implement custom distribution file parser for CentOS, so we can look
for "CentOS Linux" and "CentOS Stream"
- Two new fixtures introduced (CentOS Linux 8.1 and CentOS Stream 8)
- Removed two dicts from `Distribution` class that were seemingly not
used anywhere.
Test Plan:
- ci_complete
- New test fixtures
Tickets:
- Fixes#73027
Signed-off-by: Rick Elrod <rick@elrod.me>
The test currently only expects the literal `ansible` followed
by a semver in the first output line of `ansible --version`.
When running from within a git checkout, additional information
like the currently checkout branch are attached, which lets
the test fail. This commit allows arbitrary information to
follow the semver.
`test_prepare_multipart` fails in non Debian environments since
Debian installations map the file ending `.key` to the MIME type
`application/pgp-keys`, which is not IANA conformant. This commit
explicitly sets the corresponding file type to
`application/octet-stream` and adjusts the expected serialized
result.
* Remove unused code in ansible-test.
* Remove obsolete endpoint logic from ansible-test.
* Remove obsolete region selection in ansible-test.
* Remove obsolete port logic in ansible-test.
* Clean up ansible-test remote providers.
Do the right thing on Linux when password lock and a password hash are provided by writing
out the password hash prepended by the appropriate lock string rather than using -U and -L.
This is the correct way to set and lock the account in one command.
On BSD, run separate commands as appropriate since locking and setting the password cannot
be done in a single action.
FreeBSD requires running several commands to get the account in the desired state. As a result,
the rc, output, and error from all commands need to be combined and evaluated so an accurate
and complete summary can be given at the end of module execution.
* Improve integration tests to cover this scenario.
* Break up user integration tests into smaller files
* Properly lock account when creating a new account and password is supplied
* Simplify rc collection in FreeBSD class
Since the _handle_lock() method was added, the rc would be set to None, which could make
task change reporting incorrect. My first attempt to solve this used a set and was a bit too
complicated. Simplify it my comparing the rc from _handle_lock() and the current value of rc.
* Improve the Linux password hash and locking behavior
If password lock and hash are provided, set the hash and lock the account by using a password
hash since -L cannot be used with -p.
* Ensure -U and -L are not combined with -p since they are mutually exclusive to usermod.
* Clarify password_lock behavior.
Adds a note pointing to the datetime documentation as to_datetime does not use the time library but instead uses datetime.
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
Tests should use the Python built-in ``venv`` module on Python 3 instead of the standalone ``virtualenv`` module.
On Python 2 the ``virtualenv`` module continues to be the only option.
The version installed is either the OS packaged version or the last release to support Python 2, which is version 16.7.10.
The old example failed with:
fatal: [localhost]: FAILED! => {"msg": "to count backwards make stride negative"}
I fixed this and placed a start parameter to consolidate with the rest of the examples provided.
Co-authored-by: Jorge Rua <jorua@redhat.com>
The standard Port for ansible winrm is set to 5986/HTTPS.
To be able to use Kerberos, you need to set ansible_port: 5985/HTTP otherwiese you get error message or timeout.
HTTPS is not needed for environments with Kerberos, since kerberos has server identification built in to the protocoll.
FYI: HTTP traffic will by default also be encrypted when using kerberos
Change:
- Internally, use dnf.subject.Subject#get_best_query for state: absent
- Add a bunch of tests for removing packages, given a bunch of different
pkg specs (nv, nvr, nvra, wildcard, etc.)
Test Plan:
- New tests
- Local experiments with DNF API via PDB.
Tickets:
- Fixes#72809
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Changes to make F33 work with current tests
- Add F33 to docker.txt
- Add F33 to AZP config
Test Plan:
- ci_complete
Signed-off-by: Rick Elrod <rick@elrod.me>
* Replace the use of the function string.replace() with the method str.replace()
because that's what works for both Python 2 and 3.
* Cleanup the unused string import.
Added a changelog fragment.
* The documentation for os.write() seems a bit iffy, but in Python 3 we
definitely cannot pass it a string and we need to encode it into
bytes.
The Python documentation at
https://docs.python.org/3/howto/pyporting.html#text-versus-binary-data
says that this code will work in Python 2 as well.
Co-authored-by: Lloyd Parkes <lloyd@must-have-coffee.gen.nz>
Change:
- The FreeBSD release can contain -RC or -PRERELEASE in addition to
-RELEASE, -STABLE, or -CURRENT.
Test Plan:
- Added new fixed from an RC version of TrueNAS which uses a -PRERELEASE
version of FreeBSD.
Tickets:
- Fixes#72331
Signed-off-by: Rick Elrod <rick@elrod.me>
* ExecutionPolicy default value is RemoteSigned
##### SUMMARY
In Windows setup steps, settings `ExecutionPolicy` is restored to value `Restricted`. But actually default value for Windows server is not `Restricted` but `RemoteSigned` ([source](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_execution_policies?view=powershell-7.1#default)).
I have got issue with `win_dsc` because `ExecutionPolicy` was `Restricted`.
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
* Add changelog fragment
* Apply suggestions from code review
Co-authored-by: Sandra McCann <samccann@redhat.com>
* Change example to use role params to represent the documented outcome. Fixes#68922
* Add notes about how vars vs role params affect variable scoping, and how lazy evaluation and role-deduplicating is affected. Fixes#72650
* link to term
* Add link to all module and plugin indexes.
* Use glob setting for toctree.
* Explicitly include new file.
* Move all_plugins one level up.
* Revert "Move all_plugins one level up."
This reverts commit bd47e82ead.
* Revert "Explicitly include new file."
This reverts commit 74ae50a734.
* Adjust .gitignore
* Revert "Revert "Explicitly include new file.""
This reverts commit f30d99c861.
* Fix docs make clean
Now that there's a static file in rst/collections, we have to be more
explicit about how we clean the generated files out of that directory.
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
* arg_spec - move type checking lookup method to a function
* Change get_wanted_type name and behavior
Change the name to get_validator to bette describe what it is doing.
Change the interface to always return a value. This lines up with the behavior of get_*
functions always returning something or None and check_* functions raising an
Exception if something went wrong during the check.
* Add param to check_type_str()
Not meant to be a long term fix, but gets tests passing. More work is needed to figure
out how to solve this cleanly.
* Remove private attribute mapping types to validator
Since the function that needs it has moved to parameters.py, there is no need to have it as
a attribute of AnsibleModule.
Update tests that were referencing the private attribute.
* Use private method for 'str' type
To avoid having to put the string conversion warning behavior in the check_type_str() method,
use the private _check_type_str() method for 'str' type.
Import CHECK_ARGUMENT_TYPES_DISPATCHER for backwards compalitibility and store it as
a private attribute.
Revert changes to support plugins that are referencing serf._CHECK_ARGUMENT_TYPES_DISPATCHER.
* Add changelog
* Change function name to better reflect its... function
* Change dict name to better reflect its contents
CHECK_ARGUMENT_TYPES_DISPATCHER --> DEFAULT_TYPE_VALIDATORS
* Fix changelog
* systemd - do not overwrite unit name when searching
PR #72702 introduced a bug that changed the unit name when splitting it up for the purpose
of searching for the unit. This only happens on unit file templates on systems that have a 5.8
or newer kernel and a version of systemd that does not contain a bugfix that causes systmed
to fail to parse dbus.
* Use facts rather than a manual probe to determine if systmed is present
* Remov unnecessary block
* Use vars files instead of set_fact
* Add tests for using a templated unit file
* Update changelog fragment
* Use template to get correct path to sleep binary
* import_playbook - change additional params to deprecation
I incorrectly recommended this be set as a warning when it should have been a deprecation.
* Fix deprecation sanity test to not required a collection name when not inside a collection
Also pin virtualenv to 16.7.10 for older Mac OS X systems. This was the version being installed
anway with the previous constraint (<20).
On systems with Python 3, now prefer venv over virtualenv. Test to see if venv is functional since
some systems have a non-functional venv installation (such as Debian).
* Increase fetch depth to reduce the frequency of full test runs for out-of-date PRs.
* Update scripts to match those which will be used in collections.
* Change role argspec file used by ansible-doc
This changes the file used for role argument specs from meta/argument_specs.yml
to meta/main.yml. The argument specs are expected to be in that file under the
top-level entry of `argument_spec`.
* Switch to argument_specs
* Update yum/dnf module docs to include version comp
Fixes#61234
Signed-off-by: Adam Miller <admiller@redhat.com>
* fix up docs syntax to sanity tests are happy
Signed-off-by: Adam Miller <admiller@redhat.com>
* ansible-galaxy collection list and verify now utilize collections in site-packages.
This is a short term fix for #70147. The long term fix needs to handle
install (but that discussion is also bound up in how upgrade is going to
work and where things can get installed so it's deferred for 2.11.)
* Add test for ansible-galaxy collection list with site-packages
* Fix sanity issue
Co-authored-by: David Moreau Simard <moi@dmsimard.com>
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* Test installing a dependency compatible with multiple collection requirements
Test downloading a collection without dependencies
* Move cleanup to the end of the test suite
On python3 sys.stdin is an encoded file object that does not support
reading raw binary data. Use the supplied buffer object to do so.
Signed-off-by: Sven Wegener <sven.wegener@inovex.de>
Co-authored-by: Sven Wegener <sven.wegener@inovex.de>
This should allow users to extract specific files from an archive as
desired.
Fixes#16130, #27081.
* Rebase and make a few minor changes
* Add changelog
* Improve tests
- move to separate tasks file
- change assertions to check for exactly one file
- use remote_tmp_dir for output dir
* Make exclude and include mutually exclusive
* Don't remove files needed by other tasks
* Fix sanity tests
* Improve feature documentation
* Skip tests that use map() on CentOS 6
* Use fnmatch on include for zip archives
This matches the behavior of exclude
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Remove space before comma in '... bla , use ...'
* 'why' is inserted in the middle of a sentence, between two commas.
* Make deprecations from base.yml show source ansible-core.
* Add changelog fragment.
* Improve some more 'why's.
* Add PR URL to fragment.
* file: add symlink is in a sticky directory tests
* file: handle symlink in a sticky directory
Co-Authored-By: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Add changelog and fix unit test
The builtins import was removed since it was unused, but it is now needed.
This is in reference to the issue: https://github.com/ansible/ansible/issues/41130 (Cannot decrypt with copy module when remote_src: yes)
Autodecryption of the vaulted files does not work when we want to copy files with remote_src=yes.
Co-authored-by: ziaurrehman72 <ziaurrheman72@yahoo.com>
The recently released version of cffi fails to install on systems with an older version of gcc. In
our case, this in the CentOS 6 test image. There is a fix but it has not yet been released.
https://foss.heptapod.net/pypy/cffi/-/issues/480
Fix the order of the arguments for ansible.netcommon.reduce_on_network. Pass the network as the argument and the list of IP addresses as the filter input.
* Move _syslog_facitily to __init__
No good reason it should not be set for each object
* Move internal property setting to private method
* Create check_arguments() function
* Remove unused import
* Rename function to better match its behavior
Change the behavior to return a set, either empty or populated, with unsupported keys.
Accept legal_inputs as optional which will not required calling handle_aliases before calling
get_unsupported_parameters().
* Add changelog
* Rework function behavior and documentation
I realized I missed the original intent of this method when moving it to a function. It
is meant to compared the parameter keys to legal inputs always, not compare
parameter keys to argument spec keys, even though the argument spec keys should
be a subset of legal inputs.
* Add tests
* Fix typo.
* Set internal properties when handling suboptions
We could not find reason for the docutils install so trying out removing it.
Also bumping to latest version of ansible-runner
Skip all python2 versions because next ansible-runner drops it
Co-authored-by: Elijah DeLee <kdelee@redhat.com>
* Support listing roles in text and JSON
* Change tests for unfrack'd playbook_dir var
These tests were using '/tmp' for testing the setting of the playbook_dir
var. Now that we unfrack that var, MacOS will change this to '/private/tmp'
causing the tests to fail. We can choose a path that does not exist (since
unfrack does not validate existence) so that we can guarantee unfracking
will not change the value.
##### SUMMARY
Make clear that `ansible_play_name` is the name attribute from the play and not the file name of the playbook.
improves #57361
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
Change:
- Initial set of changes for renaming to ansible-core
- Includes changelog fragment changes from base -> core
- Does NOT include docs changes
- Modifies detection stuff in setup.py to support ansible<2.9 and ansible-base
Test Plan:
- ci_complete
* Fix type of mode field in copy module
* Remove module type checking from ignore.txt sanity checks
* Remove mode type
Co-authored-by: Matt Martz <matt@sivel.net>
* Fix super annoying Python 2.6 multiprocessing.Queue stack trace in CI
A bug exists in Python 2.6 that sometimes raises an exception during interpreter shutdown. We
encounter this frequently in our CI since we run tests on CentOS 6 as the control node, which
has Python 2.6.6 with this bug.
This PR adds a very minor sleep only on Python 2.6 which gets around this issue. I did lot of testing
using a standalon script I found that easily duplicated the issue to find the minimum sleep value
needed to avoid this issue.
CPython issue: https://bugs.python.org/issue4106
Fix in CPython: https://hg.python.org/cpython/rev/d316315a8781
* Use correct attribute
* Consolidate logic for determining whether or not session is interactive
into a single function, is_interactive()
* Increase test coverage
I wasn't able to find a good way of simulating running a backgrounded test with CI since the
whole test is essentially run not in a TTY, which is similar enough to cause the new is_interactive()
function to always return false.
* add return documentation for unarchive, add fragment for changelog, apply suggestions from code review
Co-authored-by: Amin Vakil <info@aminvakil.com>
* Clarify collection paths in docs, fix rst syntax, rephrase docs about using collection from folder/local git
Co-authored-by: Felix Fontein <felix@fontein.de>
* Added caching mechanism for Galaxy API requests
* Add cache options and split up code
* Added unit tests
* Fix sanity test
* Use modified date and fix up caching for explicit servers
* Make sure credentials are not in cached server name
* Added test for getting updated cache version
* Changes from review
Change:
- Bump default, ansible-base, distro containers
- We do NOT add fedora33 yet, because it doesn't work right on Shippable
due to an old kernel. This will be added post-AZP.
Test Plan:
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
* Reorder comment postition
* Add comment unit test
* Fix unit test
* Fix unit test
* Add changelog
* Add paramaters which would be problematic without this fix
* Fix typo
* Fix unit test
* Fix unit test
* Emit warning when running on the controller with a Python older than 3.8
* Add spaces
Co-authored-by: Matt Clay <matt@mystile.com>
* and more spaces
Co-authored-by: Matt Clay <matt@mystile.com>
* s/Py/Python/
Co-authored-by: Matt Clay <matt@mystile.com>
* Add note to Control node requirements about Py3.8 requirement
* Add collection_name to deprecated call
* more spaces
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Clarify that we are only packaging for py3.8+
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Fix 'role_name : tast_name' notation if task contains role name
* Add tests for notifying handler names which contain the role name
Co-authored-by: Thomas Wouters <thomaswouters@gmail.com>
* Support docker and k8s action groups for moved modules in community.docker and community.kubevirt.
* Also support k8s action group for community.okd.
* Also add kubernetes.core.
* Adjust PR #.
* Fix changelog fragment.
* Remove community.okd.
* Revert "Remove community.okd."
This reverts commit 812b5aa6e2.
Change:
- Docs: Add note that security/bugfix apply to dependencies too, like
the dnf command.
- dnf: security/bugfix only makes sense for updates, so limit the
package query sack to available updates.
- tests: Limit tests to our known-good test packages, so that RHEL
packages marked security/bugfix without similarly marked dependencies
don't fail our tests.
Test Plan:
- Tested with `dnf upgrade-minimal --bugfix` and reproduced the same
error currently seen in CI, showing that we are consistent with what
dnf does.
Tickets:
- Likely fixes#72316
Signed-off-by: Rick Elrod <rick@elrod.me>
* Indent Jinja2 directives.
* Show version_added and deprecations for ini settings and environment variables.
* Make default conditional, add choices. Copied from #55474.
* Add Ansible variables. Copied from #55474.
* Turn ini keys and environment variables into lists. Copied from #55474.
* Improve formatting. Copied from #55474.
* hint in doc of git module to abort on missing http password
improves #69489
* fix syntax in doc of git module
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Matt Martz <matt@sivel.net>
* ansible-doc: export has_action when --json is used.
* Remove docuri and now_data, which were not used resp. ignored in format_plugin_doc and the functions it calls anyway.
* Add function _combine_plugin_doc.
* added timeout options to adhoc and console
* added test
* fix typosesz
* fix conflict
* task_timeout
* fix timeout option, added extra vars to console
* actually use right cli switch .. DUH!
* added timeout to include 'valid' but ignored keys
* fix default
* fixes per review
* fixes for collection playbooks
- add fqcn invocation, also handles extensions
- bring import_playbook into new normal
- avoid adding collection playbooks to adjacency
- deal with extensions or lack of em
- added tests
- fix bugs with multiple playbooks supplied
- unicode/bytes/native fixes
- fix modulenotfound for < py3.7
- tests
* Update galaxy server image used in CI
* Fix port and migration issue
* Add delete step, still need to deal with pagination
* Make cleanup more efficient
* Remove testing code
* Add DarwinStrategy class and integration tests
macOS has three seprate hostname params that need to be set. One of those params, LocalHostName, has more stringent requirements than the other two, which accept special characters and spaces. Create a method to scrub the hostname to ensure it works well with the system requirements.
* Update documentation
* Account for virtualization type returned on Azure Pipelines
* Do not be dependent on order of self.name_types
Use the scrubbed name when the name type is LocalHostName
A bug existed in systemd 245 that did not properly handle unknown kernel
capabilities gracefully. This resulted in incomplete output when querying
for the service status. It is possible to get service status by other means.
This PR works around this issue by getting service status using other commands
in the event of a failure due to this bug.
Change:
- Cryptography 3.2 drops support for OpenSSL 1.0.2. Some of our CI
infrastructure still uses this version (FreeBSD, namely). For now,
just add a constraint to use old cryptography.
Test Plan:
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
* GitHub is removing the underlying API used to implement the `login` command. Since the general consensus seems to be that relatively nobody currently uses this command (in favor of explicit token passing), support was simply removed for interactive login. If a future need arises, this command should be reimplemented via OAuth Device Auth Grants.
* login or role login commands now produce a fatal error with a descriptive message
* updated 2.10 and 2.11 porting guide entries
* remove dead code/config, update messages and porting guides
* VMware: scenario guide for vmware_tools connection plugin
Scenario doc to guide user about the usage of vmware_tools
using connection plugin.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* review comments
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Fix ansible-test docker container detection.
* Attach test containers to the correct network.
* Do not assume `localhost` for accesing Docker.
* Look for containers on current network.
* Always map /var/run/docker.sock into containers.
This fixes issues when using a remote Docker host.
* Support container IP lookup from networks list.
* Fix container network attachment.
* Remove redundant container detection messages.
* Limit DOCKER_HOST parsing to TCP.
* Restore docker socket existence check.
The check is skipped if the docker hostname is not localhost.
* Correct changelog entry.
* Fix the reversed order of return values
AnsibleModule.run_command returns a tuple of return code, stdout and stderr.
The module main function of the user module expects user.create_user to
return a tuple of return code, stdout and stderr.
Fix the locations where stdout and stderr got reversed.
* Fix another missed occurance
* Add changelog
* Fix further reversed order of return values occurrences
Followup to 72c3ba62c8:
Fix further reversed order of return values occurrences.
Change:
- containerd is now recognized as container tech
Test Plan:
- Tested with Docker inside Github Actions in a private repo:
"ansible_virtualization_role": "guest",
"ansible_virtualization_tech_guest": [
"containerd",
"container",
"VirtualPC"
],
"ansible_virtualization_tech_host": [],
"ansible_virtualization_type": "containerd",
Tickets:
- Fixes#66304 because this is what docker containers show up as
in Github Actions.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Fixing missing hosts entry under the group dict.
* Added missing 'vars:' key
* Reordering arguments for consistency with the format displayed by "ansible-playbook --help"
* Fixing contradictory text
* Ensure we call action_loader.get with collection_list. Fixes#72170
* Add tests and changelog
* Remove grep, do assertion in playbook. ci_complete
* Skip old jinja2 versions
* ci_complete
* dedupe
Change:
- In this test we end up upgrading dnf (and python3-dnf) so that we can
test its new logging behavior. However, the latest Fedora 32 dnf had a
packaging issue which caused it to not pull in the latest
python3-libdnf. This is fixed, but not synced out to mirrors yet.
Fixing it in this test will get CI passing again in the meanwhile.
Test Plan:
- CI
Tickets:
- https://bugzilla.redhat.com/show_bug.cgi?id=1887502
Signed-off-by: Rick Elrod <rick@elrod.me>
make collections whitelist follow normal flow
* fixes missing set_options call and adhoc and stdout processing rules
* avoid dupes
* fixed to handle redirects
* also updated tests with new and more accurate skip message
* fix callback tests for envs with cowsay installed
* lots MOAR comments on why the code is as it is, some todos to refactor in future
Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
beside the fact that naming the variable to specify the number of machines executed in *parallel* is named *serial* - the number of hosts chosen in this example is not optimal, either: 2x2 = 4; so - 3 hosts in 2 batches should clear things up a little
Change:
- When a plugin defines `type: str` on a parameter, treat more kinds of
input as a string instead of whatever it is parsed as.
Test Plan:
- New unit tests
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
* Add support for GSSAPI/Kerberos to urls.py
* Test out changes with the latest test container
* Get remote hosts working
* Fix up httptester_krb5_password reader
* Fix tests for opensuse and macOS
* Hopefully last lot of testing changes
* Dont do CBT on macOS
* Fixes from review
* Add explicit apt tests for fnmatch and update_cache
* Add explicit apt_key tests for fetching key directly from url
* ci_complete ci_coverage
* Remove repo only by repo
* ci_complete ci_coverage
* Add apt cache update after apt_repository to show that the cache doesn't update
* ci_complete ci_coverage
* Add systemd tests for enabling and disabling a service
* ci_complete ci_coverage
* Remove incidental_zabbix_host
* ci_complete ci_coverage
* Fix various sanity errors in plugins.
* Revert callback_type -> type transform.
* Undo paramiko_ssh 'connection' change, since this discrepancy is hardcoded in the plugin loader.
* Standardize on name and type (for callbacks).
* Fix existing author entries.
* Add 'Unknown (!UNKNOWN)' as author when author is missing.
* These are actually parsed as integers in the code.
* Revert "Add 'Unknown (!UNKNOWN)' as author when author is missing."
This reverts commit 29d1438aca.
* Add a test suite for module_utils.common.dict_transformations
* ci_complete
ci_coverage
Add a wait_for test using delegate_to
* Remove incidental_ec2_instance
* Remove unused test support modules
* Requested changes
ci_complete
ci_coverage
* Oops, put everything back to test coverage again
ci_complete
ci_coverage
* Remove incidental_ec2_instance tests and supporting modules
Change:
- No longer fail due to old Fedora/RHEL and our failure to try to cast
gids to integers before trying to pass them to getgrgid() before
trying to use them.
- Add tests for user/mode for various unarchive formats.
Test Plan:
- New integration tests, ran against centos6 container
Tickets:
- Fixes#71903
Change:
- Make KubeVirt be identified as such, and with a guest role.
- This is a breaking change, since playbooks could be relying on the
incorrect behavior previously.
Test Plan:
- Issue author
Tickets:
- Fixes#72001
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- podman > 2 && < 2.2 does not support "images --format {{json .}}"
- podman also now outputs images JSON differently than docker
- Work around both of the above.
Test Plan:
- Tested with podman 2.0.6 in Fedora 31.
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Change:
- Use `sysctl -n` for openbsd uptime information
- Allow `get_sysctl()` to account for multi-line sysctl settings
- Add unit tests for `get_sysctl()`
Test Plan:
- New unit tests
Tickets:
- Fixes#71968
- Refs #72025
- Refs #72067
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Brian Coca <brian.coca+git@gmail.com>
Co-authored-by: Brian Coca <bcoca@ansible.com>
The luseradd / lusermod commands do not support the -e option. Set
the expiry time in this case via lchage after the user was
created / modified.
Fixes: #71942
In Python3 math.floor returns an integer whereas Python2 returns a float.
Hence always convert the result of math.floor to an int to ensure that
lexpires is an integer.
Move local expires tests in a separate file and import the tasks to the
main.yml to keep main.yml smaller.
Module package is actually a proxy to a native package manager
modules (internally it delegates to the module that matches the
results of ansible_pkg_mgr).
Fixes#69371
b6b238a fixed the SLES4SAP detection, which was at this time ok.
Sadly Suse changed with SLES 15 the /etc/os-release file, so the above
change will no longer work.
This commit updates the SLES4SAP detection regarding
https://www.suse.com/support/kb/doc/?id=000019341.
The symlink realpath is matched with endswith, because in SLES 12+ the
link target is SLES_SAP.prod, but in SLES 11 the link target is
SUSE_SLES_SAP.prod.
* Validate removal versions.
* Validate that removal collection versions and version_added collection versions conform to semver spec.
* Validate removal version numbers in meta/runtime.yml.
* Stricter validation for isodates (f.ex. YYYY-M-D is not allowed).
* Improve error reporting.
* Validate removal collection versions.
Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
Change:
- pip packages should get removed after, not try to add them again
- Try removing containerd.io package too
Test Plan:
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
* Use default group of staff on macos to prevent sudo issues
* Install gnu-tar for macos in git and unarchive tests
* Enable timezone module to support py3 on macos
* If the virtualenv command is missing, try python -m virtualenv
* Install passlib for filter_core on macos
* Install paramiko via pip on macos for paramiko tests
* Normalize discovered python interpreter on macos
* Get pip tests passing, by ensuring we have wheel installed
* Create /etc/ansible for ca certs on mac, list lookup_url as destructive
* Fixups for CA certs
* Include macos
* Dynamically get cafile instead of hardcoding the path
* DOCS: updates intersphinx references for docs links
* TESTS: Raise the number of bytes scanned to determine if a file is binary. The newest ansible-2.10.inv file has its first null byte at position 2261. 4096 is still a cheap chunksize to read so it still makes sense to raise this.
Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
This change preserves how all the external imports refer to this code
while allowing us to start cutting the spaghetti into more easily
maintainable pieces.
This is a start of the upcoming refactoring effort destined to
eliminate tight coupling, implicit data manipulation, god objects,
abstraction leaks and other code smells.
Essentially, `ansible.galaxy.collection` is going to be a package that
holds parts of the collection management code spread across loosely
coupled modules.
PR: #71931
Change:
- The docker-ce.repo file for centos does not work on RHEL since it uses
$releasever and on RHEL that is, e.g., "7Server".
- Instead, set up the repo manually.
- Additionally, the docker centos8 repo no longer has old versions, so
we use the (only) version in the repo instead.
Test Plan:
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
* Let get_file_attributes() work without `lsattr -v`
Change:
- module_utils's get_file_attributes() expects `lsattr -v` to work, but
in some cases, it may not.
- The function now takes an optional include_version bool parameter,
which removes this expectation.
- Places where we call get_file_attributes() without using the 'version'
it returns, we now call it with include_version=False.
Test Plan:
- New unit tests
Signed-off-by: Rick Elrod <rick@elrod.me>
* Reorder test/sanity/ignore.txt to better see changes
* Remove extra empty line causing pass fail
* Apply suggestions made by Akasurde
* Minor fix in package_facts and remove two unnecessary ignores
* Fix subversion based on a suggestion made by felixfontein
* Apply suggestions made by felixfontein and Andersson007
* Fix subversion.py as suggested by felixfontein
* Minor reformatting in yum_repository description
* Reformat changelog
* Add key to apt_key, add deprecated changelog
* Add PR url to changelog
* Ignore paramater-type-not-in-doc in favour of adding key back to apt_key
* Fix apt_key
* Remove undocumented-paramater from apt_key ignore
* Ignore doc-choices-do-not-match-spec in package_facts
* Fix package_facts
* Fix filter option in setup module
Change:
- In some cases (always with free strategy, sometimes with linear), the
default callback would not show the task banner for include_tasks.
- This only affects the include_tasks task itself, not the tasks in the
included file.
Test Plan:
- Updated default callback tests
Tickets:
- Fixes#71277
Signed-off-by: Rick Elrod <rick@elrod.me>
The test is not supported when running in a container. It now recognizes both 'docker' and 'container' as virtualization types that should cause the test to be skipped.
A recent update to cffi that was yanked is still being installed on our
Mac OS X 10.11 test image since the version of pip there is very old and
does not ignore yanked packages.
Pin the version of pyOpenSSL and its dependencies to fix this and avoid
future spontaneous failures.
* [docs] add porting guide for DNF GPG validation
Change:
- This was a breaking change (security fix), but I neglected to add a
porting guide entry for it previously.
Tickets:
- Refs #71537
- Refs #71539
- Refs #71540
- Refs #71541
Signed-off-by: Rick Elrod <rick@elrod.me>
* changes from sivel
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Previously when `security: true` and `bugfix: true` were both given,
only security updates would get applied. Filters now accumulate so
that both get applied in this case.
Test Plan:
- New integration tests for both check_mode and not. These tests make
use of a contrived yum repository which is stored in S3.
Tickets:
- Fixes#70854
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Matt Martz <matt@sivel.net>
* fix coverage output from synthetic packages
* synthetic packages (eg, implicit collection packages without `__init__.py`) were always created at runtime with empty string source, which was compiled to a code object and exec'd during the package load. When run with code coverage, it created a bogus coverage entry (since the `__synthetic__`-suffixed `__file__` entry didn't exist on disk).
* modified collection loader `get_code` to preserve the distinction between `None` (eg synthetic package) and empty string (eg empty `__init__.py`) values from `get_source`, and to return `None` when the source is `None`. This allows the package loader to skip `exec`ing things that truly have no source file on disk, thus not creating bogus coverage entries, while preserving behavior and coverage reporting for empty package inits that actually exist.
* add unit test
On some systems (tested with official Debian Buster-based Python 3.6
docker image), setting the LC_ALL environment variable to en_US.UTF-8
will trigger Python into switching its preferred encoding to ASCII.
If any python process tries to read a non-ASCII file in this scenario,
it will terminate with an error.
And this is exactly what happens to pytest when it tries to load its
configuration that ansible-test supplies because the configuration
contains an em dash.
In order to bypass this issue, we replaced the em dash with a regular
dash and things started working again.
An alternative solution would be to replace the en_US.UTF-8 locale with
something safer, but unfortunately, the en_US.UTF-8 is probably as safe
as it gets.
The setup module can now filter out multiple pattern by providing a list
to the filter parameter instead of just a string. Single string sill
works. Previous behaviour remains.
(cherry picked from commit b5c36dac483fdd74d6c570d77cc8f3e396720366)
* Use InventoryManager for inventory_hostnames lookup. Fixes#17268
* Add test for using patterns
* s/it's/its/
* Allow bypassing parse_sources in InventoryManager
* Remove unneeded method call
In order to test this piece of functionality, we create a dedicated
collection that contains files that should be ignored when building
the collection tarball.
After the collection is built, we make sure the produced tarball does
not contain any files that should be ignored.
Change:
- By default the dnf API does not gpg-verify packages. This is a feature
that is executed in its CLI code. It never made it into Ansible's
usage of the API, so packages were previously not verified.
- This fixes CVE-2020-14365.
Test Plan:
- New integration tests
Signed-off-by: Rick Elrod <rick@elrod.me>
test_dict_undefined_values is a duplicate of test_dict_undefined_values_is_defined.
It seems originally test_dict_undefined_values was supposed
to do what test_dict_undefined_values_bare does so it is safe to remove.
Remove "mode: preserve" option from documentation in doc fragments "FILE_COMMON_ARGUMENTS", as it was incorrectly included in the documentation for the `lineinfile`, `unarchive` and other file-related modules.
The `copy` and `templates` modules documentation remains untouched and still contain "mode: preserve", as intended.
* Validate salt when using crypt. Respect salt_size in password lookup. Repair salt for bcrypt. Fixes#71107. Fixes#53750. Fixes#36129.
* Handle algorithms we don't know about, and make sure to return the salt
* Account for old passlib
* Add tests for salt constraints
* Add changelog fragment
* Add test for #36129
Some parameters for systemd are ignored, add a note about such parameters in documentation.
Fixes: #23144
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
nitzmahone's blog nicely explained why Windows is not supported
as Ansible controller. Link that in documentation so users can
read about it.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* added configurable dir for tree callback
- allows usage in playbooks
- also made errors more specific if bad dir is supplied
- ensure we expand paths
* docfix
This script is mostly unmaintained and relies on the deprecated and
unmaintained `boto` library. Featuring it prominently in the docs
leads to many new users using it instead of the supported `aws_ec2`
inventory plugin.
##### SUMMARY
Per #67913, when comparing dependencies, Ansible takes into account parameters, tags and the when clause in determining whether a role is a duplicate or not.
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
* quick fix to change main back to devel
* Update docs/docsite/rst/dev_guide/developing_collections.rst
Co-authored-by: Felix Fontein <felix@fontein.de>
Change:
- Now sends meta tasks to the task start callback
- Lets callback plugins opt-in to receiving implicit tasks
Test Plan:
- New integration tests
Tickets:
- Indirectly fixes#71007 by allowing custom callbacks with this data
Signed-off-by: Rick Elrod <rick@elrod.me>
ini-style inventory does not support Ansible Vault password.
This fixes network_best_practices_2.5 doc.
Fixes: #69039
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
skip_missing parameter in subelements lookup plugin is accepted from
inside the dictionary.
Fixes: #38182
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
"command -v" is the POSIX compliant way to get the path to a command.
It returns the path to the command if it exists and exits with a
non-zero status when it does not, without any output. "which" is not
POSIX and it can have various different effects. With Zsh, it says
"python not found" on stdout.
See:
- https://pubs.opengroup.org/onlinepubs/009695399/utilities/command.html
Signed-off-by: Vincent Bernat <vincent@bernat.ch>
* check run state of current block only
* Add changelog and test
* Add test for issue 29047
* Fix for both tests
* blerg
* Change test messages
* fix tests
* Add multi-level block in rescue test case
* Add recursive rescue check and multi-level test
* Should probably run the new test
* ci_complete
* Merge new tests
* ci_complete
* Don't do conflict check on sdist and egg_info. Fixes#71279
* Add changelog
* adjust changelog PR link
Co-authored-by: Rick Elrod <rick@elrod.me>
* add warning about bdist_wheel
Co-authored-by: Rick Elrod <rick@elrod.me>
This would be a partial solution for #69364 in that the SHASUMS file can be downloaded and gpg verified but then used from the downloaded location to verify the get_url's file.
* Make checksum url parsing more explicit
Use urlsplit to test if the checksum string has a (currently tested and) supported url scheme.
* Fix whitespace
* Changelog fragment
* Added tests
* Fix typo in test setup
The message generated by systemctl has been updated in 9321e23c40, which requires a corresponding change in the systemd module.
In addition, this fixes the module when the SYSTEMD_OFFLINE environment variable is set.
* POC for supporting callback events that come from the worker
* linting fixes. ci_complete
* fix up units. ci_complete
* Try moving the sentinel put higher. ci_complete
* safeguards. ci_complete
* Move queue killing to terminate
* LINTING. ci_complete
* Subclass Queue, to add helper send_callback method
* Just use _final_q instead of adding another queue and thread
* Revert a few changes
* Add helper for inserting a TaskResult into the _final_q
* Add changelog fragment
* Address rebase issue
* ci_complete
* Add test to assert async poll callback from fork
* Don't use full path
* ci_complete
* Use _results_lock as a context manager
* Add new generic lock decorator, and use it with send_callback
MacOSX seems to want bare `+` whereas GNU expr wants escaped `+` (`\+`)
to mean match one or more. Use `\{1,\}` instead which will match one or
more on both MaxOSX and GNU-using systems.
Fixes#71053
* [docs] 2.7 is EOL, add 2.10 which is almost out
- Remove 2.7 support from the maintenance schedule
- Add 2.10 which is in RC and will be out soon enough.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Update docs/docsite/rst/reference_appendices/release_and_maintenance.rst, fix table and separate ansible-base from ansible, fix rstcheck errors, clean up sections, explain the two packages
Co-authored-by: Sandra McCann <samccann@redhat.com>
Co-authored-by: Rick Elrod <rick@elrod.me>
* Revert "Change default file permissions so they are not world readable (#70221)"
This reverts commit 5260527c4a.
* Revert "Fix warning for new default permissions when mode is not specified (#70976)"
This reverts commit dc79528cc6.
ansible_play_batch lists the currently targeted host(s) in the serial/batch, while
ansible_play_hosts lists all the hosts which will be targeted by the play.
Password lookup can be used to generate random string
with desired length. This is useful in various scenarios.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Ensure -k is set to delegated hosts without a pass
* Fix up some broken tests
* Update task_executor.py
one possible fix, the other is updating winrm to normalize on 'password' like the other connection plugins
* Add alias for winrm and fix incorrect assumption
* Make sure aliases are used for keyword options
* Conditionally run test if sshpass is present, fix sanity
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
* Start of alpine testing
* More updates
* Add forgotten file
* remove debug
* Add alpine3
* equal
* group 4
* group 4
* group 5
* Try to decrease test length
* libuser only available in testing
* Remove debug
* Make loops target work on hosts without gnu date
* Enable alpine testing
* ci_complete
* Don't specify uid for creating test user
* ci_complete
* Re-sort docker completion
* use newer container image
* ci_complete
* fix indentation
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Clay <matt@mystile.com>
* The version comparison for determining what ansible package to build
docs against was comparing the version number for ansible-base but it
needed to check the version number for ansible instead
* add a comment about some bad logic than needs to be fixed after 2.10.0
The ansible collection repository correctly renamed their default branch from `master` to `main`, which has caused a number for broken urls. This PR fixes those urls.
Change:
- Allow systems to declare multiple virt techs. For example if a system
is both a docker container, but virtualized on KVM, show both. If a
system is set up to run virtualbox and KVM VMs, show both.
- This is done by introducing new facts keys:
- virtualization_tech_guest
- virtualization_tech_host
- Backwards compatibility is preserved by keeping track of the previous
return-points and refusing to update those keys after we would have
returned, but now returning them at the end, so that the new keys can
accumulate their data.
Test Plan:
- Local
- CI
Tickets:
- Refs #66304
- Refs #17151
- Refs #17058
- Probably others
Signed-off-by: Rick Elrod <rick@elrod.me>
* Fix tty_ify bugs and refactor
* Move tty_ify() and supporting attributes to the DocCLI class as that's
the only thing using it.
* Add unittest for the code.
* Fix a bug where the substitution macros can be detected when they are
a part of another word.
* Add support for L(), R(), and HORIZONTALLINE which were added to the
website docs many years ago.
* Update test/units/cli/test_doc.py
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Clay <matt@mystile.com>
* add jinja2 global to reserved names
also allow expansion by additional context provided from caller
fixes#41955
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Change:
- Use `chmod +a` in the fallback chain to allow MacOS to use ACLs to
allow an unprivileged user to become an unprivileged user.
Test Plan:
- CI, new tests
Tickets:
- Fixes#70648
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Adds some intentional coverage around PluginLoader for cases that
incidental_azure_rm_resource covered.
- Specifically, modules starting with an underscore, and starting with
an underscore but a symlink.
Test Plan:
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Add hardware_facts test target which manually sets up some LVM devices
and tests facts against them.
Test Plan:
- New integration tests
Tickets:
- Refs #71041 and #71042 both of which I discovered during this
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Add a parameter `read_whole_file` which allows for reading the whole
file when doing a `contains` regex search.
- This allows for (for example) matching a pattern at the very end of
a file.
Test Plan:
- New integration tests
Tickets:
- Fixes#63378
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Previously, `cache_timeout=0` was seen as falsey and not passed along
to cache plugins as an option. Now only "nulley" values are not sent.
Test Plan:
- New integration tests
Tickets:
- Fixes#70702
Signed-off-by: Rick Elrod <rick@elrod.me>
Changes to sanity and unit tests now trigger the ansible-test self-test integration tests.
No changelog entry since this only affects tests for ansible itself and not collections.
Change:
- Previously CachePluginAdjudicator#flush only removed entries from the
cache backend that it knew about by using them earlier. Now it calls
the underlying plugin's flush() method.
Test Plan:
- New unit tests
Tickets:
- Fixes#68770
Signed-off-by: Rick Elrod <rick@elrod.me>
There are links from the scenario guides to collections but collections
docs aren't built in testing or locally when we're on the devel branch.
Due to that we need to make sure those references resolve to the
production docsite. We can use intersphinx to make sure that happens.
* The test for binary files wasn't reading enough of the file.
Checking for null bytes in the first 1024 bytes failed to diagnose the
ansible_2_10.inv file as binary
* Strip spaces in module names in explicit actions
Change:
- When an action is called like "action: copy foo=bar", strip spaces
around the action name.
- This allows "action: copy foo=bar" to work as expected.
Test Plan:
- New integration tests
Tickets:
- Fixes#62136
Signed-off-by: Rick Elrod <rick@elrod.me>
Update the docs to guide users to use `ansible-runner` instead of using Python API directly. In many use cases, executing Ansible playbooks are sufficient. In those use cases, `ansible-runner` is easier and much stable to use comparing with Python API, but there is no mention of it.
* Incase of network integration test for connection local
test the paramiko_ssh auth_timeout is the value of timeout
under defaults section which is 10 seconds.
* For slower connection 10sec timout value result in authentication
timeout error hence increase the timeout value to 90 seconds
* Fix changelog link title.
* Rename Ansible 2.10 and 2.11 porting guides to Ansible-base porting guides.
* Add stub for automatically generated 2.10 porting guide.
* Move things that should not be in the ansible-base porting guide to the ansible porting guide.
* Apply changes to base porting guides.
* Add remark that ansible-base is mainly for developers.
* Ansible Base -> Ansible-base
* Fix link in base porting guide.
* Add generated porting guide.
* Use same header signs as antsibull-changelog's RST builder.
* Update generated porting guide.
Change:
- Fix a bug where rc/beta versions throw off the "is this an
ansible-base release"? check.
Test Plan:
- Used it for 2.10.0rc4
Signed-off-by: Rick Elrod <rick@elrod.me>
* document how to migrate between collections
* Apply suggestions from code review
Co-authored-by: John R Barker <john@johnrbarker.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Add integration tests for unarchiving as unprivileged user
Break tasks into separate files for easier reading and maintenance
Create a user by specifying a default group of 'staff' for macOS.
The user module does not actually remove the user directory on macOS,
so explicitly remove it.
Put the removal tasks in an always block to ensure they always run
Co-authored-by: Philip Douglass <philip.douglass@amadeus.com>
Co-authored-by: Sam Doran <sdoran@redhat.com>
The iso8601_micro and iso8601 facts incorrectly called now.utcnow(), resulting
in a new timestamp at the time it was called, not a conversion of the previously
stored timestamp.
Correct this by capturing the UTC timestamp once then calculating the local
time using the UTC offset of the current system.
* Use time.time() for getting the current time
* Convert from that stored epoch timestamp to local and UTC times
* Used existing timestamp for epoch time
* Add unit tests that validate the formate of the return value rather than an exact value since mocking time and timezone is non-trivial
Follow up to #70221
Related to #67794
CVE-2020-1736
When set_mode_if_different() is called with mode of 'None', ensure we issue
a warning about the change in default permissions.
Add integration tests to ensure the warning works properly.
* Fix tests
- actually use custom module 🤦♂️
- verify file permission on created files
- use remote_tmp_dir so we're ready for split controller
- improve test module so we can skip the call to set_fs_attributes_if_different()
- fix tests for CentOS 6
Change:
- Clarify that not passing `--tags` will cause `ansible_run_tags` to
default to `["all"]`.
- Add some extra coverage around `ansible_run_tags`
Test Plan:
- New integration and unit tests
Tickets:
- Fixes#69619
Signed-off-by: Rick Elrod <rick@elrod.me>
- Combine remote and local debugging instructions.
- Update the example code to match current AnsiballZ structure and behavior
- Change reference name and update references
- Clarify how PYTHON path is modified
- Also add note about other remote debugging tools.
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
Change:
- Allow older git to verify tags again
- Enable verification tests everywhere, even if most of them only work
on newer git. Some of them work on older git and they test the --raw
parameter.
Test Plan:
- Re-enabled subset of git tests
Tickets:
- Fixes#64469
Signed-off-by: Rick Elrod <rick@elrod.me>
* Fix verbosity for var_templating test
Display the subtests
* fix specificity and set ANSIBLE_BECOME_ALLOW_SAME_USER
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Matt Martz <matt@sivel.net>
Change:
- Previously, we only showed that something would have changed, not what
would have changed. This allows us to show what will chang as well.
Test Plan:
- Local RHEL8 VM
- New integration tests
Tickets:
- Fixes#66132
Signed-off-by: Rick Elrod <rick@elrod.me>
* Also validate top-level version_added.
* Fix error code.
* Produce same version_added validation error in schema than in code (and stop returning it twice).
* Return correct error codes for invalid version_added for options and return values.
* Add changelog.
* Fix forgotten closing braket.
* Accept 'historical' for some top-level version_added.
* dnf: Add nobest option
* dnf: Fix indent, add nobest specifically to dnf not yum
* Add changelog for dnf: add nobest option
* dnf: Add nobest to yumdnf module argument_spec
* dnf: remove nobest from module paramaters in yumdnf.py
* dnf: Add test for nobest option
* dnf: Cleanup packages in nobest test at last
* dnf: Cleanup manually added repos in nobest test at last
* dnf: Remove dnf-plugins-core as well in nobest test
* dnf: Change nobest release version to 2.11
* Change changelog number according to change in PR number
* Change changelog number according to change in PR number
Change:
- On Linux, there are situations where a host might be a KVM host but
not have the kernel module enabled (it might be compiled in instead).
In these cases, /dev/kvm will still exist, and rather than reporting
NA, we should report that the host is a KVM host.
Test Plan:
- Local
Signed-off-by: Rick Elrod <rick@elrod.me>
PR #70446: it's a follow-up for #70445.
It includes a merge of `examples/scripts/uptime.py` and a similar
code snippet from `docs/docsite/rst/dev_guide/developing_api.rst`.
This patch also changes the docs RST file to include contents of
the example file instead of holding a copy of a similar code.
* Change default file permissions so they are not world readable
CVE-2020-1736
Set the default permissions for files we create with atomic_move() to 0o0660. Track
which files we create that did not exist and warn if the module supports 'mode'
and it was not specified and the module did not call set_mode_if_different(). This allows the user to take action and specify a mode rather than using the defaults.
A code audit is needed to find all instances of modules that call atomic_move()
but do not call set_mode_if_different(). The findings need to be documented in
a changelog since we are not warning. Warning in those instances would be frustrating
to the user since they have no way to change the module code.
- use a set for storing list of created files
- just check the argument spac and params rather than using another property
- improve the warning message to include the default permissions
* ensure hostvars are available on delegation
* also inventory_hostname must point to current host and not delegated one
* fix get_connection since it was still mixing original host vars and delegated ones
* also return connection vars for delegation and non delegation alike
* add test to ensure we have expected usage when directly assigning for non delegated host
Some platform such as ESXi does not implement EpollSelector,
which is selected by DefaultSelector. Use SelectSelector which is
based upon 'select' implementation. This works perfectly with
a platform like VMware ESXi.
Fixes: #70238
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Change:
- Remove "New Module Pull Request" since we're not taking new modules in
ansible/ansible
- Add "Tests Pull Request" to use for things like adding coverage that
aren't bugfixes.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Encode/Decode files in UTF-8
* Use helper function in ansible
* Add an integration test
* Use emoji in test data.
* add changelog
* Also support non-ascii chars in filepath and add tests about this.
* Also use non-ascii chars in replaced text and ensure not to break cron syntax.
* rename self.existing to self.n_existing
* rename crontab.existing to crontab.n_existing
Change:
- Refactoring to make it harder to get wrong and easier to read.
- Generalize become_unprivileged tests and fix some that never worked
but also never failed.
Test Plan:
- CI, new units/integration tests
Signed-off-by: Rick Elrod <rick@elrod.me>
* Format using `` instead of `, add line breaks for long lines, rephrase or remove useless text.
Move some text.
* Add clearer version of OpenSSh is affected by SCP bug
* Review some pages using ansible doc writing guide.
* Guard against allowing ansible to ansible-base upgrades
* newline
* use alias
* Add an explicit line detailing this is a 1 time thing
* period
* Read __version__ and __author__ rather than import, update working, and add ability to skip conflict checks
* Remove commented code
* Re introduce removed changes from rebase
* Just use open
* Nuke unused import
* Fix missing quoting for remote_tmp in second mkdir of shell module. Issue #69577
* adding changelog
* fixing typo in changelog entry
* adding test case
Adding test case written by bmillemayhias.
* using $HOME instead of ~
* fixing commit measage
* Update 69578-shell-remote_tmp-quoting.yaml
Co-authored-by: Brian Kohles <me@briankohles.com>
debconf module exposes sensitive information to logs, console.
Add a note to user about using no_log=True to hide such
information from console.
Fixes: #32386
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
From what I have observed it is the block itself that doesn't support the name attribute rather than the tasks inside the block.
* Update docs/docsite/rst/user_guide/playbooks_blocks.rst
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
* Followup to #70610: fix redirects to wrong collections
* gcp module_utils is in community.general.
* dellos10_* modules are in dellemc_networking.os10, not dellemc.os10.
* intersight_facts was renamed to intersight_info in cisco.intersight in version 1.0.7.
* Fix more dellos10 redirects.
A couple of years ago Slackware -current began using a plus (“+”) at the end of the distribution version string to indicate a future version work-in-progress.
Rearrange distribution_files unit tests to easily support more tests
- add conftest with common fixtures
- use parametrize for testing multiple scenarios
* Add changelog
* Add unit tests for Slackware distribution parsing
* Use correct fixtures for Slackware
Data comes from /etc/slackware-version
Co-authored-by: Sam Doran <sdoran@redhat.com>
Co-authored-by: <Eduard Rozenberg <eduardr@pobox.com>>
* Build documentation for Ansible-2.10 (formerly known as ACD).
Builds plugin docs from collections whose source is on galaxy
The new command downloads collections from galaxy, then finds the
plugins inside of them to get the documentation for those plugins.
* Update the python syntax checks
* docs builds can now require python 3.6+.
* Move plugin formatter code out to an external tool, antsibull-docs.
Collection owners want to be able to extract docs for their own
websites as well.
* The jinja2 filters, tests, and other support code have moved to antsibull
* Remove document_plugins as that has now been integrated into antsibull-docs
* Cleanup and bugfix to other build script code:
* The Commands class needed to have its metaclass set for abstractmethod
to work correctly
* Fix lint issues in some command plugins
* Add the docs/docsite/rst/collections to .gitignore as
everything in that directory will be generated so we don't want any of
it saved in the git repository
* gitignore the build dir and remove edit docs link on module pages
* Add docs/rst/collections as a directory to remove on make clean
* Split the collections docs from the main docs
* remove version and edit on github
* remove version banner for just collections
* clarify examples need collection keyword defined
* Remove references to plugin documentation locations that no longer exist.
* Perhaps the pages in plugins/*.rst should be deprecated
altogether and their content moved?
* If not, perhaps we want to rephrase and link into the collection
documentation?
* Or perhaps we want to link to the plugins which are present in
collections/ansible/builtin?
* Remove PYTHONPATH from the build-ansible calls
One of the design goals of the build-ansible.py script was for it to
automatically set its library path to include the checkout of ansible
and the library of code to implement itself. Because it automatically
includes the checkout of ansible, we don't need to set PYTHONPATH in
the Makefile any longer.
* Create a command to only build ansible-base plugin docs
* When building docs for devel, only build the ansible-base docs for
now. This is because antsibull needs support for building a "devel
tree" of docs. This can be changed once that is implemented
* When building docs for the sanity tests, only build the ansible-base
plugin docs for now. Those are the docs which are in this repo so
that seems appropriate for now.
A recent updated to psutil, which is a dependency of ansible-runner, fails
to install on older versions of pip.
Commit with the breaking change:
135628639b
* Get m_u.facts.utils coverage up to 100%
Change:
- Add tests to 'gathering_facts' integration target to get
module_utils.facts.utils coverage up to 100%.
- This also clears incidental coverage from incidental_selinux.
Test Plan:
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
A little further down the page is another index, ansible_loop.index, which shares a similar description but is 1 indexed.
Its zero indexed twin has a 0 suffix.
``ansible_loop.index`` The current iteration of the loop. (1 indexed)
``ansible_loop.index0`` The current iteration of the loop. (0 indexed)
To remove ambiguity around the usage of index_var, explicitly mention that this variable is 0 indexed.
Change:
- Test async_wrapper when the module it runs has stderr output
Test Plan:
- CI
- Looked at coverage report and saw green for a few lines that weren't
previously green.
Signed-off-by: Rick Elrod <rick@elrod.me>
pipe lookup plugin uses Popen with shell=True intentionally.
This is considered a security issue if user input is not validated.
Updated docs to reflect this information for the user. Also, added
Bandit B602 documentation link for further reading.
Fixes: #70159
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Change:
- Add integration tests for various cases
- Fix wrong use of "its" in an exception thrown in varnames when it
throws an AnsibleError, given a term of the wrong type.
Test Plan:
- new tests, CI
Tickets:
- Fixes#70546
Signed-off-by: Rick Elrod <rick@elrod.me>
- ensure we preserve the typeerror part of the exception so loop defereed error handling
can postpone those caused by undefined variables until the when check is done.
- fix tests to comply with the 'new normal'
- human_to_bytes and others can issue TypeError not only on 'non string'
but also bad string that is not convertable.
Co-authored-by: Sloane Hertel <shertel@redhat.com>
Co-authored-by: Sloane Hertel <shertel@redhat.com>
Change:
- Use parse_kv() for parsing in the csvfile lookup plugin. This allows
us to handle multi-word search keys and filenames. Previously, the
plugin split on space and so none of these things worked as expected.
- Add integration tests for csvfile, testing a plethora of weird cases.
Test Plan:
- New integration tests, CI
Tickets:
- Fixes#70545
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Move hostnamectl check out of GenericStrategy because it was incorrect
for everything except the SystemdStrategy which is where it belongs.
- Add some initial tests for the hostname module, though we are limited
by the fact that we can't do much testing with it in containers.
Test Plan:
- new hostname integration tests
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Sam Doran <sdoran@redhat.com>
Change:
- Followup to #70504. We need to not pass the 'warn' parameter from the
action plugin either, unless it's True. Otherwise, even though it
defaults to false, we always show the deprecation.
Test Plan:
- Local
Signed-off-by: Rick Elrod <rick@elrod.me>
* speed up modify_module
* Remove debugging
* ci_complete
* Simplify generic_visit, alias to visit, eliminate some attr lookups
* ci_complete
* Add changelog fragment
* Fix ansible-test error in community.aws
* Add changelog entry for fix
* Change check from None to string_types
* Update changelogs/fragments/70507-validate-null-author.yaml
clarify wording "or a list of strings"
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/schema.py
clarify wording - single string or not specified valid
Co-authored-by: Felix Fontein <felix@fontein.de>
* Do not fail but return None when given outside list
Co-authored-by: Felix Fontein <felix@fontein.de>
Change:
- The command warnings feature which suggests that users use modules
instead of certain commands is now deprecated. Its `warn` paramater
and `COMMAND_WARNINGS` configuration options are also deprecated.
Their use will become an error in version 2.13.
Test Plan:
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Sam Doran <sdoran@redhat.com>
Change:
Our handling of NetBSD virtualization facts led to facts that were just
plain incorrect. One example is reporting Xen even when the system is
running on something completely different (like KVM).
As stated by the reporter of #69352, NetBSD has a better sysctl setting
to use for this information, machdep.hypervisor.
This PR does the following:
- Try to use machdep.hypervisor sysctl value if the other sysctl values
we check don't end up with enough information to be useful
- Only look for /dev/xencons and assume Xen if nothing else works
(Really this should probably return 'unknown' since the file exists on
non-Xen systems and is not very useful).
- Add a few more patterns (Xen matches and also Hyper-V) to
VirtualSysctlDetectionMixin#detect_virt_product.
This change is slightly breaking:
- If the first two attempts at using sysctl worked before,
(machdep.dmi.system-product and machdep.dmi.system-vendor), they will
continue to work.
- For cases when those values didn't work, previously the existence of
/dev/xencons was checked, and if found, we reported 'xen' (even on
non-Xen systems when the file existed). After this PR, we try the
machdep.hypervisor sysctl key before still falling back to
/dev/xencons. This means that in some cases, we might go from
(wrongly) saying "xen" to giving a more accurate value such as "kvm"
or "Hyper-V".
Test Plan:
- Tested with local NetBSD VM and got 'kvm' instead of 'xen' back.
Tickets:
- Fixes#69352
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- module_utils.basic.is_special_selinux_path() used a string ==
bytestring comparison which returned False and made Ansible think that
certain filesystems aren't, in fact, special-cased, when they should
be. Ensure both sides of the == are bytestrings.
Test Plan:
- Added `copy` integration tests for this case.
Tickets:
- Fixes#70244
Signed-off-by: Rick Elrod <rick@elrod.me>
* Add type for options in the sample module shown in Developing Modules, as this is required to have the tests being
green
* Remove duplicated strings: the same sentence is in "Python tips" and "Module security", keeping the
latter.
* ansible-doc man formatter: do not crash when description isn't there.
* Change to report a better error message when description is not there.
* Add test.
Previous version initialized the `TaskQueueManager` after calling
`Play.load()` while advertising a way to inject a custom library
location path. This caused the tasks loader not to find any custom
modules because it was triggered before the path was actually added
to the module loader.
This patch changes the order of the operations to ensure that the
customized `context.CLIARGS` actually influences things.
Resolves https://github.com/ansible/ansible/issues/69758.
If a local module has no documentation, the doc command will fail without any hints of what is wrong. Add another way to confirm the presence of a local module.
* Update docs/docsite/rst/dev_guide/developing_locally.rst
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
When user uses home directory in --playbook-dir option
of ansible-inventory command, it warns user about this.
This PR suppress the warning message for user's home directory usage
in ansible-inventory command.
Fixes: #65262
Signed-off-by: Satyajit Bulage <sbulage@redhat.com>
The ssh_args were sometimes not correctly applied to the connection
when using _play_context. Use get_option() instead to ensure the
correct ssh_args are always applied.
Change:
- Update bundled six to 1.13 (last with py2.6 support)
- Make it pass lint
- Fix check to allow skipping over compat __init__.py files we authored
- Fix check to allow files that can't be updated for some reason
Test Plan:
- ansible-test sanity --docker
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
* Test against galaxy_ng
* Switch container image
* Remove redundant |default
* Re-enable
* Update image
* Update wording
* Don't use pulp as the container name
* Use the first galaxy server supporting v1 for roles. Fixes#65440
* Add changelog fragment
* This is best effort, fall back to original behavior if something bad happens
* misc collection metadata fixes
* parse collection meta with libyaml if available
* require only Mapping for validation
* add explanatory text for _meta_yml_to_dict
* ignore custom pylint rule
* this code shouldn't import a bunch of stuff from ansible, since it's run under the import sanity test
Change:
- In certain situations, such as when the input string contains null
bytes (\0), syslog.syslog will throw a TypeError. Handle that and
fail_json instead.
Test Plan:
- New test
- ansible-test --docker centos[68] (for py2 and py3 respectively)
Tickets:
- Refs #70269
Signed-off-by: Rick Elrod <rick@elrod.me>
* ansible-test - do not validate blacklisted ps modules
* Update changelogs/fragments/validate-modules-ps-doc-blacklist.yaml
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Clay <matt@mystile.com>
CIFS can be mounted using backward slash as well in /etc/fstab like
\\Windows\share /data/ cifs credentials=/root/.creds 0 0
Handle this condition while gather mount information in Linux.
Fixes: #48813
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Update documentation as per sanity tests
* Added example about custom SSH port in example section
Fixes: #29236
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Ensure not to remove existing packages while installing apt packages.
* Make all lines shorter than 160 characters
* Allow removing packages only when upgrading.
* Add integration tests
Change:
- Fix a UnicodeDecodeError in executor.module_common that could get
triggered with -vvvvv.
Test Plan:
- `ansible-test integration --docker centos7 module_utils -vvvvv`
This would show the error previously, and no loner does after this
patch.
Signed-off-by: Rick Elrod <rick@elrod.me>
In some usecases, we want to be able to clone a single branch
of a repository, without using --depth (which implies --single-branch).
* Use branch name when available
- update description of parameter
- consolidate branch or tag checking for easy reuse
* Add changelog
* Use static task imports rather than dynamic includes
* Add integration tests for single_branch
* Account for older versions of git
* Minor tweak to warnings
Co-authored-by: Laurent Coustet <laurent.coustet@clarisys.fr>
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Update docs/docsite/rst/dev_guide/developing_collections.rst
* add steps to create changelogs, add sentence about not using the tool
* add note for rerunning the command
Co-authored-by: Felix Fontein <felix@fontein.de>
* [Bugfix] curses.setupterm() error
When run playbook in celery task, curses.setupterm() will be failed
```
File "<frozen importlib._bootstrap>", line 675, in _load
File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
File "/Users/guang/.virtualenvs/ansible/lib/python3.6/site-packages/ansible/plugins/action/pause.py", line 45, in <module>
curses.setupterm()
TypeError: argument must be an int, or have a fileno() method.
```
* Add changelog
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Add description of collections keyword
* Update based on feedback.
- Add link to become plugins.
- Add note about how the collections keyword works with roles.
Handle colon appearing in filename while parsing the mimetype and charset
using file command.
Fixes: #70256
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Change:
- On Python 3.9, `crypt.crypt` will throw instead of returning `None`
when the algorithm isn't supported. So we catch that and handle it
the same way we handled the algorithm not being supported on 3.8: by
throwing AnsibleError.
Test Plan:
- CI for <=3.8.
- Local for 3.9b3:
ansible -m debug -a "msg=\"{{ 'changeme' | password_hash('bcrypt') }}\"" localhost
Before:
localhost | FAILED! => {
"msg": "Unexpected failure during module execution.",
"stdout": ""
}
After:
localhost | FAILED! => {
"msg": "crypt.crypt does not support 'bcrypt' algorithm"
}
Tickets:
- Fixes#69930
Signed-off-by: Rick Elrod <rick@elrod.me>
The `packaging` and `pyparsing` packages are now installed by `ansible-test` during provisioning of RHEL instances to match the downstream vendored versions.
Change:
- Remove _get_item() alias as it has been deprecated
- Update tests
- Remove relevant sanity curtailment
- Add changelog
Test Plan:
CI, grep
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
Removes with_* loop squashing and tests for 2.11
Test Plan:
CI, and grepped for with_items in package manager integration targets.
There might be some test cases in collections which need to stop testing
this behavior.
Signed-off-by: Rick Elrod <rick@elrod.me>
Rather than looking through tests.json to find the data file, look for it explicitly
within a given target to avoid problems processing data in tests.json.
* Disable too-many-ancestors pylint check.
This check was only enabled for ansible-test code.
In some cases this check counts 1 more ancestor in python 3.7+ than under python 3.6, making the check inconsistent and unsuitable for use in ansible-test.
* Disable additional pylint checks:
- import-error
- no-name-in-module
These checks were only enabled for ansible-test code.
These checks provide inconsistent results since they are dependent on available imports, which vary between environments.
It may be practical to enable these checks in the future if changes are made to ensure a consistent test environment for pylint.
* Test galaxy cli against pulp
* linting fix
* Renames and small fixes
* Better handling for resetting pulp
* Clean up some things, add a comment
* I can't spell
* Bump fallaxy, use alternate pulp image
* Only reset pulp when we're are executing against pulp
* Update for updated pulp container
* Update some comments with correct URLs and typos
* Linting fix
* Pin pulp-fedora31 to a digest
* Address review comments for documentation
When I try to do X with ansible-core from the devel branch on GitHub, Y
breaks in a way Z under the env E. Here are all the details I know
about this problem...
validations:
required:true
- type:dropdown
attributes:
label:Issue Type
# FIXME: Once GitHub allows defining the default choice, update this
options:
- Bug Report
validations:
required:true
- type:input
attributes:
label:Component Name
description:>-
Write the short name of the module, plugin, task or feature below,
*use your best guess if unsure*.
placeholder:dnf, apt, yum, pip, user etc.
validations:
required:true
- type:textarea
attributes:
label:Ansible Version
description:>-
Paste verbatim output from `ansible --version` between
tripple backticks.
value:|
```console (paste below)
$ ansible --version
```
validations:
required:true
- type:textarea
attributes:
label:Configuration
description:>-
Paste verbatim output from `ansible-config dump --only-changed` between quotes.
value:|
```console (paste below)
$ ansible-config dump --only-changed
```
validations:
required:true
- type:textarea
attributes:
label:OS / Environment
description:>-
Provide all relevant information below, e.g. target OS versions,
network device firmware, etc.
placeholder:RHEL 8, CentOS Stream etc.
validations:
required:true
- type:textarea
attributes:
label:Steps to Reproduce
description:|
Describe exactly how to reproduce the problem, using a minimal test-case. It would *really* help us understand your problem if you could also pased any playbooks, configs and commands you used.
**HINT:** You can paste https://gist.github.com links for larger files.
value:|
<!--- Paste example playbooks or commands between quotes below -->
```yaml (paste below)
```
validations:
required:true
- type:textarea
attributes:
label:Expected Results
description:>-
Describe what you expected to happen when running the steps above.
placeholder:>-
I expected X to happen because I assumed Y and was shocked
that it did not.
validations:
required:true
- type:textarea
attributes:
label:Actual Results
description:|
Describe what actually happened. If possible run with extra verbosity (`-vvvv`).
We document our Coding Guidelines in the `Developer Guide <https://docs.ansible.com/ansible/devel/dev_guide/>`_. We particularly suggest you review:
*`Contributing your module to Ansible <https://docs.ansible.com/ansible/devel/dev_guide/developing_modules_checklist.html>`_
*`Conventions, tips and pitfalls <https://docs.ansible.com/ansible/devel/dev_guide/developing_modules_best_practices.html>`_
*`Conventions, tips, and pitfalls <https://docs.ansible.com/ansible/devel/dev_guide/developing_modules_best_practices.html>`_
Branch Info
===========
@ -72,7 +71,7 @@ Branch Info
* The ``devel`` branch corresponds to the release actively under development.
* The ``stable-2.X`` branches correspond to stable releases.
* Create a branch based on ``devel`` and set up a `dev environment <https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_general.html#common-environment-setup>`_ if you want to open a PR.
* See the `Ansible release and maintenance <https://docs.ansible.com/ansible/latest/reference_appendices/release_and_maintenance.html>`_ page for information about active branches.
* See the `Ansible release and maintenance <https://docs.ansible.com/ansible/devel/reference_appendices/release_and_maintenance.html>`_ page for information about active branches.
Roadmap
=======
@ -84,7 +83,7 @@ Authors
=======
Ansible was created by `Michael DeHaan <https://github.com/mpdehaan>`_
and has contributions from over 4700 users (and growing). Thanks everyone!
and has contributions from over 5000 users (and growing). Thanks everyone!
`Ansible <https://www.ansible.com>`_ is sponsored by `Red Hat, Inc.
<https://www.redhat.com>`_
@ -96,12 +95,12 @@ GNU General Public License v3.0 or later
- slurp - improve the logic in the error handling and remove ``os.stat()`` call (https://github.com/ansible/ansible/pull/75038)
- ssh_connection - rename ``retries`` to ``reconnection_retries`` to avoid conflicts with task vars (https://github.com/ansible/ansible/issues/75142).
- ssh_connection - set the default for ``reconnection_retries`` back to ``0`` (https://github.com/ansible/ansible/issues/75142).
- task_executor/ssh_connection - use the ``retries`` value from ``ssh_connection`` settings, not the default from the ``Task`` field attributes (https://github.com/ansible/ansible/issues/75142).
- templating engine fix for not preserving usnafe status when trying to preserve newlines. CVE-2021-3583
Bugfixes
--------
- AnsibleModule.set_mode_if_different - don't check file existence when check_mode is activated (https://github.com/ansible/ansible/issues/61185).
- Apply ``display_failed_stderr`` callback option on loop item results. (https://github.com/ansible/ansible/issues/74864)
- Avoid task executor from ending early as vars can come from delegated to host.
- ansible-pull - update documentation for ``--directory`` option to clarify path must be absolute.
- config, ensure 'quoted' lists from ini or env do not take the quotes literally as part of the list item.
- gather_facts, package, service - fix using module_defaults for the modules in addition to the action plugins. (https://github.com/ansible/ansible/issues/72918)
- psrp - Always cleanup the last run pipeline if a second pipeline is invoked to avoid violating any resource limits.
- psrp - Fix error when resetting a connection that was initialised but not connected - (https://github.com/ansible/ansible/issues/74092).
- psrp - Try to clean up any server-side resources when resetting a connection.
- roles - make sure argspec validation task is tagged with ``always`` (https://github.com/ansible/ansible/pull/74994).
- slurp - Fix error messages for unreadable files and directories (https://github.com/ansible/ansible/issues/67340).
- slurp - handle error when ``path`` is a directory and not a file (https://github.com/ansible/ansible/pull/74930).
- ssh connection - fix interaction between transfer settings options.
- subversion - fix stack trace when getting information about the repository (https://github.com/ansible/ansible/issues/36498)
- version test - improve error message when an empty version is provided
- ansible adhoc, clarified the help to some options, also added some comments to code.
- command - update the user warning message to point out command name (https://github.com/ansible/ansible/pull/74475).
Bugfixes
--------
- Add RockyLinux to fact gathering (https://github.com/ansible/ansible/pull/74530).
- Improve resilience of ``ansible-galaxy collection`` by increasing the page size to make fewer requests overall and retrying queries with a jittered exponential backoff when rate limiting HTTP codes (520 and 429) occur. (https://github.com/ansible/ansible/issues/74191)
- Prevent ``ansible_failed_task`` from further templating (https://github.com/ansible/ansible/issues/74036)
- Remove 'default' from ssh plugin as we want to rely on default from ssh itself or ssh/config.
- The error message about the failure to import a ``gpg`` key by the ``apt_key`` module was incorrect (https://github.com/ansible/ansible/issues/74423).
- ansible-test - Avoid publishing the port used by the ``pypi-test-container`` since it is only accessed by other containers. This avoids issues when trying to run tests in parallel on a single host.
- ansible-test - Fix docker container IP address detection. The ``bridge`` network is no longer assumed to be the default.
- ansible-test - Use documented API to retrieve build information from Azure Pipelines.
- ansible.builtin.cron - Keep non-empty crontabs, when removing cron jobs (https://github.com/ansible/ansible/pull/74497).
- ansible.utils.encrypt now handles missing or unusable 'crypt' library.
- ansible_test - add constraint for ``MarkupSafe`` (https://github.com/ansible/ansible/pull/74666)
- apt_key - Binary GnuPG keys downloaded via URLs were corrupted so GnuPG could not import them (https://github.com/ansible/ansible/issues/74424).
- become - fix a regression on Solaris where chmod can return 5 which we interpret as auth failure and stop trying become tmpdir permission fallbacks
- become - work around setfacl not existing on modern Solaris (and possibly failing on some filesystems even when it does exist)
- callback default, now uses task delegate_to instead of delegate vars to display delegate to host
- callbacks, restores missing delegate_vars
- correct doc links for become on warnings over world readable settings.
- correctly use world readable setting since old constant is not 'settable' anymore.
- facts - detect homebrew installed at /opt/homebrew/bin/brew
- filter plugins - patch new versions of Jinja2 to prevent warnings/errors on renamed filter decorators (https://github.com/ansible/ansible/issues/74667)
- get_url - Fixed checksum validation for binary files (leading asterisk) in checksum files (https://github.com/ansible/ansible/pull/74502).
- hostname - Add Rocky Linux support
- ini lookup - handle errors for duplicate keys and missing sections (https://github.com/ansible/ansible/issues/74601)
- interpreter discovery - Debian 8 and lower will avoid unsupported Python3 version in interpreter discovery
- pause - ensure control characters are always set to an appropriate value (https://github.com/ansible/ansible/issues/73264)
- playbook loaded from collection subdir now does not ignore subdirs.
- plugin config now allows list type options to have multiple valid choices (#74225).
- replace - better handling of file operation exceptions (https://github.com/ansible/ansible/pull/74686).
- roles - allow for role arg specs in new meta file (https://github.com/ansible/ansible/issues/74525).
- service - compare version without LooseVersion API (https://github.com/ansible/ansible/issues/74488).
- A collection can be reinstalled with new version requirements without using the ``--force`` flag. The collection's dependencies will also be updated if necessary with the new requirements. Use ``--upgrade`` to force transitive dependency updates.
- AnsibleModule - use ``ArgumentSpecValidator`` class for validating argument spec and remove private methods related to argument spec validation. Any modules using private methods should now use the ``ArgumentSpecValidator`` class or the appropriate validation function.
- Declared ``resolvelib >= 0.5.3, < 0.6.0`` a direct dependency of
- It became possible to install Ansible Collections from local folders and namespaces folder similar to SCM structure with multiple collections.
- It became possible to upgrade Ansible collections from Galaxy servers using the ``--upgrade`` option with ``ansible-galaxy collection install``.
- Support for role argument specification validation at role execution time. When a role contains an argument spec, an implicit validation task is inserted at the start of role execution.
- add ``ArgumentSpecValidator`` class for validating parameters against an argument spec outside of ``AnsibleModule`` (https://github.com/ansible/ansible/pull/73335)
- ansible-test - Tests run with the ``centos6`` and ``default`` test containers now use a PyPI proxy container to access PyPI when Python 2.6 is used. This allows tests running under Python 2.6 to continue functioning even though PyPI is discontinuing support for non-SNI capable clients.
Minor Changes
-------------
- Add ``--format`` CLI option to ``ansible-galaxy collection list`` which allows for ``human`` (default), ``yaml``, or ``json``. (https://github.com/ansible/ansible/pull/73474)
- Add an example for using var in with_sequence (https://github.com/ansible/ansible/issues/68836).
- Add new rolespec_validate option to the import/include_role modules do allow disabling of the implicit role arg validation task on a per-role basis.
- Add option to pass extra vars to ansible-inventory
- Add path of collection location in Ansible CLI version info.
- Add standard Python 2/3 compatibility boilerplate to setup script, module_utils and docs_fragments which were missing them.
- Add support for `argument_specs` data in role metadata.
- Add support for datetime.date object type in module result (https://github.com/ansible/ansible/issues/70583).
- Add which conditional is being evaluated at each step when debugging.
- Add yum/dnf version comparison documentation for package install
- Added NO_COLOR environment var to ansible color configuration, allowing it to integrate with existing convention.
- Added name of aliases in user error (https://github.com/ansible/ansible/issues/58752).
- Added support for GSSAPI/Kerberos authentication with ``urls.py`` that is used by ``uri`` and ``get_url``.
- Added support for specify custom credentials for GSSAPI authentication.
- Allow an attribute to be passed to the min and max filters with Jinja 2.10+
- Allow for the skipped filter to be used on a registered looped task results. (https://github.com/ansible/ansible/issues/16949)
- Allow inventory plugins access to extra vars by default
- Allow unsafe_writes to be set on target via env var, for those targets that need a blanket setting.
- Also added extra vars cli option to console CLI.
- AnsiballZ - Improve performance of ``ModuleDepFinder`` by using faster lookups and reducing the object types that are walked while looking for ``import`` statements. (https://github.com/ansible/ansible/pull/70475)
- CLI - Specify jinja version in ``--version`` output
- CLI - Specify whether PyYAML includes libyaml support in version output
- CLI version displays clarified as core version
- Callbacks - Migrate more places in the ``TaskExecutor`` to sending callbacks directly over the queue, instead of sending them as ``TaskResult`` and short circuiting in the Strategy to send the callback. This enables closer to real time callbacks of retries and loop results (https://github.com/ansible/ansible/issues/73899)
- Collection routing: Cisco NSO content from community.network migrated to cisco.nso (https://github.com/ansible/ansible/pull/73046).
- Collection routing: DellEMC content from community.general migrated to dellemc.openmanage (https://github.com/ansible/ansible/pull/73046).
- Collection routing: FortiOS content from community.network migrated to community.fortios (https://github.com/ansible/ansible/pull/73046).
- Collection routing: Google content from community.general migrated to community.google (https://github.com/ansible/ansible/pull/73046).
- Collection routing: Hashi Vault content from community.general migrated to community.hashi_vault (https://github.com/ansible/ansible/pull/73046).
- Collection routing: Hetzner Robot content from community.general migrated to community.hrobot (https://github.com/ansible/ansible/pull/73046).
- Collection routing: KubeVirt content from community.general migrated to community.kubevirt (https://github.com/ansible/ansible/pull/73046).
- Collection routing: OC content from community.general migrated to community.okd (https://github.com/ansible/ansible/pull/73046).
- Collection routing: PostgreSQL content from community.general migrated to community.postgresql (https://github.com/ansible/ansible/pull/73046).
- Collection routing: RouterOS content from community.network migrated to community.routeros (https://github.com/ansible/ansible/pull/73046).
- Collection routing: docker content from community.general migrated to community.docker (https://github.com/ansible/ansible/pull/73046).
- Controller - Add warning for Ansible 2.11 when running a Python version older than Python 3.8 to inform users that 2.12 will only support Python 3.8 and newer on the controller. Starting with Ansible 2.11, the project will only be packaged for Python 3.8 and newer.
- Discourage the use of 'hexdigits' in password lookup, as it distorts expected entropy.
- Enable extra vars for inventory plugin options
- Errors - Ensure that errors passed with ``orig_exc`` include the context of that exception (https://github.com/ansible/ansible/issues/68605)
- Filters - Add new ``split`` filter for splitting strings
- Fixed ansible-doc to not substitute for words followed by parenthesis. For instance, ``IBM(International Business Machines)`` will no longer be substituted with a link to a non-existent module. https://github.com/ansible/ansible/pull/71070
- Force the template module to use non-native Jinja2 (https://github.com/ansible/ansible/issues/46169)
- Internal config entries will not be documented, to mark an entry as internal it must start with `_`.
- Interpreter Discovery - Add Python 3.8 and Python 3.9 to the fallback list
- Minor code cleanup in plugin loader.
- Module API - libselinux-python is no longer required for basic module API selinux operations (affects core modules assemble, blockinfile, copy, cron, file, get_url, lineinfile, setup, replace, unarchive, uri, user, yum_repository)
- Module API - new module_respawn API allows modules that need to run under a specific Python interpreter to respawn in place under that interpreter
- Module iptables multiport destination support added (https://github.com/ansible/ansible/pull/72928)
- Module iptables set/ipset support added (https://github.com/ansible/ansible/pull/72984)
- New 'timeout' feature added to adhoc and console CLIs, corresponding to the recent 'timeout' task keyword.
- New virtualization facts, ``virtualization_tech_guest`` and ``virtualization_tech_host`` now allow for conveying when a system is a host or guest of multiple virtualization technologies.
- Now 'choices' keyword in config definitions also restricts valid values for the entry.
- Refactored ``ansible-galaxy collection [download|install|list|verify]`` CLI subcommands with the public interface kept intact.
- Restructured _fixup_perms2() in ansible.plugins.action to make it more linear
- Shadow prompt input to ansible-vault encrypt-string unless the ``--show-input`` flag is set
- Switch to hashlib.sha256() for ansible-test to allow for FIPs mode.
- TOML inventory plugin is no longer in preview status
- Templar - reduce the complexity of ``Templar._lookup`` (https://github.com/ansible/ansible/pull/73277)
- The ``csvfile`` lookup plugin now uses ``parse_kv()`` internally. As a result, multi-word search keys can now be passed.
- The ``csvfile`` lookup plugin's documentation has been fixed; it erroneously said that the delimiter could be ``t`` which was never true. We now accept ``\t``, however, and the error in the documentation has been fixed to note that.
- The constructed inventory plugin has new option to force using vars plugins on previouslly processed inventory sources.
- The find module is now more specific about the reasons it skips candidate files.
- The logging functionality in module_utils.basic now returns a nicer error when it falls back to syslog but ends up getting a TypeError thrown back.
- The new dependency resolver prefers ``MANIFEST.json`` over ``galaxy.yml`` if it exists in the target directory.
- The plugin loader now keeps track of the collection where a plugin was resolved to, in particular whether the plugin was loaded from ansible-core's internal paths (``ansible.builtin``) or from user-supplied paths (no collection name).
- Toggle allowing usage of extra_vars in compose
- When connecting as an unprivileged user, and becoming an unprivileged user, we now fall back to also trying ``chmod +a`` which works on macOS and makes use of ACLs.
- allow tree callback plugin to be configurable, for use with playbooks.
- ansible-doc - In Windows setup steps, ``ExecutionPolicy`` should be restored to default value ``RemoteSigned`` (https://github.com/ansible/ansible/pull/72993).
- ansible-doc - provide ``has_action`` field in JSON output for modules. That information is currently only available in the text view (https://github.com/ansible/ansible/pull/72359).
- ansible-doc has new option to show keyword documentation.
- ansible-doc will now format, ``L()``, ``R()``, and ``HORIZONTALLINE`` in plugin docs just as the website docs do. https://github.com/ansible/ansible/pull/71070
- ansible-galaxy - Added caching mechanisms when retrieving collection info to speed up installs and downloads
- ansible-galaxy - Change the output verbosity level of the download message from 3 to 0 (https://github.com/ansible/ansible/issues/70010)
- ansible-galaxy - Ensure ``get_collection_versions`` returns an empty list when a collection does not exist for consistency across API versions.
- ansible-galaxy - find any collection dependencies in the globally configured Galaxy servers and not just the server the parent collection is from.
- ansible-galaxy CLI - ``collection verify`` command now exits with a non-zero exit code on verification failure
- ansible-galaxy CLI - ``collection verify`` command now supports a ``--offline`` option for local-only verification
- ansible-test - A warning is no longer emitted when a ``pip*`` or ``python*`` binary is found without a matching couterpart.
- ansible-test - Add ``macos/10.15`` as a supported value for the ``--remote`` option.
- ansible-test - Add a ``--docker-network`` option to choose the network for running containers when using the ``--docker`` option.
- ansible-test - Add constraint for ``decorator`` for Python versions prior to 3.5.
- ansible-test - Add support for running tests on Fedora 33 (https://github.com/ansible/ansible/pull/72861).
- ansible-test - Added Ubuntu 20.04 LTS image to the default completion list
- ansible-test - Added a ``--export`` option to the ``ansible-test coverage combine`` command to facilitate multi-stage aggregation of coverage in CI pipelines.
- ansible-test - Added the ``-remote rhel/7.9`` option to run tests on RHEL 7.9
- ansible-test - Allow custom ``--remote-stage`` options for development and testing.
- ansible-test - CentOS 8 container is now 8.2.2004 (https://github.com/ansible/distro-test-containers/pull/45).
- ansible-test - Changed the internal name of the custom plugin used to identify use of unwanted imports and functions.
- ansible-test - Cleaned up code to resolve warnings and errors reported by PyCharm.
- ansible-test - Code cleanup in the ``import`` sanity test.
- ansible-test - Code cleanup in the internal logic for code coverage collection of PowerShell modules.
- ansible-test - Collections can now specify pip constraints for unit and integration test requirements using ``tests/unit/constraints.txt`` and ``tests/integration/constraints.txt`` respectively.
- ansible-test - Containers used with the ``--remote`` option have been updated to version 1.29.0 to include the latest Ansible requirements.
- ansible-test - Files used to track remote instances no longer have a region suffix.
- ansible-test - Fix container hostname/IP discovery for the ``acme`` test plugin.
- ansible-test - FreeBSD 11.4 and 12.2 provisioning can now be used with the ``--python 3.8`` option.
- ansible-test - FreeBSD instances provisioned with ``--remote`` now make ``libyaml`` available for use with PyYAML installation.
- ansible-test - Generation of an ``egg-info`` directory, if needed, is now done after installing test dependencies and before running tests. When running from an installed version of ``ansible-test`` a temporary directory is used to avoid permissions issues. Previously it was done before installing test dependencies and adjacent to the installed directory.
- ansible-test - Implemented CloudStack test container selection by ENV variable `ANSIBLE_CLOUDSTACK_CONTAINER` with a default to `quay.io/ansible/cloudstack-test-container:1.4.0`.
- ansible-test - Improved handling of minimum Python version requirements for sanity tests. Supported versions are now included in warning messages displayed when tests are skipped.
- ansible-test - More sanity test requirements have been pinned to specific versions to provide consistent test results.
- ansible-test - Most sanity test specific ``pip`` constraints are now used only when running sanity tests. This should reduce conflicts with ``pip`` requirements and constraints when testing collections.
- ansible-test - Most sanity tests are now skipped on Python 3.5 and earlier with a warning. Previously this was done for Python 2.7 and earlier.
- ansible-test - Now supports freebsd/11.4 remote (https://github.com/ansible/ansible/issues/48782).
- ansible-test - Now supports freebsd/12.2 remote (https://github.com/ansible/ansible/issues/72366).
- ansible-test - OpenSuse container now uses Leap 15.2 (https://github.com/ansible/distro-test-containers/pull/48).
- ansible-test - Pin the ``virtualenv`` version used for ``--remote`` pip installs to the latest version supported by Python 2.x, which is version 16.7.10.
- ansible-test - Provisioning of RHEL instances now includes installation of pinned versions of ``packaging`` and ``pyparsing`` to match the downstream vendored versions.
- ansible-test - RHEL 8.2+ provisioning can now be used with the ``--python 3.8`` option, taking advantage of the Python 3.8 AppStream.
- ansible-test - Raise the number of bytes scanned by ansible-test to determine if a file is binary to 4096.
- ansible-test - Refactor code for installing ``cryptography`` to allow re-use in the future.
- ansible-test - Refactor code to remove unused logic for obsolete support of multiple provisioning endpoints.
- ansible-test - Remove ``pytest < 6.0.0`` constraint for managed installations on Python 3.x now that pytest 6 is supported.
- ansible-test - Remove em dash from the Pytest configuration file in order to be readable on systems where preferred encoding is ASCII. (https://github.com/ansible/ansible/issues/71739)
- ansible-test - Removed ``pip`` constraints related to integration tests that have been moved to collections. This should reduce conflicts with ``pip`` requirements and constraints when testing collections.
- ansible-test - Removed the obsolete ``--remote-aws-region`` provisioning option.
- ansible-test - Removed the obsolete ``tower`` test plugin for testing Tower modules.
- ansible-test - Removed unused provisioning code and cleaned up remote provider management logic.
- ansible-test - Rename internal functions to match associated constant names that were previously updated.
- ansible-test - Reorganize internal ``pylint`` configuration files for easier comparison and maintenance.
- ansible-test - Report the correct line number in the ``yamllint`` sanity test when reporting ``libyaml`` parse errors in module documentation.
- ansible-test - Request remote resources by provider name for all provider types.
- ansible-test - Show a warning when the obsolete ``--remote-aws-region`` option is used.
- ansible-test - Silence ``pip`` warnings about Python 3.5 being EOL when installing requirements.
- ansible-test - Support custom remote endpoints with the ``--remote-endpoint`` option.
- ansible-test - The ``--remote`` option no longer pre-installs the ``virtualenv`` module on Python 3.x instances. The Python built-in ``venv`` module should be used instead.
- ansible-test - The ``default`` container for both collections and core have been updated to versions 2.11.0 and 1.9.0 respectively.
- ansible-test - The ``pylint`` sanity test is now skipped with a warning on Python 3.9 due to unresolved upstream regressions.
- ansible-test - The ``pylint`` sanity test is now supported on Python 3.8.
- ansible-test - The ``rstcheck`` sanity test is no longer used for collections, but continues to be used for ansible-core.
- ansible-test - The generated ``resource_prefix`` variable now meets the host name syntax requirements specified in RFC 1123 and RFC 952. The value used for local tests now places the random number before the hostname component, rather than after. If the resulting value is too long, it will be truncated.
- ansible-test - Ubuntu containers as well as ``default-test-container`` and ``ansible-base-test-container`` are now slightly smaller due to apt cleanup (https://github.com/ansible/distro-test-containers/pull/46).
- ansible-test - Update ``pylint`` and its dependencies to the latest available versions to support Python 3.9.
- ansible-test - Update built-in service endpoints for the ``--remote`` option.
- ansible-test - Update distribution test containers from version 2.0.1 to 2.0.2.
- ansible-test - Update the Ansible Core and Ansible Collection default test containers to 3.2.0 and 3.2.2 respectively.
- ansible-test - Updated the default test containers to version 3.1.0.
- ansible-test - Upgrade ansible-runner version used in compatibility tests, remove some tasks that were only needed with older versions, and skip in python2 because ansible-runner is soon dropping it.
- ansible-test - Use new endpoint for Parallels based instances with the ``--remote`` option.
- ansible-test - ``default-test-container`` and ``ansible-base-test-container`` now use Python 3.9.0 instead of 3.9.0rc1.
- ansible-test - add https endpoint for ansible-test
- ansible-test - add macOS 11.1 as a remote target (https://github.com/ansible/ansible/pull/72622)
- ansible-test - add the collection plugin directories ``plugin_utils`` and ``sub_plugins`` to list of plugin types. This ensures such plugins are tested for the ``import`` sanity test (https://github.com/ansible/ansible/pull/73599).
- ansible-test - centos6 end of life - container image updated to point to vault base repository (https://github.com/ansible/distro-test-containers/pull/54)
- ansible-test - centos6 image now has multiple fallback yum repositories for CentOS Vault.
- ansible-test - default container now uses default-test-container 2.7.0 and ansible-base-test-container 1.6.0. This brings in Python 3.9.0rc1 for testing.
- ansible-test - now makes a better attempt to support podman when calling ``docker images`` and asking for JSON format.
- ansible-test - python-cryptography is now bounded at <3.2, as 3.2 drops support for OpenSSL 1.0.2 upon which some of our CI infrastructure still depends.
- ansible-test - remote macOS instances no longer install ``virtualenv`` during provisioning
- ansible-test - the ACME test container was updated, it now supports external account creation and has a basic OCSP responder (https://github.com/ansible/ansible/pull/71097, https://github.com/ansible/acme-test-container/releases/tag/2.0.0).
- ansible-test - the ``import`` sanity test now also tries to import all non-module and non-module_utils Python files in ``lib/ansible/`` resp. ``plugins/`` (https://github.com/ansible/ansible/pull/72497).
- ansible-test - virtualenv helper scripts now prefer ``venv`` on Python 3 over ``virtualenv``
- ansible-test Now supports RHEL 8.3
- ansible-test pylint - ensure that removal collection version numbers conform to the semantic versioning specification at https://semver.org/ (https://github.com/ansible/ansible/pull/71679).
- ansible-test runtime-metadata - compare deprecation and tombstone versions to the current version to ensure that they are correct (https://github.com/ansible/ansible/pull/72625).
- ansible-test runtime-metadata - ensure that removal collection version numbers conform to the semantic versioning specification at https://semver.org/ (https://github.com/ansible/ansible/pull/71679).
- ansible-test runtime-metadata - ensure that the tombstone removal date is not in the future (https://github.com/ansible/ansible/pull/72625).
- ansible-test runtime-metadata - validate removal version numbers, and check removal dates more strictly (https://github.com/ansible/ansible/pull/71679).
- ansible-test validate-modules - ensure that removal collection version numbers and version_added collection version numbers conform to the semantic versioning specification at https://semver.org/ (https://github.com/ansible/ansible/pull/71679).
- ansible-test validate-modules - no longer assume that ``default`` for ``type=bool`` options is ``false``, as the default is ``none`` and for some modules, ``none`` and ``false`` mean different things (https://github.com/ansible/ansible/issues/69561).
- ansible-test validate-modules - option names that seem to indicate they contain secret information that should be marked ``no_log=True`` are now flagged in the validate-modules sanity test. False positives can be marked by explicitly setting ``no_log=False`` for these options in the argument spec. Please note that many false positives are expected; the assumption is that it is by far better to have false positives than false negatives (https://github.com/ansible/ansible/pull/73508).
- ansible-test validate-modules - validate removal version numbers (https://github.com/ansible/ansible/pull/71679).
- ansible.utils.encrypt now returns `AnsibleError` instead of crypt.crypt's `OSError` on Python 3.9
- apt - module now works under any supported Python interpreter
- apt_repository - module now works under any supported Python interpreter
- callback plugins - ``meta`` tasks now get sent to ``v2_playbook_on_task_start``. Explicit tasks are always sent. Plugins can opt in to receiving implicit ones.
- conditionals - change the default of CONDITIONAL_BARE_VARS to False (https://github.com/ansible/ansible/issues/70682).
- config - more types are now automatically coerced to string when ``type: str`` is used and the value is parsed as a different type
- constructed - Add a toggle to allow the separator to be omitted if no prefix has been provided.
- constructed inventory plugin - Sanitize group names created from the ``groups`` option silently.
- create ``get_type_validator`` standalone function and move that functionality out of ``AnsibleModule`` (https://github.com/ansible/ansible/pull/72667)
- create ``get_unsupported_parameters`` validation function (https://github.com/ansible/ansible/pull/72447/files)
- debconf - add a note about no_log=True since module might expose sensitive information to logs (https://github.com/ansible/ansible/issues/32386).
- default callback - add ``show_task_path_on_failure`` option to display file and line number of tasks only on failed tasks when running at normal verbosity level (https://github.com/ansible/ansible/issues/64625)
- default callback - task name is now shown for ``include_tasks`` when using the ``free`` strategy (https://github.com/ansible/ansible/issues/71277).
- default callback - task name is now shown for ``include_tasks`` when using the ``linear`` strategy with ``ANSIBLE_DISPLAY_SKIPPED_HOSTS=0``.
- default_callback - moving 'check_mode_markers' documentation in default_callback doc_fragment (https://github.com/ansible-collections/community.general/issues/565).
- distribution - add facts about Amazon Linux Distribution facts (https://github.com/ansible/ansible/issues/73742).
- distribution - add support for DragonFly distribution (https://github.com/ansible/ansible/issues/43739).
- distribution - added distribution fact and hostname support for Parrot OS (https://github.com/ansible/ansible/pull/69158).
- distribution - handle NetBSD OS Family (https://github.com/ansible/ansible/issues/43739).
- distribution facts - ``distribution_release`` is now ``"Stream"`` on CentOS Stream (https://github.com/ansible/ansible/issues/73027).
- dnf - When ``state: absent``, package names are now matched similarly to how the ``dnf`` CLI matches them (https://github.com/ansible/ansible/issues/72809).
- dnf - module now works under any supported Python interpreter
- dnf - now shows specific package changes (installations/removals) under ``results`` in check_mode. (https://github.com/ansible/ansible/issues/66132)
- facts - ``/dev/kvm`` is now consulted in Linux virtualization facts, and the host is considered a KVM host if this file exists and none of the pre-existing checks matched.
- facts - add new fact ``date_time['tz_dst']``, which returns the daylight saving timezone (https://github.com/ansible/ansible/issues/69004).
- facts - add uptime to openbsd
- find module - Now has a ``read_whole_file`` boolean parameter which allows for reading the whole file and doing an ``re.search()`` regex evaluation on it when searching using the ``contains`` option. This allows (for example) for ensuring the very end of the file matches a pattern.
- galaxy - add documentation about galaxy parameters in examples/ansible.cfg (https://github.com/ansible/ansible/issues/68402).
- galaxy - handle token as dict while loading from yaml file (https://github.com/ansible/ansible/issues/70887).
- get_url - allow checksum urls to point to file:// resources, moving scheme test to function
- get_url - handle same SHA sum for checksum file (https://github.com/ansible/ansible/issues/71420).
- hash filter - fail when unsupported hash type is passed as an argument (https://github.com/ansible/ansible/issues/70258)
- inventory cache - do not show a warning when the cache file does not (yet) exist.
- iptables - add a note about ipv6-icmp in protocol parameter (https://github.com/ansible/ansible/issues/70905).
- iptables - fixed get_chain_policy API (https://github.com/ansible/ansible/issues/68612).
- iptables - reorder comment postition to be at the end (https://github.com/ansible/ansible/issues/71444).
- lineinfile - add search_string parameter for non-regexp searching (https://github.com/ansible/ansible/issues/70470)
- linux facts - Add additional check to ensure 'container' virtual fact gets added to guest_tech when appropriate (https://github.com/ansible/ansible/pull/71385)
- meta - now include a ``skip_reason`` when skipped (https://github.com/ansible/ansible/pull/71355).
- module payload builder - module_utils imports in any nested block (eg, ``try``, ``if``) are treated as optional during module payload builds; this allows modules to implement runtime fallback behavior for module_utils that do not exist in older versions of Ansible.
- module_utils - ``get_file_attributes()`` now takes an optional ``include_version`` boolean parameter. When ``True`` (default), the file's version/generation number is included in the result (but requires ``lsattr -v`` to work on the target platform).
- now !unsafe works on all types of data, not just strings, even recursively for mappings and sequences.
- package_facts - module support for apt and rpm now works under any supported Python interpreter
- pipe lookup - update docs for Popen with shell=True usages (https://github.com/ansible/ansible/issues/70159).
- plugin examples - Allow non-YAML examples, so that examples for plugins like the INI and TOML inventory plugins can be directly represented (https://github.com/ansible/ansible/pull/71184)
- plugin option validation - now the option type ``dict``/``dictionary`` is also validated by the config manager (https://github.com/ansible/ansible/pull/71928).
- reboot - add ``reboot_command`` parameter to allow specifying the command used to reboot the system (https://github.com/ansible/ansible/issues/51359)
- remove ``excommunicate`` debug command from AnsiballZ
- unarchive - add ``include`` parameter to allow extracting specific files from an archive (https://github.com/ansible/ansible/pull/40522)
- update sphinx to 2.1.2 and rstcheck to 3.3.1 for building documentation.
- uri - add ``ca_path`` argument to allow specification of a CA certificate (https://github.com/ansible/ansible/pull/71979).
- user - add new parameters ``password_expire_max`` and ``password_expire_min`` for controlling password expiration (https://github.com/ansible/ansible/issues/68775)
- varnames lookup plugin - Fixed grammar error in exception message when the plugin is given a non-string term.
- vault - Provide better error for single value encrypted values to indicate the file, line, and column of the errant vault (https://github.com/ansible/ansible/issues/72276)
- version test - Add semantic version functionality
- virtual facts - containerd cgroup is now recognized as container tech (https://github.com/ansible/ansible/issues/66304).
- virtualization facts - Detect ``vdsmd`` in addition to ``vdsm`` when trying to detect RHEV hosts. (https://github.com/ansible/ansible/issues/66147)
- winrm - Added ``ansible_winrm_kinit_args`` that can be used to control the args that are sent to the ``kinit`` call for Kerberos authentication.
- yum - module now works under any supported Python interpreter
- yum_repository - added boolean option module_hotfixes which allows to enable functionality for dnf.
Breaking Changes / Porting Guide
--------------------------------
- Made SCM collections be reinstalled regardless of ``--force`` being present.
- NetBSD virtualization facts (specifically ``ansible_virtualization_type``) now returns a more accurate value by checking the value of the ``machdep.hypervisor````sysctl`` key. This change is breaking because in some cases previously, we would erroneously report ``xen`` even when the target is not running on Xen. This prevents that behavior in most cases. (https://github.com/ansible/ansible/issues/69352)
- Replaced the in-tree dependency resolver with an external implementation that pip >= 20.3 uses now by default — ``resolvelib``. (https://github.com/ansible/ansible/issues/71784)
- The ``meta`` module now supports tags for user-defined tasks. Internal ``meta`` tasks continue to always run. (https://github.com/ansible/ansible/issues/64558)
- ansible-galaxy login command has been removed (see https://github.com/ansible/ansible/issues/71560)
Deprecated Features
-------------------
- Starting in 2.14, shell and command modules will no longer have the option to warn and suggest modules in lieu of commands. The ``warn`` parameter to these modules is now deprecated and defaults to ``False``. Similarly, the ``COMMAND_WARNINGS`` configuration option is also deprecated and defaults to ``False``. These will be removed and their presence will become an error in 2.14.
- apt_key - the paramater ``key`` does not have any effect, has been deprecated and will be removed in ansible-core version 2.14 (https://github.com/ansible/ansible/pull/70319).
- psrp - Set the minimum version of ``pypsrp`` to ``0.4.0``.
Removed Features (previously deprecated)
----------------------------------------
- Removed `SharedPluginLoaderObj` class from ansible.plugins.strategy. It was deprecated in favor of using the standard plugin loader.
- Removed `_get_item()` alias from callback plugin base class which had been deprecated in favor of `_get_item_label()`.
- The "user" parameter was previously deprecated and is now removed in favor of "scope"
- The deprecated ``ansible.constants.BECOME_METHODS`` has been removed.
- The deprecated ``ansible.constants.get_config()`` has been removed.
- The deprecated ``ansible.constants.mk_boolean()`` has been removed.
-`with_*` loops are no longer optimized for modules whose `name` parameters can take lists (mostly package managers). Use `name` instead of looping over individual names with `with_items` and friends.
Security Fixes
--------------
-**security issue** - Mask default and fallback values for ``no_log`` module options (CVE-2021-20228)
-**security issue** - copy - Redact the value of the no_log 'content' parameter in the result's invocation.module_args in check mode. Previously when used with check mode and with '-vvv', the module would not censor the content if a change would be made to the destination path. (CVE-2020-14332)
- Sanitize no_log values from any response keys that might be returned from the uri module (CVE-2020-14330).
- dnf - Previously, regardless of the ``disable_gpg_check`` option, packages were not GPG validated. They are now. (CVE-2020-14365)
Bugfixes
--------
- A handler defined within a role will now search handlers subdir for included tasks (issue https://github.com/ansible/ansible/issues/71222).
- ALLOW_WORLD_READABLE_TMP, switched to 'moved' message as 'deprecation' is misleading since config settings still work w/o needing change.
- ANSIBLE_COLLECTIONS_PATHS - remove deprecation so that users of Ansible 2.9 and 2.10+ can use the same var when specifying a collection path without a warning.
- Added unsafe_writes test.
- Address compat with rpmfluff-0.6 for integration tests
- Address the deprecation of the use of stdlib distutils in packaging. It's a short-term hotfix for the problem (https://github.com/ansible/ansible/issues/70456, https://github.com/pypa/setuptools/issues/2230, https://github.com/pypa/setuptools/commit/bd110264)
- Adjust various hard-coded action names to also include their ``ansible.builtin.`` and ``ansible.legacy.`` prefixed version (https://github.com/ansible/ansible/issues/71817, https://github.com/ansible/ansible/issues/71818, https://github.com/ansible/ansible/pull/71824).
- Allow TypeErrors on Undefined variables in filters to be handled or deferred when processing for loops.
- Allow `~` to be present in file names in galaxy roles (https://github.com/ansible/ansible/issues/72966)
- Always mention the name of the deprecated or tombstoned plugin in routing deprecation/tombstone messages (https://github.com/ansible/ansible/pull/73059).
- Ansible output now uses stdout to determine column width instead of stdin
- AnsibleModule - added arg ``ignore_invalid_cwd`` to ``AnsibleModule.run_command()``, to control its behaviour when ``cwd`` is invalid. (https://github.com/ansible/ansible/pull/72390)
- Apply ``_wrap_native_text`` only for builtin filters specified in STRING_TYPE_FILTERS.
- Automatically remove async cache files for polled async tasks that have completed (issue https://github.com/ansible/ansible/issues/73206).
- Be smarter about collection paths ending with ansible_collections, emulating a-galaxy behaviour. Issue 72628
- CLI - Restore git information in version output when running from source
- Collection callbacks were ignoring options and rules for stdout and adhoc cases.
- Collections - Ensure ``action_loader.get`` is called with ``collection_list`` to properly find collections when ``collections:`` search is specified (https://github.com/ansible/ansible/issues/72170)
- Command module now returns stdout & stderr if executable is missing or an unknown error occurs
- ConfigManager - Normalize ConfigParser between Python2 and Python3 to for handling comments (https://github.com/ansible/ansible/issues/73709)
- Continue execution when 'flatten' filter when it hits a None/null value as part of the list.
- Correct the inventory source error parse handling, specifically make the config INVENTORY_ANY_UNPARSED_IS_FAILED work as expected.
- Correctly set template_path and template_fullpath for usage in template lookup and action plugins.
- Deal with failures when sorting JSON and you have incompatible key types.
- Display - Use wcswidth to calculate printable width of a text string (https://github.com/ansible/ansible/issues/63105)
- Enabled unsafe_writes for get_url which was ignoring the paramter.
- Ensure Ansible's unique filter preserves order (https://github.com/ansible/ansible/issues/63417)
- Ensure if a traceback halts ``strategy.run`` that we still attempt to clean up (https://github.com/ansible/ansible/issues/23958)
- Ensure password passed in by -k is used on delegated hosts that do not have ansible_password set
- Ensure the correct options are used when ssh executables are used that don't match ssh executable names.
- Facts collection - get serial number of NVMe device without sg_inq (https://github.com/ansible/ansible/issues/66663).
- Fix --list-tasks format `role_name : task_name` when task name contains the role name. (https://github.com/ansible/ansible/issues/72505)
- Fix ``RecursionError`` when templating large vars structures (https://github.com/ansible/ansible/issues/71920)
- Fix ``delegate_facts: true`` when ``ansible_python_interpreter`` is not set. (https://github.com/ansible/ansible/issues/70168)
- Fix adding unrelated candidate names to the plugin loader redirect list.
- Fix an exit code for a non-failing playbook (https://github.com/ansible/ansible/issues/71306)
- Fix ansible-galaxy collection list to show collections in site-packages (https://github.com/ansible/ansible/issues/70147)
- Fix bytestring vs string comparison in module_utils.basic.is_special_selinux_path() so that special-cased filesystems which don't support SELinux context attributes still allow files to be manipulated on them. (https://github.com/ansible/ansible/issues/70244)
- Fix execution of the meta tasks 'clear_facts', 'clear_host_errors', 'end_play', 'end_host', and 'reset_connection' when the CLI flag '--flush-cache' is provided.
- Fix fileglob bug where it could return different results for different order of parameters (https://github.com/ansible/ansible/issues/72873).
- Fix incorrect msg in the results dict in loops
- Fix incorrect re-run of roles with tags (https://github.com/ansible/ansible/issues/69848)
- Fix incorrect variable scoping when using ``import with context`` in Jinja2 templates. (https://github.com/ansible/ansible/issues/72615)
- Fix jsonfile cache plugin option '_uri' to be a type path instead of a string. (https://github.com/ansible/ansible/issues/38002)
- Fix notifying handlers via `role_name : handler_name` when handler name contains the role name. (https://github.com/ansible/ansible/issues/70582)
- Fix parsing of values when using empty string as a key (https://github.com/ansible/ansible/issues/57132)
- Fix statistics reporting when rescue block contains another block (issue https://github.com/ansible/ansible/issues/61253).
- Fix to previous deprecation change (#70504) which caused command warning deprecation to show in all cases, even when not specified by the user.
- Fixed TypeError instancemethod expecting at least 2 arguments for apt_repository(issue https://github.com/ansible/ansible/issues/69308, PR https://github.com/ansible/ansible/pull/69463)
- Fixed issue when `netstat` is either missing or doesn't have execution permissions leading to incorrect command being executed.
- Fixes ``ansible-galaxy role info`` to support multiple roles on the command line (https://github.com/ansible/ansible/pull/70148)
- Fixes ansible-test traceback when plugin author is not a string or a list of strings (https://github.com/ansible/ansible/pull/70507)
- Handle more varnames that can create conflicts, expand a function in general, handle jinja2 globals in particular (https://github.com/ansible/ansible/issues/41955).
- INTERPRETER_PYTHON_DISTRO_MAP - prefer ``/usr/libexec/platform-python`` on ``oraclelinux 8`` when other pythons are present.
- Improve Ansible config deprecations to show the source of the deprecation (ansible-core). Also remove space before a comma in config deprecations (https://github.com/ansible/ansible/pull/72697).
- Improved/fixed regular expressions in ``validate-modules/validate_modules/schema.py`` and ``utils/collection_loader/_collection_finder.py`` (https://github.com/ansible/ansible/pull/73577).
- Includes - Explicitly get the include task, and not assume it is the parent (https://github.com/ansible/ansible/issues/65710)
- InventoryManager - Fix unhandled exception when given limit file was actually a directory.
- InventoryManager - Fix unhandled exception when inventory directory was empty or contained empty subdirectories (https://github.com/ansible/ansible/issues/73658).
- JSON Encoder - Ensure we treat single vault encrypted values as strings (https://github.com/ansible/ansible/issues/70784)
- Lookup user by UID in password database if login name is not found (https://github.com/ansible/ansible/issues/17029)
- OpenBSD module_utils - update sysctl variable name
- Pass expression in angle-bracket notation as filename argument to a ``compile()`` built-in function, so that Python debuggers do not try to parse it as filename.
- Pass the connection's timeout to connection plugins instead of the task's timeout.
- Provide more information in AnsibleUndefinedVariable (https://github.com/ansible/ansible/issues/55152)
- Python module_utils finder - refactor logic to eliminate many corner cases, remove recursion, fix base module_utils redirections
- Remove an embedded function from RoleMixin and add tests for it (https://github.com/ansible/ansible/pull/72754).
- Remove the warning displayed when validating the arg spec of a role with dependencies and add it to the documentation.
- Restore the ability for changed_when/failed_when to function with group_by (#70844).
- Restored unsafe_writes functionality which was being skipped.
- Restructured pipelining settings to be at the connection plugins leaving base config as global and for backwards compatiblity.
- SSH plugin - Improve error message when ssh client is not found on the host
- Setup virtualization_facts - add RHV and oVirt type. This change will fully work for VMs in clusters at cluster level 4.4 or newer (https://github.com/ansible/ansible/pull/72876).
- Skip invalid collection names when listing in ansible-doc instead of throwing exception. Issue#72257
- Skip literal_eval for string filters results in native jinja. (https://github.com/ansible/ansible/issues/70831)
- Stop adding the connection variables to the output results
- Strategy - When building the task in the Strategy from the Worker, ensure it is properly marked as finalized and squashed. Addresses an issue with ``ansible_failed_task``. (https://github.com/ansible/ansible/issues/57399)
- Suppress warning when user directory used in --playbook-dir option with ansible-inventory command (https://github.com/ansible/ansible/issues/65262).
- TOML inventory - Ensure we register dump functions for ``AnsibleUnsafe`` to support dumping unsafe values. Note that the TOML format has no functionality to mark that the data is unsafe for re-consumption. (https://github.com/ansible/ansible/issues/71307)
- Terminal plugins - add "\e[m" to the list of ANSI sequences stripped from device output
- The ``docker`` and ``k8s`` action groups / module default groups now also support the moved modules in `community.docker <https://galaxy.ansible.com/community/docker>`_, `community.kubevirt <https://github.com/ansible-collections/community.kubevirt>`_, `community.okd <https://galaxy.ansible.com/community/okd>`_, and `kubernetes.core <https://galaxy.ansible.com/kubernetes/core>`_ (https://github.com/ansible/ansible/pull/72428).
- The ``flush()`` method of ``CachePluginAdjudicator`` now calls the plugin's ``flush()`` method instead of iterating over the keys that the adjudicator knows about and deleting those from the cache. (https://github.com/ansible/ansible/issues/68770)
- The `ansible_become` value was not being treated as a boolean value when set in an INI format inventory file (fixes bug https://github.com/ansible/ansible/issues/70476).
- The machine-readable changelog ``changelogs/changelog.yaml`` is now contained in the release.
- Try to avoid kernel 'blocking' state on reading files while fact gathering.
- Updated docs and added warning on max_fail_percentage and free strategy usage. fixes issue 16666.
- VariableManager - Add the 'vars' key before getting delegated variables (https://github.com/ansible/ansible/issues/71092).
- Vault - Allow single vault encrypted values to be used directly as module parameters. (https://github.com/ansible/ansible/issues/68275)
- WorkerProcess - Implement workaround for stdout deadlock in multiprocessing shutdown to avoid process hangs.
- WorkerProcess - Python 3.5 fix for workaround for stdout deadlock in multiprocessing shutdown to avoid process hangs. (https://github.com/ansible/ansible/issues/74149)
- [set_fact] Corrected and expanded documentation as well as now raise errors that were previously ignored.
- account for bug in Python 2.6 that occurs during interpreter shutdown to avoid stack trace
- action plugins - change all action/module delegations to use FQ names while allowing overrides (https://github.com/ansible/ansible/issues/69788)
- add AlmaLinux to fact gathering (https://github.com/ansible/ansible/pull/73458)
- add constraints file for ``ansible_runner`` test since an update to ``psutil`` is now causing test failures
- add magic/connection vars updates from delegated host info.
- add support for alpine linux 'apk' package manager in package_facts
- allow become method 'su' to work on 'local' connection by allocating a fake tty.
- ansible-console - Ctrl+C (in a task) abort current task, and put you back on prompt (this behavior doesn't change) (ditto)
- ansible-console - Ctrl+C (on prompt) used to exit the shell, unlike most shells, it should just reset the current line (ie. abort it and spawn a new prompt) (https://github.com/ansible/ansible/issues/68529)
- ansible-console - Ctrl+D (on prompt) now exit the shell, this is the expected behavior in a shell (cf bash, sh, zsh, ipython, ...) (ditto)
- ansible-console - add more documentation, specifically on various commands[1] (https://github.com/ansible/ansible/issues/72195)
- ansible-console - fixes few strings' typos
- ansible-console - remove useless and poorly formatted comment section (replaced with [1])
- ansible-doc - account for an empty ``meta/main.yml`` file when displaying role information (https://github.com/ansible/ansible/pull/73590)
- ansible-doc - collection name for plugin top-level deprecation was not inserted when deprecating by version (https://github.com/ansible/ansible/pull/70344).
- ansible-doc - improve error message in text formatter when ``description`` is missing for a (sub-)option or a return value or its ``contains`` (https://github.com/ansible/ansible/pull/70046).
- ansible-doc - improve man page formatting to avoid problems when YAML anchors are used (https://github.com/ansible/ansible/pull/70045).
- ansible-doc - include the collection name in the text output (https://github.com/ansible/ansible/pull/70401).
- ansible-doc - plugin option deprecations now also get ``collection_name`` added (https://github.com/ansible/ansible/pull/71735).
- ansible-doc - properly show plugin name when ``name:`` is used instead of ``<plugin_type>:`` (https://github.com/ansible/ansible/pull/71966).
- ansible-galaxy - Cache the responses for available collection versions after getting all pages. (https://github.com/ansible/ansible/issues/73071)
- ansible-galaxy - Instead of assuming the first defined server is galaxy, filter based on the servers that support the v1 API, and return the first of those (https://github.com/ansible/ansible/issues/65440)
- ansible-galaxy - Use ``sys.exit`` instead of ``exit`` when reporting an error for the removed login command.
- ansible-galaxy - correct ``collections-path`` command line argument (https://github.com/ansible/ansible/issues/73127)
- ansible-galaxy - fixed galaxy role init command (https://github.com/ansible/ansible/issues/71977).
- ansible-galaxy collection download - fix downloading tar.gz files and collections in git repositories (https://github.com/ansible/ansible/issues/70429)
- ansible-galaxy collection install - fix fallback mechanism if the AH server did not have the collection requested - https://github.com/ansible/ansible/issues/70940
- ansible-galaxy download - fix bug when downloading a collection in a SCM subdirectory
- ansible-pull - Run all playbooks that when multiple are supplied via the command line (https://github.com/ansible/ansible/issues/72708)
- ansible-test - Add ``pytest < 6.0.0`` constraint for managed installations on Python 3.x to avoid issues with relative imports.
- ansible-test - Add a ``six < 1.14.0`` constraint for Python 2.6.
- ansible-test - Always connect additional Docker containers to the network used by the current container (if any).
- ansible-test - Always map ``/var/run/docker.sock`` into test containers created by the ``--docker`` option if the docker host is not ``localhost``.
- ansible-test - Attempt to detect the Docker hostname instead of assuming ``localhost``.
- ansible-test - Avoid using ``/tmp`` to resolve occasional failures starting tests with the ``--docker`` option.
- ansible-test - Change classification using ``--changed`` now consistently handles common configuration files for supported CI providers.
- ansible-test - Change detection now properly resolves relative imports instead of treating them as absolute imports.
- ansible-test - Correctly detect changes in a GitHub pull request when running on Azure Pipelines.
- ansible-test - Correctly detect running in a Docker container on Azure Pipelines.
- ansible-test - Do not try to validate PowerShell modules ``setup.ps1``, ``slurp.ps1``, and ``async_status.ps1``
- ansible-test - Prefer container IP at ``.NetworkSettings.Networks.{NetworkName}.IPAddress`` over ``.NetworkSettings.IPAddress``.
- ansible-test - Running tests using an installed version of ``ansible-test`` against one Python version from another no longer fails due to a missing ``egg-info`` directory. This could occur when testing plugins which import ``pkg_resources``.
- ansible-test - Running tests using an installed version of ``ansible-test`` no longer generates an error attempting to create an ``egg-info`` directory when an existing one is not found in the expected location. This could occur if the existing ``egg-info`` directory included a Python version specifier in the name.
- ansible-test - Skip installing requirements if they are already installed.
- ansible-test - Symbolic links are no longer used to inject ``python`` into the environment, since they do not work reliably in all cases. Instead, the existing Python based exec wrapper is always used.
- ansible-test - Temporarily limit ``cryptography`` to versions before 3.4 to enable tests to function.
- ansible-test - The ``--export`` option for ``ansible-test coverage`` is now limited to the ``combine`` command. It was previously available for reporting commands on which it had no effect.
- ansible-test - The ``--raw`` option for ``ansible-test shell --remote`` now uses ``sh`` for the shell instead of ``bash``, which may not be present.
- ansible-test - The ``--remote`` option has been updated for Python 2.7 to work around breaking changes in the newly released ``get-pip.py`` bootstrapper.
- ansible-test - The ``--remote`` option has been updated to use a versioned ``get-pip.py`` bootstrapper to avoid issues with future releases.
- ansible-test - The ``ansible-doc`` sanity test now works for ``netconf`` plugins.
- ansible-test - The ``ansible-test coverage combine`` option ``--export`` now exports relative paths. This avoids loss of coverage data when aggregating across systems with different absolute paths. Paths will be converted back to absolute when generating reports.
- ansible-test - The ``changelog`` sanity test has been updated to ensure ``rstcheck`` does not load the ``sphinx`` module.
- ansible-test - The ``cs`` and ``openshift`` test plugins now search for containers on the current network instead of assuming the ``bridge`` network.
- ansible-test - The ``resource_prefix`` variable provided to tests running on Azure Pipelines is now converted to lowercase to match other CI providers.
- ansible-test - Unified SSH key management for all instances created with the ``--remote`` or ``--docker`` options.
- ansible-test - Using the ``--remote`` option on Azure Pipelines now works from a job running in a container.
- ansible-test - ``cryptography`` is now limited to versions prior to 3.2 only when an incompatible OpenSSL version (earlier than 1.1.0) is detected
- ansible-test - add constraint for ``cffi`` to prevent failure on systems with older versions of ``gcc`` (https://foss.heptapod.net/pypy/cffi/-/issues/480)
- ansible-test - convert target paths to unicode on Python 2 to avoid ``UnicodeDecodeError`` (https://github.com/ansible/ansible/issues/68398, https://github.com/ansible/ansible/pull/72623).
- ansible-test - ensure the correct unit test target is given when the ``__init__.py`` file is modified inside the connection plugins directory
- ansible-test - ensure unit test paths for connection and inventory plugins are correctly identified for collections (https://github.com/ansible/ansible/issues/73876).
- ansible-test - improve classification of changes to ``.gitignore``, ``COPYING``, ``LICENSE``, ``Makefile``, and all files ending with one of ``.in`, ``.md`, ``.rst``, ``.toml``, ``.txt`` in the collection root directory (https://github.com/ansible/ansible/pull/72353).
- ansible-test - integration and unit test change detection now works for filter, lookup and test plugins
- ansible-test now always uses the ``--python`` option for ``virtualenv`` to select the correct interpreter when creating environments with the ``--venv`` option
- ansible-test sanity changelog test - bump dependency on antsibull-changelog to 0.9.0 so that `fragments that add new plugins or objects <https://github.com/ansible-community/antsibull-changelog/blob/main/docs/changelogs.rst#adding-new-roles-playbooks-test-and-filter-plugins>`_ will not fail validation (https://github.com/ansible/ansible/pull/73428).
- ansible-test units - fixed collection location code to work under pytest >= 6.0.0
- ansible-test validate-modules - ``version_added`` on module level was not validated for modules in collections (https://github.com/ansible/ansible/pull/70869).
- ansible-test validate-modules - return correct error codes ``option-invalid-version-added`` resp. ``return-invalid-version-added`` instead of the wrong error ``deprecation-either-date-or-version`` when an invalid value of ``version_added`` is specified for an option or a return value (https://github.com/ansible/ansible/pull/70869).
- ansible-test validate-modules - when a module uses ``add_file_common_args=True`` and does not use a keyword argument for ``argument_spec`` in ``AnsibleModule()``, the common file arguments were not considered added during validation (https://github.com/ansible/ansible/pull/72334).
- ansible_pkg_mgr fact - now correctly returns ``atomic_container`` when run on "RHEL for Edge" images and Fedora/RHEL/CentOS Atomic Host (https://github.com/ansible/ansible/issues/73084).
- api - time.clock is removed in Python 3.8, add backward compatible code (https://github.com/ansible/ansible/issues/70649).
- apt - add ``fail_on_autoremove`` param to apt module to avoid unintended package removals (https://github.com/ansible/ansible/issues/63231)
- apt - fix policy_rc_d parameter throwing an exception when restoring original file (https://github.com/ansible/ansible/issues/66211)
- apt - include exception message from apt python library in error output
- apt_key - Specifying ``file`` as mutually exclusive with ``data``, ``keyserver``, ``url`` (https://github.com/ansible/ansible/pull/70492).
- argument spec validation - fix behavior of ``apply_defaults=True`` when an empty dictionary is specified for such an option (https://github.com/ansible/ansible/pull/74029).
- assemble - fix decrypt argument in the module (https://github.com/ansible/ansible/issues/65450).
- async - Fix Python 3 interpreter parsing from module by comparing with bytes (https://github.com/ansible/ansible/issues/70690)
- async_wrapper - Fix race condition when ``~/.ansible_async`` folder tries to be created by multiple async tasks at the same time - https://github.com/ansible/ansible/issues/59306
- avoid possible errors accessing os.environ by not assuming existance of variables.
- basic - handle exceptions for default selectors in Python 2.7 (https://github.com/ansible/ansible/issues/71704).
- basic - use PollSelector implementation when DefaultSelector fails (https://github.com/ansible/ansible/issues/70238).
- bcrypt hashing - Ensure we repair the salt, to avoid warnings (https://github.com/ansible/ansible/issues/36129)
- blockinfile - properly insert a block at the end of a file that does not have a trailing newline character (https://github.com/ansible/ansible/issues/72055)
- blockinfile now returns name of backup file when this option is used.
- clarified changed status to reflect existing rule that had never been written down.
- collection metadata - ensure collection loader uses libyaml/CSafeLoader to parse collection metadata if available
- connection/ssh, ensure parameters come from correct source get_option, so functionality matches docs.
- connection/ssh, fix reset to use same parameters to check if socket exists as actually used, was hardcoded to default string construction previouslly.
- cron - cron file should not be empty after adding var (https://github.com/ansible/ansible/pull/71207)
- cron - encode and decode crontab files in UTF-8 explicitly to allow non-ascii chars in cron filepath and job (https://github.com/ansible/ansible/issues/69492)
- debug action, prevent setting facts when displaying ansible_facts.
- default callback - Ensure that the ``host_pinned`` strategy is not treated as lockstep (https://github.com/ansible/ansible/issues/73364)
- delegate_to - Ensure that calculating ``delegate_to`` vars with a loop uses the correct context to correctly evaluate the loop (https://github.com/ansible/ansible/issues/37132)
- display correct error information when an error exists in the last line of the file (https://github.com/ansible/ansible/issues/16456)
- distribution - add support for Pardus Linux distribution (https://github.com/ansible/ansible/issues/71636).
- distribution facts - Allow ``distribution_major_version`` and ``distribution_version`` to work for RC and PRERELEASE versions of FreeBSD (and derived distributions) (https://github.com/ansible/ansible/issues/72331).
- dnf - fix filtering to avoid dependncy conflicts (https://github.com/ansible/ansible/issues/72316)
- dnf - it is now possible to specify both ``security: true`` and ``bugfix: true`` to install updates of both types. Previously, only security would get installed if both were true. (https://github.com/ansible/ansible/issues/70854)
- ensure 'local' connection always has the correct default user for actions to consume.
- ensure delegated vars can resolve hostvars object and access vars from hostvars[inventory_hostname].
- ensure we don't clobber role vars data when getting an empty file
- expect - Operate pexpect with bytes to avoid potential encoding issues (https://github.com/ansible/ansible/issues/29351)
- facts - account for Slackware OS with ``+`` in the name (https://github.com/ansible/ansible/issues/38760)
- facts - fix distribution fact for SLES4SAP (https://github.com/ansible/ansible/pull/71559).
- facts - fix incorrect UTC timestamp in ``iso8601_micro`` and ``iso8601``
- facts - properly report virtualization facts for Linux guests running on bhyve (https://github.com/ansible/ansible/issues/73167)
- file - prevent link src from being rewritten when src is not specified explicitly (https://github.com/ansible/ansible/issues/65448)
- file - the module should warn in check_mode when path an owner/group don't exist (https://github.com/ansible/ansible/issues/67307).
- find module - Don't treat empty excludes as a match (https://github.com/ansible/ansible/issues/70640)
- find module - Stop traversing directories past the requested depth. (https://github.com/ansible/ansible/issues/73627)
- find module, fix default pattern when use_regex is true.
- fix issue with inventory_hostname and delegated host vars mixing on connection settings.
- fortimanager httpapi plugin - fix redirect to point to the ``fortinet.fortimanager`` collection (https://github.com/ansible/ansible/pull/71073).
- galaxy - handle plus sign in user token appearing in role url (https://github.com/ansible/ansible/issues/45475).
- get_sysctl now handles multiline values and does not die silently anymore.
- get_url - skip checksum during ``--check`` (https://github.com/ansible/ansible/issues/61369).
- git - Only pass ``--raw`` flag to git verify commands (verify-tag, verify-commit) when ``gpg_whitelist`` is in use. Otherwise don't pass it so that non-whitelist GPG validation still works on older Git versions. (https://github.com/ansible/ansible/issues/64469)
- gluster modules - fix redirect to point to the ``gluster.gluster`` collection (https://github.com/ansible/ansible/pull/71240).
- hostname - Fixed an issue where the hostname on the alinux could not be set.
- hostname - add Almalinux support (https://github.com/ansible/ansible/pull/73619)
- hostname - add macOS support (https://github.com/ansible/ansible/pull/54439)
- if the ``type`` for a module parameter in the argument spec is callable, do not pass ``kwargs`` to avoid errors (https://github.com/ansible/ansible/issues/70017)
- import_playbook - change warning about extra parameters to deprecation (https://github.com/ansible/ansible/issues/72745)
- improve deprecation message when using bare variable (https://github.com/ansible/ansible/pull/70687)
- inventory - pass the vars dictionary to combine_vars instead of an individual key's value (https://github.com/ansible/ansible/issues/72975).
- inventory plugins - Let plugins define the sanitization method for the constructed ``groups`` feature.
- inventory_hostnames - Use ``InventoryManager`` instead of trying to replicate its behavior (https://github.com/ansible/ansible/issues/17268)
- is_string/vault - Ensure the is_string helper properly identifies AnsibleVaultEncryptedUnicode as a string (https://github.com/ansible/ansible/pull/71609)
- j2 plugin loader clarified comments, made note with better fqcn detection.
- lineinfile - fix not subscriptable error in exception handling around file creation
- linux network facts - get the correct value for broadcast address (https://github.com/ansible/ansible/issues/64384)
- notify keyword is not ignored anymore on import_tasks, also able to apply to blocks now.
- package - use list of built in package managers from facts rather than creating a new list
- paramiko connection plugin - Ensure we only reset the connection when one has been previously established (https://github.com/ansible/ansible/issues/65812)
- password hashing - Ensure we validate salts against allowed characters and length when using ``crypt`` (https://github.com/ansible/ansible/issues/71107)
- password lookup - Try to automatically generate salts using known salt sizes (https://github.com/ansible/ansible/issues/53750)
- pause - Fix indefinite hang when using a pause task on a background process (https://github.com/ansible/ansible/issues/32142)
- pause - catch additional error on setting up curses (https://github.com/ansible/ansible/pull/73588).
- pause - do not accept enter to continue when a timeout is set (https://github.com/ansible/ansible/issues/73948)
- pause - do not warn when running in the background if a timeout is provided (https://github.com/ansible/ansible/issues/73042)
- pause - handle exception when there is no stdout (https://github.com/ansible/ansible/pull/47851)
- powershell - fix escaping of strings that broken modules like fetch when dealing with special chars - https://github.com/ansible/ansible/issues/62781
- powershell - fix the CLIXML parser when it contains nested CLIXML objects - https://github.com/ansible/ansible/issues/69550
- powershell - remove getting the PowerShell version from the env var ``POWERSHELL_VERSION``. This feature never worked properly and can cause conflicts with other libraries that use this var
- psrp - Fix hang when copying an empty file to the remote target
- psrp - Use native PSRP mechanism when copying files to support custom endpoints
- quote filter - normalize how ``None`` is handled, to match Python3 behavior (https://github.com/ansible/ansible/issues/32174)
- reboot - Add support for the runit init system, used on Void Linux, that does not support the normal Linux syntax.
- remove contradictory recomendation from template docs. https://github.com/ansible/ansible/issues/63484
- remove redundant remote_user setting in play_context for local as plugin already does it, also removes fork/thread issue from use of pwd library.
- reset logging level to INFO due to CVE-2019-14846.
- restrict module valid JSON parsed output to objects as lists are not valid responses.
- runas - create a new token when running as ``SYSTEM`` to ensure it has the full privileges assigned to that account
- service - Fix for the BSD rcconf code using a Python 2 specific string replace function
- set_mode_if_different - handle symlink if it is inside a directory with sticky bit set (https://github.com/ansible/ansible/pull/45198)
- setup module, fix error handling on bad subset given
- setup, don't give up on all local facts gathering if one script file fails.
- several fixes to make apt_key better at identifying needs for change and also to avoid changes in check_mode.
- shell - fix quoting of mkdir command in creation of remote_tmp in order to allow spaces and other special characters (https://github.com/ansible/ansible/issues/69577).
- splunk httpapi plugin - switch from splunk.enterprise_security to splunk.es in runtime.yml to reflect upstream change of Collection Name
- ssh connection plugin - use ``get_option()`` rather than ``_play_context`` to ensure ``ANSBILE_SSH_ARGS`` are applied properly (https://github.com/ansible/ansible/issues/70437)
- stat - handle colons in filename while parsing the mimetype output (https://github.com/ansible/ansible/issues/70256).
- strftime filter - Input epoch is allowed to be a float (https://github.com/ansible/ansible/issues/71257)
- su become plugin, ensure correct type for localization option.
- systemd - account for templated unit files using ``@`` when searching for the unit file (https://github.com/ansible/ansible/pull/72347#issuecomment-730626228)
- systemd - fixed chroot usage on new versions of systemd, that broke because of upstream changes in systemctl output
- systemd - follow up fix to https://github.com/ansible/ansible/issues/72338 to use ``list-unit-files`` rather than ``list-units`` in order to show all units files on the system.
- systemd - made the systemd module work correctly when the SYSTEMD_OFFLINE environment variable is set
- systemd - preserve the full unit name when using a templated service and ``systemd`` failed to parse dbus due to a known bug in ``systemd`` (https://github.com/ansible/ansible/pull/72985)
- systemd - work around bug with ``systemd`` 245 and 5.8 kernel that does not correctly report service state (https://github.com/ansible/ansible/issues/71528)
- task parsing - strip spaces from action name when using ``action: foo bar=baz`` form. (https://github.com/ansible/ansible/issues/62136)
- templating - fix error message for ``x in y`` when y is undefined (https://github.com/ansible/ansible/issues/70984)
- the unvault lookup plugin returned a byte string. Now returns a real string.
- to_text(stdout) before json.loads in psrp.Connection.put_file in case stdout is bytes
- unarchive - ``zip`` unarchive no longer errors on RHEL/CentOS 6 and old Fedora when attempting to use a numeric gid (https://github.com/ansible/ansible/issues/71903).
- unarchive - check ``fut_gid`` against ``run_gid`` in addition to supplemental groups (https://github.com/ansible/ansible/issues/49284)
- undeprecate hash_merge setting and add more docs clarifying its use and why not to use it.
- uri - ``status_code`` elements are type ``int``
- url lookup - make sure that options supplied in ansible.cfg are actually used (https://github.com/ansible/ansible/pull/71736).
- url lookup - set default user agent to ``ansible-httpget`` (https://github.com/ansible/ansible/pull/72324)
- urls - Close filedescriptor of certificate chain tempfile to prevent stale filedescriptor leakage (https://github.com/ansible/ansible/pull/71825).
- user - AnsibleModule.run_command returns a tuple of return code, stdout and stderr. The module main function of the user module expects user.create_user to return a tuple of return code, stdout and stderr. Fix the locations where stdout and stderr got reversed.
- user - Local users with an expiry date cannot be created as the ``luseradd`` / ``lusermod`` commands do not support the ``-e`` option. Set the expiry time in this case via ``lchage`` after the user was created / modified. (https://github.com/ansible/ansible/issues/71942)
- user - do the right thing when ``password_lock=True`` and ``password`` are used together (https://github.com/ansible/ansible/issues/72992)
- user - don't create home directory and missing parents when create_home == false (https://github.com/ansible/ansible/pull/70600).
- validate-modules - do not raise an ``AttributeError`` if a value is assigned to a module attribute in a try/except block.
- vault - Support reading raw binary data from stdin under python3
- virtual facts - kubevirt is now identified as "KubeVirt" and with a "guest" role instead of "kvm" and "host" role (https://github.com/ansible/ansible/issues/72001).
- wait_for - catch and ignore errors when getting active connections with psutil (https://github.com/ansible/ansible/issues/72322)
- wait_for module, move missing socket into function to get proper comparrison in time.
- win setup - Fix redirection path for the windows setup module
- windows async - use full path when calling PowerShell to reduce reliance on environment vars being correct - https://github.com/ansible/ansible/issues/70655
- winrm - preserve winrm forensic data on put_file failures
- yamllint - do not raise an ``AttributeError`` if a value is assigned to a module attribute at the top of the module.
Known Issues
------------
- ansible-test - The ``pylint`` sanity test no longer correctly detects "bad" variable names for non-constants. See https://github.com/PyCQA/pylint/issues/3701 for additional details.
New Modules
-----------
- validate_argument_spec - Validate role argument specs.
- ssh - connection plugin now supports a new variable ``sshpass_prompt`` which gets passed to ``sshpass`` allowing the user to set a custom substring to search for a password prompt (requires sshpass 1.06+)
- apt_repository - Implemented an exponential backoff behaviour when retrying to update the apt cache with new params ``update_cache_retry_max_delay`` and ``update_cache_retries`` to control the behavior.
- combine filter - ``[dict1, [dict2]] | combine`` now raise an error; previously ``combine`` had an undocumented behaviour where it was flattening the list before combining it (https://github.com/ansible/ansible/pull/57894#discussion_r339517518).
minor_changes:
- combine filter - now accept a ``list_merge`` argument which modifies its behaviour when the hashes to merge contain arrays/lists.
- CLI - the `ANSIBLE_PLAYBOOK_DIR` envvar or `playbook_dir` config can now substitute for the --playbook-dir arg on CLIs that support it (https://github.com/ansible/ansible/issues/59464)
- cron and cronvar - use get_bin_path utility to locate the default crontab executable instead of the hardcoded /usr/bin/crontab. (https://github.com/ansible/ansible/pull/59765)
- apt - Implemented an exponential backoff behaviour when retrying to update the cache with new params ``update_cache_retry_max_delay`` and ``update_cache_retries`` to control the behavior.
- Handle exception encountered while parsing the argument description in module when invoked via ansible-doc command (https://github.com/ansible/ansible/issues/60587).
- vars plugins - Support vars plugins in collections by adding the ability to whitelist plugins.
- host_group_vars plugin - Require whitelisting and whitelist by default.
- Add a global toggle to control when vars plugins are executed (per task by default for backward compatibility or after importing inventory).
- Add a per-plugin stage option to override the global toggle to control the execution of individual vars plugins (per task, after inventory, or both).
- lineinfile - fix bug that caused multiple line insertions (https://github.com/ansible/ansible/issues/58923).
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.