Commit Graph

68 Commits

Author SHA1 Message Date
6627ba9ace Unit test cleanup (#81556)
* Fix cleanup_tmp_file unit tests

* Simplify the problematic wcwidth chars unit test
2023-08-21 15:54:03 -07:00
48dfe8e215 add Python 3.12 support to ansible-test (#80834)
* update docker containers versions to use newer ansible-test ref in the pre-built venvs

* Allow invoking ansible-test with Python 3.12

* Add python3.12 to the INTERPRETER_PYTHON_FALLBACK

* changelog

* add Python 3.12 as a non-default Python version for the test containers

* Update mypy ignores for Python 3.12

* Add Python 3.12 to CI matrix for unit tests, generic tests, and galaxy

* Update unit test for using the Python 2 collection loader path with Python 3.

Skip the existing test on Python 3.12, since find_module is removed.

Suppress the pre-existing deprecation warnings using the Python 2
codepath with Python 3.

Add a test for Python >= 3.12, which doesn't call find_module.

* Ignore sanity test errors on systems without libselinux present.
2023-07-21 10:23:14 -07:00
63946eaed6 Remove unreachable code in curses unit test (#81138) 2023-06-27 23:24:42 -07:00
a7d2a4e032 display, also proxy warnings and deprecations (#81060)
Use generic proxy for display  methods with decorator
2023-06-22 13:57:59 -04:00
2ba24957dd Use subset checks for some collection loader tests (#80812) 2023-05-16 10:29:46 -07:00
b981a9dfcd add a worker queue to get updates from the main results thread (#79886)
* Create a queue per WorkerProcess to receive intra-task updates
* Update `pause` action to use the worker queue
* Deprecate ConnectionBase()._new_stdin
* Add new `Display`  convenience method `prompt_until` to manage both controller- and worker-sourced prompting without cross-fork stdin sharing, in-worker mechanism to handle request-response over new worker queue.
2023-02-28 12:21:48 -08:00
1ba100a3b4 Fix pointless statements in unit tests (#79940) 2023-02-07 14:00:38 -08:00
25d859f8fb Clean up unused imports in unit tests (#79893) 2023-02-02 15:52:38 -08:00
91807695c3 Remove unused unit test code (#79879)
* Remove unused test fixtures

* Removed unused _old_dump_load_cycle method

* Remove Python 2.x compat

* Remove unused code

* Remove unused context manager

* Fix cowsay test

- The test no longer depends on another test to initialize config.
- Also remove unreachable code.

* Remove Python 2.x compat
2023-02-01 16:00:57 -08:00
56d142350d Add support for importlib.resources (#78915)
* Add support for importlib.resources

* Remove the importlib.resources imports

* return the correct data

* Some code comments, and re-order for consistency

* Disallow traversing packages below an individual collection

* Add a traversable class for namespaces

* Re-use variable

* Utilize itertools.chain.from_iterable

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>

* Simplify logic to check for packages from ansible loaders

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>

* Just a generator expression, instead of a generator

* docstrings

* Add comment about find_spec for our namespaces

* Add some initial unit tests for importlib.resources

* normalize

* Utilize importlib.resources for listing collections

* collections_path is already in config, just use config

* install uses a different default for collections_path

* Remove unused import

* Remove duplicate __truediv__

* Bring back TraversableResources

* Apply some small suggestions from code review

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>

* Remove cross contamination between plugin loader code and CLI code

* Remove unused import

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
2023-01-10 11:09:57 -06:00
d8cb9117ac Skip test if no there are no problematic chars for wcswidth (#78645) 2022-08-29 09:06:37 -05:00
b1dd2af4ca Preflight utf8 encodings check (#78175) 2022-07-11 09:22:27 -05:00
5e369604e1 Forked display via queue (#77056)
* Forked Display via queue

* Docs and simple code cleanup

* Only proxy Display.display

* Remove unused import

* comment

* Update deadlock comment, remove py3 check

* Don't flush display, and don't lock from forks

* clog frag

* ci_complete ci_coverage

* Add units for queue proxying

* Cleanup flush

* ci_complete

* Only lock the write, switch to RLock

* Remove unused import
2022-06-07 08:31:56 -07:00
3a9a23fb1a Use unittest.mock instead of mock. (#77883)
This can only be done for controller tests.
2022-05-23 11:04:56 -07:00
1a5853d794 Remove obsolete units.compat.mock compat layer. (#77118)
* Remove obsolete units.compat.mock compat layer.
* Update remaining units.compat.mock references.
2022-02-22 18:20:11 -08:00
ed6581e4db check finder type before passing path (#76448)
* check finder type before passing path

ci_complete

* Reduce nesting

* Test find_module does not cause a traceback with Python 3 FileFinder

* Update lib/ansible/utils/collection_loader/_collection_finder.py
2021-12-09 16:05:58 -08:00
472028c869 catch the case that cowsay is broken (#76326)
* catch the case that cowsay is broken

fixes https://github.com/ansible/ansible/issues/72582

  add changelog
  raise Exception for broken cowsay
  add test for broken cowsay

Co-authored-by: Matthias Bernt <m.bernt@ufz.de>
2021-12-07 09:11:06 -05:00
57359d0174 use passlib default if password_hash option isn't set (#75043)
* add changelog fragment
* password_hash|length == 120
* mention the new default bcrypt format in the porting guide
2021-09-21 17:35:53 -04:00
1bd7dcf339 encrypt: add new paramter 'ident' (#74595)
Add a new parameter `ident` for specifying version of BCrypt
algorithm. This parameter is only valid for `blowfish` hash type.
2021-05-24 11:46:37 -04:00
8d1cf7f266 Vendor distutils.version (#74644)
* Vendor distutils.version

* Fix import order. ci_complete

* remove distutils warning filter

* Don't remove warnings filter from importer

* ci_complete

* Add pylint config for preventing distutils.version

* Add changelog fragment
2021-05-11 12:33:51 -05:00
4494ef3a9d defend against bad or missing crypt (#74304)
* defend against bad or missing crypt

  fixes #74279
2021-04-20 11:47:34 -04:00
9ba6cf9a72 Relocate misplaced unit tests. 2021-04-16 22:23:40 -07:00
920b68f5f2 Fixed/improved regular expresssion for collection names (#73577)
* added changelog fragment
* added a couple of tests to coll name validation
2021-03-01 10:40:38 -05:00
f327e65d11 Use iskeyword and str.isidentifier for "is FQCN" (#73279)
* Use valid FQCN in test_verbosity_arguments

* Use iskeyword and str.isidentifier for "is FQCN"
2021-02-11 18:23:57 -08:00
d22804c4fb saner path dir management (#72648)
* saner path dir management

   fixes #72628

   ensure we always store paths w/o a_c

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
2020-12-14 19:30:13 -05:00
e05c62547b run playbook from collections (#67435)
* fixes for collection playbooks

 - add fqcn invocation, also handles extensions
 - bring import_playbook into new normal
 - avoid adding collection playbooks to adjacency
 - deal with extensions or lack of em
 - added tests
 - fix bugs with multiple playbooks supplied
 - unicode/bytes/native fixes
 - fix modulenotfound for < py3.7
 - tests
2020-10-28 11:46:39 -04:00
1107aace1b Add deprecation collection name to plugin options (#71735)
* Add changelog fragment.

* Use correct field that's expected by lib/ansible/cli/__init__.py..

* Add basic unit tests.
2020-10-02 09:43:23 -04:00
e813b0151c fix coverage output from synthetic packages (#71727)
* fix coverage output from synthetic packages

* synthetic packages (eg, implicit collection packages without `__init__.py`) were always created at runtime with empty string source, which was compiled to a code object and exec'd during the package load. When run with code coverage, it created a bogus coverage entry (since the `__synthetic__`-suffixed `__file__` entry didn't exist on disk).
* modified collection loader `get_code` to preserve the distinction between `None` (eg synthetic package) and empty string (eg empty `__init__.py`) values from `get_source`, and to return `None` when the source is `None`. This allows the package loader to skip `exec`ing things that truly have no source file on disk, thus not creating bogus coverage entries, while preserving behavior and coverage reporting for empty package inits that actually exist.

* add unit test
2020-09-14 09:14:23 -07:00
fdf5dd02b3 Updates for password hashing (#71120)
* Validate salt when using crypt. Respect salt_size in password lookup. Repair salt for bcrypt. Fixes #71107. Fixes #53750. Fixes #36129.

* Handle algorithms we don't know about, and make sure to return the salt

* Account for old passlib

* Add tests for salt constraints

* Add changelog fragment

* Add test for #36129
2020-08-26 14:54:38 -05:00
1223ce656a reset logging to INFO (#70878)
- due to CVE-2019-14846
 - also added comments and test to avoid 'oportunistic' reversion
2020-07-24 15:53:17 -04:00
1fedb95e4b Use libc wcwidth to calculate print width in display (#66214)
* Use libc wcwidth to calculate print width in display. Fixes #63105

* Remove errantly added blank lines

* Fixes

* Move setlocale, adjust tests to work around py2 oddity with characters following null

* Don't change cli stub

* emojis

* Remove to_text call

* Special accounting for deletions

* Add initialization function, expand tests, ensure fallback to len

* get_text_width requires text, ensure banner deals with it

* Handle setlocale errors

* Move variable decrement

* Remove unused import
2020-06-22 16:57:01 -05:00
98a0995fd0 Clean up unit test boilerplate. 2020-06-22 14:20:33 -07:00
47d14a33bd config: singular ANSIBLE_COLLECTIONS_PATH (#70007)
* config: singular ANSIBLE_COLLECTIONS_PATH

Every other *_PATH setting in ansible is singular, and the traditional
$PATH variable is also singular despite containing a list of
directories.  Let's be consistent both internally and with POSIX
tradition.

* update all ANSIBLE_COLLECTIONS_PATHS env references to be singular

* deprecate plural ANSIBLE_COLLECTIONS_PATHS setting
2020-06-11 11:40:13 -07:00
fa81cc6a0c fix delegated loading when path is not a directory (#69713)
* find_module can't pop ImportError- we need to just translate to `None` since this is a normal condition with files on sys.path (eg `/usr/lib/python36.zip`)
* added test
2020-05-26 15:55:17 -07:00
f7dfa817ae collection routing (#67684)
* `meta/` directory in collections
* runtime metadata for redirection/deprecation/removal of plugin loads
* a compatibility layer to keep existing content working on ansible-base + collections
* a Python import redirection layer to keep collections-hosted (and otherwise moved) content importable by things that don't know better
* supported Ansible version validation on collection loads
2020-05-26 09:42:06 -07:00
1ee70fc272 ansible.utils.vars.isidentifier improvements (#58278)
ci_complete
2020-05-21 19:54:40 +02:00
5b9418c06c Fix SemanticVersion comparison (#69395)
* Fix SemanticVersion comparison.

* Complete tests for _Alpha and _Numeric comparators.

* Linting, and add comment.
2020-05-11 11:58:15 -05:00
2068131589 Fix colorization to not extend across newline boundary (#68517)
* Fix colorization to not extend across newline boundary

* Fix unit test to look for the newline outside the coloration

* Add changelog fragment
2020-03-27 17:10:38 -07:00
d3ec31f8d5 Support pre-releases via new SemanticVersion (#68258)
* Support pre-releases via new SemanticVersion. Fixes #64905

* Don't treat buildmeta as prerelease

* Don't inherit from str and int

* Add helper method to try and construct a SemanticVersion from a LooseVersion

* Don't count major 0 as pre-release, it's different

* Guard against invalid or no version in LooseVersion

* return a bool

* Add integration tests for pre-release

* Fix up lingering issues with comparisons

* typo fix

* Always allow pre-releases in verify

* Move pre-release filtering into CollectionRequirement, add messaging when a collection only contains pre-releases

* Update changelog

* If explicit requirement allow pre releases

* Enable pre-releases for tar installs, and collections already installed when they are pre-releases

* Drop --pre-release alias, make arg name more clear

* Simplify code into a single line

* Remove build metadata precedence, add some comments, and is_stable helper

* Improve from_loose_version

* Increase test coverage

* linting fix

* Update changelog
2020-03-23 16:04:07 -05:00
7e8dae5790 Remove unused unit test files. 2020-03-23 11:14:21 -05:00
53e043b5fe combine filter: fine list handling (option b) (#57894) 2020-02-12 16:40:36 -05:00
8e195adda5 display - remove extra new line after warning message (#65199)
Add unit tests for display
2020-01-24 12:10:04 -05:00
f8654de851 Ensure that data within a tuple is marked as unsafe (#65918)
* Use is_sequence, and Mapping throughout, add support for tuples. Fixes #65722

* Address tests

* Remove unused import

* Add changelog

* Add docstring for clarity

* Argh, linting fix

* Not chasing this rabbit

* wrap_var doesn't return a ref to the original item

* no ref tests

* Remove unused import
2020-01-07 08:41:37 -06:00
c5d61953b1 Remove empty setUp/tearDown/tearDownClass methods in test classes. 2019-11-06 08:14:29 -08:00
8528fbc790 Amazon related Sanity test fixups (remaining batch 2) (#64358)
* Remove sanity/ignore.txt entries

* replace use of "_" as a variable name

* Cleanup boilerplate

* Remove default values from mandatory parameters

* Sanity test documentation updates

* Remove unused imports from iam_role
2019-11-04 16:56:14 +00:00
bfa004930a add subdir support to collection loading (#60682)
* add subdir support to collection loading

* collections may now load plugins from subdirs under a plugin type or roles dir, eg `ns.coll.subdir1.subdir2.myrole`->ns.coll's roles/subdir1/subdir2/myrole, `ns.coll.subdir1.mymodule`->ns.coll's plugins/modules/subdir1/mymodule.py
* centralize parsing/validation in AnsibleCollectionRef class
* fix issues loading Jinja2 plugins from multiple sources 
* resolves #59462, #59890,

* sanity test fixes

* string fixes

* add changelog entry
2019-08-21 10:45:04 +01:00
697b566971 Update units to pass on macOS (#60435)
* Update units to pass on macOS. Fixes #27810

* raising=False
2019-08-12 15:13:07 -05:00
6c09b5c659 Unit test cleanup. (#60315)
* Clean up option parsing unit tests.
* Remove unused unit test marker and import.
2019-08-09 08:31:13 -07:00
164881d871 Remove UnsafeProxy (#59711)
* Remove UnsafeProxy

Move the work from UnsafeProxy to wrap_var and add support for bytes.
Where wrap_var is not needed, use AnsibleUnsafeBytes/AnsibleUnsafeText
directly.

Fixes #59606

* item is not always text

* Address issues from reviews

* ci_complete
2019-08-07 10:39:01 -05:00
32de51adac Unit tests: remove unused imports (#59740) 2019-07-29 22:10:13 +02:00