Commit Graph

128 Commits

Author SHA1 Message Date
53af57b08b Fix type discrepancy in ansible-test. 2019-07-30 10:46:05 -07:00
66654475e1 Update ansible-test Python version handling.
Minimum version requirements for sanity tests have been standardized:

- All single version sanity tests now require Python 3.5 or later.
- All multiple version sanity tests continue to use all supported Python versions.
- All version neutral sanity tests continue to work on any supported Python version.

Previously some tests required 3.5 or later with most of the remaining tests requiring 2.7 or later.

When using the `--python` option to specify a Python version:

- Tests which do not support the specified Python version will be skipped with a warning.
- If the specified Python version is not available, any test attempting to use it will generate an error.

When not using the `--python` option to specify a Python version:

- Multiple version tests will attempt to run on all supported versions.
- Single version tests will use the current version if supported and available, or if no supported version is available.
- Single version tests will use the lowest available and supported version if the current version is not supported.
- Any versions which are not available or supported will be skipped with a warning.

Unit tests automatically skip unavailable Python versions unless `--python` was used to specify a version.
2019-07-29 11:54:03 -07:00
79eca9c8fb Initial ansible-test support for collections. (#59197)
* Initial ansible-test support for collections.
* Include cloud config in delegation payload.
* Add missing types import and fix `t` shadowing.
* Fix plugin traceback when config_path not set.
* Fix encoding issues.
* Remove unused imports.
* More encoding fixes.
* Handle delegation outside exception handler.
* Inject ssh keys only if not already in place.
* More defensive approach to getting remote pwd.
* Add missing string format var.
* Correct PowerShell require regex.
* Rename `is_install` and `INSTALL_ROOT`.
2019-07-22 19:24:48 -07:00
4255ee8e5a Prepare ansible-test for collections support.
Another round of changes to prepare ansible-test for supporting collections to help keep later PRs a manageable size.
2019-07-15 17:46:29 -07:00
f81238012b Clean up pylint issues in test infra. 2019-07-15 14:45:30 -07:00
6645054329 Code cleanup for test infrastructure. (#59046)
* Remove useless object inheritance in test code.
* Remove unnecessary pass statements from test code.
* Comment on why certain pylint rules are ignored.
* Add more pylint test contexts.
* Fix import order.
* Remove unused variables.
* Remove unnecessary pass statement.
* Fix bad continuations.
* Remove unnecessary elif.
* Allow legitimate broad except statements.
* Allow legitimate protected access.
* Clean up names to make pylint pass.
2019-07-12 13:17:20 -07:00
3f2b766d10 Add future and metaclass boilerplate to test code.
Continue to ignore:

- test/integration/
- test/legacy/
- test/units/
2019-07-12 09:18:24 -07:00
6327a6114b Add ansible-test units --num-workers option. 2019-07-11 17:15:16 -07:00
bc3a599678 Fix invalid type hints in ansible-test. 2019-07-11 14:25:57 -07:00
86ad4c99ba Reorganize util code in ansible-test.
Code in util.py that depends on CommonConfig is now in util_common.py.
2019-07-11 08:54:13 -07:00
c79afb1cba Move ansible-test unit coverage code into plugin. 2019-07-10 10:09:46 -07:00
1e1463401d Prepare ansible-test for supporting collections. (#58886)
This is a small but incomplete set of the initial changes for supporting testing of collections with ansible-test.
2019-07-09 17:31:04 -07:00
4403f866e3 Simplify Git class in ansible-test. 2019-06-27 18:01:02 -07:00
b06f5634c5 Simplify ansible-test target expansion.
Targets are always expanded to full lists now instead of optimizing
for shorter lists by collapsing directories.

This change only affects unit tests and the ansible-doc sanity test,
as they were the only remaining tests using the old behavior.
2019-06-27 08:27:48 -07:00
817ec5ab48 Add PyYAML check for libyaml support. 2019-06-05 15:20:47 -07:00
484c023316 ansible-test: prefer shlex.quote (#56823) 2019-05-24 22:10:33 +02:00
864bd941af Add support for isolated virtualenv in tests. 2019-05-06 16:28:52 -04:00
a4d0c24bbf Fix ansible-test inventory and vars path handling. (#55186)
* Fix ansible-test inventory and vars path handling.
* Improve explanation for missing commit.
2019-04-12 11:52:44 -07:00
a4c42ba687 Update the default test container to 1.7.0. (#54930)
* Update the default test container to 1.7.0.
* Run `pip check` after installing test requirements.
* Support older versions of pip.
* Remove obsolete pip error handler.
2019-04-05 22:34:38 -07:00
785afc7a53 Improve ansible-test python interpreter selection. (#54445) 2019-03-27 16:40:27 -07:00
89a022b6a0 Update handling of the non_local test alias.
Previously the alias disabled all code coverage collection.
Now it only disables Ansible module code coverage collection.
2019-03-22 16:05:24 -07:00
a8e328f474 Overhaul ansible-test code coverage and injector. (#53510) 2019-03-13 07:14:12 -07:00
8535c05b29 Add ansible-test finally block after tests run.
This will record data from passing tests even when tests fail.
2019-03-07 18:31:02 -08:00
8ef2e6da05 Add support for Windows hosts in the SSH connection plugin (#47732)
* Add support for Windows hosts in the SSH connection plugin

* fix Python 2.6 unit test and sanity issues

* fix up connection tests in CI, disable SCP for now

* ensure we don't pollute the existing environment during the test

* Add connection_windows_ssh to classifier

* use test dir for inventory file

* Required powershell as default shell and fix tests

* Remove exlicit become_methods on connection

* clarify console encoding comment

* ignore recent SCP errors in integration tests

* Add cmd shell type and added more tests

* Fix some doc issues

* revises windows faq

* add anchors for windows links

* revises windows setup page

* Update changelogs/fragments/windows-ssh.yaml

Co-Authored-By: jborean93 <jborean93@gmail.com>
2019-03-07 16:38:02 -08:00
156613f025 Fix coverage handling for non_local tests. 2019-03-06 16:30:37 -08:00
eeaff731de Overhaul ansible-test cloud test plugins. (#53044) 2019-02-28 18:25:49 -08:00
6a82827ddd Add Python 3.8 to ansible-test and CI. 2019-02-14 23:24:24 -08:00
ce8db479f0 ansible-test: make the httptester for Windows more resiliant around the shell chosen (#51416) 2019-01-30 07:50:23 +10:00
b834b29e43 Run integration tests from temporary directory.
ci_complete
2019-01-17 17:22:51 -08:00
eee3dc888b Pass ansible-test inventory path to test funcs.
This avoids declaring the inventory path/names in multiple places.
Also expose the inventory path using the INVENTORY_PATH env var.
2019-01-15 18:07:13 -08:00
3b705efc93 Fix ansible-test skip warning message. 2018-11-27 07:50:22 -08:00
0826a00803 Add --raw option to ansible-test shell command.
It is currently supported only with the `--remote` option.

This makes it easier to troubleshoot new instances which are not
yet supported by the setup scripts used by ansible-test.
2018-11-16 17:38:15 -08:00
8066acc90c Support skip of platforms by version in tests. (#48826)
* Support skip of platforms by version in tests.

Previously a remote platform could be skipped completely using the alias:

`skip/{platform}` such as `skip/rhel`

Now a specific platform version can be skipped using the alias:

`skip/{platform}{version}` such as `skip/rhel7.6`

This feature is available for platforms specified with the `--remote` option.

* Add skip by version to the docs.
2018-11-16 16:38:47 -08:00
fa53b4805b Fix ansible-test interpreter tracking.
Track the interpreter for each copy of the injector by the interpreter
path instead of the interpreter version. This avoids the possibility
of mixing different interpreters with the same version.
2018-11-14 13:58:03 -08:00
eec21a3d12 Use correct interpreter for ansible-test injector. 2018-11-13 22:50:34 -08:00
9a5561da0f ansible-test: setup up http runner in between each target (#47100)
* ansible-test: setup up http runner in between each target

* review changes
2018-10-16 18:09:17 +10:00
69e1088ec1 ansible-test: run win httptester with bypass policy (#47090) 2018-10-16 08:07:02 +10:00
38eba60849 Fix ansible-test --exclude with delegation.
Previously the option worked with integration commands but not units or sanity.
2018-10-15 14:05:37 -07:00
6e2897647c win test: add http tester container to Windows tests (#46606) 2018-10-13 08:20:00 +10:00
0f5331645f ansible-test: add skip/windows/... alias to skip tests on specific Windows versions (#46845)
* ansible-test: add skip/windows/... alias to skip tests on specific Windows versions

* show what tests were skipped

* changes to logic to only skip if all Windows targets are set to skip

* codestyle improvements

* change warning message based on review

* check args type before running the Windows path
2018-10-12 15:32:46 +10:00
ec88510fd4 Add OUTPUT_DIR env var to integration tests. 2018-10-11 20:34:12 -07:00
712ad9ed64 Fix ansible-test custom docker image traceback. 2018-10-11 10:49:38 -07:00
0dc7f38787 Improve ansible-test environment checking between tests. (#46459)
* Add unified diff output to environment validation.

This makes it easier to see where the environment changed.

* Compare Python interpreters by version to pip shebangs.

This helps expose cases where pip executables use a different
Python interpreter than is expected.

* Query `pip.__version__` instead of using `pip --version`.

This is a much faster way to query the pip version. It also more
closely matches how we invoke pip within ansible-test.

* Remove redundant environment scan between tests.

This reuses the environment scan from the end of the previous test
as the basis for comparison during the next test.
2018-10-03 21:41:27 -07:00
e53390b3b1 Fix ansible-test multi-group smoke test handling. (#46363)
* Fix ansible-test smoke tests across groups.
* Fix ansible-test list arg defaults.
* Fix ansible-test require and exclude delegation.
* Fix detection of Windows specific changes.
* Add minimal Windows testing for Python 3.7.
2018-10-02 12:26:14 -07:00
54937ba784 Fix ansible-test docker python version handling.
This removes the old name based version detection behavior and
uses versions defined in the docker completion file instead, as
the new containers do not follow the old naming scheme.
2018-09-21 20:11:38 -07:00
ac492476e5 Bug fixes and cleanup for ansible-test. (#45991)
* Remove unused imports.
* Clean up ConfigParser usage in ansible-test.
* Fix bare except statements in ansible-test.
* Miscellaneous cleanup from PyCharm inspections.
* Enable pylint no-self-use for ansible-test.
* Remove obsolete pylint ignores for Python 3.7.
* Fix shellcheck issuers under newer shellcheck.
* Use newer path for ansible-test.
* Fix issues in code-smell tests.
2018-09-21 11:38:22 -07:00
27c10fa502 add skip/docker alias to skip tests when running on Docker (#45955)
* add skip/docker alias to skip tests when running on Docker

* changed warning message wording
2018-09-21 16:09:54 +10:00
abe8e4c9e8 Run unit tests in parallel. (#45812) 2018-09-18 13:58:22 -07:00
379a7f4f5a Fix ansible-test unit test execution. (#45772)
* Fix ansible-test units requirements install.
* Run unit tests as unprivileged user under Docker.
2018-09-18 08:37:14 -07:00
c1f9efabf4 Overhaul httptester support in ansible-test. (#39892)
- Works with the --remote option.
- Can be disabled with the --disable-httptester option.
- Change image with the --httptester option.
- Only load and run httptester for targets that require it.
2018-05-09 09:24:40 -07:00