CVE-2017-7481
Lookup returns wrap the result in unsafe, however when used through the
standard templar engine, this does not result in the jinja2 environment being
marked as unsafe as a whole. This means the lookup result looses the unsafe
protection and may become simple unicode strings, which can result in bad
things being re-templated.
This also adds a global lookup param and cfg options for lookups to allow
unsafe returns, so users can force the previous (insecure) behavior.
(cherry picked from commit 72dfb1570d22ac519350a8c09e76c458789120ed)
(cherry picked from commit fadccda7c7a2e8d0650f4dee8e3cea93cf17acfd)
* Use packaging.version.Version instead of LooseVersion for better support of prerelease tags (eg, 0.30.0 > 0.30.0rc6)
* Add explicit check/error for msrestazure package
(cherry picked from commit d12c93ef2b)
Previously, the Conditional class did a simple check when an
AnsibleUndefinedVariable error was raised to see if certain strings were
present. This patch tries to be smarter by evaluating the variable contained
in the error string and compared to the defined/not defined conditionals in
the conditional string.
This also modifies the UndefinedError message from HostVars slightly to
match the format returned jinja2 in general, making it easier to match the
error message in the Conditional code.
Fixes#18514
(cherry picked from commit 81aa12eb1b)
(cherry picked from commit cfd57fcae2)
This provides better error handling, and prevents errors like KeyError
from bubbliing up to code in odd places.
Fixes#17482
(cherry picked from commit 85bbce9d6b)
This forces basic auth to be used. Using the normal HTTPPasswordMgrWithDefaultRealm
password manager from urllib2 fails since collins doesn't send a 401 retry on failure.
More about this can be seen here http://stackoverflow.com/questions/2407126/python-urllib2-basic-auth-problem.
I added a small comment about the format of the host so others don't waste time like i did.
(cherry picked from commit 21813ed83e)
* Moved the _inventory.clear_group_dict_cache() from creating a group which doesn't exist, to adding members to the group.
* Update __init__.py
Update to use changed: block to catch all changes for cache clear as suggested
(cherry picked from commit b91d4d884d)
This allows meta refresh_inventory to work with relative paths
Added option to unfrackpath to not resolv symlinks
fixes#16857
(cherry picked from commit 8217c1c39c)
* Fix unbound method call for JSONEncoder
The way it is currently it will lead to unbound method error
```python
In [1]: import json
In [2]: json.JSONEncoder.default('object_here')
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-2-872fdacfda50> in <module>()
----> 1 json.JSONEncoder.default('object_here')
TypeError: unbound method default() must be called with JSONEncoder instance as first argument (got str instance instead)
```
But what is really wanted is to let the json module to raise the "is not serializable error" which demands a bounded instance of `JSONEncoder()`
```python
In [3]: json.JSONEncoder().default('object_here')
---------------------------------------------------------------------------
TypeError: 'object_here' is not JSON serializable
```
BTW: I think it would try to call `.to_json` of object before raising as it is a common pattern.
* Calling JSONEncoder bounded `default` method using super()
(cherry picked from commit b06fb2022c)
Searching the DEFAULT_ROLES_PATH and the roles basedir should come
before this, and it has been a long standing oversight.
Fixes#17882
(cherry picked from commit 0a86ddc251)
* Add tests for the mount module
* Switch from unmounted to absent...
the code for mounting always modifies fstab so we need to always modify
fstab to cleanup as well.
* Fix comments and copyright
6eefc11c converted task.loop_control into an object, but while the other
callers were updated to use .loop_var instead of .get('loop_var'), this
site was overlooked.
This can be reproduced by including with loop_control a file that does
set_fact; a simple regression test along these lines is included.
(cherry picked from commit 950cc26aab)
The cherry-pick allows module parameters to convert int to float, fixing
bugs where a float type is required. However, it used types from the
six library for python3 compatibility as well. Since we don't have
six in 2.1.0 this was buggy. Merging the unittests for the problem
here: adc158a499 made this buggy behaviour
apparent.
A parameter of type int should accept int and string, but not float.
A parameter of type float should accept float, int, and string.
Also reset the arguments in another test so that it runs cleanly. This
agrees with what all the other tests are doing.
When using hostvars to get extra connection-specific vars for connection
plugins, use this raw lookup to avoid prematurely templating all of the
hostvar data (triggering unnecessary lookups).
Fixes#17024
(cherry picked from commit ac5ddf4aa092e12f9e1c85c6b74aa30b7ef0a382)
Copying the TaskInclude task (which is the parent) before loading the blocks
makes the code much more simple and clean, and fixes a bug introduced during
the performance improvement changes (and specifically the change which moved
things to a single-parent model).
Fixes#17064
(cherry picked from commit f4237b2151)
* Pass the absolute path to dirname when assigning basedir
If no path is specified when calling the playbook, os.path.dirname(playbook_path) returns ''
This will cause failure when creating the retry file.
Fixes#17456
* Updated to use os.pathdirname(os.path.abspath())
When using the no_proxy option, the temp ca files that are created for
setting up an ssl context were not being deleted.
This small patch fixes that issue by removing the temp file.
Also added a deprecation notice for _fixup_perms.
Resolves issue #17352 (assumes custom actions use recursive=False).
(cherry picked from commit 94a0d2afb4)
* when including statically, make sure that all parents were also included
statically (issue #16990)
* properly resolve nested static include paths
* print a message when a file is statically included
Fixes#16990
(cherry picked from commit 1c7e0c73c9)
When a task result has an empty results list, the
list should be ignored when determining the results
of `_check_key`. Here the empty list is treated the
same as a non-existent list.
This fixes a bug that manifests itself with squashed
items - namely the task result contains the correct
value for the key, but an empty results list. The
empty results list was treated as zero failures
when deciding which handler to call - so the task
show as a success in the output, but is deemed to
have failed when deciding whether to continue.
This also demonstrates a mismatch between task
result processing and play iteration.
A test is also added for this case, but it would not
have caught the bug - because the bug is really in
the display, and not the success/failure of the
task (visually the test is more accurate).
Fixesansible/ansible-modules-core#4214
(cherry picked from commit eb2a3a91a8)
The calculation for max_fail_percentage was moved into the linear
strategy a while back, and works better there in the stategy layer
rather than at the PBE layer. This patch removes it from the PBE layer
and tweaks the logic controlling whether or not the next batch is run.
Fixes#15954
(cherry picked from commit 890e096b2b)
The flag new_pb_basedir is not being utilized in Inventory._get_hostgroup_vars,
leading to the situation where an inventory with no playbook basedir set will
read host/group vars from the $CWD, regardless of the inventory and/or playbook
relative location. This patch corrects that by not using the playbook basedir
if it is unset (None).
This patch also corrects a bug in which the VariableManager would accumulate
host/group vars files, which could lead to incorrect vars files being used when
playbooks are run from different directories containing their own group/host vars
directories.
Fixes#16953
(cherry picked from commit b617d62203)
We want to NOT consider the async task as failed if the result is
not parsed, which was the intent of:
https://github.com/ansible/ansible/pull/16458
However, the logic doesn't actually do that because we default
the 'parsed' value to True. It should default to False so that
we continue waiting, as intended.
(cherry picked from commit bf8c871801)
This is needed so that async_status can use ansible_python_interpreter
for the host being contacted.
Fixes#14101
(cherry picked from commit 38ccd11cce)
* switch cwd to basedir of task
This restores previous behaviour in pre 2.0 and allows for 'local type' plugins
and actions to have a more predictable relative path.
fixes#14489
* removed FIXME since prev commit 'fixes' this
* fix tests, now they need a loader (thanks jimi!)
(cherry picked from commit e2f17f8d9b)
Run setfacl/chown/chmod on each temp dir and file.
This fixes temp file permissions handling on platforms such as FreeBSD
which always return success when using find -exec. This is done by
eliminating the use of find when setting up temp files and directories.
(cherry picked from commit 72cca01cd4)
Due to the way we load plugins, internally to Python there can be issues when
the debug strategy is loaded after the linear strategy. To work around this,
we're changing the import line for the linear strategy to avoid the problem.
Related to #16825
(cherry picked from commit 1714279b5e)
uri:
follow_redirects: no
Will lead yaml to set follow_redirects=False. This is problematic when
the module parameter is not a boolean value but a string. For instance:
follow_redirects = dict(required=False, default='safe', choices=['all', 'safe', 'none', 'yes', 'no']),
Our parameter validation code ends up getting follow_redirects="False"
instead of "no". The 100% fix is for the user to quote their strings in
playbooks like:
uri:
follow_redirects: "no"
But we can fix quite a few common cases by trying to switch "False" back
into the string that it was specified as. We only do this if there is
only one correct choices value that could have been specified. In the
follow_redirects example, a value of "True" only maps back to "yes" and
a value of "False" only maps back to "no" so we can do this. If choices
also contained "on" and "off" then we couldn't map back safely and would
need to force the module author to change the module to handle this
case.
Fixes parts of the following PRs:
* https://github.com/ansible/ansible-modules-core/pull/4220
* https://github.com/ansible/ansible-modules-extras/pull/2593
(cherry picked from commit 6db6edfc4f)
* These can still race when multiple ansible processes are created at
the same time.
* Reverse order of expanduser and expandvars in unfrakpath(). So that
tildes in environment variables will be handled.
(cherry picked from commit 1ecf51d87e)
* Revert "There can be only one localhost"
This reverts commit 5f1bbb4fcd.
this broke several usages of localhost, see #16882, #16898 and #16886
* ensure there is only 1 localhost
fixes#16886, #16882 and #16898
- make sure localhost exists before returning it
- optimzed host caching
- ensure we always return a host object
(cherry picked from commit f7570f1dc4)
This removes the extra layer of quotes around values in the 'args' file.
These quotes were there before the pipes.quote() call was added, but
were not removed, resulting in too much quoting.
Problem: When setting the file permissions on the remote server for
unprivileged users ansible expects that a chown will fail for unprivileged
users. For some systems (e.g. HP-UX) this is not the case.
Solution: Change the order how ansible sets the remote permissions.
* If the remote_user sudo's to an unprivileged user then we attempt to
grant the unprivileged user access via file system acls.
* If granting file system acls fails we try to change the owner of the
file with chown which only works in case the remote_user is privileged
or the remote systems allows chown calls by unprivileged users (e.g.
HP-UX)
* If the chown fails we can set the file to be world readable so that
the second unprivileged user can read the file. Since this could allow
other users to get access to private information we only do this
ansible is configured with "allow_world_readable_tmpfiles" in the
ansible.cfg
Reinstates some functionality removed by commit 0ba9a6a but
makes the expensive operation optional (and default to off)
(cherry picked from commit 7dc09adaf4)
A simple import of cryptography can throw several types of errors. For example,
if `setuptools` is less than cryptography's minimum requirement of 11.3, then
this import of cryptography will throw a VersionConflict here. An earlier case
threw a DistributionNotFound exception.
An optional dependency should not stop ansible. If the error is more than
an ImportError, log a warning, so that errors can be fixed in ansible or
elsewhere.
This bug was introduced in 3ced6d3, where getting vars from a role
did not follow the dep chain. This was originally hidden by the fact
that we got vars twice (from the block and from the roles directly).
Fixes#16729
(cherry picked from commit d8a3feb976)
* Add a function to check for killed processes so that if any
threads are sigkilled or sigtermed, the entire playbook execution is aborted.
(cherry picked from commit 238c6461f6)
2e003adb added the ability for tasks using any_errors_fatal to fail
when there were unreachable hosts. However that patch used the running
unreachable hosts data rather than the results from the current task,
which causes failures when any run_once or BYPASS_HOST_LOOP task is hit
after an unreachable host causes a failure. This patch corrects that by
using the current set of results to determine if any hosts were
unreachable during the last task only.
Fixesansible/ansible-modules-core#4160
(cherry picked from commit 245ce9461d)
I suspect this problem was masked previously as older versions of pywinrm
where significantly slower, allowing more time for the windows installer service to complete.
* Don't treat parsing problems as async task timeout
If there is a problem reading/writing the status file that manifests as
not being able to parse the data, that doesn't mean the task timed out,
it means there was what was likely a tempoarary problem. Move on and
keep polling for success. The only things that should cause the async
status to not be parseable are bugs in the async_runner.
* Add comment explaining not bailing out of loop
* Return different error when result is unparseable
* Remove extraneous else
* Instead of rebuilding the handler list all over the place, we now
compile the handlers at the point the play is post-validated so that
the view of the play in the PlayIterator contains the definitive list
* Assign the dep_chain to the handlers as they're compiling, just as we
do for regular tasks
(cherry picked from commit 930d090507)
Since the rest of the file already use a non 2.4 syntax
(such as format), I didn't bother using the 2.4 syntax for
exceptions.
(cherry picked from commit eb52dc9af0)
We want to update host vars for all hosts (even those that might
have failed), and the in case of a refresh_inventory, the code has
a stale restrictions list at this point anyway.
(cherry picked from commit 0c4effb8a6)
This patch corrects the way we look in the inventory hosts list for
implicit localhost entries when localhost aliases are used.
Fixes#16568
(cherry picked from commit 83e4a4048b)
* Parse async response in async action.
* Add async test for non-JSON data before module output.
* Fix existing async unit test.
Resolves#16156
(cherry picked from commit 292785ff2b)
Tests now use '/shared' instead of '/tmp/shared-dir' when using
COPY_SOURCE. This avoids issues with containers purging '/tmp'.
(cherry picked from commit fbfadc47c7)
In the case of using YAML anchors/aliases, YAML actually uses references
to the duplicated object so any modifications to the original impacts
later uses of the object.
Fixes#13575
(cherry picked from commit af249b83e6)
* Lookup unencrypted password must not include salt
* Integration test lookup: remove previous directory
* Test that lookup password doesn't return salt
* Lookup password: test behavior with empty encrypt parameter
Closes#16189
(cherry picked from commit b361bf90d7)
* Remove unnecessary copying of values from parents to role deps, as
this can cause problems when roles have multiple parents (or the same
parents with different params speficied through deps)
* Since we're already checking the dep chain in the block for role
things (which every task in a role should have), it is not necessary
to check the role directly in case it improperly grabs something
Fixes#14438
(cherry picked from commit 3e4755f7e4)
Our custom encoder for the to_json filter was simply returning the
object if it was not a HostVars object, leading in some cases to a
TypeError when the data contained an undefined variable. This lead
to an odd error message being propagated up, so we now properly catch
this as an undefined variable error.
Fixes#15610
(cherry picked from commit c24c0f5f6b)
The find_mount_point function does not resolve the mount point of paths with a soft-link correctly and returns the wrong mount-point.
I have mounted an NFS filesystem on /nfs-mount. This directory contains a directory called "directory". I also created a soft-link to this last directory: /soft-link-to-directory -> /nfs-mount/directory. I created the following task to copy a file into /soft-link-to-directory:
- name: copy file to nfs-mount
copy:
src: "file"
dest: "/soft-link-to-directory/file"
This throws an exception:
invalid selinux context: [Errno 95] Operation not supported
This is caused by the find_mount_point function to return '/' as the mount point for '/soft-link-to-directory/file'. This should have been /nfs-mount. Because the find_mount_point returns the wrong mount-point, the is_special_selinux_path function does not recognise the file is on an NFS mount and tries to set the default SELinux context (system_u:object_r:default_t:s0), which fails. The context should have been: system_u:object_r:nfs_t:s0
Full Ansible output:
TASK [copy file to nfs-mount] **************************************************
fatal: [hostname]: FAILED! => {"changed": false, "checksum": "f34b60930a5d6d689cf49a4c16bd7f9806be608c", "cur_context": ["system_u", "object_r", "nfs_t", "s0"], "failed": true, "gid": 24170, "group": "foundation", "input_was": ["system_u", "object_r", "default_t", "s0"], "mode": "0644", "msg": "invalid selinux context: [Errno 95] Operation not supported", "new_context": ["system_u", "object_r", "default_t", "s0"], "owner": "root", "path": "/soft-link-to-directory/.ansible_tmpWCT6Z4file", "secontext": "system_u:object_r:nfs_t:s0", "size": 37, "state": "file", "uid": 0}
(cherry picked from commit 23349911f1)
Again, as we're carrying failed/unreachable hosts forward from play to play via
internal structures, we need to remember which ones had previously failed so that
unrelated host failures don't inflate the numbers for a given serial batch in the
PlaybookExecutor causing a premature exit.
Fixes#16364
(cherry picked from commit 4c1601e9f2)
When the PYTHONPATH is an empty string python will treat it as though
the cwd is in the PYTHONPATH. This can be undesirable. So make sure we
delete PYTHONPATH from the environment altgether in this case.
Fixes#16195
Due to the fact that roles may be instantiated with different sets of
params (multiple inclusions of the same role or via role dependencies),
simply tracking notified handlers by name does not work. This patch
changes the way we track handler notifications by using the handler
object itself instead of just the name, allowing for multiple internal
instances. Normally this would be bad, but we also modify the way we
search for handlers by first looking at the notifying tasks dependency
chain (ensuring that roles find their own handlers first) and then at
the main list of handlers, using the first match it finds.
This patch also modifies the way we setup the internal list of handlers,
which should allow us to correctly identify if a notified handler exists
more easily.
Fixes#15084
Symlinks inside of the chroot were failng because we weren't able to
determine if they were pointing to a real file or not. We could write
some complicated code to walk the symlink path taking into account where
the root of the tree is but that could be fragile. Since this is just
a sanity check, instead we just assume that the chroot is fine if we
find that /bin/sh in the chroot is a symlink. Can revisit if it turns
out that many chroots have a /bin/sh that's a broken symlink.
Fixes#16097
The junos network module will now properly use the ssh key file if its
passed from the playbook to authenticate to the remote device. Prior
to this commit, the ssh keyfile was ignored.
When setuptools installs a python module (as is done via python setup.py
install) It puts the module into a subdirectory of site-packages and
then creates an entry in easy-install.pth to load that directory. This
makes it difficult for Ansiballz to function correctly as the .pth file
overrides the sys.path that the wrapper constructs. Using
sitecustomize.py fixes this because sitecustomize overrides the
directories handled in .pth files.
Fixes#16187
* Fix: create retry_files_save_path if it doesn't exist
Ansible documentation states that retry_files_save_path directory will be
created if it does not already exist. It currently doesn't, so this patch
fixes it :)
* Use makedirs_safe to ensure thread-safe dir creation
@bcoca suggested to use the makedirs_safe helper function :)
(cherry picked from commit 828b73dd2d)
The changes to exclude implicit localhosts from group patterns exposed
the bug that we sometimes create multiple implicit localhosts, which
caused some bugs with things like includes, where the host was used as
an entry into a dict, so having multiple meant that the incorrect host
(with a different uuid) was found and includes were not executed for
implicit localhosts.
This allows the PlaybookExecutor to receive more information regarding
what happened internal to the TaskQueueManager and strategy, to determine
things like whether or not the play iteration should stop.
Fixes#15523
(cherry picked from commit fbec2d9692)
The nxos cli provider would not properly handle ssh key files passed
from the playbook task. The ssh_keyfile argument is now properly
passed to the ssh authentication method
This fix address the bug reported in #3862
Also updates doc on variable precedence, as it was incorrect for the
order of play vars/vars_prompt/vars_files in relation to set_fact and
registered variables.
Fixes#14702Fixes#14826
Since we now use the PlayIterator to carry forward failures from previous
play executions, in the event that some hosts which had previously failed
are not in the current inventory we now create a stub state instead of
raising an error.
Exception was raised when trying to use ssh-agent for authentication to
ios devices. This fix enables ssh-agent and enable use of password
protected ssh keys. There is one additional fix to capture authentication
exceptions nicely.
These tests were not part of CI at the time 2.1 was released.
They are not currently passing in stable-2.1, as changes necessary
for them to pass were made after the release.
* alias module is very basic and removing it leads to the suse default
config failing
* future improvements might test different modules and the effect of
them being removed
(cherry picked from commit cf62a62b83)
* reduce async sleep time in test
* make zypper test less destructive (don't break following uses of zypper)
* fix ca cert on suse
* fix/enable postgres/mysql on opensuseleap
* fix mysql test for mysql versions 5.7.6 and newer
* skip sni_host check on ubuntu1604
* add HTTPTESTER flag for test_uri
ubuntu 16.04 uses dash which drops env variables containing a dot
we work around this by adding an explicit env variable to enable httptester
(cherry picked from commit c06884eff0)
* Use httptester docker image for http tests
* When not running with an httptester linked container, use public test sites
(cherry picked from commit accf40d8a8)
* In the VariableManager, we were not properly tracking if a file
had already been loaded, so we continuously append data to the end
of the list there for host and group vars, meaning large sets of data
are duplicated multiple times
* In the inventory, we were merging the host/group vars with the vars
local to the host needlessly, as the VariableManager already handles that.
This leads to needless duplication of the data and makes combining the
vars in VariableManager take even longer.
Ansible excessively checks the file system for the potential presence of
`group_vars` and `host_vars` files.
For large numbers of groups this leads to combinatorial performance
issues.
This commit generates a set of group_vars and host_vars filenames using
`os.listdir()` in every possible location and then checks against the sets
before making a stat of the file system.
Also included in this commit is caching of the base directory lookup
for the inventory.
Has already been transferred as a tempfile.
This fixes the error in https://github.com/ansible/ansible/issues/16125
but there may be higher level issues that should be fixed as well (other
modules might be able to cause status fields like failed and changed to
return a censored string instead of a bool). So leaving 16125 open for
now.
Child blocks (whether nested or via includes) don't get a copy of the
dependency chain, so the above method should be used to ensure the block
looks at its parents dep chain.
Fixes#15996
* Catch DistributionNotFound when pycrypto is absent
On Solaris 11, module `pkg_resources` throws `DistributionNotFound` on import if `cryptography` is installed but `pycrypto` is not. This change causes that situation to be handled gracefully.
I'm not using Paramiko or Vault, so I my understanding is that I don't
need `pycrpto`. I could install `pycrypto` to make the error go away, but:
- The latest released version of `pycrypto` doesn't build cleanly on Solaris (https://github.com/dlitz/pycrypto/issues/184).
- Solaris includes an old version of GMP that triggers warnings every time Ansible runs (https://github.com/ansible/ansible/issues/6941). I notice that I can silence these warnings with `system_warnings` in `ansible.cfg`, but not installing `pycrypto` seems like a safer solution.
* Ignore only `pkg_resources.DistributionNotFound`, not other exceptions.
With some earlier changes, continuing to forward failed hosts on
to the iterator with each TQM run() call was causing plays with
max_fail_pct set to fail, as hosts which failed in previous plays
were counting those old failures against the % calculation.
Also changed the linear strategy's calculation to use the internal
failed list, rather than the iterator, as this now represents the
hosts failed during the current run only.
As noted in the comment, the TQM may be used for more than one play. As such,
after creating the new PlayIterator object it is necessary to mark any failed
hosts from previous calls to run() as failed in the iterator, so they are
properly skipped during any future calls to run().
By default the `Shell` class disables ssh agents. The `junos_netconf`
module uses this class, but doesn't re-enable agents.
Here it's explicitly enabled again, so an ssh agent can be used to
connect to and configure Junos devices.
Since this is now the default package manager, it got moved
to another location on Netbsd :
netbsd# type pkgin
pkgin is a tracked alias for /usr/pkg/bin/pkgin
netbsd# uname -a
NetBSD netbsd.example.org 6.1.4 NetBSD 6.1.4 (GENERIC) amd64
But since the package manager is also used outside of NetBSD, we
have to keep the /opt/local path too.
The change is needed to support the multiple include statements
inside the jinja2 template file, as in '{% include ['another.j2'] %}'.
statement. I need this capability, as OpenSwitch `switch` role needs
to handle multiple *.j2 files and supporting the include statement
inside jinja2 file is essential, otherwise I need to combine multiple
template files into a single file, which easily causes conflicts
between developers working on different parts of the teamplate, ports
and interface.
Prior to this patch, the retry/until logic would fail any task that
succeeded if it took all of the alloted retries to succeed. This patch
reworks the retry/until logic to make things more simple and clear.
Fixes#15697
When using run_once, there is only one dict of facts so passing that
to the VariableManager results in the fact cache containing the same
dictionary reference for all hosts in inventory. This patch fixes that
by making sure we pass a copy of the facts dict to VariableManager.
Fixes#14279
Previously the changed code was necessary, however it is now problematic
as we've started using the is_failed() method in other places in the code.
Additional changes at the strategy layer should make this safe to remove
now.
Fixes#15625
In VariableManager, we fetch the params specifically in the next step,
so including them in the prior step is unnecessary and could lead to things
being overridden in an improper order.
In Block, we should not be getting the params for the role as they are
included earlier via the VariableManager.
Fixes#14411
Fixes#15745
Applies conditional forwarding to all tasks/roles within the included playbook.
The existing line only applies forwarded conditionals to the main Task block, and misses pre_, post_, and roles.
Typo ::
Made a selection mistake when I copied over the one line change
* Update GCE module to use JSON credentials
* Ensure minimum libcloud version when using JSON crednetials for GCE
* Relax langauge around libcloud requirements
In the free strategy, we mark a host as blocked when it has work to do
(the PlayIterator returns a task) to prevent multiple tasks from being sent
to the host. However, we check for role duplicates after setting the blocked
flag, but were not clearing that when the task was skipped leading to an
infinite loop. This patch corrects that by clearing the blocked flag when
the task is skipped.
Fixes#15681
In `lib/ansible/executor/play_iterator.py`, ansible sets a host's
`_gathered_facts` property to `True` without checking to see if there
are any tasks to be executed. In the event that the entire play is
skipped, `_gathered_facts` will be `True` even though the `setup`
module was never run.
This patch modifies the logic to only set `_gathered_facts` to `True`
when there are tasks to execute.
Closes#15744.
Issue #15633 observes that a meta: inventory_refresh task causes the playbook
to exit. An inventory refresh flushes all caches and rebuilds all host
objects, assigning new UUIDs to each. These new host UUIDs currently fail to
match those on host objects stored for restrictions in the inventory, causing
the playbook to exit for having no hosts to run further tasks against.
This changeset attempts to address this issue by storing host restrictions
by name, and comparing inventory host names against these names when applying
restrictions in get_hosts.
* Don't filter hosts remaining based on their failed state. Instead rely
on the PlayIterator to return None/ITERATING_COMPLETE when the host is
failed.
* In the free strategy, make sure we wait outside the host loop for all
pending results to be processed.
* Use the internal _set_failed_state() instead of manually setting things
when a failed child state is hit
Fixes#15623
This changeset addresses the issue reported here:
ansible/ansible-modules-core#1765
The yum module (at least) includes its task results as strings, rather than
dicts, and the code this changeset replaces assumed that in that instance the
task was skipped. The updated behaviour assumes that the task has been
skipped only if:
* results exist, and
* all results are dicts that include a truthy skipped value
* When var name is the same as var content, try to template it before reporting that var is not defined.
Add asserts in test_var_blending to check this special corner case.
* Fix integration tests when using debug with list or dict.
It wasn't doing anything that a literal couldn't do and used
unicode_escape which only understands latin1 (The author of the code
seems to have thought it took an encoding argument but it looks like
that was silently ignored.)
This was reinitialized every time we forked before so we weren't sharing
the same Locks. It also was not accounting for modules which were
directly invoked by an action plugin instead of going through the
strategy plguins.
* Make ziploader's ansible and ansible.module_utils libraries into
namespace packages.
* Move __version__ and __author__ from ansible/__init__ to
ansible/release.py. This is because namespace packages only load one
__init__.py. If that is not the __init__.py with the author and
version info then those won't be available.
* In ziploader, move the version ito ANSIBLE_CONSTANTS.
* Change PluginLoader to properly construct the path to the plugins even
when namespace packages are present.
* extend zypper integration tests
* fix caching issue of local RPMS on openSUSE 42.1 Leap
* add tests for simultaneous install/remove via prefixes +-
* test fail cases (rm patch or URL)
* test patch install (succes, unchanged second run, fail on wrong name)
* add test for pattern install
Hi! Thanks for your interest in contributing to Ansible.
Hi! Thanks for interest in contributing to Ansible.
Here are some guidelines for contributing code. The purpose of this document is to establish what we're looking for in code contributions, and to make sure
new contributors know some of the conventions that we've been using.
Here are some guidelines for contributing code. The purpose of this document are to establish what we're looking for in code contributions, and to make sure
new contributions know some of the conventions that we've been using.
We don't think much of this should be too strange to readers familiar with contributing to Python projects, though it helps if we all get on the same page.
@ -12,27 +12,27 @@ Language
========
* While not all components of Ansible must be in Python, core contributions to the Ansible repo must be written in Python. This is to maximize the ability of everyone to contribute.
* If you want to write non-Python ansible modules or inventory scripts, that's fine, but they are not going to get merged in most likely. Sorry!!
* If you want to write non-Python ansible modules or inventory scripts, that's fine, but they are not going to get merged in most likely. Sorry!!
PEP8 and basic style checks
============================
PEP8 and basic style checks
===========================
*[PEP 8](https://www.python.org/dev/peps/pep-0008/) is a great Python style guide, which you should read.
* PEP8 must not be strictly followed in all aspects, but most of it is good advice.
*The line is limited to 160 characters.
* To run checks for things we care about, use [ansible-test](https://docs.ansible.com/ansible/devel/dev_guide/testing_sanity.html).
* Similarly, additional checks can be made with "make pyflakes".
* There is no need to submit code changes for PEP 8 and pyflakes fixes, as these break attribution history. Project leadership will make these periodically.
* Do not submit pull requests that simply adjust whitespace in the code.
* PEP8 is a great Python style guide, which you should read.
* PEP8 must not be strictly followed in all aspects, but most of it is good advice
*In particular, we don't really care about line lengths. Buy a bigger monitor!
* To run checks for things we care about, run "make pep8"
* Similarly, additional checks can be made with "make pyflakes"
* There is no need to submit code changes for pep8 and pyflake fixes, as these break attribution history. Project leadership will make these periodically.
* Do not submit pull requests that simply adjust whitespace in the code
Testing
=======
* Much of Ansible's testing needs are in integration, not unit tests. Add module tests there.
* Much of ansible's testing needs are in integration, not unit tests. Add module tests there.
* That being said, there are unit tests too!
* Code written must absolutely pass tests (i.e. "make tests")
* You should anticipate any error paths in your code and test down those error paths.
* Additions to tests for core code are welcome, but not always possible. Be sure things are at least well tested manually in that case.
* Additions to tests for core code is welcome, but not always possible. Be sure things are at least well tested manually in that case.
Whitespace
==========
@ -42,7 +42,7 @@ Whitespace
Shebang Lines
=============
* /usr/bin/scripts should start with '/usr/bin/env python'
* module code should still use '/usr/bin/python' as this is replaced automatically by 'ansible_python_interpreter', see the FAQ in the docs for more info.
@ -60,13 +60,13 @@ Classes
* With the exception of module code (where inline is better), it is desirable to see classes in their own files.
* Classes should generally not cause side effects as soon as they are instantiated, move meaningful behavior to methods rather than constructors.
Functions and Methods
=====================
* In general, functions should not be 'too long' and should describe a meaningful amount of work
* When code gets too nested, that's usually the sign the loop body could benefit from being a function
* Parts of our existing code are not the best examples of this at times.
* Parts of our existing code are not the best examples of this at times.
* Functions should have names that describe what they do, along with docstrings
* Functions should be named with_underscores
* "Don't repeat yourself" is generally a good philosophy
@ -74,7 +74,7 @@ Functions and Methods
Variables
=========
* Use descriptive variable names instead of variables like 'x', unless x is an obvious loop index
* Use descriptive variable names instead of variables like 'x', unless x is a obvious loop index
* Ansible python code uses identifiers like 'ClassesLikeThis and variables_like_this
* Module parameters should also use_underscores and not runtogether
@ -83,7 +83,7 @@ Module Security
* Modules must take steps to avoid passing user input from the shell and always check return codes
* always use module.run_command instead of subprocess or Popen or os.system -- this is mandatory
* if you need to use the shell you must pass use_unsafe_shell=True to module.run_command
* if you use need the shell you must pass use_unsafe_shell=True to module.run_command
* if you do not need the shell, avoid using the shell
* any variables that can come from the user input with use_unsafe_shell=True must be wrapped by pipes.quote(x)
* downloads of https:// resource urls must import module_utils.urls and use the fetch_url method
@ -101,22 +101,22 @@ Use the dict constructor where possible when allocating dictionaries:
# this:
foo = dict(
a=12,
b=34,
a = 12,
b = 34
)
Do not line up variables
Line up variables
# not this
a = 12
foosball = 34
xyz = 'dog'
# this
a = 12
foosball = 34
xyz = 'dog'
# this
a = 12
foosball = 34
xyz = 'dog'
Don't use line continuations:
# no
@ -131,13 +131,13 @@ Don't use line continuations:
Spacing:
# no
x = [1,2,3]
x=[1,2,3]
# no
x = [1,2, 3 ]
x = [1,2,3]
# yes
x = [1, 2, 3]
x = [1, 2, 3]
Spacing continued:
@ -161,24 +161,24 @@ All contributions to the core repo should preserve original licenses and new con
Module Documentation
====================
All module pull requests must include a DOCUMENTATION docstring (YAML format,
see other modules for examples) as well as an EXAMPLES docstring, which is free form.
All module pull requests must include a DOCUMENTATION docstring (YAML format,
see other modules for examples) as well as an EXAMPLES docstring, which is free form.
When adding new modules, any new parameter must have a "version_added" attribute.
When submitting a new module, the module should have a "version_added" attribute in the
When adding new modules, any new parameter must have a "version_added" attribute.
When submitting a new module, the module should have a "version_added" attribute in the
pull request as well, set to the current development version.
Be sure to check grammar and spelling.
It's frequently the case that modules get submitted with YAML that isn't valid,
so you can run "make webdocs" from the checkout to preview your module's documentation.
If it fails to build, take a look at your DOCUMENTATION string
It's frequently the case that modules get submitted with YAML that isn't valid,
so you can run "make webdocs" from the checkout to preview your module's documentation.
If it fails to build, take a look at your DOCUMENTATION string
or you might have a Python syntax error in there too.
Python Imports
==============
To make it clear what a module is importing, imports should not be sprinkled throughout the code.
To make it clear what a module is importing, imports should not be sprinkled throughout the code.
Python Imports should happen at the top of the file, exempting code from module_utils.
@ -203,7 +203,7 @@ Exceptions
==========
In the main body of the code, use typed exceptions where possible:
# not this
raise Exception("panic!")
@ -265,7 +265,7 @@ Use 'in':
# not this:
if x.find('foo') != -1:
# this:
# this:
if 'foo' in x:
String checks
@ -277,8 +277,7 @@ To test if something is a string, consider that it may be unicode.
if type(x) == str:
# yes
from ansible.module_utils.six import string_types
if isinstance(x, string_types):
if isinstance(x, basestring):
Cleverness
==========
@ -290,19 +289,19 @@ In particular, metaclasses are probably not appropriate, however entertaining th
Git Practices
=============
Pull requests cannot be accepted if they contain merge commits.
Pull requests cannot be accepted that contain merge commits.
Always do "git pull --rebase" and "git rebase" vs "git pull" or "git merge". See [rebasing a pull request](https://docs.ansible.com/ansible/latest/dev_guide/developing_rebasing.html) for more information.
Always create a new branch for each pull request to avoid intermingling different features or fixes on the same branch.
Always do "git pull --rebase" and "git rebase" vs "git pull" or "git merge".
Always create a new branch for each pull request to avoid intertwingling different features or fixes on the same branch.
Python Version Compliance
=========================
All code in Ansible core must support a minimum version of Python 2.6.
Module code must support a minimum of Python 2.4, with occasional exceptions for modules that require code that themselves require 2.6 and later.
Module code must support a minimum of Python 2.4, with occasional exception for modules that require code that themselves require 2.6 and later.
A quick reminder is that list comprehensions in Python 2.4 are not as fully fleshed out, there are no 'dict' comprehensions, and there is no 'with' statement.
But otherwise it's pretty much all the same.
@ -314,3 +313,7 @@ This was not meant to be a scary document, so we hope it wasn't, but we also hop
If you have questions about this document, please ask on the ansible-devel mailing list.
Please see the [community guide](https://docs.ansible.com/ansible/latest/community/index.html) for information on how to ask questions on the [mailing lists](https://docs.ansible.com/ansible/latest/community/communication.html#mailing-list-information) and IRC.
Please see the [community page](http://docs.ansible.com/community.html) for information on how to ask questions on the [mailing lists](http://docs.ansible.com/community.html#mailing-list-information) and IRC.
The GitHub issue tracker is not the best place for questions for various reasons, but both IRC and the mailing list are very helpful places for those things, as the community page explains best.
## CONTRIBUTING ?
By contributing you agree that these contributions are your own (or approved by your employer) and you grant a full, complete, irrevocable copyright license to all users and developers of the project, present and future, pursuant to the license of the project.
Please see the [community page](http://docs.ansible.com/community.html) for information regarding the contribution process. Important license agreement information is also included on that page.
Please see the [community guide](https://docs.ansible.com/ansible/latest/community/index.html) for information regarding the contribution process. Important license agreement information is also included on that page.
## BUG TO REPORT ?
First and foremost, also check the [community guide](https://docs.ansible.com/ansible/latest/community/index.html).
First and foremost, also check the [community page](http://docs.ansible.com/community.html).
You can report bugs or make enhancement requests at the [Ansible GitHub issue page](http://github.com/ansible/ansible/issues/new/choose) by filling out the issue template that will be presented.
You can report bugs or make enhancement requests at the [Ansible GitHub issue page](http://github.com/ansible/ansible/issues/new) by filling out the issue template that will be presented.
Also please make sure you are testing on the latest released version of Ansible or the development branch. You can find the latest releases and development branch at:
The Ansible Module Maintainer Guidelines can now be found in the [official Ansible community guide](https://docs.ansible.com/ansible/latest/community/maintainers.html).
tar -C deb-build/$${DIST} -xvf dist/$(NAME)-$(VERSION).tar.gz ;\
cp -a packaging/debian deb-build/$${DIST}/$(NAME)-$(VERSION)/ ;\
sed -ie "s|%VERSION%|$(DEB_VERSION)|g;s|%RELEASE%|$(DEB_RELEASE)|;s|%DIST%|$${DIST}|g;s|%DATE%|$(DEB_DATE)|g" deb-build/$${DIST}/$(NAME)-$(VERSION)/debian/changelog ;\
sed -ie "s|%VERSION%|$(VERSION)|g;s|%RELEASE%|$(DEB_RELEASE)|;s|%DIST%|$${DIST}|g;s|%DATE%|$(DEB_DATE)|g" deb-build/$${DIST}/$(NAME)-$(VERSION)/debian/changelog ;\
Ansible is a radically simple IT automation system. It handles configuration-management, application deployment, cloud provisioning, ad-hoc task-execution, and multinode orchestration - including trivializing things like zero downtime rolling updates with load balancers.
Read the documentation and more at http://ansible.com/
Many users run straight from the development branch (it's generally fine to do so), but you might also wish to consume a release.
You can find instructions [here](http://docs.ansible.com/intro_getting_started.html) for a variety of platforms. If you decide to go with the development branch, be sure to run `git submodule update --init --recursive` after doing a checkout.
If you want to download a tarball of a release, go to [releases.ansible.com](http://releases.ansible.com/ansible), though most users use `yum` (using the EPEL instructions linked above), `apt` (using the PPA instructions linked above), or `pip install ansible`.
Design Principles
=================
* Have a dead simple setup process and a minimal learning curve
* Manage machines very quickly and in parallel
* Avoid custom-agents and additional open ports, be agentless by leveraging the existing SSH daemon
* Describe infrastructure in a language that is both machine and human friendly
* Focus on security and easy auditability/review/rewriting of content
* Manage new remote machines instantly, without bootstrapping any software
* Allow module development in any dynamic language, not just Python
* Be usable as non-root
* Be the easiest IT automation system to use, ever.
Get Involved
============
* Read [Community Information](http://docs.ansible.com/community.html) for all kinds of ways to contribute to and interact with the project, including mailing list information and how to submit bug reports and code to Ansible.
* All code submissions are done through pull requests. Take care to make sure no merge commits are in the submission, and use `git rebase` vs `git merge` for this reason. If submitting a large code change (other than modules), it's probably a good idea to join ansible-devel and talk about what you would like to do or add first and to avoid duplicate efforts. This not only helps everyone know what's going on, it also helps save time and effort if we decide some changes are needed.
* Development list: [ansible-devel](http://groups.google.com/group/ansible-devel)
* Announcement list: [ansible-announce](http://groups.google.com/group/ansible-announce) - read only
* irc.freenode.net: #ansible
Branch Info
===========
* Releases are named after Led Zeppelin songs. (Releases prior to 2.0 were named after Van Halen songs.)
* The devel branch corresponds to the release actively under development.
* As of 1.8, modules are kept in different repos, you'll want to follow [core](https://github.com/ansible/ansible-modules-core) and [extras](https://github.com/ansible/ansible-modules-extras)
* Various release-X.Y branches exist for previous releases.
* We'd love to have your contributions, read [Community Information](http://docs.ansible.com/community.html) for notes on how to get started.
Authors
=======
Ansible was created by [Michael DeHaan](https://github.com/mpdehaan) (michael.dehaan/gmail/com) and has contributions from over 1000 users (and growing). Thanks everyone!
Ansible is sponsored by [Ansible, Inc](http://ansible.com)
This document is now the location for published Ansible Core roadmaps.
The roadmap will be updated by version. Based on team and community feedback, an initial roadmap will be published for a major or minor version (2.0, 2.1). Subminor versions will generally not have roadmaps published.
This is the first time Ansible has published this and asked for feedback in this manner. So feedback on the roadmap and the new process is quite welcome. The team is aiming for further transparency and better inclusion of both community desires and submissions.
These roadmaps are the team's *best guess* roadmaps based on the Ansible team's experience and are also based on requests and feedback from the community. There are things that may not make it on due to time constraints, lack of community maintainers, etc. And there may be things that got missed, so each roadmap is published both as an idea of what is upcoming in Ansible, and as a medium for seeking further feedback from the community. Here are the good places for you to submit feedback:
* Ansible's google-group: ansible-devel
* Ansible Fest conferences.
* IRC freenode channel: #ansible-devel (this one may have things lost in lots of conversation, so a caution).
2.1 Roadmap, Targeted for the End of April
==========
## Windows, General
* Figuring out privilege escalation (runas w/ username/password)
* Implement kerberos encryption over http
* pywinrm conversion to requests (Some mess here on pywinrm/requests. will need docs etc.)
* NTLM support
## Modules
* Windows
* Finish cleaning up tests and support for post-beta release
* Strict mode cleanup (one module in core)
* Domain user/group management
* Finish win\_host and win\_rm in the domain/workgroup modules.
* Close 2 existing PRs (These were deemed insufficient)
* Replicate python module API in PS/C# (deprecate hodgepodge of stuff from module_utils/powershell.ps1)
* Network
* Cisco modules (ios, iosxr, nxos, iosxe)
* Arista modules (eos)
* Juniper modules (junos)
* OpenSwitch
* Cumulus
* Dell (os10) - At risk
* Netconf shared module
* Hooks for supporting Tower credentials
* VMware (This one is a little at risk due to staffing. We're investigating some community maintainers and shifting some people at Ansible around, but it is a VERY high priority).
* vsphere\_guest brought to parity with other vmware modules (vs Viasat and 'whereismyjetpack' provided modules)
* VMware modules moved to official pyvmomi bindings
* VMware inventory script updates for pyvmomi, adding tagging support
* Azure (Notes: We've made progress here now that Microsoft has swaped out the code generator on the Azure Python SDK. We have basic modules working against all of these resources at this time. Could ship it against current SDK, but may break. Or should the version be pinned?)
* Minimal Azure coverage using new ARM api
* Resource Group
* Virtual Network
* Subnet
* Public IP
* Network Interface
* Storage Account
* Security Group
* Virtual Machine
* Update of inventory script to use new API, adding tagging support
* Docker:
* Start Docker module refactor
* Update to match current docker CLI capabilities
* Docker exec support
* Upgrade other cloud modules or work with community maintainers to upgrade. (In order)
* AWS (Community maintainers)
* Openstack (Community maintainers)
* Google (Google/Community)
* Digital Ocean (Community)
* Ziploader:
* Write code to create the zipfile that gets passed across the wire to be run on the remote python
* Port most of the functionality in module\_utils to be usage in ziploader instead
* Port a few essential modules to use ziploader instead of module-replacer as proof of concept
* New modules will be able to use ziploader. Old modules will need to be ported in future releases (Some modules will not need porting but others will)
* Better testing of modules, caching of modules clientside(Have not yet arrived at an architecture for this that we like), better code sharing between ansible/ansible and modules
* ziploader is a helpful building block for: python3 porting(high priority), better code sharing between modules(medium priority)
* ziploader is a good idea before: enabling users to have custom module_utils directories
* Expand module diff support (already in progress in devel)
* Framework done. Need to add to modules, test etc.
* Coordinate with community to update their modules
* Things being kicking down the road that we said we’d do
* NOT remerging core with ansible/ansible this release cycle
* Community stuff
* Define the process/ETA for reviewing PR’s from community
* Publish better docs and how-tos for submitting code/features/fixes
from ansible.errors import AnsibleError, AnsibleOptionsError, AnsibleParserError
from ansible.module_utils._text import to_text
# Used for determining if the system is running a new enough python version
# and should only restrict on our documented minimum versions
_PY3_MIN = sys.version_info[:2] >= (3, 5)
_PY2_MIN = (2, 6) <= sys.version_info[:2] < (3,)
_PY_MIN = _PY3_MIN or _PY2_MIN
if not _PY_MIN:
raise SystemExit('ERROR: Ansible requires a minimum of Python2 version 2.6 or Python3 version 3.5. Current version: %s' % ''.join(sys.version.splitlines()))
from ansible.utils.display import Display
from ansible.utils.unicode import to_unicode
########################################
### OUTPUT OF LAST RESORT ###
class LastResort(object):
# OUTPUT OF LAST RESORT
def display(self, msg, log_only=None):
def display(self, msg):
print(msg, file=sys.stderr)
def error(self, msg, wrap_text=None):
print(msg, file=sys.stderr)
########################################
if __name__ == '__main__':
display = LastResort()
try: # bad ANSIBLE_CONFIG or config options can force ugly stacktrace
import ansible.constants as C
from ansible.utils.display import Display
except AnsibleOptionsError as e:
display.error(to_text(e), wrap_text=False)
sys.exit(5)
cli = None
me = os.path.basename(sys.argv[0])
@ -78,51 +70,33 @@ if __name__ == '__main__':
display.debug("starting run")
sub = None
target = me.split('-')
if target[-1][0].isdigit():
# Remove any version or python version info as downstreams
- win_nssm - Fix several issues and idempotency problems (https://github.com/ansible/ansible/pull/44755)
- win_nssm - Fix service not started when ``state=started`` (https://github.com/ansible/ansible/issues/35442)
- win_nssm - Correctly escape argument line when a parameter contains spaces, quotes or backslashes
- win_nssm - Add missing space between parameters with ``app_parameters``
- win_nssm - Fix extra space added in argument line with ``app_parameters`` or ``app_parameters_free_form`` when a parameter start by a dash and is followed by a period (https://github.com/ansible/ansible/issues/44079)
- win_nssm - Fix error when several services were given to the ``dependencies`` option
minor_changes:
- win_nssm - Drop support of literal YAML dictionnary for ``app_parameters`` option. Use the ``key=value;`` string form instead
- "ACME modules support `POST-as-GET <https://community.letsencrypt.org/t/acme-v2-scheduled-deprecation-of-unauthenticated-resource-gets/74380>`__ and will be able to access Let's Encrypt ACME v2 endpoint after November 1st, 2019."
- "docker_container - fix ``ipc_mode`` and ``pid_mode`` idempotency if the ``host:<container-name>`` form is used (as opposed to ``host:<container-id>``)."
- "This reverts some changes from commit 723daf3. If a line is found in the file, exactly or via regexp matching, it must not be added again. `insertafter`/`insertbefore` options are used only when a line is to be inserted, to specify where it must be added."
- Ansible JSON Decoder - Switch from decode to object_hook to support nested use of __ansible_vault and __ansible_unsafe (https://github.com/ansible/ansible/pull/45514)
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
