Commit Graph

401 Commits (87928ff56bd3471460d248e87960616e48cbdcdb)

Author SHA1 Message Date
James Cammarata 4c1601e9f2 Take previously failed/unreachable hosts into account when checking the batch
Again, as we're carrying failed/unreachable hosts forward from play to play via
internal structures, we need to remember which ones had previously failed so that
unrelated host failures don't inflate the numbers for a given serial batch in the
PlaybookExecutor causing a premature exit.

Fixes #16364
2016-06-21 16:47:38 -05:00
James Cammarata 15e648dd94 Fix handler listeners as a list
The listen statement on handlers should have supported a list, however
it was broken in the revision of the pub/sub feature based on the handler
revamp. This patch corrects the bug, so this works again:

- name: some handler
  ...
  listen:
  - some target
  - another target

Fixes #16378
2016-06-21 10:00:10 -05:00
James Cammarata ca6ee4c789 FEATURE: handler listeners
Fixes ansible/proposals#8
2016-06-20 09:30:20 -05:00
James Cammarata 8218591fec Track notified handlers by object rather than simply their name
Due to the fact that roles may be instantiated with different sets of
params (multiple inclusions of the same role or via role dependencies),
simply tracking notified handlers by name does not work. This patch
changes the way we track handler notifications by using the handler
object itself instead of just the name, allowing for multiple internal
instances. Normally this would be bad, but we also modify the way we
search for handlers by first looking at the notifying tasks dependency
chain (ensuring that roles find their own handlers first) and then at
the main list of handlers, using the first match it finds.

This patch also modifies the way we setup the internal list of handlers,
which should allow us to correctly identify if a notified handler exists
more easily.

Fixes #15084
2016-06-20 09:30:20 -05:00
Toshio Kuratomi facea1ff0c Remove unneeded pass 2016-06-13 09:43:51 -07:00
Toshio Kuratomi 94dc4554db Workaround bad interaction with .pth files. (#16238)
When setuptools installs a python module (as is done via python setup.py
install)  It puts the module into a subdirectory of site-packages and
then creates an entry in easy-install.pth to load that directory.  This
makes it difficult for Ansiballz to function correctly as the .pth file
overrides the sys.path that the wrapper constructs.  Using
sitecustomize.py fixes this because sitecustomize overrides the
directories handled in .pth files.

Fixes #16187
2016-06-13 07:56:18 -07:00
Brian Coca 734bbcb1d3 better handling of retry files 2016-06-10 14:03:22 -04:00
nitzmahone ece1ed09d5 fix for psuedo-connection hostvars not propagating to connection
(mostly done by jimi-c, tested working)
2016-06-10 10:14:41 -07:00
J. Javier Maestro 828b73dd2d Fix: create retry_files_save_path if it doesn't exist (#15659)
* Fix: create retry_files_save_path if it doesn't exist

Ansible documentation states that retry_files_save_path directory will be
created if it does not already exist. It currently doesn't, so this patch
fixes it :)

* Use makedirs_safe to ensure thread-safe dir creation

@bcoca suggested to use the makedirs_safe helper function :)
2016-06-10 09:30:06 -04:00
James Cammarata fbec2d9692 Expand return code values returned by TQM and strategies
This allows the PlaybookExecutor to receive more information regarding
what happened internal to the TaskQueueManager and strategy, to determine
things like whether or not the play iteration should stop.

Fixes #15523
2016-06-08 10:21:57 -05:00
James Cammarata fa368934bd Create state in PlayIterator for unknown hosts rather than raise errors
Since we now use the PlayIterator to carry forward failures from previous
play executions, in the event that some hosts which had previously failed
are not in the current inventory we now create a stub state instead of
raising an error.
2016-06-07 09:42:39 -05:00
James Cammarata 1f18f5c92d Tweak the poll time to 0.005 to improve perf with large playbooks 2016-06-06 11:24:32 -05:00
James Cammarata b9e743b478 Merge branch 'devel' of https://github.com/andreyshestakov/ansible into andreyshestakov-devel 2016-06-06 11:03:28 -05:00
Toshio Kuratomi 5a3493be5f Port urls.py to python3 and other byte vs text fixes (#16124)
* Port urls.py to python3

Fixes (largely normalizing byte vs text strings) for python3

* Rework what we do with attributes that aren't set already.

* Comments
2016-06-04 16:19:57 -07:00
Andrey Shestakov 295c7ce1c8 Fix sleep intervals (14219) 2016-05-31 09:33:58 +03:00
Toshio Kuratomi b75895df9b Fix the error handling for loop squashing to restore the name argument into the task args 2016-05-21 07:07:26 -07:00
James Cammarata 74a252d0a5 More playbook executor compatibility fixes
With some earlier changes, continuing to forward failed hosts on
to the iterator with each TQM run() call was causing plays with
max_fail_pct set to fail, as hosts which failed in previous plays
were counting those old failures against the % calculation.

Also changed the linear strategy's calculation to use the internal
failed list, rather than the iterator, as this now represents the
hosts failed during the current run only.
2016-05-18 14:09:21 -04:00
James Cammarata f0e26d0b76 In TQM run() mark any entries in _failed_hosts as failed in the PlayIterator
As noted in the comment, the TQM may be used for more than one play. As such,
after creating the new PlayIterator object it is necessary to mark any failed
hosts from previous calls to run() as failed in the iterator, so they are
properly skipped during any future calls to run().
2016-05-18 08:17:39 -04:00
Vincent Roy 7a1309c98a Make sure that serial failures are handled correctly. 2016-05-18 08:17:39 -04:00
Vincent Roy e193e80f04 Don't stop executing plays after failure.
https://github.com/ansible/ansible/pull/13750/files
2016-05-18 08:17:39 -04:00
Vincent Roy ba89b2a095 Backward compatibility execution failures with 1.9 2016-05-18 08:17:39 -04:00
James Cammarata 6f6456dff5 Adding a deprecation message for accelerated mode 2016-05-13 17:01:07 -04:00
James Cammarata 4f0be29d65 Reworking retry/until logic to fix bugs
Prior to this patch, the retry/until logic would fail any task that
succeeded if it took all of the alloted retries to succeed. This patch
reworks the retry/until logic to make things more simple and clear.

Fixes #15697
2016-05-13 16:09:38 -04:00
Toshio Kuratomi 186337db28 Ship constants to the modules via internal module params rather than a secondary dict. 2016-05-12 20:30:05 -07:00
Matt Martz 196453b9b2 Merge pull request #13771 from sivel/binary-modules
First pass at allowing binary modules
2016-05-12 18:36:34 -05:00
Toshio Kuratomi b4e78caf25 Merge pull request #15825 from abadger/fix-squashing-traceback
If we can't squash for any reason, then simply do not optimize the items loop.
2016-05-12 10:59:48 -07:00
Matt Martz ca22783086 modify_module does not need to return module_path, as the calling code already has access to it 2016-05-12 12:53:49 -05:00
Matt Martz 34adb54734 Make _is_binary use already read module_data, move _is_binary check to the top of the stack 2016-05-12 12:53:49 -05:00
Matt Martz 0faddfa168 Move binary module detection into executor/module_common.py 2016-05-12 12:25:09 -05:00
Toshio Kuratomi 292f0ed0d6 If we can't squash for any reason, then simply do not optimize the items loop.
Also add more squashing testcases

Fixes #15649
2016-05-12 10:06:18 -07:00
James Cammarata 08f6283c3f Merge pull request #15747 from larsks/bug/15744
do not erroneously set gathered_facts=True
2016-05-12 10:22:45 -04:00
James Cammarata 0d7530e6da Change error about loop variable in use to a warning 2016-05-11 21:50:12 -04:00
James Cammarata 9d9a451b34 Simply being in an always block shouldn't mean a host did not fail
Previously the changed code was necessary, however it is now problematic
as we've started using the is_failed() method in other places in the code.
Additional changes at the strategy layer should make this safe to remove
now.

Fixes #15625
2016-05-11 19:13:12 -04:00
James Cammarata 09c90f7f2f Fixing bugs in strategies
* Don't filter hosts remaining based on their failed state. Instead rely
  on the PlayIterator to return None/ITERATING_COMPLETE when the host is
  failed.
* In the free strategy, make sure we wait outside the host loop for all
  pending results to be processed.
* Use the internal _set_failed_state() instead of manually setting things
  when a failed child state is hit

Fixes #15623
2016-05-06 09:22:11 -04:00
Lars Kellogg-Stedman 21ac95402f do not erroneously set gathered_facts=True
In `lib/ansible/executor/play_iterator.py`, ansible sets a host's
`_gathered_facts` property to `True` without checking to see if there
are any tasks to be executed.  In the event that the entire play is
skipped, `_gathered_facts` will be `True` even though the `setup`
module was never run.

This patch modifies the logic to only set `_gathered_facts` to `True`
when there are tasks to execute.

Closes #15744.
2016-05-05 23:45:06 -04:00
nitzmahone 133395db30 add jimi-c's unit test for squashed skip results, tweaked is_skipped() logic to pass 2016-05-05 15:29:10 -07:00
Andrew Taumoefolau 85868e07a9 Don't assume a task with non-dict loop results has been skipped.
This changeset addresses the issue reported here:

  ansible/ansible-modules-core#1765

The yum module (at least) includes its task results as strings, rather than
dicts, and the code this changeset replaces assumed that in that instance the
task was skipped. The updated behaviour assumes that the task has been
skipped only if:

* results exist, and
* all results are dicts that include a truthy skipped value
2016-05-05 15:29:10 -07:00
Brian Coca fb7940fc50 check that variable first
before using string methods to check for magic interpreter var
2016-05-05 11:14:11 -04:00
Toshio Kuratomi 98feafb411 Fix the mapping of module_name to Locks
This was reinitialized every time we forked before so we weren't sharing
the same Locks.  It also was not accounting for modules which were
directly invoked by an action plugin instead of going through the
strategy plguins.
2016-04-29 20:47:51 -07:00
Toshio Kuratomi 487e6562ca Fix ziploader for the cornercase of ansible invoking ansible.
* Make ziploader's ansible and ansible.module_utils libraries into
  namespace packages.
* Move __version__ and __author__ from ansible/__init__ to
  ansible/release.py.  This is because namespace packages only load one
  __init__.py.  If that is not the __init__.py with the author and
  version info then those won't be available.
* In ziplaoder, move the version ito ANSIBLE_CONSTANTS.
* Change PluginLoader to properly construct the path to the plugins even
  when namespace packages are present.
2016-04-29 08:47:49 -07:00
Toshio Kuratomi 01cd4fd06f Handle "/usr/bin/env python" style ansible_*interpreter settings.
Fixes #15635
2016-04-27 20:22:31 -07:00
Toshio Kuratomi 2fd7f194c8 Convert byte strings to unicode strings in our task results (#15581)
Fixes #15367
2016-04-25 12:35:25 -04:00
James Cammarata 1211a0fa12 Fixing the way we iterate over child states for tasks
Previously we were first checking the fail/run state of the child
state for tasks/rescue/always portions of the block. Instead we are now
always recursively iterating over the child state and then evaluating
whether the child state is failed or complete before changing the failed/
run state within the current block.

Fixes #14324
2016-04-25 11:13:44 -04:00
Toshio Kuratomi bdd73e31dc Have test-module clean up the local temp dir when it exits
Get test-module's debugger switch to do something useful with ziploader modules
2016-04-24 20:44:42 -07:00
Toshio Kuratomi 3ffd55ce7f Don't include "ansible" in the module output as there are a few files in the upper directory (args and the module) 2016-04-24 20:42:14 -07:00
Toshio Kuratomi 3c135ef3f2 Push debug files into a subdirectory to keep things cleaner.
We now have the ansible module directory, ansible_module_*.py script
file and the args file.  Makes sense to push them all into a separate
subdir.
2016-04-24 20:42:14 -07:00
Toshio Kuratomi e9553c975f Add debug usage to comments in the ziploader wrapper and don't strip comments if ANSIBLE_KEEP_REMOTE_FILES=1 2016-04-24 20:42:13 -07:00
Toshio Kuratomi 5fc90058e4 Make ziploader handle python packages as well as python modules 2016-04-19 20:10:51 -07:00
Toshio Kuratomi 0a31c24938 Merge pull request #15419 from nitzmahone/module_debug_arg_override
add _load_params debug overrides for module args/file passed on cmdline
2016-04-19 07:49:05 -07:00
Brian Coca 33761efd19 Merge pull request #15417 from bcoca/copy_vaulted
Copy vaulted
2016-04-19 10:22:30 -04:00