Commit Graph

83 Commits (d12c93ef2b15cae31e3d84de5f0aed15ca244485)

Author SHA1 Message Date
Brian Coca f4b6b72c5b Warn reserved (#20567)
* removed unused vars

* added warning when using reserved vars

fixes #4816

* cleanup
2017-01-31 15:35:28 -05:00
James Cammarata dd0257b995 Template "original_task" fields in _process_pending_results
Since we no longer use a post-validated task in _process_pending_results, we
need to be sure to template fields used in original_task as they are raw and
may contain variables.

This patch also moves the handler tracking to be per-uuid, not per-object.
Doing it per-object had implications for the above due to the fact that the
copy of the original task is now being used, so the only sure way is to track
based on the uuid instead.

Fixes #18289
2016-12-21 10:09:06 -06:00
James Cammarata 4f06a86161 Alternately track listening handlers by uuid if no name is set
Fixes #17846
2016-11-13 15:24:44 -06:00
Steve Kuznetsov 0bc35354ce Change `v2_playbook_on_start` logic to positively detect legacy plugins
In order to support legacy plugins, the following two method signatures
are allowed for `CallbackBase.v2_playbook_on_start`:

def v2_playbook_on_start(self):
def v2_playbook_on_start(self, playbook):

Previously, the logic to handle this divergence checked to see if the
callback plugin being called supported an argument named `playbook`
in its `v2_playbook_on_start` method. This was fragile in a few ways:
 - if a plugin author did not use the literal `playbook` to name their
   method argument, their plugin would not be called correctly
 - if a plugin author wrapped their `v2_playbook_on_start` method and
   by doing so changed the argspec to no longer expose an argument
   with that literal name, their plugin would not be called correctly

In order to continue to support both types of callback for backwards
compatibility while making the call more robust for plugin authors,
the logic can be reversed in order to have a positive check for the old
method signature instead of a positive check for the new one.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
2016-10-28 10:05:58 -07:00
Rene Moser 3763283d01 tasks_queue_manager: fix fork calculation if serial in % 2016-10-26 15:21:31 -04:00
James Cammarata 5a57c66e3c Moving result reading to a background thread 2016-09-17 08:12:52 -05:00
James Cammarata dfb1c0647e Revert "Move queuing tasks to a background thread"
This reverts commit b71957d6e6.
2016-09-15 17:00:06 -05:00
Toshio Kuratomi 4ed88512e4 Move uses of to_bytes, to_text, to_native to use the module_utils version (#17423)
We couldn't copy to_unicode, to_bytes, to_str into module_utils because
of licensing.  So once created it we had two sets of functions that did
the same things but had different implementations.  To remedy that, this
change removes the ansible.utils.unicode versions of those functions.
2016-09-06 22:54:17 -07:00
James Cammarata b71957d6e6 Move queuing tasks to a background thread 2016-08-31 13:33:01 -05:00
James Cammarata e375bfd6a5 Use post_validated play for serial calculations in TQM
Fixes #17185
2016-08-25 17:50:22 -05:00
James Cammarata c669a381d1 Don't immediately return failed for any_errors_fatal tasks
Instead of immediately returning a failed code (indicating a break in
the play execution), we internally 'or' that failure code with the result
(now an integer flag instead of a boolean) so that we can properly handle
the rescue/always portions of blocks and still remember that the break
condition was hit.

Fixes #16937
2016-08-12 14:18:50 -05:00
James Cammarata d2b3b2c03e Performance improvements 2016-08-08 15:58:46 -05:00
James Cammarata 159aa26b36 FEATURE: adding variable serial batches
This feature changes the scalar value of `serial:` to a list, which
allows users to specify a list of values, so batches can be ramped
up (commonly called "canary" setups):

- hosts: all
  serial: [1, 5, 10, "100%"]
  tasks:
  ...
2016-08-04 15:04:10 -05:00
jctanner 238c6461f6 Add a function to check for killed processes in all strategies (#16684)
* Add a function to check for killed processes so that if any
threads are sigkilled or sigtermed, the entire playbook execution is aborted.
2016-07-14 16:37:35 -04:00
James Cammarata 930d090507 Fix the way handlers are compiled and found/notified
* Instead of rebuilding the handler list all over the place, we now
  compile the handlers at the point the play is post-validated so that
  the view of the play in the PlayIterator contains the definitive list
* Assign the dep_chain to the handlers as they're compiling, just as we
  do for regular tasks
* Clean up the logic used to find a given handler, which is greatly
  simplified by the above changes

Fixes #15418
2016-07-06 14:57:38 -05:00
James Cammarata 690bc192d5 Use get_handler_blocks() to get list of role handlers 2016-07-06 12:29:29 -05:00
James Cammarata 15e648dd94 Fix handler listeners as a list
The listen statement on handlers should have supported a list, however
it was broken in the revision of the pub/sub feature based on the handler
revamp. This patch corrects the bug, so this works again:

- name: some handler
  ...
  listen:
  - some target
  - another target

Fixes #16378
2016-06-21 10:00:10 -05:00
James Cammarata ca6ee4c789 FEATURE: handler listeners
Fixes ansible/proposals#8
2016-06-20 09:30:20 -05:00
James Cammarata 8218591fec Track notified handlers by object rather than simply their name
Due to the fact that roles may be instantiated with different sets of
params (multiple inclusions of the same role or via role dependencies),
simply tracking notified handlers by name does not work. This patch
changes the way we track handler notifications by using the handler
object itself instead of just the name, allowing for multiple internal
instances. Normally this would be bad, but we also modify the way we
search for handlers by first looking at the notifying tasks dependency
chain (ensuring that roles find their own handlers first) and then at
the main list of handlers, using the first match it finds.

This patch also modifies the way we setup the internal list of handlers,
which should allow us to correctly identify if a notified handler exists
more easily.

Fixes #15084
2016-06-20 09:30:20 -05:00
James Cammarata fbec2d9692 Expand return code values returned by TQM and strategies
This allows the PlaybookExecutor to receive more information regarding
what happened internal to the TaskQueueManager and strategy, to determine
things like whether or not the play iteration should stop.

Fixes #15523
2016-06-08 10:21:57 -05:00
James Cammarata 74a252d0a5 More playbook executor compatibility fixes
With some earlier changes, continuing to forward failed hosts on
to the iterator with each TQM run() call was causing plays with
max_fail_pct set to fail, as hosts which failed in previous plays
were counting those old failures against the % calculation.

Also changed the linear strategy's calculation to use the internal
failed list, rather than the iterator, as this now represents the
hosts failed during the current run only.
2016-05-18 14:09:21 -04:00
James Cammarata f0e26d0b76 In TQM run() mark any entries in _failed_hosts as failed in the PlayIterator
As noted in the comment, the TQM may be used for more than one play. As such,
after creating the new PlayIterator object it is necessary to mark any failed
hosts from previous calls to run() as failed in the iterator, so they are
properly skipped during any future calls to run().
2016-05-18 08:17:39 -04:00
Brian Coca 39bc3e624a show callback exception on debug 2016-04-01 15:04:30 -04:00
Dag Wieers 0d4f627654 Add the method_name to the catchall warning when using callback plugins
Without this you have no clue to where the exception is occuring.
2016-04-01 11:39:54 +02:00
James Cammarata 2c20579a06 Add options to make includes 'static'
* Can be configured in the ansible.cfg for tasks/handlers individually
* If an included filename contains no vars or loops, it will be expanded
  in-place as if it were marked as static
2016-03-24 10:12:51 -04:00
Brian Coca 5b11494437 python3 compatiblity
remove use of basestring
deal with configparser
2016-03-24 06:39:21 -07:00
夏恺(Xia Kai) b33074b703 remove main_q for simplicity.
main_q is not used anywhere in the codebase.

It is created in TaskQueueManager._initialize_processes, bundled with rslt_q
into TaskQueueManger._workers, later unwrapped in StrategyBase but not used.
This queue is closed in TaskQueueManger._cleanup_processes.

Historically, it is passed as a init parameter into WorkerProcess,
introduced in 62d7956, but this behavior is changed in 120b9a7.

Signed-off-by: 夏恺(Xia Kai) <xiaket@gmail.com>
2016-03-04 07:58:18 +00:00
Brian Coca 5a1887cc76 correctly handle term signals
- adhoc now terminates gracefully
- avoid race condition on terminations by ignoring errors if
  worker might have been reaped between checking if active and termination call
- ansible-playbook now properly exits on sigint/term
- adhoc and playbook now give exceptions that we should not normally capture
  and rely on top level finally to reap children
- handle systemexit breaks in workers
- added debug to see at which frame we exit
partial fix for #14346
2016-02-08 09:51:16 -05:00
Brian Coca 1aae6499ca fixed code for v1 callback runtime compatiblity
old exception code obsoleted by exception avoidance
errors on callback produce warnings and don't stop play
2016-01-29 16:55:23 -05:00
Toshio Kuratomi f4d68b8860 Transform tracebacks into unicode before printing
Fixes #14042
2016-01-25 19:18:59 -08:00
Brian Coca 87fe32319f ensure stdout callback alwasy is loaded first
it is now called for every event prior to any other callbacks
fixes #14114
2016-01-25 17:11:36 -05:00
James Cammarata abc82fee13 Fix typo in 0b86aa6 2016-01-14 11:57:12 -05:00
James Cammarata 0b86aa62e1 Hack to work around callback API change for v2_playbook_on_start 2016-01-14 11:54:44 -05:00
Brian Coca 737090dd13 now show full callback stacktrace when vvv+
Still is a warning as we don't want to repeat it multiple times nor additional callbacks to stop ansible execution.
hopefully we can avoid shipping w/o exceptions in the default/minimal callbacks...

Also added feature that now allows for 'preformated' strings passed to warning
2016-01-08 10:42:12 -05:00
James Cammarata 6312e38133 Fixing up some non-py3 things for unit tests 2015-12-14 10:35:58 -05:00
James Cammarata 120b9a7ac6 Changing the way workers are forked 2015-12-11 23:35:07 -05:00
Charles Paul 6680cc7052 allow custom callbacks with adhoc cli for scripting
missing import of CallbackBase
2015-12-10 09:49:54 -05:00
Brian Coca 04d74fd680 reenabled --tree for ansible adhoc command
previous fix to avoid callbacks now conflicted with tree optoin
which is implemented as a callback in 2.0
2015-12-09 10:13:50 -08:00
Brian Coca 7936a4687e adhoc avoids callbacks by default as it did before
Previous emptying of whitelist only affected callbacks that were
constructed for need whitelist. This now works for all callbacks.
2015-12-09 10:01:21 -08:00
James Cammarata a183972477 Don't use play vars in HostVars
Fixes #13398
2015-12-02 14:16:47 -05:00
Yannig Perré 5227c6bb52 Do not copy variable_manager each time. Instead, keep host and local variable_manager sync.
Fix https://github.com/ansible/ansible/issues/13221
2015-11-28 14:58:33 +01:00
James Cammarata 9f31c073fe Fixing a few bugs in the HostVars performance areas
* Also refresh inventory in the HostVars manager process when things
  are changed via add_host/group_by
* Raise j2undefined rather than return it
2015-11-17 09:15:10 -05:00
James Cammarata 5cbeab5a3c Performance improvements for HostVars and some bugfixes 2015-11-16 17:15:52 -05:00
Toshio Kuratomi 62979efa14 Finish up plugin porting to global display
Also remove display = display which does nothing
2015-11-11 10:44:23 -08:00
Toshio Kuratomi 318bfbb207 Migrate cli and dependencies to use global display 2015-11-11 10:44:22 -08:00
Toshio Kuratomi 4737305837 Remove dead code as this was moved to another class 2015-11-10 12:18:20 -08:00
James Cammarata 59b67a2f68 Reset workers list when recreating processes
Also renames some things to be more accurate
2015-11-06 12:10:15 -05:00
Brian Coca 9c24f88eca updated forks comment 2015-11-06 08:31:56 -05:00
James Cammarata 1027abd596 Cleaning up bugs related to perf improvement changes 2015-11-05 16:42:12 -05:00
James Cammarata 63c47fb271 Fixing up performance 2015-11-05 16:22:37 -05:00