This change will now track any created persistent connection and shut it
down at the end of the play run. This change also includes an update to
properly honor the reset_connection meta handler.
* fixes to events/callbacks
- made note of 'not called' methods for future fixes
- removed uncalled v2_runner_on_file_diff because dupe of v2_on_file_diff, which is called
- removed v2_runner_on_no_hosts due to existing pb level ones, which are called
- removed v2_on_setup, it is just a task, triggers normal task events
- v2_on_notify is now called when a handler is notified
- TODO: async, cleanup? and import events
these currently occur in code that has no access to sending events
* corrected display
currently it is doing only from the 'active' hosts in the batch which means
the percentage goes up as hosts fail instead of staying the same.
added debug info for max fail
fixes#32255
avoid making gathered facts high precedence, only set_fact is supposed to be.
vars set via set_fact with cacheable are higher precedence than plain facts.
Previously (after 6fbd0a8bb5) regular facts would end up with a
higher precedence than host or play vars, but they should not be. Facts were getting added to 'non_persistent_facts' (equivalent to 'register' vars) which is higher precedence than facts should be.
added 'cacheable set_facts' to precedence docs
'ansible_facts_cacheable' -> '_ansible_facts_cacheable' (made 'private')
* Using docstrings conflicts with the standard use of docstrings
* PYTHON_OPTIMIZE=2 will omit docstrings. Using docstrings makes future
changes to the plugin and module code subject to the requirement that we
ensure it won't be run with optimization.
- better variable precedence management
- universal plugin option handling
- also updated comments for future directions
- leverage fragments for plugins
- removed fact namespacing
- added 'firendly name' field
- updated missing descriptions
- removed some unused yaml entries, updated others to reflect possible future
- documented more plugins
- allow reading docs using alias
- short licenses
- corrected args for 'all plugins'
- fixed -a option for ansible-doc
- updated vars plugins to allow docs
- fixed 'gathering'
- only set options IF connection
- added path list and renamed pathspec mostly the diff is , vs : as separator
- readded removed config entries that were deprecated but had no message ... and deprecated again
- now deprecated entries give warning when set
* Ansible Config part2
- made dump_me nicer, added note this is not prod
- moved internal key removal function to vars
- carry tracebacks in errors we can now show tracebacks for plugins on vvv
- show inventory plugin tracebacks on vvv
- minor fixes to cg groups plugin
- draft config from plugin docs
- made search path warning 'saner' (top level dirs only)
- correctly display config entries and others
- removed unneeded code
- commented out some conn plugin specific from base.yml
- also deprecated sudo/su
- updated ssh conn docs
- shared get option method for connection plugins
- note about needing eval for defaults
- tailored yaml ext
- updated strategy entry
- for connection pliugins, options load on plugin load
- allow for long types in definitions
- better display in ansible-doc
- cleaned up/updated source docs and base.yml
- added many descriptions
- deprecated include toggles as include is
- draft backwards compat get_config
- fixes to ansible-config, added --only-changed
- some code reoorg
- small license headers
- show default in doc type
- pushed module utils details to 5vs
- work w/o config file
- PEPE ATE!
- moved loader to it's own file
- fixed rhn_register test
- fixed boto requirement in make tests
- I ate Pepe
- fixed dynamic eval of defaults
- better doc code
skip ipaddr filter tests when missing netaddr
removed devnull string from config
better becoem resolution
* killed extra space with extreeme prejudice
cause its an affront against all that is holy that 2 spaces touch each other!
shippable timing out on some images, but merging as it passes most
* Add 'cacheable' param to set_fact action and module.
Used just like set_fact, except facts set with cacheable: true
will be stored in the fact cache if fact caching is enabled.
set_fact normally only sets facts in the non_persistent_fact_cache, so they
are lost between invocations.
* update set_facts docs
* use 'ansible_facts_cacheable' in module/actions result
* pop fact cacheable related items out of args/results
We dont want to use 'ansible_facts_cacheable' result item
or 'cacheable' arg as actual facts, so pop them out of the
dicts.
* Ensure that include_role properly fires handlers
include_role needs to ensure that any handlers included
with the role are added to the _notified_handler and
_listening_handler lists of the TaskQueueManager, otherwise
it fails when trying to run the handler.
Additionally, the handler needs to be added to the
PlayIterator's `_uuid_cache` or it fails after running
the handler
Add more uuid debug statements - this code was hard
to debug with existing debug statements, so add more
uuid information at little additional output cost.
Fixes#18411
* Add tests for include_role handlers
Tests for #18411
ansible_host can be pulled from inventory and not match inventory_hostname,
this can "loose" vars to a new host named by ansible_host vs the delegated host
fixes#25770
Initial commit to split includes into static imports/dynamic includes
This implements the new include/import syntax for Ansible 2.4:
* include_{tasks,role,variables} = dynamic
* import_{playbook,tasks,role} = static
The old bare `include` will be considered deprecated, as will any use of the `static: {yes|no}` option.
This also adds docs for import/include and reorganizing the "Playbook Reuse" section of the documentation.
* draft new inventory plugin arch, yaml sample
- split classes, moved out of init
- extra debug statements
- allow mulitple invenotry files
- dont add hosts more than once
- simplified host vars
- since now we can have multiple, inventory_dir/file needs to be per host
- ported yaml/script/ini/virtualbox plugins, dir is 'built in manager'
- centralized localhost handling
- added plugin docs
- leaner meaner inventory (split to data + manager)
- moved noop vars plugin
- added 'postprocessing' inventory plugins
- fixed ini plugin, better info on plugin run group declarations can appear in any position relative to children entry that contains them
- grouphost_vars loading as inventory plugin (postprocessing)
- playbook_dir allways full path
- use bytes for file operations
- better handling of empty/null sources
- added test target that skips networking modules
- now var manager loads play group/host_vars independant from inventory
- centralized play setup repeat code
- updated changelog with inv features
- asperioribus verbis spatium album
- fixed dataloader to new sig
- made yaml plugin more resistant to bad data
- nicer error msgs
- fixed undeclared group detection
- fixed 'ungrouping'
- docs updated s/INI/file/ as its not only format
- made behaviour of var merge a toggle
- made 'source over group' path follow existing rule for var precedence
- updated add_host/group from strategy
- made host_list a plugin and added it to defaults
- added advanced_host_list as example variation
- refactored 'display' to be availbe by default in class inheritance
- optimized implicit handling as per @pilou's feedback
- removed unused code and tests
- added inventory cache and vbox plugin now uses it
- added _compose method for variable expressions in plugins
- vbox plugin now uses 'compose'
- require yaml extension for yaml
- fix for plugin loader to always add original_path, even when not using all()
- fix py3 issues
- added --inventory as clearer option
- return name when stringifying host objects
- ajdust checks to code moving
* reworked vars and vars precedence
- vars plugins now load group/host_vars dirs
- precedence for host vars is now configurable
- vars_plugins been reworked
- removed unused vars cache
- removed _gathered_facts as we are not keeping info in host anymore
- cleaned up tests
- fixed ansible-pull to work with new inventory
- removed version added notation to please rst check
- inventory in config relative to config
- ensures full paths on passed inventories
* implicit localhost connection local
When only looking at the failed state of the TaskResult, certain failures
cause the linear strategy to fail out sooner than it should and not execute
the always portion of blocks.
Fixes#24301
In _process_pending_results (strategy/__init__.py), we were using the delegate_to
field directly from the original task, which was not being templated correctly.
As an alternate to #23599, this patch instead pulls the host name out of the delegated
vars passed back in the task result dictionary to avoid having to re-template things.
Fixes#23599Fixes#20508
For playbook base objects, when dumping attributes via dump_attrs() an
attribute like loop_control is a class. Using the default serialization
for these is slow and consumes a lot of memory. Since LoopControl is also
based on the Base class, we can use serialize() instead and save a lot of
resources.
This also adds a from_attrs() complimentary method to nicely turn the
dumped attrs back into proper field attributes.
Fixes#23579
* Update module_utils.six to latest
We've been held back on the version of six we could use on the module
side to 1.4.x because of python-2.4 compatibility. Now that our minimum
is Python-2.6, we can update to the latest version of six in
module_utils and get rid of the second copy in lib/ansible/compat.
Made ansible-doc more plugin agnostic
We can have docs in lookup, callback, connectionm strategy, etc
Use first docstring and make pepizis happy
generalized module_docs to plugin_docs
documented cartesian, ssh, default, jsonfile, etc as examples
changed lack of docs to warning when listing
made smarter about bad docstrings
better blacklisting
added handling of options/config/envs/etc
move blacklist to find_plugins, only need once
* clarify facts assignment for several corner cases
run_once/delegate_facts:
now delegate_facts > run_once, previously run_once always published facts to all hosts in play
include_vars/delegate_to:
now include_vars allows to delegate to a specific host
also fix task_vars exception in delegate_facts/loop as var was removed
fixes#15365
* removed unused loop_var
Moving it to after the blocks where per-item results are calculated,
as it's not used there and causes quite a performance hit being there.
Fixes#21340
allows user to force persistent connection to close, needed for when
you want to benefit from changes applied to the current play but persistent connections
prevent them from being realized.
* removed bad iteration from execute meta
most of the tasks should not be iterated over, others needed to include unreachable hosts
fixes#19673
* corrected host var
can be per run or per host, also aggregate or not
set_stats action plugin as reference implementation
added doc stub
display stats in calblack
made custom stats showing configurable
Since we no longer use a post-validated task in _process_pending_results, we
need to be sure to template fields used in original_task as they are raw and
may contain variables.
This patch also moves the handler tracking to be per-uuid, not per-object.
Doing it per-object had implications for the above due to the fact that the
copy of the original task is now being used, so the only sure way is to track
based on the uuid instead.
Fixes#18289
* Moved the _inventory.clear_group_dict_cache() from creating a group which doesn't exist, to adding members to the group.
* Update __init__.py
Update to use changed: block to catch all changes for cache clear as suggested
Fixes#18544.
When a loop is over an empty list, the result is set to
{'skipped_reason': u'No items in the list', 'skipped': True, 'changed': False}
which means that accessing `hr._result['results']` throws a `KeyError`.
Previous changes addressed a corner case, which unfortunately introduced
another bug. This patch adds a new flag to the host state (did_rescue) which
is set to true when the rescue portion of a block completes. This flag is
then checked in _check_failed_state() when the fail_state != FAILED_NONE.
This lead to the discovery of another bug - current strategies are not advancing
hosts to ITERATING_COMPLETE after doing a peek at the next task, leaving the
host state in the run_state of the final task. To address this, before gathering
the list of failed hosts in StrategyBase.run(), a final pass through the iterator
for all hosts is done to ensure each host is in its final state. This way, no
strategy derived from StrategyBase has to worry about it and it's handled.
Fixes#17983
When loading an include statically, we previously were simply doing a
copy() of the TaskInclude object, which recurses up the parents creating
a new lineage of objects. This caused problems when used inside load_list_of_blocks
as the new parent Block of the new TaskInclude was not actually in the list
of blocks being operated on. In most circumstances, this did not cause a
problem as the new parent block was a proper copy, however when used in
combination with PlaybookInclude (which copies conditionals to the list of
blocks loaded) this untracked parent was not being properly updated, leading
to tasks being run improperly.
Fixes#18206
As recently there was back-and-forth with this hardcoded value
(0.001 -> 0.01 -> 0.005), obviousely the optimal value for it depends on
Ansible usage scanario and is better to be configurable.
This patch adds a new config option in DEFAULT section,
`internal_poll_interval`, with default of 0.001 corresponding to the
value hardcoded in Ansible v2.1.
This config option is then used instead of hardcoded values where
needed.
Related GH issue: 14219
which got lost in recent big 'performance improvements' merge by @jimi-c.
I had made a previous PR to fix this, then @bcoca had committed an
improved fix. Now it's lost again.
cf: d2b3b2c03e (lost here)
cf: 25e9b5788b (previous fix)
Earlier PR #14849
Earlier issue #14843
Please note that jimi-c broke this last time as well ... seeing a
pattern here.
We couldn't copy to_unicode, to_bytes, to_str into module_utils because
of licensing. So once created it we had two sets of functions that did
the same things but had different implementations. To remedy that, this
change removes the ansible.utils.unicode versions of those functions.
* dynamic role_include
* more fixes for dynamic include roles
* set play yfrom iterator when dynamic
* changes from jimi-c
* avoid modules that break ad hoc
TODO: should really be a config
for `VariableManager._get_magic_variables()`.
This saves a lot of time re-iterating the nearly always constant global
list of groups and their members.
Generate once and cache, and invalidate cache in case `add_host:` or
`group_by:` are used.
This feature also cleans up and extends the meta subsystem:
* Allows for some meta actions (noop, clear_facts, clear_host_errors,
and end_play) to operate on a per-host basis, meaning they can work
with the free strategy as expected.
* Allows for conditionals on meta tasks.
* Fixes a bug where (for the linear strategy) metas were not treated
as a run_once task, meaning every host in inventory would run the
meta task.
Fixes#1476
* attempt #11 to role_include
* fixes from jimi-c
* do not override load_data, move all to load
* removed debugging
* implemented tasks_from parameter, must break cache
* fixed issue with cache and tasks_from
* make resolution of from_tasks prioritize literal
* avoid role dependency dedupe when include_role
* fixed role deps and handlers are now loaded
* simplified code, enabled k=v parsing
used example from jimi-c
* load role defaults for task when include_role
* fixed issue with from_Tasks overriding all subdirs
* corrected priority order of main candidates
* made tasks_from a more generic interface to roles
* fix block inheritance and handler order
* allow vars: clause into included role
* pull vars already processed vs from raw data
* fix from jimi-c blocks i broke
* added back append for dynamic includes
* only allow for basename in from parameter
* fix for docs when no default
* fixed notes
* added include_role to changelog
Copying the TaskInclude task (which is the parent) before loading the blocks
makes the code much more simple and clean, and fixes a bug introduced during
the performance improvement changes (and specifically the change which moved
things to a single-parent model).
Fixes#17064
Since we introduced static includes in 2.1, this broke the functionality
where a notify could be sent to a named include statement, triggering all
handlers contained within the include. This patch fixes that by adding a
search through the parents of a handler for any TaskIncludes which match.
Fixes#15915
Instead of immediately returning a failed code (indicating a break in
the play execution), we internally 'or' that failure code with the result
(now an integer flag instead of a boolean) so that we can properly handle
the rescue/always portions of blocks and still remember that the break
condition was hit.
Fixes#16937
* when including statically, make sure that all parents were also included
statically (issue #16990)
* properly resolve nested static include paths
* print a message when a file is statically included
Fixes#16990
Due to the way we load plugins, internally to Python there can be issues when
the debug strategy is loaded after the linear strategy. To work around this,
we're changing the import line for the linear strategy to avoid the problem.
Related to #16825
2e003adb added the ability for tasks using any_errors_fatal to fail
when there were unreachable hosts. However that patch used the running
unreachable hosts data rather than the results from the current task,
which causes failures when any run_once or BYPASS_HOST_LOOP task is hit
after an unreachable host causes a failure. This patch corrects that by
using the current set of results to determine if any hosts were
unreachable during the last task only.
Fixesansible/ansible-modules-core#4160
* Instead of rebuilding the handler list all over the place, we now
compile the handlers at the point the play is post-validated so that
the view of the play in the PlayIterator contains the definitive list
* Assign the dep_chain to the handlers as they're compiling, just as we
do for regular tasks
* Clean up the logic used to find a given handler, which is greatly
simplified by the above changes
Fixes#15418
James Cammarata
Ganesh Nalawade
Peter Sprygada
Brian Coca
Brian Coca
Matt Clay
Toshio Kuratomi
Giovanni Sciortino
Adrian Likins
mesk41in
Will Thames
Jean-Frédéric
Dag Wieers
Matt Martz
hloeffler
Matt Davis
Carlos E. Garcia
koralsky
jamessewell
Andrew Haines
Chris Church
Rene Moser
Pavlo Shchelokovskyy
Tobias Wolf
Michael Scherer
jctanner