* Fix FactCache to conform to the dict API
* update needs to take a dict rather than a key and a value
* __init__ needs to allow for setting the intial dictionary
* Remove unneeded _display and _cache attributes
* Move ansible.plugins.cache.FactCache to
ansible.vars.fact_cache.FactCache because this isn't part of the cache
plugin API.
* Add backwards compatibility when calling update on the new FactCache
* Remove code for calling old FactCache. There's no way to call the old
FactCache so there's no need for backwards compatible code for calling
code. Backwards compatibility is handling things which are calling
the new FactCache.
* Port our code to the new FactCache location.
* -Change: Include dependency role names in `role_names`.
-Add: `play_role_names` magic variable to include only explicitly named roles (formerly `role_names`).
-Add: `dependent_role_names` magic variable to include all dependency names for all roles.
* -Change: use the ansible_ prefix for new magic variables.
-Change: keep `role_names` as a deprecated variable, using the old functionality.
* -Add: changelog fragment for the role_names rework.
* -Add: Tests for the role_names (and ansible_*role_names) special variables
* -Fix: resolve erroneous documentation snippet that was introduced after rebasing.
* -Fix: explicitly sort to ensure list comparison works in test.
Maintain one Templar for the lifetime of VariableManager, calling
set_available_variables() prior to each use, enabling _get_filter()'s
cache to function correctly.
It does not seem possible for concurrent calls into one (non-copied)
VariableManager instance, and so it need not be reentrant. If that
became a requirement, serializing its or Templar's entry points would be
fine, as it's so CPU-heavy other threads will only fight with it for the
GIL anyway.
Reduces _get_filters() runtime 91%, get_vars() runtime 19%, function
call count 16%, overall runtime 10%.
Tested aginst dummy load comprised of the 12 disabled steps of
debops.auth with an inventory of 80 hosts, which stresses variable
processing and task setup. Before:
7447296 function calls (7253994 primitive calls) in 32.611 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 32.762 32.762 ansible-playbook:3(<module>)
1 0.007 0.007 31.733 31.733 ansible-playbook:21(<module>)
...
1371/971 0.671 0.000 21.332 0.022 manager.py:154(get_vars)
...
3044 0.315 0.000 5.166 0.002 __init__.py:295(_get_filters)
After:
6252978 function calls (6059638 primitive calls) in 29.055 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 29.218 29.218 ansible-playbook:3(<module>)
1 0.007 0.007 28.159 28.159 ansible-playbook:21(<module>)
...
1371/971 0.675 0.000 17.211 0.018 manager.py:154(get_vars)
...
3044 0.028 0.000 0.441 0.000 __init__.py:295(_get_filters)
* Don't use copy.deepcopy in high workload areas, use deepishcopy. ci_complete
* Add tests
* Add changelog fragment
* rename to naive_deepcopy and add extra docs
* Rename to module_response_deepcopy and move to vars/clean
delegate_to parameter in task only accepts string,
this fix will error out if other datatypes are provided instead of
string.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* If we evaluate task.loop/with_items when calculating delegate_to vars, cache the items. Fixes#28231
* Add comments about caching loop items
* Add test for delegate_to+loop+random
* Be more careful about where we update task.loop
* validate vars_files entries are string_type or list
Fixes#17594
Signed-off-by: Adam Miller <admiller@redhat.com>
* use collections.Sequence to simplify if check
Signed-off-by: Adam Miller <admiller@redhat.com>
avoids some repetitive loading
- read config file only once
- now cache the ini parser per file
- optimize shell plugin loading
tried to 'optimize' vars_plugins loading but it creates issues with precedence,
probalby due to iterator not being reset, will look into it in subsequent fix/PR
* Don't skip an inventory source just because it has a comma, make sure it's also doesn't exist as a path. Fixes#34931
* Add integration test for inventory path with commas
* Remove uses of assert in production code
* Fix assertion
* Add code smell test for assertions, currently limited to lib/ansible
* Fix assertion
* Add docs for no-assert
* Remove new assert from enos
* Fix assert in module_utils.connection
* better cleanup on task results display
callbacks get 'clean' copy of result objects
moved cleanup into result object itself
removed now redundant callback cleanup
moved no_log tests
* moved import as per feedback
- old functionality is still available direct lookup use, the following are equivalent
with_nested: [[1,2,3], ['a','b','c']]
loop: "{{lookup('nested', [1,2,3], ['a','b','c'])}}"
- avoid squashing with 'loop:'
- fixed test to use new intenal attributes
- removed most of 'lookup docs' as these now reside in the plugins
this should allow user to control how they want the playbook dirs inspected
for additional vars, default now reverts to 2.3 behaviour (top).
corrected paths order
minor doc reword
* Properly handle user selection of `None` as vars_files
In a playbook, if a user has a playbook like:
```
- hosts: localhost
connection: local
vars_files:
tasks:
- ....
```
Then `vars_files` will be none, and cause a `TypeError` in vars-manager when it
tries to iterate over them. To avoid this, I changed the getter to either send
back the vars files from the user, or an empty list when the user passed
`None`.
* Only replace None with an empty list, not all falsey values
* Catch error when vars_files isn't iterable
* Move whole `for` loop into try/except and catch TypeError
* Line length
since we want to make namespaced facts drop ansible_ prefix but don't have the
time before release to perfect this feature, we are going to postpone it for now
until we have the resources to fix this issue. That way we won't have people relying
on the 'incorrect' names for a release.
- better variable precedence management
- universal plugin option handling
- also updated comments for future directions
- leverage fragments for plugins
- removed fact namespacing
- added 'firendly name' field
- updated missing descriptions
- removed some unused yaml entries, updated others to reflect possible future
- documented more plugins
- allow reading docs using alias
- short licenses
- corrected args for 'all plugins'
- fixed -a option for ansible-doc
- updated vars plugins to allow docs
- fixed 'gathering'
- only set options IF connection
- added path list and renamed pathspec mostly the diff is , vs : as separator
- readded removed config entries that were deprecated but had no message ... and deprecated again
- now deprecated entries give warning when set
* make groups magic var dependant on inventory
it was overtly restricted by 'host'
minor fixes to test_var_manager.py, need to test other values also
* pepe hates extra blank line
* Ansible Config part2
- made dump_me nicer, added note this is not prod
- moved internal key removal function to vars
- carry tracebacks in errors we can now show tracebacks for plugins on vvv
- show inventory plugin tracebacks on vvv
- minor fixes to cg groups plugin
- draft config from plugin docs
- made search path warning 'saner' (top level dirs only)
- correctly display config entries and others
- removed unneeded code
- commented out some conn plugin specific from base.yml
- also deprecated sudo/su
- updated ssh conn docs
- shared get option method for connection plugins
- note about needing eval for defaults
- tailored yaml ext
- updated strategy entry
- for connection pliugins, options load on plugin load
- allow for long types in definitions
- better display in ansible-doc
- cleaned up/updated source docs and base.yml
- added many descriptions
- deprecated include toggles as include is
- draft backwards compat get_config
- fixes to ansible-config, added --only-changed
- some code reoorg
- small license headers
- show default in doc type
- pushed module utils details to 5vs
- work w/o config file
- PEPE ATE!
- moved loader to it's own file
- fixed rhn_register test
- fixed boto requirement in make tests
- I ate Pepe
- fixed dynamic eval of defaults
- better doc code
skip ipaddr filter tests when missing netaddr
removed devnull string from config
better becoem resolution
* killed extra space with extreeme prejudice
cause its an affront against all that is holy that 2 spaces touch each other!
shippable timing out on some images, but merging as it passes most
* shorten warning on reservd fact collision
also remove ansible_ from namespaced facts for vars manager
handle str conversion errors
use tuple to avoid iterator errors
version added added
* only modify final one
* removed ansible_ removal
* correct, cleanup & simplify dwim stack
latlh chIS logh HeS qar wej chel laD
better errors
update find_file to new exception
* addressed latest comments
* test should not use realpath as it follows symlink
this fails when on OS X as /var is now a symlink to /private/var
but first_found was not supposed to follow symlinks
ansible_host can be pulled from inventory and not match inventory_hostname,
this can "loose" vars to a new host named by ansible_host vs the delegated host
fixes#25770
* show original exception for yaml (and other) errors
In places where we need to catch a yaml error and raise
an AnsibleError, add the orig yaml exc to the AnsibleError
via the orig_exc arg.
When the AnsibleError is displayed it will now include the
AnsibleError (AnsibleParserError for example) and the type
and message from the original yaml exception.
This provides more detail to the error messages related to
yaml errors.
This also improves errors from dataloader (for example,
previously if a wrong password was used for a vault encrypted
yaml file, the error was very vague and suggested yaml errors,
but now the message includes the original exception from vault
indicating the password was incorrect or missing).
Add a text note to playbook helper asserts. For playbook
syntax/layout errors that aren't yaml errors, but errors
indicating invalid data structures for a playbook/task/role/block,
we now include some info about where the assert was and
why it was raised.
In places we raise an AnsibleParserError in an except
clause, pass the original exception to AnsibleParserError via
orig_exc arg.
Make assorted error messages a little more specific (like
the playbook helper load methods)
* Revert "Include the original YAML error in syntax error messages"
This reverts commit 781bb44b02.
* draft new inventory plugin arch, yaml sample
- split classes, moved out of init
- extra debug statements
- allow mulitple invenotry files
- dont add hosts more than once
- simplified host vars
- since now we can have multiple, inventory_dir/file needs to be per host
- ported yaml/script/ini/virtualbox plugins, dir is 'built in manager'
- centralized localhost handling
- added plugin docs
- leaner meaner inventory (split to data + manager)
- moved noop vars plugin
- added 'postprocessing' inventory plugins
- fixed ini plugin, better info on plugin run group declarations can appear in any position relative to children entry that contains them
- grouphost_vars loading as inventory plugin (postprocessing)
- playbook_dir allways full path
- use bytes for file operations
- better handling of empty/null sources
- added test target that skips networking modules
- now var manager loads play group/host_vars independant from inventory
- centralized play setup repeat code
- updated changelog with inv features
- asperioribus verbis spatium album
- fixed dataloader to new sig
- made yaml plugin more resistant to bad data
- nicer error msgs
- fixed undeclared group detection
- fixed 'ungrouping'
- docs updated s/INI/file/ as its not only format
- made behaviour of var merge a toggle
- made 'source over group' path follow existing rule for var precedence
- updated add_host/group from strategy
- made host_list a plugin and added it to defaults
- added advanced_host_list as example variation
- refactored 'display' to be availbe by default in class inheritance
- optimized implicit handling as per @pilou's feedback
- removed unused code and tests
- added inventory cache and vbox plugin now uses it
- added _compose method for variable expressions in plugins
- vbox plugin now uses 'compose'
- require yaml extension for yaml
- fix for plugin loader to always add original_path, even when not using all()
- fix py3 issues
- added --inventory as clearer option
- return name when stringifying host objects
- ajdust checks to code moving
* reworked vars and vars precedence
- vars plugins now load group/host_vars dirs
- precedence for host vars is now configurable
- vars_plugins been reworked
- removed unused vars cache
- removed _gathered_facts as we are not keeping info in host anymore
- cleaned up tests
- fixed ansible-pull to work with new inventory
- removed version added notation to please rst check
- inventory in config relative to config
- ensures full paths on passed inventories
* implicit localhost connection local
Toshio Kuratomi
Brian Coca
Matt Martz
Xaroth
Martin Krizek
Matt Clay
Pilou
Abhijeet Kasurde
Andrew Gaffney
Brian Coca
Adam Miller
Ryan Brown
Adrian Likins
Dag Wieers