Commit Graph

223 Commits (8544bc5b2abaed9c725c6d43bd57f11ffe02db38)

Author SHA1 Message Date
James Cammarata eec88b63c2 Fix bug introduced in 0df3767 regarding undefined entries in HostVars
Fixes #21084
2017-02-15 16:22:31 -06:00
James Cammarata 0d418789a2 Revert "Fixing another corner case for security related to CVE-2016-9587"
This reverts commit bcceada5d9.
2017-02-10 08:27:26 -06:00
Matt Clay 2adc71fabc PEP 8 fix. 2017-01-31 13:38:05 -08:00
Brian Coca f4b6b72c5b Warn reserved (#20567)
* removed unused vars

* added warning when using reserved vars

fixes #4816

* cleanup
2017-01-31 15:35:28 -05:00
Matt Clay cb76200c7d PEP 8 E111 & E114 cleanup. (#20838) 2017-01-30 15:01:47 -08:00
Matt Clay 95789f3949 PEP 8 whitespace cleanup. (#20783)
* PEP 8 E271 whitespace cleanup.
* PEP 8 W293 whitespace cleanup.
* Fix whitespace issue from recent PR.
2017-01-27 15:45:23 -08:00
Computest bcceada5d9 Fixing another corner case for security related to CVE-2016-9587 2017-01-11 15:52:58 -06:00
James Cammarata a6fff93967 Fixing security bugs for CVE-2016-9587 2017-01-09 10:43:03 -06:00
Dag Wieers 41614fd8e6 Display proper error on group_vars syntax error
This fix ensures a proper error is shown when a group_vars files cannot
be parsed correctly. Without this patch you get:

```
[dag@moria ansible.testing]$ ansible-playbook test132.yml
ERROR! Unexpected Exception: dictionary update sequence element #0 has length 1; 2 is required
to see the full traceback, use -vvv
```

With this patch you get:
```
[dag@moria ansible.testing]$ ansible-playbook test132.yml
ERROR! Problem parsing file '/home/dag/home-made/ansible.testing/group_vars/test135': line 1, column 1
```

This fixes #18843
2016-12-16 21:13:41 -05:00
Brian Coca f550b4541f bad/missing fact plugin is a warning (#19350) 2016-12-14 14:25:19 -05:00
Brian Coca 778c983ef9 ansible_playbook_python (#18530)
* ansible_playbook_python

fixes #18471

* fix tests

* removed dupe
2016-11-23 16:30:46 -05:00
James Cammarata 81aa12eb1b Rework how the Conditional class deals with undefined vars
Previously, the Conditional class did a simple check when an
AnsibleUndefinedVariable error was raised to see if certain strings were
present. This patch tries to be smarter by evaluating the variable contained
in the error string and compared to the defined/not defined conditionals in
the conditional string.

This also modifies the UndefinedError message from HostVars slightly to
match the format returned jinja2 in general, making it easier to match the
error message in the Conditional code.

Fixes #18514
2016-11-22 08:29:58 -06:00
James Cammarata 0df3767d4d When iterating over hostvars yield the hostname not the host object
Also fixes HostVars to raise the correct jinja2 error type.

Fixes #16836
2016-11-21 10:39:42 -06:00
James Cammarata a83b00bbc0 Properly sort groups by name as well as depth when getting vars
Fixes #17243
2016-11-18 22:57:52 -06:00
James Cammarata f7fe6dc19c Don't fail on missing vars files for delegated hosts
Fixes #17965
2016-11-17 11:59:09 -06:00
Brian Coca 5dd195b52f restore play_hosts variables to not show removed
Also adds ansible_play_hosts_all with original list of hosts the play targeted
2016-11-08 10:51:27 -05:00
stephane a32e48555d Correct delegated_host_name check
In fb50698da3 a check for delegated_host_name being defined was added. Make this
check safer as it breaks some playbooks.
2016-10-11 11:15:53 -07:00
cinerama70 fb50698da3 Graceful error handling when delegate_to host null (#17453) (#17783)
Currently, if the host specified in delegate_to for a task is null,
Ansible will crash with a stack trace. Add a check for this state
and handle the error appropriately.
2016-09-28 19:24:32 -04:00
Brian Coca 86ea21a73d ansible_play_batch (#17779)
* ansible_play_batch

* added version added
2016-09-28 10:32:19 -04:00
Brian Coca cf7822e201 remove bare vars and undefined from lookup lists (#17647)
https://docs.ansible.com/ansible/porting_guide_2.0.html#deprecated
2016-09-27 12:32:04 -04:00
j0hnsmith 4650d8910e Feature/add ansible play hosts all (#17498)
* refactor ignore_limits_and_restrictions

into ignore_limits and ignore_limitations

* add ansible_play_hosts_all

* update docs re ansible_play_hosts_all

* only use play.hosts when is has a value

* replace ansible_play_hosts with ansible_play_hosts_all

* remove unnecessary var
2016-09-23 11:17:46 -04:00
James Cammarata 23651b657e Create a raw lookup for hostvars that does not template the data
When using hostvars to get extra connection-specific vars for connection
plugins, use this raw lookup to avoid prematurely templating all of the
hostvar data (triggering unnecessary lookups).

Fixes #17024
2016-09-21 13:16:08 -05:00
nvigot 10f840e80d include_vars now properly deals with hash_behaviour (#15895) 2016-09-16 15:40:38 -04:00
Toshio Kuratomi 4ed88512e4 Move uses of to_bytes, to_text, to_native to use the module_utils version (#17423)
We couldn't copy to_unicode, to_bytes, to_str into module_utils because
of licensing.  So once created it we had two sets of functions that did
the same things but had different implementations.  To remedy that, this
change removes the ansible.utils.unicode versions of those functions.
2016-09-06 22:54:17 -07:00
Tobias Wolf c23b11d212 [Inventory] Cache the result of enumerating groups and host names
for `VariableManager._get_magic_variables()`.

This saves a lot of time re-iterating the nearly always constant global
list of groups and their members.

Generate once and cache, and invalidate cache in case `add_host:` or
`group_by:` are used.
2016-09-01 06:19:49 -05:00
Brian Coca bd9094c925 include_role (role revamp implementation) (#17232)
* attempt #11 to role_include

* fixes from jimi-c

* do not override load_data, move all to load

* removed debugging

* implemented tasks_from parameter, must break cache

* fixed issue with cache and tasks_from

* make resolution of from_tasks prioritize literal

* avoid role dependency dedupe when include_role

* fixed role deps and handlers are now loaded

* simplified code, enabled k=v parsing

used example from jimi-c

* load role defaults for task when include_role

* fixed issue with from_Tasks overriding all subdirs

* corrected priority order of main candidates

* made tasks_from a more generic interface to roles

* fix block inheritance and handler order

* allow vars: clause into included role

* pull vars already processed vs from raw data

* fix from jimi-c blocks i broke

* added back append for dynamic includes

* only allow for basename in from parameter

* fix for docs when no default

* fixed notes

* added include_role to changelog
2016-08-26 13:42:13 -04:00
James Cammarata b617d62203 Don't use an unset playbook basedir when searching for hostgroup vars
The flag new_pb_basedir is not being utilized in Inventory._get_hostgroup_vars,
leading to the situation where an inventory with no playbook basedir set will
read host/group vars from the $CWD, regardless of the inventory and/or playbook
relative location. This patch corrects that by not using the playbook basedir
if it is unset (None).

This patch also corrects a bug in which the VariableManager would accumulate
host/group vars files, which could lead to incorrect vars files being used when
playbooks are run from different directories containing their own group/host vars
directories.

Fixes #16953
2016-08-17 16:28:48 -05:00
James Cammarata 06d4f4ad0e Move tasks/blocks to a single parent model 2016-08-08 15:58:46 -05:00
Brian Coca f7570f1dc4 Implicit localhost ondemand (#16900)
* Revert "There can be only one localhost"

This reverts commit 5f1bbb4fcd.
this broke several usages of localhost, see #16882, #16898 and #16886

* ensure there is only 1 localhost

fixes #16886, #16882 and #16898

- make sure localhost exists before returning it
- optimzed host caching
- ensure we always return a host object
2016-08-02 10:39:00 -04:00
James Cammarata d8a3feb976 Fix bug where getting role vars does not follow the dep chain
This bug was introduced in 3ced6d3, where getting vars from a role
did not follow the dep chain. This was originally hidden by the fact
that we got vars twice (from the block and from the roles directly).

Fixes #16729
2016-07-19 16:59:11 -04:00
James Cammarata 83e4a4048b Fix the way pull localhosts out of inventory for delegate_to
This patch corrects the way we look in the inventory hosts list for
implicit localhost entries when localhost aliases are used.

Fixes #16568
2016-07-04 11:25:53 -05:00
James Cammarata b51d0dd69b Fix hostvars lookup of locahost based on 5f1bbb4 2016-06-08 14:14:02 -05:00
James Cammarata 068e447fda Further tweaks to variable precedence to make it match our docs
Also removes looking at role variables from the Block, as those are
merged in separately via VariableManager
2016-06-07 12:08:01 -05:00
James Cammarata b37b51dcea Fix variable precedence issue where set facts beat role params
Also updates doc on variable precedence, as it was incorrect for the
order of play vars/vars_prompt/vars_files in relation to set_fact and
registered variables.

Fixes #14702
Fixes #14826
2016-06-07 11:25:02 -05:00
James Cammarata add2d1947e Fix recursive call of _load_inventory_file() in VariableManager
Fixes #16128
2016-06-06 11:25:15 -05:00
James Cammarata 273a965dcc Fix the way host and group vars are loaded
* In the VariableManager, we were not properly tracking if a file
  had already been loaded, so we continuously append data to the end
  of the list there for host and group vars, meaning large sets of data
  are duplicated multiple times
* In the inventory, we were merging the host/group vars with the vars
  local to the host needlessly, as the VariableManager already handles that.
  This leads to needless duplication of the data and makes combining the
  vars in VariableManager take even longer.
2016-06-02 16:52:37 -04:00
James Cammarata 3e03004b6f Use get_dep_chain() instead of directly using a blocks _dep_chain
Child blocks (whether nested or via includes) don't get a copy of the
dependency chain, so the above method should be used to ensure the block
looks at its parents dep chain.

Fixes #15996
2016-05-26 15:48:07 -04:00
James Cammarata 33de7707c9 Do not include params when getting role vars in certain situations
In VariableManager, we fetch the params specifically in the next step,
so including them in the prior step is unnecessary and could lead to things
being overridden in an improper order.

In Block, we should not be getting the params for the role as they are
included earlier via the VariableManager.

Fixes #14411
2016-05-11 13:17:32 -04:00
Matthew Stoltenberg ddd9f92255 add repr for hostvars (#15793)
* allows passing full hostvars to a module
2016-05-10 10:23:10 -04:00
James Cammarata a0dea1ac35 Fix var precedence bug introduced in ff9f5d7d
Fixes #14067
2016-04-19 12:40:31 -04:00
nitzmahone eac4cf1177 fix options_vars iteration for py3 2016-04-07 19:16:15 -07:00
Strahinja Kustudić eed6cf5dad Adds 'ansible_check_mode' boolean magic variable
* Makes it possible to pass any options variable to VariableManager
  by changing `load_options_vars(options)` in `lib/ansible/utils/vars.py`
2016-04-08 00:26:02 +02:00
James Cammarata 0f2b1244d2 Make sure VariableManager has a view of HostVars
Fixes #15261
2016-04-05 11:32:28 -04:00
Brian Coca 5dfc013e18 Merge pull request #14971 from bcoca/clear_facts
meta: clear_facts
2016-03-30 19:09:38 -07:00
James Cammarata 4fc0aadd84 Only combine vars in include_vars if both are mappings
Fixes #15184
2016-03-30 11:11:26 -04:00
James Cammarata f427955640 Make default var lookup for tasks follow the dep chain
Related to #14296 (@jjshoe's follow-up example on that issue)
2016-03-24 09:51:36 -04:00
James Cammarata 9d2fe2fb2c Fixing role param precedence issues
* Make role param resolution follow the role dependency chain, rather
  than using all roles within the play
* Also move params to be merged in after role vars in general, to match
  our variable precedence rules
* Changes to the way var lookup is done in role helper methods for
  get_vars(), get_inherited_vars(), and get_role_params() to make the
  above possible without trampling on previously merged vars

Fixes #14472
Fixes #14475
2016-03-20 11:09:32 -04:00
Brian Coca 61f05c2e8d strip keys recursively 2016-03-16 13:18:29 -07:00
Brian Coca 09699d0cd0 meta: clear_facts
draft implementation
2016-03-14 21:24:26 -07:00
Brian Coca d6546a7513 make sure group_names is always sorted
this makes it consistent with previous ansilbe versions and other
paths that create the group_names variable
2016-03-04 13:12:35 -05:00
Brian Coca a327420b67 properly combine vars if merge is set
hash_behaviour = merge now also applies to include vars
2016-02-12 00:55:17 -05:00
Brian Coca bbdfaf0522 move hostvars.vars to vars
this fixes duplication under hostvars and exposes all vars in the vars dict
which makes dynamic reference possible on 'non hostvars'
2015-12-22 00:24:35 -05:00
Yannig Perré a4674906c6 Merge role params into variables separately from other variables
Fixes #13617
2015-12-21 13:03:09 -05:00
James Cammarata 4426b7f6e0 fix sorting of groups for host vars
Fixes #13371
2015-12-03 14:23:02 -05:00
James Cammarata a183972477 Don't use play vars in HostVars
Fixes #13398
2015-12-02 14:16:47 -05:00
James Cammarata cc36eedf76 Ensure port is (re)set for delegated-to hosts
Fixes #13265
2015-11-30 14:40:23 -05:00
James Cammarata fbc9553bd4 Use text_type instead of unicode 2015-11-30 10:33:36 -05:00
James Cammarata c6a30f7000 Make sure the uuid in vars is string 2015-11-30 10:26:12 -05:00
James Cammarata dfa576b037 Merge pull request #13307 from Yannig/devel_fix_big_include_vars
Fix for https://github.com/ansible/ansible/issues/13221
2015-11-29 23:14:03 -05:00
James Cammarata 5b6162a166 Re-adding role_name/role_uuid variables 2015-11-28 09:08:24 -05:00
Yannig Perré 5227c6bb52 Do not copy variable_manager each time. Instead, keep host and local variable_manager sync.
Fix https://github.com/ansible/ansible/issues/13221
2015-11-28 14:58:33 +01:00
James Cammarata 91500f8f5f Fix include param precedence in variable manager 2015-11-19 09:01:51 -05:00
James Cammarata 180159b01d Adding vars back in and trying to add a little more speed by avoiding copies 2015-11-17 14:44:46 -05:00
James Cammarata f10d2c57c8 Restoring templating of hostvars returned by __getitem__ 2015-11-17 10:19:56 -05:00
James Cammarata 9f31c073fe Fixing a few bugs in the HostVars performance areas
* Also refresh inventory in the HostVars manager process when things
  are changed via add_host/group_by
* Raise j2undefined rather than return it
2015-11-17 09:15:10 -05:00
James Cammarata 5cbeab5a3c Performance improvements for HostVars and some bugfixes 2015-11-16 17:15:52 -05:00
nitzmahone 3bb1f7a561 hostvars should return j2 undefined as instance, not type
Looks like someone forgot to create an instance of undefined here- we were returning the undefined type object, which broke all the undefined checks.

Added an integration test around add_host that will catch this (separate PR to follow)
2015-11-16 09:58:23 -08:00
Toshio Kuratomi 62979efa14 Finish up plugin porting to global display
Also remove display = display which does nothing
2015-11-11 10:44:23 -08:00
James Cammarata 566144461f Remove internal use of display and use the top-level import 2015-11-10 12:02:29 -05:00
James Cammarata 09994c18f3 Correct logic around determining environment variable existence
Fixes #13105
2015-11-09 21:04:24 -05:00
James Cammarata ccbcfcddfc Move where we add environment and make it conditional on not existing
Also displays a warning now, because users should not be using that variable
name as it causes a collision with the internal variable of the same name.
2015-11-09 16:28:54 -05:00
James Cammarata 937584cd52 Ensure 'environment' is set in the magic variables
Fixes #13033
2015-11-09 14:50:39 -05:00
James Cammarata 7e04947599 Be more selective about what variables we cache during templating
Fixes #13087
2015-11-09 14:07:48 -05:00
James Cammarata dd4ca66e1e Make sure hostvars cache is recreated during serialization 2015-11-06 22:21:50 -05:00
James Cammarata 5040abaaf2 Merge branch 'devel_cache_for_do_template_call' of https://github.com/Yannig/ansible into Yannig-devel_cache_for_do_template_call 2015-11-06 22:04:52 -05:00
James Cammarata 63c47fb271 Fixing up performance 2015-11-05 16:22:37 -05:00
Brian Coca 483c25a609 Revert "switched host patterns to use sets, simplified logic which now uses buitins"
need this to be lists, for order and other considerations
This reverts commit 8e2f0b3f2c.
2015-11-05 15:41:17 -05:00
Brian Coca 8e2f0b3f2c switched host patterns to use sets, simplified logic which now uses buitins 2015-11-05 15:02:06 -05:00
Yannig Perre ccbdd6229a Use static vars when computing host vars known to be static (inventory_hostname, inventory_dir etc.). 2015-11-04 22:16:14 +01:00
Yannig Perré 30f827d92d Fix python3 test. 2015-11-03 11:52:09 +01:00
Yannig Perré 4a8d1703d4 New patch against hostvars.py. With this patch, Ansible run lose 50% of time.
Little rewrite of previous patch to use sha1 signature.
Use fail_on_undefined to compute sha1 signature.
2015-11-02 21:04:20 +01:00
Toshio Kuratomi 4203850d1a Break apart a looped dependency to show a warning when parsing playbooks
Display a warning when a dict key is overwritten by pyyaml
Fixes #12888
2015-10-27 12:39:42 -07:00
James Cammarata cace91df56 Allow hostvars cache to be invalidated so hostvars contain all hosts after add_host
Fixes #12925
2015-10-27 14:12:17 -04:00
Brian Coca 7ebfe72015 added missing string_types 2015-10-24 15:49:30 -04:00
Brian Coca 8ce864db6f sanatize results 2015-10-24 15:23:12 -04:00
James Cammarata 498c27d09a Re-adding role_names magic variable
Fixes #12863
2015-10-22 11:06:47 -04:00
Brian Coca 08c9e70307 made display part of variable manager and corrected reference to it
fixes #12859
2015-10-21 22:29:27 -04:00
Toshio Kuratomi 99e7bb35c1 Simplify UnsafeProxy as we don't need to use it for byte strings, only text 2015-10-20 10:57:23 -07:00
Toshio Kuratomi f34b55ac2b Add python3-compat boilerplate to all .py files in lib/ansible 2015-10-19 18:36:19 -07:00
James Cammarata 0bbe9d5bd0 Make hostvars json/yaml serializable in filters
Fixes #12615
2015-10-18 10:09:05 -04:00
Toshio Kuratomi baa309309d Bundle a new version of python-six for compatibility along with some code to make it easy for distributions to override the bunndled copy if they have a new enough version. 2015-10-16 08:21:28 -07:00
Brian Coca aee6de5a76 group_names should not include implied 'all', fixes #12763 2015-10-15 11:46:43 -04:00
Marius Gedminas e511d6d458 Make UnsafeProxy work on Python 3
Instead of str and unicode we have to deal with bytes and str.
2015-10-13 09:41:08 +03:00
Marius Gedminas af6a316d9b Remove unused import
Fixes ImportError: No module named '__builtin__' on Python 3.
2015-10-13 09:37:48 +03:00
James Cammarata cd2cb178ae Totally rework the way UnsafeProxy does things 2015-10-12 18:13:09 -04:00
James Cammarata d15fcf05f0 Fix bug in the way temp vars were created for templating vars_files
Fixes #12711
2015-10-12 12:17:45 -04:00
Ben Keith 1f82971283 Adding the inventory_file var back 2015-10-10 21:54:28 -04:00
Matthew Jones 7da478a4f4 Fix an issue where cache plugins weren't updated
The first call to persisting facts would work due to the assignment of a
MutableMapping calling __setitem__ but subsequent module fact data would
not be propogated to the fact cache plugins because update() doesn't
invoke __setitem__.  This changes the behavior a little bit and ensures
set() is called on cache plugins.
2015-10-09 11:16:41 -04:00
James Cammarata d91b3c6b9d Make sure magic variables are available when templating vars_files
Also does some reorganization/cleanup on the magic vars/delegated
variable generation portions of VariableManager to make the above
possible.

Fixes #12633
2015-10-05 16:32:58 -04:00
James Cammarata 3bd71d0619 Use delegated_to field for play context remote_addr, if none other exists
Fixes #12527
2015-09-30 09:27:29 -04:00