Commit Graph

533 Commits (a4131197e04c6663ba504f63f8d2cd3b9e1936c8)

Author SHA1 Message Date
Dag Wieers 630185cb20 ansible/executor: PEP8 compliancy (#24695)
- Make PEP8 compliant
2017-05-30 18:13:53 +01:00
Will Thames 30fec68931 Expose variable_manager to PlayIterator
`variable_manager` is passed to the constructor of `PlayIterator`,
and then used to access the fact cache when fact gathering.
Make `_variable_manager` an attribute of the `PlayIterator` class
2017-05-24 21:59:13 -04:00
Brian Coca 8f97aef1a3 Transition inventory into plugins (#23001)
* draft new inventory plugin arch, yaml sample

 - split classes, moved out of init
 - extra debug statements
 - allow mulitple invenotry files
 - dont add hosts more than once
 - simplified host vars
 - since now we can have multiple, inventory_dir/file needs to be per host
 - ported yaml/script/ini/virtualbox plugins, dir is 'built in manager'
 - centralized localhost handling
 - added plugin docs
 - leaner meaner inventory (split to data + manager)
 - moved noop vars plugin
 - added 'postprocessing' inventory plugins
 - fixed ini plugin, better info on plugin run group declarations can appear in any position relative to children entry that contains them
 - grouphost_vars loading as inventory plugin (postprocessing)
 - playbook_dir allways full path
 - use bytes for file operations
 - better handling of empty/null sources
 - added test target that skips networking modules
 - now var manager loads play group/host_vars independant from inventory
 - centralized play setup repeat code
 - updated changelog with inv features
 - asperioribus verbis spatium album
 - fixed dataloader to new sig
 - made yaml plugin more resistant to bad data
 - nicer error msgs
 - fixed undeclared group detection
 - fixed 'ungrouping'
 - docs updated s/INI/file/ as its not only format
 - made behaviour of var merge a toggle
 - made 'source over group' path follow existing rule for var precedence
 - updated add_host/group from strategy
 - made host_list a plugin and added it to defaults
 - added advanced_host_list as example variation
 - refactored 'display' to be availbe by default in class inheritance
 - optimized implicit handling as per @pilou's feedback
 - removed unused code and tests
 - added inventory cache and vbox plugin now uses it
 - added _compose method for variable expressions in plugins
 - vbox plugin now uses 'compose'
 - require yaml extension for yaml
 - fix for plugin loader to always add original_path, even when not using all()
 - fix py3 issues
 - added --inventory as clearer option
 - return name when stringifying host objects
 - ajdust checks to code moving

* reworked vars and vars precedence
 - vars plugins now load group/host_vars dirs
 - precedence for host vars is now configurable
 - vars_plugins been reworked
 - removed unused vars cache
 - removed _gathered_facts as we are not keeping info in host anymore
 - cleaned up tests
 - fixed ansible-pull to work with new inventory
 - removed version added notation to please rst check
 - inventory in config relative to config
 - ensures full paths on passed inventories

* implicit localhost connection local
2017-05-23 17:16:49 -04:00
Matt Martz 1eed6c5b3d Convert some more base64 strings to text for winrm, to support async and become (#24796) 2017-05-18 16:06:51 -07:00
Matt Davis 569377e951 add segfault detection to TQM
* prevents indefinite hangs when a worker segfaults
2017-05-18 14:59:07 -04:00
Matt Martz e7d8ebf080 Base changes required to allow winrm to work on py3 (#24744) 2017-05-17 14:26:32 -07:00
Ricardo Carrillo Cruz 14b1405ee4 Fix conn_type assignment (#24697)
This should be a single equal, not a double-one
2017-05-16 23:14:17 +02:00
Brian Coca c50cf22d52 deal with null/none connections
fixes #23621
pushed 'connection resolution' to play_context
override fieldattribute getter
2017-05-11 15:46:14 -04:00
Toshio Kuratomi 1c05ed7951 Fix circular import with unsafe_proxy, template, and vars
template/__init__.py imported unsafe_proxy from vars which caused
vars/__init__.py to load.  vars/__init__.py needed template/__init__.py
which caused issues.  Loading unsafe_proxy from another location fixes
that.
2017-04-28 15:35:30 -07:00
Toshio Kuratomi 500de1f557 Remove hack for backwards compatible v2_playbook_on_start callback
Just after release of 2.0.0 (in 2.0.0.1) we had a change to the API of
callbacks without bumping the API version.  We added the playbook to the
arguments passed to the callbacks.

This wasn't in the Tower callback at the time.  In order to prevent
breaking that callback we added a temporary hack to inspect the
callback's API to decide if we needed to call it with arguments or not.

We scheduled the hack for removal in January 2017.  Since that's now
past, removing the hack.

Change signed off by matburt on the Tower side.
2017-04-28 13:44:43 -07:00
Brian Coca 31045d58c6 added versions to dep notices (#24019)
* added versions to dep notices

* pep7

* string
2017-04-27 12:48:31 -04:00
Brian Coca e29dc49a49 moved to exceptions for basic skip/fails
better handling of checkmode and async
fix test to follow new flow control
2017-04-25 14:48:56 -04:00
Matt Davis 02ed223d49 template hostvars for set_host_overrides (#23839)
* fixes #23586
* temporary solution until connection-specific vars are handled by play_context
2017-04-20 20:33:48 -07:00
Brian Coca b4b96143fe also skip polling when async failed 2017-04-20 15:47:24 -04:00
Brian Coca 586fcae398 tolerate 'batch' systems that mess with stdin (#23596)
* tolerate 'batch' systems taht mess with stdin

fixes #23541

* have pause on windows tolerate devnull

* tuplie
2017-04-18 11:39:49 -04:00
Toshio Kuratomi a06014663c display path to module_utils files in verbosity vvv. 2017-04-17 15:08:14 -07:00
Brian Coca 952e842720 less code 2017-04-13 11:21:11 -04:00
smaftoul 9812d211e7 typo: psuedo -> pseudo (#23177) 2017-03-31 10:48:29 -04:00
Matt Davis 73f50b4f9f fix Windows env handling (#22927)
* fixes #22441
* fixes #22655
* moves all env handling into the exec wrapper; this should work for everything but raw, which is consistent with non-Windows.
2017-03-23 17:48:15 -07:00
Toshio Kuratomi 2fff690caa Update module_utils.six to latest (#22855)
* Update module_utils.six to latest

We've been held back on the version of six we could use on the module
side to 1.4.x because of python-2.4 compatibility.  Now that our minimum
is Python-2.6, we can update to the latest version of six in
module_utils and get rid of the second copy in lib/ansible/compat.
2017-03-23 13:35:05 -07:00
Matt Martz 3164e8b561 E501 fixes (#22879) 2017-03-22 20:50:28 -05:00
Matt Martz 02f66b9369 E1 legacy pep8 fixes (#21933)
* E1 pep8 fixes

* e111 fix for rds.py
2017-03-21 21:19:40 -05:00
Brian Coca dd8d699981 namespace facts (#18445)
* namespace facts

always namespace facts, make the polluting of 'main' conditional on config

* updated to 2.4

* Update intro_configuration.rst
2017-03-15 17:12:16 -07:00
Brian Coca 7ad6ce7ea1 moved network module magic from hardcoded to conf 2017-03-09 21:49:02 -05:00
Toshio Kuratomi 89bf14e467 Change warning message as advised by bcoca 2017-02-27 09:41:41 -08:00
Toshio Kuratomi 49db03c384 Fix for traceback when we encounter non-utf8 characters in diff
We can diff non-utf8 files (as part of copy, for instance) but when we
try to turn the bytes into text for display, the characters cause
a traceback.  Since diff output is only informational, we can replace
those problematic bytes with replacement characters.  We do not want to
do this to other fields because those fields may be used inside of the
playbook (for templating another variable or matching in a conditional).

Fixes #21803
Fixes #21804
2017-02-24 12:18:32 -08:00
Brian Coca 9ac697a155 simplified persistent connection quandry 2017-02-24 15:15:18 -05:00
Adrian Likins 8830cde28d Fix 'task name is not templated in retry callback' (add task_name property to TaskResult) (#21214)
Fix 'task name is not templated in retry callback'

Add a task_name property to TaskResult that knows to
check in TaskResult._task_fields.

Add integration test for v2_retry_runner callback

Fixes #18236
2017-02-24 12:33:24 -05:00
James Cammarata 3d65482927 Stash post-validated fields of the task in the TaskResult
This allows us to have a snapshot of the fields, which we can restore
on the pre-fork side so as to avoid having to re-template fields.
2017-02-21 14:35:13 -06:00
jctanner 40235d7b99 Skip fact gathering if the entire play was included via conditional and False (#21734)
Addresses #21528
2017-02-21 14:46:10 -05:00
Matt Davis 8527013fbe Complete rewrite of Windows exec wrapper (#21510)
* supports pipelining for faster execution
* supports become (runas), creates interactive subsession under WinRM batch logon
* supports usage of arbitrary module_utils files
* modular exec wrapper payload supports easier extension
* integrates async wrapper behavior for pipelined/become'd async
* module_utils are loaded as true Powershell modules, no more runtime modifications to module code
2017-02-17 00:09:56 -08:00
Toshio Kuratomi eeaec56ed5 Use isinstance instead of type() comparisons. isinstance is more robust 2017-02-16 16:34:43 -08:00
Peter Sprygada 48b02336ab code clean of old code from network modules updates (#21469)
* removes unused code
* removes module_utils/local.py
* removes plugins/action/network.py
* removes action_handler from connection plugins
* removes code to use action_handler in task_executor
* updates action plugins to subclass from normal
2017-02-15 12:46:30 -05:00
Brian Coca e9b009510f added fixme note 2017-02-10 08:46:33 -05:00
Peter Sprygada 21157aa1bc loads action handler for a group of network modules (#21194)
* loads action handler based on module group prefix for network modules
2017-02-09 14:26:09 -05:00
stephane 1b8c2a2b0b Restore "skipping: no hosts matched" message
In 159aa26b36
the result of _get_serialized_branches when no hosts were matched
changed from [[]] to []. Thus, the v2_playbook_on_no_hosts_matched
callback would not fire. Change the check so we get the error message
again. Fixes #17706
2017-02-08 16:03:29 -05:00
Brian Coca c86a17b7a0 refactoring async
- centralized skipping
- also fixed module name broken by previous refactor
- let action modules handle async processing
- moved async into base action class's module exec
- action plugins can now run final action as async
- actually skip copy if base skips
- fixed normal for new paths
- ensure internal stat is never async
- default poll to 10 as per docs
- added hint for callback fix on poll
- restructured late tmp, now a pipeline query
- moving action handler to connection as networking does
- fixed network assumption invocation is always passed
- centralized key cleanup, normalized internal var
- _supress_tmpdir_delete now in _ansible_xxx and gets removed from results
- delay internal key removal till after we use em
- nicer tmp removing, using existing methods
- moved cleanup tmp flag to mking tmp func
2017-02-06 19:32:00 -05:00
Toshio Kuratomi 1df7d95cec Module utils default path (#20913)
* Make the module_utils path configurable
* Add a config value to define the path site module_utils files
* Handle module_utils that do not have source as an error
* Make an integration test for module_utils envvar working
* Add documentation for the ANSIBLE_MODULE_UTILS config option/envvar
* Add it to the sample ansible.cfg
* Add it to intro_configuration.
* Also modify intro_configuration to place envvars on equal footing with
  the config options (will need to document the envvar names in the
  future)
* Also add the ANSIBLE_LIBRARY use case from
  https://github.com/ansible/ansible/issues/15432 so we can close out
  that bug.
2017-02-02 17:48:53 -08:00
Brian Coca f4b6b72c5b Warn reserved (#20567)
* removed unused vars

* added warning when using reserved vars

fixes #4816

* cleanup
2017-01-31 15:35:28 -05:00
Brian Coca e6f92bc3a9 reworded conditional skipping to not imply error
fixes #19068
2017-01-31 13:33:29 -05:00
Toshio Kuratomi 5c38f3cea2 Combine jimi-c and bcoca's ideas and work on hooking module-utils into PluginLoader.
This version just gets the relevant paths from PluginLoader and then
uses the existing imp.find_plugin() calls in the AnsiballZ code to load
the proper module_utils.

Modify PluginLoader to optionally omit subdirectories (module_utils
needs to operate on top level dirs, not on subdirs because it has
a hierarchical namespace whereas all other plugins use a flat
namespace).

Rename snippet* variables to module_utils*

Add a small number of unittests for recursive_finder

Add a larger number of integration tests to demonstrate that
module_utils is working.

Whitelist module-style shebang in test target library dirs

Prefix module_data variable with b_ to be clear that it holds bytes data
2017-01-31 09:41:24 -08:00
Matt Clay 10d9318de7 PEP 8 indent cleanup. (#20800)
* PEP 8 E121 cleanup.

* PEP 8 E126 cleanup.

* PEP 8 E122 cleanup.
2017-01-29 07:28:53 +00:00
Matt Clay 95789f3949 PEP 8 whitespace cleanup. (#20783)
* PEP 8 E271 whitespace cleanup.
* PEP 8 W293 whitespace cleanup.
* Fix whitespace issue from recent PR.
2017-01-27 15:45:23 -08:00
James Cammarata 9d549c3498 Fixing iterator bug related to reworking of end-of-role detection
Bug was introduced in cae682607
2017-01-19 10:10:07 -06:00
James Cammarata cae682607c Reworking the way end of role detection is done
Rather than trying to enumerate tasks or track an ever changing cur_role
flag in PlayIterator, this change simply sets a flag on the last block in
the list of blocks returned by Role.compile(). The PlayIterator then checks
for that flag when the cur_block number is incremented, and marks the role
as complete if the given host had any tasks run in that role.

Fixes #20224
2017-01-18 17:29:52 -06:00
Alvaro Aleman 1b2ad94496 Configurable fact path (#18147)
* Make fact_path configurable

* Add docs for fact_path

* Add tests for localfacts

* Default fact gathering settings in PlayContext
2017-01-12 10:49:04 -05:00
Toshio Kuratomi 03510ec4ce Allow OSError to skip scriptdir removal
On Ubuntu the scriptdir gets placed into sys.path.  This makes some
modules (copy) fail because the ansible module gets loaded instead of
the stdlib copy module.  So we remove scriptdir there.  Unfortunately,
the scriptdir code uses abspath().  When pipelining, abspath() has to
find the cwd.  On OSX, finding the cwd when that directory is not
executable by the user raises an OSError.  Since OSX does not suffer
from the scriptdir problem we're able to just skip scriptdir handling if
we get that exception.

Fixes #19729
2017-01-05 15:50:56 -08:00
Brian Coca 08e0f6ada5 allow modules to set custom stats (#18946)
can be per run or per host, also aggregate or not
set_stats action plugin as reference implementation
added doc stub
display stats in calblack
made custom stats showing configurable
2017-01-05 16:38:36 -05:00
James Cammarata ac00c8a660 Fixing an iteration bug introduced by fed079e4 2017-01-04 10:34:06 -06:00
James Cammarata fed079e4cb Fix role completion detection problem
When the same role is listed consecutively in a play, the previous role
completion detection failed to mark it as complete as it only checked to
see if the role changed.

This patch addresses that by also keeping track of which task in the role
we are on, so that even if the same role is encountered during later passes
the task number will be less than or equal to the last noted task position.

Related to #15409
2017-01-03 14:32:44 -06:00