Commit Graph

689 Commits (60d134aeaf0feb963e5d35c67fc396d461c528bb)

Author SHA1 Message Date
Brian Coca 960388143f
faster config loading (#48333)
* faster config loading

  - used already loaded module var instead of doc functions
  - add preload to populate config defs cache
  - avoid debug work when not in debug
  - generators, force consumption
2019-01-23 12:06:54 -05:00
Martin Krizek 3dc5ba6186
Remove unnecessary calls to cache_block_tasks noop (#50715) 2019-01-17 08:29:26 +01:00
Brian Coca 119b65f919
Clarify clear facts (#50667)
* Revert "avoid x2 setting of set_fact when 'cacheable' (#50564)"

This reverts commit 207848f354.

* clarify clear_facts with set_fact cacheable

 revert previous 'fix' as it will break playbooks by changing precedence
 opted to leave current behaviour but document it on both plugins to mitigate confusion

 fixes #50556

 also fix grammer, add comment, remove unused e
2019-01-15 15:20:33 -05:00
Abhijeet Kasurde db8702cdb8 Close all open filehandle (#50544)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-01-11 10:14:08 -05:00
Martin Krizek 4c72ef124b tqm._initialize_notified_handlers() is noop now (#50713) 2019-01-09 10:15:11 -05:00
Brian Coca 207848f354 avoid x2 setting of set_fact when 'cacheable' (#50564)
* avoid x2 setting of set_fact when 'cacheable'

  fixes #50556

* ammend docs to new behaviour

* added period
2019-01-08 10:12:49 -05:00
Abhijeet Kasurde bc6cd13874
Handle exception raised in recursive_finder API (#49590)
User module can contain Indentation errors or syntax errors.
Handle AST exceptions rather than showing traceback while importing such module.

Fixes: #21707

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-01-04 10:33:37 +05:30
Toshio Kuratomi afdbb0d9d5 Save the command line arguments into a global context
* Once cli args are parsed, they're constant.  So, save the parsed args
  into the global context for everyone else to use them from now on.
* Port cli scripts to use the CLIARGS in the context
* Refactor call to parse cli args into the run() method
* Fix unittests for changes to the internals of CLI arg parsing
* Port callback plugins to use context.CLIARGS
  * Got rid of the private self._options attribute
  * Use context.CLIARGS in the individual callback plugins instead.
  * Also output positional arguments in default and unixy plugins
  * Code has been simplified since we're now dealing with a dict rather
    than Optparse.Value
2019-01-03 18:12:23 -08:00
Nathaniel Case 1829a72885
Allow persistent connection plugins to queue messages back to ansible-connection (#49977)
* Connections can queue messages to be returned from ansible-connection

* Provide fallback for invalid display level

* Strip display from plugins

* Route messages through helper method to try to avoid improper appends
2018-12-19 10:54:42 -05:00
Toshio Kuratomi 3fba006207 Update bare exceptions to specify Exception.
This will keep us from accidentally catching program-exiting exceptions
like KeyboardInterupt and SystemExit.
2018-12-16 15:03:19 -08:00
Jordan Borean 190d1ed7f1 win become: refactor and add support for passwordless become (#48082)
* win become: refactor and add support for passwordless become

* make tests more stable

* fix up dep message for Load-CommandUtils

* Add further check for System impersonation token

* re-add support for become with accounts that have no password

* doc fixes and slight code improvements

* fix doc sanity issue
2018-12-12 17:15:25 -08:00
Nathaniel Case 02432565cd
Remove cliconf from httpapi connection (#46813)
* Bare minimum rip out cliconf

* nxapi changeover

* Update documentation, move options

* Memoize device_info

* Gratuitous rename to underscore use of local api implementation

Fixup eos module_utils like nxos

* Streamline version and image scans

* Expose get_capabilities through module_utils

* Add load_config to module_utils

* Support rpcs using both args and kwargs

* Add get_config for nxos

* Add get_diff

* module context, pulled from nxapi

We could probably do this correctly later

* Fix eos issues

* Limit connection._sub_plugin to only one plugin
2018-12-11 16:26:59 -05:00
Matt Martz 2a469fd959 Consolidate handler tracking (#49338)
* Consolidate handler tracking
  -  Remove unused code. ci_complete
  - unit test fixes. ci_complete
  - Restore previous behavior of matching a single handler
  - when notifying a host for a handler, return True if it was added, False otherwise, to reduce copied logic
  - rename funcitons for clarity. ci_complete
  - Remove handler logic for static includes which was disabled previously
2018-12-11 10:58:19 -05:00
Matt Martz 9007dbec2f
Add info about loop based on jinja2 loop var (#42134)
* Add info about loop based on jinja2 loop var

* ansible_loop

* Update test count

* Add extended loop_control that defines whether ansible_loop should be added

* Extended needs to be defaulted

* Revert "Update test count"

This reverts commit f1e93ee469825f4cdcd90fb28667d29aa088275c.

* Add docs about loop_control.extended

* Add revindex and revindex0

* Document ansible_loop in special vars

* Add changelog fragment

* Add tests, change items to allitems so that dot notation works, fix logic error with previtem
2018-12-07 13:49:50 -06:00
Matt Martz 69d230f4bf Make squash deprecation messages less misleading (#49551) 2018-12-06 10:49:01 -05:00
Monty Taylor 824c72da5f Set play context on stdout callback plugin (#35910)
The loop on self._callback_plugins does not include the stdout callback,
so the stdout_callback never has set_play_context called.
2018-11-29 23:15:56 -05:00
Matt Davis 8c1f701e6e
split PS wrapper and payload (CVE-2018-16859) (#49142)
* prevent scriptblock logging from logging payload contents
* added tests to verify no payload contents in PS Operational event log
* fix script action to send split-aware wrapper
* fix CLIXML error parser (return to -EncodedCommand exposed problems with it)
2018-11-26 15:28:21 -08:00
Matt Martz 9773a1f289
Add a Singleton metaclass, use it with Display (#48935)
* Add a Singleton class, use it with Display

* update six import

* Move remaining failes to display singleton

* Fix rebase issues

* Singleton improvements

* Add code-smell for 'from __main__ import display'. ci_complete

* s/self/cls/g

* Add docs for no-main-display

* Address linting issues

* Add changelog fragment. ci_complete

* Implement reentrant lock for class instantiation in Singleton

* Add Display singleton porting guide
2018-11-20 17:06:51 -06:00
Matt Martz 64bb54dab4 Catch per loop label templating exceptions. Fixes #48879 (#48896) 2018-11-20 15:29:44 -05:00
Abhijeet Kasurde a6fde8aef9 Add timeout information into async failure message
Add some extra information to an async failure message to reflect the
actually timeout value of the failure.

Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-11-20 08:46:27 -05:00
Martin Krizek 869d866aa1 display.error on no matching --start-at-task (#45221) 2018-11-15 19:35:51 -05:00
Jordan Borean 4fc013a7fa powershell - fix async for Ansible.Basic (#48404)
* powershell - fix async for Ansible.Basic

* moved old out outside of the try block
2018-11-13 12:08:27 -08:00
Brian Coca b047591dee fix tagged implicit gathering (#48294)
* fix tagged implicit gathering

 - None is not tHe default anymore, its an empty list

* Update changelogs/fragments/fix_taggged_gather.yml

Co-Authored-By: bcoca <bcoca@users.noreply.github.com>
2018-11-09 13:41:07 +00:00
Matt Martz 0d068f1e3a
Support list of dicts with omit. Fixes #45907 (#45923) 2018-11-07 13:41:22 -06:00
Jordan Borean 501acae5ab
Added basic equivalent to PowerShell modules (#44705)
* Added basic equivalent to PowerShell modules

* changes based on latest review

* Added tests

* ignore sanity test due to how tests are set up

* Changes to work with PSCore

* Added documentation and change updated more modules

* Add some speed optimisations to AddType

* fix some issues in the doc changes

* doc changes
2018-11-07 10:53:17 +10:00
Brian Coca 4ec2423a04
make fact gathering obey play tags (#44717)
fixes #44251
2018-11-06 13:19:44 -05:00
Matt Martz 77d32b8f57
Don't use the task for a cache, return a special cache var (#47243)
* Don't use task to cache loop results, use hostvars. Fixes #47207

* Avoid a race condition, supply _ansible_loop_cache through get_vars directly

* Add tests

* Add changelog fragment

* Remove unnecessary copy

* Remove unnecessary host from _get_delegated_vars signature
2018-10-18 15:25:43 -05:00
Matt Martz 02f4d0a57f
Ensure that an empty literal list with loop skips the task (#47129) 2018-10-16 13:35:10 -05:00
Eitan Adler 063a9dd911 Reduce noise in warnings from ansible (#40754)
When python is compiled in debug mode, or certain command line flags are
passed, python issues helpful warnings to let users know of files opened
by not closed.

This can be fairly spammy on an ansible run.  This change reduces the
number of such warnings by a factor of 10.
2018-10-09 18:53:03 -04:00
Matt Clay a11f631ee4 Python 3.8 collections compatibility fixes.
Includes a new pylint blacklist plugin to prevent regressions.
2018-10-08 11:26:37 -07:00
Jordan Borean e972287c35 win_exec: refactor PS exec runner (#45334)
* win_exec: refactor PS exec runner

* more changes for PSCore compatibility

* made some changes based on the recent review

* split up module exec scripts for smaller payload

* removed C# module support to focus on just error msg improvement

* cleaned up c# test classifier code
2018-10-02 15:55:53 -07:00
Nathaniel Case cdaf5bd806
Add configuration to override location of ansible-connection (#46128)
* Add configuration to override location of ansible-connection

* Check var or argv[0] before $PATH
2018-10-01 14:29:59 -04:00
Nathaniel Case 406b59aeba
Move persistent connections to only use registered variables (#45616)
* Try to intuit proper plugins to send to ansible-connection

* Move sub-plugins to init so that vars will be populated in executor

* Fix connection unit tests
2018-09-20 09:56:43 -04:00
Jordan Borean 5c73d4f4bd
async: use async_dir for the async results file directory (#45461)
* win async: use async_dir for the async results file directory

* tried to unify POSIX and PowerShell async implementations of async_dir

* fix sanity issue
2018-09-20 19:37:54 +10:00
Nathaniel Case 3d0f19d8cd
Exclude the whole ansible_failed_task block (#45587)
It has lots of FieldAttributes that won't serialize.
Also the connection doesn't care.
2018-09-12 21:17:24 -04:00
Jordan Borean d81249994e
win_script: add support for become and centralise exec wrapper builder (#45582)
* win_script: add support for become and centralise exec wrapper builder

* satisfying the pep8 gods

* do not scan for module dependencies when running as a script
2018-09-13 08:50:13 +10:00
Brian Coca 4dea9d84ed fix issue with no_log in py3
the error percolates differently from undefined vars in jinja2,
so fallback to use play_context as it is not involved in current templating
2018-08-27 11:57:02 -07:00
Toshio Kuratomi 55503004ec Fix recursive_finder so it doesn't re-read files multiple times
This closes a corner case where recursive_finder could read a file
multiple times.
2018-08-24 15:37:13 -07:00
Toshio Kuratomi 14b3ae1120 Fix new AnsiballZ code to always include module_utils/basic
The AnsiballZ optimization which only uses one pyton interpreter
currently monkeypatches the arguments into a global argument in module_utils
so we need to always include basic.py.  In the future we should fix this
so that it monkeypatches its own file.  That way we won't need to always
include basic.py
2018-08-24 15:37:13 -07:00
Ryan Brown 4c8808ec9d
Extend `module_defaults` by adding default groups for cloud modules (#44127)
Extends `module_defaults` by adding a prefix to defaults `group/` which denotes a builtin list of modules. Initial groups are: `group/aws`, `group/azure`, and `group/gcp`
2018-08-22 21:33:27 -04:00
Brian Coca bda074d34e
fix tempating issues with no_log and loops (#44468)
* fix tempating issues with no_log and loops

 - task is no log if any item is
 - added test cases

fixes #43294
2018-08-21 21:53:56 -04:00
Matt Martz 9b2baebe64
Don't use copy.deepcopy in high workload areas, use naive_deepcopy (#44337)
* Don't use copy.deepcopy in high workload areas, use deepishcopy. ci_complete

* Add tests

* Add changelog fragment

* rename to naive_deepcopy and add extra docs

* Rename to module_response_deepcopy and move to vars/clean
2018-08-20 15:08:29 -05:00
Matt Davis 6b81c36533
restore task arg splatting (#43798)
* restore task arg splatting

* reverts #41804
* supersedes #41295
* fixes #42192
* after lots of discussion amongst the core team, we decided to preserve this feature, clarify the runtime warnings/docs, and prioritize a path toward fixing the underlying behavior that causes this feature to be insecure (un-namespaced facts).

* update faq text

note that warning is disabled when inject_facts_as_vars is

* wordsmithing FAQ entry
2018-08-14 12:58:00 -07:00
Matt Martz 17c89d1ffa
Remove unused rslt_q, rename the one queue to final_q everywhere (#43894)
* Remove unused rslt_q, rename the one queue to final_q everywhere

* Test update
2018-08-13 13:43:28 -05:00
Sam Doran 6d38167d49
Only template values in vars_prompt rather than all vars (#39304)
* Only template values in vars_prompt rather than all vars

This allows the use of variables in vars_prompt fields but allows variables entered in the prompt to affect play vars rather than throwing an undefined error.

Only post validate if there was a vars_prompt

* Add tests for vars_prompt
2018-08-13 12:54:31 -04:00
Nathaniel Case f221105882
Prevent data being truncated over persistent connection socket (#43885)
* Change how data is sent to the persistent connection socket.

We can't rely on readline(), so send the size of the data first. We can
then read that many bytes from the stream on the recieving end.

* Set pty to noncanonical mode before sending

* Now that we send data length, we don't need a sentinel anymore

* Copy socket changes to persistent, too

* Use os.write instead of fdopen()ing and using that.

* Follow pickle with sha1sum of pickle

* Swap order of vars and init being passed to ansible-connection
2018-08-10 09:26:58 -04:00
Brian Coca 2e9a7e2564 remove deprecated 2018-08-01 15:17:28 -04:00
Brian Coca 21fc6a0c38 nicer error message on multiprocessing fail
fixes #43090
2018-08-01 15:14:23 -04:00
Nathaniel Case 9169daa326
\r does not round-trip the local socket, escape and restore on the other side (#43507) 2018-08-01 08:44:39 -04:00
Matt Martz a7f45c0660 Add basedir to ansible_search_path before running loops. Fixes #43302 (#43431) 2018-07-31 11:51:32 -04:00