Commit Graph

679 Commits (f2eef2d30b2a9ae91e51923e8fb81ccfd77d0019)

Author SHA1 Message Date
Jordan Borean 190d1ed7f1 win become: refactor and add support for passwordless become (#48082)
* win become: refactor and add support for passwordless become

* make tests more stable

* fix up dep message for Load-CommandUtils

* Add further check for System impersonation token

* re-add support for become with accounts that have no password

* doc fixes and slight code improvements

* fix doc sanity issue
2018-12-12 17:15:25 -08:00
Nathaniel Case 02432565cd
Remove cliconf from httpapi connection (#46813)
* Bare minimum rip out cliconf

* nxapi changeover

* Update documentation, move options

* Memoize device_info

* Gratuitous rename to underscore use of local api implementation

Fixup eos module_utils like nxos

* Streamline version and image scans

* Expose get_capabilities through module_utils

* Add load_config to module_utils

* Support rpcs using both args and kwargs

* Add get_config for nxos

* Add get_diff

* module context, pulled from nxapi

We could probably do this correctly later

* Fix eos issues

* Limit connection._sub_plugin to only one plugin
2018-12-11 16:26:59 -05:00
Matt Martz 2a469fd959 Consolidate handler tracking (#49338)
* Consolidate handler tracking
  -  Remove unused code. ci_complete
  - unit test fixes. ci_complete
  - Restore previous behavior of matching a single handler
  - when notifying a host for a handler, return True if it was added, False otherwise, to reduce copied logic
  - rename funcitons for clarity. ci_complete
  - Remove handler logic for static includes which was disabled previously
2018-12-11 10:58:19 -05:00
Matt Martz 9007dbec2f
Add info about loop based on jinja2 loop var (#42134)
* Add info about loop based on jinja2 loop var

* ansible_loop

* Update test count

* Add extended loop_control that defines whether ansible_loop should be added

* Extended needs to be defaulted

* Revert "Update test count"

This reverts commit f1e93ee469825f4cdcd90fb28667d29aa088275c.

* Add docs about loop_control.extended

* Add revindex and revindex0

* Document ansible_loop in special vars

* Add changelog fragment

* Add tests, change items to allitems so that dot notation works, fix logic error with previtem
2018-12-07 13:49:50 -06:00
Matt Martz 69d230f4bf Make squash deprecation messages less misleading (#49551) 2018-12-06 10:49:01 -05:00
Monty Taylor 824c72da5f Set play context on stdout callback plugin (#35910)
The loop on self._callback_plugins does not include the stdout callback,
so the stdout_callback never has set_play_context called.
2018-11-29 23:15:56 -05:00
Matt Davis 8c1f701e6e
split PS wrapper and payload (CVE-2018-16859) (#49142)
* prevent scriptblock logging from logging payload contents
* added tests to verify no payload contents in PS Operational event log
* fix script action to send split-aware wrapper
* fix CLIXML error parser (return to -EncodedCommand exposed problems with it)
2018-11-26 15:28:21 -08:00
Matt Martz 9773a1f289
Add a Singleton metaclass, use it with Display (#48935)
* Add a Singleton class, use it with Display

* update six import

* Move remaining failes to display singleton

* Fix rebase issues

* Singleton improvements

* Add code-smell for 'from __main__ import display'. ci_complete

* s/self/cls/g

* Add docs for no-main-display

* Address linting issues

* Add changelog fragment. ci_complete

* Implement reentrant lock for class instantiation in Singleton

* Add Display singleton porting guide
2018-11-20 17:06:51 -06:00
Matt Martz 64bb54dab4 Catch per loop label templating exceptions. Fixes #48879 (#48896) 2018-11-20 15:29:44 -05:00
Abhijeet Kasurde a6fde8aef9 Add timeout information into async failure message
Add some extra information to an async failure message to reflect the
actually timeout value of the failure.

Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-11-20 08:46:27 -05:00
Martin Krizek 869d866aa1 display.error on no matching --start-at-task (#45221) 2018-11-15 19:35:51 -05:00
Jordan Borean 4fc013a7fa powershell - fix async for Ansible.Basic (#48404)
* powershell - fix async for Ansible.Basic

* moved old out outside of the try block
2018-11-13 12:08:27 -08:00
Brian Coca b047591dee fix tagged implicit gathering (#48294)
* fix tagged implicit gathering

 - None is not tHe default anymore, its an empty list

* Update changelogs/fragments/fix_taggged_gather.yml

Co-Authored-By: bcoca <bcoca@users.noreply.github.com>
2018-11-09 13:41:07 +00:00
Matt Martz 0d068f1e3a
Support list of dicts with omit. Fixes #45907 (#45923) 2018-11-07 13:41:22 -06:00
Jordan Borean 501acae5ab
Added basic equivalent to PowerShell modules (#44705)
* Added basic equivalent to PowerShell modules

* changes based on latest review

* Added tests

* ignore sanity test due to how tests are set up

* Changes to work with PSCore

* Added documentation and change updated more modules

* Add some speed optimisations to AddType

* fix some issues in the doc changes

* doc changes
2018-11-07 10:53:17 +10:00
Brian Coca 4ec2423a04
make fact gathering obey play tags (#44717)
fixes #44251
2018-11-06 13:19:44 -05:00
Matt Martz 77d32b8f57
Don't use the task for a cache, return a special cache var (#47243)
* Don't use task to cache loop results, use hostvars. Fixes #47207

* Avoid a race condition, supply _ansible_loop_cache through get_vars directly

* Add tests

* Add changelog fragment

* Remove unnecessary copy

* Remove unnecessary host from _get_delegated_vars signature
2018-10-18 15:25:43 -05:00
Matt Martz 02f4d0a57f
Ensure that an empty literal list with loop skips the task (#47129) 2018-10-16 13:35:10 -05:00
Eitan Adler 063a9dd911 Reduce noise in warnings from ansible (#40754)
When python is compiled in debug mode, or certain command line flags are
passed, python issues helpful warnings to let users know of files opened
by not closed.

This can be fairly spammy on an ansible run.  This change reduces the
number of such warnings by a factor of 10.
2018-10-09 18:53:03 -04:00
Matt Clay a11f631ee4 Python 3.8 collections compatibility fixes.
Includes a new pylint blacklist plugin to prevent regressions.
2018-10-08 11:26:37 -07:00
Jordan Borean e972287c35 win_exec: refactor PS exec runner (#45334)
* win_exec: refactor PS exec runner

* more changes for PSCore compatibility

* made some changes based on the recent review

* split up module exec scripts for smaller payload

* removed C# module support to focus on just error msg improvement

* cleaned up c# test classifier code
2018-10-02 15:55:53 -07:00
Nathaniel Case cdaf5bd806
Add configuration to override location of ansible-connection (#46128)
* Add configuration to override location of ansible-connection

* Check var or argv[0] before $PATH
2018-10-01 14:29:59 -04:00
Nathaniel Case 406b59aeba
Move persistent connections to only use registered variables (#45616)
* Try to intuit proper plugins to send to ansible-connection

* Move sub-plugins to init so that vars will be populated in executor

* Fix connection unit tests
2018-09-20 09:56:43 -04:00
Jordan Borean 5c73d4f4bd
async: use async_dir for the async results file directory (#45461)
* win async: use async_dir for the async results file directory

* tried to unify POSIX and PowerShell async implementations of async_dir

* fix sanity issue
2018-09-20 19:37:54 +10:00
Nathaniel Case 3d0f19d8cd
Exclude the whole ansible_failed_task block (#45587)
It has lots of FieldAttributes that won't serialize.
Also the connection doesn't care.
2018-09-12 21:17:24 -04:00
Jordan Borean d81249994e
win_script: add support for become and centralise exec wrapper builder (#45582)
* win_script: add support for become and centralise exec wrapper builder

* satisfying the pep8 gods

* do not scan for module dependencies when running as a script
2018-09-13 08:50:13 +10:00
Brian Coca 4dea9d84ed fix issue with no_log in py3
the error percolates differently from undefined vars in jinja2,
so fallback to use play_context as it is not involved in current templating
2018-08-27 11:57:02 -07:00
Toshio Kuratomi 55503004ec Fix recursive_finder so it doesn't re-read files multiple times
This closes a corner case where recursive_finder could read a file
multiple times.
2018-08-24 15:37:13 -07:00
Toshio Kuratomi 14b3ae1120 Fix new AnsiballZ code to always include module_utils/basic
The AnsiballZ optimization which only uses one pyton interpreter
currently monkeypatches the arguments into a global argument in module_utils
so we need to always include basic.py.  In the future we should fix this
so that it monkeypatches its own file.  That way we won't need to always
include basic.py
2018-08-24 15:37:13 -07:00
Ryan Brown 4c8808ec9d
Extend `module_defaults` by adding default groups for cloud modules (#44127)
Extends `module_defaults` by adding a prefix to defaults `group/` which denotes a builtin list of modules. Initial groups are: `group/aws`, `group/azure`, and `group/gcp`
2018-08-22 21:33:27 -04:00
Brian Coca bda074d34e
fix tempating issues with no_log and loops (#44468)
* fix tempating issues with no_log and loops

 - task is no log if any item is
 - added test cases

fixes #43294
2018-08-21 21:53:56 -04:00
Matt Martz 9b2baebe64
Don't use copy.deepcopy in high workload areas, use naive_deepcopy (#44337)
* Don't use copy.deepcopy in high workload areas, use deepishcopy. ci_complete

* Add tests

* Add changelog fragment

* rename to naive_deepcopy and add extra docs

* Rename to module_response_deepcopy and move to vars/clean
2018-08-20 15:08:29 -05:00
Matt Davis 6b81c36533
restore task arg splatting (#43798)
* restore task arg splatting

* reverts #41804
* supersedes #41295
* fixes #42192
* after lots of discussion amongst the core team, we decided to preserve this feature, clarify the runtime warnings/docs, and prioritize a path toward fixing the underlying behavior that causes this feature to be insecure (un-namespaced facts).

* update faq text

note that warning is disabled when inject_facts_as_vars is

* wordsmithing FAQ entry
2018-08-14 12:58:00 -07:00
Matt Martz 17c89d1ffa
Remove unused rslt_q, rename the one queue to final_q everywhere (#43894)
* Remove unused rslt_q, rename the one queue to final_q everywhere

* Test update
2018-08-13 13:43:28 -05:00
Sam Doran 6d38167d49
Only template values in vars_prompt rather than all vars (#39304)
* Only template values in vars_prompt rather than all vars

This allows the use of variables in vars_prompt fields but allows variables entered in the prompt to affect play vars rather than throwing an undefined error.

Only post validate if there was a vars_prompt

* Add tests for vars_prompt
2018-08-13 12:54:31 -04:00
Nathaniel Case f221105882
Prevent data being truncated over persistent connection socket (#43885)
* Change how data is sent to the persistent connection socket.

We can't rely on readline(), so send the size of the data first. We can
then read that many bytes from the stream on the recieving end.

* Set pty to noncanonical mode before sending

* Now that we send data length, we don't need a sentinel anymore

* Copy socket changes to persistent, too

* Use os.write instead of fdopen()ing and using that.

* Follow pickle with sha1sum of pickle

* Swap order of vars and init being passed to ansible-connection
2018-08-10 09:26:58 -04:00
Brian Coca 2e9a7e2564 remove deprecated 2018-08-01 15:17:28 -04:00
Brian Coca 21fc6a0c38 nicer error message on multiprocessing fail
fixes #43090
2018-08-01 15:14:23 -04:00
Nathaniel Case 9169daa326
\r does not round-trip the local socket, escape and restore on the other side (#43507) 2018-08-01 08:44:39 -04:00
Matt Martz a7f45c0660 Add basedir to ansible_search_path before running loops. Fixes #43302 (#43431) 2018-07-31 11:51:32 -04:00
Toshio Kuratomi dce0a8d052 Switch to imp.load_module 2018-07-26 20:07:25 -07:00
Toshio Kuratomi 52449cc01a AnsiballZ improvements
Now that we don't need to worry about python-2.4 and 2.5, we can make
some improvements to the way AnsiballZ handles modules.

* Change AnsiballZ wrapper to use import to invoke the module
  We need the module to think of itself as a script because it could be
  coded as:

      main()

  or as:

      if __name__ == '__main__':
          main()

  Or even as:

      if __name__ == '__main__':
          random_function_name()

  A script will invoke all of those.  Prior to this change, we invoked
  a second Python interpreter on the module so that it really was
  a script.  However, this means that we have to run python twice (once
  for the AnsiballZ wrapper and once for the module).  This change makes
  the module think that it is a script (because __name__ in the module ==
  '__main__') but it's actually being invoked by us importing the module
  code.

  There's three ways we've come up to do this.
  * The most elegant is to use zipimporter and tell the import mechanism
    that the module being loaded is __main__:
    * 5959f11c9d/lib/ansible/executor/module_common.py (L175)
    * zipimporter is nice because we do not have to extract the module from
      the zip file and save it to the disk when we do that.  The import
      machinery does it all for us.
    * The drawback is that modules do not have a __file__ which points
      to a real file when they do this.  Modules could be using __file__
      to for a variety of reasons, most of those probably have
      replacements (the most common one is to find a writable directory
      for temporary files.  AnsibleModule.tmpdir should be used instead)
      We can monkeypatch __file__ in fom AnsibleModule initialization
      but that's kind of gross.  There's no way I can see to do this
      from the wrapper.

  * Next, there's imp.load_module():
    * https://github.com/abadger/ansible/blob/340edf7489/lib/ansible/executor/module_common.py#L151
    * imp has the nice property of allowing us to set __name__ to
      __main__ without changing the name of the file itself
    * We also don't have to do anything special to set __file__ for
      backwards compatibility (although the reason for that is the
      drawback):
    * Its drawback is that it requires the file to exist on disk so we
      have to explicitly extract it from the zipfile and save it to
      a temporary file

  * The last choice is to use exec to execute the module:
    * https://github.com/abadger/ansible/blob/f47a4ccc76/lib/ansible/executor/module_common.py#L175
    * The code we would have to maintain for this looks pretty clean.
      In the wrapper we create a ModuleType, set __file__ on it, read
      the module's contents in from the zip file and then exec it.
    * Drawbacks: We still have to explicitly extract the file's contents
      from the zip archive instead of letting python's import mechanism
      handle it.
    * Exec also has hidden performance issues and breaks certain
      assumptions that modules could be making about their own code:
      http://lucumr.pocoo.org/2011/2/1/exec-in-python/

  Our plan is to use imp.load_module() for now, deprecate the use of
  __file__ in modules, and switch to zipimport once the deprecation
  period for __file__ is over (without monkeypatching a fake __file__ in
  via AnsibleModule).

* Rename the name of the AnsiBallZ wrapped module
  This makes it obvious that the wrapped module isn't the module file that
  we distribute.  It's part of trying to mitigate the fact that the module
  is now named __main)).py in tracebacks.

* Shield all wrapper symbols inside of a function
  With the new import code, all symbols in the wrapper become visible in
  the module.  To mitigate the chance of collisions, move most symbols
  into a toplevel function.  The only symbols left in the global namespace
  are now _ANSIBALLZ_WRAPPER and _ansiballz_main.

revised porting guide entry

Integrate code coverage collection into AnsiballZ.

ci_coverage
ci_complete
2018-07-26 20:07:25 -07:00
Brian Coca cdc762dd9e
don't assume all delegation subkeys exist (#43032)
* don't assume all delegation subkeys exist

fixes #43026

* add tests
2018-07-24 08:54:58 -04:00
Jordan Borean 48ad90ac8e
powershell: support recursive requires statements (#43143) 2018-07-24 08:16:16 +10:00
Jordan Borean d723b8541d changed winrm _reset to reset and make ssh reset show warning (#42651)
* changed winrm _reset to reset and make ssh reset show warning

* minor changelog update
2018-07-11 20:22:01 -07:00
Brian Coca e115e6496f
preserve delegation info on no_log (#42577)
* preserve delegation info on no_log

fixes #42344
2018-07-11 20:41:37 -04:00
Brian Coca 27c43daab8
fix minor issues with debug and item labels (#41331)
* fix minor issues with debug and item labels

 - no more `item=None`, we always have a label now
 - debug should only show expected information, either msg= or the var in var=
 - also fixed method name, deprecated misleading _get_item
2018-06-21 16:14:57 -04:00
Toshio Kuratomi 3ea936312a Remove an unnecessary import from the ansiballz wrapper (#41747) 2018-06-21 12:04:55 +10:00
Brian Coca 336b3762b2 no_log even when task_result doesn't provide key
- now also checks task property
 - added reproducer to tests for unreachable status on item loop
2018-06-12 10:25:30 -07:00
Matt Martz 079318bf4a Honor ignore_errors when invoking the debugger, add config to disable this behavior (#39868)
* Honor ignore_errors when invoking the debugger, add config to disable this behavior. Fixes #39863

* Limit ignore_errors logic to failed
2018-05-31 11:14:26 -04:00