Commit Graph

963 Commits (dfaa72297ccec757aed0c6eeaadaa0188e8ec34f)

Author SHA1 Message Date
Brian Coca 27dcf8aaab
Not native, but text (#55676)
* use to_text instead of to_native
* cleaned up some imports and other pyflakisms
* fix missing lib messages
2019-05-24 14:49:04 -04:00
Martin Krizek 34e9d6781b Templar: encapsulate _available_variables (#55435)
Ensure variables are reset between iterations
2019-05-20 11:49:54 -04:00
Brian Coca 674a57c3fa
fix module defaults (#56020)
* fix module defaults

 - corrected precedence (specific module > group)
 - made into reusable function
 - use from gather_facts/service/package to match 'actual module used'
2019-05-14 09:59:18 -04:00
Nilashish Chakraborty 30ccc8e1bd
Fix VyOS cli prompt issues (#56389)
Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
2019-05-14 19:17:00 +05:30
Nilashish Chakraborty 7608bc821e
Fix vyos cli prompt issues (#55589)
Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
2019-05-07 20:51:35 +05:30
Ganesh Nalawade 9271b4e368
Fix net_get and net_put task run failure (#56145)
* net_get and net_put action plugin class need
  to inherit from ActionBase class as the action
  class implements the entire get and put logic
2019-05-07 06:34:36 -04:00
Ganesh Nalawade 1a66121f03
Fix invalid src option return response for network config modules (#56076)
*  Add changed key in faliure case to maintain backward compatibility
2019-05-04 07:23:12 -04:00
Matt Davis 03cac394cc
mark entire module result untrusted as template (#55717)
* prevents accidental templating on intra-action postprocessing of an untrusted module result
* makes the view of a module result within an action consistent with the way it would be stored for future use (eg facts, register)
2019-04-24 15:15:20 -07:00
Brian Coca 24b44e1772
Ensure discovery unsafe (#55295)
* Ensure safe discovery by marking it Unsafe
2019-04-17 12:08:51 -04:00
xuxiaowei 1017f15c38 fix bugs for ce (#54750)
* Update ce.py

            while to_text(out, errors='surrogate_then_replace').strip().endswith(']'):
                display.vvvv('wrong context, sending exit to device', self._play_context.remote_addr)
                conn.exec_command('return')
                out = conn.get_prompt()

connetion has no send_command function and ce device has no 'exit' command to return user-view(a correct context),but 'return' .command.

* Add files via upload

Some bugs fix.

* Add files via upload

fix some bugs

* fix a bug for ce_command

Running a command with prompt via ce_command, It doesn't work.The reason is that the key word for network_cli recognition is answer not response.

* fix bugs

fix bugs for ce modules

* Update ce.py

* Delete ce_ftp.py

need modify

* Delete ce_lacp.py

* Add files via upload

* Delete ce_aaa_server.py

* Delete ce_aaa_server_host.py

* Compatible with Python 3

Compatible with Python 3 and fix bugs for ce

* Update ce_aaa_server.py

* Add files via upload

modify doc

* Add files via upload

Compatible with Python 3 and fix bugs

* Add files via upload

Compatible with Python 3 and fix bugs

* Add files via upload

Cancellation of change

* Update ce_netconf.py

It is a bug that response has no xml attribute:line 183

* Add files via upload

* Add files via upload

Compatible with Python 3 and fix bugs

* updatp ce_config.py

a bug for this module.
2019-04-17 12:51:58 +05:30
Jordan Borean 49655a452d
psrp - fix test_command rc for win_reboot (#55354) 2019-04-17 08:38:42 +10:00
Martin Krizek d15812fabf
Fix copy module to reset filesystem acls (#51868)
The controller's fixup_perms2 uses filesystem acls to make the temporary
file for copy readable by an unprivileged become user. On Python3, the
acls are then copied to the destination filename so we have to remove
them from there.

We can't remove them prior to the copy because we may not have
permission to read the file if the acls are not present. We can't
remove them in atomic_move() because the move function shouldn't know
anything about controller features. We may want to generalize this into
a helper function, though.

Fixes #44412

Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
2019-04-03 18:37:59 +02:00
Nilashish Chakraborty ee08b009dd
Make prompt inspection explicit for IOS (#54324)
* Be explicit with prompt inspection

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

* Use regex to evaluate prompt for sub-level config sessions

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

* Add new line at the end of file

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
2019-04-03 10:55:20 +05:30
Matt Davis f86345f777
Collection content loading (#52194)
* basic plugin loading working (with many hacks)

* task collections working

* play/block-level collection module/action working

* implement PEP302 loader

* implicit package support (no need for __init.py__ in collections)
* provides future options for secure loading of content that shouldn't execute inside controller (eg, actively ignore __init__.py on content/module paths)
* provide hook for synthetic collection setup (eg ansible.core pseudo-collection for specifying built-in plugins without legacy path, etc)

* synthetic package support

* ansible.core.plugins mapping works, others don't

* synthetic collections working for modules/actions

* fix direct-load legacy

* change base package name to ansible_collections

* note

* collection role loading

* expand paths from installed content root vars

* feature complete?

* rename ansible.core to ansible.builtin

* and various sanity fixes

* sanity tweaks

* unittest fixes

* less grabby error handler on has_plugin

* probably need to replace with a or harden callers

* fix win_ping test

* disable module test with explicit file extension; might be able to support in some scenarios, but can't see any other tests that verify that behavior...

* fix unicode conversion issues on py2

* attempt to keep things working-ish on py2.6

* python2.6 test fun round 2

* rename dirs/configs to "collections"

* add wrapper dir for content-adjacent

* fix pythoncheck to use localhost

* unicode tweaks, native/bytes string prefixing

* rename COLLECTION_PATHS to COLLECTIONS_PATHS

* switch to pathspec

* path handling cleanup

* change expensive `all` back to or chain

* unused import cleanup

* quotes tweak

* use wrapped iter/len in Jinja proxy

* var name expansion

* comment seemingly overcomplicated playbook_paths resolution

* drop unnecessary conditional nesting

* eliminate extraneous local

* zap superfluous validation function

* use slice for rolespec NS assembly

* misc naming/unicode fixes

* collection callback loader asks if valid FQ name instead of just '.'
* switch collection role resolution behavior to be internally `text` as much as possible

* misc fixmes

* to_native in exception constructor
* (slightly) detangle tuple accumulation mess in module_utils __init__ walker

* more misc fixmes

* tighten up action dispatch, add unqualified action test

* rename Collection mixin to CollectionSearch

* (attempt to) avoid potential confusion/conflict with builtin collections, etc

* stale fixmes

* tighten up pluginloader collections determination

* sanity test fixes

* ditch regex escape

* clarify comment

* update default collections paths config entry

* use PATH format instead of list

* skip integration tests on Python 2.6

ci_complete
2019-03-28 10:41:39 -07:00
Brian Coca 3637ce4538
Handle gracefully a bad module 'results' key (#53973)
* Handle gracefully a bad module 'results' key

  fixes #53962
2019-03-27 18:41:32 -04:00
Mike Wiebe 4c2a3bfed5 Fix nxos action plugin for nxos_install_os (#53768)
* Fix nxos action plugin for nxos_install_os

* Update module docs
2019-03-18 09:14:40 +05:30
Matt Martz 97161ed2ab
Strip None values from mod_args in gather_facts action (#53758) 2019-03-14 10:19:04 -05:00
Jordan Borean 3d23e47c53
win_reboot - Fix rc validation when using psrp and add extra docs (#53711)
* win_reboot - Fix rc validation when using psrp and add extra docs

* Revert boot time command and fix docs
2019-03-13 10:43:02 +10:00
Matt Martz e9f9bcae6a
Don't raise AnsibleConnectionFailure if the ssh process has already died. (#53534)
* Don't raise AnsibleConnectionFailure if the ssh_process has already died. Fixes #53487

* Better support for file not found messages

* Add changelog fragment
2019-03-12 10:41:30 -05:00
Brian Coca 8940732b58
Configurable and parallel gather facts (#49399)
* Configurable list of facts modules (#31783)

 - allow for args dict for specific modules
 - add way to pass parameters
 - avoid facts poluting test
 - move to 'facts gathered' flag
 - add 'gathering' setting tests
 - allow parallel option in case serialization is too slow
 - added support to automatically map network facts
   uses "smart" connection mapping
2019-03-08 13:08:37 -05:00
Abhijeet Kasurde a5d3647eca
include_vars: include main.yml (#51926)
Enhance the conditional check to include main.yml if it is not
from 'role/vars/'

Fixes: #29135

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-03-08 10:03:01 +05:30
Jordan Borean 8ef2e6da05 Add support for Windows hosts in the SSH connection plugin (#47732)
* Add support for Windows hosts in the SSH connection plugin

* fix Python 2.6 unit test and sanity issues

* fix up connection tests in CI, disable SCP for now

* ensure we don't pollute the existing environment during the test

* Add connection_windows_ssh to classifier

* use test dir for inventory file

* Required powershell as default shell and fix tests

* Remove exlicit become_methods on connection

* clarify console encoding comment

* ignore recent SCP errors in integration tests

* Add cmd shell type and added more tests

* Fix some doc issues

* revises windows faq

* add anchors for windows links

* revises windows setup page

* Update changelogs/fragments/windows-ssh.yaml

Co-Authored-By: jborean93 <jborean93@gmail.com>
2019-03-07 16:38:02 -08:00
Brian Coca efd5222c17
fix locals not getting updated (#41798)
* fix locals not getting updated

switched to updating the args/assignments as locals doesn't work

* add missing import
2019-03-07 13:38:07 -05:00
Abhijeet Kasurde d39f35c1d6
assert: allow list of strings in msg or failed_msg (#50531)
Added check for assert module for msg and failed_msg as a list or string types.

Fixes: #48547

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-03-05 11:12:17 +05:30
Sam Doran 1c87ea6fa6 Reboot - add parameter for paths to search for shutdown command (#51194)
* Look in /lib/molly-guard for shutdown command

Signed-off-by: Sam Doran <sdoran@redhat.com>

* Add parameter for extra_search_paths

Signed-off-by: Sam Doran <sdoran@redhat.com>

* Change option to search_paths

- Update docs
- Make the parameter replace the default values
- Add some sanity checking on the data type

Signed-off-by: Sam Doran <sdoran@redhat.com>

* Correct parameter in tests, change conditional for molly-guard tasks

Signed-off-by: Sam Doran <sdoran@redhat.com>

* Address feedback

* Simplify field validation

Needed the try except in case a non-iterable type is put in the search_paths field
2019-02-28 13:36:33 -08:00
rwagnergit d2bdbadb03 adding quiet option to assert (ansible#27124) (#52032)
* adding quiet option to assert (ansible#27124)

* adding doc for quiet to assert.py

* fixing PEP8 failure

* improving example

* improving docs

* adding changelog fragment

* adding . at end of descriptions

* removing trailing blank line

* adding integration test for assert

* fixing CI complaints

* disabling gather_facts in quiet.yml test

* rerunning to capture skip

* cleaning up python 2 vs 3

* following rebase

* fixing CI complaints

* fixing CI complaints

* fixing CI complaints

* fixing CI complaints

* fixing CI complaints
2019-02-28 10:25:41 -05:00
Matt Davis 4d3a6123d5
Python interpreter discovery (#50163)
* Python interpreter discovery

* No longer blindly default to only `/usr/bin/python`
* `ansible_python_interpreter` defaults to `auto_legacy`, which will discover the platform Python interpreter on some platforms (but still favor `/usr/bin/python` if present for backward compatibility). Use `auto` to always use the discovered interpreter, append `_silent` to either value to suppress warnings.
* includes new doc utility method `get_versioned_doclink` to generate a major.minor versioned doclink against docs.ansible.com (or some other config-overridden URL)

* docs revisions for python interpreter discovery

(cherry picked from commit 5b53c0012ab7212304c28fdd24cb33fd8ff755c2)

* verify output on some distros, cleanup
2019-02-27 23:52:02 -08:00
Ganesh Nalawade cb67235eab
Fix network config module invalid src option error (#52912)
Fixes #52911

Raise AnsibleError execption if the file
path mentioned in src option in not found or
failed to load
2019-02-25 16:46:33 +05:30
Dag Wieers 3d1dd0e599 Windows: Add backup parameter to modules (#50033)
* Windows: Add backup parameter to modules

This PR adds a backup infrastructure for modules.

* Fixes based on review feedback

* Various fixes to check-mode and backup

* Add integration tests

* Fix win_xml integration test

* Add backup support to copy action plugin

* Added integration tests

* Improve test efficiencies and other minor impv
2019-02-25 11:37:25 +10:00
Sam Doran f52a088862
Add option to ignore, warn, or error when a module parameter is converted to a string (#51404)
* Add new module property to Windows modules
* Add brief pause to file tests to ensure the stat times are not equal, which was happening sometimes.
* Raise TypeError on error rather than fail_json()
* Rework error message to be less verbose
* Add porting guide entry
2019-02-22 16:44:32 -05:00
Nathaniel Case 1b9745722e
Fix issues with enable become plugin (#52657)
* Work around become_user not being set
2019-02-20 16:15:45 -05:00
Martin Krizek f8d1fa80e9 debug: prevent failing when printing var=rc (#52074)
Fixes #52073
2019-02-19 15:24:11 -05:00
Abhijeet Kasurde 24d1886499
template: add additional variable for dest path (#52015)
Signed-off-by: Dustin Spicuzza <dustin@virtualroadside.com>
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-02-19 14:15:19 +05:30
Sayed Anisul Hoque 7d6a94e7dc Fixed typos (#52426) 2019-02-17 22:49:40 +01:00
Dag Wieers c2fb581414 Fix various sonarcloud issues
This fixes various reported bugs through sonarcloud at:
https://sonarcloud.io/project/issues?id=Rodney-Reis_ansible&resolved=false&types=BUG
2019-02-15 07:19:04 -08:00
Matt Martz b34d141eed
Disallow use of remote home directories containing .. in their path (CVE-2019-3828) (#52133)
* Disallow use of remote home directories containing .. in their path

* Add CVE to changelog
2019-02-13 10:38:28 -06:00
Abhijeet Kasurde 4e0e09d2de docs: update docs about pipelining (#51856)
Additional documentation about `keep_remote_files` and
`pipelining` configurations.

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-02-12 12:49:00 -05:00
Matt Martz 445ff39f94
Become plugins (#50991)
* [WIP] become plugins

Move from hardcoded method to plugins for ease of use, expansion and overrides
  - load into connection as it is going to be the main consumer
  - play_context will also use to keep backwards compat API
  - ensure shell is used to construct commands when needed
  - migrate settings remove from base config in favor of plugin specific configs
  - cleanup ansible-doc
  - add become plugin docs
  - remove deprecated sudo/su code and keywords
  - adjust become options for cli
  - set plugin options from context
  - ensure config defs are avaialbe before instance
  - refactored getting the shell plugin, fixed tests
     - changed into regex as they were string matching, which does not work with random string generation
     - explicitly set flags for play context tests
 - moved plugin loading up front
 - now loads for basedir also
 - allow pyc/o for non m modules
 - fixes to tests and some plugins
 - migrate to play objects fro play_context
 - simiplify gathering
 -  added utf8 headers
 - moved option setting
 - add fail msg to dzdo
 - use tuple for multiple options on fail/missing
 - fix relative plugin paths
 - shift from play context to play
 - all tasks already inherit this from play directly
 - remove obsolete 'set play'
 - correct environment handling
 - add wrap_exe option to pfexec
 - fix runas to noop
 - fixed setting play context
 - added password configs
 - removed required false
 - remove from doc building till they are ready

future development:
  - deal with 'enable' and 'runas' which are not 'command wrappers' but 'state flags' and currently hardcoded in diff subsystems

* cleanup

  remove callers to removed func
  removed --sudo cli doc refs
  remove runas become_exe
  ensure keyerorr on plugin
  also fix backwards compat, missing method is attributeerror, not ansible error
  get remote_user consistently
  ignore missing system_tmpdirs on plugin load
  correct config precedence
  add deprecation
  fix networking imports
  backwards compat for plugins using BECOME_METHODS

* Port become_plugins to context.CLIARGS

This is a work in progress:
* Stop passing options around everywhere as we can use context.CLIARGS
  instead

* Refactor make_become_commands as asked for by alikins

* Typo in comment fix

* Stop loading values from the cli in more than one place

Both play and play_context were saving default values from the cli
arguments directly.  This changes things so that the default values are
loaded into the play and then play_context takes them from there.

* Rename BECOME_PLUGIN_PATH to DEFAULT_BECOME_PLUGIN_PATH

As alikins said, all other plugin paths are named
DEFAULT_plugintype_PLUGIN_PATH.  If we're going to rename these, that
should be done all at one time rather than piecemeal.

* One to throw away

This is a set of hacks to get setting FieldAttribute defaults to command
line args to work.  It's not fully done yet.

After talking it over with sivel and jimi-c this should be done by
fixing FieldAttributeBase and _get_parent_attribute() calls to do the
right thing when there is a non-None default.

What we want to be able to do ideally is something like this:

class Base(FieldAttributeBase):
    _check_mode = FieldAttribute([..] default=lambda: context.CLIARGS['check'])

class Play(Base):
    # lambda so that we have a chance to parse the command line args
    # before we get here.  In the future we might be able to restructure
    # this so that the cli parsing code runs before these classes are
    # defined.

class Task(Base):
    pass

And still have a playbook like this function:

---
- hosts:
  tasks:
  - command: whoami
    check_mode: True

(The check_mode test that is added as a separate commit in this PR will
let you test variations on this case).

There's a few separate reasons that the code doesn't let us do this or
a non-ugly workaround for this as written right now.  The fix that
jimi-c, sivel, and I talked about may let us do this or it may still
require a workaround (but less ugly) (having one class that has the
FieldAttributes with default values and one class that inherits from
that but just overrides the FieldAttributes which now have defaults)

* Revert "One to throw away"

This reverts commit 23aa883cbed11429ef1be2a2d0ed18f83a3b8064.

* Set FieldAttr defaults directly from CLIARGS

* Remove dead code

* Move timeout directly to PlayContext, it's never needed on Play

* just for backwards compat, add a static version of BECOME_METHODS to constants

* Make the become attr on the connection public, since it's used outside of the connection

* Logic fix

* Nuke connection testing if it supports specific become methods

* Remove unused vars

* Address rebase issues

* Fix path encoding issue

* Remove unused import

* Various cleanups

* Restore network_cli check in _low_level_execute_command

* type improvements for cliargs_deferred_get and swap shallowcopy to default to False

* minor cleanups

* Allow the su plugin to work, since it doesn't define a prompt the same way

* Fix up ksu become plugin

* Only set prompt if build_become_command was called

* Add helper to assist connection plugins in knowing they need to wait for a prompt

* Fix tests and code expectations

* Doc updates

* Various additional minor cleanups

* Make doas functional

* Don't change connection signature, load become plugin from TaskExecutor

* Remove unused imports

* Add comment about setting the become plugin on the playcontext

* Fix up tests for recent changes

* Support 'Password:' natively for the doas plugin

* Make default prompts raw

* wording cleanups. ci_complete

* Remove unrelated changes

* Address spelling mistake

* Restore removed test, and udpate to use new functionality

* Add changelog fragment

* Don't hard fail in set_attributes_from_cli on missing CLI keys

* Remove unrelated change to loader

* Remove internal deprecated FieldAttributes now

* Emit deprecation warnings now
2019-02-11 11:27:44 -06:00
chronidev 240d1a6afb Add coherency between check and normal mode for copy plugin action see issue #24633 (#51582)
* Add coherency between check and normal mode see issue #24633

* Add changelog fragment for the PR

* Make change following PR comment

* Remove trailing whitespace
2019-02-01 13:47:23 +10:00
Jordan Borean 6a2aac487d
win_stat - add follow option and fix broken tests (#51522)
* win_stat - add follow option and fix broken tests

* fix docs issues
2019-01-31 15:56:06 +10:00
Nilashish Chakraborty 92b0cd8e0e
Add more return values to *_config modules (#50702)
* Add more return values to *_config modules

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

* Add more rv

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

* Exclude from cli_config docs for now

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

* Add new rv docs for junos_config

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

* Fix CI errors

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

* Support date time for configurable backup path

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

* Change logic based on configurable path

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

* Remove unwanted import

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

* Fix docs

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

* Add filename rv

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

* Change dosc filename rv

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

* Make new rv code more readable

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
2019-01-28 20:49:56 +05:30
Ganesh Nalawade 70bf9b9919
Add backup filename and dir path option for config network modules (#50801)
* Add configurable backup path option for network config modules

Fixes #50283
Fixes #32724

*  Add back_options in network config module argspec
*  Handle backup path options in network action plugin

* Fix review comments

* Add integration tests

* Update changelog
2019-01-24 09:36:16 +05:30
Matt Davis 2cb9ec9f0e fix include_vars non-ASCII error handling (#51198) 2019-01-22 19:35:34 -05:00
Jordan Borean b7620c161a
win_updates - fix category return value to be a list (#51001)
* win_updates - fix category return value to be a list

* remove unstable test alias
2019-01-18 05:35:10 +10:00
Jordan Borean 9fa46e7f94 reboot - Fix connection timeout reset (#51000)
* only reset if we could retrieve the conn timeout
2019-01-17 10:45:41 -05:00
Matt Clay b971ebd343 More path encoding fixes. 2019-01-16 15:44:50 -08:00
Trishna Guha a1ea7e430a
fix ansible_connect_timeout variable in connection plugins and nxos_install_os timeout check (#50965)
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
2019-01-16 14:10:26 +05:30
Abhijeet Kasurde db8702cdb8 Close all open filehandle (#50544)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-01-11 10:14:08 -05:00
trogdor_the_burninator 1dac10e5c3 Add AIX support to reboot module (#50353)
* Add ability for reboot module to work for AIX

* changelog for AIX reboot support
2019-01-08 18:12:30 -05:00
Ganesh Nalawade 1e09e21ba1
Fix issue in junos and bigip action plugins (#50636)
*  tmp is not required to be passed to in run()
   within action plugin
2019-01-08 08:43:15 +05:30