Commit Graph

227 Commits (1d640182c61af2d59cc1005f1ab5b217a56dcf15)

Author SHA1 Message Date
Martin Krizek 6d984c7988 low_level_execute_command: add stderr_lines 2018-05-14 12:05:03 -04:00
Toshio Kuratomi 6b6c4914d2 Initial cleanup of file module
* Remove use of six.b as Python-2.6+ have byte literals.
* Make AnsibleModule a global object so we'll have access to it in all
  the functions we're going to break this up into.
* Rework the parameters so things that are in file_common_args are used
  from file_common_args or the reason for deviation is documented.
* Remove validate as a parameter: this should be taken care of by
  removing it from params before the copy and template action plugin
  invoke file.
* Rename diff_peek to _diff_peek as it is an internal parameter.
* add module_name execute_module call to assemble so that it is more greppable
2018-05-10 14:48:28 -07:00
Abhijit Menon-Sen 0a119c391a Only expand '~' based on become_user/remote_user
Change brought forward from #00c023e in stable-2.5.

Closes #39281 (bug report)
Closes #39540 (PR with substantially the same change)
2018-05-02 22:50:41 +05:30
Jordan Borean 25dea1f72c Change except Error when trying to get a missing option 2018-04-27 09:47:18 -04:00
Brian Coca cc1c7c63db
ensure 'home' translations (#36755)
* ensure 'home' translations

* removed slash as it created issues on diff plats
2018-04-11 11:53:05 -04:00
Pilou 7908f78fa6 module_common: handle None value for templar (#36651)
* module_common: set required parameter templar

Fix the following error (related to b455901):

  $ ./hacking/test-module -m ./lib/ansible/modules/system/ping.py -I ansible_python_interpreter=/usr/bin/python
  Traceback (most recent call last):
    File "./hacking/test-module", line 268, in <module>
      main()
    File "./hacking/test-module", line 249, in main
      (modfile, modname, module_style) = boilerplate_module(options.module_path, options.module_args, interpreters, options.check, options.filename)
    File "./hacking/test-module", line 152, in boilerplate_module
      task_vars=task_vars
    File "ansible/lib/ansible/executor/module_common.py", line 910, in modify_module
      environment=environment)
    File "ansible/lib/ansible/executor/module_common.py", line 736, in _find_module_utils
      shebang, interpreter = _get_shebang(u'/usr/bin/python', task_vars, templar)
    File "ansible/lib/ansible/executor/module_common.py", line 452, in _get_shebang
      interpreter = templar.template(task_vars[interpreter_config].strip())
  AttributeError: 'NoneType' object has no attribute 'template'

* module_common.modify_module: templar is required
2018-03-29 13:54:48 -04:00
Toshio Kuratomi 06f73ad578
Normalize usage of temp and tmp on tmp (#36221)
* Normalize usage of temp and tmp on tmp
* Rename system_tmps system_tmpdirs
* Add ANSIBLE_REMOTE_TMP spelling of environment variables
2018-02-15 09:01:02 -08:00
Toshio Kuratomi 5c085b2436 Fix aysnc failure by forgetting the remote tempdir that async_wrapper will delete on the remote end 2018-02-08 06:57:09 -08:00
Toshio Kuratomi 71f46d69d6 First bit of fixing temporary to have one source of truth (#35747)
* First bit of fixing temporary to have one source of truth

* Fix pep8

* Remove explicit make_tmp_path() in copy

The copy action plugin sets TRANSFER_FILES=True so it does not need to
set the temporary directory explicitly; the base class's run() method
will do that for us.

* Fix for calling a module's run when a shell has already created a temp path.

* Remember to inform the rest of the world when tempdir is removed

* New strategy for how to warn on passing tmp

Now we just warn when calling the parent class run() early.  If the
module does a late call to the parent run() and doesn't make use of the
temporary directory, then we don't check for the possibility that the
user mistakenly is sending tmp in.  If we truly deprecate this (rather
than ignoring it forever) then we might want to switch back to checking
for someone passing a value in as tmp.

* Remove tmp parameter from _execute_module as well

* Port all action plugins to not send tmp explicitly

This is now handled inside of _execute_module via the
_connection._shell.tempdir attribute.

Also update warnings and docs to tell people to set the attribute
instead of using _execute_module's tmp parameter.

* Always set local tempdir variable
2018-02-07 15:11:36 -08:00
Brian Coca 3b0d864683 fix keep remote files 2018-02-07 16:58:27 -05:00
Toshio Kuratomi ca8e4c806a Pass tempdir via module args instead of the environment 2018-01-22 23:36:34 -08:00
Toshio Kuratomi 62bc714dae Move setting for world-readable-tempfiles back to global config.
The code that depends on this is all in the action plugins so we should
leave it there until we either move that action plugin code over
(fixup_perms2) or we give action plugins the ability to register new
config.
2018-01-22 18:38:15 -08:00
Jordan Borean d0e6889f93 win_become: another option to support become flags for runas (#34551)
* win_become: another option to support become flags for runas

* removed uneeded entries

* fixed up whitespace issue

* Copy edit
2018-01-19 13:58:10 -08:00
Brian Coca bbd6b8bb42 Temporary (#31677)
* allow shells to have per host options, remote_tmp

added language to shell
removed module lang setting from general as  plugins have it now
use get to avoid bad powershell plugin
more resilient tmp discovery, fall back to `pwd`
add shell to docs
fixed options for when frags are only options
added shell set ops in t_e and fixed option frags
normalize tmp dir usag4e

- pass tmpdir/tmp/temp options as env var to commands, making it default for tempfile
- adjusted ansiballz tmpdir
- default local tempfile usage to the configured local tmp
- set env temp in action

add options to powershell
shift temporary to internal envvar/params
ensure tempdir is set if we pass var
ensure basic and url use expected tempdir
ensure localhost uses local tmp
give /var/tmp priority, less perms issues
more consistent tempfile mgmt for ansiballz
made async_dir configurable
better action handling, allow for finally rm tmp
fixed tmp issue and no more tempdir in ballz
hostvarize world readable and admin users
always set shell tempdir
added comment to discourage use of exception/flow control

* Mostly revert expand_user as it's not quite working.

This was an additional feature anyhow.

Kept the use of pwd as a fallback but moved it to a second ssh
connection.  This is not optimal but getting that to work in a single
ssh connection was part of the problem holding this up.

(cherry picked from commit 395b714120522f15e4c90a346f5e8e8d79213aca)

* fixed script and other action plugins

ensure tmpdir deletion
allow for connections that don't support new options (legacy, 3rd party)
fixed tests
2018-01-15 21:15:04 -08:00
Jordan Borean 25a21f46bd
win async: removed async_wrapper module and fixed async for action plugins to use shell code (#34434)
* win async: removed async_wrapper module and fixed async for action plugins to use shell code

* fixed pep8 issue
2018-01-04 12:51:53 +10:00
Toshio Kuratomi 235bdfb996 Remove shell module specialcases
Shell is implemented via the command module.  There was a special case
in mod_args to do that.  Make shell into an action plugin to handle that
instead.

Also move the special case for the command nanny into a command module
action plugin.  This is more appropriate as we then do not have to send
a parameter that is only for the command module to every single module.
2017-12-21 20:42:44 -08:00
Brian Coca a0c171ce1b fix command warnings, add action warnings (#34060)
* fix command warnings, add action warnings
2017-12-21 08:35:32 -08:00
Sloane Hertel 7bb35e8781
Fix using loops with environment and add tests (#32796) 2017-12-08 10:13:22 -05:00
Toshio Kuratomi b455901904
Fix shebang. shebang and interpreter path weren't being templated (#33698)
* Fix shebang.  shebang and interpreter path weren't being templated

Fixes #18665
Fixes #33696
2017-12-08 06:59:24 -08:00
Jordan Borean 8386201242 add deprecation for stat get_md5 (#33002) 2017-11-27 18:58:55 -05:00
James Cammarata d8ae4dfbf2 Adding aliases for field attributes and renaming async attribute (#33141)
* Adding aliases for field attributes and renaming async attribute

As of Python 3.7, the use of async raises an error, whereas before the use
of the reserved word was ignored. This adds an alias field for field attrs
so that both async and async_val (interally) work. This allows us to be
backwards-compatible with 3rd party plugins that may still reference Task.async,
but for the core engine to work on Py3.7+.

* Remove files fixed for 'async' usage from the python 3.7 skip list
2017-11-22 12:35:58 -08:00
Brian Coca 23b1dbacaf
Config continued (#31024)
* included inventory and callback in new config

allow inventory to be configurable
updated connection options settings
also updated winrm to work with new configs
removed now obsolete set_host_overrides
added notes for future bcoca, current one is just punting, it's future's problem
updated docs per feedback
added remove group/host methods to inv data
moved fact cache from data to constructed
cleaner/better options
fix when vars are added
extended ignore list to config dicts
updated paramiko connection docs
removed options from base that paramiko already handles
left the look option as it is used by other plugin types
resolve delegation
updated cache doc options
fixed test_script
better fragment merge for options
fixed proxy command
restore ini for proxy
normalized options
moved pipelining to class
updates for host_key_checking
restructured mixins

* fix typo
2017-11-16 13:49:57 -05:00
Nathaniel Case 9c0275a879
Connection plugins network_cli and netconf (#32521)
* implements jsonrpc message passing for ansible-connection

* implements more generic mechanism for persistent connections
* starts persistent connection in task_executor if enabled and supported
* supports using network_cli as top level connection plugin
* enhances logging for persistent connection to stdout

* Update action plugins

* Fix Python3 RPC

* Fix Junos bytes<-->str issues

* supports using netconf as top level connection plugin

* Error message when running netconf on an unsupported platform
* Update tests

* Fix `authorize: yes` for `connection: local`

* Handle potentially JSON data in terminal

* Add clarifying detail if possible on ConnectionError
2017-11-09 15:04:40 -05:00
Brian Coca db749de5b8 namespace facts
updated action plugins to use new guranteed facts
updated tests to new data clean
added cases for ansible_local and some docstrings
2017-11-09 09:48:14 -05:00
Brian Coca f2ade09dce use configured ansible_shell_executable
refine args/shell/executable hanlding
2017-10-06 08:56:21 -04:00
Brian Coca ae29245e05 decelerate! (#30160)
removed accelerate code
removed keyczar dep for accelerate
2017-09-16 23:35:50 -04:00
Matt Clay 442af3744e Miscellaneous pylint fixes.
The following rules are no longer disabled:

- bad-format-string
- duplicate-key
- lost-exception
- trailing-newlines
- unexpected-keyword-arg
- useless-suppression
- using-constant-test
2017-09-13 01:53:08 -07:00
Matt Clay 68aeaa58a8 Fix dangerous default args. (#29839) 2017-09-12 00:11:13 -07:00
Brian Coca 075ead8fb0 fixes to config/setting retrieval
- better variable precedence management
- universal plugin option handling
- also updated comments for future directions
- leverage fragments for plugins
- removed fact namespacing
- added 'firendly name' field
- updated missing descriptions
- removed some unused yaml entries, updated others to reflect possible future
- documented more plugins
- allow reading docs using alias
- short licenses
- corrected args for 'all plugins'
- fixed -a option for ansible-doc
- updated vars plugins to allow docs
- fixed 'gathering'
- only set options IF connection
- added path list and renamed pathspec mostly the diff is , vs : as separator
- readded removed config entries that were deprecated but had no message ... and deprecated again
- now deprecated entries give warning when set
2017-09-09 09:48:22 -07:00
Brian Coca c06f8a3f9b allow pesky 'bridge' facts to bypass facts filter (#28401)
* allow pesky 'bridge' facts to bypass facts filter

fixes #27729, #23577

* I ate pepe
2017-08-29 09:47:40 -04:00
Brian Coca f921369445 Ansible Config part2 (#27448)
* Ansible Config part2

- made dump_me nicer, added note this is not prod
- moved internal key removal function to vars
- carry tracebacks in errors we can now show tracebacks for plugins on vvv
- show inventory plugin tracebacks on vvv
- minor fixes to cg groups plugin
- draft config from plugin docs
- made search path warning 'saner' (top level dirs only)
- correctly display config entries and others
- removed unneeded code
- commented out some conn plugin specific from base.yml
- also deprecated sudo/su
- updated ssh conn docs
- shared get option method for connection plugins
- note about needing eval for defaults
- tailored yaml ext
- updated strategy entry
- for connection pliugins, options load on plugin load
- allow for long types in definitions
- better display in ansible-doc
- cleaned up/updated source docs and base.yml
- added many descriptions
- deprecated include toggles as include is
- draft backwards compat get_config
- fixes to ansible-config, added --only-changed
- some code reoorg
- small license headers
- show default in doc type
- pushed module utils details to 5vs
- work w/o config file
- PEPE ATE!
- moved loader to it's own file
- fixed rhn_register test
- fixed boto requirement in make tests
- I ate Pepe
- fixed dynamic eval of defaults
- better doc code

skip ipaddr filter tests when missing netaddr
removed devnull string from config
better becoem resolution

* killed extra space with extreeme prejudice

cause its an affront against all that is holy that 2 spaces touch each other!

shippable timing out on some images, but merging as it passes most
2017-08-15 16:38:59 -04:00
Brian Coca 84367eacef added ansible_remote_tmp inventory var (#26754)
* added ansible_remote_tmp inventory var

* fixed horrible crime of no space after #
2017-07-13 15:31:10 -07:00
Brian Coca a5007f2f88 Namespaced facts fixes (#26615)
* shorten warning on reservd fact collision

also remove ansible_ from namespaced facts for vars manager
handle str conversion errors
use tuple to avoid iterator errors
version added added

* only modify final one

* removed ansible_ removal
2017-07-12 19:27:05 -04:00
Karim BEN YOUSSEF 804f40f7a5 script: Add support for chdir argument 2017-07-07 09:40:02 -07:00
Brian Coca 8f758204cf correct, cleanup & simplify dwim stack (#25956)
* correct, cleanup & simplify dwim stack

latlh chIS logh HeS qar wej chel laD
better errors
update find_file to new exception

* addressed latest comments

* test should not use realpath as it follows symlink

this fails when on OS X as /var is now a symlink to /private/var
but first_found was not supposed to follow symlinks
2017-07-03 15:27:53 -04:00
Tomas Tomecek 2683c36a01 plugins,action: log stderr output if verbose (#25734)
otherwise ansible eats out important error messages

Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
2017-06-28 15:55:39 -04:00
Matt Davis 36ad934156 re-enable non-pipelined mode for Powershell (#25012)
* fixes #23986
* fixes 3rd-party Windows connection plugins that don't support pipelining (eg awsrun)
2017-06-26 22:58:09 -07:00
Dag Wieers 5553b20828 Collated PEP8 fixes (#25293)
- Make PEP8 compliant
2017-06-02 12:14:11 +01:00
Toshio Kuratomi a0dfa8616a Make fetch default to fail on errors
Fixes #23501
2017-05-03 09:37:31 -07:00
Toshio Kuratomi 1c05ed7951 Fix circular import with unsafe_proxy, template, and vars
template/__init__.py imported unsafe_proxy from vars which caused
vars/__init__.py to load.  vars/__init__.py needed template/__init__.py
which caused issues.  Loading unsafe_proxy from another location fixes
that.
2017-04-28 15:35:30 -07:00
Brian Coca e29dc49a49 moved to exceptions for basic skip/fails
better handling of checkmode and async
fix test to follow new flow control
2017-04-25 14:48:56 -04:00
Brian Coca 3965689328 finer grained failed/skipped on async for actions
fixes #23729
2017-04-20 10:58:50 -04:00
Toshio Kuratomi 2fdb8e7f90 Fix a traceback with python3 and diff output
When retrieving file contents for diffing we need to get the contents as
binary.  Otherwise python3 will try to convert the file to text and fail
with non-decodable contents.

Fixes #23171
2017-04-07 14:35:50 -07:00
Matt Davis 73f50b4f9f fix Windows env handling (#22927)
* fixes #22441
* fixes #22655
* moves all env handling into the exec wrapper; this should work for everything but raw, which is consistent with non-Windows.
2017-03-23 17:48:15 -07:00
Toshio Kuratomi 2fff690caa Update module_utils.six to latest (#22855)
* Update module_utils.six to latest

We've been held back on the version of six we could use on the module
side to 1.4.x because of python-2.4 compatibility.  Now that our minimum
is Python-2.6, we can update to the latest version of six in
module_utils and get rid of the second copy in lib/ansible/compat.
2017-03-23 13:35:05 -07:00
Matt Martz 0ced29c51b Don't skip action/__init__.py (#22911) 2017-03-23 13:40:02 -05:00
Matt Martz 3164e8b561 E501 fixes (#22879) 2017-03-22 20:50:28 -05:00
Brian Coca 5e98be2e62 better handling of parsed
updated tests to match new result output
2017-03-22 12:03:43 -04:00
Matt Martz 02f66b9369 E1 legacy pep8 fixes (#21933)
* E1 pep8 fixes

* e111 fix for rds.py
2017-03-21 21:19:40 -05:00
Ted Timmons 887456ab8e Fix stderr false return value (#22845)
* ensure exit_json does not fail from stderr=False

- do a little bit of safety-checking in exit_json to not try to .splitlines() on a boolean
- remove the stderr=boolean from uri.py, this is the only spot that uses it (at least so obviously)
- add unit tests that call exit_json. These are useless because the stderr parsing is in _execute_module and is difficult to mock; deleting these tests after the commit.

* remove added unit tests per prev commit

exit_json doesn't do the param parsing, that is buried deep inside _execute_module.
2017-03-21 18:11:11 -04:00