* These can still race when multiple ansible processes are created at
the same time.
* Reverse order of expanduser and expandvars in unfrakpath(). So that
tildes in environment variables will be handled.
This feature changes the scalar value of `serial:` to a list, which
allows users to specify a list of values, so batches can be ramped
up (commonly called "canary" setups):
- hosts: all
serial: [1, 5, 10, "100%"]
tasks:
...
* add load_config() for loading a set of configuration commands
* add load_candidate() function for loading a candidate config
* updates shared module to provide NetworKModule instead of get_module
* fixes Cli transport implementation for 2.2 refactor
* updates ios documentation fragments with new options
Manifests as the following stack trace
File "/usr/local/Cellar/ansible/2.0.1.0/libexec/lib/python2.7/site-packages/ansible/utils/display.py", line 259, in error
new_msg = u"ERROR! " + msg
TypeError: coercing to Unicode: need string or buffer, AnsibleParserError found
Ensure that initial setup in creating the key directory for ansible
accelerate mode keys, and generation/storage of the key for a
particular host are completed in a multiprocess safe manner.
Creating directories/files and then assigning permissions and contents
to them means that paths may exist and satisfy the os.path.exists()
method in python for other processes before they are usable.
Use a combination of locking around operations with use of unique named
files and an OS file system move to ensure that the conditions of
checking whether a file or directory exists, where it is potentially
created by another thread, will only succeed when the file has both the
correct contents and permissions.
Fixes#13850
* Could only have one alias before. Subsequent aliases overrode the
previous ones. Now multiple aliases work.
* Fix BLACKLISTED_MODULES. Previously, modules were listed in the
generated documentation despite being blacklisted
* Deprecated modules form extras were showing the (E) tag and not the
(D) tag. Reversed that now (Probably not necessary to also show the
E tag).
* Sort the deprecated modules alphabetically in the Category docs as
well as the list of all modules
* Optimization: Previously rendered the modules to rst twice once in all
group and once in individual categories. Fixed to only render them
once.
* Add fireball to blacklist and remove async_status (as people need to
use that).
A few of the docs fragments have the available choices for some params
defined as "BOOLEANS". Because choices accepts a list, it treats
"BOOLEANS" as an iterable and then generates a list composed of each
letter.
Instead, define the available choices as a list of `['yes', 'no']`,
as is common in most other modules.
For example:
$ ansible web --list-hosts | head -n1
hosts (7):
ERROR! Unexpected Exception: [Errno 32] Broken pipe
Traceback (most recent call last):
File "/home/lamby/git/private/lamby-ansible2/.venv/bin/ansible", line 114, in <module>
display.display("to see the full traceback, use -vvv")
File "/home/lamby/git/private/lamby-ansible2/.venv/local/lib/python2.7/site-packages/ansible/utils/display.py", line 133, in display
sys.stdout.flush()
IOError: [Errno 32] Broken pipe
Such a pipe target will close up shop early when its seen enough input,
causing ansible to print an ugly traceback.
Signed-off-by: Chris Lamb <chris@chris-lamb.co.uk>
This is the same fix we applied to v1.9 in PR #14565, however it does not fix#14678 completely !
The dictionaries are not being merged as tey are on v1.9.
This is related to #14559, but only the part for Ansible v2.0
This commit makes merging empty dicts, or equal dicts more efficient.
I noticed that while debugging merge_hash a lot of merges related to empty dictionaries and sometimes also identical dictionaries.
The module docs and vault changes solve issues where tracebacks can
happen. The galaxy changes are mostly refactoring to be more pythonic
with a small chance that a unicode traceback could have occurred there
without the changes. The change in __init__.py when we actually call
the pager makes things more robust but could hide places where we had
bytes coming in already so I didn't want to change that without auditing
where the text was coming from.
Fixes#14178
The provider argument accepts the set of device common arguments as a
dict object. Individual connection arguments can still be included and
take priority over the provider argument. This update includes additions
to the nxos doc fragment
New argument `provider` added to the ios shared module that provides
the ability to pass all of the common ios arguments as a dict. This commit
includes some minor bugfixes and refactoring of names. It also includes
udpates to the ios documentation fragment for the new argument
Adds a new argument `provider` to the eos shared module and updates the
eos doc fragment. This commit includes some additional minor fixes and
code refactors for naming conventions. The `provider` argument allows the
shared module arguments to be passed as a dict object instead of having
to pass each argument invididually.
This commit adds a new argument `provider` to the iosxr shared module that
allows common connection parameters to be passed as a dict object. The
constraints on the args still applies. This commit also updates the iosxr
doc fragment.
Adds new argument `provider` to the openswitch shared module. The provider
argument can pass all openswitch connection arguments as a dict object. This
update includes adding the provider argument to the openswitch doc fragment
This commit adds a new argument `provider` to the junos shared module. The
argument allows the set of common connection args to be passed to the
junos shared module. This commit also updates the junos doc fragment
This commit adds a new shared module for working with network devices running
the Juniper Junos operating system. The commit includes a new document
fragment junos to be used when building modules. The junos shared module
currently only supports CLI
This commit adds a new shared module openswitch for building modules that
work with OpenSwitch. This shared module supports connectivity to
OpenSwitch devices over SSH, CLI or REST. It also adds an openswitch
documentation fragment for use in modules
This commit refactors the nxapi into a new shared module nxos that supports
connectivity over both ssh (cli) and nxapi. It supercedes the nxapi shared
module and removes it from module_utils. This commit also adds a
documentation fragement supporting the nxos shared module
This commit adds a new shared module for working with Cisco IOS XR devices over
CLI (SSH). It also provides a documentation fragement for the commmon arguments
provided by the iosxr module.
This update refactor the ios shared module to use the new shell shared
library instead of issh and cli. It also adds the ios documentation
fragment to be used when building ios based modules.
This adds a shared module for communicating with Arista EOS devices over
SSH (cli) or JSON-RPC (eapi). This modules replaces the eapi.py module
previously added to module_utils. This commit includes a documentation
fragment that describes the eos common arguments
pushed it to use the existing propmpt from display and moved the vars prompt code there also for uniformity
changed vars_prompt to check extra vars vs the empty play.vars to restore 1.9 behaviour
sipmlified the code as it didn't need to check for syntax again (tqm is made none prior based on that)
fixes#13770
Still is a warning as we don't want to repeat it multiple times nor additional callbacks to stop ansible execution.
hopefully we can avoid shipping w/o exceptions in the default/minimal callbacks...
Also added feature that now allows for 'preformated' strings passed to warning
better error reporting on fetching errors
use scm if it exists over src
unified functions in requirements
simplified logic
added verbose to tests
cleanup code refs, unused options and dead code
moved get_opt to base class
fixes#11920fixes#12612fixes#10454
On Python 2, shlex.split() raises if you pass it a unicode object with
non-ASCII characters in it. The Ansible codebase copes by explicitly
converting the string using to_bytes() before passing it to
shlex.split().
On Python 3, shlex.split() raises ('bytes' object has no attribute 'read')
if you pass a bytes object. Oops.
This commit introduces a new wrapper function, shlex_split, that
transparently performs the to_bytes/to_unicode conversions only on
Python 2.
Currently I've only converted one call site (the one that was causing a
unit test to fail on Python 3). If this approach is deemed suitable,
I'll convert them all.
Required some rewiring in inventory code to make sure we're using
the DataLoader class for some data file operations, which makes mocking
them much easier.
Also identified two corner cases not currently handled by the code, related
to inventory variable sources and which one "wins". Also noticed we weren't
properly merging variables from multiple group/host_var file locations
(inventory directory vs. playbook directory locations) so fixed as well.
You cannot call bytes(obj) to get a simple representation of obj on
Python 3! E.g. bytes(42) returns a byte string with 42 NUL characters
instead of b'42'.
Since we explicitly set convert_bare=False in the template lookup
code, but still want individual looks that call listify directly to
convert bare variables if needed.
Fixes bugs related to creating Templar() objects on the fly, where
the shared loader objects (serialized to TaskExecutor) aren't used
so information loaded into plugin loaders after forking is lost.
Fixes#11815
This is unsafe and we debated re-adding it to the v2/2.0 codebase,
however it is a common-enough feature that we will simply mark it
as deprecated for now and remove it at some point in the future.
Fixes#11718
added deprecation warning
fixed display.deprecated to make version optional (code already assumed this)
turned warning + 'deprecated' in plugin loader into actual call to deprecated()
This allows not messing up the wonderful error reporting that is
carefully created. Instead of:
$ ansible-playbook foo.yml
[ERROR]: ERROR! 'foo' is not a valid attribute for a Task The error appears
to have been in '/Users/marca/dev/git-repos/ansible/foo.yml': line 4, column 7,
but may be elsewhere in the file depending on the exact syntax problem. The
offending line appears to be: tasks: - name: do something ^ here
we get:
$ ansible-playbook foo.yml
ERROR! 'foo' is not a valid attribute for a Task
The error appears to have been in '/Users/marca/dev/git-repos/ansible/foo.yml': line 4, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
tasks:
- name: do something
^ here
which is much nicer.
We're being too strict - there is a third possibility, which is that a
user will have defined the OS_* environment variables and expect them to
pass through.
inventory vars: make loading from a directory obey the same rules as
when checking the base paths, looking at the file name extensions
as defined in CONSTANTS.YAML_FILENAME_EXTENSIONS
Fixes Github issue #11017
Before this change if a variable was of type int or bool and the variable was referenced
by another variable, the type would change to string.
eg. defaults/main.yml
```
PORT: 4567
OTHER_CONFIG:
secret1: "so_secret"
secret2: "even_more_secret"
CONFIG:
hostname: "some_hostname"
port: "{{ PORT }}"
secrets: "{{ OTHER_CONFIG }}"
```
If you output `CONFIG` to json or yaml, the port would get represented in the output as a
string instead of as a number, but secrets would get represented as a dictionary. This is
a mis-match in behaviour where some "types" are retained and others are not. This change
should fix the issue.
Update template test to also test var retainment.
Make the template changes in v2.
Update to only short-circuit for booleans and numbers.
Added an entry to the changelog.
The rest of ansible uses validate_certs, so make that the main
documented parameter. However, leave verify as an alias since that's the
passthrough value to the underlying libraries.
- become constants inherit existing sudo/su ones
- become command line options, marked sudo/su as deprecated and moved sudo/su passwords to runas group
- changed method signatures as privlege escalation is collapsed to become
- added tests for su and become, diabled su for lack of support in local.py
- updated playbook,play and task objects to become
- added become to runner
- added whoami test for become/sudo/su
- added home override dir for plugins
- removed useless method from ask pass
- forced become pass to always be string also uses to_bytes
- fixed fakerunner for tests
- corrected reference in synchronize action plugin
- added pfexec (needs testing)
- removed unused sudo/su in runner init
- removed deprecated info
- updated pe tests to allow to run under sudo and not need root
- normalized become options into a funciton to avoid duplication and inconsistencies
- pushed suppored list to connection classs property
- updated all connection plugins to latest 'become' pe
- includes fixes from feedback (including typos)
- added draft docs
- stub of become_exe, leaving for future v2 fixes
There is an old PR that shows a great use case for having a different
set of states for the server module. Before the other modules start
being in real use, pull this out so that we don't get ourselves into a
pickle.
We have been caching the paths to the plugins but before we would only
cache the specific plugin that we were looking for. This meant that we
might search through all of the plugin directories before finding the
specific module we were interested in. The next plugin we needed we
might again search through all the plugin directories before finding the
plugin we wanted.
This new code will cache all the potential plugins in each directory
searched. For a play that only uses one plugin we'll only search
through directories until we encounter the directory that has the proper
plugin. For a large playbook with many plugins used we'll search
through each directory at most once each.
This should alleviate #10165
- make sure it calls itself correctly, now passes same params as it recieves
- vars is reserved, changed for templatevars to avoid confustion
- forcing mustaches again since the removal broke 'listification' as per #9622
- fixes incorrectly successful tests using undefined var, now it is defined
- now returns empty list if items is None to avoid errors
'_', they will be loaded after non prefixed modules are checked they can
be full modules or symlinks to existing ones (alias)
also updated ansible doc to ignore these, will eventually add selective
display
If the repo_path contained multiple '.git' strings, the _git_repo_info()
call resulted in a traceback. This change removes the trailing '.git'
and resolves the traceback.
Output a useful message if `git submodule update --init --recursive` not
yet performed
```
$ ansible --version
ansible 1.8 (submodule_ansible_version 59ae596484) last updated 2014/09/28 13:20:51 (GMT +1000)
lib/ansible/modules/core: (detached HEAD 617a52b20d) last updated 2014/09/28 13:15:25 (GMT +1000)
lib/ansible/modules/extras: not found - use git submodule update --init lib/ansible/modules/extras
```
`ansible --version` etc. now include information about submodules
```
ansible 1.8 (submodule_ansible_version ffee9a8fe0) last updated 2014/09/28 11:03:14 (GMT +1000)
lib/ansible/modules/core: (ec2_snapshot_remove 3a77c31ecb) last updated 2014/09/27 18:23:31 (GMT +1000)
lib/ansible/modules/extras: (detached HEAD 110250d344) last updated 2014/09/27 14:33:42 (GMT +1000)
```
Also improved handling of detached HEAD when printing out version
information.
Also modifies the template action plugin to use this new param
when executing the file/copy modules for templating so that links
are preserved correctly.
Fixes#8998
Ensure that the automated scm determination for github.com
repos still copes with .tar.gz archive files.
Handling .zip archives is left as an exercise for the interested reader
* Roles can now be given a friendly name as third field in role spec csv
* Roles can be installed from URL (not just from archived SCMs)
* Integration tests to demonstrate this
* Unit tests to ensure that role spec parsing works as expected
(B) This is a work in progress but the idea is to have text snippets where we can respond to tickets with a bit more helpful info.
Not active just yet.
filter_leading_non_json_lines effectively does
re.match(".*\w+=\w+.*", line)
for every line of output. This has abysmal performance in case of large
Base64-encoded data (which ultimately does not match the regex but does
match the .*\w+= part) as returned e.g. by the template module (diffs).
Replacing the match with
re.search("\w=\w", line)
drops the complexity back to linear, and actually usable with large
diffs from the template module (a 150 KB Base64 diff kept Ansible
spinning at 100% cpu for minutes).
Also, check the easy cases (line.startswith) first while we're here.
Closes: #8932
* Strip lookup calls out of inventory variables and clean unsafe data
returned from lookup plugins (CVE-2014-4966)
* Make sure vars don't insert extra parameters into module args and prevent
duplicate params from superseding previous params (CVE-2014-4967)
At the point the exception is handled, it is likely due to error loading
a sub-template included from main template. Besides file not found, it
can be caused also by include path failing Jinja2 checks. Now, when
rendering the exception from Jinja, it will include exception name. This
will give basic context or *what* the error is.
Fixes#7103Fixes#7105
If a lookup plugin is run by the lookup() template method it should pass
along any AnsibleError (or child exception classes) rather than just eat
them. These exceptions are purposely raised by the plugin.
Using sets for these operations is dangerous because sets cannot contain
certain object types (such as lists) and their iteration order is
undefined.
Fixes#7596
process_common_errors() was called thrice, each time appending to the
existing error message, and leading to confusing repetition in the
message that was finally displayed.
Fixes#7498
Addresses multiple issues when using su on freebsd including
* su prompt differs between platforms, so turned that check into a
regex comparison instead of a simple string comparison
* not using '-c' after su causes problems, so added that for all
platforms
* fixed quoting issues due to multiple uses of '-c' introduced by
the above fix
Fixes#7503Fixes#7507
Created common module doc fragment, and applied to all
modules that use ec2_connect or connect_to_aws as
they definitely share the common doc fragments
Andrew Gaffney
John R Barker
Brian Coca
Tim Rupp
Toshio Kuratomi
James Cammarata
Alexandre Garnier
Peter Sprygada
chouseknecht
Brian Coca
Peter Sprygada
art-at-sky
vlna
nitzmahone
Bede Carroll
ogenstad
René Moser
Jiri Tyr
Chris Houseknecht
Jonathan Davila
Darragh Bailey
Nathaniel Case
Strahinja Kustudić
David Wittman
Matt Clay
Brian Coca
Chris Lamb
Dag Wieers
Toshio Kuratomi
Chrrrles Paul
Charles Paul
Robin Roth
Abhijit Menon-Sen
Thilo Uttendorfer
Jonathan Mainguy
Marius Gedminas
soarpenguin
Andriy Yurchuk
Jeff Widman
Marc Abramowitz
Monty Taylor
Matt Martz
Serge van Ginderachter
Carlos E. Garcia
Feanil Patel
Shirou WAKAYAMA
Alejandro Guirao
jensdepuydt
Cove Schneider
Nicolas Le Manchet
Henk Wiedig
Mick Bass
Nathan Cahill
Nicolas Rémond
Chris Church
Rob Howard
James Laska
Michael DeHaan
Will Thames
Bruno BAILLUET
Michael DeHaan
Niko Felger
Victor Lin
y_nk
Jakub Paweł Głazik
Josh Drake
Marcus Cobden
Alexander Lozovskoy
Grzegorz Nosek
Jacob Leverich
Henry Finucane
Paul Sokolovsky
Jack Neely
Marc Pujol
Abhijit Menon-Sen
Christian Berendt
James Tanner
Cristian Ciupitu