Commit Graph

91 Commits (28b86b11485444d2dd46ed60dc8ae27fd2b81303)

Author SHA1 Message Date
Dag Wieers 9a6615a905
Make newer stuff PEP8 compliant (#26951)
So we fixed everything that was not a module to be PEP8 compliant, and
in the meantime these 5 new files were additionally disabled from PEP8
testing.

This fixes it.

Also update Copyright/License statements.
2017-11-05 11:27:41 +01:00
Sloane Hertel aa658c64ec Allow any_errors_fatal to be set in playbook - fixes #30691 (#30839)
* Allow any_errors_fatal to be set in playbook.
* Default to the config file value for any_errors_fatal only if it isn't already provided.
* add _get_attr method
2017-10-04 12:52:27 -07:00
Matt Clay 68aeaa58a8 Fix dangerous default args. (#29839) 2017-09-12 00:11:13 -07:00
Brian Coca fe3b4325c2 less confusing 'args' message (#29053)
* less confusing 'args' message

* fix test

* gone native
2017-09-06 18:20:06 -04:00
Brian Coca 725d4d4bc8 added diff as new keyword (#28581)
added diff to fake options in ec2_vpc_nat_gateway test
2017-08-24 14:43:19 +02:00
Adrian Likins 01c0b2f714 Handle errors in jmespath in json_query better (#22109)
* Handle errors in jmespath in json_query better

Catch any exceptions raised from jmespath and raise
an AnsibleFilterError instead.

Avoid a traceback.

Fixes #20379

* pep8
2017-08-17 17:16:35 -04:00
Brian Coca f921369445 Ansible Config part2 (#27448)
* Ansible Config part2

- made dump_me nicer, added note this is not prod
- moved internal key removal function to vars
- carry tracebacks in errors we can now show tracebacks for plugins on vvv
- show inventory plugin tracebacks on vvv
- minor fixes to cg groups plugin
- draft config from plugin docs
- made search path warning 'saner' (top level dirs only)
- correctly display config entries and others
- removed unneeded code
- commented out some conn plugin specific from base.yml
- also deprecated sudo/su
- updated ssh conn docs
- shared get option method for connection plugins
- note about needing eval for defaults
- tailored yaml ext
- updated strategy entry
- for connection pliugins, options load on plugin load
- allow for long types in definitions
- better display in ansible-doc
- cleaned up/updated source docs and base.yml
- added many descriptions
- deprecated include toggles as include is
- draft backwards compat get_config
- fixes to ansible-config, added --only-changed
- some code reoorg
- small license headers
- show default in doc type
- pushed module utils details to 5vs
- work w/o config file
- PEPE ATE!
- moved loader to it's own file
- fixed rhn_register test
- fixed boto requirement in make tests
- I ate Pepe
- fixed dynamic eval of defaults
- better doc code

skip ipaddr filter tests when missing netaddr
removed devnull string from config
better becoem resolution

* killed extra space with extreeme prejudice

cause its an affront against all that is holy that 2 spaces touch each other!

shippable timing out on some images, but merging as it passes most
2017-08-15 16:38:59 -04:00
Toshio Kuratomi ff22528b07 Consolidate boolean/mk_boolean conversion functions into a single location
Consolidate the module_utils, constants, and config functions that
convert values into booleans into a single function in module_utils.

Port code to use the module_utils.validate.convert_bool.boolean function
isntead of mk_boolean.
2017-07-17 11:48:05 -07:00
Brian Coca 32fa4db232 add any_errors_fatal global config 2017-07-03 17:05:44 -04:00
James Cammarata c5b8196ff1 Do not set default value for any_errors_fatal in Base
Setting default values for FieldAttribute values created in the Base class
prevents the _get_parent_attribute() code from working correctly, as the value
is always non-None.

Related to #22924
2017-06-16 00:56:06 -05:00
Brian Coca 74842adc07 1st part of ansible config, adds ansible-config to view/manage configs (#12797)
* Start of ansible config project

moved configuration definitions to external yaml file vs hardcoded
 * updated constants to be a data strcutures that are looped over and also return origin of setting
changed to manager/data scheme for base classes
new cli ansible-config to view/manage ansible configuration settings
 * prints green for default/unchanged and yellow for those that have been overriden
 * added list action to show all configurable settings and their associated ini and env var names
 * allows specifying config file to see what result would look like
 * TBD update, edit and view options

removed test for functions that have been removed

env_Vars are now list of dicts
allows for version_added and deprecation in future
added a couple of descriptions for future doc autogeneration
ensure test does not fail if delete_me exists
normalized 'path expansion'
added yaml config to setup packaging
removed unused imports
better encoding handling

updated as per feedback

* pep8
2017-06-14 11:08:34 -04:00
Abhijeet Kasurde b89cb95609 Fix spelling mistakes (comments only) (#25564)
Original Author : klemens <ka7@github.com>

Taking over previous PR as per
https://github.com/ansible/ansible/pull/23644#issuecomment-307334525

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2017-06-12 07:55:19 +01:00
Adrian Likins 4befefd78c Try to show original exception info for yaml (and other) errors (#24468)
* show original exception for yaml (and other) errors

In places where we need to catch a yaml error and raise
an AnsibleError, add the orig yaml exc to the AnsibleError
via the orig_exc arg.

When the AnsibleError is displayed it will now include the
AnsibleError (AnsibleParserError for example) and the type
and message from the original yaml exception.

This provides more detail to the error messages related to
yaml errors.

This also improves errors from dataloader (for example,
previously if a wrong password was used for a vault encrypted
yaml file, the error was very vague and suggested yaml errors,
but now the message includes the original exception from vault
indicating the password was incorrect or missing).

Add a text note to playbook helper asserts. For playbook
syntax/layout errors that aren't yaml errors, but errors
indicating invalid data structures for a playbook/task/role/block,
we now include some info about where the assert was and
why it was raised.

In places we raise an AnsibleParserError in an except
clause, pass the original exception to AnsibleParserError via
orig_exc arg.

Make assorted error messages a little more specific (like
the playbook helper load methods)

* Revert "Include the original YAML error in syntax error messages"

This reverts commit 781bb44b02.
2017-06-09 13:13:15 -04:00
Dag Wieers 5553b20828 Collated PEP8 fixes (#25293)
- Make PEP8 compliant
2017-06-02 12:14:11 +01:00
James Cammarata 78478e80ea Avoid default serialization of classes in dump_attrs
For playbook base objects, when dumping attributes via dump_attrs() an
attribute like loop_control is a class. Using the default serialization
for these is slow and consumes a lot of memory. Since LoopControl is also
based on the Base class, we can use serialize() instead and save a lot of
resources.

This also adds a from_attrs() complimentary method to nicely turn the
dumped attrs back into proper field attributes.

Fixes #23579
2017-05-11 16:28:07 -05:00
Toshio Kuratomi 2fff690caa Update module_utils.six to latest (#22855)
* Update module_utils.six to latest

We've been held back on the version of six we could use on the module
side to 1.4.x because of python-2.4 compatibility.  Now that our minimum
is Python-2.6, we can update to the latest version of six in
module_utils and get rid of the second copy in lib/ansible/compat.
2017-03-23 13:35:05 -07:00
Brian Coca 30c6a77045 moved any_errors_fatal to base (#21585) 2017-03-02 10:28:20 -05:00
James Cammarata 3d65482927 Stash post-validated fields of the task in the TaskResult
This allows us to have a snapshot of the fields, which we can restore
on the pre-fork side so as to avoid having to re-template fields.
2017-02-21 14:35:13 -06:00
James Cammarata 4cbe610263 Use a custom id generator over uuid4
>>> timeit.timeit("uuid.uuid4()", setup="import uuid")
9.518647909164429
>>> timeit.timeit("get_unique_id()", setup="from __main__ import get_unique_id")
0.40436601638793945

This will mainly be beneficial when a very large inventory is being
used, however it may also help with some very large playbooks.
2017-02-10 14:38:38 -06:00
Brian Coca ca1514cf2a unified boolean function
optimized boolean function
fixes #17815
2016-11-29 12:34:30 -05:00
James Cammarata 4859e0a419 Look for _get_parent_attribute method in both src and dst dict
When determining which getter style to use for the object in question,
the BaseMeta class should look at both dict's to try and locate the method.

Fixes #18522
2016-11-23 12:51:38 -06:00
James Cammarata 435ca620b2 Add option to prepend inherited attributes when extending values
Fixes #18483
2016-11-14 16:33:01 -06:00
James Cammarata 5b87951d6c Don't copy the parent block of TaskIncludes when loading statically
When loading an include statically, we previously were simply doing a
copy() of the TaskInclude object, which recurses up the parents creating
a new lineage of objects. This caused problems when used inside load_list_of_blocks
as the new parent Block of the new TaskInclude was not actually in the list
of blocks being operated on. In most circumstances, this did not cause a
problem as the new parent block was a proper copy, however when used in
combination with PlaybookInclude (which copies conditionals to the list of
blocks loaded) this untracked parent was not being properly updated, leading
to tasks being run improperly.

Fixes #18206
2016-11-11 08:09:43 -06:00
James Cammarata 4794b5da45 Merge class dict with parent dict when creating meta attributes
In some situations, where the Base class defines an Attribute, the
BaseMeta class doesn't properly see the _get_parent_attribute or
_get_attr_<whatever> methods because of multiple layers of subclasses
(ie. Handler, which subclasses Task). This addresses that by merging
the __dict__ of the parent with the current classes __dict__ meaning
all future iterations see available special methods.

Fixes #18378
2016-11-07 21:30:01 -06:00
Toshio Kuratomi 6a3893c518 Remove direct calls to print and cleanup imports
All display of information should go through display instead of through print.
2016-11-07 17:12:36 -05:00
Toshio Kuratomi 4ed88512e4 Move uses of to_bytes, to_text, to_native to use the module_utils version (#17423)
We couldn't copy to_unicode, to_bytes, to_str into module_utils because
of licensing.  So once created it we had two sets of functions that did
the same things but had different implementations.  To remedy that, this
change removes the ansible.utils.unicode versions of those functions.
2016-09-06 22:54:17 -07:00
Brian Coca ff34f5548d Dynamic role include (#17401)
* dynamic role_include

* more fixes for dynamic include roles

* set play yfrom iterator when dynamic

* changes from jimi-c

* avoid modules that break ad hoc

TODO: should really be a config
2016-09-05 20:07:58 -04:00
James Cammarata 9ecec6c28e Don't do parent stuff during serialize if squashed or finalized 2016-08-31 13:33:01 -05:00
James Cammarata cddf1cf98e Some further cleanup in the meta branch
* adds squashing to objects, which allows them to be squashed down
  to a final "view" before post_validate to avoid expensive evaluations
  of parent attributes
2016-08-31 13:33:01 -05:00
James Cammarata 96e2be9bf8 Selectively assign the getter for better performance
Introduces the `inherit` param for FieldAttributes, which is now used
in BaseMeta when constructing the getter property to enhance performance
by reducing the amount of work the getter generally has to do.
2016-08-31 13:33:01 -05:00
James Cammarata 3a51587220 Meta meta meta 2016-08-31 13:33:01 -05:00
James Cammarata f4237b2151 Rework the way params are assigned to TaskIncludes when they're dynamic
Copying the TaskInclude task (which is the parent) before loading the blocks
makes the code much more simple and clean, and fixes a bug introduced during
the performance improvement changes (and specifically the change which moved
things to a single-parent model).

Fixes #17064
2016-08-16 15:45:24 -05:00
James Cammarata 57fca2dde2 Validate new blocks during copy
It is possible that a block is copied prior to validation, in which case
some fields (like when) which should be something other than a string might
not be. Using validate() in copy() is relatively harmless and ensures the
blocks are in the proper structure.

This also cleans up some of the finalized logic from an earlier commit and
adds similar logic for validated.

Fixes #17018
2016-08-10 08:27:59 -05:00
James Cammarata 7503875ed7 Mark playbook objects as finalized after post_validate
After post_validate() is called on an object, there should be no
need to continue looking up at parent attributes. This patch adds a
new flag (_finalized) which is set to True at the end of post_validate,
and getattr will not look beyond its own attributes from that point on.
2016-08-09 08:57:21 -05:00
James Cammarata 06d4f4ad0e Move tasks/blocks to a single parent model 2016-08-08 15:58:46 -05:00
James Cammarata d2b3b2c03e Performance improvements 2016-08-08 15:58:46 -05:00
Robin Roth 2b28beb1d7 add check_mode option for tasks (#16056)
* add check_mode option for tasks

includes example testcases for the template module

* extend check_mode option

* replace always_run, see also proposal rename_always_run
* rename always_run where used and add deprecation warning
* add some documentation

* have check_mode overwrite always_run

* use unique template name to prevent conflicts

test_check_mode was right before, but failed due to using the same filename as other roles

* still mention always_run in the docs

* set deprecation of always_run to version 2.4

* fix rst style

* expand documentation on per-task check mode
2016-07-22 20:40:14 -04:00
James Cammarata 04fca42b62 Fixing issues with getattr caused by 5a3493b 2016-06-24 15:43:03 -05:00
nitzmahone ac04ffb9b1 apply @zazard's fix to parent attribute lookup 2016-06-23 14:11:07 -07:00
James Cammarata c24c0f5f6b Properly wrap objects using json default encoder
Our custom encoder for the to_json filter was simply returning the
object if it was not a HostVars object, leading in some cases to a
TypeError when the data contained an undefined variable. This lead
to an odd error message being propagated up, so we now properly catch
this as an undefined variable error.

Fixes #15610
2016-06-23 09:19:32 -05:00
Toshio Kuratomi 5a3493be5f Port urls.py to python3 and other byte vs text fixes (#16124)
* Port urls.py to python3

Fixes (largely normalizing byte vs text strings) for python3

* Rework what we do with attributes that aren't set already.

* Comments
2016-06-04 16:19:57 -07:00
Toshio Kuratomi 09605ce806 Fix facts for the FieldAttribute list change in 30a38f9 2016-04-22 16:31:53 -07:00
James Cammarata 30a38f94ce Create a special class of list FieldAttribute for splitting on commas
Which we're use on a case-by-case basis if we find people were actually
using comma-separated strings for list values outside of hosts. Support
for doing so is now deprecated and users should instead use the full
YAML syntax for lists of values.

Fixes #15291
2016-04-22 14:21:14 -04:00
James Cammarata 6eefc11c39 Make the loop variable (item by default) settable per task
Required for include+with* tasks which may include files that also
have tasks containing a with* loop.

Fixes #12736
2016-04-19 01:42:40 -04:00
Toshio Kuratomi d665911bab Allow FieldAttribute lists and sets to be set from a comma separated string 2016-03-14 14:53:29 -07:00
James Cammarata 299d93f6e9 Updating unit tests for PlayIterator
This knowingly introduces a broken test, planning to fix that later.
2016-03-07 13:03:06 -05:00
John Barker 8b4ebd8e4a Make it easier to read error message 2016-03-07 12:49:41 +00:00
Thomas Quinot 2eb22d5536 Fix minor typo in comment 2016-01-10 10:27:50 +01:00
Brian Coca b92cb93539 unconditionally set vars on init to avoid issues with var precedence 2015-12-01 21:24:46 -08:00
Brian Coca a6f6a80caa avoid inheritance issues with default=dict declaration at class level
this should avoid the issue of subsequent plays not prompting for a var
prompted for in a previous play.
2015-12-01 08:18:31 -08:00