Commit Graph

49 Commits (9b8abc8a7fc677bd2ab58bbe19b228de533bf303)

Author SHA1 Message Date
Brian Coca f921369445 Ansible Config part2 (#27448)
* Ansible Config part2

- made dump_me nicer, added note this is not prod
- moved internal key removal function to vars
- carry tracebacks in errors we can now show tracebacks for plugins on vvv
- show inventory plugin tracebacks on vvv
- minor fixes to cg groups plugin
- draft config from plugin docs
- made search path warning 'saner' (top level dirs only)
- correctly display config entries and others
- removed unneeded code
- commented out some conn plugin specific from base.yml
- also deprecated sudo/su
- updated ssh conn docs
- shared get option method for connection plugins
- note about needing eval for defaults
- tailored yaml ext
- updated strategy entry
- for connection pliugins, options load on plugin load
- allow for long types in definitions
- better display in ansible-doc
- cleaned up/updated source docs and base.yml
- added many descriptions
- deprecated include toggles as include is
- draft backwards compat get_config
- fixes to ansible-config, added --only-changed
- some code reoorg
- small license headers
- show default in doc type
- pushed module utils details to 5vs
- work w/o config file
- PEPE ATE!
- moved loader to it's own file
- fixed rhn_register test
- fixed boto requirement in make tests
- I ate Pepe
- fixed dynamic eval of defaults
- better doc code

skip ipaddr filter tests when missing netaddr
removed devnull string from config
better becoem resolution

* killed extra space with extreeme prejudice

cause its an affront against all that is holy that 2 spaces touch each other!

shippable timing out on some images, but merging as it passes most
2017-08-15 16:38:59 -04:00
Toshio Kuratomi 90b1d780eb Fix redis cache for python3
According to the redis-py docs, zrank will return the 0 based index for
the value in the sorted set.  So the logic here wasn't right to begin
with (It just means that a value at the 0-th position would never show
up as cached).  Need to compare against None to know if the value
exists in the cache.

https://redis-py.readthedocs.io/en/latest/#redis.StrictRedis.zrank

Fixes #25590
2017-06-13 09:50:46 -04:00
Brian Coca 5717f4f161 updated cache descriptions 2017-06-09 23:48:47 -04:00
Dag Wieers 5553b20828 Collated PEP8 fixes (#25293)
- Make PEP8 compliant
2017-06-02 12:14:11 +01:00
Toshio Kuratomi 84a59e472b Be explicit about pickle protocol and encoding (#24454)
On Python3 and Python2 use pickle slightly differently so we need to be
explicit about some things.

If pickles could be shared between python2 and python3, as in
ansible-connection and the pickle cache, we need to specify the protocol
to use when dumping and the encoding to use for byte strings when
loading.

The dumping protocol needs to be no higher than 2 as python-2 only
supports up to protocol 2.  The encoding should usually be 'bytes' so
that python2 str type becomes python3 bytes type.  However, doing this
means that we must make sure that the objects being serialized properly
make their strings into text strings except when they're supposed to be
bytes.  If strings are improperly byte strings, they may cause
tracebacks on the receiving end
2017-05-12 18:04:48 -07:00
Toshio Kuratomi 2fff690caa Update module_utils.six to latest (#22855)
* Update module_utils.six to latest

We've been held back on the version of six we could use on the module
side to 1.4.x because of python-2.4 compatibility.  Now that our minimum
is Python-2.6, we can update to the latest version of six in
module_utils and get rid of the second copy in lib/ansible/compat.
2017-03-23 13:35:05 -07:00
Brian Coca 7839f70e36 Enable documentation in plugins
Made ansible-doc more plugin agnostic
We can have docs in lookup, callback, connectionm strategy, etc
Use first docstring and make pepizis happy
generalized module_docs to plugin_docs
documented cartesian, ssh, default, jsonfile, etc as examples
changed lack of docs to warning when listing
made smarter about bad docstrings
better blacklisting
added handling of options/config/envs/etc
move blacklist to find_plugins, only need once
2017-03-23 01:27:19 -04:00
Brian Coca d3115ae8f3 put cache base classes in __init__ (#22809)
* put cache base classes in __init__

* added placeholder
2017-03-23 01:26:10 -04:00
Matt Clay 0b250016b0 Fix PEP 8 issue. 2017-03-08 10:05:19 -08:00
Brian Coca 406505bfe5 fixes cache file modules with bad path permissions
now module will fail to load and report an error (turned into warning as this should not be fatal)
fixes #13093
2017-03-08 10:58:53 -05:00
Toshio Kuratomi 45251f910c Make BaseFileCache into an abstractbaseclass so it's a proper interface
Push the opening and closing of files into the _load and _dump methods
so that we don't invoke the slow codec machinery without reason.
2017-02-22 10:27:29 -05:00
Brian Coca 374af06cbf added pickle and yaml cache plugins
added new base class for file based cache plugins as 99% of code was common
now also catches unexpected decoding exceptions
allows per module file modes and encoding
moved jsonfile code to base
2017-02-22 10:27:29 -05:00
Brian Coca f550b4541f bad/missing fact plugin is a warning (#19350) 2016-12-14 14:25:19 -05:00
Adrian Likins 6c4f554f5a Provide slightly better msg on fact cache error (#18759)
If the configured fact_cache plugin (fact_caching config)
fails, raise a fatal error instead of failing mysteriously
later.

Fixes #18751
2016-12-13 14:34:58 -05:00
Toshio Kuratomi 6a3893c518 Remove direct calls to print and cleanup imports
All display of information should go through display instead of through print.
2016-11-07 17:12:36 -05:00
Adrian Likins 3e754086b6 Fix error using jsonfile with incomplete config (#17567)
If 'fact_caching=jsonfile' was configured, but
'fact_caching_connection' was not configured, jsonfile
would fail and ansible-playbook would exit with a traceback.

Fixes #17566
2016-09-16 15:08:02 -04:00
Adrian Likins 8438da2a34 Make jsonfile cache plugin errors less vague. (#17568) 2016-09-14 14:10:12 -04:00
Adrian Likins c633022fca [wip] Let jsonfile and memcached cache plugins understand fact_caching_timeout=0 (#17565)
* Add support for no-expiration to jsonfile cache

* Let memcached cache use fact_caching_timeout=0

If fact_cache=memcached and fact_caching_timeout=0
memcached would hit a NameError on _expire_keys
2016-09-14 13:27:42 -04:00
Toshio Kuratomi 4ed88512e4 Move uses of to_bytes, to_text, to_native to use the module_utils version (#17423)
We couldn't copy to_unicode, to_bytes, to_str into module_utils because
of licensing.  So once created it we had two sets of functions that did
the same things but had different implementations.  To remedy that, this
change removes the ansible.utils.unicode versions of those functions.
2016-09-06 22:54:17 -07:00
Dag Wieers 718f431466 Allow to make the jsonfile cache files pretty (indented and sorted) (#17000)
* Allow to make the jsonfile cache files pretty (indented and sorted)

Since the json cache files are condensed, it is not very practical to look for something in them. Having indented/sorted cache files makes debugging and playbook/inventory development a lot easier to do.

I made it configurable in case people would object to the performance hit this would have, but to be honest, then they probably should be looking at other cache plugins instead IMO.

* Removed the config option and documentation changes
2016-08-08 17:15:19 -04:00
Brian Coca 07846f7bec ignore closing file error when file not open
fixes #16268
2016-06-14 09:34:41 -04:00
Brian Coca f576082949 restore old jsonfile behaviour on key expiration
fixes #14456, now it won't expire keys in middle of a play when they
were 'valid' at 'gather time'.
2016-05-10 15:55:33 -04:00
Toshio Kuratomi edab8d338d Expand tilde after expanding vars 2016-04-05 22:27:07 -07:00
Brian Coca af68ae1e87 switched from threading to multiprocessing
functions same, just keeping names from confusing people
2016-02-11 22:25:17 -05:00
Daniel Menet 0918416b83 substitute tilde and env vars before storing C.CACHE_PLUGIN_CONNECTION as instance attribute 2015-11-12 16:52:02 +01:00
Toshio Kuratomi 62979efa14 Finish up plugin porting to global display
Also remove display = display which does nothing
2015-11-11 10:44:23 -08:00
Toshio Kuratomi 2bd695ed42 Port cache plugins to global display 2015-11-11 10:44:22 -08:00
James Cammarata 86de1429e5 Cleaning up FIXMEs 2015-10-22 16:03:50 -04:00
Toshio Kuratomi f34b55ac2b Add python3-compat boilerplate to all .py files in lib/ansible 2015-10-19 18:36:19 -07:00
Toshio Kuratomi 0888d78b84 Cleanup some pyflakes warning (1 real error) 2015-10-19 11:42:46 -07:00
James Cammarata 0aa018337a Fixing logic in json cache plugin get()
Fixes #12722
2015-10-17 11:50:14 -04:00
Toshio Kuratomi baa309309d Bundle a new version of python-six for compatibility along with some code to make it easy for distributions to override the bunndled copy if they have a new enough version. 2015-10-16 08:21:28 -07:00
Brian Coca f5227d494b added missing cachefile and changed str(e) to to_bytes(e) 2015-10-13 11:15:10 -04:00
James Cammarata 426187489f Add variable for format parameter 2015-10-13 02:33:18 -04:00
James Cammarata 5c5806d669 Fixing bugs related to jfonfile cache plugin
* corrupt/invalid file causes tracebacks
* incorrect initialization of display/_display in BaseCacheModule class
* tweaking the way errors in get() on jsonfile caches work, to raise
  a proper AnsibleError in that situation so the playbook/task is stopped

Fixes #12708
2015-10-12 13:06:41 -04:00
Matthew Jones 7da478a4f4 Fix an issue where cache plugins weren't updated
The first call to persisting facts would work due to the assignment of a
MutableMapping calling __setitem__ but subsequent module fact data would
not be propogated to the fact cache plugins because update() doesn't
invoke __setitem__.  This changes the behavior a little bit and ensures
set() is called on cache plugins.
2015-10-09 11:16:41 -04:00
Brian Coca 2463922306 fixed error reporting when missing redis import 2015-10-01 18:49:51 -04:00
Marius Gedminas a32bf1ec18 Simplify FactCache.copy()
Also fix the bug (missing from six import iteritems) I introduced in
823677b490.
2015-09-03 09:44:44 +03:00
Marius Gedminas 823677b490 Replace .iteritems() with six.iteritems()
Replace .iteritems() with six.iteritems() everywhere except in
module_utils (because there's no 'six' on the remote host).  And except
in lib/ansible/galaxy/data/metadata_template.j2, because I'm not sure
six is available there.
2015-09-03 09:23:27 +03:00
James Cammarata 66ed397360 Fix redis fact_caching_timeout=0 bug
Fixes #12018
2015-08-28 11:47:06 -04:00
Marius Gedminas 0c6ce31f76 Use 'except ... as' syntax
This syntax works on Python 2.6 through 3.x.  lib/ansible/module_utils
(and lib/ansible/modules) need to support Python 2.4, so I didn't touch
those.
2015-08-27 22:15:04 +03:00
Brian Coca 851ed45bbf adding display to plugins and start moving debug to display 2015-07-23 10:26:12 -04:00
James Cammarata d977da5b41 Fixing up fact_cache use in VariableManager 2015-07-13 11:06:03 -04:00
James Cammarata 932d1e57f7 Removing stray debugging print 2015-07-13 10:56:09 -04:00
James Cammarata 91c9bb96e3 Moving jsonfile cache plugin over and fixing #10883
Fixes #10883
2015-07-13 10:42:10 -04:00
Toshio Kuratomi 7dd3ef7b60 Older python-six from early RHEL and ubuntu do not have add_metaclass but do have with_metaclass 2015-06-03 13:29:24 -07:00
James Cammarata 0f1eb3cfc2 Better fix for serializing/deserializing cache plugins (v2) 2015-05-07 23:56:33 -05:00
James Cammarata cd6d1f9221 Fix pickling errors with cache plugins (v2)
Fixes #10945
2015-05-07 21:15:02 -05:00
James Cammarata ce3ef7f4c1 Making the switch to v2 2015-05-03 21:47:26 -05:00