Commit Graph

107 Commits (9ecec6c28ed048dcceacdfd3d98dcee7c05f3d23)

Author SHA1 Message Date
Michael Scherer acd69bcc77 Fix url lookup for python 3 (#17295)
* Use six instead of urllib2, for python 3 compat

* Open the certificate file using binary mode

On python3, os.write requires 'bytes'. Also avoid
using a too broad exception, since the issue was hard
to spot due to it.

* Do not add the header User-agent if not set

Python3 module do raise a exception if a header is
not a string-like object, and the default value is None.
2016-08-31 07:03:20 -07:00
Michael Scherer 9245c786db Fix the lines lookup to work on python3 (#17291)
Since stdout is (on python3) of type 'bytes', callbacks
plugins fail in the test suite, because calls backs expect
a string.
2016-08-30 10:38:55 -07:00
Michael Scherer 8f364b549b Fix indexed_items.py to run on python3 (#17292)
On python3, zip is a iterator so we need
to explictily create the list from that.
2016-08-30 09:49:58 -07:00
Toshio Kuratomi 0a39700b36 Fix octal output in a few more places (#17250)
Fix filetree lookup plugin for python3 (octal output and selinux API
takes native strings)
2016-08-25 14:58:35 -07:00
Toshio Kuratomi 7b4f808a21 One more izip_longest => zip_longest fix (#17229) 2016-08-24 13:50:14 -07:00
Toshio Kuratomi 51ec35378d xrange and izip_longest aren't available in vanilla python3 (#17226)
Fixes for these are either rewriting to get rid of the need for the
functions or using six.moves to get equivalent functions for both
python2 and python3
2016-08-24 12:28:02 -07:00
Toshio Kuratomi a22909c226 Migrate basestring to a python3 compatible type (#17199) 2016-08-23 13:13:44 -07:00
Toshio Kuratomi 76f9935634 Add some missing imports from last night's py3 fixes (#17196) 2016-08-23 08:06:20 -07:00
Marius Gedminas 73caff58e8 Add missing to_unicode import 2016-08-23 09:24:02 +03:00
Marius Gedminas 483c83b088 There's no ansible.module_utils_unicode 2016-08-23 09:22:03 +03:00
Toshio Kuratomi 313d4b2c9e Move a path being passed around as a byte string to being passed around as a text string. (#17190)
This is enough to get minimal copy module working on python3

We have t omodify dataloader's path_dwim_relative_stack and everything
that calls it to use text paths instead of byte string paths
2016-08-22 21:55:30 -07:00
Dag Wieers 1268f4778d Introduce new 'filetree' lookup plugin (#14332)
* Introduce new 'filetree' lookup plugin

The new "filetree" lookup plugin makes it possible to recurse over a tree of files within the task loop. This makes it possible to e.g. template a complete tree of files to a target system with little effort while retaining permissions and ownership.

The module supports directories, files and symlinks.

The item dictionary consists of:
 - src
 - root
 - path
 - mode
 - state
 - owner
 - group
 - seuser
 - serole
 - setype
 - selevel
 - uid
 - gid
 - size
 - mtime
 - ctime

EXAMPLES:
Here is an example of how we use with_filetree within a role:

```yaml
 - name: Create directories
   file:
     path: /web/{{ item.path }}
     state: directory
     mode: '{{ item.mode }}'
     owner: '{{ item.owner }}'
     group: '{{ item.group }}'
     force: yes
   with_filetree: web/
   when: item.state == 'directory'

 - name: Template complete tree
   file:
     src: '{{ item.src }}'
     dest: /web/{{ item.path }}
     state: 'link'
     mode: '{{ item.mode }}'
     owner: '{{ item.owner }}'
     group: '{{ item.group }}'
   with_filetree: web/
   when: item.state == 'link'

 - name: Template complete tree
   template:
     src: '{{ item.src }}'
     dest: /web/{{ item.path }}
     mode: '{{ item.mode }}'
     owner: '{{ item.owner }}'
     group: '{{ item.group }}'
     force: yes
   with_filetree: web/
   when: item.state == 'file'
```

SPECIAL USE:
The following properties also have its special use:

 - root: Makes it possible to filter by original location
 - path: Is the relative path to root
 - uid, gid: Makes it possible to force-create by exact id, rather than by name
 - size, mtime, ctime: Makes it possible to filter out files by size, mtime or ctime

TODO:
 - Add snippets to documentation

* Small fixes for Python 3

* Return the portion of the file’s mode that can be set by os.chmod()

And remove the exists=True, which is redundant.

* Use lstat() instead of stat() since we support symlinks

* Avoid a few possible stat() calls

* Bring in line with v1.9 and hybrid plugin

* Remove glob module since we no longer use it

* Included suggestions from @RussellLuo

- Two blank lines will be better. See PEP 8
- I think if props is not None is more conventional 😄

* Support failed pwd/grp lookups

* Implement first-found functionality in the path-order
2016-08-11 23:33:54 -04:00
jctanner 39d764c7ad Restore previous behavior of ignoring missing files via with_fileglob (#17053)
Fixes #16801
2016-08-11 19:55:21 -04:00
victoru 14901b65d9 raise AnsibleError in hashi_vault lookup plugin when hvac module is not installed (#16859) 2016-08-04 10:06:12 -07:00
Brian Coca 3c39bb5633 fixed lookup search path (#16630)
* fixed lookup search path

added ansible_search_path var that contains the proper list and in order
removed roledir var which was only used by first_found, rest used role_path
added needle function for lookups that mirrors the action plugin one, now
both types of plugins use same pathing.

* added missing os import

* renamed as per feedback

* fixed missing rename in first_found

* also fixed first_found

* fixed import to match new error class

* fixed getattr ref
2016-07-13 10:06:34 -04:00
jctanner 4ba60d00c8 Refactor the parameter splitting in ini lookup to handle more path formats (#16477)
Refactor the parameter splitting in ini lookup to handle more path formats.

Fixes #16468
2016-06-28 13:24:57 -04:00
Pilou b361bf90d7 Lookup password omit salt (#16361)
* Lookup unencrypted password must not include salt
* Integration test lookup: remove previous directory
* Test that lookup password doesn't return salt
* Lookup password: test behavior with empty encrypt parameter

Closes #16189
2016-06-27 10:44:25 -07:00
James Cammarata aefa12a20f Merge pull request #13709 from traveloka/credstash-support-encr-context
Make credstash lookup plugin support encryption contexts
2016-06-21 11:00:43 -05:00
rdezavalia a7274774ed Support etcd v2. Use this version by default (#12312)
* Support etcd v2. Use this version by default

* default to etcd v1
2016-06-06 11:26:12 -04:00
feliksik 95cf095222 hashi_vault lookup: be more rebust, and allow fields with other name than 'value' (#13690)
* more robust hashi_vault module, and allow querying specific field in secret-dict

* allow fetching entire secret dict with trailing ':'

* process comment by bcoca for PR #13690
2016-05-14 21:48:31 -04:00
Toshio Kuratomi c75b5ba85f Use to_bytes and to_str instead of str as str is unsafe. 2016-04-04 12:57:32 -07:00
Dag Wieers 055aae55f7 Implement own CSVRecoder and CSVReader from documentation
As detailed in the python manual:
https://docs.python.org/2/library/csv.html
2016-04-04 20:32:08 +02:00
Dag Wieers ac75069ee0 Support different encoding types in csvfile lookup plugin
This fixes #15266.
2016-04-04 14:11:44 -04:00
Brian Coca 0bd6f3a5e8 correctly adds searchpath as list 2016-04-01 23:14:56 -04:00
Toshio Kuratomi b70bf3b056 Use io.StringIO and io.BytesIO instead of StringIO.StringIO for compat with py3 2016-02-26 16:43:05 -08:00
Brian Coca a8aa5ff4eb fix cartesian lookup
it seems that this was not working in 1.9 but we swallowed up the error
fixes #14437
2016-02-11 10:35:37 -05:00
Dag Wieers dee5dba82a Handle when the lookup-plugin is used in jinja-style
This fixes #14190.
2016-02-03 17:10:40 +01:00
Brian Coca dd4787701f fix unicode input for template lookup
fixes #14207
2016-01-29 10:42:52 -05:00
Brian Coca c2ac1507ea corrected host/group match in inventory_hostnames
now the lookup works when using ! and & operators
fixes #13997
2016-01-20 18:32:39 -05:00
Michel Alexandre Salim afb2abf980 Make credstash lookup plugin support encryption contexts
Previously, the lookup plugin passes all its keyword arguments to
credstash's `getSecret`; while this works for passing the standard
parameters (version, region and table), this does not allow passing
a dictionary of key-value pairs as `getSecret`'s context parameter.

Instead, pop `version`, `region` and `table` from `kwargs`, supplying
the default value if they are not defined, and pass the rest of the `kwargs`
as the `context` parameter.
2016-01-02 15:23:27 +07:00
Brian Coca 89646d4253 no need to be executable 2015-11-16 14:40:35 -08:00
Sebi Calbaza 1f052d5ce6 fixed super invocation 2015-11-15 13:48:40 +02:00
Toshio Kuratomi 958fb29b78 Merge pull request #12881 from Scypho/devel
fix for issue #11885 - allowing json in lookup template calls
2015-11-12 15:39:35 -08:00
Toshio Kuratomi 62979efa14 Finish up plugin porting to global display
Also remove display = display which does nothing
2015-11-11 10:44:23 -08:00
Toshio Kuratomi b05d0b8c9c Port lookup plugins to global display. 2015-11-11 10:44:22 -08:00
Toshio Kuratomi db5d3f03a9 Fix first_found not finding files in roles (only templates) 2015-11-09 13:47:17 -08:00
Brian Coca 57d3245d0d remove commented out prints 2015-10-30 14:02:41 -04:00
Jens Peter Schroer ecae636160 allowing json in lookup template calls 2015-10-23 14:21:46 +02:00
James Cammarata 86de1429e5 Cleaning up FIXMEs 2015-10-22 16:03:50 -04:00
Toshio Kuratomi f34b55ac2b Add python3-compat boilerplate to all .py files in lib/ansible 2015-10-19 18:36:19 -07:00
Toshio Kuratomi 7aa9f6754a Cleanup more pyflakes warnings (2 real problems) 2015-10-19 12:01:01 -07:00
Toshio Kuratomi 0888d78b84 Cleanup some pyflakes warning (1 real error) 2015-10-19 11:42:46 -07:00
Brian Coca 6d87bd405d Merge pull request #12765 from bcoca/lookup_inventory_hostname
implemented inventory_hostname lookup in v2
2015-10-16 15:05:23 -04:00
Toshio Kuratomi baa309309d Bundle a new version of python-six for compatibility along with some code to make it easy for distributions to override the bunndled copy if they have a new enough version. 2015-10-16 08:21:28 -07:00
Brian Coca 9b088a1e40 implemented inventory_hostname lookup in v2 2015-10-15 13:11:42 -04:00
Toshio Kuratomi 8277747a04 Fix password lookup when password file exists and the password is not encrypted 2015-10-12 15:52:03 -07:00
Toshio Kuratomi b49e0995cb Fix typo 2015-10-12 13:29:04 -07:00
Toshio Kuratomi b58d7470d4 Cleanup logic and be more paranoid about passwords with spaces 2015-10-12 12:55:33 -07:00
Brian Coca f5f3bf06e4 Merge pull request #12550 from redrampage/remove-pw-salt-deletion
Removed deletion of salt by 'password' lookup
2015-10-12 11:33:19 -04:00
Brian Coca 4ba5ebd1c8 Merge pull request #12538 from yesbox/url_lookup_plugin_httperror_fix
Fix url lookup plugin HTTPError message not being reachable
2015-10-02 21:30:16 -04:00