Commit Graph

114 Commits (5ec7f401969839da66018b8753edf5caf045fb0c)

Author SHA1 Message Date
James Cammarata ed56f51f18 Fixing security issue with lookup returns not tainting the jinja2 environment
CVE-2017-7481

Lookup returns wrap the result in unsafe, however when used through the
standard templar engine, this does not result in the jinja2 environment being
marked as unsafe as a whole. This means the lookup result looses the unsafe
protection and may become simple unicode strings, which can result in bad
things being re-templated.

This also adds a global lookup param and cfg options for lookups to allow
unsafe returns, so users can force the previous (insecure) behavior.
2017-05-08 12:43:46 -05:00
Toshio Kuratomi 1c05ed7951 Fix circular import with unsafe_proxy, template, and vars
template/__init__.py imported unsafe_proxy from vars which caused
vars/__init__.py to load.  vars/__init__.py needed template/__init__.py
which caused issues.  Loading unsafe_proxy from another location fixes
that.
2017-04-28 15:35:30 -07:00
Brian Coca 31045d58c6 added versions to dep notices (#24019)
* added versions to dep notices

* pep7

* string
2017-04-27 12:48:31 -04:00
Brian Coca a108f0fcd3 avoid hosts 'pre templated warnings' (#23878)
* avoid hosts 'pre templated warnings'

fixed is_template failing on undefined
fixes #23057

* py3 compat
2017-04-24 15:19:52 -04:00
Brian Coca 4594bee65a keep unsafe .. unsafe (#23742)
* keep unsafe .. unsafe

fixes #23734, which was broken in previous fix that allowed non string types to be templated
use new 'is_template' function vs bastardizing others
refactored clean_data to allow for arbitrary data structures to clean
fixed/removed some tests

* deal with complex data for is_template

* typos
2017-04-21 16:07:38 -04:00
Brian Coca 5b7e8a7bd6 centralize ansible_managed and other template vars
now template lookup supports these again.
2017-04-13 11:03:11 -04:00
Dag Wieers ac43a1bbbc Windows: Use the correct newline sequence for the platform (#21846)
This change to the template action plugin make template use the
platform's native newline_sequence for Jinja.

We also added the option `newline_sequence` to change the newline
sequence using by Jinja if you need to use another newline sequence than
the platform default.

This was previously discussed in
https://github.com/ansible/ansible/issues/16255#issuecomment-278289414

And also relates to issue #21128
2017-03-23 19:47:10 -07:00
Toshio Kuratomi 2fff690caa Update module_utils.six to latest (#22855)
* Update module_utils.six to latest

We've been held back on the version of six we could use on the module
side to 1.4.x because of python-2.4 compatibility.  Now that our minimum
is Python-2.6, we can update to the latest version of six in
module_utils and get rid of the second copy in lib/ansible/compat.
2017-03-23 13:35:05 -07:00
Matt Martz 3164e8b561 E501 fixes (#22879) 2017-03-22 20:50:28 -05:00
James Cammarata 86beb55a90 Also clean template data even if marked unsafe
Fixes #20568
2017-02-10 10:08:55 -06:00
James Cammarata 0d418789a2 Revert "Fixing another corner case for security related to CVE-2016-9587"
This reverts commit bcceada5d9.
2017-02-10 08:27:26 -06:00
Adrian Likins df3e4cd7f4 Don't check for var._obj in template._clean_data (#20868)
* Don't check for var._obj in template._clean_data

AnsibleUnsafe or other unsafe vars used to have a
'_obj' slot but no longer do. This was causing attribute
errors if a object was 'unsafe' but not a string.

Add tests for AnsibleUnsafe, lookups, and AnsibleContext
2017-02-08 16:09:34 -05:00
Matt Clay cb76200c7d PEP 8 E111 & E114 cleanup. (#20838) 2017-01-30 15:01:47 -08:00
Matt Clay 10d9318de7 PEP 8 indent cleanup. (#20800)
* PEP 8 E121 cleanup.

* PEP 8 E126 cleanup.

* PEP 8 E122 cleanup.
2017-01-29 07:28:53 +00:00
James Cammarata 188c3c608a Don't restrict local jinja2 variables to those that start with l_
Per a change in jinja2 2.9, local variables no longer are prefixed
with l_, so this updates AnsibleJ2Vars to pull in all locals (while
excluding some) regardless of name.

Fixes #20063

(cherry picked from commit 4d49b317929b86e1fc1b0cbace825ff73b372dc7)
2017-01-20 07:15:51 -06:00
James Cammarata b7cdc21aee Additional security fixes for CVE-2016-9587 2017-01-13 16:22:53 -06:00
Brian Coca f078946ed3 no calls plz 2017-01-12 10:35:26 -05:00
James Cammarata a94a48f85f Partial revert of 76f7ce55 2017-01-11 15:53:04 -06:00
James Cammarata d316068831 Additional fixes for security related to CVE-2016-9587 2017-01-11 15:53:04 -06:00
Computest bcceada5d9 Fixing another corner case for security related to CVE-2016-9587 2017-01-11 15:52:58 -06:00
Toshio Kuratomi 80357e07de Add some comments about how we're cleaning callables 2017-01-11 13:16:08 -08:00
Toshio Kuratomi 027b126b42 Remove _clean_data_struct() advice as that was a 1.x design
data now goes through this extra cleaning in the template rather than
being an explicit other step.
2017-01-11 11:43:56 -08:00
James Cammarata a6fff93967 Fixing security bugs for CVE-2016-9587 2017-01-09 10:43:03 -06:00
James Cammarata 85bbce9d6b Wrap unhandled errors from lookups in an AnsibleError
This provides better error handling, and prevents errors like KeyError
from bubbliing up to code in odd places.

Fixes #17482
2016-12-21 11:48:18 -06:00
Lumír 'Frenzy' Balhar 84544ee8fd Fix AST nodes for Python 3 and enable dependent test_uri (#18597)
* Enable tests on python 3 for uri

* Added one more node type to SAFE_NODES into safe_eval module.

ast.USub represents unary operators. This is necessary for
parsing some unusual but still valid JSON files during testing
with Python 3.
2016-12-13 07:37:39 -08:00
Brian Coca bd70397e24 always template when called from template (#18490)
* Have template action plugin call do_template

Avoids all the magic done for 'inline templating' for ansible plays.
renamed _do_template to do_template in templar to make externally accessible.
fixes #18192

* added backwards compat as per feedback
2016-11-15 15:16:46 -05:00
Dag Wieers cd784cd345 Remove unnecessary warnings (#18121) 2016-10-20 22:49:49 -04:00
Dag Wieers d52a9cee46 Support late evaluation, suppress warnings on SyntaxError (#17462)
See #14304 comments (https://github.com/ansible/ansible/pull/14304#issuecomment-245453209) for more information.
2016-09-09 09:33:23 -04:00
Toshio Kuratomi 4ed88512e4 Move uses of to_bytes, to_text, to_native to use the module_utils version (#17423)
We couldn't copy to_unicode, to_bytes, to_str into module_utils because
of licensing.  So once created it we had two sets of functions that did
the same things but had different implementations.  To remedy that, this
change removes the ansible.utils.unicode versions of those functions.
2016-09-06 22:54:17 -07:00
Brian Coca ff34f5548d Dynamic role include (#17401)
* dynamic role_include

* more fixes for dynamic include roles

* set play yfrom iterator when dynamic

* changes from jimi-c

* avoid modules that break ad hoc

TODO: should really be a config
2016-09-05 20:07:58 -04:00
Adrian Likins e396d5d508 Implement vault encrypted yaml variables. (#16274)
Make !vault-encrypted create a AnsibleVaultUnicode
yaml object that can be used as a regular string object.

This allows a playbook to include a encrypted vault
blob for the value of a yaml variable. A 'secret_password'
variable can have it's value encrypted instead of having
to vault encrypt an entire vars file.

Add __ENCRYPTED__ to the vault yaml types so
template.Template can treat it similar
to __UNSAFE__ flags.

vault.VaultLib api changes:
    - Split VaultLib.encrypt to encrypt and encrypt_bytestring

    - VaultLib.encrypt() previously accepted the plaintext data
      as either a byte string or a unicode string.
      Doing the right thing based on the input type would fail
      on py3 if given a arg of type 'bytes'. To simplify the
      API, vaultlib.encrypt() now assumes input plaintext is a
      py2 unicode or py3 str. It will encode to utf-8 then call
      the new encrypt_bytestring(). The new methods are less
      ambiguous.

    - moved VaultLib.is_encrypted logic to vault module scope
      and split to is_encrypted() and is_encrypted_file().

Add a test/unit/mock/yaml_helper.py
It has some helpers for testing parsing/yaml

Integration tests added as roles test_vault and test_vault_embedded
2016-08-23 20:03:11 -04:00
Dag Wieers 56148291e9 Emit warnings when safe_eval() raises a SyntaxError or other Exception (#14304)
This change is related to reported issue #14291 and pull request #14293.

Without the fix from #14293, this change will emit a warning as shown below, on the following playbook:

``yaml
---
- hosts: localhost
  gather_facts: no
  vars:
    works:
      key1: 'string'
      key2: 1234
    fails:
      key1: 'string'
      key2: 1234
      key3: false
  tasks:
  - debug: msg={{ works | to_json }}
  - debug: msg={{ fails | to_json }}
```

On error, this results in a proper warning:

```
[dag@moria ansible.dag]$ ansible-playbook test49.yml

PLAY ***************************************************************************

TASK [debug] *******************************************************************
ok: [localhost] => {
    "msg": {
        "key1": "string",
        "key2": 1234
    }
}

TASK [debug] *******************************************************************
 [WARNING]: Error in expression "{"key3": false, "key2": 1234, "key1": "string"}". (name 'false' is not defined)

ok: [localhost] => {
    "msg": "{\"key3\": false, \"key2\": 1234, \"key1\": \"string\"}"
}

PLAY RECAP *********************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0
```
2016-08-12 09:29:54 -04:00
Colin Hoglund 5a2b34e159 allow spaces before the filter string (#16312) 2016-06-15 14:17:17 -04:00
Brian Coca 005dc84aa7 moved last utils.debug to display.debug 2016-04-08 16:00:36 -04:00
Brian Coca 0b2f631197 Merge pull request #14504 from bcoca/template_comments
template also when only comments present
2016-03-06 10:45:18 -05:00
Brian Coca b901b15d3b Merge pull request #14537 from Yannig/devel_better_error_handling
Improve message content to get more hint about the raised error.
2016-03-03 18:11:04 -05:00
Brian Coca c022a43a47 fixed str to to_str 2016-03-03 14:40:35 -05:00
Brian Coca 4cfd042839 more complete and informative errors
mostly templating issues now all return data templated and actual error
fixes #14781
2016-03-03 13:52:09 -05:00
Toshio Kuratomi 1dbbd2a80c Remove debugging and fix the place where bytes are being handed to StringIO 2016-03-02 09:31:06 -08:00
Toshio Kuratomi 093b3422ec more information 2016-03-02 08:58:15 -08:00
Toshio Kuratomi 5aef65edcd Testing whether this fixes jenkins 2016-03-02 07:47:32 -08:00
James Cammarata 3247390123 Conditionally import StringIO in template so we only use io.StringIO on py3 2016-03-02 09:34:43 -05:00
Toshio Kuratomi b70bf3b056 Use io.StringIO and io.BytesIO instead of StringIO.StringIO for compat with py3 2016-02-26 16:43:05 -08:00
Matt Martz 439a385215 Catch TypeError on join, and if caught just return a single item, or a list of many 2016-02-20 16:17:58 -06:00
Yannig Perré 588045cc46 Improve message content to get more hint about the raised error. 2016-02-17 21:30:34 +01:00
Brian Coca 6102a4b9b4 template also when only comments present 2016-02-16 09:42:33 -05:00
Marc Poirier 0792983ec8 Display the string which is causing an exception to be raised.
In the ansible template module, when there is an error while
expanding a templated string, displaying the string causing the
exception is very useful.
2016-02-11 15:42:50 -05:00
Brian Coca 6149685652 Merge pull request #14293 from dagwieers/fix-eval-json-booleans
Defined JSON booleans in global context for python eval()
2016-02-04 11:33:30 -05:00
Dag Wieers 2e171610e0 Also add 'null' as a possible JSON value 2016-02-04 14:19:46 +01:00
Dag Wieers dc48d27dd2 Defined JSON booleans in global context for python eval()
We define 'false' and 'true' as variables so that python eval() recognizes them as False and True.

This fixes #14291.
2016-02-03 19:11:55 +01:00