Commit Graph

265 Commits (4d78c784a6743b9c3aad03cb602d4b505f73303a)

Author SHA1 Message Date
James Cammarata 04da466c7b Fixing up serialize_args utility function 2014-08-27 10:20:07 -05:00
James Cammarata e99db078b4 Additional fixes for the new omit parameter variable 2014-08-27 10:06:55 -05:00
Victor Lin 559c04f324 Implement omit for module args, also add tests for it 2014-08-27 10:04:47 -05:00
Victor Lin 0b4d7f1574 Implement default omit filter 2014-08-27 10:03:07 -05:00
y_nk affdadda25 handling case of space in path (given would start with single quote) 2014-08-25 21:26:47 +05:30
Michael DeHaan 8a0045baad Update tests for ansible-galaxy CLI additions. 2014-08-22 14:01:15 -04:00
Michael DeHaan d87830e1cc Merge branch 'galaxy_github_archive' of git://github.com/willthames/ansible into devel
Conflicts:
	lib/ansible/utils/__init__.py
2014-08-22 13:58:50 -04:00
James Cammarata d01188033f Checking for localized versions of the su password prompt
Fixes #8681
2014-08-22 09:59:40 -05:00
Michael DeHaan 0286c31e85 Assume master branch for downloads if not specified. 2014-08-22 10:01:13 -04:00
Michael DeHaan 0cd40fc372 Remove debug 2014-08-22 09:58:27 -04:00
Will Thames 850963790d Allow roles to be installed from archives on github
Ensure that the automated scm determination for github.com
repos still copes with .tar.gz archive files.

Handling .zip archives is left as an exercise for the interested reader
2014-08-22 10:06:21 +10:00
Michael DeHaan bcb44276fe Assume github.com URLs are git protocol if not specified. 2014-08-21 17:15:23 -04:00
Michael DeHaan 68e21aa4b0 Add comments about what these new functions do. 2014-08-21 17:15:23 -04:00
Will Thames 6e9abefc11 Corrected bug where role_path was wrong for roles in subdirectories
Fixed role name for
    - { role: 'lives/in/a/subdirectory' }
Should be 'lives/in/a/subdirectory', not just 'subdirectory'
2014-08-21 17:15:23 -04:00
Will Thames ada9074276 Allow installation of roles from yaml roles file
Added docs
Added more tests
Improved how roles are returned from the parsers
2014-08-21 17:15:23 -04:00
Will Thames 46b59b02ed Friendly Role Names and roles from URLs
* Roles can now be given a friendly name as third field in role spec csv
* Roles can be installed from URL (not just from archived SCMs)
* Integration tests to demonstrate this
* Unit tests to ensure that role spec parsing works as expected
2014-08-21 17:15:23 -04:00
Will Thames 4803e923ff Improved handling of role_version in repo_url_to_role_name 2014-08-21 17:15:23 -04:00
Will Thames c2fe33f9f4 Tidied up a little, added tests
Moved repo_url_to_role_name to common method in ansible.utils
Added unit test for repo_url_to_role_name
Added integration tests for galaxy
2014-08-21 17:15:23 -04:00
James Cammarata bae73e5793 Merge branch 'alozovskoy-alozovskoy-debug-ascii-patch' into devel 2014-08-21 15:57:43 -05:00
James Cammarata 4fafd3baa8 Add error checking around json.dumps when not forcing ascii mode 2014-08-21 15:52:36 -05:00
James Cammarata b81e77cfec Tweak error language in dict validation 2014-08-21 13:24:53 -05:00
Serge van Ginderachter b8d057296a variable merging: detect if both vars are really dicts when combining/merging dicts 2014-08-21 18:06:02 +02:00
James Cammarata 8a1fbed5d6 Correct variable blending from vars_files with hostvars in them
Fixes #8638
2014-08-20 15:49:01 -05:00
James Cammarata 379e31883c Fix check mode issues with copy module
Fixes #8639
2014-08-18 16:02:45 -05:00
Michael DeHaan 99c39b1ff3 By popular request, make the no_log attribute also censor tasks from callbacks. 2014-08-12 13:36:05 -04:00
Michael DeHaan fb5a1403dd Merge 2014-08-11 12:22:55 -04:00
James Cammarata 25e9d1197e Merge branch 'optimise_utils_clean_data' of https://github.com/leth/ansible into leth-optimise_utils_clean_data 2014-08-11 10:21:02 -05:00
Marcus Cobden c47d1f5265 Pre-load whole string and use seek to alter tags 2014-08-11 09:10:06 +01:00
Marcus Cobden c8bfd157f8 Switch cStringIO to StringIO for unicode support
The performance difference isn't too bad
2014-08-11 08:48:37 +01:00
Marcus Cobden 84114e5c0b Fix copy/paste error 2014-08-11 08:43:40 +01:00
Marcus Cobden ef65d3dbf6 Optimise string handling in ansible.utils._clean_data 2014-08-09 23:47:08 +01:00
James Cammarata 63bf2f6d9a Preserve all line breaks in literal blocks
Fixes #8512
2014-08-08 12:11:15 -05:00
Michael DeHaan 8c4161d4a1 Merge branch 'retf_py' of git://github.com/berendt/ansible into devel
Conflicts:
	lib/ansible/inventory/vars_plugins/group_vars.py
	plugins/callbacks/mail.py
2014-08-08 12:17:37 -04:00
Michael DeHaan 48a0236ec6 Merge pull request #6619 from sergevanginderachter/ansible_version_var
Introduce an ansible_version dict as runner variable
2014-08-08 11:20:17 -04:00
James Cammarata a45c3b84f3 Avoid templating raw lookup strings 2014-08-06 13:25:14 -05:00
Alexander Lozovskoy 5165dfa2df Issue with printing unicode non-english characters #8425 patch 2014-08-06 12:38:14 +04:00
James Cammarata 6d94ae64ec Remove unnecessary unicode conversions from parse_kv
Fixes #8425
2014-08-04 09:59:08 -05:00
Grzegorz Nosek 7f33580eba Fix exponential regex performance issue
filter_leading_non_json_lines effectively does

re.match(".*\w+=\w+.*", line)

for every line of output. This has abysmal performance in case of large
Base64-encoded data (which ultimately does not match the regex but does
match the .*\w+= part) as returned e.g. by the template module (diffs).

Replacing the match with

re.search("\w=\w", line)

drops the complexity back to linear, and actually usable with large
diffs from the template module (a 150 KB Base64 diff kept Ansible
spinning at 100% cpu for minutes).

Also, check the easy cases (line.startswith) first while we're here.

Closes: #8932
2014-08-01 14:46:00 +02:00
James Cammarata 189824dd76 Remove whitespace around args
Fixes #8343
2014-07-30 08:51:27 -05:00
James Cammarata e6fa50a306 Move splitter to module_utils so modules can use it and fix command arg splitting
Fixes #8338
2014-07-29 14:55:16 -05:00
James Cammarata aee940aaca Reworking _clean_data() to be smarter about replaces
Fixes #8228
2014-07-28 16:50:18 -05:00
Michael DeHaan 8d42f5cbfa Smush ds removal 2014-07-24 20:38:02 -05:00
Michael DeHaan 630f080cf0 Start of unit tests for split_args function, moved split_args to utils since not needed by modules (so far). 2014-07-24 20:15:04 -04:00
James Cammarata 43154e5101 Using custom splitting function for module param counting 2014-07-24 14:42:35 -05:00
James Cammarata 84759faa09 Security fixes:
* Strip lookup calls out of inventory variables and clean unsafe data
  returned from lookup plugins (CVE-2014-4966)
* Make sure vars don't insert extra parameters into module args and prevent
  duplicate params from superseding previous params (CVE-2014-4967)
2014-07-21 11:20:49 -05:00
James Cammarata 4fc8d4b6fe Merge pull request #7649 from sivel/vault-password-script
Allow --vault-password-file to work with a script as well as a flat file
2014-07-14 10:57:16 -05:00
James Cammarata db96344966 Re-fixing ability to have empty json files after inventory refactoring 2014-07-11 13:20:00 -05:00
James Cammarata fe892fccb1 Merge remote-tracking branch 'svg/varsplugins_refactor' into svg_and_inventory_refactor 2014-07-07 14:16:37 -05:00
James Cammarata 8ed6350e65 When parsing json from untrusted sources, remove templating tags 2014-07-01 14:14:14 -05:00
James Cammarata eeb597360e Further safe_eval fixes 2014-07-01 14:14:14 -05:00
James Cammarata 35368e531b Additional fixes for safe_eval 2014-06-25 21:26:32 -05:00
James Cammarata 5429b85b9f Change safe_eval to a strict white list 2014-06-25 14:00:21 -05:00
Matt Martz 19f5ce2c9c Allow --vault-password-file to work with a script as well as a flat file 2014-06-24 15:02:24 -05:00
Chris Church 243cd877ae Add integration tests for fetch/slurp, make powershell fetch/slurp work as close as possible to existing fetch/slurp modules. 2014-06-19 14:25:28 -05:00
Marc Pujol d0f82e94e8 Ensure there are no duplicates in the merged/intersected lists 2014-05-30 15:47:18 +02:00
Marc Pujol f892cc798c Merge and intersect lists without using sets.
Using sets for these operations is dangerous because sets cannot contain
certain object types (such as lists) and their iteration order is
undefined.

Fixes #7596
2014-05-30 09:07:57 +02:00
James Cammarata 92f16b3d6f Merge pull request #7539 from jimi-c/issue_7503_freebsd_su_fixes
Fixes for su on freebsd
2014-05-25 15:09:58 -05:00
Abhijit Menon-Sen f6792b724a Don't double (or triple) up common error messages
process_common_errors() was called thrice, each time appending to the
existing error message, and leading to confusing repetition in the
message that was finally displayed.

Fixes #7498
2014-05-23 21:05:42 +05:30
James Cammarata 1e672a0fec Fixes for su on freebsd
Addresses multiple issues when using su on freebsd including
* su prompt differs between platforms, so turned that check into a
  regex comparison instead of a simple string comparison
* not using '-c' after su causes problems, so added that for all
  platforms
* fixed quoting issues due to multiple uses of '-c' introduced by
  the above fix

Fixes #7503
Fixes #7507
2014-05-23 10:06:09 -05:00
James Cammarata 85bd6810bb Pass vars_files on to included playbooks too
Fixes #7384
2014-05-14 08:37:47 -05:00
Christian Berendt 6676720ce5 fixed typos found by RETF rules in PY files
rules are avaialble at https://en.wikipedia.org/wiki/Wikipedia:AutoWikiBrowser/Typos
2014-05-03 18:40:05 +02:00
James Cammarata 8e45fa9b63 Moving display-related functions to new module in utils 2014-04-30 15:33:46 -05:00
James Cammarata 6069ff6e9e Adding a new system_warnings config option to supress warnings 2014-04-30 14:44:10 -05:00
James Cammarata 4cadcccc48 Catch pycrypto warning about gmp and show a nice warning on stderr 2014-04-29 14:28:14 -05:00
James Cammarata b9d8b3b911 Merge pull request #7132 from jimi-c/issue_6601_hide_vault_yaml
Hide YAML content on syntax errors when a vault password is specified
2014-04-24 00:28:12 -05:00
James Cammarata 5e598c5337 Fix handling of non-JSON lines in responses
Garbage lines with ' = ' in them were causing parsing errors,
where key=val lines should not have spaces around the equals.

Fixes #6962
2014-04-23 15:17:31 -05:00
James Cammarata 93b5769d94 Hide YAML content on syntax errors when a vault password is specified
Fixes #6601
2014-04-23 09:23:07 -05:00
James Cammarata 2cc4ac2e75 Catch traceback caused by permissions errors during a local md5 sum
Fixes #6948
2014-04-14 12:41:21 -05:00
Serge van Ginderachter d240e2b9fb Introduce an ansible_version dict as runner variable
Given the version:
    "1.6 (ansible_version_var 14499e8bf3) last updated 2014/03/21 17:07:50 (GMT +200)"

  We get the special variable:
    "ansible_version": {
        "full": "1.6",
        "major": 1,
        "minor": 6,
        "revision": 0,
        "string": "1.6 (ansible_version_var 14499e8bf3) last updated 2014/03/21 17:07:50 (GMT +200)"
       }

	modified:   lib/ansible/playbook/__init__.py
	modified:   lib/ansible/runner/__init__.py
	modified:   lib/ansible/utils/__init__.py
2014-04-08 17:14:42 +02:00
James Cammarata a4df906fc9 Fixes to safe_eval 2014-04-01 16:23:22 -05:00
Cristian Ciupitu 1eaf85b89f Micro-optimization: replace s.find(x)!=-1 with x in s
timeit shows a speedup of ~3x on Python 2.7.5 x86_64.
It also makes the code a bit shorter.
2014-03-27 20:35:20 +02:00
Serge van Ginderachter b0ff1ea425 performance optimisation in hash merge logic
rewrite deepcopy in util.merge_hash and just iterate
on an inventory with 500 groups and 800 hosts this brings back the
inventory initialisation from 13s to 3s (with hash_behaviour=merge)
2014-03-26 11:37:31 +01:00
Serge van Ginderachter d4634983f0 Move group/host_vars parsing into core inventory
modified:   lib/ansible/inventory/__init__.py
	deleted:    lib/ansible/inventory/vars_plugins/group_vars.py
	modified:   lib/ansible/utils/__init__.py
2014-03-26 11:37:26 +01:00
Michael DeHaan c42dbf80a2 Merge pull request #6586 from sivel/probline-logic-fix
Fix logic in process_common_errors for unbalanced quotes
2014-03-19 17:45:49 -04:00
James Tanner e71857fbdf Addresses #6579 Only strip vault passwords if given 2014-03-19 16:08:35 -04:00
James Tanner 8c2e1e2baa Addresses #6579 Disallow vault passwords with newline characters by stripping them in utils 2014-03-19 15:56:14 -04:00
James Cammarata a9000e7f3a Making the error messages for bad key/dir permissions more understandable
Fixes #6254
2014-03-19 14:09:03 -05:00
Matt Martz 2aaecc5afa Fix logic in process_common_errors for unbalanced quotes 2014-03-19 13:49:31 -05:00
Michael DeHaan 94e3350b38 Catch a unquoted line error. Fixes #6532 2014-03-17 17:15:42 -04:00
James Tanner b14c658532 Fix concatenation for with_file 2014-03-17 10:38:22 -04:00
Cristian Ciupitu 0749112286 Micro-optimization: replace s.find(x)!=-1 with x in s
timeit shows a speedup of ~3x on Python 2.7.5 x86_64.
It also makes the code a bit shorter.
2014-03-16 13:10:28 -04:00
Matt Martz 4b400ca5e9 Only used stripped data for testing if the file is json, but used unstripped when actually parsing. Fixes #6348 2014-03-10 13:43:34 -05:00
Michael DeHaan e486dbab04 Use the loaded variable since we have it assigned, not just for error detection. 2014-03-06 22:15:21 -05:00
Michael DeHaan 3f07ec3d73 Fixup JSON error reporting in previous commits. 2014-03-06 22:07:35 -05:00
Michael DeHaan 79799f6819 Allow any file that can be in YAML to also be in JSON. This is primarily done to support non-visual editors better. 2014-03-06 21:42:51 -05:00
Michael DeHaan 94a7fb60fe Allow escaped comments in inventory files. 2014-03-05 20:10:25 -05:00
Richard C Isaacson b41d8106ff Revert "Merge pull request #6287 from risaacson/fix_hash_in_var"
This reverts commit a808287451, reversing
changes made to 6129ea7566.
2014-03-05 14:49:30 -06:00
Richard C Isaacson 38abd5e20e Break this out into a reusable function and document regex shortcomings. 2014-03-05 10:50:58 -06:00
Richard C Isaacson 9c8bbe2904 column must always be less than len(probline)
Fixes related to GH-5773.
2014-02-26 16:28:34 -06:00
Richard C Isaacson 585766201d When looking for double colon len(line) is longer then 1.
Addresses GH-5116.

It comes up that when parsing json that if you are missing the last double quote on the last variable and the next line is just '}' we will get an out of range error. In this instance we will also then make sure that the line is long enough to have two colons.
2014-02-26 14:25:56 -06:00
James Tanner 35def422a3 Addresses #6188 Add --vault-password-file to bin/ansible and bin/ansible-playbook 2014-02-26 11:01:03 -05:00
James Tanner 52a8efefba Vault rewrite, pass 1 2014-02-24 13:09:36 -05:00
James Tanner 84467a8f12 Change retype to confirm 2014-02-19 11:06:53 -05:00
James Tanner aa46ab1d86 Fix function name 2014-02-19 11:05:08 -05:00
James Tanner 427b8dc78d Ansible vault: a framework for encrypting any playbook or var file. 2014-02-18 15:35:12 -05:00
James Cammarata b8646991db Filter out messages that match the pattern username:password@host
This filtering is done in both the module invocation logging and in
the regular verbose() logging output.

Fixes #4087
2014-02-13 14:23:49 -06:00
James Cammarata bb79b16d59 Specify mode in the makedirs call for fireball/accelerate keys 2014-02-10 13:27:43 -06:00
xyrix 20d97416e5 fixed thinkoes 2014-02-06 13:02:11 +00:00
xyrix 0af40374ed fixed file perms checking 2014-02-06 12:48:34 +00:00
xyrix 1de45bf687 made accelerate keys directory configurable, and permissions for the file and dir configurable, and gave them a safe default 2014-02-06 08:53:43 +00:00