Commit Graph

32 Commits (b3a57a40147ae090686c12c49871c90d17294350)

Author SHA1 Message Date
Will Thames ef8c9798d3 include_role handlers bug fix (#26335)
* Ensure that include_role properly fires handlers

include_role needs to ensure that any handlers included
with the role are added to the _notified_handler and
_listening_handler lists of the TaskQueueManager, otherwise
it fails when trying to run the handler.

Additionally, the handler needs to be added to the
PlayIterator's `_uuid_cache` or it fails after running
the handler

Add more uuid debug statements - this code was hard
to debug with existing debug statements, so add more
uuid information at little additional output cost.

Fixes #18411

* Add tests for include_role handlers

Tests for #18411
2017-07-19 15:02:32 -05:00
Toshio Kuratomi e238ae999b Cyptography pr 20566 rebase (#25560)
Make pyca/cryptography the preferred backend for cryptographic needs (mainly vault) falling back to pycrypto

pyca/cryptography is already implicitly a dependency in many cases
through paramiko (2.0+) as well as the new openssl_publickey module,
which requires pyOpenSSL 16.0+. Additionally, pyca/cryptography is
an optional dep for better performance with vault already.

This commit leverages cryptography's padding, constant time comparisons,
and CBC/CTR modes to reduce the amount of code ansible needs to
maintain.

* Handle wrong password given for VaultAES format

* Do not display deprecation warning for cryptography on python-2.6

* Namespace all of the pycrypto imports and always import them

  Makes unittests better and the code less likely to get stupid mistakes
  (like using HMAC from cryptogrpahy when the one from pycrypto is needed)

* Add back in atfork since we need pycrypto to reinitialize its RNG just in case we're being used with old paramiko

* contrib/inventory/gce: Remove spurious require on pycrypto

(cherry picked from commit 9e16b9db275263b3ea8d1b124966fdebfc9ab271)

* Add cryptography to ec2_win_password module requirements
  * Fix python3 bug which would pass text strings to a function which
    requires byte strings.

* Attempt to add pycrypto version to setup deps

* Change hacking README for dual pycrypto/cryptography

* update dependencies for various CI scripts

* additional CI dockerfile/script updates

* add paramiko to the windows and sanity requirement set

  This is needed because ansible lists it as a requirement. Previously
  the missing dep wasn't enforced, but cryptography imports pkg_resources
  so you can't ignore a requirement any more

* Add integration test cases for old vault and for wrong passwords

* helper script for manual testing of pycrypto/cryptography

* Skip the pycrypto tests so that users without it installed can still run the unittests

* Run unittests for vault with both cryptography and pycrypto backend
2017-06-27 06:00:15 -07:00
Dag Wieers 630185cb20 ansible/executor: PEP8 compliancy (#24695)
- Make PEP8 compliant
2017-05-30 18:13:53 +01:00
Brian Coca 586fcae398 tolerate 'batch' systems that mess with stdin (#23596)
* tolerate 'batch' systems taht mess with stdin

fixes #23541

* have pause on windows tolerate devnull

* tuplie
2017-04-18 11:39:49 -04:00
James Cammarata 3d65482927 Stash post-validated fields of the task in the TaskResult
This allows us to have a snapshot of the fields, which we can restore
on the pre-fork side so as to avoid having to re-template fields.
2017-02-21 14:35:13 -06:00
James Cammarata dfb1c0647e Revert "Move queuing tasks to a background thread"
This reverts commit b71957d6e6.
2016-09-15 17:00:06 -05:00
Toshio Kuratomi 4ed88512e4 Move uses of to_bytes, to_text, to_native to use the module_utils version (#17423)
We couldn't copy to_unicode, to_bytes, to_str into module_utils because
of licensing.  So once created it we had two sets of functions that did
the same things but had different implementations.  To remedy that, this
change removes the ansible.utils.unicode versions of those functions.
2016-09-06 22:54:17 -07:00
James Cammarata b71957d6e6 Move queuing tasks to a background thread 2016-08-31 13:33:01 -05:00
nitzmahone a9322e8b8b py3-friendly multi-exception fix for missing fileno under debugger 2016-08-23 19:55:39 -07:00
nitzmahone dbf7df4439 fix pycharm debugger issue on missing fileno
Under pycharm debugger, stdin has no fileno, but throws AttributeError instead of ValueError. Account for both.
2016-08-23 17:00:14 -07:00
James Cammarata d2b3b2c03e Performance improvements 2016-08-08 15:58:46 -05:00
Toshio Kuratomi b4b058794e Correct the worker.run documentation 2016-04-12 08:01:07 -07:00
Brian Coca 005dc84aa7 moved last utils.debug to display.debug 2016-04-08 16:00:36 -04:00
James Cammarata e02b98274b issue callbacks per item and retry fails
- now workers passes queue to task_executor so it can send back events per item and on retry attempt
- updated result class to pass along events to strategy
- base strategy updated to forward new events to callback
- callbacks now remove 'items' on final result but process them directly when invoked per item
- new callback method to deal with retry attempt messages (also now obeys nolog)
- updated tests to match new signature of task_executor

fixes #14558
fixes #14072
2016-02-23 15:07:06 -05:00
Brian Coca 5a1887cc76 correctly handle term signals
- adhoc now terminates gracefully
- avoid race condition on terminations by ignoring errors if
  worker might have been reaped between checking if active and termination call
- ansible-playbook now properly exits on sigint/term
- adhoc and playbook now give exceptions that we should not normally capture
  and rely on top level finally to reap children
- handle systemexit breaks in workers
- added debug to see at which frame we exit
partial fix for #14346
2016-02-08 09:51:16 -05:00
Toshio Kuratomi f4d68b8860 Transform tracebacks into unicode before printing
Fixes #14042
2016-01-25 19:18:59 -08:00
James Cammarata 120b9a7ac6 Changing the way workers are forked 2015-12-11 23:35:07 -05:00
James Cammarata 5cbeab5a3c Performance improvements for HostVars and some bugfixes 2015-11-16 17:15:52 -05:00
James Cammarata efbc6054a4 Add variable compression option 2015-11-05 16:22:37 -05:00
James Cammarata 63c47fb271 Fixing up performance 2015-11-05 16:22:37 -05:00
James Cammarata 86de1429e5 Cleaning up FIXMEs 2015-10-22 16:03:50 -04:00
Toshio Kuratomi a526eae543 Clean up a few more pyflakes warnings 2015-10-19 11:20:21 -07:00
Toshio Kuratomi baa309309d Bundle a new version of python-six for compatibility along with some code to make it easy for distributions to override the bunndled copy if they have a new enough version. 2015-10-16 08:21:28 -07:00
James Cammarata 31d5f88a1d Use the task loop to calculate multiple delegated hosts
Due to the way we're now calculating delegate_to, if that value is based
on a loop variable ('item') we need to calculate all of the possible
delegated_to variables for that loop.

Fixes #12499
2015-09-25 01:41:09 -04:00
James Cammarata bde5ed9672 Catch worker errors that may be subclassed on others that lead to incorrect exits 2015-09-17 14:46:53 -04:00
Marius Gedminas 0c6ce31f76 Use 'except ... as' syntax
This syntax works on Python 2.6 through 3.x.  lib/ansible/module_utils
(and lib/ansible/modules) need to support Python 2.4, so I didn't touch
those.
2015-08-27 22:15:04 +03:00
Toshio Kuratomi 6e107d2f22 Comments pointing the way towards substituting cryptography for pycrypto 2015-08-24 15:50:37 -07:00
James Cammarata e7b2308b66 Performance improvements
* Speed up serialization of hostvars by simply using the internal
  dictionary used for cached lookups
* Use blocking gets/puts on queues instead of spin locking
* Merge sequential implicitly created blocks
2015-08-17 13:46:57 -04:00
James Cammarata 42cfacf83b Switch up the task/host overrides for PlayContext to use the compiled vars dict
Fixes #11436
2015-07-28 16:25:44 -04:00
James Cammarata e64989beb4 Moving ConnectionInformation -> PlayContext
Also making PlayContext a child class of the Playbook Base class,
which gives it access to all of the FieldAttribute code to ensure
field values are correctly typed after post_validation

Fixes #11381
2015-07-21 12:13:50 -04:00
James Cammarata daf533c80e V2 fixes
* PluginLoader class will now be more selective about loading some
  plugin classes, if a required base class is specified (used to avoid
  loading v1 plugins that have changed significantly in their apis)
* Added ability for the connection info class to read values from a
  given hosts variables, to support "magic" variables
* Added some more magic variables to the VariableManager output
* Fixed a bug in the ActionBase class, where the module configuration
  code was not correctly handling unicode
2015-05-11 11:29:48 -05:00
James Cammarata ce3ef7f4c1 Making the switch to v2 2015-05-03 21:47:26 -05:00