Commit Graph

67 Commits (1ad55ec9de630ca543bb2a59adf44af4e6d97820)

Author SHA1 Message Date
James Cammarata dfb1c0647e Revert "Move queuing tasks to a background thread"
This reverts commit b71957d6e6.
2016-09-15 17:00:06 -05:00
Toshio Kuratomi 4ed88512e4 Move uses of to_bytes, to_text, to_native to use the module_utils version (#17423)
We couldn't copy to_unicode, to_bytes, to_str into module_utils because
of licensing.  So once created it we had two sets of functions that did
the same things but had different implementations.  To remedy that, this
change removes the ansible.utils.unicode versions of those functions.
2016-09-06 22:54:17 -07:00
James Cammarata b71957d6e6 Move queuing tasks to a background thread 2016-08-31 13:33:01 -05:00
nitzmahone a9322e8b8b py3-friendly multi-exception fix for missing fileno under debugger 2016-08-23 19:55:39 -07:00
nitzmahone dbf7df4439 fix pycharm debugger issue on missing fileno
Under pycharm debugger, stdin has no fileno, but throws AttributeError instead of ValueError. Account for both.
2016-08-23 17:00:14 -07:00
James Cammarata d2b3b2c03e Performance improvements 2016-08-08 15:58:46 -05:00
Abhijit Menon-Sen 950cc26aab Use loop_control.loop_var directly
6eefc11c converted task.loop_control into an object, but while the other
callers were updated to use .loop_var instead of .get('loop_var'), this
site was overlooked.

This can be reproduced by including with loop_control a file that does
set_fact; a simple regression test along these lines is included.
2016-07-01 07:03:42 +05:30
James Cammarata 1f18f5c92d Tweak the poll time to 0.005 to improve perf with large playbooks 2016-06-06 11:24:32 -05:00
Andrey Shestakov 295c7ce1c8 Fix sleep intervals (14219) 2016-05-31 09:33:58 +03:00
James Cammarata 6eefc11c39 Make the loop variable (item by default) settable per task
Required for include+with* tasks which may include files that also
have tasks containing a with* loop.

Fixes #12736
2016-04-19 01:42:40 -04:00
Toshio Kuratomi b4b058794e Correct the worker.run documentation 2016-04-12 08:01:07 -07:00
Brian Coca 005dc84aa7 moved last utils.debug to display.debug 2016-04-08 16:00:36 -04:00
James Cammarata 0f2b1244d2 Make sure VariableManager has a view of HostVars
Fixes #15261
2016-04-05 11:32:28 -04:00
James Cammarata 13f3cbaf3b Renaming per-item and retry callbacks 2016-03-18 15:17:43 -04:00
Brian Coca 66ea464ebd Merge pull request #14793 from xiaket/devel
remove main_q for simplicity.
2016-03-07 12:51:38 -05:00
Brian Coca 25e9b5788b add per item diff handling
fixes #14843
2016-03-07 09:49:18 -05:00
夏恺(Xia Kai) b33074b703 remove main_q for simplicity.
main_q is not used anywhere in the codebase.

It is created in TaskQueueManager._initialize_processes, bundled with rslt_q
into TaskQueueManger._workers, later unwrapped in StrategyBase but not used.
This queue is closed in TaskQueueManger._cleanup_processes.

Historically, it is passed as a init parameter into WorkerProcess,
introduced in 62d7956, but this behavior is changed in 120b9a7.

Signed-off-by: 夏恺(Xia Kai) <xiaket@gmail.com>
2016-03-04 07:58:18 +00:00
James Cammarata e02b98274b issue callbacks per item and retry fails
- now workers passes queue to task_executor so it can send back events per item and on retry attempt
- updated result class to pass along events to strategy
- base strategy updated to forward new events to callback
- callbacks now remove 'items' on final result but process them directly when invoked per item
- new callback method to deal with retry attempt messages (also now obeys nolog)
- updated tests to match new signature of task_executor

fixes #14558
fixes #14072
2016-02-23 15:07:06 -05:00
Brian Coca 5a1887cc76 correctly handle term signals
- adhoc now terminates gracefully
- avoid race condition on terminations by ignoring errors if
  worker might have been reaped between checking if active and termination call
- ansible-playbook now properly exits on sigint/term
- adhoc and playbook now give exceptions that we should not normally capture
  and rely on top level finally to reap children
- handle systemexit breaks in workers
- added debug to see at which frame we exit
partial fix for #14346
2016-02-08 09:51:16 -05:00
Toshio Kuratomi f4d68b8860 Transform tracebacks into unicode before printing
Fixes #14042
2016-01-25 19:18:59 -08:00
James Cammarata 120b9a7ac6 Changing the way workers are forked 2015-12-11 23:35:07 -05:00
James Cammarata 80db6bacc4 Make sure run_once tasks properly set variables for all active hosts
Fixes #13267
2015-11-30 11:27:05 -05:00
James Cammarata 25807f5404 Don't modify things we've put on the queue
Fixes #12937
2015-11-17 14:22:52 -05:00
James Cammarata 5cbeab5a3c Performance improvements for HostVars and some bugfixes 2015-11-16 17:15:52 -05:00
James Cammarata 2b25b7974c Remove unnecessary sleep from result process 2015-11-13 16:45:45 -05:00
James Cammarata 13750f68ee Reduce spinning while waiting on queues 2015-11-12 14:41:09 -05:00
James Cammarata efbc6054a4 Add variable compression option 2015-11-05 16:22:37 -05:00
James Cammarata 63c47fb271 Fixing up performance 2015-11-05 16:22:37 -05:00
Brian Coca f5b095f6c8 properly sanitize registered vars 2015-10-30 16:42:06 -04:00
Brian Coca b6c92dacb5 removed key striping as they are read up the chain 2015-10-30 14:03:37 -04:00
Brian Coca 8ce864db6f sanatize results 2015-10-24 15:23:12 -04:00
James Cammarata 86de1429e5 Cleaning up FIXMEs 2015-10-22 16:03:50 -04:00
James Cammarata 1fa975d81a Simplify group_by by removing BYPASS_HOST_LOOP from the action
Fixes #12825
2015-10-19 14:57:07 -04:00
Toshio Kuratomi a526eae543 Clean up a few more pyflakes warnings 2015-10-19 11:20:21 -07:00
Toshio Kuratomi baa309309d Bundle a new version of python-six for compatibility along with some code to make it easy for distributions to override the bunndled copy if they have a new enough version. 2015-10-16 08:21:28 -07:00
James Cammarata a3e913da62 Don't assume handlers exist in the same role (if any)
Fixes #12536
2015-09-29 14:52:43 -04:00
James Cammarata 31d5f88a1d Use the task loop to calculate multiple delegated hosts
Due to the way we're now calculating delegate_to, if that value is based
on a loop variable ('item') we need to calculate all of the possible
delegated_to variables for that loop.

Fixes #12499
2015-09-25 01:41:09 -04:00
James Cammarata bde5ed9672 Catch worker errors that may be subclassed on others that lead to incorrect exits 2015-09-17 14:46:53 -04:00
James Cammarata c2115c3715 Re-adding part of 926f127 which was actually fixing #12277 2015-09-11 09:45:27 -04:00
James Cammarata 083aef65ad Revert "set_fact should put facts in the fact_cache, not vars_cache"
This reverts commit 926f127245.
2015-09-11 09:45:27 -04:00
James Cammarata 926f127245 set_fact should put facts in the fact_cache, not vars_cache
Also fixes a template path lookup issue when using an include

Fixes #12277
2015-09-09 11:10:18 -04:00
Marius Gedminas 37be9539ff Python 3: use six.text_type instead of unicode
Replace 'unicode' with six.text_type, everywhere but in module_utils.
2015-09-04 08:40:10 +03:00
Marius Gedminas 823677b490 Replace .iteritems() with six.iteritems()
Replace .iteritems() with six.iteritems() everywhere except in
module_utils (because there's no 'six' on the remote host).  And except
in lib/ansible/galaxy/data/metadata_template.j2, because I'm not sure
six is available there.
2015-09-03 09:23:27 +03:00
Marius Gedminas 0c6ce31f76 Use 'except ... as' syntax
This syntax works on Python 2.6 through 3.x.  lib/ansible/module_utils
(and lib/ansible/modules) need to support Python 2.4, so I didn't touch
those.
2015-08-27 22:15:04 +03:00
Toshio Kuratomi 6e107d2f22 Comments pointing the way towards substituting cryptography for pycrypto 2015-08-24 15:50:37 -07:00
Brian Coca 2198f48f54 changed check to avoid OS X broken sem_getvalue()
fixes #11988
2015-08-18 03:46:20 -04:00
James Cammarata e7b2308b66 Performance improvements
* Speed up serialization of hostvars by simply using the internal
  dictionary used for cached lookups
* Use blocking gets/puts on queues instead of spin locking
* Merge sequential implicitly created blocks
2015-08-17 13:46:57 -04:00
James Cammarata 42cfacf83b Switch up the task/host overrides for PlayContext to use the compiled vars dict
Fixes #11436
2015-07-28 16:25:44 -04:00
James Cammarata cb262449c7 Reworking internal result flags and making sure include_vars hides vault data
Fixes #10194
2015-07-27 14:04:31 -04:00
James Cammarata e64989beb4 Moving ConnectionInformation -> PlayContext
Also making PlayContext a child class of the Playbook Base class,
which gives it access to all of the FieldAttribute code to ensure
field values are correctly typed after post_validation

Fixes #11381
2015-07-21 12:13:50 -04:00