2641f70e47
Reset self._groups_cache in refresh_inventory
2015-08-19 11:35:27 +05:30
73d6da757f
Make get_group run in O(n*log(n)) instead of O(n^2) by using a cache
2015-08-19 11:35:27 +05:30
b8865314ec
improve ansible start up time for very large inventory
...
first off, we add an oddly slow basic test of 10k item inventory
Before:
```
Ran 229 tests in 13.214s
OK
real 0m13.403s
user 0m12.106s
sys 0m1.155s
```
After:
```
Ran 230 tests in 21.328s
OK
real 0m21.516s
user 0m20.099s
sys 0m1.275s
```
since that seems like a bit long for the test to add to runtime, lets profile
`python -m cProfile -s time ./bin/ansible all -i test/units/inventory_test_data/huge_range --list-hosts`
Before:
```
1272607 function calls (1259689 primitive calls) in 8.497 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
10000 4.393 0.000 4.396 0.000 __init__.py:395(_get_host)
20000 2.695 0.000 2.697 0.000 __init__.py:341(__append_host_to_results)
40369 0.113 0.000 0.113 0.000 {posix.lstat}
50006 0.102 0.000 0.153 0.000 __init__.py:1490(combine_vars)
40008 0.089 0.000 0.202 0.000 __init__.py:1546(_load_vars_from_path)
20195 0.088 0.000 0.088 0.000 {posix.stat}
10011 0.087 0.000 0.087 0.000 {posix.getcwd}
```
The top two lines are promising optimization targets
- populate Inventory's host cache more in _get_host, as we are looping
over all the groups anyways.
- eliminate duplicate check of whether we've already included a host
in the construction around __append_host_to_results we can infer
presence of a host in the results list implies the presence of its
name in the hostnames set, allowing us to only to the less expensive
of the two checks
After:
```
1252610 function calls (1239692 primitive calls) in 1.320 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
40369 0.105 0.000 0.105 0.000 {posix.lstat}
50006 0.094 0.000 0.141 0.000 __init__.py:1490(combine_vars)
40008 0.081 0.000 0.184 0.000 __init__.py:1546(_load_vars_from_path)
10011 0.080 0.000 0.080 0.000 {posix.getcwd}
20195 0.074 0.000 0.074 0.000 {posix.stat}
10002 0.069 0.000 0.261 0.000 __init__.py:1517(load_vars)
```
2015-08-19 09:45:02 +05:30
8f231ec469
Tweak deprecation warning, and apply it to --limit also
2015-08-14 10:55:13 +05:30
4dba30ccd0
Cleanup missed merge conflict sections
2015-08-13 15:05:58 -04:00
d41db8edd6
Don't accept undocumented "p1;p2" patterns; note in changelog
...
The correct (and now only) way to say p1 OR p2 is "p1:p2".
2015-08-13 14:54:12 -04:00
1437b8e120
Introduce _split_pattern to parse an "x:y:z" pattern
...
This function takes a string like 'foo:bar[1:2]:baz[x:y]-quux' and
returns a list of patterns ['foo', 'bar[1:2]', 'baz[x:y]-quux'], i.e.
splits the string on colons that are not part of a range specification.
2015-08-13 14:54:12 -04:00
32034feec5
Add deprecation warning regarding host patterns
2015-08-13 14:51:08 -04:00
6bf6b9fdbb
Rename some internal functions, clarify what they do
...
get_hosts → used externally, not changed
_get_hosts → _evaluate_patterns (takes a list, evaluates ! and &)
__get_hosts → _match_one_pattern (takes one pattern only, ignores !&)
2015-08-13 14:51:08 -04:00
eff33bac49
Don't accept undocumented "p1;p2" patterns; note in changelog
...
The correct (and now only) way to say p1 OR p2 is "p1:p2".
2015-08-13 14:51:08 -04:00
8aaa06dd50
Introduce _split_pattern to parse an "x:y:z" pattern
...
This function takes a string like 'foo:bar[1:2]:baz[x:y]-quux' and
returns a list of patterns ['foo', 'bar[1:2]', 'baz[x:y]-quux'], i.e.
splits the string on colons that are not part of a range specification.
2015-08-13 14:51:08 -04:00
baf637b9ae
Remove dead code: Inventory.also_restrict_to/lift_also_restriction
...
This was used earlier to implement serial, but that's now done using
restrict_to_hosts() (whose docstring is also suitably adjusted here)
and there are no more callers.
2015-08-13 14:51:08 -04:00
b905f51341
fixed typo
2015-08-13 11:05:20 -04:00
fbc7224066
Merge pull request #11643 from bcoca/meta_inventory_and_fixes
...
Meta inventory and fixes
2015-08-13 10:36:57 -04:00
5fec11f513
capture parsing exceptions and give nice error and location
2015-08-12 23:27:00 -04:00
49d88cef9c
meta: refresh_inventory and several inventory fixes
2015-08-12 10:35:12 -04:00
8e30e3791e
Merge pull request #11811 from amenonsen/fixme-range
...
FIXME in host range parsing
2015-08-12 09:06:39 -04:00
26d5a17b59
Make sure inventory loads files with non-relative paths
...
This is a slightly different fix than we originally committed, but fixes
the problem in a less invasive way (and I believe it's generally better
that we don't deal with relative paths internally past this point)
Fixes #11789
2015-08-06 10:18:58 -04:00
b9433650d1
Revert "Path of group_vars and host_vars were getting the basedir added twice."
...
in view of simpler solution incomming from james
This reverts commit bae7a02be5
2015-08-06 10:09:43 -04:00
bae7a02be5
Path of group_vars and host_vars were getting the basedir added twice.
...
Fix inventory so this won't happen and fix DataLoader so that it will
test relative paths relative to self._basedir
Fixes #11789
2015-08-05 17:41:17 -07:00
f0431eaea9
Make ansible_python_interpreter work with synchronize
...
Fixes #11836
2015-08-03 13:32:24 -07:00
7d9689c1c5
Slice ascii_letters directly instead of using 'for x in range()'
2015-08-02 23:47:18 +05:30
bfe708a189
Make host range parsing errors issue better messages
...
Now we always say "host range must …specific thing…"
2015-08-02 23:47:18 +05:30
1413496292
Implement step for alphabetic ranges: [a:e:2] => a,c,e
2015-08-02 23:47:18 +05:30
f16b745f17
now properly uses default port if defined
2015-07-31 12:01:58 -04:00
845784b572
Remove outdated FIXMEs; all use combine_vars already
2015-07-30 23:47:41 +05:30
77fc3ce759
removed unused import
2015-07-28 08:28:52 -04:00
49a6601856
Further cleanup of internal use of ansible_ssh_host
2015-07-27 10:42:39 -04:00
a1a8997e89
Merge pull request #11663 from whereismyjetpack/fix_ansible_ssh_host
...
only set ansible_ssh_host if not already set
2015-07-26 23:46:21 -04:00
65ae9780fa
Merge pull request #11079 from amegianeg/fix_broad_except_in_inventory
...
Avoid too broad except in Inventory
2015-07-23 13:36:02 -04:00
3e79bfa2d6
only set ansible_ssh_host if not already set
2015-07-20 15:43:55 -04:00
d23ab261e1
fixes 11607, allows ansible_ssh_port to be overridden from group or host_vars
2015-07-16 23:00:17 +01:00
4e94bb64d8
Fix group/host var loading relative to playbook basedir
2015-07-14 09:26:24 -04:00
a9712bb0fb
Fixing some delegate_to bugs
...
* Moving connection creation until after the task is post_validated,
to make sure all fields are properly templated (#11230 )
* Fixing problems related to the connection method and remote address
lookup on the delegated-to host
Fixes #11230
2015-07-09 08:25:08 -04:00
ff251a0dcc
Catch runtime errors due to recursion when calculating group depth
...
Fixes #7708
2015-06-22 02:06:07 -04:00
f17bdc4d61
Set the ansible_ssh_port variable instead of saving it internally for Host
...
Fixes #11330
2015-06-22 00:38:41 -04:00
87ca475704
Exclude the all/ungrouped groups from pattern matching results
...
Fixes #5375
2015-06-17 16:25:58 -04:00
90445ee67d
Add ::1 where we see 127.0.0.1, for better ipv6 support
...
Fixes #5764
2015-06-17 16:03:19 -04:00
a385744426
Add inventory file to "Unable to find" error msg
...
E.g.:
$ ansible gabriel -m ping -i ssh_config.py
ERROR! Unable to find an inventory file (ssh_config.py), specify one with -i ?
2015-06-17 11:00:13 -07:00
30c1a2d861
Have group/host var file loading check for YAML extensions too
...
Fixes #11132
2015-06-16 11:55:56 -04:00
f9b56a5d7c
Fix raising AnsibleError
2015-06-05 11:41:23 -07:00
96836412aa
Make error messages tell which inventory script the error came from
2015-06-03 11:51:33 -07:00
1c8527044b
Fix error handling when pasing output from dynamic inventory
2015-06-03 11:29:20 -07:00
620fad9f8d
Fixing an oops in inventory/__init__.py where the slots are incorrect
2015-06-01 20:02:15 -05:00
fe41f109a9
Merge branch 'v2_final' into devel_switch_v2
2015-06-01 16:42:59 -05:00
4bc7703db3
Fixing some small bugs related to integration tests (v2)
2015-06-01 16:42:10 -05:00
2bad888f28
Merge branch 'v2_final' into devel_switch_v2
...
Conflicts:
lib/ansible/inventory/__init__.py
lib/ansible/modules/core
lib/ansible/utils/__init__.py
lib/ansible/utils/module_docs.py
2015-05-28 15:26:03 -05:00
37ae5aab31
Capture only IOError when reading shebang from inventory file, to avoid ignoring other possible exceptions like timeouts from a task
2015-05-28 12:19:32 +02:00
0913b8263c
made special treatment of certain filesystem for selinux configurable
2015-05-15 13:52:27 -04:00
e2de336a23
made special treatment of certain filesystem for selinux configurable
2015-05-15 13:19:15 -04:00
ce3ef7f4c1
Making the switch to v2
2015-05-03 21:47:26 -05:00
c3d34a5386
Merge pull request #8936 from srvg/inventoryscript_onlychildren
...
Allow InventoryScript JSON with childgroups only
2015-03-05 17:11:04 -05:00
084fbbdea9
Reverse the conditional so there's one less return from this function
2015-02-26 22:50:11 -08:00
2e929cf0ce
Fixes bug #10281 - Trailing zeros were truncated from strings
2015-02-21 19:54:38 +01:00
a14248ffe1
Fixes bug #10281 - Trailing zeros were truncated from strings
2015-02-21 16:40:50 +01:00
751701c6f2
Merge pull request #9672 from quinot/topic/lineno-in-inventory-err
...
Report location (filename and line number) for inventory syntax errors
2015-01-29 12:36:30 -05:00
a63e4c595f
Another place that needs to be json_dict_bytes_to_unicode
2015-01-26 22:04:51 -08:00
0f6b87d33e
Revert "Fix import statements" -- those impotrt statements were right
...
before... something else is fishy
This reverts commit 8ab536fbf2
2015-01-26 22:01:52 -08:00
8ab536fbf2
Fix import statements
2015-01-26 21:49:09 -08:00
915d232d5f
jinja2 cannot handle byte strs with non-ascii. So we need to transform potential byte str into unicode type. This fix is for dynamic inventory.
...
Fixes #10007
2015-01-26 20:39:09 -08:00
2ef5d6be1e
Merge pull request #9702 from bcoca/better_errors_on_delegate
...
better exception handling with delegated hosts
2015-01-19 15:25:01 -05:00
6b3d0f4788
now captures returncode errors on inventory scripts
2015-01-15 13:38:25 -05:00
61a30e5f49
better exception handling with delegated hosts
2014-12-03 07:26:42 -05:00
23d959db71
Report location (filename and line number) for inventory syntax errors
...
When AnsibleError is raised for a syntax error in an inventory file,
report filename and line number to help pinpointing the error.
2014-11-30 10:41:09 +01:00
d19fe8d95d
Fetch vars for host directly when calculating the delegated user
...
This fixes the case in which the delegated to host may not be in the
specified hosts list, in which cases facts/vars for the host were
not available in the injected hostvars.
This also fixes the inventory variable fetching function, so that an
unknown host raises a proper error as opposed to a NoneType exception.
Fixes #8224
2014-10-10 01:18:18 -05:00
a10d10f647
Workaround more python-2.6 shlex not being able to handle unicode strings
2014-10-08 14:30:36 -04:00
cef2a8795f
Make dynamic inventory return byte str, not unicode
2014-10-02 20:27:06 -04:00
26cdddaebf
Tracebacks are now catchable with ignore_errors and have streamlined output. Also removes 'baby-JSON' for bash modules.
2014-09-11 12:27:21 -04:00
69740b86e8
Allow InventoryScript JSON with childgroups only
...
and without hosts and vars
Without this patch, the simplified syntax is triggered when a group
is defined like this:
"platforms": {
"children": [
"cloudstack"
]
}
Which results in a group 'platforms' with 1 host 'platforms'.
modified: lib/ansible/inventory/script.py
2014-09-09 11:37:54 +02:00
3a228b9d55
InventoryScript: better syntax checking for json stream
2014-08-21 18:06:07 +02:00
a1f09bd89f
Merge variables for hosts instead of updating in get_variables()
2014-08-20 01:54:46 -05:00
0ab721d51e
better logic for all-group variables for implicit localhost
2014-08-19 16:41:51 +02:00
a2cfe8770b
Inventory: fix logic mistake in loading/retrieving variables for groups
2014-08-19 16:34:26 +02:00
e3895840d1
Make sure group_vars/all is loaded for ungrouped hosts
...
Fixes #6563
2014-08-18 12:29:46 -05:00
b4476c238f
Use match() instead of search() for pattern matching
...
Also related to f48fa3, fix for #8614
2014-08-14 12:26:52 -05:00
f48fa3759a
Fix host pattern matching and enhance error detection
...
Fixes #8614
2014-08-14 12:13:40 -05:00
bea0845322
Fix vault_password positional args error
2014-08-14 10:28:08 -05:00
8c4161d4a1
Merge branch 'retf_py' of git://github.com/berendt/ansible into devel
...
Conflicts:
lib/ansible/inventory/vars_plugins/group_vars.py
plugins/callbacks/mail.py
2014-08-08 12:17:37 -04:00
7c96f1d803
InventoryDir: fixing yet another corner case
2014-08-08 08:54:11 +02:00
e8b45a9ef7
Invalidate host/group variables cache when loading
...
extra varsiables based on playbook basedir.
2014-07-14 15:21:33 +02:00
d39ff55688
Fix logic bug when loading vars for a new playbook basedir.
2014-07-14 15:12:53 +02:00
4e5eb7559e
Merge branch 'svg_and_inventory_refactor' into devel
2014-07-10 14:43:07 -05:00
307a3e8a9c
Fixed regex square-bracket bug.
...
Regexes were being parsed like ordinary ansible host patterns, so
square-bracket groups were getting interpolated wrongly.
2014-07-10 00:08:12 +00:00
aa261bdd14
Optimizing groups_for_host() lookup in inventory
2014-07-07 21:08:39 -05:00
fe892fccb1
Merge remote-tracking branch 'svg/varsplugins_refactor' into svg_and_inventory_refactor
2014-07-07 14:16:37 -05:00
11a5fc85f4
Merge branch 'inventory_refactoring' into svg_and_inventory_refactor
2014-07-07 13:11:58 -05:00
bcf83cbc24
Merge branch 'sergevanginderachter-inventorytree' into svg_and_inventory_refactor
2014-07-07 13:11:52 -05:00
ff4119adc0
Performance optimization in resolving host patterns
...
Avoid resolving a pattern that is a plain host. When matching a hostname in the
hosts_cache, just use the host object from there.
When running a task on say 750 hosts, this yields a huge improvement.
2014-07-07 12:50:50 -05:00
ff04b2b532
Allow empty yaml vars files
...
Fixes #7843
2014-07-04 23:10:32 -05:00
c2ac8fda1e
Data from inventory scripts should be able to use nested Jinja2 expressions.
2014-07-03 16:11:14 -04:00
8ed6350e65
When parsing json from untrusted sources, remove templating tags
2014-07-01 14:14:14 -05:00
539426f612
Performance tuning inventory functions for large inventories
2014-06-26 23:15:39 -05:00
d8a7b49b59
Merge branch 'inventorytree' of https://github.com/sergevanginderachter/ansible into sergevanginderachter-inventorytree
2014-06-24 15:21:08 -05:00
92bd755b47
Fixed extension filtering in InventoryDirectory
2014-05-14 23:43:14 -07:00
393124e462
Add pyc/pyo to ignore list for inventory directories
...
Fixes #7308
2014-05-07 09:59:13 -05:00
6676720ce5
fixed typos found by RETF rules in PY files
...
rules are avaialble at https://en.wikipedia.org/wiki/Wikipedia:AutoWikiBrowser/Typos
2014-05-03 18:40:05 +02:00
e36e2d38fe
InventoryDir: another fix for the host.groups list
...
In some cases, where a host is mentioned in multiple groups, and those
groups are referenced in multiple ini files, a group could still contain
multiple instances of a group in its host,groups list, where only one of them
is the right group, that exists in the inventory.
2014-04-04 09:27:44 +02:00
96ff1444f0
Micro-optimization: replace s.find(x)==0 with s.startswith(x)
...
timeit shows a speedup of ~1.4x on Python 2.7.5 x86_64.
It also makes the code a bit shorter.
2014-03-27 20:35:20 +02:00
d4634983f0
Move group/host_vars parsing into core inventory
...
modified: lib/ansible/inventory/__init__.py
deleted: lib/ansible/inventory/vars_plugins/group_vars.py
modified: lib/ansible/utils/__init__.py
2014-03-26 11:37:26 +01:00
f8ea93c732
Move inventory.set_playbook_basedir from ansible-playbook to playbook constructor
2014-03-25 22:12:01 +01:00
Abhijit Menon-Sen
Harald Laabs
Will West
James Cammarata
Brian Coca
Brian Coca
Toshio Kuratomi
Dann Bohn
Gerard Lynch
Benno Joy
Marc Abramowitz
alberto
Alvaro Lopez Ortega
Thomas Quinot
Michael DeHaan
Serge van Ginderachter
Michael DeHaan
Dan Slimmon
Kevin Bell
Christian Berendt
Cristian Ciupitu