The main purpose of this PR is to add the subnet associations to the
dict returned by ec2_vpc_route_table_facts. This commit also
re-formats code to make it PEP8 compliant.
New module by @tedder for handling granting/revoking access to KMS secrets.
For example:
```
- name: grant user-style access to production secrets
kms:
args:
mode: grant
key_alias: "alias/my_production_secrets"
role_name: "prod-appServerRole-1R5AQG2BSEL6L"
grant_types: "role,role grant"
```
- Consistent capitalisation in the descriptions
- Removed redundant 'optional' notes when this is covered by the Boolean `optional` column
- Clarified `instance_id` description
* set is_public 'true' or 'false'
Despite being a boolean property, https://github.com/melta/boto/blob/master/boto/ec2/image.py:63 sets is_public = True only if the argument is passed in as the string 'true'. Likewise for False/'false'.
This is a workaround for that bug in boto2, to allow the documented parameter to work with valid yaml values.
fixes#5600
* only set is_public if true
Actual doc give this error
```
fatal: [localhost]: FAILED! => {
"changed": false,
"failed": true,
"invocation": {
"module_args": {
"command": "facts",
"register": "new_database_facts"
},
"module_name": "rds"
},
"msg": "unsupported parameter for module: register"
}
```
Register should be at the module `rds` level and not at args level in this example :
```
- rds:
command: facts
instance_name: new-database
- register: new_database_facts
+ register: new_database_facts
```
* Add new parameters to taskdefinition module - network_mode and task_role_arn
* Add version_added field for doco
* Change version_added parameter to 2.3
* Allow re-using an existing template when updating a stack by not passing 'template' or 'template_url'. This is a big one for me as our deploy process creates a new stack and then modifies the old one; to avoid changing the resources inside the old one, we have had to avoid using the Ansible module and use the AWS CLI instead in order to pass `--use-previous-template`.
* Split create and update logic into separate functions
* Remove dead `update` variable
PR move of https://github.com/ansible/ansible-modules-core/pull/3588
##### ISSUE TYPE
- Docs Pull Request
##### COMPONENT NAME
ec2_group.py
##### ANSIBLE VERSION
```
ansible 2.0.2.0
config file = /Users/tpai/src/cm-secure/ansible.cfg
configured module search path = Default w/o overrides
```
##### SUMMARY
Make it clear you can specify the created group in the rules list, allowing idempotent use for group<->group networking rules.
This is a really useful feature that isn't obvious enough in the docs.
* Change example syntax on supervisorctl module
* Change example syntax or _ec2_ami_search module
* Change example syntax on cloudformation module
* Change example syntax on ec2 module
* Change example syntax on ec2_facts module
* Change example syntax on ec2_eip module
* Change example syntax on rds module
* Change example syntax on route53 module
* Change example syntax on s3 module
* Change example syntax on digital_ocean module
* Change example syntax on docker_service module
* Change example syntax on cloudformation module
* Change example syntax on gc_storage module
* Change example syntax on gce module
* Change example syntax on gce_mig module
* Change example syntax on _glance_image module
* Change example syntax on _keystone_user module
* Change example syntax on _nova_keypair module
* Change example syntax on _quantum_floating module
* Change example syntax on _quantum_floating_ip_associate module
* Change example syntax on _quantum_network module
* Change example syntax on _quantum_router module
* Change example syntax on _quantum_router_gateway module
* Change example syntax on _quantum_router_interface module
* Change example syntax on _quantum_subnet module
* SQUASH _quantum_subnet
* Add missing quotes
Moving the "check if min_size/max_size/desired_capacity..." code to execute BEFORE the desired_capacity code is used in the following operation:
num_new_inst_needed = desired_capacity - len(new_instances)
Otherwise the following exception occurs when desired_capacity is not specified and you're replacing instances:
num_new_inst_needed = desired_capacity - len(new_instances)
TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'
Stack Trace:
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
File "/var/lib/awx/.ansible/tmp/ansible-tmp-1478229985.74-62334493713074/ec2_asg", line 3044, in <module>
main()
File "/var/lib/awx/.ansible/tmp/ansible-tmp-1478229985.74-62334493713074/ec2_asg", line 3038, in main
replace_changed, asg_properties=replace(connection, module)
File "/var/lib/awx/.ansible/tmp/ansible-tmp-1478229985.74-62334493713074/ec2_asg", line 2778, in replace
num_new_inst_needed = desired_capacity - len(new_instances)
TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "invocation": {"module_name": "ec2_asg"}, "module_stderr": "Traceback (most recent call last):\n File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1478229985.74-62334493713074/ec2_asg\", line 3044, in <module>\n main()\n File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1478229985.74-62334493713074/ec2_asg\", line 3038, in main\n replace_changed, asg_properties=replace(connection, module)\n File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1478229985.74-62334493713074/ec2_asg\", line 2778, in replace\n num_new_inst_needed = desired_capacity - len(new_instances)\nTypeError: unsupported operand type(s) for -: 'NoneType' and 'int'\n", "module_stdout": "", "msg": "MODULE FAILURE", "parsed": false}
to retry, use: --limit @
In cases where a CFN stack could not complete (due to lack of
permissions or similar) but also failed to roll back, the gathering of
stack resources would fail because successfully deleted items in the
rollback would no longer have a `PhysicalResourceId` property.
This PR fixes that by soft-failing when there's no physical ID
associated to a resource.
Support the new native YAML format in the CloudFormation API. This means
the existing `template_format` parameter is deprecated. This commit also
adds a warning for the deprecated parameter.
- Don't rewrite the result; this is causing 'changed=true' on update
- Move AWSRetry import to top since it's a decorator, and is needed at definition-time
- removed star-imports, which wasn't possible in Ansible 1.x
- boto doesn't have any of the modern features (most notably, changesets), so this rewrite goes all-in on boto3.
- tags are updateable, at least in boto3. Fix documentation.
- staying with "ansible yaml to json conversion" because I'm trying to keep this scoped properly. The next PR will have AWS-native yaml support.
- documented the output. Tried to leave it backwards-compatible but the changes to 'events' might break someone's flow. However, the existing data wasn't terribly useful so I don't assume it will hurt.
- split up the code into functions. This should make unit testing possible.
- added forward-facing code: 'six' for iterating, started using AWSRetry, common tag conversion.
- add todo list
- Pass `exception` parameter to fail_json
* Restart EC2 instances with multiple network interfaces
A previous bug, #3234, caused instances with multiple ENI's to fail when being
started or stopped because sourceDestCheck is a per-interface attribute, but we
use the boto global access to it (which only works when there's a single ENI).
This patch handles a variant of that bug that only surfaced when restarting an
instance, and catches the same type of exception.
* Default termination_protection to None instead of False
AWS defaults the value of termination_protection to False, so we don't
need to explicitly send `False` when the user hasn't specified a
termination protection level. Before this patch, the below pair of tasks
would:
1. Create an instance (enabling termination_protection)
2. Restart that instance (disabling termination_protection)
Now, the default None value would prevent the restart task from
disabling termination_protection.
```
- name: make an EC2 instance
ec2:
vpc_subnet_id: {{ subnet }}
instance_type: t2.micro
termination_protection: yes
exact_count: 1
count_tag:
Name: TestInstance
instance_tags:
Name: TestInstance
group_id: {{ group }}
image: ami-7172b611
wait: yes
- name: restart a protected EC2 instance
ec2:
vpc_subnet_id: {{ subnet }}
state: restarted
instance_tags:
Name: TestInstance
group_id: {{ group }}
image: ami-7172b611
wait: yes
```
Per #3877, the code to wait for spot instance requests to finish would
hang for the full wait time if any spot request failed for any reason.
This commit introduces status checks for spot requests, so if the
request fails, finishes, or is cancelled the task will fail/succeed
accordingly.
One edge case introduced here is tha if a user terminates the instance
associated with the request manually it won't fail the play, under the
presumption that the user *wants* the instance terminated.
The AWS API requires that any termination policy list that includes
`Default` must end with Default. The attribute sorting caused any list
of attributes to be lexically sorted, so a list like
`["OldestLaunchConfiguration", "Default"]` would be changed to
`["Default", "OldestLaunchConfiguration"]` because default is earlier
alphabetically. This caused calls to fail with BotoServerError per #4069
This commit also adds proper tracebacks to all botoservererror fail_json
calls.
Closes#4069
Previously calculation of the number of instances that have been
terminated assumed all instances were in the first reservation returned
by AWS. If this is not the case the calculated number of instances
terminated never reaches the number of instances and the module always
times out. By unpacking the instances we get an accurate number and the
module correctly exits.
Currently instances with multiple ENI's can't be started or stopped
because sourceDestCheck is a per-interface attribute, but we use the
boto global access to it (which only works when there's a single ENI).
This patch handles multiple ENI's and applies the sourcedestcheck across
all interfaces the same way.
Fixes#3234
If you apply `wait=yes` and use `instance_tags` as your filter for
stopping/starting EC2 instances, this stack trace happens:
```
An exception occurred during task execution. The full traceback is: │~
Traceback (most recent call last): │~
File "/tmp/ryansb/ansible_FwE8VR/ansible_module_ec2.py", line 1540, in <module> │~
main() │~
File "/tmp/ryansb/ansible_FwE8VR/ansible_module_ec2.py", line 1514, in main │~
(changed, instance_dict_array, new_instance_ids) = startstop_instances(module, ec2, instance_ids, state, instance_tags) │~
File "/tmp/ryansb/ansible_FwE8VR/ansible_module_ec2.py", line 1343, in startstop_instances │~
if len(matched_instances) < len(instance_ids): │~
TypeError: object of type 'NoneType' has no len() │~
│~
fatal: [localhost -> localhost]: FAILED! => {"changed": false, "failed": true, "invocation": {"module_name": "ec2"}, "module_stderr": "Traceb│~
ack (most recent call last):\n File \"/tmp/ryansb/ansible_FwE8VR/ansible_module_ec2.py\", line 1540, in <module>\n main()\n File \"/tmp/│~
ryansb/ansible_FwE8VR/ansible_module_ec2.py\", line 1514, in main\n (changed, instance_dict_array, new_instance_ids) = startstop_instances│~
(module, ec2, instance_ids, state, instance_tags)\n File \"/tmp/ryansb/ansible_FwE8VR/ansible_module_ec2.py\", line 1343, in startstop_insta│~
nces\n if len(matched_instances) < len(instance_ids):\nTypeError: object of type 'NoneType' has no len()\n", "module_stdout": "", "msg": "│~
MODULE FAILURE", "parsed": false}
```
That's because the `instance_ids` variable is None if not supplied
in the task. That means the instances that result from the instance_tags
query aren't going to be included in the wait loop. To fix this, a list
needs to be kept of instances with matching tags and that list needs to
be added to `instance_ids` before the wait loop.
Before this, all spot instance requests would fail because the code
_always_ called module.fail_json when the parameter was set (which it
always was, because the module parameter's default was set to 'stop').
As the comment said, this parameter doesn't make sense for spot
instances at all, so the error message was also misleading.
Due to a mixup of the group/role/user and policy names, policies with
the same name as the group/role/user they are attached to would never be
updated after creation. To fix that, we needed two changes to the logic
of policy comparison:
- Compare the new policy name to *all* matching policies, not just the
first in lexicographical order
- Compare the new policy name to the matching ones, not to the IAM
object the policy is attached to
* Check mode fixes for ec2_vpc_net module
Returns VPC object information
Detects state change for VPC, DHCP options, and tags in check mode
* Early exit on VPC creation in check mode
The default VPC egress rules was being left in the egress rules for
purging in check mode. This ensures that the module returns the correct
change state during check mode.
AWS security groups are unique by name only by VPC (Restated, the VPC
and group name form a unique key).
When attaching security groups to an ELB, the ec2_elb_lb module would
erroneously find security groups of the same name in other VPCs thus
causing an error stating as such.
To eliminate the error, we check that we are attaching subnets (implying
that we are in a VPC), grab the vpc_id of the 0th subnet, and filtering
the list of security groups on this VPC. In other cases, no such filter
is applied (filters=None).
EC2 Security Group names are unique given a VPC. When a group_name
value is specified in a rule, if the group_name does not exist in the
provided vpc_id it should create the group as per the documentation.
The groups dictionary uses group_names as keys, so it is possible to
find a group in another VPC with the name that is desired. This causes
an error as the security group being acted on, and the security group
referenced in the rule are in two different VPCs.
To prevent this issue, we check to see if vpc_id is defined and if so
check that VPCs match, else we treat the group as new.
While from the documentation[1] one would assume that replacing
CAPABILITY_IAM with CAPABILITY_NAMED_IAM; this as empirically been shown
to not be the case.
1: "If you have IAM resources, you can specify either capability. If you
have IAM resources with custom names, you must specify
CAPABILITY_NAMED_IAM."
http://docs.aws.amazon.com/AWSCloudFormation/latest/APIReference/API_CreateStack.html
The `source_dest_check` and `termination_protection` variables are being
assigned twice in ec2.py, likely due to an incorrect merge somewhere
along the line.
The IAM group modules were not receiving the `module` object, but they
use `module.fail_json()` in their exception handlers. This patch passes
through the module object so the real errors from boto are exposed,
rather than errors about "NoneType has no method `fail_json`".
Ceph Object Gateway (Ceph RGW) is an object storage interface built on top of
librados to provide applications with a RESTful gateway to Ceph Storage
Clusters:
http://docs.ceph.com/docs/master/radosgw/
This patch adds the required bits to use the RGW S3 RESTful API properly.
Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>
The default pagination is every 100 items with a maximum of 1000 from
Amazon. This properly uses the marker returned by Amazon to concatenate
the various pages from the results.
This fixes#2440.
* Fixing error exception handling for python. Does not need to be compatible with Python2.4 b/c boto is Python 2.6 and above.
* Fixing error exception handling for python. Does not need to be compatible with Python2.4 b/c boto is Python 2.6 and above.
* Fixing compile time errors IRT error exception handling for Python 3.5.
This does not need to be compatible with Python2.4 b/c Boto is Python 2.6 and above.
This is to address this error:
fatal: [site]: FAILED! => {"changed": false, "failed": true, "msg": "Failed to connect to S3: Region does not seem to be available for awsmodule boto.s3. If the region definitely exists, you may need to upgrade boto or extend with endpoints_path"}
Commit 0dd58e9 changed the logic so an exception is thrown (by
`connect_to_aws`) before the `s3 is None` check is performed. This
changes the `None` check to a catch so the old logic can compensate.
* Add more example on how to use module ec2_tags to list tags on an instance
* Add more example on how to use module ec2_tags to list tags on an instance
'key_ids' is referenced before it is assigned, causing the module to fail with a UnboundLocalError instead of failing gracefully with a helpful error message. This very small patch moves the assignment of 'key_ids' to before the variable is referenced.
ec2_elb_lb doesn't react well to AWS API throttling errors. This
implements an exponential backoff operation around some of the AWS API
calls (with random jitter, in line with AWS recommendations) to make
this more resilient.
* Speed up AMI code by not attempting to create the AMI without checking on the name first. Also simplifies code for reporting errors from AMI creation, greatly.
* remove sys.exit
In case role policy was deleted, we did not handle at all if there
was authorization issue to do the deletion. Also add message when
role is not found and the policy is skipped.
route53 creates Record objects using `health check` and `failover`
parameters. Those parameters only became available in boto 2.28.0.
As some prominent LTS Linux releases (e.g.: Ubuntu 14.04) only ship
older boto versions (e.g.: 2.20.1 for Ubuntu 14.04), users are getting
unhelpful error messages like
TypeError: __init__() got an unexpected keyword argument 'health_check'
when running Ansible 2 against their LTS install's default boto.
We improve upon this error message by checking the boto version
beforehand.
Fixesansible/ansible#13646
OCD is making me fix the inconsistency with how None is typed. First Letter Capitalized All Over Now.
cleaning up the default object that was created for the cache_security_groups and removing checks dealing with it.
clean up space
Changing default cache_security_groups from [default] to None.
Now module will assume that if the argument is a string it is already formated as json
and will only try to convert non strings into json string.
Also removed unused 'msg' var declarations and the ifs that set them
fixes#2009
If this is not set, Ansible parses the parameter as a string.
This is fine if the parameter is not provided by the caller, but
if it is set to False or True explicitly, ec2_vol receives this as
the string 'False' or the string 'True', both of which are truthy.
Thus, without this fix, setting the parameter results in encryption
always enabled.
Previously, the `promote` command in the `rds` module would always return OK and never actually promote an instance. This was because `promote_db_instance()` had its conditions backwards: if the instance had the `replication_source` attribute indicating that it **was** a replica, it would set `changed = False` and do nothing. If the instance **wasn't** a replica, it would attempt to run `boto.rds.promote_read_replica()`, which would always fail.
'exact_count' and 'state' are mutually exclusive options they should not be in the following examples:
- # Enforce that 5 running instances named "database" with a "dbtype" of "postgres" example and
- # Enforce that 5 instances with a tag "foo" are running
There was no db restore example. I've provided one that shows how to do the restore, then add a security group (you cannot add the security group during the restore step -- it has to be done in a modify step afterward). Also, I show how to get the endpoint.
Absent unction was not working on user with login profile
also fixed the exception handling
fixed the delete user function
now works with or without loginprofile (password)
typo
When this was treated as a boolean, sphinx was leaving the Default
column on http://docs.ansible.com/ansible/ec2_module.html blank,
implying it would use AWS's default. In reality, it passes False, which
overrides the defaults at AWS (it's possible to boot an instance which
AWS claims will always have EBS optimization without it because of this
silently passed False).
This PR #1950 implements duplicate functionality to #1353
This reverts commit b04efa22c4403ca869e94e7918721306d23afa8d.
Conflicts:
cloud/amazon/elasticache.py
Toshio Kuratomi
howinator
Will Thames
John R Barker
Emlyn Stokes
Ted Timmons
Enis Ozgen
Ryan S. Brown
Rob
Kash
Rob White
Tom Paine
Lujeni
joernheissler
Laurent Goujon
Sean Summers
Ivan Nikolov
Chris Soyars
Gaël Lambert
Alex Sergeyev
tedder
Carlos E. Garcia
Andrea Tartaglia
Yury V. Zaytsev
José Sabater Montes
Adam Chainz
Maarten Bezemer
Andy Freeland
Matt Clay
James Cammarata
Fabio Alessandro Locati
Fabio Alessandro Locati
Rob White
Sam Doran
Ben Tomasik
einarc
Zaius Dr
Andrew Gaffney
jctanner
mickael-ange
Eduard Iskandarov
Jamie Dyer
Charles Paul
jjshoe
Elena Washington
Pradeep
Denis Tiago
René Moser
Shawn Siefkas
mzizzi
Christopher Kotfila
Abhijit Menon-Sen
Florian Dambrine
Matt Ferrante
Kenny Woodson
Rick Mendes
Ilja Bauer
Richard Adams
Shawn Siefkas
Rick Mendes
Lyle Mantooth
Jasmine Hegman
Shaun Brady
chrisweaver
Matthew Martin
Michael Baydoun
Kai Kousa
Adrian Moisey
Hrishikesh Barua
Javier M. Mellid
Adam Butler
Toshio Kuratomi
Bradley Phipps
Matt Davis
mansunkuo
Jonathan A. Sternberg
Michael Pappas
@
David Fischer
Roberto Bampi
Doug Luce
Nathan Brown
kwerey
Lloyd Hazlett
Kaz Cheng
hyperized
cspollar
Brian Coca
Bill W
deyvsh
Sumit Roy
Constantin
Dennis Conrad
Marcin
Joel Thompson
Michael Moussa
William Holroyd
Kalle Lehtonen
Christian Aistleitner
Arnaud Lachaume
Ashwanth Kumar
Taras Lipatov
Fabian Fülling
Jonathan Davila
Neil Saunders
Joel
Constantin07
Tom Bamford
Ryan-Neal Mes
Will Thames
Jonathan Davila
Scott Brown
Daniel Petty
Matt Martz
Michael Baydoun
Ben Visser
Shawn Silva
techraf
Yoichi Imai
Jason Paige
Bermudez, Jaime
Henrique Rodrigues
Constantin07
autotune
Joost Cassee
krdlab
Jon Hadfield
Etherdaemon
Mike Buzzetti
Andy Nelson
HAMSIK Adam
Michel Alexandre Salim
Arthur Clement
Jay Rogers
Michel Alexandre Salim
Max Rothman
joshuaeke
Keith Hassen
Daniel Donckers
Mike Riddle
J Levitt
Rabenstein
Kevin Falcone
Adrian Bridgett
Kai Webber