* Allow re-using an existing template when updating a stack by not passing 'template' or 'template_url'. This is a big one for me as our deploy process creates a new stack and then modifies the old one; to avoid changing the resources inside the old one, we have had to avoid using the Ansible module and use the AWS CLI instead in order to pass `--use-previous-template`.
* Split create and update logic into separate functions
* Remove dead `update` variable
PR move of https://github.com/ansible/ansible-modules-core/pull/3588
##### ISSUE TYPE
- Docs Pull Request
##### COMPONENT NAME
ec2_group.py
##### ANSIBLE VERSION
```
ansible 2.0.2.0
config file = /Users/tpai/src/cm-secure/ansible.cfg
configured module search path = Default w/o overrides
```
##### SUMMARY
Make it clear you can specify the created group in the rules list, allowing idempotent use for group<->group networking rules.
This is a really useful feature that isn't obvious enough in the docs.
* Change example syntax on supervisorctl module
* Change example syntax or _ec2_ami_search module
* Change example syntax on cloudformation module
* Change example syntax on ec2 module
* Change example syntax on ec2_facts module
* Change example syntax on ec2_eip module
* Change example syntax on rds module
* Change example syntax on route53 module
* Change example syntax on s3 module
* Change example syntax on digital_ocean module
* Change example syntax on docker_service module
* Change example syntax on cloudformation module
* Change example syntax on gc_storage module
* Change example syntax on gce module
* Change example syntax on gce_mig module
* Change example syntax on _glance_image module
* Change example syntax on _keystone_user module
* Change example syntax on _nova_keypair module
* Change example syntax on _quantum_floating module
* Change example syntax on _quantum_floating_ip_associate module
* Change example syntax on _quantum_network module
* Change example syntax on _quantum_router module
* Change example syntax on _quantum_router_gateway module
* Change example syntax on _quantum_router_interface module
* Change example syntax on _quantum_subnet module
* SQUASH _quantum_subnet
* Add missing quotes
Moving the "check if min_size/max_size/desired_capacity..." code to execute BEFORE the desired_capacity code is used in the following operation:
num_new_inst_needed = desired_capacity - len(new_instances)
Otherwise the following exception occurs when desired_capacity is not specified and you're replacing instances:
num_new_inst_needed = desired_capacity - len(new_instances)
TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'
Stack Trace:
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
File "/var/lib/awx/.ansible/tmp/ansible-tmp-1478229985.74-62334493713074/ec2_asg", line 3044, in <module>
main()
File "/var/lib/awx/.ansible/tmp/ansible-tmp-1478229985.74-62334493713074/ec2_asg", line 3038, in main
replace_changed, asg_properties=replace(connection, module)
File "/var/lib/awx/.ansible/tmp/ansible-tmp-1478229985.74-62334493713074/ec2_asg", line 2778, in replace
num_new_inst_needed = desired_capacity - len(new_instances)
TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "invocation": {"module_name": "ec2_asg"}, "module_stderr": "Traceback (most recent call last):\n File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1478229985.74-62334493713074/ec2_asg\", line 3044, in <module>\n main()\n File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1478229985.74-62334493713074/ec2_asg\", line 3038, in main\n replace_changed, asg_properties=replace(connection, module)\n File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1478229985.74-62334493713074/ec2_asg\", line 2778, in replace\n num_new_inst_needed = desired_capacity - len(new_instances)\nTypeError: unsupported operand type(s) for -: 'NoneType' and 'int'\n", "module_stdout": "", "msg": "MODULE FAILURE", "parsed": false}
to retry, use: --limit @
In cases where a CFN stack could not complete (due to lack of
permissions or similar) but also failed to roll back, the gathering of
stack resources would fail because successfully deleted items in the
rollback would no longer have a `PhysicalResourceId` property.
This PR fixes that by soft-failing when there's no physical ID
associated to a resource.
Support the new native YAML format in the CloudFormation API. This means
the existing `template_format` parameter is deprecated. This commit also
adds a warning for the deprecated parameter.
- Don't rewrite the result; this is causing 'changed=true' on update
- Move AWSRetry import to top since it's a decorator, and is needed at definition-time
- removed star-imports, which wasn't possible in Ansible 1.x
- boto doesn't have any of the modern features (most notably, changesets), so this rewrite goes all-in on boto3.
- tags are updateable, at least in boto3. Fix documentation.
- staying with "ansible yaml to json conversion" because I'm trying to keep this scoped properly. The next PR will have AWS-native yaml support.
- documented the output. Tried to leave it backwards-compatible but the changes to 'events' might break someone's flow. However, the existing data wasn't terribly useful so I don't assume it will hurt.
- split up the code into functions. This should make unit testing possible.
- added forward-facing code: 'six' for iterating, started using AWSRetry, common tag conversion.
- add todo list
- Pass `exception` parameter to fail_json
* Restart EC2 instances with multiple network interfaces
A previous bug, #3234, caused instances with multiple ENI's to fail when being
started or stopped because sourceDestCheck is a per-interface attribute, but we
use the boto global access to it (which only works when there's a single ENI).
This patch handles a variant of that bug that only surfaced when restarting an
instance, and catches the same type of exception.
* Default termination_protection to None instead of False
AWS defaults the value of termination_protection to False, so we don't
need to explicitly send `False` when the user hasn't specified a
termination protection level. Before this patch, the below pair of tasks
would:
1. Create an instance (enabling termination_protection)
2. Restart that instance (disabling termination_protection)
Now, the default None value would prevent the restart task from
disabling termination_protection.
```
- name: make an EC2 instance
ec2:
vpc_subnet_id: {{ subnet }}
instance_type: t2.micro
termination_protection: yes
exact_count: 1
count_tag:
Name: TestInstance
instance_tags:
Name: TestInstance
group_id: {{ group }}
image: ami-7172b611
wait: yes
- name: restart a protected EC2 instance
ec2:
vpc_subnet_id: {{ subnet }}
state: restarted
instance_tags:
Name: TestInstance
group_id: {{ group }}
image: ami-7172b611
wait: yes
```
Per #3877, the code to wait for spot instance requests to finish would
hang for the full wait time if any spot request failed for any reason.
This commit introduces status checks for spot requests, so if the
request fails, finishes, or is cancelled the task will fail/succeed
accordingly.
One edge case introduced here is tha if a user terminates the instance
associated with the request manually it won't fail the play, under the
presumption that the user *wants* the instance terminated.
The AWS API requires that any termination policy list that includes
`Default` must end with Default. The attribute sorting caused any list
of attributes to be lexically sorted, so a list like
`["OldestLaunchConfiguration", "Default"]` would be changed to
`["Default", "OldestLaunchConfiguration"]` because default is earlier
alphabetically. This caused calls to fail with BotoServerError per #4069
This commit also adds proper tracebacks to all botoservererror fail_json
calls.
Closes#4069
Previously calculation of the number of instances that have been
terminated assumed all instances were in the first reservation returned
by AWS. If this is not the case the calculated number of instances
terminated never reaches the number of instances and the module always
times out. By unpacking the instances we get an accurate number and the
module correctly exits.
Currently instances with multiple ENI's can't be started or stopped
because sourceDestCheck is a per-interface attribute, but we use the
boto global access to it (which only works when there's a single ENI).
This patch handles multiple ENI's and applies the sourcedestcheck across
all interfaces the same way.
Fixes#3234
If you apply `wait=yes` and use `instance_tags` as your filter for
stopping/starting EC2 instances, this stack trace happens:
```
An exception occurred during task execution. The full traceback is: │~
Traceback (most recent call last): │~
File "/tmp/ryansb/ansible_FwE8VR/ansible_module_ec2.py", line 1540, in <module> │~
main() │~
File "/tmp/ryansb/ansible_FwE8VR/ansible_module_ec2.py", line 1514, in main │~
(changed, instance_dict_array, new_instance_ids) = startstop_instances(module, ec2, instance_ids, state, instance_tags) │~
File "/tmp/ryansb/ansible_FwE8VR/ansible_module_ec2.py", line 1343, in startstop_instances │~
if len(matched_instances) < len(instance_ids): │~
TypeError: object of type 'NoneType' has no len() │~
│~
fatal: [localhost -> localhost]: FAILED! => {"changed": false, "failed": true, "invocation": {"module_name": "ec2"}, "module_stderr": "Traceb│~
ack (most recent call last):\n File \"/tmp/ryansb/ansible_FwE8VR/ansible_module_ec2.py\", line 1540, in <module>\n main()\n File \"/tmp/│~
ryansb/ansible_FwE8VR/ansible_module_ec2.py\", line 1514, in main\n (changed, instance_dict_array, new_instance_ids) = startstop_instances│~
(module, ec2, instance_ids, state, instance_tags)\n File \"/tmp/ryansb/ansible_FwE8VR/ansible_module_ec2.py\", line 1343, in startstop_insta│~
nces\n if len(matched_instances) < len(instance_ids):\nTypeError: object of type 'NoneType' has no len()\n", "module_stdout": "", "msg": "│~
MODULE FAILURE", "parsed": false}
```
That's because the `instance_ids` variable is None if not supplied
in the task. That means the instances that result from the instance_tags
query aren't going to be included in the wait loop. To fix this, a list
needs to be kept of instances with matching tags and that list needs to
be added to `instance_ids` before the wait loop.
Before this, all spot instance requests would fail because the code
_always_ called module.fail_json when the parameter was set (which it
always was, because the module parameter's default was set to 'stop').
As the comment said, this parameter doesn't make sense for spot
instances at all, so the error message was also misleading.
Due to a mixup of the group/role/user and policy names, policies with
the same name as the group/role/user they are attached to would never be
updated after creation. To fix that, we needed two changes to the logic
of policy comparison:
- Compare the new policy name to *all* matching policies, not just the
first in lexicographical order
- Compare the new policy name to the matching ones, not to the IAM
object the policy is attached to
* Check mode fixes for ec2_vpc_net module
Returns VPC object information
Detects state change for VPC, DHCP options, and tags in check mode
* Early exit on VPC creation in check mode
The default VPC egress rules was being left in the egress rules for
purging in check mode. This ensures that the module returns the correct
change state during check mode.
AWS security groups are unique by name only by VPC (Restated, the VPC
and group name form a unique key).
When attaching security groups to an ELB, the ec2_elb_lb module would
erroneously find security groups of the same name in other VPCs thus
causing an error stating as such.
To eliminate the error, we check that we are attaching subnets (implying
that we are in a VPC), grab the vpc_id of the 0th subnet, and filtering
the list of security groups on this VPC. In other cases, no such filter
is applied (filters=None).
EC2 Security Group names are unique given a VPC. When a group_name
value is specified in a rule, if the group_name does not exist in the
provided vpc_id it should create the group as per the documentation.
The groups dictionary uses group_names as keys, so it is possible to
find a group in another VPC with the name that is desired. This causes
an error as the security group being acted on, and the security group
referenced in the rule are in two different VPCs.
To prevent this issue, we check to see if vpc_id is defined and if so
check that VPCs match, else we treat the group as new.
While from the documentation[1] one would assume that replacing
CAPABILITY_IAM with CAPABILITY_NAMED_IAM; this as empirically been shown
to not be the case.
1: "If you have IAM resources, you can specify either capability. If you
have IAM resources with custom names, you must specify
CAPABILITY_NAMED_IAM."
http://docs.aws.amazon.com/AWSCloudFormation/latest/APIReference/API_CreateStack.html
The `source_dest_check` and `termination_protection` variables are being
assigned twice in ec2.py, likely due to an incorrect merge somewhere
along the line.
The IAM group modules were not receiving the `module` object, but they
use `module.fail_json()` in their exception handlers. This patch passes
through the module object so the real errors from boto are exposed,
rather than errors about "NoneType has no method `fail_json`".
Ceph Object Gateway (Ceph RGW) is an object storage interface built on top of
librados to provide applications with a RESTful gateway to Ceph Storage
Clusters:
http://docs.ceph.com/docs/master/radosgw/
This patch adds the required bits to use the RGW S3 RESTful API properly.
Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>
Ryan S. Brown
tedder
Carlos E. Garcia
Toshio Kuratomi
Andrea Tartaglia
Kash
Yury V. Zaytsev
José Sabater Montes
Adam Chainz
Maarten Bezemer
Tom Paine
Andy Freeland
Matt Clay
James Cammarata
Fabio Alessandro Locati
Fabio Alessandro Locati
Ted Timmons
Rob White
Sam Doran
Ben Tomasik
einarc
Zaius Dr
Andrew Gaffney
jctanner
Sean Summers
mickael-ange
Eduard Iskandarov
Jamie Dyer
Charles Paul
John R Barker
jjshoe
Elena Washington
Pradeep
Denis Tiago
René Moser
Shawn Siefkas
mzizzi
Christopher Kotfila
Abhijit Menon-Sen
Florian Dambrine
Matt Ferrante
Kenny Woodson
Rick Mendes
Ilja Bauer
Richard Adams
Shawn Siefkas
Rick Mendes
Lyle Mantooth
Jasmine Hegman
Shaun Brady
chrisweaver
Matthew Martin
Michael Baydoun
Kai Kousa
Adrian Moisey
Hrishikesh Barua
Javier M. Mellid
Adam Butler
Toshio Kuratomi
Bradley Phipps