Allow to update server attributes such as its description.

Changed default value of server attribute 'security_groups' from
['default'] to [] because the latter is the default in
python-openstackclient [1] and the former behavior causes issues
with existing servers [2]: Previously, when no 'security_groups'
parameter was given, the server module would change existing
servers to use the default security group, dropping all other
security groups assigned to the server.
Our (undocumented) guideline when writing modules is to only
add or change what has been requested by the user and to stick to
defaults from openstacksdk and python-openstackclient whenever
possible. Since we have to break backward compatibility with the
next release anyway, we take this opportunity to clean up this odd
behavior. Now, when no security groups are given, then security
groups of an existing server will not be touched.
Closes story #2007893 [2].
Note, Nova will create a server in the default security group,
if the security_groups parameter is omitted.

Dropped 'openstack' field from server module's results. This
variable expanded to additional server information which might
be useful for Ansible inventories and was filled from
openstacksdk's get_openstack_vars() function [3]. Variables in
this function can make additional cloud queries to retrieve
additional data, so calling this function can be expensive [4].
Users can use *_info modules to retrieve this data on-demand.

Dropped 'availabity_zone' attribute from generic OpenStackModule
arguments and inserted it into server and volume modules because
it is relevant to those two modules only. This is completes what
was started years ago [5] and is possible now since we have
breaking changes anyway.

Switched attribute name 'userdata' with its alias 'user_data' to
match openstacksdk's attribute names which are used e.g. in module
results. The previous attribute name 'userdata' is now used as an
alias and 'user_data' is used as the attribute name to keep backward
compatibility.

Wait for server to get into 'ACTIVE' state when creating a server
and attribute 'wait' has been set to true.

Sorted argument specs and documentation of the server module and
marked attributes which are not updatable. Changed unstable bash
script example in server module documentation.

Renamed server's module attribute 'delete_fip' to 'delete_ips' to
match openstacksdk and clarify that it includes all floating ip
addresses of the server.

Renamed server_info's module attribute 'server' to 'name' and added
the former as an alias to be consistent with other *_info modules.

Added RETURN fields documentation for the module results of both
server and server_info modules.

Added description and examples of how to use the 'filters' attribute
of the server_info module. Closes story #2007873 [6].

Removed 'openstack_' prefix from module results because the prefix is
not consistently used across modules, is more to type without any
benefit and removal of the prefix allows us to signal to users that
their code for handling module results has to be updated. Many modules
have different return values with openstacksdk >= 0.99.0 because it
consistently uses resource proxies now.

Added assertions for module results to catch future changes in the
openstacksdk and our Ansible modules.

Added integration tests to check the update mechanism of the server
module.

Fixed indentation in integration tests.

Ensure proper creation and deletion of resources such as networks,
subnets and servers in integration tests of server_action module.

Renamed ci/roles/server/defaults/main.yaml to main.yml, removing the
'a' in the file extension to be consistent with other filenames.

Dropped deprecated function openstack_find_nova_addresses() and
incorporated its code directly into the server module because it
is not used anywhere else.

[1] https://opendev.org/openstack/python-openstackclient/src/commit/e49ad1795b9dd57d5a82fb6f8f365fa20041cf29/openstackclient/compute/v2/server.py#L1070
[2] https://storyboard.openstack.org/#!/story/2007893
[3] https://opendev.org/openstack/openstacksdk/src/commit/9e9fc9879583943a08f854980cca5dfb3a5832f7/openstack/cloud/_compute.py#L1772
[4] https://opendev.org/openstack/openstacksdk/src/commit/9e9fc9879583943a08f854980cca5dfb3a5832f7/openstack/cloud/meta.py#L482
[5] https://github.com/ansible/ansible/commit/9bf33e56dd49e9478ba091a1ba12d17b7caeac24
[6] https://storyboard.openstack.org/#!/story/2007873



Signed-off-by: Jakob Meng <code@jakobmeng.de>
Change-Id: I2f955519a7e8c782b1dab8f94f7a019ed384b81d

- Change sdk calls to use proxies
- Convert return values to
- Update module docs
- Change argspec to more closely match the new sdk (and therefore the
  API) without breaking backward compatibility

Change-Id: I0f9bc573fd0c69cab65bd808145d628732bb0830

Be consistent with Ansible docs [1], [2], [3].

[1] https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_documenting.html
[2] https://docs.ansible.com/ansible-core/devel/dev_guide/testing/sanity/shebang.html
[3] https://github.com/ansible/ansible/blob/b86a18bd273499f5d10e581821a47571690660e1/test/lib/ansible_test/_util/controller/sanity/code-smell/shebang.py

Change-Id: Ia3976bb3fcca662cefa219edeef057bcb7143c01

distutils is deprecated in 3.10: https://peps.python.org/pep-0632/
Ansible requires it to be replaced[1]

[1] https://github.com/ansible-community/community-topics/issues/96
https://github.com/ansible-collections/news-for-maintainers/issues/18

Change-Id: I2bae37f206319e8f9ace468f5b94f6be643b6a3c

Removed default=None, type='str' and required=False from all module's
argument_specs with

sed -i \
    -e 's/default=None, //g' \
    -e 's/default=None,//g' \
    -e 's/default=None//g' \
    -e "s/type='str', //g" \
    -e "s/type='str',//g" \
    -e "s/type='str'//g" \
    -e 's/required=False, //g' \
    -e 's/required=False,//g' \
    -e 's/required=False//g' \
    plugins/modules/*.py plugins/module_utils/*.py

and manually cleaned the results because those options values are the
default in Ansible.

Ref.: https://docs.ansible.com/ansible/latest/dev_guide/developing_program_flow_modules.html#argument-spec

Change-Id: Icbc3bb84269d3b8205fac259300902ebdaf6a3ae

Bifrost downgrades openstacksdk to a release <0.99.0 [1], which is
incompatible to our master branch of the Ansible OpenStack
collection. By adding openstacksdk to Zuul variable job.required-\
projects [2], Bifrost will install openstacksdk from its master
branch, overwriting any existing (older) openstacksdk releases.

[1] https://opendev.org/openstack/bifrost/commit/03ddd026567f7b61b89d83b0e7e7cb35ed466ae6
[2] https://review.opendev.org/c/openstack/bifrost/+/849563/5/playbooks/test-bifrost.yaml

Change-Id: I46dadc70338bb45d459e8176cc26e67c20c1d4ae

Change-Id: Ia9fbc5b136347145d1a7ddc2d874f014c06558ae

With Ansible OpenStack collection 2.0.0 we break backward
compatibility to older releases, mainly due to breaking changes
coming with openstacksdk >=0.99.0. For example, results will change
for most Ansible modules in this collection.

We take this opportunity to drop the symbolic links with prefix
os_ in plugins/modules and the plugin routing in meta/runtime.yml.
This means users have to call modules of the Ansible OpenStack
collection using their FQCN (Fully Qualified Collection Name) such
as openstack.cloud.server. Short module names such as os_server
will now raise an Ansible error. This also decreases the likelihood
of incompatible Ansible code going undetected.

Symbolic links were introduced to keep our collection backward
compatible to user code which was written for old(er) Ansible releases
which did not have support for collections and where OpenStack modules
where named with a prefix os_ such as os_server which is nowadays
known and stored as openstack.cloud.server.

In Ansible aka ansible-base 2.10, a internal routing table
lib/ansible/config/ansible_builtin_runtime.yml [1] was introduced which
Ansible uses to resolve deprecated module names missing the FQCN (Fully
Qualified Collection Name). Additionally, collections can define their
own plugin routing table in meta/runtime.yml [2] which we did.

In ansible-base 2.10 and ansible-core 2.11 or later, if a user uses a
short module name and the collections keyword is not used, Ansible
will first look in the internal routing table, get an FQCN, and then
looks in the collection for that FQCN. If there is another routing
entry for that new name in that collection's meta/runtime.yml,
Ansible will continue with that redirect. If it does not find another
redirect, Ansible will look for the plugin itself, so it will not
find a redirect in the collection before looking at its internal
redirects. Except if the user uses a FQCN, then it looks directly in
that collection.

Ansible 2.9 and 2.8 do not have any notion of these redirects with a
plugin routing table, backward compatibility with deprecated os_*
module names is solely achieved with symbolic links. Ansible releases
older than 2.11 are EOL [3], so usage of os_* symlinks should reduce
soon.

[1] https://github.com/ansible/ansible/blob/devel/lib/ansible/config/ansible_builtin_runtime.yml
[2] https://github.com/openstack/ansible-collections-openstack/blob/master/meta/runtime.yml
[3] https://docs.ansible.com/ansible/devel/reference_appendices/release_and_maintenance.html

Change-Id: I28cc05c95419b72552899c926721eb87fb6f0868

openstacksdk will start using cleaning in 1.0.0 [1], so we need to be
able to wait for it to finish.

[1] https://review.opendev.org/c/openstack/openstacksdk/+/849402

Change-Id: I9695b1a5acfa007ad474c9b5b07a4fdb4c1b1198

Make it compatible with new openstack sdk 1.0.0

Change-Id: I911eecd31ee69dbde1da02a74e152746c1e3edfa

This reapplies commit 4bfa135b [1]
which got reverted in commit 1b59c19a
[2] due to issues in how TripleO Quickstart installed the Ansible
OpenStack collection. TripleO Quickstart has now been fixed and
will install code from our stable/1.0.0 branch instead of master for
all RDO branches which have openstacksdk <0.99.0 [3][4].

[1] https://opendev.org/openstack/ansible-collections-openstack/commit/4bfa135b20caeb6d6c2717049059ac52fd08e0e1
[2] https://opendev.org/openstack/ansible-collections-openstack/commit/1b59c19a24c55aa236d80552dcbf70c9c7b5088e
[3] https://review.opendev.org/c/openstack/tripleo-ci/+/849500
[4] https://review.opendev.org/c/openstack/tripleo-quickstart/+/849620/

Change-Id: I918d776c1560f03a4a84df371feb013d47043aa0

No RDO release for CentOS8 will get a OpenStackSDK 1.x.x release.

Change-Id: I728f0f282717c11a782be30859f999008f3fc8cb

Build and install latest RPM for openstacksdk from its master branch
instead of using the (pinned) RPM from RDO. This is necessary because
openstacksdk in RDO is currently pinned to <0.99.0 for all RDO
releases. Variable artg_change_list is used to define what code is
being build from source. The RPM of the Ansible OpenStack collection
does not have build from source because TripleO Quickstart installs
the collection from job.required-projects [1]. The latter shadows the
RPM release which is installed later by TripleO because it has a
higher precedence in ansible.cfg [2][3].

Changed the job hierarchy to other base jobs tripleo-ci-centos-8-\
standalone-build and tripleo-ci-centos-9-standalone-build. This reduces
the number of variables we have to define. It is also cleaner since our
CentOS9 job is no longer based on the CentOS8 job which prevents
issues with job variant collections due to our branched repository.

Added more Ansible modules to files which trigger TripleO jobs,
because Ansible role os_tempest [4] requires those modules and is
called in TripleO jobs. Modules which have been added include:
* openstack.cloud.compute_flavor
* openstack.cloud.image
* openstack.cloud.network
* openstack.cloud.router
* openstack.cloud.subnet

Added tripleo-ci-centos-8-standalone-osa to experimental jobs so that we
can actually run this job on demand easily.

[1] https://opendev.org/openstack/tripleo-quickstart/src/commit/cb1595223b09c991c3e8451c6d1b332a78f65a94/quickstart.sh#L123
[2] https://opendev.org/openstack/tripleo-quickstart/src/commit/cb1595223b09c991c3e8451c6d1b332a78f65a94/ansible.cfg#L19
[3] https://opendev.org/openstack/tripleo-quickstart/src/commit/cb1595223b09c991c3e8451c6d1b332a78f65a94/quickstart.sh#L595
[4] https://opendev.org/openstack/openstack-ansible-os_tempest.git

Change-Id: Ibde318678a3e44fdc297a6f29761eb0c7d77cbc9

Replaced custom code for interface listing with call to openstacksdk.
The original idea was to reduce the number of calls to the OpenStack
API but this kind of optimization is better to be implemented in the
SDK itself [1]. Reimplementing code like this increases our
maintenance burden, does not help other SDK users and increases the
likeliness of bugs. For example, variable allowed_device_owners
introduced a bug, it is not 'network_router_interface_distributed'
but 'network:router_interface_distributed'.

[1] https://review.opendev.org/c/openstack/openstacksdk/+/849967

Change-Id: I9c52de03c53ef29d7cecdf26253c0c00a7cf3689

- Change sdk calls to use proxy objects
- Convert return values to dict before updating
- Adds additional test values

Change-Id: I187a27af4a5b8aa7cd4b60a1a876b5e5e6975144

Signed-off-by: Jakob Meng <code@jakobmeng.de>
Change-Id: If8befd62ac59ef19debf0d63abe76f11b47c2da3

Make routers_info module compatible with the new sdk 1.0.0

Change-Id: I43ea8ccc0d882ca63bc294dd99fb4f010f3806a5

Updated documentation of return values and added test to verify
return values

Function self.conn.search_security_groups() cannot be used here.
Arguments for filtering such as 'description' would have to be passed
to self.conn.search_security_groups() in its 'filters' argument [1].
The latter is passed to both as query arguments to OpenStack API
and later to _utils._filter_list() [2] for filtering the results.
Some arguments such as 'any_tags' are only used as query arguments
by openstacksdk [3] when querying OpenStack API. They are no valid
attributes in security_group.py [4]. Whenever those non-attribute
arguments are passed to self.conn.search_security_groups(),
_utils._filter_list() [2] would drop all results because no result
would have a matching attributes.

[1] https://opendev.org/openstack/openstacksdk/src/commit/0898398415ae7b0e2447d61226acf50f01567cdd/openstack/cloud/_security_group.py#L31
[2] https://opendev.org/openstack/openstacksdk/src/commit/0898398415ae7b0e2447d61226acf50f01567cdd/openstack/cloud/_utils.py#L63
[3] https://opendev.org/openstack/openstacksdk/src/commit/0898398415ae7b0e2447d61226acf50f01567cdd/openstack/common/tag.py#L19
[4] https://opendev.org/openstack/openstacksdk/src/commit/0898398415ae7b0e2447d61226acf50f01567cdd/openstack/network/v2/security_group.py

Change-Id: Ie7fe9d2e973d38751c48e71e6bd55e56a591ac1f

Previously, all security_group{,_info} and security_group_rule{,_info}
modules were tested in the same Ansible role. This patch splits
tests into two separate Ansible roles to increase readibility and
prevent variable name conflicts, e.g. for expected_fields.

Change-Id: Ifc28435147b3bfe88d4ee5e176469a53b7395dc0

The solution is based on implementation of logging option
in Open Telekom Cloud collections.

Change-Id: Ie8b309d2aaa8da57794888848fc5414de207e54f

- Update docs
- Change calls from cloud to proxy layer
- Make sure return value is a dict
- Improve test coverage

Change-Id: I857d7ba7b7ca1b23100ee7e85e90e98430d68462

Change-Id: Iba1604ee9c0b922b8fb7c6a278acf90d080a63e7

- Use proxy layer where possible
  - Image upload has some tricky logic so that stays in the cloud layer
- Convert return value to dict
- Document return values
- Update visibility logic for glance v2 api
- Increase test coverage
- General refactoring to bring more in line with rest of collection
- Deprecate is_public attribute which has been replaced with
  visibility.
- Deprecate volume attribute which has been made obsolete with
  openstack.cloud.volume module. Removed examples showing the volume
  attribute since users are encouraged to use openstack.cloud.volume
  module.

Change-Id: I1d8034a3b9a391444ea275b68b06ee3a291c73c3

Regions have IDs, but do not have names.
Ref.: https://docs.openstack.org/api-ref/identity/v3/#regions

Change-Id: I2512bbde6e96e2ab0f1fef0230295223f46105dd

Module ansible.builtin.user in ansible-core 2.13.0 and 2.13.1 is
affected by #78017 [1] which results in an exception being raised
in ci/roles/keypair/tasks/main.yml [2]. Until this issue is fixed,
we will exclude the broken versions 2.13.0 and 2.13.1 in
requirements.txt [3].

[1] https://github.com/ansible/ansible/issues/78017
[2] https://opendev.org/openstack/ansible-collections-openstack/src/commit/802e46d554a1f899ad56be5574d2f7e2323ee6d6/ci/roles/keypair/tasks/main.yml#L72
[3] https://opendev.org/openstack/ansible-collections-openstack/src/branch/master/tests/requirements.txt



Change-Id: I61bec4e62ecbcf357f3c1279a7373049077cb8d4
Signed-off-by: Jakob Meng <code@jakobmeng.de>

Make project_info module compatible with the new sdk 1.0.0 and also add
ansible tests for project_info module

Change-Id: I413200cf6a9b8bada7e5d78087246b888d53fac2

With this change we replace zuul.projects with zuul.project that will
imply any project which will run the job. Also we read galaxy.yml as
vars file to predict packed collection naming for futher upload.

Change-Id: I66e27f3026689ad719384203fe66d65f5bca46ce
Needed-By: https://review.opendev.org/c/openstack/ansible-config_template/+/846391

Without any parameters supplied openstack.cloud.baremetal_node_info is
supposed to gather and return information about all nodes.
This is done with a call to cloud.list_machines(), which itself calls
self.baremetal.nodes().
Unfortunately this will not return detailed information about each
machine as the details parameter of self.baremetal.nodes() defaults to
false.
This commit rewrites the module to use the baremetal service proxy of
openstacksdk to get machines with details and converts them using the
`to_dict()` method.

Story: 2010017
Task: 45207
Change-Id: Ib06aea5f59f799d6ed81b30264c8a168301c1a9b

This makes keypair_info compatible with new sdk version

Change-Id: I09c75717a620272904b023179c726a19c4bca000

Update endpoint to new openstack sdk

Change-Id: Ibd647d0c2cd2f90310f381e56088e7e8e93f76fc

Updating role_assignment module to use the new openstacksdk

Change-Id: I09258e18d50acb57501ea1b47d9422dad857607e

This patch do the following:

* Update catalog_service to use new openstacksdk
* Add catalog_service role to test catalog_service module

Change-Id: I6778f5e91cb0ead63cede28af0111d7ffbbf3ab1