On host reboot the VMs on VMware Integrated Opentack are in "Error" state

Hi All,

One of the three hosts managing the VIO resource cluster, was rebooted.The VMs that were present in that host are currently in "Error" state and I am unable to start or stop the VMs.

This issue occurs whenever, the VMs are in "Powered ON" state and the host is rebooted.

I can just see the following options,

Nova logs,

2016-11-27 11:53:27.126 2040 INFO nova.api.ec2 [-] 0.99s 10.110.50.84 GET / None:None 200 [None] text/plain text/plain

2016-11-27 11:53:27.127 2040 INFO nova.metadata.wsgi.server [-] 10.110.50.73,10.110.50.84 "GET / HTTP/1.0" status: 200 len: 234 time: 0.0006559

2016-11-27 11:53:37.130 2037 INFO nova.api.ec2 [-] 0.103s 10.110.50.84 GET / None:None 200 [None] text/plain text/plain

2016-11-27 11:53:37.131 2037 INFO nova.metadata.wsgi.server [-] 10.110.50.73,10.110.50.84 "GET / HTTP/1.0" status: 200 len: 234 time: 0.0006850

2016-11-27 11:53:40.385 2017 DEBUG nova.api.openstack.wsgi [req-06afc57d-69b5-4ceb-82da-6f219dd5ff2f 8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -] Calling method '<bound method Controller.detail of <nova.api.openstack.compute.servers.Controller object at 0x7f6352864310>>' _process_stack /usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py:783

2016-11-27 11:53:40.386 2017 DEBUG nova.compute.api [req-06afc57d-69b5-4ceb-82da-6f219dd5ff2f 8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -] Searching by: {'deleted': False, u'project_id': u'aeca5808a8da42bf97be22cb3571e172', u'limit': u'21'} get_all /usr/lib/python2.7/dist-packages/nova/compute/api.py:1962

2016-11-27 11:53:40.449 2017 DEBUG keystoneclient.session [req-06afc57d-69b5-4ceb-82da-6f219dd5ff2f 8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -] REQ: curl -g -i -X GET http://10.110.50.72:9696/v2.0/ports.json?device_id=25954535-35f9-45ed-9d3e-acea39e71b5d&device_id=d3c94db6-f54f-4ab4-a7ad-d25c25b35e83&device_id=1bc45884-782f-4e4d-a8c6-719aff0d8345&device_id=e0367e0f-11e0-430f-a179-72d99d4c6252&device_id=4ca20b85-81fd-4d6a-974e-f2bb6f6b1671&device_id=6ab3368f-7adb-451e-a6f8-8f787438390c&device_id=a2cb1373-d7a3-4fc2-a6ca-b688e1409902&device_id=6b6ee57d-8841-4681-b63e-a0eaaf4ffc22 -H "User-Agent: python-neutronclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}bf91f44b193e72921358b1872f3e584c42e1bd5b" _http_log_request /usr/lib/python2.7/dist-packages/keystoneclient/session.py:195

2016-11-27 11:53:40.471 2017 DEBUG keystoneclient.session [req-06afc57d-69b5-4ceb-82da-6f219dd5ff2f 8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -] RESP: [200] date: Sun, 27 Nov 2016 11:53:40 GMT content-length: 8454 content-type: application/json; charset=UTF-8 x-openstack-request-id: req-3466cffc-ddf4-4908-9e10-0b39fb3a3093

RESP BODY: {"ports": [{"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": 0, "fixed_ips": [{"subnet_id": "d1760e9a-0931-480a-a59a-eadfcf293bd5", "ip_address": "192.169.30.34"}], "id": "1cadf9bb-e6db-4f0d-907e-eb33336f5d4a", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "d3c94db6-f54f-4ab4-a7ad-d25c25b35e83", "name": "", "admin_state_up": true, "network_id": "8d47954d-3cf9-4270-8064-9cd36ee8f997", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:5f:2f:65"}, {"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": 0, "fixed_ips": [{"subnet_id": "be1ce0be-21a7-419b-9d9a-e78f244e7175", "ip_address": "192.169.10.26"}], "id": "273d98d0-2c57-42e5-9284-8bb0c83b6927", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "1bc45884-782f-4e4d-a8c6-719aff0d8345", "name": "", "admin_state_up": true, "network_id": "d2b7f23c-aaaf-4f37-aafd-ed2d260365fa", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:d4:25:2a"}, {"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": 0, "fixed_ips": [{"subnet_id": "be1ce0be-21a7-419b-9d9a-e78f244e7175", "ip_address": "192.169.10.20"}], "id": "56af5ce1-75ff-4d19-b7ac-0b5d4c93dfe1", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "6ab3368f-7adb-451e-a6f8-8f787438390c", "name": "", "admin_state_up": true, "network_id": "d2b7f23c-aaaf-4f37-aafd-ed2d260365fa", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:51:97:9b"}, {"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": 3, "fixed_ips": [{"subnet_id": "c47d8fd7-4dea-479f-8b1f-64f1da586fb5", "ip_address": "192.169.18.30"}], "id": "687519f6-d021-495c-84ce-c96c71a46f62", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "d3c94db6-f54f-4ab4-a7ad-d25c25b35e83", "name": "", "admin_state_up": true, "network_id": "5d2fa3f6-e50c-4e76-a7a9-ff5527891fe3", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:ef:1e:d2"}, {"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": 1, "fixed_ips": [{"subnet_id": "87f8e064-fdc2-47b1-9d58-344d52346e3b", "ip_address": "192.169.31.36"}], "id": "6d63ac9b-6050-4826-a9e4-f95d00141b38", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "d3c94db6-f54f-4ab4-a7ad-d25c25b35e83", "name": "", "admin_state_up": true, "network_id": "d1cd0cad-ecea-434d-b36d-61a62bf359c2", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:a0:45:e3"}, {"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": null, "fixed_ips": [{"subnet_id": "e45e70fc-6f7a-476a-bf64-eaa1e895c59a", "ip_address": "192.169.14.106"}], "id": "72d6533d-42b5-43e3-bdc2-a67772c0c309", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "25954535-35f9-45ed-9d3e-acea39e71b5d", "name": "", "admin_state_up": true, "network_id": "dd699c81-7ae2-4340-9c4b-8596f9b3efba", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:08:71:47"}, {"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": 0, "fixed_ips": [{"subnet_id": "be1ce0be-21a7-419b-9d9a-e78f244e7175", "ip_address": "192.169.10.22"}], "id": "7bee6785-0e98-45b7-8ab2-d60841f1d71e", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "e0367e0f-11e0-430f-a179-72d99d4c6252", "name": "", "admin_state_up": true, "network_id": "d2b7f23c-aaaf-4f37-aafd-ed2d260365fa", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:af:bc:94"}, {"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": 0, "fixed_ips": [{"subnet_id": "be1ce0be-21a7-419b-9d9a-e78f244e7175", "ip_address": "192.169.10.92"}], "id": "7dcabfa9-0575-4c2f-8a51-8ab338b546d4", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "6b6ee57d-8841-4681-b63e-a0eaaf4ffc22", "name": "", "admin_state_up": true, "network_id": "d2b7f23c-aaaf-4f37-aafd-ed2d260365fa", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:2c:12:7e"}, {"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": 2, "fixed_ips": [{"subnet_id": "09743c9b-1e02-4a87-8a2b-cdc1ed51ff20", "ip_address": "192.169.32.34"}], "id": "cf55ef26-698b-4ba6-997d-405127116654", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "d3c94db6-f54f-4ab4-a7ad-d25c25b35e83", "name": "", "admin_state_up": true, "network_id": "18f65fc9-7691-46e7-90cc-8ce10474d500", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:17:60:e9"}, {"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": 0, "fixed_ips": [{"subnet_id": "be1ce0be-21a7-419b-9d9a-e78f244e7175", "ip_address": "192.169.10.21"}], "id": "e05f92eb-a1bd-4499-b6a8-e40469e8edc0", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "4ca20b85-81fd-4d6a-974e-f2bb6f6b1671", "name": "", "admin_state_up": true, "network_id": "d2b7f23c-aaaf-4f37-aafd-ed2d260365fa", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:4f:ec:e6"}, {"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": 0, "fixed_ips": [{"subnet_id": "be1ce0be-21a7-419b-9d9a-e78f244e7175", "ip_address": "192.169.10.19"}], "id": "f3de5052-4f8a-459a-8f33-018fb6de5d2d", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "a2cb1373-d7a3-4fc2-a6ca-b688e1409902", "name": "", "admin_state_up": true, "network_id": "d2b7f23c-aaaf-4f37-aafd-ed2d260365fa", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:1f:94:a5"}, {"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": null, "fixed_ips": [{"subnet_id": "be1ce0be-21a7-419b-9d9a-e78f244e7175", "ip_address": "192.169.10.45"}], "id": "ffaa5b99-1deb-4420-98dc-41bd781bd11b", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "25954535-35f9-45ed-9d3e-acea39e71b5d", "name": "", "admin_state_up": true, "network_id": "d2b7f23c-aaaf-4f37-aafd-ed2d260365fa", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:e3:6f:78"}]}

_http_log_response /usr/lib/python2.7/dist-packages/keystoneclient/session.py:223

2016-11-27 11:53:40.472 2017 DEBUG keystoneclient.session [req-06afc57d-69b5-4ceb-82da-6f219dd5ff2f 8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -] REQ: curl -g -i -X GET http://10.110.50.72:9696/v2.0/security-groups.json?id=c576f2be-bf17-40d8-9d65-7ae2ee136c61 -H "User-Agent: python-neutronclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}bf91f44b193e72921358b1872f3e584c42e1bd5b" _http_log_request /usr/lib/python2.7/dist-packages/keystoneclient/session.py:195

2016-11-27 11:53:40.486 2017 DEBUG keystoneclient.session [req-06afc57d-69b5-4ceb-82da-6f219dd5ff2f 8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -] RESP: [200] date: Sun, 27 Nov 2016 11:53:40 GMT content-length: 4109 content-type: application/json; charset=UTF-8 x-openstack-request-id: req-4bbf1430-7ee7-4d22-8272-75c77899d97a

RESP BODY: {"security_groups": [{"tenant_id": "aeca5808a8da42bf97be22cb3571e172", "description": "Default security group", "id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "security_group_rules": [{"local_ip_prefix": null, "direction": "egress", "protocol": "tcp", "port_range_max": 443, "id": "0325d054-7209-4f03-a367-cb439ce52f97", "remote_group_id": null, "remote_ip_prefix": "0.0.0.0/0", "security_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "port_range_min": 443, "ethertype": "IPv4"}, {"local_ip_prefix": null, "direction": "egress", "protocol": null, "port_range_max": null, "id": "0fbe0ce2-92da-462f-a66a-2df51ac325bc", "remote_group_id": null, "remote_ip_prefix": null, "security_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "port_range_min": null, "ethertype": "IPv4"}, {"local_ip_prefix": null, "direction": "ingress", "protocol": "tcp", "port_range_max": 80, "id": "22574145-399e-4cb9-a422-1cafffe1069a", "remote_group_id": null, "remote_ip_prefix": "0.0.0.0/0", "security_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "port_range_min": 80, "ethertype": "IPv4"}, {"local_ip_prefix": null, "direction": "ingress", "protocol": null, "port_range_max": null, "id": "2a67c97c-cdee-4c3a-9ff0-328b540d3f21", "remote_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "remote_ip_prefix": null, "security_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "port_range_min": null, "ethertype": "IPv6"}, {"local_ip_prefix": null, "direction": "egress", "protocol": "icmp", "port_range_max": null, "id": "364fb2bc-c171-4b38-8971-d6ebdbbaee0b", "remote_group_id": null, "remote_ip_prefix": "0.0.0.0/0", "security_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "port_range_min": null, "ethertype": "IPv4"}, {"local_ip_prefix": null, "direction": "ingress", "protocol": "tcp", "port_range_max": 3389, "id": "3cc60008-00c8-4c1e-93f4-accc86e88ff0", "remote_group_id": null, "remote_ip_prefix": "0.0.0.0/0", "security_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "port_range_min": 3389, "ethertype": "IPv4"}, {"local_ip_prefix": null, "direction": "ingress", "protocol": "tcp", "port_range_max": 22, "id": "448f06e5-a374-487b-9956-3800c4c0bed4", "remote_group_id": null, "remote_ip_prefix": "0.0.0.0/0", "security_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "port_range_min": 22, "ethertype": "IPv4"}, {"local_ip_prefix": null, "direction": "ingress", "protocol": "tcp", "port_range_max": 443, "id": "5786f368-ee20-4303-af20-12228043ca19", "remote_group_id": null, "remote_ip_prefix": "0.0.0.0/0", "security_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "port_range_min": 443, "ethertype": "IPv4"}, {"local_ip_prefix": null, "direction": "egress", "protocol": null, "port_range_max": null, "id": "83e9d70b-5c54-44b5-8944-4847cf361de8", "remote_group_id": null, "remote_ip_prefix": null, "security_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "port_range_min": null, "ethertype": "IPv6"}, {"local_ip_prefix": null, "direction": "ingress", "protocol": "icmp", "port_range_max": null, "id": "9d32ca7a-279c-4bfb-a787-2def28131f43", "remote_group_id": null, "remote_ip_prefix": "0.0.0.0/0", "security_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "port_range_min": null, "ethertype": "IPv4"}, {"local_ip_prefix": null, "direction": "ingress", "protocol": null, "port_range_max": null, "id": "ce032e23-7b82-419b-acf1-2e5ce9eaabb8", "remote_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "remote_ip_prefix": null, "security_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "port_range_min": null, "ethertype": "IPv4"}], "name": "default"}]}

_http_log_response /usr/lib/python2.7/dist-packages/keystoneclient/session.py:223

2016-11-27 11:53:40.487 2017 DEBUG nova.policy [req-06afc57d-69b5-4ceb-82da-6f219dd5ff2f 8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -] Policy check for compute_extension:hide_server_addresses failed with credentials {'domain': None, 'project_name': u'admin', 'project_domain': None, 'timestamp': '2016-11-27T11:53:40.384783', 'remote_address': '10.110.50.78', 'quota_class': None, 'resource_uuid': None, 'is_admin': True, 'user': u'8754985c51914ddd92cfb0132925474d', 'service_catalog': [{u'endpoints': [{u'adminURL': u'http://10.110.50.72:8776/v1/aeca5808a8da42bf97be22cb3571e172', u'region': u'nova', u'internalURL': u'http://10.110.50.72:8776/v1/aeca5808a8da42bf97be22cb3571e172', u'publicURL': u'https://10.110.81.6:8776/v1/aeca5808a8da42bf97be22cb3571e172'}], u'type': u'volume', u'name': u'cinder'}, {u'endpoints': [{u'adminURL': u'http://10.110.50.72:8776/v2/aeca5808a8da42bf97be22cb3571e172', u'region': u'nova', u'internalURL': u'http://10.110.50.72:8776/v2/aeca5808a8da42bf97be22cb3571e172', u'publicURL': u'https://10.110.81.6:8776/v2/aeca5808a8da42bf97be22cb3571e172'}], u'type': u'volumev2', u'name': u'cinderv2'}], 'tenant': u'aeca5808a8da42bf97be22cb3571e172', 'read_only': False, 'project_id': u'aeca5808a8da42bf97be22cb3571e172', 'user_id': u'8754985c51914ddd92cfb0132925474d', 'show_deleted': False, 'roles': [u'admin', u'_member_'], 'user_identity': '8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -', 'read_deleted': 'no', 'request_id': 'req-06afc57d-69b5-4ceb-82da-6f219dd5ff2f', 'instance_lock_checked': False, 'user_domain': None, 'user_name': u'admin'} enforce /usr/lib/python2.7/dist-packages/nova/policy.py:104

2016-11-27 11:53:40.510 2017 INFO nova.osapi_compute.wsgi.server [req-06afc57d-69b5-4ceb-82da-6f219dd5ff2f 8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -] 10.110.50.78,10.110.50.84 "GET /v2/aeca5808a8da42bf97be22cb3571e172/servers/detail?limit=21&project_id=aeca5808a8da42bf97be22cb3571e172 HTTP/1.1" status: 200 len: 16474 time: 0.1276321

2016-11-27 11:53:40.916 2015 DEBUG nova.api.openstack.wsgi [req-2984e8a3-daeb-41a4-9bea-f7d4a66bbecf 8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -] Calling method '<bound method Controller.detail of <nova.api.openstack.compute.servers.Controller object at 0x7f6352864310>>' _process_stack /usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py:783

2016-11-27 11:53:40.916 2015 DEBUG nova.compute.api [req-2984e8a3-daeb-41a4-9bea-f7d4a66bbecf 8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -] Searching by: {'deleted': False, u'project_id': u'aeca5808a8da42bf97be22cb3571e172', u'limit': u'21'} get_all /usr/lib/python2.7/dist-packages/nova/compute/api.py:1962

2016-11-27 11:53:40.955 2015 DEBUG keystoneclient.session [req-2984e8a3-daeb-41a4-9bea-f7d4a66bbecf 8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -] REQ: curl -g -i -X GET http://10.110.50.72:9696/v2.0/ports.json?device_id=25954535-35f9-45ed-9d3e-acea39e71b5d&device_id=d3c94db6-f54f-4ab4-a7ad-d25c25b35e83&device_id=1bc45884-782f-4e4d-a8c6-719aff0d8345&device_id=e0367e0f-11e0-430f-a179-72d99d4c6252&device_id=4ca20b85-81fd-4d6a-974e-f2bb6f6b1671&device_id=6ab3368f-7adb-451e-a6f8-8f787438390c&device_id=a2cb1373-d7a3-4fc2-a6ca-b688e1409902&device_id=6b6ee57d-8841-4681-b63e-a0eaaf4ffc22 -H "User-Agent: python-neutronclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}bf91f44b193e72921358b1872f3e584c42e1bd5b" _http_log_request /usr/lib/python2.7/dist-packages/keystoneclient/session.py:195

2016-11-27 11:53:40.977 2015 DEBUG keystoneclient.session [req-2984e8a3-daeb-41a4-9bea-f7d4a66bbecf 8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -] RESP: [200] date: Sun, 27 Nov 2016 11:53:40 GMT content-length: 8454 content-type: application/json; charset=UTF-8 x-openstack-request-id: req-9220cfc5-02f7-4205-850c-3c1d2ff0560a

RESP BODY: {"ports": [{"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": 0, "fixed_ips": [{"subnet_id": "d1760e9a-0931-480a-a59a-eadfcf293bd5", "ip_address": "192.169.30.34"}], "id": "1cadf9bb-e6db-4f0d-907e-eb33336f5d4a", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "d3c94db6-f54f-4ab4-a7ad-d25c25b35e83", "name": "", "admin_state_up": true, "network_id": "8d47954d-3cf9-4270-8064-9cd36ee8f997", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:5f:2f:65"}, {"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": 0, "fixed_ips": [{"subnet_id": "be1ce0be-21a7-419b-9d9a-e78f244e7175", "ip_address": "192.169.10.26"}], "id": "273d98d0-2c57-42e5-9284-8bb0c83b6927", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "1bc45884-782f-4e4d-a8c6-719aff0d8345", "name": "", "admin_state_up": true, "network_id": "d2b7f23c-aaaf-4f37-aafd-ed2d260365fa", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:d4:25:2a"}, {"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": 0, "fixed_ips": [{"subnet_id": "be1ce0be-21a7-419b-9d9a-e78f244e7175", "ip_address": "192.169.10.20"}], "id": "56af5ce1-75ff-4d19-b7ac-0b5d4c93dfe1", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "6ab3368f-7adb-451e-a6f8-8f787438390c", "name": "", "admin_state_up": true, "network_id": "d2b7f23c-aaaf-4f37-aafd-ed2d260365fa", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:51:97:9b"}, {"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": 3, "fixed_ips": [{"subnet_id": "c47d8fd7-4dea-479f-8b1f-64f1da586fb5", "ip_address": "192.169.18.30"}], "id": "687519f6-d021-495c-84ce-c96c71a46f62", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "d3c94db6-f54f-4ab4-a7ad-d25c25b35e83", "name": "", "admin_state_up": true, "network_id": "5d2fa3f6-e50c-4e76-a7a9-ff5527891fe3", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:ef:1e:d2"}, {"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": 1, "fixed_ips": [{"subnet_id": "87f8e064-fdc2-47b1-9d58-344d52346e3b", "ip_address": "192.169.31.36"}], "id": "6d63ac9b-6050-4826-a9e4-f95d00141b38", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "d3c94db6-f54f-4ab4-a7ad-d25c25b35e83", "name": "", "admin_state_up": true, "network_id": "d1cd0cad-ecea-434d-b36d-61a62bf359c2", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:a0:45:e3"}, {"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": null, "fixed_ips": [{"subnet_id": "e45e70fc-6f7a-476a-bf64-eaa1e895c59a", "ip_address": "192.169.14.106"}], "id": "72d6533d-42b5-43e3-bdc2-a67772c0c309", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "25954535-35f9-45ed-9d3e-acea39e71b5d", "name": "", "admin_state_up": true, "network_id": "dd699c81-7ae2-4340-9c4b-8596f9b3efba", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:08:71:47"}, {"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": 0, "fixed_ips": [{"subnet_id": "be1ce0be-21a7-419b-9d9a-e78f244e7175", "ip_address": "192.169.10.22"}], "id": "7bee6785-0e98-45b7-8ab2-d60841f1d71e", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "e0367e0f-11e0-430f-a179-72d99d4c6252", "name": "", "admin_state_up": true, "network_id": "d2b7f23c-aaaf-4f37-aafd-ed2d260365fa", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:af:bc:94"}, {"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": 0, "fixed_ips": [{"subnet_id": "be1ce0be-21a7-419b-9d9a-e78f244e7175", "ip_address": "192.169.10.92"}], "id": "7dcabfa9-0575-4c2f-8a51-8ab338b546d4", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "6b6ee57d-8841-4681-b63e-a0eaaf4ffc22", "name": "", "admin_state_up": true, "network_id": "d2b7f23c-aaaf-4f37-aafd-ed2d260365fa", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:2c:12:7e"}, {"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": 2, "fixed_ips": [{"subnet_id": "09743c9b-1e02-4a87-8a2b-cdc1ed51ff20", "ip_address": "192.169.32.34"}], "id": "cf55ef26-698b-4ba6-997d-405127116654", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "d3c94db6-f54f-4ab4-a7ad-d25c25b35e83", "name": "", "admin_state_up": true, "network_id": "18f65fc9-7691-46e7-90cc-8ce10474d500", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:17:60:e9"}, {"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": 0, "fixed_ips": [{"subnet_id": "be1ce0be-21a7-419b-9d9a-e78f244e7175", "ip_address": "192.169.10.21"}], "id": "e05f92eb-a1bd-4499-b6a8-e40469e8edc0", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "4ca20b85-81fd-4d6a-974e-f2bb6f6b1671", "name": "", "admin_state_up": true, "network_id": "d2b7f23c-aaaf-4f37-aafd-ed2d260365fa", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:4f:ec:e6"}, {"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": 0, "fixed_ips": [{"subnet_id": "be1ce0be-21a7-419b-9d9a-e78f244e7175", "ip_address": "192.169.10.19"}], "id": "f3de5052-4f8a-459a-8f33-018fb6de5d2d", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "a2cb1373-d7a3-4fc2-a6ca-b688e1409902", "name": "", "admin_state_up": true, "network_id": "d2b7f23c-aaaf-4f37-aafd-ed2d260365fa", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:1f:94:a5"}, {"status": "ACTIVE", "binding:host_id": "compute01", "allowed_address_pairs": [], "device_owner": "compute:nova", "port_security_enabled": true, "vnic_index": null, "fixed_ips": [{"subnet_id": "be1ce0be-21a7-419b-9d9a-e78f244e7175", "ip_address": "192.169.10.45"}], "id": "ffaa5b99-1deb-4420-98dc-41bd781bd11b", "security_groups": ["c576f2be-bf17-40d8-9d65-7ae2ee136c61"], "device_id": "25954535-35f9-45ed-9d3e-acea39e71b5d", "name": "", "admin_state_up": true, "network_id": "d2b7f23c-aaaf-4f37-aafd-ed2d260365fa", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "binding:vif_details": {"port_filter": true}, "binding:vnic_type": "normal", "binding:vif_type": "dvs", "mac_address": "fa:16:3e:e3:6f:78"}]}

_http_log_response /usr/lib/python2.7/dist-packages/keystoneclient/session.py:223

2016-11-27 11:53:40.978 2015 DEBUG keystoneclient.session [req-2984e8a3-daeb-41a4-9bea-f7d4a66bbecf 8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -] REQ: curl -g -i -X GET http://10.110.50.72:9696/v2.0/security-groups.json?id=c576f2be-bf17-40d8-9d65-7ae2ee136c61 -H "User-Agent: python-neutronclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}bf91f44b193e72921358b1872f3e584c42e1bd5b" _http_log_request /usr/lib/python2.7/dist-packages/keystoneclient/session.py:195

2016-11-27 11:53:40.991 2015 DEBUG keystoneclient.session [req-2984e8a3-daeb-41a4-9bea-f7d4a66bbecf 8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -] RESP: [200] date: Sun, 27 Nov 2016 11:53:40 GMT content-length: 4109 content-type: application/json; charset=UTF-8 x-openstack-request-id: req-7e22a705-205f-406b-b569-2867bb2d6c65

RESP BODY: {"security_groups": [{"tenant_id": "aeca5808a8da42bf97be22cb3571e172", "description": "Default security group", "id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "security_group_rules": [{"local_ip_prefix": null, "direction": "egress", "protocol": "tcp", "port_range_max": 443, "id": "0325d054-7209-4f03-a367-cb439ce52f97", "remote_group_id": null, "remote_ip_prefix": "0.0.0.0/0", "security_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "port_range_min": 443, "ethertype": "IPv4"}, {"local_ip_prefix": null, "direction": "egress", "protocol": null, "port_range_max": null, "id": "0fbe0ce2-92da-462f-a66a-2df51ac325bc", "remote_group_id": null, "remote_ip_prefix": null, "security_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "port_range_min": null, "ethertype": "IPv4"}, {"local_ip_prefix": null, "direction": "ingress", "protocol": "tcp", "port_range_max": 80, "id": "22574145-399e-4cb9-a422-1cafffe1069a", "remote_group_id": null, "remote_ip_prefix": "0.0.0.0/0", "security_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "port_range_min": 80, "ethertype": "IPv4"}, {"local_ip_prefix": null, "direction": "ingress", "protocol": null, "port_range_max": null, "id": "2a67c97c-cdee-4c3a-9ff0-328b540d3f21", "remote_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "remote_ip_prefix": null, "security_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "port_range_min": null, "ethertype": "IPv6"}, {"local_ip_prefix": null, "direction": "egress", "protocol": "icmp", "port_range_max": null, "id": "364fb2bc-c171-4b38-8971-d6ebdbbaee0b", "remote_group_id": null, "remote_ip_prefix": "0.0.0.0/0", "security_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "port_range_min": null, "ethertype": "IPv4"}, {"local_ip_prefix": null, "direction": "ingress", "protocol": "tcp", "port_range_max": 3389, "id": "3cc60008-00c8-4c1e-93f4-accc86e88ff0", "remote_group_id": null, "remote_ip_prefix": "0.0.0.0/0", "security_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "port_range_min": 3389, "ethertype": "IPv4"}, {"local_ip_prefix": null, "direction": "ingress", "protocol": "tcp", "port_range_max": 22, "id": "448f06e5-a374-487b-9956-3800c4c0bed4", "remote_group_id": null, "remote_ip_prefix": "0.0.0.0/0", "security_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "port_range_min": 22, "ethertype": "IPv4"}, {"local_ip_prefix": null, "direction": "ingress", "protocol": "tcp", "port_range_max": 443, "id": "5786f368-ee20-4303-af20-12228043ca19", "remote_group_id": null, "remote_ip_prefix": "0.0.0.0/0", "security_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "port_range_min": 443, "ethertype": "IPv4"}, {"local_ip_prefix": null, "direction": "egress", "protocol": null, "port_range_max": null, "id": "83e9d70b-5c54-44b5-8944-4847cf361de8", "remote_group_id": null, "remote_ip_prefix": null, "security_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "port_range_min": null, "ethertype": "IPv6"}, {"local_ip_prefix": null, "direction": "ingress", "protocol": "icmp", "port_range_max": null, "id": "9d32ca7a-279c-4bfb-a787-2def28131f43", "remote_group_id": null, "remote_ip_prefix": "0.0.0.0/0", "security_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "port_range_min": null, "ethertype": "IPv4"}, {"local_ip_prefix": null, "direction": "ingress", "protocol": null, "port_range_max": null, "id": "ce032e23-7b82-419b-acf1-2e5ce9eaabb8", "remote_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "remote_ip_prefix": null, "security_group_id": "c576f2be-bf17-40d8-9d65-7ae2ee136c61", "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "port_range_min": null, "ethertype": "IPv4"}], "name": "default"}]}

_http_log_response /usr/lib/python2.7/dist-packages/keystoneclient/session.py:223

2016-11-27 11:53:40.992 2015 DEBUG nova.policy [req-2984e8a3-daeb-41a4-9bea-f7d4a66bbecf 8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -] Policy check for compute_extension:hide_server_addresses failed with credentials {'domain': None, 'project_name': u'admin', 'project_domain': None, 'timestamp': '2016-11-27T11:53:40.915809', 'remote_address': '10.110.50.78', 'quota_class': None, 'resource_uuid': None, 'is_admin': True, 'user': u'8754985c51914ddd92cfb0132925474d', 'service_catalog': [{u'endpoints': [{u'adminURL': u'http://10.110.50.72:8776/v1/aeca5808a8da42bf97be22cb3571e172', u'region': u'nova', u'internalURL': u'http://10.110.50.72:8776/v1/aeca5808a8da42bf97be22cb3571e172', u'publicURL': u'https://10.110.81.6:8776/v1/aeca5808a8da42bf97be22cb3571e172'}], u'type': u'volume', u'name': u'cinder'}, {u'endpoints': [{u'adminURL': u'http://10.110.50.72:8776/v2/aeca5808a8da42bf97be22cb3571e172', u'region': u'nova', u'internalURL': u'http://10.110.50.72:8776/v2/aeca5808a8da42bf97be22cb3571e172', u'publicURL': u'https://10.110.81.6:8776/v2/aeca5808a8da42bf97be22cb3571e172'}], u'type': u'volumev2', u'name': u'cinderv2'}], 'tenant': u'aeca5808a8da42bf97be22cb3571e172', 'read_only': False, 'project_id': u'aeca5808a8da42bf97be22cb3571e172', 'user_id': u'8754985c51914ddd92cfb0132925474d', 'show_deleted': False, 'roles': [u'admin', u'_member_'], 'user_identity': '8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -', 'read_deleted': 'no', 'request_id': 'req-2984e8a3-daeb-41a4-9bea-f7d4a66bbecf', 'instance_lock_checked': False, 'user_domain': None, 'user_name': u'admin'} enforce /usr/lib/python2.7/dist-packages/nova/policy.py:104

2016-11-27 11:53:41.026 2015 INFO nova.osapi_compute.wsgi.server [req-2984e8a3-daeb-41a4-9bea-f7d4a66bbecf 8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -] 10.110.50.78,10.110.50.84 "GET /v2/aeca5808a8da42bf97be22cb3571e172/servers/detail?limit=21&project_id=aeca5808a8da42bf97be22cb3571e172 HTTP/1.1" status: 200 len: 16474 time: 0.1126778

2016-11-27 11:53:41.331 2013 DEBUG nova.api.openstack.wsgi [req-e3aa5194-3e07-442f-83a8-e343132c7d92 8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -] Calling method '<bound method LimitsController.index of <nova.api.openstack.compute.limits.LimitsController object at 0x7f6352430690>>' _process_stack /usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py:783

2016-11-27 11:53:41.340 2013 DEBUG nova.quota [req-e3aa5194-3e07-442f-83a8-e343132c7d92 8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -] Getting all quota usages for project: aeca5808a8da42bf97be22cb3571e172 get_project_quotas /usr/lib/python2.7/dist-packages/nova/quota.py:290

2016-11-27 11:53:41.349 2013 INFO nova.osapi_compute.wsgi.server [req-e3aa5194-3e07-442f-83a8-e343132c7d92 8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -] 10.110.50.78,10.110.50.84 "GET /v2/aeca5808a8da42bf97be22cb3571e172/limits?reserved=1 HTTP/1.1" status: 200 len: 712 time: 0.0198190

2016-11-27 11:53:41.509 2014 DEBUG nova.api.openstack.wsgi [req-58098bad-554d-4522-a11b-dcdb23df8962 8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -] Calling method '<bound method LimitsController.index of <nova.api.openstack.compute.limits.LimitsController object at 0x7f6352430690>>' _process_stack /usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py:783

2016-11-27 11:53:41.518 2014 DEBUG nova.quota [req-58098bad-554d-4522-a11b-dcdb23df8962 8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -] Getting all quota usages for project: aeca5808a8da42bf97be22cb3571e172 get_project_quotas /usr/lib/python2.7/dist-packages/nova/quota.py:290

2016-11-27 11:53:41.526 2014 INFO nova.osapi_compute.wsgi.server [req-58098bad-554d-4522-a11b-dcdb23df8962 8754985c51914ddd92cfb0132925474d aeca5808a8da42bf97be22cb3571e172 - - -] 10.110.50.78,10.110.50.84 "GET /v2/aeca5808a8da42bf97be22cb3571e172/limits?reserved=1 HTTP/1.1" status: 200 len: 712 time: 0.0190151

2016-11-27 11:53:47.135 2037 INFO nova.api.ec2 [-] 0.101s 10.110.50.84 GET / None:None 200 [None] text/plain text/plain

2016-11-27 11:53:47.135 2037 INFO nova.metadata.wsgi.server [-] 10.110.50.73,10.110.50.84 "GET / HTTP/1.0" status: 200 len: 234 time: 0.0007861

2016-11-27 11:53:57.139 2040 INFO nova.api.ec2 [-] 0.102s 10.110.50.84 GET / None:None 200 [None] text/plain text/plain

2016-11-27 11:53:57.139 2040 INFO nova.metadata.wsgi.server [-] 10.110.50.73,10.110.50.84 "GET / HTTP/1.0" status: 200 len: 234 time: 0.0009539

2016-11-27 11:54:07.143 2040 INFO nova.api.ec2 [-] 0.101s 10.110.50.84 GET / None:None 200 [None] text/plain text/plain

2016-11-27 11:54:07.143 2040 INFO nova.metadata.wsgi.server [-] 10.110.50.73,10.110.50.84 "GET / HTTP/1.0" status: 200 len: 234 time: 0.0009680

2016-11-27 11:54:17.146 2040 INFO nova.api.ec2 [-] 0.102s 10.110.50.84 GET / None:None 200 [None] text/plain text/plain

2016-11-27 11:54:17.147 2040 INFO nova.metadata.wsgi.server [-] 10.110.50.73,10.110.50.84 "GET / HTTP/1.0" status: 200 len: 234 time: 0.0009642

2016-11-27 11:54:27.150 2040 INFO nova.api.ec2 [-] 0.100s 10.110.50.84 GET / None:None 200 [None] text/plain text/plain

2016-11-27 11:54:27.151 2040 INFO nova.metadata.wsgi.server [-] 10.110.50.73,10.110.50.84 "GET / HTTP/1.0" status: 200 len: 234 time: 0.0009348

2016-11-27 11:54:32.362 2014 DEBUG keystoneclient.auth.identity.v2 [-] Making authentication request to http://10.110.50.72:35357/v2.0/tokens get_auth_ref /usr/lib/python2.7/dist-packages/keystoneclient/auth/identity/v2.py:76

2016-11-27 11:54:32.443 2014 DEBUG keystoneclient.session [-] REQ: curl -g -i -X GET http://10.110.50.72:35357/v3/auth/tokens -H "X-Subject-Token: {SHA1}a533179109fb2f8b1351e74c1e63da305647e41d" -H "User-Agent: python-keystoneclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}92d95b27e7a86d293c5689a44bb42d15acc2a092" _http_log_request /usr/lib/python2.7/dist-packages/keystoneclient/session.py:195

2016-11-27 11:54:32.487 2014 DEBUG keystoneclient.session [-] RESP: [200] content-length: 5733 x-subject-token: {SHA1}a533179109fb2f8b1351e74c1e63da305647e41d vary: X-Auth-Token x-distribution: Ubuntu date: Sun, 27 Nov 2016 11:54:32 GMT content-type: application/json x-openstack-request-id: req-8472c3c6-2b21-4a29-b037-1f462cf06752

RESP BODY: {"token": {"methods": ["password"], "roles": [{"id": "2c4f511851084da58e3a5dccd2c9a44f", "name": "admin"}, {"id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}], "expires_at": "2016-11-27T13:54:31.270761Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "9a2fd84f759f4d5bb3dbbdedf04d9d6d", "name": "service"}, "catalog": "<removed>", "extras": {}, "user": {"domain": {"id": "default", "name": "Default"}, "id": "2b1a66047ea74d32b933c3a72fdf51c1", "name": "ceilometer"}, "audit_ids": ["rsoleBDUSZakKJNaTlMDuQ"], "issued_at": "2016-11-27T11:54:31.000000Z"}}

_http_log_response /usr/lib/python2.7/dist-packages/keystoneclient/session.py:223

2016-11-27 11:54:32.489 2014 DEBUG nova.api.openstack.wsgi [req-87131690-c5ce-46a4-9fce-026ace08ac96 2b1a66047ea74d32b933c3a72fdf51c1 9a2fd84f759f4d5bb3dbbdedf04d9d6d - - -] Calling method '<bound method FloatingIPController.index of <nova.api.openstack.compute.contrib.floating_ips.FloatingIPController object at 0x7f63523986d0>>' _process_stack /usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py:783

2016-11-27 11:54:32.490 2014 DEBUG keystoneclient.session [req-87131690-c5ce-46a4-9fce-026ace08ac96 2b1a66047ea74d32b933c3a72fdf51c1 9a2fd84f759f4d5bb3dbbdedf04d9d6d - - -] REQ: curl -g -i -X GET http://10.110.50.72:9696/v2.0/floatingips.json?tenant_id=9a2fd84f759f4d5bb3dbbdedf04d9d6d -H "User-Agent: python-neutronclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}a533179109fb2f8b1351e74c1e63da305647e41d" _http_log_request /usr/lib/python2.7/dist-packages/keystoneclient/session.py:195

2016-11-27 11:54:32.510 2014 DEBUG keystoneclient.session [req-87131690-c5ce-46a4-9fce-026ace08ac96 2b1a66047ea74d32b933c3a72fdf51c1 9a2fd84f759f4d5bb3dbbdedf04d9d6d - - -] RESP: [200] date: Sun, 27 Nov 2016 11:54:32 GMT content-length: 19 content-type: application/json; charset=UTF-8 x-openstack-request-id: req-02ab8763-1b91-4581-a17c-b6fd92c3a5ee

RESP BODY: {"floatingips": []}

_http_log_response /usr/lib/python2.7/dist-packages/keystoneclient/session.py:223

2016-11-27 11:54:32.510 2014 DEBUG keystoneclient.session [req-87131690-c5ce-46a4-9fce-026ace08ac96 2b1a66047ea74d32b933c3a72fdf51c1 9a2fd84f759f4d5bb3dbbdedf04d9d6d - - -] REQ: curl -g -i -X GET http://10.110.50.72:9696/v2.0/networks.json?router%3Aexternal=True -H "User-Agent: python-neutronclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}a533179109fb2f8b1351e74c1e63da305647e41d" _http_log_request /usr/lib/python2.7/dist-packages/keystoneclient/session.py:195

2016-11-27 11:54:32.568 2014 DEBUG keystoneclient.session [req-87131690-c5ce-46a4-9fce-026ace08ac96 2b1a66047ea74d32b933c3a72fdf51c1 9a2fd84f759f4d5bb3dbbdedf04d9d6d - - -] RESP: [200] date: Sun, 27 Nov 2016 11:54:32 GMT content-length: 1261 content-type: application/json; charset=UTF-8 x-openstack-request-id: req-4e958017-5d81-4928-a177-ab8d06b27849

RESP BODY: {"networks": [{"status": "ACTIVE", "subnets": ["198e16c2-1eb1-4974-a6dc-dd9c6603fb28"], "name": "CoE_Bangalore_Ext", "provider:physical_network": "dvportgroup-3894", "router:external": true, "tenant_id": "4ba3a4434957426da0436d1520e2ea94", "admin_state_up": true, "provider:network_type": "portgroup", "shared": true, "port_security_enabled": true, "id": "0ca0a694-07b1-4bfe-8327-8352ca09fa81", "provider:segmentation_id": 0}, {"status": "ACTIVE", "subnets": ["5dd5e2c5-7cd3-4c89-afd9-c88a65c91cf1"], "name": "Certification_Ext_Network", "provider:physical_network": "dvportgroup-395", "router:external": true, "tenant_id": "aeca5808a8da42bf97be22cb3571e172", "admin_state_up": true, "provider:network_type": "portgroup", "shared": false, "port_security_enabled": true, "id": "1d527abe-3e9f-468f-afd1-08632f00158e", "provider:segmentation_id": 0}, {"status": "ACTIVE", "subnets": ["4dfbd5bf-7b53-4a41-b6eb-15f8a0ca0cdd"], "name": "CoE_Noida_External", "provider:physical_network": "dvportgroup-395", "router:external": true, "tenant_id": "2fed5018ae7541e99b7c18807dbaa50b", "admin_state_up": true, "provider:network_type": "portgroup", "shared": true, "port_security_enabled": true, "id": "5ed4efcb-25c3-4ec6-b6ac-ad34432ee042", "provider:segmentation_id": 0}]}

_http_log_response /usr/lib/python2.7/dist-packages/keystoneclient/session.py:223

2016-11-27 11:54:32.569 2014 DEBUG keystoneclient.session [req-87131690-c5ce-46a4-9fce-026ace08ac96 2b1a66047ea74d32b933c3a72fdf51c1 9a2fd84f759f4d5bb3dbbdedf04d9d6d - - -] REQ: curl -g -i -X GET http://10.110.50.72:9696/v2.0/ports.json?tenant_id=9a2fd84f759f4d5bb3dbbdedf04d9d6d -H "User-Agent: python-neutronclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}a533179109fb2f8b1351e74c1e63da305647e41d" _http_log_request /usr/lib/python2.7/dist-packages/keystoneclient/session.py:195

2016-11-27 11:54:32.650 2014 DEBUG keystoneclient.session [req-87131690-c5ce-46a4-9fce-026ace08ac96 2b1a66047ea74d32b933c3a72fdf51c1 9a2fd84f759f4d5bb3dbbdedf04d9d6d - - -] RESP: [200] date: Sun, 27 Nov 2016 11:54:32 GMT content-length: 13 content-type: application/json; charset=UTF-8 x-openstack-request-id: req-93ba853e-3c52-4af1-8153-5f634e5ccde2

RESP BODY: {"ports": []}

_http_log_response /usr/lib/python2.7/dist-packages/keystoneclient/session.py:223

2016-11-27 11:54:32.651 2014 INFO nova.osapi_compute.wsgi.server [req-87131690-c5ce-46a4-9fce-026ace08ac96 2b1a66047ea74d32b933c3a72fdf51c1 9a2fd84f759f4d5bb3dbbdedf04d9d6d - - -] 10.110.50.86,10.110.50.84 "GET /v2/9a2fd84f759f4d5bb3dbbdedf04d9d6d/os-floating-ips HTTP/1.1" status: 200 len: 211 time: 0.2899580

2016-11-27 11:54:34.777 2013 DEBUG nova.api.openstack.wsgi [req-867c2e4d-616d-4de3-bb06-3785a233ed75 2b1a66047ea74d32b933c3a72fdf51c1 9a2fd84f759f4d5bb3dbbdedf04d9d6d - - -] Calling method '<bound method Controller.show of <nova.api.openstack.compute.flavors.Controller object at 0x7f6352430c50>>' _process_stack /usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py:783

2016-11-27 11:54:34.786 2013 INFO nova.osapi_compute.wsgi.server [req-867c2e4d-616d-4de3-bb06-3785a233ed75 2b1a66047ea74d32b933c3a72fdf51c1 9a2fd84f759f4d5bb3dbbdedf04d9d6d - - -] 10.110.50.87,10.110.50.84 "GET /v2/9a2fd84f759f4d5bb3dbbdedf04d9d6d/flavors/810c73a0-7d35-4c74-840c-b246ce074b51 HTTP/1.1" status: 200 len: 724 time: 0.0113170

2016-11-27 11:54:34.970 2013 DEBUG nova.api.openstack.wsgi [req-48bc2e39-cf92-44b0-bddd-88d8282e857f 2b1a66047ea74d32b933c3a72fdf51c1 9a2fd84f759f4d5bb3dbbdedf04d9d6d - - -] Calling method '<bound method Controller.show of <nova.api.openstack.compute.flavors.Controller object at 0x7f6352430c50>>' _process_stack /usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py:783

2016-11-27 11:54:34.979 2013 INFO nova.osapi_compute.wsgi.server [req-48bc2e39-cf92-44b0-bddd-88d8282e857f 2b1a66047ea74d32b933c3a72fdf51c1 9a2fd84f759f4d5bb3dbbdedf04d9d6d - - -] 10.110.50.87,10.110.50.84 "GET /v2/9a2fd84f759f4d5bb3dbbdedf04d9d6d/flavors/2 HTTP/1.1" status: 200 len: 618 time: 0.0102479

2016-11-27 11:54:35.059 2013 DEBUG nova.api.openstack.wsgi [req-d7674615-c0ad-4fd8-b416-3ef12ae5ca27 2b1a66047ea74d32b933c3a72fdf51c1 9a2fd84f759f4d5bb3dbbdedf04d9d6d - - -] Calling method '<bound method Controller.show of <nova.api.openstack.compute.images.Controller object at 0x7f635241ef10>>' _process_stack /usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py:783

2016-11-27 11:54:35.149 2013 INFO nova.osapi_compute.wsgi.server [req-d7674615-c0ad-4fd8-b416-3ef12ae5ca27 2b1a66047ea74d32b933c3a72fdf51c1 9a2fd84f759f4d5bb3dbbdedf04d9d6d - - -] 10.110.50.87,10.110.50.84 "GET /v2/9a2fd84f759f4d5bb3dbbdedf04d9d6d/images/c5c13f1b-c8f2-404e-97b7-14e7879ffdc7 HTTP/1.1" status: 200 len: 1014 time: 0.0922749

2016-11-27 11:54:35.166 2013 DEBUG nova.api.openstack.wsgi [req-3b556c51-6eb2-44ae-9ce0-92243a9b78a1 2b1a66047ea74d32b933c3a72fdf51c1 9a2fd84f759f4d5bb3dbbdedf04d9d6d - - -] Calling method '<bound method Controller.show of <nova.api.openstack.compute.images.Controller object at 0x7f635241ef10>>' _process_stack /usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py:783

2016-11-27 11:54:35.326 2013 INFO nova.osapi_compute.wsgi.server [req-3b556c51-6eb2-44ae-9ce0-92243a9b78a1 2b1a66047ea74d32b933c3a72fdf51c1 9a2fd84f759f4d5bb3dbbdedf04d9d6d - - -] 10.110.50.87,10.110.50.84 "GET /v2/9a2fd84f759f4d5bb3dbbdedf04d9d6d/images/af331abe-2e8a-4bf5-b01f-9a358d240e32 HTTP/1.1" status: 200 len: 1029 time: 0.1622040

Can you help on this?

Thanks

Ratnajit