2026-06-10 08:27:38.278 14 DEBUG aodh.storage [-] looking for 'mysql+pymysql' driver in 'aodh.storage' get_connection_from_config /usr/lib/python3.9/site-packages/aodh/storage/__init__.py:60
2026-06-10 08:27:38.578 18 DEBUG aodh.storage [-] looking for 'mysql+pymysql' driver in 'aodh.storage' get_connection_from_config /usr/lib/python3.9/site-packages/aodh/storage/__init__.py:60
2026-06-10 08:27:38.610 14 INFO aodh.coordination [-] Coordination backend started successfully.
2026-06-10 08:27:38.616 20 DEBUG aodh.storage [-] looking for 'mysql+pymysql' driver in 'aodh.storage' get_connection_from_config /usr/lib/python3.9/site-packages/aodh/storage/__init__.py:60
2026-06-10 08:27:38.632 14 INFO aodh.coordination [-] Joined partitioning group alarm_evaluator
2026-06-10 08:27:38.799 20 INFO aodh.coordination [-] Coordination backend started successfully.
2026-06-10 08:27:38.813 20 INFO aodh.coordination [-] Joined partitioning group alarm_evaluator
2026-06-10 08:27:38.863 16 DEBUG aodh.storage [-] looking for 'mysql+pymysql' driver in 'aodh.storage' get_connection_from_config /usr/lib/python3.9/site-packages/aodh/storage/__init__.py:60
2026-06-10 08:27:39.025 18 INFO aodh.coordination [-] Coordination backend started successfully.
2026-06-10 08:27:39.045 18 INFO aodh.coordination [-] Joined partitioning group alarm_evaluator
2026-06-10 08:27:39.173 16 INFO aodh.coordination [-] Coordination backend started successfully.
2026-06-10 08:27:39.177 16 INFO aodh.coordination [-] Joined partitioning group alarm_evaluator
2026-06-10 08:28:38.637 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:28:38.675 14 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'7a28bea5-d075-44c7-a026-e40d440e67a7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:28:38.689 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:28:38.690 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:28:38.691 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:28:38.820 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:28:38.857 20 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'0cc56aaf-a660-4a2d-bd1e-955746045e98' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:28:38.864 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:28:38.864 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:28:38.865 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:28:39.052 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:28:39.101 18 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'518417dc-b941-4c6e-aae8-10988b15be26' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:28:39.110 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:28:39.110 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:28:39.111 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:28:39.181 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:28:39.214 16 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'e3d9a634-a908-4928-acb3-e9e7558495d7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:28:39.217 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:28:39.217 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:28:39.218 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:29:38.633 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:29:38.644 14 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'7a28bea5-d075-44c7-a026-e40d440e67a7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:29:38.647 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:29:38.647 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:29:38.647 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:29:38.815 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:29:38.824 20 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'0cc56aaf-a660-4a2d-bd1e-955746045e98' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:29:38.828 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:29:38.828 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:29:38.828 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:29:39.047 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:29:39.065 18 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'518417dc-b941-4c6e-aae8-10988b15be26' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:29:39.074 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:29:39.075 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:29:39.075 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:29:39.178 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:29:39.186 16 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'e3d9a634-a908-4928-acb3-e9e7558495d7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:29:39.191 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:29:39.192 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:29:39.192 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:30:38.634 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:30:38.647 14 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'7a28bea5-d075-44c7-a026-e40d440e67a7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:30:38.653 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:30:38.654 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:30:38.654 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:30:38.815 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:30:38.824 20 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'0cc56aaf-a660-4a2d-bd1e-955746045e98' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:30:38.827 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:30:38.827 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:30:38.827 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:30:39.048 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:30:39.099 18 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'518417dc-b941-4c6e-aae8-10988b15be26' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:30:39.102 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:30:39.102 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:30:39.102 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:30:39.178 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:30:39.188 16 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'e3d9a634-a908-4928-acb3-e9e7558495d7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:30:39.192 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:30:39.192 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:30:39.193 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:31:38.634 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:31:38.647 14 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'7a28bea5-d075-44c7-a026-e40d440e67a7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:31:38.654 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:31:38.654 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:31:38.654 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:31:38.815 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:31:38.824 20 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'0cc56aaf-a660-4a2d-bd1e-955746045e98' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:31:38.827 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:31:38.827 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:31:38.827 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:31:39.049 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:31:39.059 18 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'518417dc-b941-4c6e-aae8-10988b15be26' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:31:39.064 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:31:39.064 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:31:39.064 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:31:39.179 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:31:39.192 16 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'e3d9a634-a908-4928-acb3-e9e7558495d7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:31:39.199 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:31:39.199 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:31:39.199 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:32:38.635 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:32:38.645 14 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'7a28bea5-d075-44c7-a026-e40d440e67a7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:32:38.649 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:32:38.649 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:32:38.649 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:32:38.816 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:32:38.829 20 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'0cc56aaf-a660-4a2d-bd1e-955746045e98' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:32:38.835 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:32:38.835 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:32:38.835 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:32:39.049 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:32:39.060 18 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'518417dc-b941-4c6e-aae8-10988b15be26' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:32:39.063 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:32:39.064 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:32:39.064 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:32:39.179 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:32:39.190 16 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'e3d9a634-a908-4928-acb3-e9e7558495d7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:32:39.194 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:32:39.194 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:32:39.194 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:33:38.636 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:33:38.648 14 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'7a28bea5-d075-44c7-a026-e40d440e67a7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:33:38.652 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:33:38.652 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:33:38.653 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:33:38.816 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:33:38.827 20 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'0cc56aaf-a660-4a2d-bd1e-955746045e98' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:33:38.834 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:33:38.834 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:33:38.834 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:33:39.049 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:33:39.058 18 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'518417dc-b941-4c6e-aae8-10988b15be26' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:33:39.062 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:33:39.063 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:33:39.063 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:33:39.179 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:33:39.191 16 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'e3d9a634-a908-4928-acb3-e9e7558495d7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:33:39.197 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:33:39.198 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:33:39.198 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:34:38.635 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:34:38.816 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:34:39.050 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:34:39.180 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:34:39.473 14 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'7a28bea5-d075-44c7-a026-e40d440e67a7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:34:39.473 20 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'0cc56aaf-a660-4a2d-bd1e-955746045e98' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:34:39.476 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:34:39.477 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:34:39.477 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:34:39.479 18 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'518417dc-b941-4c6e-aae8-10988b15be26' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:34:39.481 16 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'e3d9a634-a908-4928-acb3-e9e7558495d7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:34:39.481 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:34:39.482 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:34:39.482 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:34:39.482 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:34:39.483 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:34:39.483 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:34:39.484 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:34:39.485 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:34:39.485 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:35:38.635 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:35:38.645 14 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'7a28bea5-d075-44c7-a026-e40d440e67a7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:35:38.648 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:35:38.648 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:35:38.648 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:35:38.817 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:35:38.829 20 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'0cc56aaf-a660-4a2d-bd1e-955746045e98' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:35:38.835 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:35:38.835 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:35:38.835 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:35:39.050 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:35:39.062 18 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'518417dc-b941-4c6e-aae8-10988b15be26' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:35:39.070 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:35:39.070 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:35:39.070 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:35:39.181 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:35:39.193 16 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'e3d9a634-a908-4928-acb3-e9e7558495d7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:35:39.200 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:35:39.200 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:35:39.201 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:36:38.636 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:36:38.651 14 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'7a28bea5-d075-44c7-a026-e40d440e67a7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:36:38.655 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:36:38.656 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:36:38.656 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:36:38.817 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:36:38.828 20 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'0cc56aaf-a660-4a2d-bd1e-955746045e98' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:36:38.831 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:36:38.831 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:36:38.831 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:36:39.050 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:36:39.060 18 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'518417dc-b941-4c6e-aae8-10988b15be26' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:36:39.063 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:36:39.064 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:36:39.064 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:36:39.181 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:36:39.192 16 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'e3d9a634-a908-4928-acb3-e9e7558495d7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:36:39.197 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:36:39.197 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:36:39.198 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:37:38.636 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:37:38.650 14 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'7a28bea5-d075-44c7-a026-e40d440e67a7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:37:38.656 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:37:38.656 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:37:38.656 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:37:38.818 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:37:38.830 20 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'0cc56aaf-a660-4a2d-bd1e-955746045e98' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:37:38.836 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:37:38.836 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:37:38.836 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:37:39.051 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:37:39.062 18 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'518417dc-b941-4c6e-aae8-10988b15be26' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:37:39.071 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:37:39.072 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:37:39.072 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:37:39.181 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:37:39.193 16 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'e3d9a634-a908-4928-acb3-e9e7558495d7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:37:39.199 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:37:39.199 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:37:39.200 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:38:38.636 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:38:38.649 14 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'7a28bea5-d075-44c7-a026-e40d440e67a7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:38:38.655 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:38:38.655 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:38:38.656 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:38:38.818 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:38:38.833 20 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'0cc56aaf-a660-4a2d-bd1e-955746045e98' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:38:38.837 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:38:38.837 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:38:38.837 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:38:39.052 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:38:39.064 18 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'518417dc-b941-4c6e-aae8-10988b15be26' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:38:39.070 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:38:39.071 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:38:39.071 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:38:39.182 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:38:39.193 16 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'e3d9a634-a908-4928-acb3-e9e7558495d7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:38:39.201 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:38:39.201 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:38:39.201 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:39:28.326 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.327 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.327 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.328 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.337 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.337 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.337 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.337 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.340 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.341 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.343 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.343 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.347 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.347 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.348 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.349 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.350 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.352 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.352 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.353 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.353 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.356 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.358 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.360 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.362 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.363 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.365 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.367 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.367 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.369 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.369 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.370 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.370 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.371 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.371 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.372 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.373 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.374 18 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:28.374 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.376 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.376 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.377 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.378 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.379 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.380 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.381 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.374 18 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:28.374 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:28.374 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:28.374 18 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:28.374 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:28.374 18 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:28.374 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:28.374 18 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:28.374 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:28.374 18 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:28.374 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:28.374 18 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:28.374 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:28.374 18 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:28.374 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:28.374 18 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:28.374 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:28.374 18 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:28.374 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:28.374 18 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:28.374 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:28.374 18 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:28.374 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:28.374 18 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:28.374 18 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:28.374 18 ERROR aodh.coordination 
2026-06-10 08:39:28.374 18 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:28.374 18 ERROR aodh.coordination 
2026-06-10 08:39:28.374 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:28.374 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:28.374 18 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:28.374 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:28.374 18 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:28.374 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:28.374 18 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:28.374 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:28.374 18 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:28.374 18 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:28.374 18 ERROR aodh.coordination 
2026-06-10 08:39:28.386 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.387 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.390 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.992 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.078 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.475 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.803 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.813 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.813 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.814 16 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:29.815 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.817 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.818 20 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:29.820 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.818 20 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.818 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.818 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:29.818 20 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:29.818 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:29.818 20 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:29.818 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:29.818 20 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:29.818 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:29.818 20 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:29.818 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:29.818 20 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:29.818 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:29.818 20 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:29.818 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:29.818 20 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:29.818 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:29.818 20 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:29.818 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:29.818 20 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:29.818 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:29.818 20 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:29.818 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:29.818 20 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:29.818 20 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:29.818 20 ERROR aodh.coordination 
2026-06-10 08:39:29.818 20 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:29.818 20 ERROR aodh.coordination 
2026-06-10 08:39:29.818 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.818 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:29.818 20 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:29.818 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:29.818 20 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:29.818 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:29.818 20 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:29.818 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:29.818 20 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:29.818 20 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.818 20 ERROR aodh.coordination 
2026-06-10 08:39:29.815 16 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.815 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.815 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:29.815 16 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:29.815 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:29.815 16 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:29.815 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:29.815 16 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:29.815 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:29.815 16 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:29.815 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:29.815 16 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:29.815 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:29.815 16 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:29.815 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:29.815 16 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:29.815 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:29.815 16 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:29.815 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:29.815 16 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:29.815 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:29.815 16 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:29.815 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:29.815 16 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:29.815 16 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:29.815 16 ERROR aodh.coordination 
2026-06-10 08:39:29.815 16 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:29.815 16 ERROR aodh.coordination 
2026-06-10 08:39:29.815 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.815 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:29.815 16 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:29.815 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:29.815 16 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:29.815 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:29.815 16 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:29.815 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:29.815 16 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:29.815 16 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.815 16 ERROR aodh.coordination 
2026-06-10 08:39:29.823 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.826 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.830 14 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:29.830 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.830 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.831 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.831 14 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.831 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.831 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:29.831 14 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:29.831 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:29.831 14 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:29.831 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:29.831 14 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:29.831 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:29.831 14 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:29.831 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:29.831 14 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:29.831 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:29.831 14 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:29.831 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:29.831 14 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:29.831 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:29.831 14 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:29.831 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:29.831 14 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:29.831 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:29.831 14 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:29.831 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:29.831 14 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:29.831 14 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:29.831 14 ERROR aodh.coordination 
2026-06-10 08:39:29.831 14 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:29.831 14 ERROR aodh.coordination 
2026-06-10 08:39:29.831 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.831 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:29.831 14 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:29.831 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:29.831 14 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:29.831 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:29.831 14 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:29.831 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:29.831 14 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:29.831 14 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.831 14 ERROR aodh.coordination 
2026-06-10 08:39:29.834 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.834 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.834 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.844 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.844 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.844 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.845 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.848 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.848 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.848 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.849 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.852 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.853 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.853 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.853 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.854 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.855 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.860 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.860 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.860 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.861 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.861 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.862 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.863 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.863 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.863 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.863 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.864 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.866 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.869 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.869 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.869 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.871 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.872 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.872 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.872 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.873 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.874 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.874 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.874 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.875 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.875 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.876 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.876 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.879 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.879 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.879 16 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:29.879 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.880 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.880 18 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:29.879 16 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.879 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.879 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:29.879 16 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:29.879 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:29.879 16 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:29.879 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:29.879 16 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:29.879 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:29.879 16 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:29.879 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:29.879 16 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:29.879 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:29.879 16 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:29.879 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:29.879 16 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:29.879 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:29.879 16 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:29.879 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:29.879 16 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:29.879 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:29.879 16 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:29.879 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:29.879 16 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:29.879 16 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:29.879 16 ERROR aodh.coordination 
2026-06-10 08:39:29.879 16 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:29.879 16 ERROR aodh.coordination 
2026-06-10 08:39:29.879 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.879 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:29.879 16 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:29.879 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:29.879 16 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:29.879 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:29.879 16 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:29.879 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:29.879 16 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:29.879 16 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.879 16 ERROR aodh.coordination 
2026-06-10 08:39:29.881 18 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.881 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.881 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:29.881 18 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:29.881 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:29.881 18 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:29.881 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:29.881 18 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:29.881 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:29.881 18 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:29.881 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:29.881 18 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:29.881 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:29.881 18 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:29.881 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:29.881 18 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:29.881 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:29.881 18 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:29.881 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:29.881 18 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:29.881 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:29.881 18 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:29.881 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:29.881 18 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:29.881 18 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:29.881 18 ERROR aodh.coordination 
2026-06-10 08:39:29.881 18 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:29.881 18 ERROR aodh.coordination 
2026-06-10 08:39:29.881 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.881 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:29.881 18 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:29.881 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:29.881 18 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:29.881 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:29.881 18 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:29.881 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:29.881 18 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:29.881 18 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.881 18 ERROR aodh.coordination 
2026-06-10 08:39:29.882 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.883 14 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:29.883 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.883 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.883 14 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.883 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.883 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:29.883 14 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:29.883 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:29.883 14 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:29.883 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:29.883 14 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:29.883 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:29.883 14 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:29.883 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:29.883 14 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:29.883 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:29.883 14 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:29.883 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:29.883 14 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:29.883 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:29.883 14 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:29.883 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:29.883 14 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:29.883 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:29.883 14 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:29.883 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:29.883 14 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:29.883 14 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:29.883 14 ERROR aodh.coordination 
2026-06-10 08:39:29.883 14 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:29.883 14 ERROR aodh.coordination 
2026-06-10 08:39:29.883 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.883 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:29.883 14 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:29.883 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:29.883 14 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:29.883 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:29.883 14 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:29.883 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:29.883 14 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:29.883 14 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.883 14 ERROR aodh.coordination 
2026-06-10 08:39:29.885 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.885 20 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:29.885 20 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.885 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.885 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:29.885 20 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:29.885 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:29.885 20 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:29.885 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:29.885 20 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:29.885 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:29.885 20 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:29.885 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:29.885 20 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:29.885 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:29.885 20 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:29.885 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:29.885 20 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:29.885 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:29.885 20 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:29.885 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:29.885 20 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:29.885 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:29.885 20 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:29.885 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:29.885 20 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:29.885 20 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:29.885 20 ERROR aodh.coordination 
2026-06-10 08:39:29.885 20 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:29.885 20 ERROR aodh.coordination 
2026-06-10 08:39:29.885 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.885 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:29.885 20 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:29.885 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:29.885 20 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:29.885 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:29.885 20 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:29.885 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:29.885 20 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:29.885 20 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.885 20 ERROR aodh.coordination 
2026-06-10 08:39:29.890 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.892 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.895 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.896 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.900 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.901 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.903 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.905 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.906 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.910 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.913 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.915 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.917 18 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:29.918 18 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.918 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.918 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:29.918 18 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:29.918 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:29.918 18 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:29.918 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:29.918 18 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:29.918 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:29.918 18 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:29.918 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:29.918 18 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:29.918 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:29.918 18 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:29.918 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:29.918 18 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:29.918 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:29.918 18 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:29.918 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:29.918 18 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:29.918 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:29.918 18 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:29.918 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:29.918 18 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:29.918 18 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:29.918 18 ERROR aodh.coordination 
2026-06-10 08:39:29.918 18 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:29.918 18 ERROR aodh.coordination 
2026-06-10 08:39:29.918 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.918 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:29.918 18 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:29.918 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:29.918 18 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:29.918 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:29.918 18 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:29.918 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:29.918 18 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:29.918 18 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.918 18 ERROR aodh.coordination 
2026-06-10 08:39:34.562 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.562 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.564 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.565 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.566 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.568 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.570 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.571 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.570 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.572 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.574 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.574 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.573 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.574 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.575 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.575 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.575 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.576 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.577 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.577 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.577 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.578 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.579 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.579 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.579 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.580 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.580 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.581 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.583 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.584 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.584 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.584 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.585 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.586 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.587 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.589 18 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:34.590 18 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.590 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.590 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:34.590 18 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:34.590 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:34.590 18 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:34.590 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:34.590 18 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:34.590 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:34.590 18 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:34.590 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:34.590 18 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:34.590 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:34.590 18 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:34.590 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:34.590 18 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:34.590 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:34.590 18 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:34.590 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:34.590 18 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:34.590 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:34.590 18 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:34.590 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:34.590 18 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:34.590 18 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:34.590 18 ERROR aodh.coordination 
2026-06-10 08:39:34.590 18 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:34.590 18 ERROR aodh.coordination 
2026-06-10 08:39:34.590 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.590 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:34.590 18 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:34.590 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:34.590 18 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:34.590 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:34.590 18 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:34.590 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:34.590 18 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:34.590 18 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.590 18 ERROR aodh.coordination 
2026-06-10 08:39:34.594 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.597 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.599 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.600 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.600 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.601 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.602 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.602 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.603 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.603 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.604 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.604 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.605 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.606 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.606 20 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:34.606 20 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.606 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.606 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:34.606 20 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:34.606 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:34.606 20 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:34.606 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:34.606 20 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:34.606 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:34.606 20 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:34.606 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:34.606 20 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:34.606 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:34.606 20 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:34.606 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:34.606 20 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:34.606 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:34.606 20 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:34.606 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:34.606 20 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:34.606 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:34.606 20 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:34.606 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:34.606 20 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:34.606 20 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:34.606 20 ERROR aodh.coordination 
2026-06-10 08:39:34.606 20 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:34.606 20 ERROR aodh.coordination 
2026-06-10 08:39:34.606 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.606 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:34.606 20 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:34.606 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:34.606 20 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:34.606 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:34.606 20 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:34.606 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:34.606 20 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:34.606 20 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.606 20 ERROR aodh.coordination 
2026-06-10 08:39:34.607 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.607 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.609 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.609 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.610 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.611 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.611 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.613 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.613 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.614 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.614 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.615 14 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:34.615 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.615 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.615 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.616 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.615 14 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.615 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.615 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:34.615 14 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:34.615 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:34.615 14 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:34.615 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:34.615 14 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:34.615 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:34.615 14 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:34.615 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:34.615 14 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:34.615 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:34.615 14 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:34.615 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:34.615 14 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:34.615 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:34.615 14 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:34.615 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:34.615 14 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:34.615 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:34.615 14 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:34.615 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:34.615 14 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:34.615 14 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:34.615 14 ERROR aodh.coordination 
2026-06-10 08:39:34.615 14 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:34.615 14 ERROR aodh.coordination 
2026-06-10 08:39:34.615 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.615 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:34.615 14 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:34.615 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:34.615 14 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:34.615 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:34.615 14 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:34.615 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:34.615 14 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:34.615 14 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.615 14 ERROR aodh.coordination 
2026-06-10 08:39:34.617 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.617 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.618 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.618 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.619 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.619 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.620 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.620 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.620 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.621 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.621 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.621 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.622 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.622 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.623 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.623 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.624 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.624 18 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:34.625 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.624 18 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.624 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.624 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:34.624 18 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:34.624 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:34.624 18 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:34.624 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:34.624 18 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:34.624 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:34.624 18 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:34.624 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:34.624 18 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:34.624 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:34.624 18 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:34.624 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:34.624 18 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:34.624 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:34.624 18 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:34.624 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:34.624 18 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:34.624 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:34.624 18 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:34.624 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:34.624 18 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:34.624 18 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:34.624 18 ERROR aodh.coordination 
2026-06-10 08:39:34.624 18 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:34.624 18 ERROR aodh.coordination 
2026-06-10 08:39:34.624 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.624 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:34.624 18 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:34.624 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:34.624 18 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:34.624 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:34.624 18 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:34.624 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:34.624 18 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:34.624 18 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.624 18 ERROR aodh.coordination 
2026-06-10 08:39:34.626 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.627 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.627 16 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:34.627 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.628 16 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.628 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.628 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:34.628 16 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:34.628 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:34.628 16 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:34.628 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:34.628 16 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:34.628 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:34.628 16 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:34.628 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:34.628 16 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:34.628 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:34.628 16 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:34.628 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:34.628 16 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:34.628 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:34.628 16 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:34.628 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:34.628 16 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:34.628 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:34.628 16 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:34.628 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:34.628 16 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:34.628 16 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:34.628 16 ERROR aodh.coordination 
2026-06-10 08:39:34.628 16 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:34.628 16 ERROR aodh.coordination 
2026-06-10 08:39:34.628 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.628 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:34.628 16 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:34.628 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:34.628 16 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:34.628 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:34.628 16 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:34.628 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:34.628 16 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:34.628 16 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.628 16 ERROR aodh.coordination 
2026-06-10 08:39:34.628 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.629 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.629 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.629 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.630 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.630 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.630 20 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:34.631 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.630 20 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.630 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.630 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:34.630 20 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:34.630 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:34.630 20 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:34.630 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:34.630 20 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:34.630 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:34.630 20 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:34.630 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:34.630 20 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:34.630 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:34.630 20 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:34.630 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:34.630 20 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:34.630 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:34.630 20 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:34.630 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:34.630 20 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:34.630 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:34.630 20 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:34.630 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:34.630 20 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:34.630 20 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:34.630 20 ERROR aodh.coordination 
2026-06-10 08:39:34.630 20 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:34.630 20 ERROR aodh.coordination 
2026-06-10 08:39:34.630 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.630 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:34.630 20 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:34.630 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:34.630 20 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:34.630 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:34.630 20 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:34.630 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:34.630 20 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:34.630 20 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.630 20 ERROR aodh.coordination 
2026-06-10 08:39:34.631 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.632 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.632 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.633 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.633 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.636 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.636 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.637 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.637 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.638 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.638 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.639 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.639 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.640 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.640 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.641 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.641 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.642 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.642 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.643 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.643 14 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:34.643 14 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.643 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.643 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:34.643 14 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:34.643 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:34.643 14 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:34.643 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:34.643 14 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:34.643 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:34.643 14 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:34.643 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:34.643 14 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:34.643 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:34.643 14 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:34.643 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:34.643 14 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:34.643 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:34.643 14 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:34.643 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:34.643 14 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:34.643 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:34.643 14 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:34.643 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:34.643 14 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:34.643 14 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:34.643 14 ERROR aodh.coordination 
2026-06-10 08:39:34.643 14 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:34.643 14 ERROR aodh.coordination 
2026-06-10 08:39:34.643 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.643 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:34.643 14 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:34.643 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:34.643 14 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:34.643 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:34.643 14 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:34.643 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:34.643 14 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:34.643 14 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.643 14 ERROR aodh.coordination 
2026-06-10 08:39:34.644 16 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:34.644 16 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.644 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.644 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:34.644 16 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:34.644 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:34.644 16 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:34.644 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:34.644 16 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:34.644 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:34.644 16 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:34.644 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:34.644 16 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:34.644 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:34.644 16 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:34.644 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:34.644 16 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:34.644 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:34.644 16 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:34.644 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:34.644 16 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:34.644 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:34.644 16 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:34.644 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:34.644 16 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:34.644 16 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:34.644 16 ERROR aodh.coordination 
2026-06-10 08:39:34.644 16 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:34.644 16 ERROR aodh.coordination 
2026-06-10 08:39:34.644 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.644 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:34.644 16 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:34.644 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:34.644 16 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:34.644 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:34.644 16 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:34.644 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:34.644 16 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:34.644 16 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.644 16 ERROR aodh.coordination 
2026-06-10 08:39:36.135 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.140 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.141 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.142 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.142 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.155 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.155 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.157 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.156 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.157 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.157 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.158 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.159 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.159 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.160 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.161 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.161 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.162 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.162 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.163 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.163 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.164 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.164 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.165 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.165 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.166 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.166 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.167 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.166 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.167 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.168 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.168 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.168 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.169 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.169 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.170 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.171 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.171 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.171 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.171 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.172 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.172 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.172 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.173 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.173 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.174 18 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:36.174 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.174 18 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.174 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.174 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:36.174 18 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:36.174 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:36.174 18 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:36.174 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:36.174 18 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:36.174 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:36.174 18 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:36.174 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:36.174 18 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:36.174 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:36.174 18 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:36.174 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:36.174 18 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:36.174 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:36.174 18 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:36.174 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:36.174 18 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:36.174 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:36.174 18 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:36.174 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:36.174 18 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:36.174 18 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:36.174 18 ERROR aodh.coordination 
2026-06-10 08:39:36.174 18 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:36.174 18 ERROR aodh.coordination 
2026-06-10 08:39:36.174 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.174 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:36.174 18 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:36.174 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:36.174 18 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:36.174 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:36.174 18 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:36.174 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:36.174 18 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:36.174 18 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.174 18 ERROR aodh.coordination 
2026-06-10 08:39:36.175 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.175 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.175 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.176 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.176 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.177 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.177 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.178 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.179 14 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:36.179 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.179 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.179 14 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.179 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.179 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:36.179 14 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:36.179 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:36.179 14 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:36.179 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:36.179 14 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:36.179 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:36.179 14 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:36.179 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:36.179 14 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:36.179 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:36.179 14 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:36.179 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:36.179 14 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:36.179 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:36.179 14 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:36.179 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:36.179 14 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:36.179 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:36.179 14 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:36.179 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:36.179 14 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:36.179 14 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:36.179 14 ERROR aodh.coordination 
2026-06-10 08:39:36.179 14 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:36.179 14 ERROR aodh.coordination 
2026-06-10 08:39:36.179 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.179 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:36.179 14 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:36.179 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:36.179 14 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:36.179 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:36.179 14 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:36.179 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:36.179 14 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:36.179 14 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.179 14 ERROR aodh.coordination 
2026-06-10 08:39:36.180 16 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:36.181 20 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:36.181 16 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.181 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.181 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:36.181 16 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:36.181 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:36.181 16 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:36.181 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:36.181 16 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:36.181 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:36.181 16 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:36.181 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:36.181 16 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:36.181 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:36.181 16 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:36.181 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:36.181 16 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:36.181 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:36.181 16 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:36.181 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:36.181 16 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:36.181 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:36.181 16 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:36.181 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:36.181 16 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:36.181 16 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:36.181 16 ERROR aodh.coordination 
2026-06-10 08:39:36.181 16 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:36.181 16 ERROR aodh.coordination 
2026-06-10 08:39:36.181 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.181 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:36.181 16 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:36.181 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:36.181 16 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:36.181 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:36.181 16 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:36.181 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:36.181 16 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:36.181 16 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.181 16 ERROR aodh.coordination 
2026-06-10 08:39:36.181 20 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.181 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.181 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:36.181 20 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:36.181 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:36.181 20 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:36.181 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:36.181 20 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:36.181 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:36.181 20 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:36.181 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:36.181 20 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:36.181 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:36.181 20 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:36.181 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:36.181 20 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:36.181 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:36.181 20 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:36.181 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:36.181 20 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:36.181 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:36.181 20 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:36.181 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:36.181 20 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:36.181 20 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:36.181 20 ERROR aodh.coordination 
2026-06-10 08:39:36.181 20 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:36.181 20 ERROR aodh.coordination 
2026-06-10 08:39:36.181 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.181 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:36.181 20 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:36.181 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:36.181 20 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:36.181 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:36.181 20 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:36.181 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:36.181 20 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:36.181 20 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.181 20 ERROR aodh.coordination 
2026-06-10 08:39:36.972 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.973 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.974 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.975 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.975 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.976 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.977 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.977 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.978 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.977 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.978 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.978 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.979 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.979 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.979 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.980 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.981 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.981 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.981 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.982 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.982 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.981 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.982 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.983 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.983 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.983 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.983 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.984 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.984 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.984 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.985 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.985 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.985 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.986 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.986 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.986 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.986 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.987 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.987 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.987 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.987 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.988 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.988 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.988 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.988 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.989 16 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:36.989 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.989 14 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:36.989 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.990 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.990 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.989 14 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.989 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.989 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:36.989 14 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:36.989 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:36.989 14 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:36.989 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:36.989 14 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:36.989 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:36.989 14 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:36.989 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:36.989 14 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:36.989 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:36.989 14 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:36.989 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:36.989 14 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:36.989 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:36.989 14 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:36.989 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:36.989 14 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:36.989 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:36.989 14 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:36.989 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:36.989 14 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:36.989 14 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:36.989 14 ERROR aodh.coordination 
2026-06-10 08:39:36.989 14 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:36.989 14 ERROR aodh.coordination 
2026-06-10 08:39:36.989 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.989 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:36.989 14 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:36.989 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:36.989 14 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:36.989 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:36.989 14 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:36.989 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:36.989 14 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:36.989 14 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.989 14 ERROR aodh.coordination 
2026-06-10 08:39:36.989 16 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.989 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.989 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:36.989 16 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:36.989 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:36.989 16 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:36.989 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:36.989 16 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:36.989 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:36.989 16 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:36.989 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:36.989 16 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:36.989 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:36.989 16 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:36.989 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:36.989 16 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:36.989 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:36.989 16 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:36.989 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:36.989 16 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:36.989 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:36.989 16 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:36.989 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:36.989 16 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:36.989 16 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:36.989 16 ERROR aodh.coordination 
2026-06-10 08:39:36.989 16 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:36.989 16 ERROR aodh.coordination 
2026-06-10 08:39:36.989 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.989 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:36.989 16 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:36.989 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:36.989 16 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:36.989 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:36.989 16 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:36.989 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:36.989 16 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:36.989 16 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.989 16 ERROR aodh.coordination 
2026-06-10 08:39:36.991 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.991 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.992 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.992 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.993 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.994 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.994 18 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:36.995 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.994 18 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.994 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.994 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:36.994 18 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:36.994 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:36.994 18 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:36.994 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:36.994 18 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:36.994 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:36.994 18 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:36.994 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:36.994 18 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:36.994 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:36.994 18 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:36.994 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:36.994 18 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:36.994 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:36.994 18 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:36.994 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:36.994 18 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:36.994 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:36.994 18 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:36.994 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:36.994 18 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:36.994 18 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:36.994 18 ERROR aodh.coordination 
2026-06-10 08:39:36.994 18 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:36.994 18 ERROR aodh.coordination 
2026-06-10 08:39:36.994 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.994 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:36.994 18 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:36.994 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:36.994 18 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:36.994 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:36.994 18 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:36.994 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:36.994 18 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:36.994 18 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.994 18 ERROR aodh.coordination 
2026-06-10 08:39:36.997 20 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:36.998 20 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.998 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.998 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:36.998 20 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:36.998 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:36.998 20 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:36.998 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:36.998 20 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:36.998 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:36.998 20 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:36.998 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:36.998 20 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:36.998 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:36.998 20 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:36.998 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:36.998 20 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:36.998 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:36.998 20 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:36.998 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:36.998 20 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:36.998 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:36.998 20 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:36.998 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:36.998 20 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:36.998 20 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:36.998 20 ERROR aodh.coordination 
2026-06-10 08:39:36.998 20 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:36.998 20 ERROR aodh.coordination 
2026-06-10 08:39:36.998 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.998 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:36.998 20 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:36.998 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:36.998 20 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:36.998 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:36.998 20 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:36.998 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:36.998 20 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:36.998 20 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.998 20 ERROR aodh.coordination 
2026-06-10 08:39:38.637 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:39:38.653 14 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'7a28bea5-d075-44c7-a026-e40d440e67a7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:39:38.658 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:39:38.659 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:39:38.659 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:39:38.818 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:39:38.831 20 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'0cc56aaf-a660-4a2d-bd1e-955746045e98' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:39:38.834 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:39:38.834 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:39:38.834 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:39:39.053 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:39:39.068 18 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'518417dc-b941-4c6e-aae8-10988b15be26' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:39:39.074 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:39:39.074 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:39:39.075 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:39:39.182 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:39:39.195 16 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'e3d9a634-a908-4928-acb3-e9e7558495d7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:39:39.201 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:39:39.202 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:39:39.202 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:40:38.637 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:40:38.651 14 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'7a28bea5-d075-44c7-a026-e40d440e67a7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:40:38.655 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:40:38.655 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:40:38.655 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:40:38.819 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:40:38.831 20 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'0cc56aaf-a660-4a2d-bd1e-955746045e98' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:40:38.837 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:40:38.837 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:40:38.837 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:40:39.053 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:40:39.067 18 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'518417dc-b941-4c6e-aae8-10988b15be26' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:40:39.075 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:40:39.075 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:40:39.075 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:40:39.182 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:40:39.194 16 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'e3d9a634-a908-4928-acb3-e9e7558495d7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:40:39.199 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:40:39.199 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:40:39.200 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:41:38.638 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:41:38.650 14 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'7a28bea5-d075-44c7-a026-e40d440e67a7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:41:38.656 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:41:38.656 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:41:38.656 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:41:38.819 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:41:38.831 20 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'0cc56aaf-a660-4a2d-bd1e-955746045e98' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:41:38.837 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:41:38.838 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:41:38.838 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:41:39.053 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:41:39.065 18 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'518417dc-b941-4c6e-aae8-10988b15be26' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:41:39.071 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:41:39.072 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:41:39.072 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:41:39.183 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:41:39.195 16 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'e3d9a634-a908-4928-acb3-e9e7558495d7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:41:39.202 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:41:39.202 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:41:39.202 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:42:38.638 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:42:38.657 14 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'7a28bea5-d075-44c7-a026-e40d440e67a7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:42:38.666 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:42:38.667 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:42:38.667 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:42:38.819 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:42:38.830 20 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'0cc56aaf-a660-4a2d-bd1e-955746045e98' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:42:38.833 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:42:38.833 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:42:38.833 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:42:39.054 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:42:39.065 18 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'518417dc-b941-4c6e-aae8-10988b15be26' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:42:39.074 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:42:39.074 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:42:39.075 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:42:39.183 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:42:39.195 16 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'e3d9a634-a908-4928-acb3-e9e7558495d7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:42:39.198 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:42:39.198 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:42:39.198 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:43:38.638 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:43:38.648 14 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'7a28bea5-d075-44c7-a026-e40d440e67a7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:43:38.652 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:43:38.652 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:43:38.652 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:43:38.820 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:43:38.832 20 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'0cc56aaf-a660-4a2d-bd1e-955746045e98' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:43:38.838 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:43:38.839 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:43:38.839 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:43:39.054 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:43:39.066 18 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'518417dc-b941-4c6e-aae8-10988b15be26' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:43:39.072 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:43:39.073 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:43:39.073 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:43:39.184 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:43:39.195 16 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'e3d9a634-a908-4928-acb3-e9e7558495d7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:43:39.202 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:43:39.202 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:43:39.203 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:44:38.639 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:44:38.651 14 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'7a28bea5-d075-44c7-a026-e40d440e67a7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:44:38.659 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:44:38.659 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:44:38.659 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:44:38.820 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:44:38.834 20 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'0cc56aaf-a660-4a2d-bd1e-955746045e98' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:44:38.840 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:44:38.840 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:44:38.841 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:44:39.055 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:44:39.067 18 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'518417dc-b941-4c6e-aae8-10988b15be26' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:44:39.073 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:44:39.073 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:44:39.073 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:44:39.184 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:44:39.197 16 DEBUG aodh.coordination [-] Members of group: {b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'518417dc-b941-4c6e-aae8-10988b15be26', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'f71381e2-8934-4f65-a238-d89d9292d45e', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c'}, Me: b'e3d9a634-a908-4928-acb3-e9e7558495d7' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:44:39.203 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:44:39.203 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:44:39.203 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:45:04.752 2 INFO cotyledon._service_manager [-] Caught SIGTERM signal, graceful exiting of master process
2026-06-10 08:45:04.854 14 INFO cotyledon._service [-] Caught SIGTERM signal, graceful exiting of service AlarmEvaluationService(0) [14]
2026-06-10 08:45:04.854 18 INFO cotyledon._service [-] Caught SIGTERM signal, graceful exiting of service AlarmEvaluationService(2) [18]
2026-06-10 08:45:04.854 20 INFO cotyledon._service [-] Caught SIGTERM signal, graceful exiting of service AlarmEvaluationService(3) [20]
2026-06-10 08:45:04.855 16 INFO cotyledon._service [-] Caught SIGTERM signal, graceful exiting of service AlarmEvaluationService(1) [16]
2026-06-10 08:45:04.855 18 INFO aodh.coordination [-] Left partitioning group alarm_evaluator
2026-06-10 08:45:04.855 14 INFO aodh.coordination [-] Left partitioning group alarm_evaluator
2026-06-10 08:45:04.855 20 INFO aodh.coordination [-] Left partitioning group alarm_evaluator
2026-06-10 08:45:04.856 16 INFO aodh.coordination [-] Left partitioning group alarm_evaluator