2026-06-10 08:27:40.257 18 DEBUG aodh.storage [-] looking for 'mysql+pymysql' driver in 'aodh.storage' get_connection_from_config /usr/lib/python3.9/site-packages/aodh/storage/__init__.py:60
2026-06-10 08:27:40.403 18 INFO aodh.coordination [-] Coordination backend started successfully.
2026-06-10 08:27:40.424 14 DEBUG aodh.storage [-] looking for 'mysql+pymysql' driver in 'aodh.storage' get_connection_from_config /usr/lib/python3.9/site-packages/aodh/storage/__init__.py:60
2026-06-10 08:27:40.456 18 INFO aodh.coordination [-] Joined partitioning group alarm_evaluator
2026-06-10 08:27:40.597 16 DEBUG aodh.storage [-] looking for 'mysql+pymysql' driver in 'aodh.storage' get_connection_from_config /usr/lib/python3.9/site-packages/aodh/storage/__init__.py:60
2026-06-10 08:27:40.622 20 DEBUG aodh.storage [-] looking for 'mysql+pymysql' driver in 'aodh.storage' get_connection_from_config /usr/lib/python3.9/site-packages/aodh/storage/__init__.py:60
2026-06-10 08:27:40.690 14 INFO aodh.coordination [-] Coordination backend started successfully.
2026-06-10 08:27:40.718 14 INFO aodh.coordination [-] Joined partitioning group alarm_evaluator
2026-06-10 08:27:40.904 20 INFO aodh.coordination [-] Coordination backend started successfully.
2026-06-10 08:27:40.924 20 INFO aodh.coordination [-] Joined partitioning group alarm_evaluator
2026-06-10 08:27:40.966 16 INFO aodh.coordination [-] Coordination backend started successfully.
2026-06-10 08:27:40.976 16 INFO aodh.coordination [-] Joined partitioning group alarm_evaluator
2026-06-10 08:28:40.464 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:28:40.500 18 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'6b0b6e6f-3527-43b7-896a-2707c2ed1552' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:28:40.507 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:28:40.508 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:28:40.508 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:28:40.726 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:28:40.765 14 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:28:40.772 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:28:40.773 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:28:40.774 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:28:40.931 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:28:40.975 20 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:28:40.982 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:28:40.984 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:28:40.984 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:28:40.995 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:28:41.038 16 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:28:41.042 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:28:41.042 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:28:41.042 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:29:40.460 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:29:40.470 18 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'6b0b6e6f-3527-43b7-896a-2707c2ed1552' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:29:40.475 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:29:40.475 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:29:40.475 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:29:40.720 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:29:40.728 14 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:29:40.731 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:29:40.731 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:29:40.731 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:29:40.926 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:29:40.935 20 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:29:40.941 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:29:40.942 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:29:40.942 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:29:40.978 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:29:40.989 16 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:29:40.995 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:29:40.996 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:29:40.996 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:30:40.460 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:30:40.470 18 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'6b0b6e6f-3527-43b7-896a-2707c2ed1552' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:30:40.474 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:30:40.474 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:30:40.474 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:30:40.721 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:30:40.736 14 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:30:40.746 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:30:40.746 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:30:40.746 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:30:40.926 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:30:40.936 20 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:30:40.943 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:30:40.943 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:30:40.944 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:30:40.978 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:30:40.994 16 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:30:40.998 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:30:40.998 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:30:40.998 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:31:40.461 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:31:40.470 18 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'6b0b6e6f-3527-43b7-896a-2707c2ed1552' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:31:40.477 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:31:40.477 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:31:40.477 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:31:40.721 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:31:40.731 14 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:31:40.738 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:31:40.738 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:31:40.738 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:31:40.926 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:31:40.938 20 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:31:40.945 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:31:40.945 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:31:40.945 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:31:40.979 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:31:40.988 16 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:31:40.994 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:31:40.994 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:31:40.995 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:32:40.461 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:32:40.470 18 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'6b0b6e6f-3527-43b7-896a-2707c2ed1552' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:32:40.477 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:32:40.478 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:32:40.478 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:32:40.721 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:32:40.729 14 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:32:40.732 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:32:40.732 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:32:40.732 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:32:40.927 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:32:40.937 20 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:32:40.944 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:32:40.944 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:32:40.945 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:32:40.979 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:32:40.998 16 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:32:41.006 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:32:41.006 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:32:41.006 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:33:40.462 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:33:40.476 18 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'6b0b6e6f-3527-43b7-896a-2707c2ed1552' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:33:40.481 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:33:40.481 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:33:40.481 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:33:40.722 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:33:40.732 14 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:33:40.739 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:33:40.740 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:33:40.740 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:33:40.927 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:33:40.938 20 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:33:40.944 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:33:40.944 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:33:40.944 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:33:40.979 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:33:40.989 16 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:33:40.996 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:33:40.996 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:33:40.997 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:34:40.462 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:34:40.473 18 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'6b0b6e6f-3527-43b7-896a-2707c2ed1552' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:34:40.479 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:34:40.480 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:34:40.480 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:34:40.722 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:34:40.732 14 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:34:40.738 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:34:40.739 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:34:40.739 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:34:40.928 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:34:40.938 20 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:34:40.944 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:34:40.944 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:34:40.944 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:34:40.980 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:34:40.992 16 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:34:40.998 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:34:40.999 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:34:40.999 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:35:40.463 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:35:40.475 18 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'6b0b6e6f-3527-43b7-896a-2707c2ed1552' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:35:40.481 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:35:40.482 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:35:40.482 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:35:40.723 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:35:40.734 14 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:35:40.741 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:35:40.741 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:35:40.741 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:35:40.928 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:35:40.937 20 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:35:40.941 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:35:40.941 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:35:40.941 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:35:40.980 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:35:40.991 16 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:35:40.998 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:35:40.998 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:35:40.998 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:36:40.463 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:36:40.474 18 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'6b0b6e6f-3527-43b7-896a-2707c2ed1552' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:36:40.478 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:36:40.478 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:36:40.479 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:36:40.723 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:36:40.731 14 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:36:40.735 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:36:40.735 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:36:40.736 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:36:40.929 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:36:40.940 20 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:36:40.946 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:36:40.946 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:36:40.946 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:36:40.981 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:36:40.991 16 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:36:40.997 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:36:40.998 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:36:40.998 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:37:40.463 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:37:40.477 18 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'6b0b6e6f-3527-43b7-896a-2707c2ed1552' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:37:40.483 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:37:40.484 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:37:40.484 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:37:40.723 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:37:40.733 14 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:37:40.740 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:37:40.740 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:37:40.740 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:37:40.929 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:37:40.939 20 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:37:40.945 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:37:40.945 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:37:40.946 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:37:40.981 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:37:40.996 16 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:37:41.002 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:37:41.002 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:37:41.003 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:38:40.464 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:38:40.516 18 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'6b0b6e6f-3527-43b7-896a-2707c2ed1552' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:38:40.519 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:38:40.519 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:38:40.519 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:38:40.724 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:38:40.736 14 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:38:40.743 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:38:40.743 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:38:40.743 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:38:40.929 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:38:40.940 20 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:38:40.948 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:38:40.949 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:38:40.949 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:38:40.981 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:38:40.993 16 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:38:40.999 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:38:40.999 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:38:40.999 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:39:26.445 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.446 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.446 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.447 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.447 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.447 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.448 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.448 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.448 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.449 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.449 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.449 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.450 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.450 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.450 20 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:26.450 20 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:26.450 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:26.450 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:26.450 20 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:26.450 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:26.450 20 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:26.450 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:26.450 20 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:26.450 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:26.450 20 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:26.450 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:26.450 20 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:26.450 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:26.450 20 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:26.450 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:26.450 20 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:26.450 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:26.450 20 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:26.450 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:26.450 20 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:26.450 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:26.450 20 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:26.450 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:26.450 20 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:26.450 20 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:26.450 20 ERROR aodh.coordination 
2026-06-10 08:39:26.450 20 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:26.450 20 ERROR aodh.coordination 
2026-06-10 08:39:26.450 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:26.450 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:26.450 20 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:26.450 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:26.450 20 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:26.450 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:26.450 20 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:26.450 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:26.450 20 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:26.450 20 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:26.450 20 ERROR aodh.coordination 
2026-06-10 08:39:26.750 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.750 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.751 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.751 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.751 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.752 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.752 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.752 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.752 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.753 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.753 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.753 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.754 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.754 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:26.754 20 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:26.754 20 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:26.754 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:26.754 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:26.754 20 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:26.754 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:26.754 20 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:26.754 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:26.754 20 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:26.754 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:26.754 20 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:26.754 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:26.754 20 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:26.754 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:26.754 20 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:26.754 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:26.754 20 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:26.754 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:26.754 20 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:26.754 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:26.754 20 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:26.754 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:26.754 20 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:26.754 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:26.754 20 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:26.754 20 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:26.754 20 ERROR aodh.coordination 
2026-06-10 08:39:26.754 20 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:26.754 20 ERROR aodh.coordination 
2026-06-10 08:39:26.754 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:26.754 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:26.754 20 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:26.754 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:26.754 20 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:26.754 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:26.754 20 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:26.754 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:26.754 20 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:26.754 20 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:26.754 20 ERROR aodh.coordination 
2026-06-10 08:39:27.750 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.751 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.751 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.752 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.752 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.752 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.753 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.753 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.753 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.753 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.754 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.754 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.754 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.755 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.755 20 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:27.755 20 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:27.755 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:27.755 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:27.755 20 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:27.755 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:27.755 20 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:27.755 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:27.755 20 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:27.755 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:27.755 20 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:27.755 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:27.755 20 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:27.755 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:27.755 20 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:27.755 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:27.755 20 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:27.755 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:27.755 20 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:27.755 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:27.755 20 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:27.755 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:27.755 20 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:27.755 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:27.755 20 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:27.755 20 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:27.755 20 ERROR aodh.coordination 
2026-06-10 08:39:27.755 20 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:27.755 20 ERROR aodh.coordination 
2026-06-10 08:39:27.755 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:27.755 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:27.755 20 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:27.755 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:27.755 20 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:27.755 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:27.755 20 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:27.755 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:27.755 20 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:27.755 20 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:27.755 20 ERROR aodh.coordination 
2026-06-10 08:39:27.987 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.991 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.992 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.992 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.993 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.993 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.994 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.994 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.995 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.995 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.995 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.996 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.999 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:27.999 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.000 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.000 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.000 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.000 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.001 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.001 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.001 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.001 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.001 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.002 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.002 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.002 14 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:28.002 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.004 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.004 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.005 16 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:28.002 14 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:28.002 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:28.002 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:28.002 14 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:28.002 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:28.002 14 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:28.002 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:28.002 14 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:28.002 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:28.002 14 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:28.002 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:28.002 14 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:28.002 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:28.002 14 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:28.002 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:28.002 14 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:28.002 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:28.002 14 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:28.002 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:28.002 14 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:28.002 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:28.002 14 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:28.002 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:28.002 14 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:28.002 14 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:28.002 14 ERROR aodh.coordination 
2026-06-10 08:39:28.002 14 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:28.002 14 ERROR aodh.coordination 
2026-06-10 08:39:28.002 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:28.002 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:28.002 14 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:28.002 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:28.002 14 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:28.002 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:28.002 14 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:28.002 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:28.002 14 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:28.002 14 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:28.002 14 ERROR aodh.coordination 
2026-06-10 08:39:28.005 16 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:28.005 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:28.005 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:28.005 16 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:28.005 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:28.005 16 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:28.005 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:28.005 16 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:28.005 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:28.005 16 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:28.005 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:28.005 16 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:28.005 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:28.005 16 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:28.005 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:28.005 16 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:28.005 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:28.005 16 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:28.005 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:28.005 16 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:28.005 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:28.005 16 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:28.005 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:28.005 16 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:28.005 16 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:28.005 16 ERROR aodh.coordination 
2026-06-10 08:39:28.005 16 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:28.005 16 ERROR aodh.coordination 
2026-06-10 08:39:28.005 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:28.005 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:28.005 16 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:28.005 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:28.005 16 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:28.005 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:28.005 16 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:28.005 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:28.005 16 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:28.005 16 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:28.005 16 ERROR aodh.coordination 
2026-06-10 08:39:28.341 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.352 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.352 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.356 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.356 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.357 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.357 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.359 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.360 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.361 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.361 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.361 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.362 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.363 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.363 18 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:28.363 18 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:28.363 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:28.363 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:28.363 18 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:28.363 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:28.363 18 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:28.363 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:28.363 18 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:28.363 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:28.363 18 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:28.363 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:28.363 18 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:28.363 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:28.363 18 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:28.363 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:28.363 18 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:28.363 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:28.363 18 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:28.363 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:28.363 18 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:28.363 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:28.363 18 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:28.363 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:28.363 18 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:28.363 18 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:28.363 18 ERROR aodh.coordination 
2026-06-10 08:39:28.363 18 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:28.363 18 ERROR aodh.coordination 
2026-06-10 08:39:28.363 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:28.363 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:28.363 18 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:28.363 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:28.363 18 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:28.363 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:28.363 18 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:28.363 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:28.363 18 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:28.363 18 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:28.363 18 ERROR aodh.coordination 
2026-06-10 08:39:28.996 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.997 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.997 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.998 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.998 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.998 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.999 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.999 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.999 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:28.999 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.000 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.000 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.000 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.000 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.001 16 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:29.001 16 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.001 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.001 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:29.001 16 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:29.001 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:29.001 16 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:29.001 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:29.001 16 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:29.001 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:29.001 16 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:29.001 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:29.001 16 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:29.001 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:29.001 16 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:29.001 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:29.001 16 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:29.001 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:29.001 16 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:29.001 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:29.001 16 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:29.001 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:29.001 16 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:29.001 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:29.001 16 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:29.001 16 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:29.001 16 ERROR aodh.coordination 
2026-06-10 08:39:29.001 16 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:29.001 16 ERROR aodh.coordination 
2026-06-10 08:39:29.001 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.001 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:29.001 16 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:29.001 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:29.001 16 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:29.001 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:29.001 16 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:29.001 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:29.001 16 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:29.001 16 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.001 16 ERROR aodh.coordination 
2026-06-10 08:39:29.811 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.812 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.812 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.813 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.814 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.824 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.829 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.830 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.829 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.830 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.831 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.831 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.833 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.835 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.835 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.840 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.841 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.845 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.845 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.846 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.846 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.847 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.847 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.847 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.847 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.849 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.853 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.855 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.859 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.860 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.861 20 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:29.861 20 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.861 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.861 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:29.861 20 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:29.861 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:29.861 20 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:29.861 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:29.861 20 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:29.861 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:29.861 20 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:29.861 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:29.861 20 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:29.861 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:29.861 20 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:29.861 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:29.861 20 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:29.861 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:29.861 20 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:29.861 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:29.861 20 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:29.861 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:29.861 20 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:29.861 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:29.861 20 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:29.861 20 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:29.861 20 ERROR aodh.coordination 
2026-06-10 08:39:29.861 20 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:29.861 20 ERROR aodh.coordination 
2026-06-10 08:39:29.861 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.861 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:29.861 20 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:29.861 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:29.861 20 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:29.861 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:29.861 20 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:29.861 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:29.861 20 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:29.861 20 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.861 20 ERROR aodh.coordination 
2026-06-10 08:39:29.862 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.864 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.865 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.865 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.872 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.872 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.873 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.874 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.875 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.875 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.880 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.881 14 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:29.881 14 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.881 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.881 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:29.881 14 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:29.881 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:29.881 14 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:29.881 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:29.881 14 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:29.881 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:29.881 14 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:29.881 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:29.881 14 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:29.881 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:29.881 14 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:29.881 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:29.881 14 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:29.881 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:29.881 14 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:29.881 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:29.881 14 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:29.881 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:29.881 14 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:29.881 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:29.881 14 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:29.881 14 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:29.881 14 ERROR aodh.coordination 
2026-06-10 08:39:29.881 14 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:29.881 14 ERROR aodh.coordination 
2026-06-10 08:39:29.881 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.881 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:29.881 14 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:29.881 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:29.881 14 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:29.881 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:29.881 14 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:29.881 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:29.881 14 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:29.881 14 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.881 14 ERROR aodh.coordination 
2026-06-10 08:39:29.882 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.882 18 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:29.882 18 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.882 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.882 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:29.882 18 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:29.882 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:29.882 18 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:29.882 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:29.882 18 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:29.882 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:29.882 18 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:29.882 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:29.882 18 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:29.882 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:29.882 18 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:29.882 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:29.882 18 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:29.882 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:29.882 18 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:29.882 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:29.882 18 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:29.882 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:29.882 18 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:29.882 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:29.882 18 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:29.882 18 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:29.882 18 ERROR aodh.coordination 
2026-06-10 08:39:29.882 18 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:29.882 18 ERROR aodh.coordination 
2026-06-10 08:39:29.882 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.882 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:29.882 18 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:29.882 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:29.882 18 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:29.882 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:29.882 18 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:29.882 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:29.882 18 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:29.882 18 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.882 18 ERROR aodh.coordination 
2026-06-10 08:39:29.987 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.988 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.988 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.988 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.988 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.989 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.989 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.989 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.990 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.990 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.990 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.990 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.991 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.991 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:29.991 16 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:29.991 16 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.991 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.991 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:29.991 16 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:29.991 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:29.991 16 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:29.991 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:29.991 16 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:29.991 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:29.991 16 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:29.991 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:29.991 16 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:29.991 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:29.991 16 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:29.991 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:29.991 16 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:29.991 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:29.991 16 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:29.991 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:29.991 16 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:29.991 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:29.991 16 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:29.991 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:29.991 16 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:29.991 16 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:29.991 16 ERROR aodh.coordination 
2026-06-10 08:39:29.991 16 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:29.991 16 ERROR aodh.coordination 
2026-06-10 08:39:29.991 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:29.991 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:29.991 16 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:29.991 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:29.991 16 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:29.991 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:29.991 16 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:29.991 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:29.991 16 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:29.991 16 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:29.991 16 ERROR aodh.coordination 
2026-06-10 08:39:30.808 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.809 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.809 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.809 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.809 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.809 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.809 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.809 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.810 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.810 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.810 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.810 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.810 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.810 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.810 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.810 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.810 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.811 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.811 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.811 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.811 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.811 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.811 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.811 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.811 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.811 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.812 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.812 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.812 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.812 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.812 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.812 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.813 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.813 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.813 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.813 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.813 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.813 14 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:30.813 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.814 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.814 18 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:30.814 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.814 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.814 14 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:30.814 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:30.814 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:30.814 14 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:30.814 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:30.814 14 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:30.814 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:30.814 14 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:30.814 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:30.814 14 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:30.814 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:30.814 14 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:30.814 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:30.814 14 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:30.814 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:30.814 14 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:30.814 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:30.814 14 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:30.814 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:30.814 14 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:30.814 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:30.814 14 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:30.814 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:30.814 14 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:30.814 14 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:30.814 14 ERROR aodh.coordination 
2026-06-10 08:39:30.814 14 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:30.814 14 ERROR aodh.coordination 
2026-06-10 08:39:30.814 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:30.814 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:30.814 14 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:30.814 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:30.814 14 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:30.814 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:30.814 14 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:30.814 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:30.814 14 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:30.814 14 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:30.814 14 ERROR aodh.coordination 
2026-06-10 08:39:30.814 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:30.815 20 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:30.814 18 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:30.814 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:30.814 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:30.814 18 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:30.814 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:30.814 18 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:30.814 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:30.814 18 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:30.814 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:30.814 18 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:30.814 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:30.814 18 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:30.814 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:30.814 18 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:30.814 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:30.814 18 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:30.814 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:30.814 18 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:30.814 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:30.814 18 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:30.814 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:30.814 18 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:30.814 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:30.814 18 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:30.814 18 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:30.814 18 ERROR aodh.coordination 
2026-06-10 08:39:30.814 18 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:30.814 18 ERROR aodh.coordination 
2026-06-10 08:39:30.814 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:30.814 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:30.814 18 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:30.814 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:30.814 18 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:30.814 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:30.814 18 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:30.814 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:30.814 18 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:30.814 18 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:30.814 18 ERROR aodh.coordination 
2026-06-10 08:39:30.815 20 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:30.815 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:30.815 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:30.815 20 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:30.815 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:30.815 20 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:30.815 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:30.815 20 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:30.815 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:30.815 20 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:30.815 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:30.815 20 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:30.815 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:30.815 20 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:30.815 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:30.815 20 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:30.815 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:30.815 20 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:30.815 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:30.815 20 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:30.815 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:30.815 20 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:30.815 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:30.815 20 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:30.815 20 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:30.815 20 ERROR aodh.coordination 
2026-06-10 08:39:30.815 20 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:30.815 20 ERROR aodh.coordination 
2026-06-10 08:39:30.815 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:30.815 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:30.815 20 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:30.815 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:30.815 20 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:30.815 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:30.815 20 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:30.815 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:30.815 20 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:30.815 20 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:30.815 20 ERROR aodh.coordination 
2026-06-10 08:39:31.809 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:31.809 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:31.809 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:31.810 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:31.810 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:31.810 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:31.810 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:31.810 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:31.811 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:31.811 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:31.811 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:31.811 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:31.811 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:31.812 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:31.813 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:31.813 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:31.813 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:31.814 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:31.816 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:31.817 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:31.817 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:31.818 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:31.818 18 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:31.818 18 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:31.818 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:31.818 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:31.818 18 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:31.818 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:31.818 18 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:31.818 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:31.818 18 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:31.818 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:31.818 18 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:31.818 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:31.818 18 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:31.818 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:31.818 18 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:31.818 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:31.818 18 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:31.818 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:31.818 18 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:31.818 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:31.818 18 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:31.818 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:31.818 18 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:31.818 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:31.818 18 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:31.818 18 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:31.818 18 ERROR aodh.coordination 
2026-06-10 08:39:31.818 18 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:31.818 18 ERROR aodh.coordination 
2026-06-10 08:39:31.818 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:31.818 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:31.818 18 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:31.818 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:31.818 18 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:31.818 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:31.818 18 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:31.818 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:31.818 18 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:31.818 18 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:31.818 18 ERROR aodh.coordination 
2026-06-10 08:39:32.810 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:32.810 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:32.811 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:32.811 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:32.811 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:32.812 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:32.812 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:32.812 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:32.813 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:32.813 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:32.813 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:32.814 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:32.814 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:32.814 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:32.815 18 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:32.815 18 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:32.815 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:32.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:32.815 18 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:32.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:32.815 18 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:32.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:32.815 18 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:32.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:32.815 18 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:32.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:32.815 18 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:32.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:32.815 18 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:32.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:32.815 18 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:32.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:32.815 18 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:32.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:32.815 18 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:32.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:32.815 18 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:32.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:32.815 18 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:32.815 18 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:32.815 18 ERROR aodh.coordination 
2026-06-10 08:39:32.815 18 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:32.815 18 ERROR aodh.coordination 
2026-06-10 08:39:32.815 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:32.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:32.815 18 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:32.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:32.815 18 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:32.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:32.815 18 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:32.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:32.815 18 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:32.815 18 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:32.815 18 ERROR aodh.coordination 
2026-06-10 08:39:33.810 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:33.811 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:33.811 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:33.812 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:33.812 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:33.812 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:33.813 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:33.813 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:33.814 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:33.814 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:33.815 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:33.815 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:33.815 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:33.816 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:33.816 18 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:33.816 18 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:33.816 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:33.816 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:33.816 18 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:33.816 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:33.816 18 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:33.816 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:33.816 18 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:33.816 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:33.816 18 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:33.816 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:33.816 18 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:33.816 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:33.816 18 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:33.816 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:33.816 18 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:33.816 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:33.816 18 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:33.816 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:33.816 18 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:33.816 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:33.816 18 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:33.816 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:33.816 18 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:33.816 18 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:33.816 18 ERROR aodh.coordination 
2026-06-10 08:39:33.816 18 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:33.816 18 ERROR aodh.coordination 
2026-06-10 08:39:33.816 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:33.816 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:33.816 18 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:33.816 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:33.816 18 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:33.816 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:33.816 18 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:33.816 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:33.816 18 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:33.816 18 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:33.816 18 ERROR aodh.coordination 
2026-06-10 08:39:34.561 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.563 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.563 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.563 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.564 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.564 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.565 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.565 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.567 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.568 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.569 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.569 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.570 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.570 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.570 20 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:34.571 20 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.571 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.571 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:34.571 20 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:34.571 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:34.571 20 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:34.571 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:34.571 20 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:34.571 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:34.571 20 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:34.571 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:34.571 20 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:34.571 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:34.571 20 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:34.571 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:34.571 20 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:34.571 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:34.571 20 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:34.571 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:34.571 20 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:34.571 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:34.571 20 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:34.571 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:34.571 20 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:34.571 20 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:34.571 20 ERROR aodh.coordination 
2026-06-10 08:39:34.571 20 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:34.571 20 ERROR aodh.coordination 
2026-06-10 08:39:34.571 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.571 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:34.571 20 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:34.571 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:34.571 20 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:34.571 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:34.571 20 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:34.571 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:34.571 20 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:34.571 20 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.571 20 ERROR aodh.coordination 
2026-06-10 08:39:34.572 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.573 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.573 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.574 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.574 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.574 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.575 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.575 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.575 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.575 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.576 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.576 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.576 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.577 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.577 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.577 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.577 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.579 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.579 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.580 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.580 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.580 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.581 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.580 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.581 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.581 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.583 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.583 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.583 16 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:34.583 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.584 16 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.584 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.584 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:34.584 16 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:34.584 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:34.584 16 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:34.584 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:34.584 16 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:34.584 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:34.584 16 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:34.584 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:34.584 16 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:34.584 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:34.584 16 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:34.584 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:34.584 16 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:34.584 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:34.584 16 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:34.584 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:34.584 16 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:34.584 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:34.584 16 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:34.584 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:34.584 16 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:34.584 16 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:34.584 16 ERROR aodh.coordination 
2026-06-10 08:39:34.584 16 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:34.584 16 ERROR aodh.coordination 
2026-06-10 08:39:34.584 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.584 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:34.584 16 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:34.584 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:34.584 16 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:34.584 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:34.584 16 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:34.584 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:34.584 16 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:34.584 16 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.584 16 ERROR aodh.coordination 
2026-06-10 08:39:34.585 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.585 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.586 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.586 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.587 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.587 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.589 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.590 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.590 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.591 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.591 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.591 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.594 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.591 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.596 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.596 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.597 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.597 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.599 16 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:34.599 16 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.599 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.599 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:34.599 16 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:34.599 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:34.599 16 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:34.599 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:34.599 16 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:34.599 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:34.599 16 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:34.599 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:34.599 16 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:34.599 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:34.599 16 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:34.599 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:34.599 16 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:34.599 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:34.599 16 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:34.599 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:34.599 16 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:34.599 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:34.599 16 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:34.599 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:34.599 16 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:34.599 16 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:34.599 16 ERROR aodh.coordination 
2026-06-10 08:39:34.599 16 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:34.599 16 ERROR aodh.coordination 
2026-06-10 08:39:34.599 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.599 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:34.599 16 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:34.599 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:34.599 16 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:34.599 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:34.599 16 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:34.599 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:34.599 16 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:34.599 16 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.599 16 ERROR aodh.coordination 
2026-06-10 08:39:34.601 20 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:34.601 20 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.601 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.601 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:34.601 20 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:34.601 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:34.601 20 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:34.601 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:34.601 20 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:34.601 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:34.601 20 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:34.601 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:34.601 20 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:34.601 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:34.601 20 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:34.601 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:34.601 20 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:34.601 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:34.601 20 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:34.601 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:34.601 20 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:34.601 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:34.601 20 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:34.601 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:34.601 20 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:34.601 20 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:34.601 20 ERROR aodh.coordination 
2026-06-10 08:39:34.601 20 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:34.601 20 ERROR aodh.coordination 
2026-06-10 08:39:34.601 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.601 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:34.601 20 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:34.601 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:34.601 20 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:34.601 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:34.601 20 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:34.601 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:34.601 20 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:34.601 20 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.601 20 ERROR aodh.coordination 
2026-06-10 08:39:34.601 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.603 14 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:34.603 14 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.603 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.603 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:34.603 14 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:34.603 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:34.603 14 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:34.603 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:34.603 14 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:34.603 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:34.603 14 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:34.603 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:34.603 14 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:34.603 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:34.603 14 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:34.603 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:34.603 14 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:34.603 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:34.603 14 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:34.603 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:34.603 14 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:34.603 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:34.603 14 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:34.603 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:34.603 14 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:34.603 14 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:34.603 14 ERROR aodh.coordination 
2026-06-10 08:39:34.603 14 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:34.603 14 ERROR aodh.coordination 
2026-06-10 08:39:34.603 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.603 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:34.603 14 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:34.603 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:34.603 14 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:34.603 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:34.603 14 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:34.603 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:34.603 14 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:34.603 14 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.603 14 ERROR aodh.coordination 
2026-06-10 08:39:34.612 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.614 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.615 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.616 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.617 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.617 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.618 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.619 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.619 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.620 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.621 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.627 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.628 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.628 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.629 14 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:34.629 14 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.629 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.629 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:34.629 14 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:34.629 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:34.629 14 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:34.629 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:34.629 14 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:34.629 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:34.629 14 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:34.629 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:34.629 14 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:34.629 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:34.629 14 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:34.629 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:34.629 14 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:34.629 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:34.629 14 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:34.629 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:34.629 14 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:34.629 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:34.629 14 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:34.629 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:34.629 14 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:34.629 14 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:34.629 14 ERROR aodh.coordination 
2026-06-10 08:39:34.629 14 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:34.629 14 ERROR aodh.coordination 
2026-06-10 08:39:34.629 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.629 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:34.629 14 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:34.629 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:34.629 14 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:34.629 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:34.629 14 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:34.629 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:34.629 14 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:34.629 14 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.629 14 ERROR aodh.coordination 
2026-06-10 08:39:34.810 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.811 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.811 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.811 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.812 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.812 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.812 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.812 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.813 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.813 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.813 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.814 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.814 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.814 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:34.814 18 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:34.815 18 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.815 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:34.815 18 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:34.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:34.815 18 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:34.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:34.815 18 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:34.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:34.815 18 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:34.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:34.815 18 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:34.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:34.815 18 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:34.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:34.815 18 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:34.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:34.815 18 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:34.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:34.815 18 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:34.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:34.815 18 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:34.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:34.815 18 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:34.815 18 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:34.815 18 ERROR aodh.coordination 
2026-06-10 08:39:34.815 18 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:34.815 18 ERROR aodh.coordination 
2026-06-10 08:39:34.815 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:34.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:34.815 18 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:34.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:34.815 18 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:34.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:34.815 18 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:34.815 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:34.815 18 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:34.815 18 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:34.815 18 ERROR aodh.coordination 
2026-06-10 08:39:35.573 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.575 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.576 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.576 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.576 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.577 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.577 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.577 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.578 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.578 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.578 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.579 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.579 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.579 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.580 20 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:35.580 20 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:35.580 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:35.580 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:35.580 20 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:35.580 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:35.580 20 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:35.580 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:35.580 20 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:35.580 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:35.580 20 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:35.580 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:35.580 20 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:35.580 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:35.580 20 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:35.580 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:35.580 20 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:35.580 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:35.580 20 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:35.580 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:35.580 20 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:35.580 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:35.580 20 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:35.580 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:35.580 20 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:35.580 20 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:35.580 20 ERROR aodh.coordination 
2026-06-10 08:39:35.580 20 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:35.580 20 ERROR aodh.coordination 
2026-06-10 08:39:35.580 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:35.580 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:35.580 20 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:35.580 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:35.580 20 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:35.580 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:35.580 20 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:35.580 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:35.580 20 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:35.580 20 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:35.580 20 ERROR aodh.coordination 
2026-06-10 08:39:35.585 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.586 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.586 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.586 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.586 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.587 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.587 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.588 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.588 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.588 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.589 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.590 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.592 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.593 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.594 16 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:35.594 16 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:35.594 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:35.594 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:35.594 16 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:35.594 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:35.594 16 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:35.594 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:35.594 16 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:35.594 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:35.594 16 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:35.594 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:35.594 16 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:35.594 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:35.594 16 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:35.594 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:35.594 16 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:35.594 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:35.594 16 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:35.594 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:35.594 16 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:35.594 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:35.594 16 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:35.594 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:35.594 16 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:35.594 16 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:35.594 16 ERROR aodh.coordination 
2026-06-10 08:39:35.594 16 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:35.594 16 ERROR aodh.coordination 
2026-06-10 08:39:35.594 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:35.594 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:35.594 16 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:35.594 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:35.594 16 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:35.594 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:35.594 16 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:35.594 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:35.594 16 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:35.594 16 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:35.594 16 ERROR aodh.coordination 
2026-06-10 08:39:35.612 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.612 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.613 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.613 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.614 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.614 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.615 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.615 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.615 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.616 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.616 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.617 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.617 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.617 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.618 14 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:35.618 14 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:35.618 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:35.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:35.618 14 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:35.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:35.618 14 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:35.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:35.618 14 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:35.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:35.618 14 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:35.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:35.618 14 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:35.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:35.618 14 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:35.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:35.618 14 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:35.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:35.618 14 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:35.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:35.618 14 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:35.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:35.618 14 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:35.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:35.618 14 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:35.618 14 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:35.618 14 ERROR aodh.coordination 
2026-06-10 08:39:35.618 14 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:35.618 14 ERROR aodh.coordination 
2026-06-10 08:39:35.618 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:35.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:35.618 14 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:35.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:35.618 14 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:35.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:35.618 14 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:35.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:35.618 14 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:35.618 14 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:35.618 14 ERROR aodh.coordination 
2026-06-10 08:39:35.861 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.862 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.862 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.862 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.863 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.863 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.863 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.864 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.864 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.864 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.865 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.865 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.865 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.866 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:35.866 18 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:35.866 18 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:35.866 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:35.866 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:35.866 18 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:35.866 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:35.866 18 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:35.866 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:35.866 18 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:35.866 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:35.866 18 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:35.866 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:35.866 18 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:35.866 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:35.866 18 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:35.866 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:35.866 18 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:35.866 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:35.866 18 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:35.866 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:35.866 18 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:35.866 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:35.866 18 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:35.866 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:35.866 18 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:35.866 18 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:35.866 18 ERROR aodh.coordination 
2026-06-10 08:39:35.866 18 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:35.866 18 ERROR aodh.coordination 
2026-06-10 08:39:35.866 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:35.866 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:35.866 18 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:35.866 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:35.866 18 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:35.866 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:35.866 18 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:35.866 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:35.866 18 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:35.866 18 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:35.866 18 ERROR aodh.coordination 
2026-06-10 08:39:36.573 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.578 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.578 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.579 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.581 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.582 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.583 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.584 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.584 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.585 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.585 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.586 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.586 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.586 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.586 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.586 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.587 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.587 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.587 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.587 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.588 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.588 20 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.588 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.588 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.588 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.588 20 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:36.589 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.589 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.590 16 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.589 20 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.589 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.589 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:36.589 20 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:36.589 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:36.589 20 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:36.589 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:36.589 20 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:36.589 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:36.589 20 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:36.589 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:36.589 20 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:36.589 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:36.589 20 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:36.589 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:36.589 20 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:36.589 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:36.589 20 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:36.589 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:36.589 20 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:36.589 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:36.589 20 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:36.589 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:36.589 20 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:36.589 20 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:36.589 20 ERROR aodh.coordination 
2026-06-10 08:39:36.589 20 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:36.589 20 ERROR aodh.coordination 
2026-06-10 08:39:36.589 20 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.589 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:36.589 20 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:36.589 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:36.589 20 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:36.589 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:36.589 20 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:36.589 20 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:36.589 20 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:36.589 20 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.589 20 ERROR aodh.coordination 
2026-06-10 08:39:36.590 16 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:36.590 16 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.590 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.590 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:36.590 16 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:36.590 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:36.590 16 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:36.590 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:36.590 16 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:36.590 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:36.590 16 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:36.590 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:36.590 16 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:36.590 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:36.590 16 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:36.590 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:36.590 16 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:36.590 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:36.590 16 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:36.590 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:36.590 16 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:36.590 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:36.590 16 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:36.590 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:36.590 16 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:36.590 16 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:36.590 16 ERROR aodh.coordination 
2026-06-10 08:39:36.590 16 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:36.590 16 ERROR aodh.coordination 
2026-06-10 08:39:36.590 16 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.590 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:36.590 16 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:36.590 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:36.590 16 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:36.590 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:36.590 16 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:36.590 16 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:36.590 16 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:36.590 16 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.590 16 ERROR aodh.coordination 
2026-06-10 08:39:36.612 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.613 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.613 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.614 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.614 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.615 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.615 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.615 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.616 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.616 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.617 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.617 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.617 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.618 14 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.618 14 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:36.618 14 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.618 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:36.618 14 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:36.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:36.618 14 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:36.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:36.618 14 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:36.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:36.618 14 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:36.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:36.618 14 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:36.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:36.618 14 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:36.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:36.618 14 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:36.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:36.618 14 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:36.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:36.618 14 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:36.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:36.618 14 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:36.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:36.618 14 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:36.618 14 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:36.618 14 ERROR aodh.coordination 
2026-06-10 08:39:36.618 14 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:36.618 14 ERROR aodh.coordination 
2026-06-10 08:39:36.618 14 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:36.618 14 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:36.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:36.618 14 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:36.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:36.618 14 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:36.618 14 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:36.618 14 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:36.618 14 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.618 14 ERROR aodh.coordination 
2026-06-10 08:39:36.861 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.863 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.863 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.864 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.864 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.865 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.865 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.866 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.866 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.867 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.867 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.868 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.868 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.868 18 DEBUG tooz.drivers.redis [-] Redis connection error, will retry - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:65
2026-06-10 08:39:36.869 18 DEBUG tooz.drivers.redis [-] Redis connection error, retry limit has been reached, aborting - Connection closed by server. wrapper /usr/lib/python3.9/site-packages/tooz/drivers/redis.py:57
2026-06-10 08:39:36.869 18 ERROR aodh.coordination [-] Error sending a heartbeat to coordination backend.: tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.869 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.869 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 52, in wrapper
2026-06-10 08:39:36.869 18 ERROR aodh.coordination     return func(*args, **kwargs)
2026-06-10 08:39:36.869 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 539, in heartbeat
2026-06-10 08:39:36.869 18 ERROR aodh.coordination     self._client.psetex(beat_id, time_ms=expiry_ms,
2026-06-10 08:39:36.869 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 1733, in psetex
2026-06-10 08:39:36.869 18 ERROR aodh.coordination     return self.execute_command('PSETEX', name, time_ms, value)
2026-06-10 08:39:36.869 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
2026-06-10 08:39:36.869 18 ERROR aodh.coordination     conn = self.connection or pool.get_connection(command_name, **options)
2026-06-10 08:39:36.869 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
2026-06-10 08:39:36.869 18 ERROR aodh.coordination     connection.connect()
2026-06-10 08:39:36.869 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
2026-06-10 08:39:36.869 18 ERROR aodh.coordination     self.on_connect()
2026-06-10 08:39:36.869 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 643, in on_connect
2026-06-10 08:39:36.869 18 ERROR aodh.coordination     auth_response = self.read_response()
2026-06-10 08:39:36.869 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
2026-06-10 08:39:36.869 18 ERROR aodh.coordination     response = self._parser.read_response()
2026-06-10 08:39:36.869 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 324, in read_response
2026-06-10 08:39:36.869 18 ERROR aodh.coordination     raw = self._buffer.readline()
2026-06-10 08:39:36.869 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 256, in readline
2026-06-10 08:39:36.869 18 ERROR aodh.coordination     self._read_from_socket()
2026-06-10 08:39:36.869 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/redis/connection.py", line 201, in _read_from_socket
2026-06-10 08:39:36.869 18 ERROR aodh.coordination     raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
2026-06-10 08:39:36.869 18 ERROR aodh.coordination redis.exceptions.ConnectionError: Connection closed by server.
2026-06-10 08:39:36.869 18 ERROR aodh.coordination 
2026-06-10 08:39:36.869 18 ERROR aodh.coordination The above exception was the direct cause of the following exception:
2026-06-10 08:39:36.869 18 ERROR aodh.coordination 
2026-06-10 08:39:36.869 18 ERROR aodh.coordination Traceback (most recent call last):
2026-06-10 08:39:36.869 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/aodh/coordination.py", line 153, in heartbeat
2026-06-10 08:39:36.869 18 ERROR aodh.coordination     self._coordinator.heartbeat()
2026-06-10 08:39:36.869 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/drivers/redis.py", line 61, in wrapper
2026-06-10 08:39:36.869 18 ERROR aodh.coordination     utils.raise_with_cause(
2026-06-10 08:39:36.869 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/tooz/utils.py", line 224, in raise_with_cause
2026-06-10 08:39:36.869 18 ERROR aodh.coordination     excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
2026-06-10 08:39:36.869 18 ERROR aodh.coordination   File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 142, in raise_with_cause
2026-06-10 08:39:36.869 18 ERROR aodh.coordination     raise exc_cls(message, *args, **kwargs) from kwargs.get('cause')
2026-06-10 08:39:36.869 18 ERROR aodh.coordination tooz.coordination.ToozConnectionError: Connection closed by server.
2026-06-10 08:39:36.869 18 ERROR aodh.coordination 
2026-06-10 08:39:40.464 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:39:40.544 18 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'6b0b6e6f-3527-43b7-896a-2707c2ed1552' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:39:40.550 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:39:40.550 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:39:40.551 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:39:40.724 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:39:40.930 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:39:40.982 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines [-] Database connection was found disconnected; reconnecting: oslo_db.exception.DBConnectionError: (pymysql.err.OperationalError) (2006, "MySQL server has gone away (TimeoutError(110, 'Connection timed out'))")
[SQL: SELECT 1]
(Background on this error at: http://sqlalche.me/e/13/e3q8)
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines Traceback (most recent call last):
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 705, in _write_bytes
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     self._sock.sendall(data)
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines TimeoutError: [Errno 110] Connection timed out
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines 
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines During handling of the above exception, another exception occurred:
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines 
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines Traceback (most recent call last):
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     self.dialect.do_execute(
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     cursor.execute(statement, parameters)
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib/python3.9/site-packages/pymysql/cursors.py", line 163, in execute
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     result = self._query(query)
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib/python3.9/site-packages/pymysql/cursors.py", line 321, in _query
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     conn.query(q)
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 504, in query
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     self._execute_command(COMMAND.COM_QUERY, sql)
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 763, in _execute_command
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     self._write_bytes(packet)
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 708, in _write_bytes
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     raise err.OperationalError(
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines pymysql.err.OperationalError: (2006, "MySQL server has gone away (TimeoutError(110, 'Connection timed out'))")
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines 
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines The above exception was the direct cause of the following exception:
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines 
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines Traceback (most recent call last):
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib/python3.9/site-packages/oslo_db/sqlalchemy/engines.py", line 73, in _connect_ping_listener
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     connection.scalar(select([1]))
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 941, in scalar
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     return self.execute(object_, *multiparams, **params).scalar()
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     return meth(self, multiparams, params)
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib64/python3.9/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     return connection._execute_clauseelement(self, multiparams, params)
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     ret = self._execute_context(
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     self._handle_dbapi_exception(
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1508, in _handle_dbapi_exception
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     util.raise_(newraise, with_traceback=exc_info[2], from_=e)
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib64/python3.9/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     raise exception
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     self.dialect.do_execute(
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     cursor.execute(statement, parameters)
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib/python3.9/site-packages/pymysql/cursors.py", line 163, in execute
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     result = self._query(query)
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib/python3.9/site-packages/pymysql/cursors.py", line 321, in _query
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     conn.query(q)
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 504, in query
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     self._execute_command(COMMAND.COM_QUERY, sql)
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 763, in _execute_command
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     self._write_bytes(packet)
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines   File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 708, in _write_bytes
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines     raise err.OperationalError(
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines oslo_db.exception.DBConnectionError: (pymysql.err.OperationalError) (2006, "MySQL server has gone away (TimeoutError(110, 'Connection timed out'))")
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines [SQL: SELECT 1]
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines (Background on this error at: http://sqlalche.me/e/13/e3q8)
2026-06-10 08:39:40.984 16 ERROR oslo_db.sqlalchemy.engines 
2026-06-10 08:39:42.666 20 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:39:42.666 14 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:39:42.667 16 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:39:42.674 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:39:42.675 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:39:42.675 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:39:42.675 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:39:42.676 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:39:42.676 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:39:42.678 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:39:42.679 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:39:42.679 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:40:40.465 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:40:40.475 18 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'6b0b6e6f-3527-43b7-896a-2707c2ed1552' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:40:40.482 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:40:40.482 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:40:40.482 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:40:40.724 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:40:40.737 14 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:40:40.743 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:40:40.743 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:40:40.744 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:40:40.930 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:40:40.942 20 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:40:40.949 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:40:40.949 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:40:40.950 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:40:40.982 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:40:40.995 16 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:40:41.002 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:40:41.002 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:40:41.002 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:41:40.465 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:41:40.474 18 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'6b0b6e6f-3527-43b7-896a-2707c2ed1552' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:41:40.478 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:41:40.478 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:41:40.478 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:41:40.724 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:41:40.732 14 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:41:40.738 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:41:40.738 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:41:40.738 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:41:40.931 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:41:40.941 20 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:41:40.947 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:41:40.947 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:41:40.947 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:41:40.983 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:41:40.993 16 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:41:41.002 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:41:41.002 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:41:41.002 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:42:40.465 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:42:40.476 18 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'6b0b6e6f-3527-43b7-896a-2707c2ed1552' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:42:40.482 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:42:40.483 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:42:40.483 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:42:40.725 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:42:40.736 14 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:42:40.748 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:42:40.748 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:42:40.749 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:42:40.931 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:42:40.940 20 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:42:40.942 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:42:40.943 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:42:40.943 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:42:40.983 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:42:40.995 16 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:42:41.001 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:42:41.002 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:42:41.002 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:43:40.466 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:43:40.477 18 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'6b0b6e6f-3527-43b7-896a-2707c2ed1552' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:43:40.483 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:43:40.483 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:43:40.483 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:43:40.725 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:43:40.735 14 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:43:40.741 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:43:40.742 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:43:40.742 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:43:40.932 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:43:40.943 20 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:43:40.950 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:43:40.950 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:43:40.950 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:43:40.984 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:43:40.994 16 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:43:41.000 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:43:41.000 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:43:41.001 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:44:40.466 18 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:44:40.477 18 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'6b0b6e6f-3527-43b7-896a-2707c2ed1552' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:44:40.480 18 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:44:40.480 18 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:44:40.480 18 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:44:40.726 14 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:44:40.738 14 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:44:40.741 14 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:44:40.741 14 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:44:40.741 14 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:44:40.932 20 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:44:40.942 20 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:44:40.949 20 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:44:40.949 20 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:44:40.949 20 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:44:40.984 16 WARNING oslo_db.sqlalchemy.utils [-] Unique keys not in sort_keys. The sorting order may be unstable.
2026-06-10 08:44:40.996 16 DEBUG aodh.coordination [-] Members of group: {b'518417dc-b941-4c6e-aae8-10988b15be26', b'dd4db7e1-1f6c-45e6-8a3b-bf38d09704bb', b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2', b'6b0b6e6f-3527-43b7-896a-2707c2ed1552', b'e3d9a634-a908-4928-acb3-e9e7558495d7', b'f7d44190-b5f1-4c19-a56a-7db52fbb893c', b'4f56fb5e-0f27-474c-a05f-86dad1a10775', b'd8c5993f-a7ff-4504-83b7-53443b516f5a', b'0cc56aaf-a660-4a2d-bd1e-955746045e98', b'7a28bea5-d075-44c7-a026-e40d440e67a7', b'ff3a4ab9-5c89-4045-bc16-81bf417bc74c', b'f71381e2-8934-4f65-a238-d89d9292d45e'}, Me: b'3e0bf06a-c9f9-4d7a-8fe3-1117982879b2' extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:230
2026-06-10 08:44:41.003 16 DEBUG aodh.coordination [-] Universal set: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:240
2026-06-10 08:44:41.003 16 DEBUG aodh.coordination [-] My subset: [] extract_my_subset /usr/lib/python3.9/site-packages/aodh/coordination.py:243
2026-06-10 08:44:41.004 16 INFO aodh.evaluator [-] initiating evaluation cycle on 0 alarms
2026-06-10 08:45:05.964 2 INFO cotyledon._service_manager [-] Caught SIGTERM signal, graceful exiting of master process
2026-06-10 08:45:06.066 20 INFO cotyledon._service [-] Caught SIGTERM signal, graceful exiting of service AlarmEvaluationService(3) [20]
2026-06-10 08:45:06.066 16 INFO cotyledon._service [-] Caught SIGTERM signal, graceful exiting of service AlarmEvaluationService(1) [16]
2026-06-10 08:45:06.066 14 INFO cotyledon._service [-] Caught SIGTERM signal, graceful exiting of service AlarmEvaluationService(0) [14]
2026-06-10 08:45:06.066 18 INFO cotyledon._service [-] Caught SIGTERM signal, graceful exiting of service AlarmEvaluationService(2) [18]
2026-06-10 08:45:06.067 16 INFO aodh.coordination [-] Left partitioning group alarm_evaluator
2026-06-10 08:45:06.067 18 INFO aodh.coordination [-] Left partitioning group alarm_evaluator
2026-06-10 08:45:06.067 14 INFO aodh.coordination [-] Left partitioning group alarm_evaluator
2026-06-10 08:45:06.067 20 INFO aodh.coordination [-] Left partitioning group alarm_evaluator