2026-06-11T10:27:51.526351573-04:00 stderr F /usr/local/lib/python3.11/site-packages/nodepool/version.py:18: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
2026-06-11T10:27:51.526351573-04:00 stderr F   import pkg_resources
2026-06-11T10:27:51.972129549-04:00 stdout F 2026-06-11 14:27:51,971 INFO nodepool.NodePool: Nodepool launcher 11.0.1.dev1 8472a37a starting
2026-06-11T10:27:52.073198643-04:00 stdout F 2026-06-11 14:27:52,072 INFO nodepool.ComponentRegistry: Noticed new launcher component nodepool-launcher-76bd4b5bf4-kxrbp0000000004
2026-06-11T10:27:52.074851698-04:00 stdout F 2026-06-11 14:27:52,074 INFO nodepool.ComponentRegistry: Component launcher nodepool-launcher-76bd4b5bf4-kxrbp0000000004 updated: {'hostname': 'nodepool-launcher-76bd4b5bf4-kxrbp', 'kind': 'launcher', 'model_api': 0, 'state': 'running', 'version': '11.0.1.dev1 8472a37a'}
2026-06-11T10:27:52.077135292-04:00 stdout F 2026-06-11 14:27:52,077 INFO nodepool.ComponentRegistry: System minimum data model version 0; this component 0
2026-06-11T10:27:52.077579135-04:00 stdout F 2026-06-11 14:27:52,077 INFO nodepool.Component: Registering component in ZooKeeper /nodepool/components/launcher/nodepool-launcher-6d9b78f848-znd8v
2026-06-11T10:27:52.084469268-04:00 stdout F 2026-06-11 14:27:52,084 INFO nodepool.ComponentRegistry: Noticed new launcher component nodepool-launcher-6d9b78f848-znd8v0000000005
2026-06-11T10:27:52.087630846-04:00 stdout F 2026-06-11 14:27:52,086 INFO nodepool.ComponentRegistry: Component launcher nodepool-launcher-6d9b78f848-znd8v0000000005 updated: {'hostname': 'nodepool-launcher-6d9b78f848-znd8v', 'kind': 'launcher', 'model_api': 0, 'state': 'running', 'version': '11.0.1.dev1 8472a37a'}
2026-06-11T10:27:52.088461789-04:00 stdout F 2026-06-11 14:27:52,088 INFO nodepool.CleanupWorker: Starting
2026-06-11T10:27:52.089281512-04:00 stdout F 2026-06-11 14:27:52,089 INFO nodepool.DeletedNodeWorker: Starting
2026-06-11T10:27:52.090046954-04:00 stdout F 2026-06-11 14:27:52,089 INFO nodepool.StatsWorker: Starting
2026-06-11T10:27:56.348632127-04:00 stdout F 2026-06-11 14:27:56,348 INFO nodepool.ComponentRegistry: Noticed new builder component nodepool-builder-00000000007
2026-06-11T10:27:56.350748586-04:00 stdout F 2026-06-11 14:27:56,350 INFO nodepool.ComponentRegistry: Component builder nodepool-builder-00000000007 updated: {'hostname': 'nodepool-builder-0', 'kind': 'builder', 'model_api': 0, 'state': 'stopped', 'version': '11.0.1.dev1 06cc723a'}
2026-06-11T10:28:17.642322561-04:00 stdout F 2026-06-11 14:28:17,642 INFO nodepool.StatsWorker: Won stats reporter election
2026-06-11T10:28:17.644068850-04:00 stdout F 2026-06-11 14:28:17,643 INFO nodepool.ComponentRegistry: Noticed launcher component nodepool-launcher-76bd4b5bf4-kxrbp0000000004 disappeared
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider: Couldn't load context openshiftpods from config
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:   Traceback (most recent call last):
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:     File "/usr/local/lib/python3.11/site-packages/nodepool/driver/utils_k8s.py", line 43, in get_client
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:       conf = _get_conf(log, context)
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:              ^^^^^^^^^^^^^^^^^^^^^^^
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:     File "/usr/local/lib/python3.11/site-packages/nodepool/driver/utils_k8s.py", line 26, in _get_conf
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:       return k8s_config.new_client_from_config(context=context)
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:     File "/usr/local/lib/python3.11/site-packages/kubernetes/config/kube_config.py", line 878, in new_client_from_config
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:       load_kube_config(config_file=config_file, context=context,
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:     File "/usr/local/lib/python3.11/site-packages/kubernetes/config/kube_config.py", line 819, in load_kube_config
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:       loader = _get_kube_config_loader(
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:                ^^^^^^^^^^^^^^^^^^^^^^^^
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:     File "/usr/local/lib/python3.11/site-packages/kubernetes/config/kube_config.py", line 779, in _get_kube_config_loader
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:       return KubeConfigLoader(
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:              ^^^^^^^^^^^^^^^^^
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:     File "/usr/local/lib/python3.11/site-packages/kubernetes/config/kube_config.py", line 206, in __init__
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:       self.set_active_context(active_context)
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:     File "/usr/local/lib/python3.11/site-packages/kubernetes/config/kube_config.py", line 259, in set_active_context
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:       self._current_context = self._config['contexts'].get_with_name(
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:     File "/usr/local/lib/python3.11/site-packages/kubernetes/config/kube_config.py", line 662, in get_with_name
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:       raise ConfigException(
2026-06-11T10:32:48.965085142-04:00 stdout F 2026-06-11 14:32:48,963 ERROR nodepool.driver.openshiftpods.OpenshiftPodsProvider:   kubernetes.config.config_exception.ConfigException: Invalid kube-config file. Expected object with name openshiftpods in /var/lib/nodepool/.kube/config/contexts list
2026-06-11T10:32:48.965510474-04:00 stdout F 2026-06-11 14:32:48,965 INFO nodepool.NodePool: Starting PoolWorker.microshiftLocal-nodepool
2026-06-11T10:32:48.966301167-04:00 stdout F 2026-06-11 14:32:48,966 INFO nodepool.Component: Registering component in ZooKeeper /nodepool/components/pool/nodepool-launcher-6d9b78f848-znd8v
2026-06-11T10:32:48.973956712-04:00 stdout F 2026-06-11 14:32:48,973 INFO nodepool.ComponentRegistry: Noticed new pool component nodepool-launcher-6d9b78f848-znd8v0000000000
2026-06-11T10:32:48.974534269-04:00 stdout F 2026-06-11 14:32:48,974 INFO nodepool.ComponentRegistry: Component pool nodepool-launcher-6d9b78f848-znd8v0000000000 updated: {'hostname': 'nodepool-launcher-6d9b78f848-znd8v', 'id': 'nodepool-launcher-6d9b78f848-znd8v-PoolWorker.microshiftLocal-nodepool-0445810e86af4bd191cb4f1fb8b4230c', 'kind': 'pool', 'model_api': 0, 'name': 'nodepool', 'paused': False, 'priority': 100, 'provider_name': 'microshiftLocal', 'state': 'running', 'supported_labels': ['zuul-worker-sf-operator-ci-syxrfjxz'], 'version': '11.0.1.dev1 8472a37a'}
2026-06-11T10:38:39.045354898-04:00 stdout F 2026-06-11 14:38:39,044 INFO nodepool.PoolWorker.microshiftLocal-nodepool: [e: 20376fb0584c40b386452500eee7b2b6] [node_request: 200-0000000000] Assigning node request <NodeRequest {'state': 'requested', 'state_time': 1781188718.8330073, 'declined_by': [], 'node_types': ['zuul-worker-sf-operator-ci-syxrfjxz'], 'nodes': [], 'reuse': True, 'requestor': '9f5815949cff418384674ac04fd75e1d', 'requestor_data': {'build_set_uuid': 'bcf3be79b283468b8575c73f4587cea8', 'tenant_name': 'demo-tenant', 'pipeline_name': 'check', 'job_uuid': 'd0b12269cfc345d582a9ad7bc3fc4122', 'job_name': 'microshift-pod', 'span_info': {'name': 'RequestNodes', 'trace_id': 285109200542263368018417367319916436624, 'span_id': 17600237073260229320, 'trace_flags': 1, 'start_time': 1781188718833016576, 'parent': {'trace_id': 285109200542263368018417367319916436624, 'span_id': 12647667885678272494, 'is_remote': True}}}, 'provider': None, 'relative_priority': 0, 'event_id': '20376fb0584c40b386452500eee7b2b6', 'created_time': 1781188718.8330166, 'tenant_name': 'demo-tenant', 'id': '200-0000000000', 'stat': ZnodeStat(czxid=18742, mzxid=18742, ctime=1781188718833, mtime=1781188718833, version=0, cversion=0, aversion=0, ephemeralOwner=0, dataLength=819, numChildren=0, pzxid=18742)}>
2026-06-11T10:38:39.088968208-04:00 stdout F 2026-06-11 14:38:39,085 ERROR nodepool.NodeLauncher: [e: 20376fb0584c40b386452500eee7b2b6] [node_request: 200-0000000000] [node: 0000000000] Launch attempt 1/3 failed for node 0000000000:
2026-06-11T10:38:39.088968208-04:00 stdout F 2026-06-11 14:38:39,085 ERROR nodepool.NodeLauncher:   Traceback (most recent call last):
2026-06-11T10:38:39.088968208-04:00 stdout F 2026-06-11 14:38:39,085 ERROR nodepool.NodeLauncher:     File "/usr/local/lib/python3.11/site-packages/nodepool/driver/openshift/handler.py", line 69, in launch
2026-06-11T10:38:39.088968208-04:00 stdout F 2026-06-11 14:38:39,085 ERROR nodepool.NodeLauncher:       self._launchLabel()
2026-06-11T10:38:39.088968208-04:00 stdout F 2026-06-11 14:38:39,085 ERROR nodepool.NodeLauncher:     File "/usr/local/lib/python3.11/site-packages/nodepool/driver/openshiftpods/handler.py", line 29, in _launchLabel
2026-06-11T10:38:39.088968208-04:00 stdout F 2026-06-11 14:38:39,085 ERROR nodepool.NodeLauncher:       self.handler.manager.createPod(self.node, self.handler.pool.name,
2026-06-11T10:38:39.088968208-04:00 stdout F 2026-06-11 14:38:39,085 ERROR nodepool.NodeLauncher:     File "/usr/local/lib/python3.11/site-packages/nodepool/driver/openshift/provider.py", line 231, in createPod
2026-06-11T10:38:39.088968208-04:00 stdout F 2026-06-11 14:38:39,085 ERROR nodepool.NodeLauncher:       self.k8s_client.create_namespaced_pod(project, pod_body)
2026-06-11T10:38:39.088968208-04:00 stdout F 2026-06-11 14:38:39,085 ERROR nodepool.NodeLauncher:       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-06-11T10:38:39.088968208-04:00 stdout F 2026-06-11 14:38:39,085 ERROR nodepool.NodeLauncher:   AttributeError: 'NoneType' object has no attribute 'create_namespaced_pod'
2026-06-11T10:38:39.089907234-04:00 stdout F 2026-06-11 14:38:39,089 ERROR nodepool.NodeLauncher: [e: 20376fb0584c40b386452500eee7b2b6] [node_request: 200-0000000000] [node: 0000000000] Launch attempt 2/3 failed for node 0000000000:
2026-06-11T10:38:39.089907234-04:00 stdout F 2026-06-11 14:38:39,089 ERROR nodepool.NodeLauncher:   Traceback (most recent call last):
2026-06-11T10:38:39.089907234-04:00 stdout F 2026-06-11 14:38:39,089 ERROR nodepool.NodeLauncher:     File "/usr/local/lib/python3.11/site-packages/nodepool/driver/openshift/handler.py", line 69, in launch
2026-06-11T10:38:39.089907234-04:00 stdout F 2026-06-11 14:38:39,089 ERROR nodepool.NodeLauncher:       self._launchLabel()
2026-06-11T10:38:39.089907234-04:00 stdout F 2026-06-11 14:38:39,089 ERROR nodepool.NodeLauncher:     File "/usr/local/lib/python3.11/site-packages/nodepool/driver/openshiftpods/handler.py", line 29, in _launchLabel
2026-06-11T10:38:39.089907234-04:00 stdout F 2026-06-11 14:38:39,089 ERROR nodepool.NodeLauncher:       self.handler.manager.createPod(self.node, self.handler.pool.name,
2026-06-11T10:38:39.089907234-04:00 stdout F 2026-06-11 14:38:39,089 ERROR nodepool.NodeLauncher:     File "/usr/local/lib/python3.11/site-packages/nodepool/driver/openshift/provider.py", line 231, in createPod
2026-06-11T10:38:39.089907234-04:00 stdout F 2026-06-11 14:38:39,089 ERROR nodepool.NodeLauncher:       self.k8s_client.create_namespaced_pod(project, pod_body)
2026-06-11T10:38:39.089907234-04:00 stdout F 2026-06-11 14:38:39,089 ERROR nodepool.NodeLauncher:       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-06-11T10:38:39.089907234-04:00 stdout F 2026-06-11 14:38:39,089 ERROR nodepool.NodeLauncher:   AttributeError: 'NoneType' object has no attribute 'create_namespaced_pod'
2026-06-11T10:38:39.090655865-04:00 stdout F 2026-06-11 14:38:39,090 ERROR nodepool.NodeLauncher: [e: 20376fb0584c40b386452500eee7b2b6] [node_request: 200-0000000000] [node: 0000000000] Launch attempt 3/3 failed for node 0000000000:
2026-06-11T10:38:39.090655865-04:00 stdout F 2026-06-11 14:38:39,090 ERROR nodepool.NodeLauncher:   Traceback (most recent call last):
2026-06-11T10:38:39.090655865-04:00 stdout F 2026-06-11 14:38:39,090 ERROR nodepool.NodeLauncher:     File "/usr/local/lib/python3.11/site-packages/nodepool/driver/openshift/handler.py", line 69, in launch
2026-06-11T10:38:39.090655865-04:00 stdout F 2026-06-11 14:38:39,090 ERROR nodepool.NodeLauncher:       self._launchLabel()
2026-06-11T10:38:39.090655865-04:00 stdout F 2026-06-11 14:38:39,090 ERROR nodepool.NodeLauncher:     File "/usr/local/lib/python3.11/site-packages/nodepool/driver/openshiftpods/handler.py", line 29, in _launchLabel
2026-06-11T10:38:39.090655865-04:00 stdout F 2026-06-11 14:38:39,090 ERROR nodepool.NodeLauncher:       self.handler.manager.createPod(self.node, self.handler.pool.name,
2026-06-11T10:38:39.090655865-04:00 stdout F 2026-06-11 14:38:39,090 ERROR nodepool.NodeLauncher:     File "/usr/local/lib/python3.11/site-packages/nodepool/driver/openshift/provider.py", line 231, in createPod
2026-06-11T10:38:39.090655865-04:00 stdout F 2026-06-11 14:38:39,090 ERROR nodepool.NodeLauncher:       self.k8s_client.create_namespaced_pod(project, pod_body)
2026-06-11T10:38:39.090655865-04:00 stdout F 2026-06-11 14:38:39,090 ERROR nodepool.NodeLauncher:       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-06-11T10:38:39.090655865-04:00 stdout F 2026-06-11 14:38:39,090 ERROR nodepool.NodeLauncher:   AttributeError: 'NoneType' object has no attribute 'create_namespaced_pod'
2026-06-11T10:38:39.093423942-04:00 stdout F 2026-06-11 14:38:39,090 ERROR nodepool.NodeLauncher: [e: 20376fb0584c40b386452500eee7b2b6] [node_request: 200-0000000000] [node: 0000000000] Launch failed for node 0000000000: tenant=demo-tenant label=['zuul-worker-sf-operator-ci-syxrfjxz'] hostname=None
2026-06-11T10:38:39.093423942-04:00 stdout F 2026-06-11 14:38:39,090 ERROR nodepool.NodeLauncher:   Traceback (most recent call last):
2026-06-11T10:38:39.093423942-04:00 stdout F 2026-06-11 14:38:39,090 ERROR nodepool.NodeLauncher:     File "/usr/local/lib/python3.11/site-packages/nodepool/driver/utils.py", line 78, in run
2026-06-11T10:38:39.093423942-04:00 stdout F 2026-06-11 14:38:39,090 ERROR nodepool.NodeLauncher:       self.launch()
2026-06-11T10:38:39.093423942-04:00 stdout F 2026-06-11 14:38:39,090 ERROR nodepool.NodeLauncher:     File "/usr/local/lib/python3.11/site-packages/nodepool/driver/openshift/handler.py", line 69, in launch
2026-06-11T10:38:39.093423942-04:00 stdout F 2026-06-11 14:38:39,090 ERROR nodepool.NodeLauncher:       self._launchLabel()
2026-06-11T10:38:39.093423942-04:00 stdout F 2026-06-11 14:38:39,090 ERROR nodepool.NodeLauncher:     File "/usr/local/lib/python3.11/site-packages/nodepool/driver/openshiftpods/handler.py", line 29, in _launchLabel
2026-06-11T10:38:39.093423942-04:00 stdout F 2026-06-11 14:38:39,090 ERROR nodepool.NodeLauncher:       self.handler.manager.createPod(self.node, self.handler.pool.name,
2026-06-11T10:38:39.093423942-04:00 stdout F 2026-06-11 14:38:39,090 ERROR nodepool.NodeLauncher:     File "/usr/local/lib/python3.11/site-packages/nodepool/driver/openshift/provider.py", line 231, in createPod
2026-06-11T10:38:39.093423942-04:00 stdout F 2026-06-11 14:38:39,090 ERROR nodepool.NodeLauncher:       self.k8s_client.create_namespaced_pod(project, pod_body)
2026-06-11T10:38:39.093423942-04:00 stdout F 2026-06-11 14:38:39,090 ERROR nodepool.NodeLauncher:       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-06-11T10:38:39.093423942-04:00 stdout F 2026-06-11 14:38:39,090 ERROR nodepool.NodeLauncher:   AttributeError: 'NoneType' object has no attribute 'create_namespaced_pod'
2026-06-11T10:38:42.189345778-04:00 stdout F 2026-06-11 14:38:42,189 INFO nodepool.DeletedNodeWorker: Deleting failed instance None from microshiftLocal
2026-06-11T10:38:42.194193313-04:00 stdout F 2026-06-11 14:38:42,193 INFO nodepool.NodeDeleter: Deleting ZK node id=0000000000, state=deleting, external_id=None
2026-06-11T10:39:05.802075796-04:00 stdout F 2026-06-11 14:39:05,801 INFO paste.httpserver.ThreadPool: kill_hung_threads status: 10 threads (0 working, 10 idle, 0 starting) ave time N/A, max time 0.00sec, killed 0 workers
2026-06-11T10:50:15.802266999-04:00 stdout F 2026-06-11 14:50:15,802 INFO paste.httpserver.ThreadPool: kill_hung_threads status: 10 threads (0 working, 10 idle, 0 starting) ave time N/A, max time 0.00sec, killed 0 workers