2026-06-10 09:48:14.835169 | Job console starting 2026-06-10 09:48:14.964859 | Updating git repos 2026-06-10 09:48:15.452176 | Cloning repos into workspace 2026-06-10 09:48:16.036949 | Restoring repo states 2026-06-10 09:48:16.056485 | Merging changes 2026-06-10 09:48:16.656344 | Checking out repos 2026-06-10 09:48:16.833924 | Preparing playbooks 2026-06-10 09:48:19.291300 | Running Ansible setup 2026-06-10 09:48:22.819301 | PRE-RUN START: [trusted : gitlab.com/softwarefactory-project/centosinfra-prod/sf-config/playbooks/base/pre.yaml@main] 2026-06-10 09:48:23.469568 | 2026-06-10 09:48:23.469700 | PLAY [localhost] 2026-06-10 09:48:23.476881 | 2026-06-10 09:48:23.476949 | TASK [Gathering Facts] 2026-06-10 09:48:24.494151 | localhost | ok 2026-06-10 09:48:24.508232 | 2026-06-10 09:48:24.508407 | TASK [Setup log path fact] 2026-06-10 09:48:24.543610 | localhost | ok 2026-06-10 09:48:24.565872 | 2026-06-10 09:48:24.566018 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-06-10 09:48:24.608507 | localhost | ok 2026-06-10 09:48:24.619740 | 2026-06-10 09:48:24.619887 | TASK [emit-job-header : Print job information] 2026-06-10 09:48:24.672461 | # Job Information 2026-06-10 09:48:24.672724 | Ansible Version: 2.16.18 2026-06-10 09:48:24.672791 | Job: sf-operator-minikube-integration 2026-06-10 09:48:24.672834 | Pipeline: check 2026-06-10 09:48:24.672872 | Executor: 38.102.83.47 2026-06-10 09:48:24.672908 | Triggered by: https://softwarefactory-project.io/r/c/software-factory/sf-operator/+/35214 2026-06-10 09:48:24.672949 | Log URL (when completed): https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/logs/316/sf/31684378a9474c86a3ad644d73a2adf1/ 2026-06-10 09:48:24.672988 | Event ID: ef0ed42b55424baca131364e6dbf9542 2026-06-10 09:48:24.678407 | 2026-06-10 09:48:24.678598 | LOOP [emit-job-header : Print node information] 2026-06-10 09:48:24.783321 | localhost | ok: 2026-06-10 09:48:24.783672 | localhost | # Node Information 2026-06-10 09:48:24.783729 | localhost | Inventory Hostname: controller 2026-06-10 09:48:24.783770 | localhost | Hostname: np0000069565 2026-06-10 09:48:24.783809 | localhost | Username: zuul-worker 2026-06-10 09:48:24.783853 | localhost | Distro: CentOS 9 2026-06-10 09:48:24.783889 | localhost | Provider: vexxhost-nodepool-sf 2026-06-10 09:48:24.783925 | localhost | Region: RegionOne 2026-06-10 09:48:24.783960 | localhost | Label: cloud-centos-9-big 2026-06-10 09:48:24.783995 | localhost | Product Name: OpenStack Nova 2026-06-10 09:48:24.784031 | localhost | Interface IP: 38.129.56.167 2026-06-10 09:48:24.799910 | 2026-06-10 09:48:24.800152 | TASK [log-inventory : Ensure Zuul Ansible directory exists] 2026-06-10 09:48:25.274509 | localhost -> localhost | changed 2026-06-10 09:48:25.283724 | 2026-06-10 09:48:25.283861 | TASK [log-inventory : Copy ansible inventory to logs dir] 2026-06-10 09:48:26.328306 | localhost -> localhost | changed 2026-06-10 09:48:26.334865 | 2026-06-10 09:48:26.334921 | PLAY [all] 2026-06-10 09:48:26.345100 | 2026-06-10 09:48:26.345310 | TASK [include_role : start-zuul-console] 2026-06-10 09:48:26.368394 | controller | ok 2026-06-10 09:48:26.383213 | 2026-06-10 09:48:26.383321 | TASK [start-zuul-console : Start zuul_console daemon.] 2026-06-10 09:48:26.781665 | controller | ok 2026-06-10 09:48:26.788156 | 2026-06-10 09:48:26.788221 | TASK [include_role : validate-host] 2026-06-10 09:48:26.816285 | controller | ok 2026-06-10 09:48:26.837943 | 2026-06-10 09:48:26.838038 | TASK [validate-host : Define zuul_info_dir fact] 2026-06-10 09:48:26.876478 | controller | ok 2026-06-10 09:48:26.879977 | 2026-06-10 09:48:26.880036 | TASK [validate-host : Ensure Zuul Ansible directory exists] 2026-06-10 09:48:27.180525 | controller -> localhost | ok 2026-06-10 09:48:27.191061 | 2026-06-10 09:48:27.191194 | TASK [validate-host : Collect information about the host] 2026-06-10 09:48:27.924399 | controller | ok 2026-06-10 09:48:27.977938 | 2026-06-10 09:48:27.978096 | TASK [validate-host : Sanitize hostname] 2026-06-10 09:48:28.049896 | controller | ok 2026-06-10 09:48:28.054337 | 2026-06-10 09:48:28.054425 | TASK [validate-host : Write out all ansible variables/facts known for each host] 2026-06-10 09:48:28.571716 | controller -> localhost | changed 2026-06-10 09:48:28.582126 | 2026-06-10 09:48:28.582279 | TASK [validate-host : Collect information about zuul worker] 2026-06-10 09:48:28.968898 | controller | ok 2026-06-10 09:48:28.976049 | 2026-06-10 09:48:28.976212 | TASK [validate-host : Write out all zuul information for each host] 2026-06-10 09:48:29.471913 | controller -> localhost | changed 2026-06-10 09:48:29.487551 | 2026-06-10 09:48:29.487674 | TASK [include_role : prepare-workspace] 2026-06-10 09:48:29.523152 | controller | ok 2026-06-10 09:48:29.557349 | 2026-06-10 09:48:29.557521 | TASK [prepare-workspace : Start zuul_console daemon.] 2026-06-10 09:48:29.800786 | controller | ok 2026-06-10 09:48:29.817303 | 2026-06-10 09:48:29.817558 | TASK [prepare-workspace : Synchronize src repos to workspace directory.] 2026-06-10 09:48:31.054630 | controller | Output suppressed because no_log was given 2026-06-10 09:48:31.066151 | 2026-06-10 09:48:31.066373 | TASK [include_role : add-build-sshkey] 2026-06-10 09:48:31.101278 | controller | ok 2026-06-10 09:48:31.134019 | 2026-06-10 09:48:31.134127 | TASK [add-build-sshkey : Check to see if ssh key was already created for this build] 2026-06-10 09:48:31.423768 | controller -> localhost | ok 2026-06-10 09:48:31.430139 | 2026-06-10 09:48:31.430236 | TASK [add-build-sshkey : Create a new key in workspace based on build UUID] 2026-06-10 09:48:31.464332 | controller | ok 2026-06-10 09:48:31.489971 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml 2026-06-10 09:48:31.496388 | 2026-06-10 09:48:31.496515 | TASK [add-build-sshkey : Create Temp SSH key] 2026-06-10 09:48:32.484944 | controller -> localhost | Generating public/private rsa key pair. 2026-06-10 09:48:32.485164 | controller -> localhost | Your identification has been saved in /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/work/31684378a9474c86a3ad644d73a2adf1_id_rsa 2026-06-10 09:48:32.485195 | controller -> localhost | Your public key has been saved in /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/work/31684378a9474c86a3ad644d73a2adf1_id_rsa.pub 2026-06-10 09:48:32.485215 | controller -> localhost | The key fingerprint is: 2026-06-10 09:48:32.485235 | controller -> localhost | SHA256:dlMQ/cELAG1PUAPqNafi/3A97wmYGoliulT2x70CL1o zuul-build-sshkey 2026-06-10 09:48:32.485253 | controller -> localhost | The key's randomart image is: 2026-06-10 09:48:32.485271 | controller -> localhost | +---[RSA 3072]----+ 2026-06-10 09:48:32.485288 | controller -> localhost | | .o*Bo. | 2026-06-10 09:48:32.485307 | controller -> localhost | | .o.+.o | 2026-06-10 09:48:32.485325 | controller -> localhost | | ..oooo o | 2026-06-10 09:48:32.485342 | controller -> localhost | | . . =. o | 2026-06-10 09:48:32.485359 | controller -> localhost | | o S + | 2026-06-10 09:48:32.485376 | controller -> localhost | | o .+ooo.o. | 2026-06-10 09:48:32.485392 | controller -> localhost | | . o E+=.+..o | 2026-06-10 09:48:32.485410 | controller -> localhost | | . o o..+oo. .o.| 2026-06-10 09:48:32.485431 | controller -> localhost | | o... ..oo. oo| 2026-06-10 09:48:32.485482 | controller -> localhost | +----[SHA256]-----+ 2026-06-10 09:48:32.485529 | controller -> localhost | ok: Runtime: 0:00:00.510464 2026-06-10 09:48:32.488840 | 2026-06-10 09:48:32.488897 | TASK [add-build-sshkey : Remote setup ssh keys (linux)] 2026-06-10 09:48:32.528166 | controller | ok 2026-06-10 09:48:32.535338 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml 2026-06-10 09:48:32.541601 | 2026-06-10 09:48:32.541661 | TASK [add-build-sshkey : Remove previously added zuul-build-sshkey] 2026-06-10 09:48:32.576165 | controller | skipping: Conditional result was False 2026-06-10 09:48:32.580532 | 2026-06-10 09:48:32.580602 | TASK [add-build-sshkey : Enable access via build key on all nodes] 2026-06-10 09:48:33.078973 | controller | changed 2026-06-10 09:48:33.086464 | 2026-06-10 09:48:33.086635 | TASK [add-build-sshkey : Make sure user has a .ssh] 2026-06-10 09:48:33.327025 | controller | ok 2026-06-10 09:48:33.336366 | 2026-06-10 09:48:33.336549 | TASK [add-build-sshkey : Install build private key as SSH key on all nodes] 2026-06-10 09:48:34.084613 | controller | changed 2026-06-10 09:48:34.092081 | 2026-06-10 09:48:34.092222 | TASK [add-build-sshkey : Install build public key as SSH key on all nodes] 2026-06-10 09:48:34.869136 | controller | changed 2026-06-10 09:48:34.876821 | 2026-06-10 09:48:34.876952 | TASK [add-build-sshkey : Remote setup ssh keys (windows)] 2026-06-10 09:48:34.912936 | controller | skipping: Conditional result was False 2026-06-10 09:48:34.922587 | 2026-06-10 09:48:34.922727 | TASK [remove-zuul-sshkey : Remove master key from local agent] 2026-06-10 09:48:35.358284 | controller -> localhost | changed 2026-06-10 09:48:35.374877 | 2026-06-10 09:48:35.375019 | TASK [add-build-sshkey : Add back temp key] 2026-06-10 09:48:35.703107 | controller -> localhost | Identity added: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/work/31684378a9474c86a3ad644d73a2adf1_id_rsa (zuul-build-sshkey) 2026-06-10 09:48:35.703504 | controller -> localhost | ok: Runtime: 0:00:00.019239 2026-06-10 09:48:35.710988 | 2026-06-10 09:48:35.711130 | TASK [add-build-sshkey : Verify we can still SSH to all nodes] 2026-06-10 09:48:36.122049 | controller | ok 2026-06-10 09:48:36.128869 | 2026-06-10 09:48:36.129113 | TASK [add-build-sshkey : Verify we can still SSH to all nodes (windows)] 2026-06-10 09:48:36.167200 | controller | skipping: Conditional result was False 2026-06-10 09:48:36.182978 | 2026-06-10 09:48:36.183215 | TASK [include_role : prepare-workspace-openshift] 2026-06-10 09:48:36.212285 | controller | skipping: Conditional result was False 2026-06-10 09:48:36.221800 | 2026-06-10 09:48:36.221957 | TASK [include_role : remove-zuul-sshkey] 2026-06-10 09:48:36.249284 | controller | skipping: Conditional result was False 2026-06-10 09:48:36.259882 | 2026-06-10 09:48:36.260030 | LOOP [ensure-output-dirs : Empty Zuul Output directories by removing them] 2026-06-10 09:48:36.511146 | controller | ok: "logs" 2026-06-10 09:48:36.511838 | controller | ok: All items complete 2026-06-10 09:48:36.511901 | 2026-06-10 09:48:36.749291 | controller | ok: "artifacts" 2026-06-10 09:48:36.989909 | controller | ok: "docs" 2026-06-10 09:48:37.001274 | 2026-06-10 09:48:37.001537 | LOOP [ensure-output-dirs : Ensure Zuul Output directories exist] 2026-06-10 09:48:37.308870 | controller | changed: "logs" 2026-06-10 09:48:37.536974 | controller | changed: "artifacts" 2026-06-10 09:48:37.791290 | controller | changed: "docs" 2026-06-10 09:48:37.815131 | 2026-06-10 09:48:37.815400 | LOOP [Tune dnf options] 2026-06-10 09:48:38.281604 | controller | changed: option added 2026-06-10 09:48:38.282050 | controller | changed: All items complete 2026-06-10 09:48:38.282110 | 2026-06-10 09:48:38.507565 | controller | changed: option added 2026-06-10 09:48:38.795748 | controller | changed: option added 2026-06-10 09:48:38.809994 | 2026-06-10 09:48:38.810207 | PLAY RECAP 2026-06-10 09:48:38.810303 | controller | ok: 24 changed: 11 unreachable: 0 failed: 0 skipped: 5 rescued: 0 ignored: 0 2026-06-10 09:48:38.810358 | localhost | ok: 6 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-06-10 09:48:38.810397 | 2026-06-10 09:48:39.004904 | PRE-RUN END RESULT_NORMAL: [trusted : gitlab.com/softwarefactory-project/centosinfra-prod/sf-config/playbooks/base/pre.yaml@main] 2026-06-10 09:48:39.007103 | PRE-RUN START: [trusted : gitlab.com/softwarefactory-project/centosinfra-prod/sf-config/playbooks/rhel_subscription/register.yaml@main] 2026-06-10 09:48:39.702884 | 2026-06-10 09:48:39.703027 | PLAY [all] 2026-06-10 09:48:39.712936 | 2026-06-10 09:48:39.713007 | TASK [Register system] 2026-06-10 09:48:39.788359 | 2026-06-10 09:48:39.788630 | TASK [Get minor release for EUS activation] 2026-06-10 09:48:39.845320 | controller | skipping: Conditional result was False 2026-06-10 09:48:39.854900 | 2026-06-10 09:48:39.855100 | LOOP [Manage Extended Update Support repos] 2026-06-10 09:48:39.902788 | controller | skipping: Conditional result was False 2026-06-10 09:48:39.905594 | controller | skipping: Conditional result was False 2026-06-10 09:48:39.924181 | 2026-06-10 09:48:39.924373 | TASK [Upgrade all packages] 2026-06-10 09:48:40.002213 | controller | skipping: Conditional result was False 2026-06-10 09:48:40.013740 | 2026-06-10 09:48:40.013889 | TASK [Reboot to use new kernel] 2026-06-10 09:48:40.072688 | controller | skipping: Conditional result was False 2026-06-10 09:48:40.075079 | 2026-06-10 09:48:40.075175 | PLAY RECAP 2026-06-10 09:48:40.075278 | controller | ok: 0 changed: 0 unreachable: 0 failed: 0 skipped: 5 rescued: 0 ignored: 0 2026-06-10 09:48:40.075322 | 2026-06-10 09:48:40.299157 | PRE-RUN END RESULT_NORMAL: [trusted : gitlab.com/softwarefactory-project/centosinfra-prod/sf-config/playbooks/rhel_subscription/register.yaml@main] 2026-06-10 09:48:40.301653 | PRE-RUN START: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/install-minikube.yaml@master] 2026-06-10 09:48:40.974777 | 2026-06-10 09:48:40.974944 | PLAY [controller] 2026-06-10 09:48:40.987217 | 2026-06-10 09:48:40.987350 | TASK [Create ramdisk folder] 2026-06-10 09:48:41.489598 | controller | changed 2026-06-10 09:48:41.498152 | 2026-06-10 09:48:41.498305 | TASK [Installing packages] 2026-06-10 09:49:30.817088 | controller | changed 2026-06-10 09:49:30.827271 | 2026-06-10 09:49:30.827413 | TASK [Set pids_limit in containers.conf] 2026-06-10 09:49:31.855691 | controller | changed 2026-06-10 09:49:31.865125 | 2026-06-10 09:49:31.865262 | TASK [Installing Minikube] 2026-06-10 09:49:33.673043 | controller | changed: OK (132766301 bytes) 2026-06-10 09:49:33.685514 | 2026-06-10 09:49:33.685697 | TASK [Start minikube] 2026-06-10 05:49:34.135643 | controller | * minikube v1.36.0 on Centos 9 (kvm/amd64) 2026-06-10 05:49:34.143790 | controller | Using Kubernetes 1.32.5 since patch version was unspecified 2026-06-10 05:49:34.714786 | controller | * minikube 1.38.1 is available! Download it: https://github.com/kubernetes/minikube/releases/tag/v1.38.1 2026-06-10 05:49:34.795420 | controller | * To disable this notice, run: 'minikube config set WantUpdateNotification false' 2026-06-10 05:49:34.795473 | controller | 2026-06-10 05:49:34.833073 | controller | * Using the podman driver based on user configuration 2026-06-10 05:49:35.286277 | controller | * Using Podman driver with root privileges 2026-06-10 05:49:35.290682 | controller | * Starting "minikube" primary control-plane node in "minikube" cluster 2026-06-10 05:49:35.296462 | controller | * Pulling base image v0.0.47 ... 2026-06-10 05:49:35.423949 | controller | * Downloading Kubernetes v1.32.5 preload ... 2026-06-10 05:49:54.469109 | controller | > gcr.io/k8s-minikube/kicbase...: 0 B [_______________________] ?% ? p/s ? > gcr.io/k8s-minikube/kicbase...: 0 B [_______________________] ?% ? p/s ? > gcr.io/k8s-minikube/kicbase...: 0 B [_______________________] ?% ? p/s ? > gcr.io/k8s-minikube/kicbase...: 1.61 KiB / 502.26 MiB 0.00% 2.68 KiB p/ > gcr.io/k8s-minikube/kicbase...: 1.61 KiB / 502.26 MiB 0.00% 2.68 KiB p/ > gcr.io/k8s-minikube/kicbase...: 1.61 KiB / 502.26 MiB 0.00% 2.68 KiB p/ > gcr.io/k8s-minikube/kicbase...: 1.61 KiB / 502.26 MiB 0.00% 2.51 KiB p/ > gcr.io/k8s-minikube/kicbase...: 113.72 KiB / 502.26 MiB 0.02% 2.51 KiB > gcr.io/k8s-minikube/kicbase...: 1.11 MiB / 502.26 MiB 0.22% 2.51 KiB p/ > gcr.io/k8s-minikube/kicbase...: 8.14 MiB / 502.26 MiB 1.62% 899.28 KiB > gcr.io/k8s-minikube/kicbase...: 17.98 MiB / 502.26 MiB 3.58% 899.28 KiB > gcr.io/k8s-minikube/kicbase...: 24.00 MiB / 502.26 MiB 4.78% 899.28 KiB > gcr.io/k8s-minikube/kicbase...: 33.68 MiB / 502.26 MiB 6.71% 3.57 MiB p > gcr.io/k8s-minikube/kicbase...: 45.48 MiB / 502.26 MiB 9.05% 3.57 MiB p > gcr.io/k8s-minikube/kicbase...: 52.17 MiB / 502.26 MiB 10.39% 3.57 MiB > gcr.io/k8s-minikube/kicbase...: 56.00 MiB / 502.26 MiB 11.15% 5.74 MiB > gcr.io/k8s-minikube/kicbase...: 61.77 MiB / 502.26 MiB 12.30% 5.74 MiB > gcr.io/k8s-minikube/kicbase...: 71.94 MiB / 502.26 MiB 14.32% 5.74 MiB > gcr.io/k8s-minikube/kicbase...: 78.44 MiB / 502.26 MiB 15.62% 7.78 MiB > gcr.io/k8s-minikube/kicbase...: 80.00 MiB / 502.26 MiB 15.93% 7.78 MiB > gcr.io/k8s-minikube/kicbase...: 80.00 MiB / 502.26 MiB 15.93% 7.78 MiB > gcr.io/k8s-minikube/kicbase...: 88.00 MiB / 502.26 MiB 17.52% 8.31 MiB > gcr.io/k8s-minikube/kicbase...: 98.12 MiB / 502.26 MiB 19.53% 8.31 MiB > gcr.io/k8s-minikube/kicbase...: 104.00 MiB / 502.26 MiB 20.71% 8.31 MiB > gcr.io/k8s-minikube/kicbase...: 111.49 MiB / 502.26 MiB 22.20% 10.30 Mi > gcr.io/k8s-minikube/kicbase...: 120.00 MiB / 502.26 MiB 23.89% 10.30 Mi > gcr.io/k8s-minikube/kicbase...: 126.03 MiB / 502.26 MiB 25.09% 10.30 Mi > gcr.io/k8s-minikube/kicbase...: 132.39 MiB / 502.26 MiB 26.36% 11.88 Mi > gcr.io/k8s-minikube/kicbase...: 139.22 MiB / 502.26 MiB 27.72% 11.88 Mi > gcr.io/k8s-minikube/kicbase...: 145.31 MiB / 502.26 MiB 28.93% 11.88 Mi > gcr.io/k8s-minikube/kicbase...: 152.00 MiB / 502.26 MiB 30.26% 13.22 Mi > gcr.io/k8s-minikube/kicbase...: 152.00 MiB / 502.26 MiB 30.26% 13.22 Mi > gcr.io/k8s-minikube/kicbase...: 152.00 MiB / 502.26 MiB 30.26% 13.22 Mi > gcr.io/k8s-minikube/kicbase...: 160.00 MiB / 502.26 MiB 31.86% 13.23 Mi > gcr.io/k8s-minikube/kicbase...: 161.46 MiB / 502.26 MiB 32.15% 13.23 Mi > gcr.io/k8s-minikube/kicbase...: 168.58 MiB / 502.26 MiB 33.56% 13.23 Mi > gcr.io/k8s-minikube/kicbase...: 176.00 MiB / 502.26 MiB 35.04% 14.09 Mi > gcr.io/k8s-minikube/kicbase...: 185.83 MiB / 502.26 MiB 37.00% 14.09 Mi > gcr.io/k8s-minikube/kicbase...: 192.00 MiB / 502.26 MiB 38.23% 14.09 Mi > gcr.io/k8s-minikube/kicbase...: 192.03 MiB / 502.26 MiB 38.23% 14.91 Mi > gcr.io/k8s-minikube/kicbase...: 200.00 MiB / 502.26 MiB 39.82% 14.91 Mi > gcr.io/k8s-minikube/kicbase...: 200.00 MiB / 502.26 MiB 39.82% 14.91 Mi > gcr.io/k8s-minikube/kicbase...: 201.44 MiB / 502.26 MiB 40.11% 14.96 Mi > gcr.io/k8s-minikube/kicbase...: 208.00 MiB / 502.26 MiB 41.41% 14.96 Mi > gcr.io/k8s-minikube/kicbase...: 216.00 MiB / 502.26 MiB 43.01% 14.96 Mi > gcr.io/k8s-minikube/kicbase...: 223.00 MiB / 502.26 MiB 44.40% 16.31 Mi > gcr.io/k8s-minikube/kicbase...: 232.00 MiB / 502.26 MiB 46.19% 16.31 Mi > gcr.io/k8s-minikube/kicbase...: 238.64 MiB / 502.26 MiB 47.51% 16.31 Mi > gcr.io/k8s-minikube/kicbase...: 246.49 MiB / 502.26 MiB 49.08% 17.79 Mi > gcr.io/k8s-minikube/kicbase...: 254.21 MiB / 502.26 MiB 50.61% 17.79 Mi > gcr.io/k8s-minikube/kicbase...: 262.99 MiB / 502.26 MiB 52.36% 17.79 Mi > gcr.io/k8s-minikube/kicbase...: 272.00 MiB / 502.26 MiB 54.16% 19.38 Mi > gcr.io/k8s-minikube/kicbase...: 278.52 MiB / 502.26 MiB 55.45% 19.38 Mi > gcr.io/k8s-minikube/kicbase...: 288.00 MiB / 502.26 MiB 57.34% 19.38 Mi > gcr.io/k8s-minikube/kicbase...: 296.00 MiB / 502.26 MiB 58.93% 20.71 Mi > gcr.io/k8s-minikube/kicbase...: 300.89 MiB / 502.26 MiB 59.91% 20.71 Mi > gcr.io/k8s-minikube/kicbase...: 311.58 MiB / 502.26 MiB 62.04% 20.71 Mi > gcr.io/k8s-minikube/kicbase...: 320.00 MiB / 502.26 MiB 63.71% 21.95 Mi > gcr.io/k8s-minikube/kicbase...: 328.00 MiB / 502.26 MiB 65.30% 21.95 Mi > gcr.io/k8s-minikube/kicbase...: 336.00 MiB / 502.26 MiB 66.90% 21.95 Mi > gcr.io/k8s-minikube/kicbase...: 344.00 MiB / 502.26 MiB 68.49% 23.12 Mi > gcr.io/k8s-minikube/kicbase...: 352.00 MiB / 502.26 MiB 70.08% 23.12 Mi > gcr.io/k8s-minikube/kicbase...: 352.00 MiB / 502.26 MiB 70.08% 23.12 Mi > gcr.io/k8s-minikube/kicbase...: 352.00 MiB / 502.26 MiB 70.08% 22.49 Mi > gcr.io/k8s-minikube/kicbase...: 360.00 MiB / 502.26 MiB 71.68% 22.49 Mi > gcr.io/k8s-minikube/kicbase...: 368.00 MiB / 502.26 MiB 73.27% 22.49 Mi > gcr.io/k8s-minikube/kicbase...: 376.00 MiB / 502.26 MiB 74.86% 23.62 Mi > gcr.io/k8s-minikube/kicbase...: 384.00 MiB / 502.26 MiB 76.45% 23.62 Mi > gcr.io/k8s-minikube/kicbase...: 390.96 MiB / 502.26 MiB 77.84% 23.62 Mi > gcr.io/k8s-minikube/kicbase...: 392.00 MiB / 502.26 MiB 78.05% 23.81 Mi > gcr.io/k8s-minikube/kicbase...: 395.63 MiB / 502.26 MiB 78.77% 23.81 Mi > gcr.io/k8s-minikube/kicbase...: 405.00 MiB / 502.26 MiB 80.64% 23.81 Mi > gcr.io/k8s-minikube/kicbase...: 411.88 MiB / 502.26 MiB 82.00% 24.42 Mi > gcr.io/k8s-minikube/kicbase...: 420.64 MiB / 502.26 MiB 83.75% 24.42 Mi > gcr.io/k8s-minikube/kicbase...: 424.00 MiB / 502.26 MiB 84.42% 24.42 Mi > gcr.io/k8s-minikube/kicbase...: 426.94 MiB / 502.26 MiB 85.00% 24.46 Mi > gcr.io/k8s-minikube/kicbase...: 432.00 MiB / 502.26 MiB 86.01% 24.46 Mi > gcr.io/k8s-minikube/kicbase...: 440.00 MiB / 502.26 MiB 87.60% 24.46 Mi > gcr.io/k8s-minikube/kicbase...: 440.52 MiB / 502.26 MiB 87.71% 24.34 Mi > gcr.io/k8s-minikube/kicbase...: 448.00 MiB / 502.26 MiB 89.20% 24.34 Mi > gcr.io/k8s-minikube/kicbase...: 456.00 MiB / 502.26 MiB 90.79% 24.34 Mi > gcr.io/k8s-minikube/kicbase...: 464.00 MiB / 502.26 MiB 92.38% 25.30 Mi > gcr.io/k8s-minikube/kicbase...: 471.13 MiB / 502.26 MiB 93.80% 25.30 Mi > gcr.io/k8s-minikube/kicbase...: 479.30 MiB / 502.26 MiB 95.43% 25.30 Mi > gcr.io/k8s-minikube/kicbase...: 480.00 MiB / 502.26 MiB 95.57% 25.38 Mi > gcr.io/k8s-minikube/kicbase...: 488.00 MiB / 502.26 MiB 97.16% 25.38 Mi > gcr.io/k8s-minikube/kicbase...: 493.30 MiB / 502.26 MiB 98.22% 25.38 Mi > gcr.io/k8s-minikube/kicbase...: 498.31 MiB / 502.26 MiB 99.21% 25.72 Mi > gcr.io/k8s-minikube/kicbase...: 502.26 MiB / 502.26 MiB 100.00% 28.75 ME0610 05:49:54.469049 11778 cache.go:225] Error downloading kic artifacts: not yet implemented, see issue #8426 2026-06-10 05:49:54.470656 | controller | * Creating podman container (CPUs=no-limit, Memory=no-limit) ... 2026-06-10 05:50:37.353168 | controller | * Preparing Kubernetes v1.32.5 on CRI-O 1.24.6 ... 2026-06-10 05:50:37.456189 | controller | - kubelet.housekeeping-interval=5m 2026-06-10 05:50:39.859191 | controller | - Generating certificates and keys ... 2026-06-10 05:50:42.657868 | controller | - Booting up control plane ... 2026-06-10 05:50:46.806856 | controller | - Configuring RBAC rules ... 2026-06-10 05:50:48.183644 | controller | * Configuring CNI (Container Networking Interface) ... 2026-06-10 05:50:48.487265 | controller | * Verifying Kubernetes components... 2026-06-10 05:50:48.551446 | controller | - Using image gcr.io/k8s-minikube/minikube-ingress-dns:0.0.3 2026-06-10 05:50:48.569762 | controller | - Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.5.3 2026-06-10 05:50:48.571066 | controller | - Using image registry.k8s.io/ingress-nginx/controller:v1.12.2 2026-06-10 05:50:48.577035 | controller | - Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.5.3 2026-06-10 05:50:48.581529 | controller | - Using image gcr.io/k8s-minikube/storage-provisioner:v5 2026-06-10 05:50:49.524061 | controller | * Verifying ingress addon... 2026-06-10 05:51:22.533906 | controller | * Enabled addons: ingress-dns, storage-provisioner, default-storageclass, ingress 2026-06-10 05:51:22.541795 | controller | * kubectl not found. If you need it, try: 'minikube kubectl -- get pods -A' 2026-06-10 05:51:22.543900 | controller | * Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default 2026-06-10 09:51:22.885408 | controller | ok: Runtime: 0:01:48.472741 2026-06-10 09:51:22.896379 | 2026-06-10 09:51:22.896679 | TASK [Get minikube deployment ip] 2026-06-10 05:51:23.371268 | controller | 192.168.49.2 2026-06-10 09:51:23.439612 | controller | ok: Runtime: 0:00:00.252048 2026-06-10 09:51:23.444823 | 2026-06-10 09:51:23.444904 | TASK [Get default interface] 2026-06-10 05:51:23.656500 | controller | eth0 2026-06-10 09:51:23.977647 | controller | ok: Runtime: 0:00:00.009021 2026-06-10 09:51:23.985084 | 2026-06-10 09:51:23.985207 | TASK [Test minikube] 2026-06-10 05:51:34.710467 | controller | NAMESPACE NAME READY STATUS RESTARTS AGE 2026-06-10 05:51:34.710557 | controller | ingress-nginx ingress-nginx-admission-create-8wpgz 0/1 Completed 0 42s 2026-06-10 05:51:34.710567 | controller | ingress-nginx ingress-nginx-admission-patch-k2gl7 0/1 Completed 0 42s 2026-06-10 05:51:34.710572 | controller | ingress-nginx ingress-nginx-controller-86649c845b-hdbbv 1/1 Running 0 42s 2026-06-10 05:51:34.710580 | controller | kube-system coredns-668d6bf9bc-sh7x9 1/1 Running 0 42s 2026-06-10 05:51:34.710586 | controller | kube-system etcd-minikube 1/1 Running 0 47s 2026-06-10 05:51:34.710600 | controller | kube-system kindnet-rb2lw 1/1 Running 0 42s 2026-06-10 05:51:34.710644 | controller | kube-system kube-apiserver-minikube 1/1 Running 0 47s 2026-06-10 05:51:34.710652 | controller | kube-system kube-controller-manager-minikube 1/1 Running 0 47s 2026-06-10 05:51:34.710659 | controller | kube-system kube-ingress-dns-minikube 0/1 ErrImagePull 0 45s 2026-06-10 05:51:34.710671 | controller | kube-system kube-proxy-z8pmr 1/1 Running 0 42s 2026-06-10 05:51:34.710681 | controller | kube-system kube-scheduler-minikube 1/1 Running 0 47s 2026-06-10 05:51:34.710687 | controller | kube-system storage-provisioner 1/1 Running 0 45s 2026-06-10 09:51:35.046119 | controller | ok: Runtime: 0:00:10.738417 2026-06-10 09:51:35.049295 | 2026-06-10 09:51:35.049396 | PLAY RECAP 2026-06-10 09:51:35.049533 | controller | ok: 8 changed: 8 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-06-10 09:51:35.049581 | 2026-06-10 09:51:35.201956 | PRE-RUN END RESULT_NORMAL: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/install-minikube.yaml@master] 2026-06-10 09:51:35.204182 | PRE-RUN START: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/prepare-minikube.yaml@master] 2026-06-10 09:51:35.804930 | 2026-06-10 09:51:35.805061 | PLAY [controller] 2026-06-10 09:51:35.814787 | 2026-06-10 09:51:35.814857 | TASK [Crete sf namespace] 2026-06-10 05:51:36.324180 | controller | namespace/sf created 2026-06-10 09:51:36.842887 | controller | ok: Runtime: 0:00:00.136925 2026-06-10 09:51:36.850916 | 2026-06-10 09:51:36.851076 | TASK [Set sf as the default namespace] 2026-06-10 05:51:37.201313 | controller | Context "minikube" modified. 2026-06-10 09:51:37.396490 | controller | ok: Runtime: 0:00:00.101256 2026-06-10 09:51:37.398908 | 2026-06-10 09:51:37.399011 | PLAY RECAP 2026-06-10 09:51:37.399095 | controller | ok: 2 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-06-10 09:51:37.399140 | 2026-06-10 09:51:37.529337 | PRE-RUN END RESULT_NORMAL: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/prepare-minikube.yaml@master] 2026-06-10 09:51:37.531727 | PRE-RUN START: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/install-test-dependencies.yaml@master] 2026-06-10 09:51:38.213294 | 2026-06-10 09:51:38.213417 | PLAY [controller] 2026-06-10 09:51:38.223857 | 2026-06-10 09:51:38.223939 | TASK [install-golang : Check Go version] 2026-06-10 09:51:38.751701 | controller | ok: ERROR (ignored) 2026-06-10 09:51:38.752151 | controller | { 2026-06-10 09:51:38.752237 | controller | "failed_when_result": false, 2026-06-10 09:51:38.752297 | controller | "msg": "[Errno 2] No such file or directory: b'go'", 2026-06-10 09:51:38.752414 | controller | "rc": 2 2026-06-10 09:51:38.752527 | controller | } 2026-06-10 09:51:38.763724 | 2026-06-10 09:51:38.763863 | TASK [install-golang : Ensure a clean install] 2026-06-10 09:51:39.174705 | controller | ok 2026-06-10 09:51:39.183307 | 2026-06-10 09:51:39.183483 | TASK [install-golang : Extract Go archive] 2026-06-10 09:51:53.162422 | controller | changed 2026-06-10 09:51:53.171918 | 2026-06-10 09:51:53.172064 | LOOP [install-golang : Create symlinks for Go binary] 2026-06-10 09:51:53.419136 | controller | changed: "go" 2026-06-10 09:51:53.626631 | controller | changed: "gofmt" 2026-06-10 09:51:53.642675 | 2026-06-10 09:51:53.642824 | TASK [Installing packages] 2026-06-10 09:51:54.628310 | controller | ok: Nothing to do 2026-06-10 09:51:54.633614 | 2026-06-10 09:51:54.633691 | TASK [Install pip packages] 2026-06-10 09:51:57.899925 | controller | changed 2026-06-10 09:51:57.903113 | 2026-06-10 09:51:57.903213 | PLAY RECAP 2026-06-10 09:51:57.903293 | controller | ok: 6 changed: 3 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-06-10 09:51:57.903351 | 2026-06-10 09:51:58.041996 | PRE-RUN END RESULT_NORMAL: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/install-test-dependencies.yaml@master] 2026-06-10 09:51:58.043596 | RUN START: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/run-deploy.yaml@master] 2026-06-10 09:51:58.672981 | 2026-06-10 09:51:58.673167 | PLAY [controller] 2026-06-10 09:51:58.683369 | 2026-06-10 09:51:58.683510 | TASK [Run ./hack/deploy.sh] 2026-06-10 05:51:59.050748 | controller | [+] Installing kubectl 2026-06-10 05:51:59.080453 | controller | % Total % Received % Xferd Average Speed Time Time Time Current 2026-06-10 05:51:59.080514 | controller | Dload Upload Total Spent Left Speed 2026-06-10 05:51:59.437671 | controller | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 56.7M 100 56.7M 0 0 158M 0 --:--:-- --:--:-- --:--:-- 158M 2026-06-10 05:51:59.515783 | controller | go: downloading go1.25.11 (linux/amd64) 2026-06-10 05:52:06.156128 | controller | Error from server (NotFound): pods "gerrit-0" not found 2026-06-10 05:52:06.158132 | controller | [+] Setting up gerrit 2026-06-10 05:52:06.187696 | controller | go: downloading k8s.io/client-go v0.30.2 2026-06-10 05:52:06.189400 | controller | go: downloading github.com/spf13/cobra v1.8.1 2026-06-10 05:52:06.202612 | controller | go: downloading github.com/openshift/api v0.0.0-20240715171821-e9f09d21bcb5 2026-06-10 05:52:06.203972 | controller | go: downloading go.uber.org/zap v1.27.0 2026-06-10 05:52:06.204506 | controller | go: downloading k8s.io/api v0.30.2 2026-06-10 05:52:06.207426 | controller | go: downloading k8s.io/apimachinery v0.30.2 2026-06-10 05:52:07.253380 | controller | go: downloading k8s.io/utils v0.0.0-20240711033017-18e509b52bc8 2026-06-10 05:52:07.256741 | controller | go: downloading sigs.k8s.io/controller-runtime v0.18.4 2026-06-10 05:52:07.499062 | controller | go: downloading sigs.k8s.io/yaml v1.4.0 2026-06-10 05:52:07.534671 | controller | go: downloading gopkg.in/yaml.v3 v3.0.1 2026-06-10 05:52:07.543573 | controller | go: downloading github.com/fatih/color v1.17.0 2026-06-10 05:52:07.543626 | controller | go: downloading github.com/go-sql-driver/mysql v1.8.1 2026-06-10 05:52:07.544819 | controller | go: downloading gopkg.in/ini.v1 v1.67.0 2026-06-10 05:52:07.545230 | controller | go: downloading k8s.io/kubectl v0.30.2 2026-06-10 05:52:07.552086 | controller | go: downloading go.uber.org/multierr v1.11.0 2026-06-10 05:52:07.552794 | controller | go: downloading github.com/spf13/pflag v1.0.5 2026-06-10 05:52:07.664262 | controller | go: downloading github.com/gogo/protobuf v1.3.2 2026-06-10 05:52:07.677490 | controller | go: downloading github.com/google/gofuzz v1.2.0 2026-06-10 05:52:07.689044 | controller | go: downloading k8s.io/klog/v2 v2.130.1 2026-06-10 05:52:07.694924 | controller | go: downloading github.com/go-logr/logr v1.4.2 2026-06-10 05:52:07.703294 | controller | go: downloading github.com/go-logr/zapr v1.3.0 2026-06-10 05:52:07.775683 | controller | go: downloading github.com/google/uuid v1.6.0 2026-06-10 05:52:07.815661 | controller | go: downloading golang.org/x/crypto v0.52.0 2026-06-10 05:52:07.845066 | controller | go: downloading golang.org/x/exp v0.0.0-20240716175740-e3f259677ff7 2026-06-10 05:52:07.845245 | controller | go: downloading gopkg.in/yaml.v2 v2.4.0 2026-06-10 05:52:07.849654 | controller | go: downloading k8s.io/apiextensions-apiserver v0.30.2 2026-06-10 05:52:08.000937 | controller | go: downloading github.com/imdario/mergo v0.3.16 2026-06-10 05:52:08.027628 | controller | go: downloading golang.org/x/term v0.43.0 2026-06-10 05:52:08.098589 | controller | go: downloading golang.org/x/net v0.55.0 2026-06-10 05:52:08.168178 | controller | go: downloading gopkg.in/inf.v0 v0.9.1 2026-06-10 05:52:08.169769 | controller | go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.4.1 2026-06-10 05:52:08.171900 | controller | go: downloading github.com/gorilla/websocket v1.5.1 2026-06-10 05:52:08.172187 | controller | go: downloading github.com/evanphx/json-patch/v5 v5.9.0 2026-06-10 05:52:08.174055 | controller | go: downloading github.com/mattn/go-isatty v0.0.20 2026-06-10 05:52:08.174134 | controller | go: downloading github.com/mattn/go-colorable v0.1.13 2026-06-10 05:52:08.186273 | controller | go: downloading filippo.io/edwards25519 v1.1.1 2026-06-10 05:52:08.258100 | controller | go: downloading github.com/evanphx/json-patch v5.9.0+incompatible 2026-06-10 05:52:08.320634 | controller | go: downloading golang.org/x/oauth2 v0.27.0 2026-06-10 05:52:08.322583 | controller | go: downloading golang.org/x/sys v0.45.0 2026-06-10 05:52:08.325993 | controller | go: downloading golang.org/x/time v0.5.0 2026-06-10 05:52:08.332987 | controller | go: downloading sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd 2026-06-10 05:52:08.335797 | controller | go: downloading github.com/golang/protobuf v1.5.4 2026-06-10 05:52:08.358161 | controller | go: downloading github.com/google/gnostic-models v0.6.8 2026-06-10 05:52:08.412660 | controller | go: downloading k8s.io/kube-openapi v0.0.0-20240430033511-f0e62f92d13f 2026-06-10 05:52:08.470863 | controller | go: downloading gomodules.xyz/jsonpatch/v2 v2.4.0 2026-06-10 05:52:08.492461 | controller | go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da 2026-06-10 05:52:08.497599 | controller | go: downloading github.com/prometheus/client_golang v1.18.0 2026-06-10 05:52:08.499485 | controller | go: downloading github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc 2026-06-10 05:52:08.509438 | controller | go: downloading github.com/json-iterator/go v1.1.12 2026-06-10 05:52:08.605495 | controller | go: downloading github.com/pkg/errors v0.9.1 2026-06-10 05:52:08.672143 | controller | go: downloading github.com/moby/spdystream v0.5.1 2026-06-10 05:52:08.682303 | controller | go: downloading google.golang.org/protobuf v1.34.1 2026-06-10 05:52:08.997638 | controller | go: downloading github.com/google/go-cmp v0.6.0 2026-06-10 05:52:08.997960 | controller | go: downloading github.com/fsnotify/fsnotify v1.7.0 2026-06-10 05:52:08.998083 | controller | go: downloading github.com/modern-go/reflect2 v1.0.2 2026-06-10 05:52:08.998090 | controller | go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd 2026-06-10 05:52:08.998372 | controller | go: downloading github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f 2026-06-10 05:52:08.998572 | controller | go: downloading golang.org/x/text v0.37.0 2026-06-10 05:52:09.134855 | controller | go: downloading github.com/prometheus/client_model v0.6.1 2026-06-10 05:52:09.137645 | controller | go: downloading github.com/prometheus/common v0.47.0 2026-06-10 05:52:09.140913 | controller | go: downloading github.com/beorn7/perks v1.0.1 2026-06-10 05:52:09.146058 | controller | go: downloading github.com/cespare/xxhash/v2 v2.3.0 2026-06-10 05:52:09.268949 | controller | go: downloading github.com/prometheus/procfs v0.15.0 2026-06-10 05:52:09.278983 | controller | go: downloading github.com/go-openapi/jsonreference v0.21.0 2026-06-10 05:52:09.281416 | controller | go: downloading github.com/go-openapi/swag v0.23.0 2026-06-10 05:52:09.297813 | controller | go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 2026-06-10 05:52:09.299154 | controller | go: downloading github.com/emicklei/go-restful/v3 v3.12.0 2026-06-10 05:52:09.420171 | controller | go: downloading github.com/go-openapi/jsonpointer v0.21.0 2026-06-10 05:52:09.420884 | controller | go: downloading github.com/mailru/easyjson v0.7.7 2026-06-10 05:52:09.564476 | controller | go: downloading github.com/josharian/intern v1.0.0 2026-06-10 05:52:46.141727 | controller | 2026-06-10T05:52:46-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 05:52:46.141872 | controller | 2026-06-10 05:52:46.149967 | controller | 2026-06-10T05:52:46-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 05:52:46.150011 | controller | 2026-06-10 05:52:46.445243 | controller | 2026-06-10T05:52:46-04:00 INFO Creating "admin-ssh-key" in sf 2026-06-10 05:52:46.448534 | controller | 2026-06-10T05:52:46-04:00 INFO "admin-ssh-key" created 2026-06-10 05:52:46.649078 | controller | 2026-06-10T05:52:46-04:00 INFO Creating "zuul-ssh-key" in sf 2026-06-10 05:52:46.651873 | controller | 2026-06-10T05:52:46-04:00 INFO "zuul-ssh-key" created 2026-06-10 05:52:46.653089 | controller | 2026-06-10T05:52:46-04:00 INFO Creating "gerrit-admin-api-key" in sf 2026-06-10 05:52:46.654655 | controller | 2026-06-10T05:52:46-04:00 INFO "gerrit-admin-api-key" created 2026-06-10 05:52:46.655768 | controller | 2026-06-10T05:52:46-04:00 INFO Creating "zuul-gerrit-api-key" in sf 2026-06-10 05:52:46.657415 | controller | 2026-06-10T05:52:46-04:00 INFO "zuul-gerrit-api-key" created 2026-06-10 05:52:46.658757 | controller | 2026-06-10T05:52:46-04:00 INFO Creating "gerrit-httpd" in sf 2026-06-10 05:52:46.663641 | controller | 2026-06-10T05:52:46-04:00 INFO "gerrit-httpd" created 2026-06-10 05:52:46.666525 | controller | 2026-06-10T05:52:46-04:00 INFO Creating "gerrit-sshd" in sf 2026-06-10 05:52:46.672540 | controller | 2026-06-10T05:52:46-04:00 INFO "gerrit-sshd" created 2026-06-10 05:52:46.674712 | controller | 2026-06-10T05:52:46-04:00 INFO managesf-resources 2026-06-10 05:52:46.674750 | controller | 2026-06-10T05:52:46-04:00 INFO Creating "managesf-resources-config-map" in sf 2026-06-10 05:52:46.676989 | controller | 2026-06-10T05:52:46-04:00 INFO "managesf-resources-config-map" created 2026-06-10 05:52:46.678347 | controller | 2026-06-10T05:52:46-04:00 INFO managesf-resources-tooling 2026-06-10 05:52:46.678363 | controller | 2026-06-10T05:52:46-04:00 INFO Creating "managesf-resources-tooling-config-map" in sf 2026-06-10 05:52:46.681941 | controller | 2026-06-10T05:52:46-04:00 INFO "managesf-resources-tooling-config-map" created 2026-06-10 05:52:46.685526 | controller | 2026-06-10T05:52:46-04:00 INFO Creating "gerrit" in sf 2026-06-10 05:52:46.690277 | controller | 2026-06-10T05:52:46-04:00 INFO "gerrit" created 2026-06-10 05:52:46.692138 | controller | 2026-06-10T05:52:46-04:00 INFO Waiting 10s for gerrit statefulset to be ready... 2026-06-10 05:52:56.701978 | controller | 2026-06-10T05:52:56-04:00 INFO Waiting 10s for gerrit statefulset to be ready... 2026-06-10 05:53:06.705440 | controller | 2026-06-10T05:53:06-04:00 INFO Waiting 10s for gerrit statefulset to be ready... 2026-06-10 05:53:16.708262 | controller | 2026-06-10T05:53:16-04:00 INFO Waiting 10s for gerrit statefulset to be ready... 2026-06-10 05:53:26.718559 | controller | 2026-06-10T05:53:26-04:00 INFO Waiting 10s for gerrit statefulset to be ready... 2026-06-10 05:53:36.721360 | controller | 2026-06-10T05:53:36-04:00 INFO Waiting 10s for gerrit statefulset to be ready... 2026-06-10 05:53:46.731959 | controller | 2026-06-10T05:53:46-04:00 INFO Waiting 10s for gerrit statefulset to be ready... 2026-06-10 05:53:56.738692 | controller | 2026-06-10T05:53:56-04:00 INFO Creating "post-init" in sf 2026-06-10 05:53:56.741541 | controller | 2026-06-10T05:53:56-04:00 INFO "post-init" created 2026-06-10 05:54:14.838579 | controller | 2026-06-10T05:54:14-04:00 INFO Creating "self-signed-cert" in sf 2026-06-10 05:54:14.842866 | controller | 2026-06-10T05:54:14-04:00 INFO "self-signed-cert" created 2026-06-10 05:54:14.846355 | controller | 2026-06-10T05:54:14-04:00 INFO Creating "gerrit-ingress" in sf 2026-06-10 05:54:14.854528 | controller | 2026-06-10T05:54:14-04:00 INFO "gerrit-ingress" created 2026-06-10 05:54:14.857095 | controller | 2026-06-10T05:54:14-04:00 INFO Creating "sf-ingress" in sf 2026-06-10 05:54:14.862491 | controller | 2026-06-10T05:54:14-04:00 INFO "sf-ingress" created 2026-06-10 05:54:14.864585 | controller | 2026-06-10T05:54:14-04:00 INFO Waiting for gerrit-ingress ... [attempt 1/25] 2026-06-10 05:54:21.868343 | controller | 2026-06-10T05:54:21-04:00 INFO Waiting for gerrit-ingress ... [attempt 2/25] 2026-06-10 05:54:28.878673 | controller | 2026-06-10T05:54:28-04:00 INFO Updating /etc/hosts 2026-06-10 05:54:28.878815 | controller | 2026-06-10T05:54:28-04:00 INFO Making sure Gerrit is up and ready... 2026-06-10 05:54:28.878838 | controller | 2026-06-10T05:54:28-04:00 INFO Querying Gerrit projects endpoint... [attempt 1/10] 2026-06-10 05:54:28.903405 | controller | 2026-06-10T05:54:28-04:00 INFO Gerrit is up and available 2026-06-10 05:54:28.903464 | controller | 2026-06-10T05:54:28-04:00 INFO Cloning config... 2026-06-10 05:54:28.905739 | controller | 2026-06-10T05:54:28-04:00 INFO Cloning repo https://admin:284faa8a-e398-4dd4-8979-e667575a825f@gerrit.sfop.me/a/config in deploy/config 2026-06-10 05:54:29.104366 | controller | 2026-06-10T05:54:29-04:00 INFO Configuring local repository for commits... 2026-06-10 05:54:29.112062 | controller | 2026-06-10T05:54:29-04:00 INFO Cloning demo-tenant-config... 2026-06-10 05:54:29.114080 | controller | 2026-06-10T05:54:29-04:00 INFO Cloning repo https://admin:284faa8a-e398-4dd4-8979-e667575a825f@gerrit.sfop.me/a/demo-tenant-config in deploy/demo-tenant-config 2026-06-10 05:54:29.208326 | controller | 2026-06-10T05:54:29-04:00 INFO Configuring local repository for commits... 2026-06-10 05:54:29.216451 | controller | 2026-06-10T05:54:29-04:00 INFO Cloning demo-project... 2026-06-10 05:54:29.218904 | controller | 2026-06-10T05:54:29-04:00 INFO Cloning repo https://admin:284faa8a-e398-4dd4-8979-e667575a825f@gerrit.sfop.me/a/demo-project in deploy/demo-project 2026-06-10 05:54:29.303220 | controller | 2026-06-10T05:54:29-04:00 INFO Configuring local repository for commits... 2026-06-10 05:54:29.309723 | controller | 2026-06-10T05:54:29-04:00 INFO Initialize demo-tenant's pipelines and jobs... 2026-06-10 05:54:29.317241 | controller | 2026-06-10T05:54:29-04:00 INFO Pushing local repo state to origin... 2026-06-10 05:54:29.413995 | controller | 2026-06-10T05:54:29-04:00 INFO Creating or updating tenant demo-tenant... 2026-06-10 05:54:29.418921 | controller | 2026-06-10T05:54:29-04:00 INFO Pushing local repo state to origin... 2026-06-10 05:54:29.514893 | controller | 2026-06-10T05:54:29-04:00 INFO Initialize demo-project's pipelines and jobs... 2026-06-10 05:54:29.520174 | controller | 2026-06-10T05:54:29-04:00 INFO Pushing local repo state to origin... 2026-06-10 05:54:29.613304 | controller | 2026-06-10T05:54:29-04:00 INFO Applying CRDs (did you run "make manifests" first?) 2026-06-10 05:54:29.891682 | controller | [+] Deploying sf-operator 2026-06-10 05:54:30.170288 | controller | 2026-06-10T05:54:30-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 05:54:30.170346 | controller | 2026-06-10 05:54:30.178778 | controller | 2026-06-10T05:54:30-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 05:54:30.178810 | controller | 2026-06-10 05:54:30.197872 | controller | 2026-06-10T05:54:30-04:00 INFO Deleting object, name: zookeeper 2026-06-10 05:54:30.201782 | controller | 2026-06-10T05:54:30-04:00 INFO Deleting object, name: ca-cert 2026-06-10 05:54:30.205778 | controller | 2026-06-10T05:54:30-04:00 INFO Creating secret, name: zuul-keystore-password 2026-06-10 05:54:30.205799 | controller | 2026-06-10T05:54:30-04:00 INFO Creating object, name: zuul-keystore-password 2026-06-10 05:54:30.208940 | controller | 2026-06-10T05:54:30-04:00 INFO Creating secret, name: zuul-auth-secret 2026-06-10 05:54:30.209076 | controller | 2026-06-10T05:54:30-04:00 INFO Creating object, name: zuul-auth-secret 2026-06-10 05:54:30.212727 | controller | 2026-06-10T05:54:30-04:00 INFO Creating config map name: zuul-scheduler-tooling-config-map 2026-06-10 05:54:30.212752 | controller | 2026-06-10T05:54:30-04:00 INFO Creating object, name: zuul-scheduler-tooling-config-map 2026-06-10 05:54:30.217957 | controller | 2026-06-10T05:54:30-04:00 INFO Creating config map name: git-server-pi-config-map 2026-06-10 05:54:30.218008 | controller | 2026-06-10T05:54:30-04:00 INFO Creating object, name: git-server-pi-config-map 2026-06-10 05:54:30.226559 | controller | 2026-06-10T05:54:30-04:00 INFO Creating object, name: git-server 2026-06-10 05:54:30.234114 | controller | 2026-06-10T05:54:30-04:00 INFO Creating secret, name: mariadb-root-password 2026-06-10 05:54:30.234154 | controller | 2026-06-10T05:54:30-04:00 INFO Creating object, name: mariadb-root-password 2026-06-10 05:54:30.240445 | controller | 2026-06-10T05:54:30-04:00 INFO Creating secret, name: mariadb-config-secrets 2026-06-10 05:54:30.240468 | controller | 2026-06-10T05:54:30-04:00 INFO Creating object, name: mariadb-config-secrets 2026-06-10 05:54:30.593106 | controller | 2026-06-10T05:54:30-04:00 INFO Creating secret, name: mariadb-initdb-secrets 2026-06-10 05:54:30.593211 | controller | 2026-06-10T05:54:30-04:00 INFO Creating object, name: mariadb-initdb-secrets 2026-06-10 05:54:30.795731 | controller | 2026-06-10T05:54:30-04:00 INFO Creating object, name: mariadb 2026-06-10 05:54:30.992292 | controller | 2026-06-10T05:54:30-04:00 INFO Creating ssh key, name: logserver-keys 2026-06-10 05:54:32.685215 | controller | 2026-06-10T05:54:32-04:00 INFO Creating object, name: logserver-keys 2026-06-10 05:54:32.692308 | controller | 2026-06-10T05:54:32-04:00 INFO Creating ssh key, name: logserver-uploader-keys 2026-06-10 05:54:33.226769 | controller | 2026-06-10T05:54:33-04:00 INFO Creating object, name: logserver-uploader-keys 2026-06-10 05:54:33.235396 | controller | 2026-06-10T05:54:33-04:00 INFO Creating ssh key, name: logserver-uploader-spare-keys 2026-06-10 05:54:34.223011 | controller | 2026-06-10T05:54:34-04:00 INFO Creating object, name: logserver-uploader-spare-keys 2026-06-10 05:54:34.227734 | controller | 2026-06-10T05:54:34-04:00 INFO Creating config map name: logserver-config-map 2026-06-10 05:54:34.227781 | controller | 2026-06-10T05:54:34-04:00 INFO Creating object, name: logserver-config-map 2026-06-10 05:54:34.231944 | controller | 2026-06-10T05:54:34-04:00 INFO Creating service, name: logserver 2026-06-10 05:54:34.231972 | controller | 2026-06-10T05:54:34-04:00 INFO Creating object, name: logserver 2026-06-10 05:54:34.243688 | controller | 2026-06-10T05:54:34-04:00 INFO Creating object, name: logserver 2026-06-10 05:54:34.249908 | controller | 2026-06-10T05:54:34-04:00 INFO Creating object, name: gateway 2026-06-10 05:54:34.258509 | controller | 2026-06-10T05:54:34-04:00 INFO Creating config map name: gateway-config-map 2026-06-10 05:54:34.258577 | controller | 2026-06-10T05:54:34-04:00 INFO Creating object, name: gateway-config-map 2026-06-10 05:54:34.264474 | controller | 2026-06-10T05:54:34-04:00 INFO Creating object, name: gateway 2026-06-10 05:54:34.276058 | controller | 2026-06-10T05:54:34-04:00 INFO Creating service, name: hound-search 2026-06-10 05:54:34.276133 | controller | 2026-06-10T05:54:34-04:00 INFO Creating object, name: hound-search 2026-06-10 05:54:34.305813 | controller | 2026-06-10T05:54:34-04:00 INFO Creating object, name: hound-search 2026-06-10 05:54:34.312318 | controller | 2026-06-10T05:54:34-04:00 INFO Creating object, name: logjuicer-pvc 2026-06-10 05:54:34.316738 | controller | 2026-06-10T05:54:34-04:00 INFO Creating object, name: logjuicer 2026-06-10 05:54:34.326947 | controller | 2026-06-10T05:54:34-04:00 INFO Creating object, name: logjuicer 2026-06-10 05:54:34.333885 | controller | 2026-06-10T05:54:34-04:00 INFO Deleting object, name: zookeeper-client-tls 2026-06-10 05:54:34.336024 | controller | 2026-06-10T05:54:34-04:00 INFO Deleting object, name: zookeeper-server-tls 2026-06-10 05:54:35.656349 | controller | 2026-06-10T05:54:35-04:00 INFO Creating object, name: zookeeper-client-tls 2026-06-10 05:54:37.489074 | controller | 2026-06-10T05:54:37-04:00 INFO Creating object, name: zookeeper-server-tls 2026-06-10 05:54:37.494343 | controller | 2026-06-10T05:54:37-04:00 INFO Creating config map name: zookeeper-pi-config-map 2026-06-10 05:54:37.494372 | controller | 2026-06-10T05:54:37-04:00 INFO Creating object, name: zookeeper-pi-config-map 2026-06-10 05:54:37.498486 | controller | 2026-06-10T05:54:37-04:00 INFO Creating service, name: zookeeper-headless 2026-06-10 05:54:37.498533 | controller | 2026-06-10T05:54:37-04:00 INFO Creating object, name: zookeeper-headless 2026-06-10 05:54:37.502243 | controller | 2026-06-10T05:54:37-04:00 INFO Creating service, name: zookeeper 2026-06-10 05:54:37.502270 | controller | 2026-06-10T05:54:37-04:00 INFO Creating object, name: zookeeper 2026-06-10 05:54:37.510000 | controller | 2026-06-10T05:54:37-04:00 INFO Creating object, name: zookeeper 2026-06-10 05:54:37.513958 | controller | 2026-06-10T05:54:37-04:00 INFO Waiting for Zookeeper service to be ready ... 2026-06-10 05:54:37.513983 | controller | 2026-06-10T05:54:37-04:00 INFO 2026-06-10 05:54:37.513988 | controller | - Config: Waiting ... 2026-06-10 05:54:37.513992 | controller | - Gateway: Waiting ... 2026-06-10 05:54:37.513995 | controller | - GitServer: Waiting ... 2026-06-10 05:54:37.513999 | controller | - HoundSearch: Waiting ... 2026-06-10 05:54:37.514002 | controller | - LogJuicer: Waiting ... 2026-06-10 05:54:37.514006 | controller | - Logserver: Waiting ... 2026-06-10 05:54:37.514009 | controller | - MariaDB: Waiting ... 2026-06-10 05:54:37.514013 | controller | - Zookeeper: Waiting ... 2026-06-10 05:54:37.514016 | controller | 2026-06-10 05:54:37.514020 | controller | 2026-06-10T05:54:37-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 05:54:42.522580 | controller | 2026-06-10T05:54:42-04:00 INFO Deleting object, name: ca-cert 2026-06-10 05:54:42.538941 | controller | 2026-06-10T05:54:42-04:00 INFO Creating service, name: git-server 2026-06-10 05:54:42.538989 | controller | 2026-06-10T05:54:42-04:00 INFO Creating object, name: git-server 2026-06-10 05:54:42.548589 | controller | 2026-06-10T05:54:42-04:00 INFO Creating service, name: git-server-rw 2026-06-10 05:54:42.548670 | controller | 2026-06-10T05:54:42-04:00 INFO Creating object, name: git-server-rw 2026-06-10 05:54:42.560766 | controller | 2026-06-10T05:54:42-04:00 INFO Waiting for statefulset, name: git-server 2026-06-10 05:54:52.603889 | controller | 2026-06-10T05:54:52-04:00 INFO Creating service, name: mariadb 2026-06-10 05:54:52.603964 | controller | 2026-06-10T05:54:52-04:00 INFO Creating object, name: mariadb 2026-06-10 05:54:52.696877 | controller | 2026-06-10T05:54:52-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-10 05:55:12.754519 | controller | 2026-06-10T05:55:12-04:00 INFO Creating object, name: zuul-db-connection 2026-06-10 05:55:12.760971 | controller | 2026-06-10T05:55:12-04:00 INFO Creating object, name: zuul-db-provision 2026-06-10 05:55:12.790864 | controller | 2026-06-10T05:55:12-04:00 INFO Waiting for statefulset, name: logserver 2026-06-10 05:55:16.805839 | controller | 2026-06-10T05:55:16-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 05:55:16.880305 | controller | 2026-06-10T05:55:16-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 05:55:16.885960 | controller | 2026-06-10T05:55:16-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 05:55:16.885991 | controller | 2026-06-10T05:55:16-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 05:55:16.892647 | controller | 2026-06-10T05:55:16-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-10 05:55:46.977281 | controller | 2026-06-10T05:55:46-04:00 INFO Statefulset "hound-search" not ready after 30s 2026-06-10 05:55:46.984817 | controller | 2026-06-10T05:55:46-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 05:55:46.984867 | controller | 2026-06-10T05:55:46-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 05:55:47.001872 | controller | 2026-06-10T05:55:47-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-10 05:55:47.001944 | controller | 2026-06-10T05:55:47-04:00 INFO 2026-06-10 05:55:47.001951 | controller | - Config: Waiting ... 2026-06-10 05:55:47.001956 | controller | - Gateway: OK 2026-06-10 05:55:47.001961 | controller | - GitServer: OK 2026-06-10 05:55:47.001965 | controller | - HoundSearch: Waiting ... 2026-06-10 05:55:47.001969 | controller | - LogJuicer: OK 2026-06-10 05:55:47.001973 | controller | - Logserver: OK 2026-06-10 05:55:47.001977 | controller | - MariaDB: Waiting ... 2026-06-10 05:55:47.001980 | controller | - Zookeeper: OK 2026-06-10 05:55:47.001984 | controller | 2026-06-10 05:55:47.001988 | controller | 2026-06-10T05:55:47-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 05:55:52.012310 | controller | 2026-06-10T05:55:52-04:00 INFO Deleting object, name: ca-cert 2026-06-10 05:55:52.626824 | controller | 2026-06-10T05:55:52-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 05:55:52.626896 | controller | 2026-06-10T05:55:52-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 05:55:52.634134 | controller | 2026-06-10T05:55:52-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-10 05:56:22.731788 | controller | 2026-06-10T05:56:22-04:00 INFO Statefulset "hound-search" not ready after 30s 2026-06-10 05:56:22.738515 | controller | 2026-06-10T05:56:22-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 05:56:22.738538 | controller | 2026-06-10T05:56:22-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 05:56:22.750258 | controller | 2026-06-10T05:56:22-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 05:56:22.753079 | controller | 2026-06-10T05:56:22-04:00 INFO Creating ssh key, name: zuul-spare-ssh-key 2026-06-10 05:56:23.049585 | controller | 2026-06-10T05:56:23-04:00 INFO Creating object, name: zuul-spare-ssh-key 2026-06-10 05:56:23.055153 | controller | 2026-06-10T05:56:23-04:00 INFO Creating config map name: np-statsd-config-map 2026-06-10 05:56:23.055200 | controller | 2026-06-10T05:56:23-04:00 INFO Creating object, name: np-statsd-config-map 2026-06-10 05:56:23.059175 | controller | 2026-06-10T05:56:23-04:00 INFO Creating config map name: nodepool-tooling-config-map 2026-06-10 05:56:23.059211 | controller | 2026-06-10T05:56:23-04:00 INFO Creating object, name: nodepool-tooling-config-map 2026-06-10 05:56:23.062339 | controller | 2026-06-10T05:56:23-04:00 INFO Creating config map name: nodepool-launcher-extra-config-config-map 2026-06-10 05:56:23.062365 | controller | 2026-06-10T05:56:23-04:00 INFO Creating object, name: nodepool-launcher-extra-config-config-map 2026-06-10 05:56:23.067340 | controller | 2026-06-10T05:56:23-04:00 INFO Creating object, name: nodepool-launcher 2026-06-10 05:56:23.073226 | controller | 2026-06-10T05:56:23-04:00 INFO Creating ssh key, name: nodepool-builder-ssh-key 2026-06-10 05:56:23.361481 | controller | 2026-06-10T05:56:23-04:00 INFO Creating object, name: nodepool-builder-ssh-key 2026-06-10 05:56:23.366693 | controller | 2026-06-10T05:56:23-04:00 INFO Creating config map name: nodepool-builder-extra-config-config-map 2026-06-10 05:56:23.366743 | controller | 2026-06-10T05:56:23-04:00 INFO Creating object, name: nodepool-builder-extra-config-config-map 2026-06-10 05:56:23.372544 | controller | 2026-06-10T05:56:23-04:00 INFO Creating service, name: nodepool-builder 2026-06-10 05:56:23.372621 | controller | 2026-06-10T05:56:23-04:00 INFO Creating object, name: nodepool-builder 2026-06-10 05:56:23.378571 | controller | 2026-06-10T05:56:23-04:00 INFO Creating object, name: nodepool-builder 2026-06-10 05:56:23.389743 | controller | 2026-06-10T05:56:23-04:00 INFO Creating secret, name: zuul-config 2026-06-10 05:56:23.389810 | controller | 2026-06-10T05:56:23-04:00 INFO Creating object, name: zuul-config 2026-06-10 05:56:23.393385 | controller | 2026-06-10T05:56:23-04:00 INFO Creating object, name: zuul-web 2026-06-10 05:56:23.405929 | controller | 2026-06-10T05:56:23-04:00 INFO Creating object, name: zuul-executor-headless 2026-06-10 05:56:23.738481 | controller | 2026-06-10T05:56:23-04:00 INFO Creating config map name: zuul-extra-config-map 2026-06-10 05:56:23.738547 | controller | 2026-06-10T05:56:23-04:00 INFO Creating object, name: zuul-extra-config-map 2026-06-10 05:56:24.138308 | controller | 2026-06-10T05:56:24-04:00 INFO Creating config map name: zuul-statsd-config-map 2026-06-10 05:56:24.138394 | controller | 2026-06-10T05:56:24-04:00 INFO Creating object, name: zuul-statsd-config-map 2026-06-10 05:56:24.538694 | controller | 2026-06-10T05:56:24-04:00 INFO Creating config map name: zuul-logging-config-map 2026-06-10 05:56:24.538752 | controller | 2026-06-10T05:56:24-04:00 INFO Creating object, name: zuul-logging-config-map 2026-06-10 05:56:24.740089 | controller | 2026-06-10T05:56:24-04:00 INFO Creating object, name: zuul-scheduler 2026-06-10 05:56:25.539883 | controller | 2026-06-10T05:56:25-04:00 INFO Creating object, name: zuul-executor 2026-06-10 05:56:26.340428 | controller | 2026-06-10T05:56:26-04:00 INFO Creating object, name: zuul-web 2026-06-10 05:56:27.141106 | controller | 2026-06-10T05:56:27-04:00 INFO Creating object, name: zuul-merger 2026-06-10 05:56:27.144479 | controller | 2026-06-10T05:56:27-04:00 INFO 2026-06-10 05:56:27.144524 | controller | - Config: Waiting ... 2026-06-10 05:56:27.144531 | controller | - Gateway: OK 2026-06-10 05:56:27.144535 | controller | - GitServer: OK 2026-06-10 05:56:27.144538 | controller | - HoundSearch: Waiting ... 2026-06-10 05:56:27.144542 | controller | - LogJuicer: OK 2026-06-10 05:56:27.144545 | controller | - Logserver: OK 2026-06-10 05:56:27.144549 | controller | - MariaDB: OK 2026-06-10 05:56:27.144552 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 05:56:27.144556 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 05:56:27.144560 | controller | - Zookeeper: OK 2026-06-10 05:56:27.144563 | controller | - Zuul: Waiting ... 2026-06-10 05:56:27.144567 | controller | - |_ Config Secrets: OK 2026-06-10 05:56:27.144570 | controller | - |_ Executor: Waiting ... 2026-06-10 05:56:27.144574 | controller | - |_ Merger: Waiting ... 2026-06-10 05:56:27.144577 | controller | - |_ Scheduler: Waiting ... 2026-06-10 05:56:27.144580 | controller | - |_ Web: Waiting ... 2026-06-10 05:56:27.144584 | controller | 2026-06-10 05:56:27.144592 | controller | 2026-06-10T05:56:27-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 05:56:32.149290 | controller | 2026-06-10T05:56:32-04:00 INFO Deleting object, name: ca-cert 2026-06-10 05:56:32.759528 | controller | 2026-06-10T05:56:32-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 05:56:32.759583 | controller | 2026-06-10T05:56:32-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 05:56:32.766176 | controller | 2026-06-10T05:56:32-04:00 INFO Waiting for statefulset, name: hound-search 2026-06-10 05:57:02.864226 | controller | 2026-06-10T05:57:02-04:00 INFO Statefulset "hound-search" not ready after 30s 2026-06-10 05:57:02.874230 | controller | 2026-06-10T05:57:02-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 05:57:02.874317 | controller | 2026-06-10T05:57:02-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 05:57:02.892364 | controller | 2026-06-10T05:57:02-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 05:57:02.910569 | controller | 2026-06-10T05:57:02-04:00 INFO Creating object, name: nodepool-launcher 2026-06-10 05:57:02.917436 | controller | 2026-06-10T05:57:02-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 05:57:02.917505 | controller | 2026-06-10T05:57:02-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 05:57:03.479640 | controller | 2026-06-10T05:57:03-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 05:57:33.604915 | controller | 2026-06-10T05:57:33-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 05:57:33.620143 | controller | 2026-06-10T05:57:33-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-10 05:57:57.731441 | controller | 2026-06-10T05:57:57-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 05:57:57.731526 | controller | 2026-06-10T05:57:57-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 05:57:57.745620 | controller | 2026-06-10T05:57:57-04:00 INFO 2026-06-10 05:57:57.745689 | controller | - Config: Waiting ... 2026-06-10 05:57:57.745706 | controller | - Gateway: OK 2026-06-10 05:57:57.745712 | controller | - GitServer: OK 2026-06-10 05:57:57.745717 | controller | - HoundSearch: Waiting ... 2026-06-10 05:57:57.745722 | controller | - LogJuicer: OK 2026-06-10 05:57:57.745727 | controller | - Logserver: OK 2026-06-10 05:57:57.745732 | controller | - MariaDB: OK 2026-06-10 05:57:57.745738 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 05:57:57.745743 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 05:57:57.745749 | controller | - Zookeeper: OK 2026-06-10 05:57:57.745755 | controller | - Zuul: Waiting ... 2026-06-10 05:57:57.745760 | controller | - |_ Config Secrets: OK 2026-06-10 05:57:57.745777 | controller | - |_ Executor: OK 2026-06-10 05:57:57.745782 | controller | - |_ Merger: OK 2026-06-10 05:57:57.745787 | controller | - |_ Scheduler: Waiting ... 2026-06-10 05:57:57.745793 | controller | - |_ Web: Waiting ... 2026-06-10 05:57:57.745798 | controller | 2026-06-10 05:57:57.745810 | controller | 2026-06-10T05:57:57-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-10 05:58:02.750710 | controller | 2026-06-10T05:58:02-04:00 INFO Deleting object, name: ca-cert 2026-06-10 05:58:03.370053 | controller | 2026-06-10T05:58:03-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 05:58:03.370104 | controller | 2026-06-10T05:58:03-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 05:58:03.393748 | controller | 2026-06-10T05:58:03-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 05:58:03.393816 | controller | 2026-06-10T05:58:03-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 05:58:03.767707 | controller | 2026-06-10T05:58:03-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 05:58:04.160043 | controller | 2026-06-10T05:58:04-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 05:58:04.160114 | controller | 2026-06-10T05:58:04-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 05:58:05.168311 | controller | 2026-06-10T05:58:05-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 05:58:27.248576 | controller | 2026-06-10T05:58:27-04:00 INFO Creating object, name: zuul-weeder 2026-06-10 05:58:27.257332 | controller | 2026-06-10T05:58:27-04:00 INFO Creating object, name: zuul-weeder 2026-06-10 05:58:27.292475 | controller | 2026-06-10T05:58:27-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 05:58:27.292553 | controller | 2026-06-10T05:58:27-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 05:58:27.676019 | controller | 2026-06-10T05:58:27-04:00 INFO 2026-06-10 05:58:27.676085 | controller | - Config: Waiting ... 2026-06-10 05:58:27.676090 | controller | - Gateway: OK 2026-06-10 05:58:27.676095 | controller | - GitServer: OK 2026-06-10 05:58:27.676099 | controller | - HoundSearch: OK 2026-06-10 05:58:27.676102 | controller | - LogJuicer: OK 2026-06-10 05:58:27.676106 | controller | - Logserver: OK 2026-06-10 05:58:27.676109 | controller | - MariaDB: OK 2026-06-10 05:58:27.676113 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 05:58:27.676117 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 05:58:27.676120 | controller | - Zookeeper: OK 2026-06-10 05:58:27.676124 | controller | - Zuul: Waiting ... 2026-06-10 05:58:27.676127 | controller | - |_ Config Secrets: OK 2026-06-10 05:58:27.676131 | controller | - |_ Executor: OK 2026-06-10 05:58:27.676134 | controller | - |_ Merger: OK 2026-06-10 05:58:27.676138 | controller | - |_ Scheduler: Waiting ... 2026-06-10 05:58:27.676142 | controller | - |_ Web: OK 2026-06-10 05:58:27.676145 | controller | 2026-06-10 05:58:27.676188 | controller | 2026-06-10T05:58:27-04:00 INFO [attempt #5] Waiting 5s for the next reconcile call ... 2026-06-10 05:58:32.685075 | controller | 2026-06-10T05:58:32-04:00 INFO Deleting object, name: ca-cert 2026-06-10 05:58:33.296581 | controller | 2026-06-10T05:58:33-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 05:58:33.296669 | controller | 2026-06-10T05:58:33-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 05:58:33.308698 | controller | 2026-06-10T05:58:33-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 05:58:33.308744 | controller | 2026-06-10T05:58:33-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 05:58:33.702446 | controller | 2026-06-10T05:58:33-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 05:58:34.092493 | controller | 2026-06-10T05:58:34-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 05:58:34.092591 | controller | 2026-06-10T05:58:34-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 05:58:35.103158 | controller | 2026-06-10T05:58:35-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 05:58:35.103213 | controller | 2026-06-10T05:58:35-04:00 INFO deployment rollout not ok, name: zuul-weeder 2026-06-10 05:58:36.086871 | controller | 2026-06-10T05:58:36-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 05:58:36.086929 | controller | 2026-06-10T05:58:36-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 05:58:36.893157 | controller | 2026-06-10T05:58:36-04:00 INFO 2026-06-10 05:58:36.893202 | controller | - Config: Waiting ... 2026-06-10 05:58:36.893213 | controller | - Gateway: OK 2026-06-10 05:58:36.893217 | controller | - GitServer: OK 2026-06-10 05:58:36.893221 | controller | - HoundSearch: OK 2026-06-10 05:58:36.893225 | controller | - LogJuicer: OK 2026-06-10 05:58:36.893228 | controller | - Logserver: OK 2026-06-10 05:58:36.893232 | controller | - MariaDB: OK 2026-06-10 05:58:36.893235 | controller | - NodePoolBuilder: OK 2026-06-10 05:58:36.893239 | controller | - NodePoolLauncher: OK 2026-06-10 05:58:36.893242 | controller | - Zookeeper: OK 2026-06-10 05:58:36.893246 | controller | - Zuul: Waiting ... 2026-06-10 05:58:36.893249 | controller | - |_ Config Secrets: OK 2026-06-10 05:58:36.893253 | controller | - |_ Executor: OK 2026-06-10 05:58:36.893257 | controller | - |_ Merger: OK 2026-06-10 05:58:36.893262 | controller | - |_ Scheduler: Waiting ... 2026-06-10 05:58:36.893266 | controller | - |_ Web: OK 2026-06-10 05:58:36.893271 | controller | 2026-06-10 05:58:36.893306 | controller | 2026-06-10T05:58:36-04:00 INFO [attempt #6] Waiting 5s for the next reconcile call ... 2026-06-10 05:58:41.900700 | controller | 2026-06-10T05:58:41-04:00 INFO Deleting object, name: ca-cert 2026-06-10 05:58:42.509845 | controller | 2026-06-10T05:58:42-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 05:58:42.509921 | controller | 2026-06-10T05:58:42-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 05:58:42.524993 | controller | 2026-06-10T05:58:42-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 05:58:42.525083 | controller | 2026-06-10T05:58:42-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 05:58:42.911965 | controller | 2026-06-10T05:58:42-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 05:58:43.310149 | controller | 2026-06-10T05:58:43-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 05:58:43.310252 | controller | 2026-06-10T05:58:43-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 05:58:44.319720 | controller | 2026-06-10T05:58:44-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 05:58:44.319779 | controller | 2026-06-10T05:58:44-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 05:58:45.302844 | controller | 2026-06-10T05:58:45-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 05:58:45.302919 | controller | 2026-06-10T05:58:45-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 05:58:46.300847 | controller | 2026-06-10T05:58:46-04:00 INFO Creating config map name: pymod-sf-operator-config-map 2026-06-10 05:58:46.300908 | controller | 2026-06-10T05:58:46-04:00 INFO Creating object, name: pymod-sf-operator-config-map 2026-06-10 05:58:46.703489 | controller | 2026-06-10T05:58:46-04:00 INFO Creating object, name: config-updater 2026-06-10 05:58:46.710918 | controller | 2026-06-10T05:58:46-04:00 INFO Creating object, name: config-updater-role 2026-06-10 05:58:46.715892 | controller | 2026-06-10T05:58:46-04:00 INFO Creating object, name: config-updater 2026-06-10 05:58:46.720429 | controller | 2026-06-10T05:58:46-04:00 INFO Creating the config-update service account secret 2026-06-10 05:58:46.720500 | controller | 2026-06-10T05:58:46-04:00 INFO Creating object, name: config-update-secrets 2026-06-10 05:58:46.724729 | controller | 2026-06-10T05:58:46-04:00 INFO Creating base secret job 2026-06-10 05:58:46.725075 | controller | 2026-06-10T05:58:46-04:00 INFO Creating object, name: config-base-secret-1-859-ns 2026-06-10 05:58:46.728046 | controller | 2026-06-10T05:58:46-04:00 INFO 2026-06-10 05:58:46.728069 | controller | - Config: Waiting ... 2026-06-10 05:58:46.728075 | controller | - Gateway: OK 2026-06-10 05:58:46.728080 | controller | - GitServer: OK 2026-06-10 05:58:46.728084 | controller | - HoundSearch: OK 2026-06-10 05:58:46.728089 | controller | - LogJuicer: OK 2026-06-10 05:58:46.728093 | controller | - Logserver: OK 2026-06-10 05:58:46.728098 | controller | - MariaDB: OK 2026-06-10 05:58:46.728102 | controller | - NodePoolBuilder: OK 2026-06-10 05:58:46.728106 | controller | - NodePoolLauncher: OK 2026-06-10 05:58:46.728111 | controller | - Zookeeper: OK 2026-06-10 05:58:46.728121 | controller | - Zuul: OK 2026-06-10 05:58:46.728130 | controller | 2026-06-10 05:58:46.728134 | controller | 2026-06-10T05:58:46-04:00 INFO [attempt #7] Waiting 5s for the next reconcile call ... 2026-06-10 05:58:51.736538 | controller | 2026-06-10T05:58:51-04:00 INFO Deleting object, name: ca-cert 2026-06-10 05:58:52.348406 | controller | 2026-06-10T05:58:52-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 05:58:52.348466 | controller | 2026-06-10T05:58:52-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 05:58:52.362021 | controller | 2026-06-10T05:58:52-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 05:58:52.362082 | controller | 2026-06-10T05:58:52-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 05:58:52.749806 | controller | 2026-06-10T05:58:52-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 05:58:53.156738 | controller | 2026-06-10T05:58:53-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 05:58:53.156846 | controller | 2026-06-10T05:58:53-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 05:58:54.162899 | controller | 2026-06-10T05:58:54-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 05:58:54.162974 | controller | 2026-06-10T05:58:54-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 05:58:55.142216 | controller | 2026-06-10T05:58:55-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 05:58:55.142360 | controller | 2026-06-10T05:58:55-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 05:58:56.344531 | controller | 2026-06-10T05:58:56-04:00 INFO Running tenant-reconfigure for the 'internal' tenant 2026-06-10 05:58:56.537544 | controller | 2026-06-10T05:58:56-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [zuul-scheduler tenant-reconfigure internal] 2026-06-10 05:59:02.569846 | controller | 2026-06-10T05:59:02-04:00 INFO Deleting object, name: zuul-web 2026-06-10 05:59:02.573914 | controller | 2026-06-10T05:59:02-04:00 INFO Creating object, name: zs-internal-tenant-reconfigure 2026-06-10 05:59:02.576627 | controller | 2026-06-10T05:59:02-04:00 INFO 2026-06-10 05:59:02.576645 | controller | - Config: Waiting ... 2026-06-10 05:59:02.576650 | controller | - Gateway: OK 2026-06-10 05:59:02.576654 | controller | - GitServer: OK 2026-06-10 05:59:02.576657 | controller | - HoundSearch: OK 2026-06-10 05:59:02.576661 | controller | - LogJuicer: OK 2026-06-10 05:59:02.576665 | controller | - Logserver: OK 2026-06-10 05:59:02.576668 | controller | - MariaDB: OK 2026-06-10 05:59:02.576671 | controller | - NodePoolBuilder: OK 2026-06-10 05:59:02.576675 | controller | - NodePoolLauncher: OK 2026-06-10 05:59:02.576678 | controller | - Zookeeper: OK 2026-06-10 05:59:02.576682 | controller | - Zuul: OK 2026-06-10 05:59:02.576686 | controller | 2026-06-10 05:59:02.576693 | controller | 2026-06-10T05:59:02-04:00 INFO [attempt #8] Waiting 5s for the next reconcile call ... 2026-06-10 05:59:07.579897 | controller | 2026-06-10T05:59:07-04:00 INFO Deleting object, name: ca-cert 2026-06-10 05:59:08.198852 | controller | 2026-06-10T05:59:08-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 05:59:08.198916 | controller | 2026-06-10T05:59:08-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 05:59:08.215622 | controller | 2026-06-10T05:59:08-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 05:59:08.215689 | controller | 2026-06-10T05:59:08-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 05:59:08.589406 | controller | 2026-06-10T05:59:08-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 05:59:08.991335 | controller | 2026-06-10T05:59:08-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 05:59:08.991400 | controller | 2026-06-10T05:59:08-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 05:59:09.998230 | controller | 2026-06-10T05:59:09-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 05:59:09.998311 | controller | 2026-06-10T05:59:09-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 05:59:10.984484 | controller | 2026-06-10T05:59:10-04:00 INFO Creating object, name: zuul-web 2026-06-10 05:59:11.796076 | controller | 2026-06-10T05:59:11-04:00 INFO 2026-06-10 05:59:11.796176 | controller | - Config: Waiting ... 2026-06-10 05:59:11.796184 | controller | - Gateway: OK 2026-06-10 05:59:11.796190 | controller | - GitServer: OK 2026-06-10 05:59:11.796196 | controller | - HoundSearch: OK 2026-06-10 05:59:11.796202 | controller | - LogJuicer: OK 2026-06-10 05:59:11.796207 | controller | - Logserver: OK 2026-06-10 05:59:11.796212 | controller | - MariaDB: OK 2026-06-10 05:59:11.796217 | controller | - NodePoolBuilder: OK 2026-06-10 05:59:11.796222 | controller | - NodePoolLauncher: OK 2026-06-10 05:59:11.796227 | controller | - Zookeeper: OK 2026-06-10 05:59:11.796233 | controller | - Zuul: Waiting ... 2026-06-10 05:59:11.796238 | controller | - |_ Config Secrets: OK 2026-06-10 05:59:11.796243 | controller | - |_ Executor: OK 2026-06-10 05:59:11.796248 | controller | - |_ Merger: OK 2026-06-10 05:59:11.796253 | controller | - |_ Scheduler: OK 2026-06-10 05:59:11.796280 | controller | - |_ Web: Waiting ... 2026-06-10 05:59:11.796286 | controller | 2026-06-10 05:59:11.796292 | controller | 2026-06-10T05:59:11-04:00 INFO [attempt #9] Waiting 5s for the next reconcile call ... 2026-06-10 05:59:16.804668 | controller | 2026-06-10T05:59:16-04:00 INFO Deleting object, name: ca-cert 2026-06-10 05:59:17.414154 | controller | 2026-06-10T05:59:17-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 05:59:17.414216 | controller | 2026-06-10T05:59:17-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 05:59:17.425137 | controller | 2026-06-10T05:59:17-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 05:59:17.425185 | controller | 2026-06-10T05:59:17-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 05:59:17.823798 | controller | 2026-06-10T05:59:17-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 05:59:18.212292 | controller | 2026-06-10T05:59:18-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 05:59:18.212354 | controller | 2026-06-10T05:59:18-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 05:59:19.222775 | controller | 2026-06-10T05:59:19-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 05:59:19.222862 | controller | 2026-06-10T05:59:19-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 05:59:20.209775 | controller | 2026-06-10T05:59:20-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 05:59:20.209835 | controller | 2026-06-10T05:59:20-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 05:59:21.018986 | controller | 2026-06-10T05:59:21-04:00 INFO 2026-06-10 05:59:21.019067 | controller | - Config: Waiting ... 2026-06-10 05:59:21.019083 | controller | - Gateway: OK 2026-06-10 05:59:21.019096 | controller | - GitServer: OK 2026-06-10 05:59:21.019108 | controller | - HoundSearch: OK 2026-06-10 05:59:21.019119 | controller | - LogJuicer: OK 2026-06-10 05:59:21.019130 | controller | - Logserver: OK 2026-06-10 05:59:21.019141 | controller | - MariaDB: OK 2026-06-10 05:59:21.019152 | controller | - NodePoolBuilder: OK 2026-06-10 05:59:21.019163 | controller | - NodePoolLauncher: OK 2026-06-10 05:59:21.019175 | controller | - Zookeeper: OK 2026-06-10 05:59:21.019186 | controller | - Zuul: Waiting ... 2026-06-10 05:59:21.019197 | controller | - |_ Config Secrets: OK 2026-06-10 05:59:21.019208 | controller | - |_ Executor: OK 2026-06-10 05:59:21.019220 | controller | - |_ Merger: OK 2026-06-10 05:59:21.019231 | controller | - |_ Scheduler: OK 2026-06-10 05:59:21.019243 | controller | - |_ Web: Waiting ... 2026-06-10 05:59:21.019254 | controller | 2026-06-10 05:59:21.019323 | controller | 2026-06-10T05:59:21-04:00 INFO [attempt #10] Waiting 5s for the next reconcile call ... 2026-06-10 05:59:26.026593 | controller | 2026-06-10T05:59:26-04:00 INFO Deleting object, name: ca-cert 2026-06-10 05:59:26.648114 | controller | 2026-06-10T05:59:26-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 05:59:26.648203 | controller | 2026-06-10T05:59:26-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 05:59:26.670170 | controller | 2026-06-10T05:59:26-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 05:59:26.670295 | controller | 2026-06-10T05:59:26-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 05:59:27.037695 | controller | 2026-06-10T05:59:27-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 05:59:27.434969 | controller | 2026-06-10T05:59:27-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 05:59:27.435033 | controller | 2026-06-10T05:59:27-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 05:59:28.445129 | controller | 2026-06-10T05:59:28-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 05:59:28.445210 | controller | 2026-06-10T05:59:28-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 05:59:29.430040 | controller | 2026-06-10T05:59:29-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 05:59:29.430108 | controller | 2026-06-10T05:59:29-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 05:59:30.234888 | controller | 2026-06-10T05:59:30-04:00 INFO 2026-06-10 05:59:30.234952 | controller | - Config: Waiting ... 2026-06-10 05:59:30.234959 | controller | - Gateway: OK 2026-06-10 05:59:30.234964 | controller | - GitServer: OK 2026-06-10 05:59:30.234969 | controller | - HoundSearch: OK 2026-06-10 05:59:30.234974 | controller | - LogJuicer: OK 2026-06-10 05:59:30.234979 | controller | - Logserver: OK 2026-06-10 05:59:30.234984 | controller | - MariaDB: OK 2026-06-10 05:59:30.235003 | controller | - NodePoolBuilder: OK 2026-06-10 05:59:30.235008 | controller | - NodePoolLauncher: OK 2026-06-10 05:59:30.235013 | controller | - Zookeeper: OK 2026-06-10 05:59:30.235018 | controller | - Zuul: Waiting ... 2026-06-10 05:59:30.235023 | controller | - |_ Config Secrets: OK 2026-06-10 05:59:30.235028 | controller | - |_ Executor: OK 2026-06-10 05:59:30.235032 | controller | - |_ Merger: OK 2026-06-10 05:59:30.235037 | controller | - |_ Scheduler: OK 2026-06-10 05:59:30.235042 | controller | - |_ Web: Waiting ... 2026-06-10 05:59:30.235047 | controller | 2026-06-10 05:59:30.235052 | controller | 2026-06-10T05:59:30-04:00 INFO [attempt #11] Waiting 5s for the next reconcile call ... 2026-06-10 05:59:35.246760 | controller | 2026-06-10T05:59:35-04:00 INFO Deleting object, name: ca-cert 2026-06-10 05:59:35.859559 | controller | 2026-06-10T05:59:35-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 05:59:35.859652 | controller | 2026-06-10T05:59:35-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 05:59:35.873566 | controller | 2026-06-10T05:59:35-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 05:59:35.873641 | controller | 2026-06-10T05:59:35-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 05:59:36.256966 | controller | 2026-06-10T05:59:36-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 05:59:36.656542 | controller | 2026-06-10T05:59:36-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 05:59:36.656681 | controller | 2026-06-10T05:59:36-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 05:59:37.672830 | controller | 2026-06-10T05:59:37-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 05:59:37.672913 | controller | 2026-06-10T05:59:37-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 05:59:38.653180 | controller | 2026-06-10T05:59:38-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 05:59:38.653320 | controller | 2026-06-10T05:59:38-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 05:59:39.462312 | controller | 2026-06-10T05:59:39-04:00 INFO 2026-06-10 05:59:39.462380 | controller | - Config: Waiting ... 2026-06-10 05:59:39.462387 | controller | - Gateway: OK 2026-06-10 05:59:39.462392 | controller | - GitServer: OK 2026-06-10 05:59:39.462398 | controller | - HoundSearch: OK 2026-06-10 05:59:39.462403 | controller | - LogJuicer: OK 2026-06-10 05:59:39.462408 | controller | - Logserver: OK 2026-06-10 05:59:39.462413 | controller | - MariaDB: OK 2026-06-10 05:59:39.462419 | controller | - NodePoolBuilder: OK 2026-06-10 05:59:39.462424 | controller | - NodePoolLauncher: OK 2026-06-10 05:59:39.462429 | controller | - Zookeeper: OK 2026-06-10 05:59:39.462434 | controller | - Zuul: Waiting ... 2026-06-10 05:59:39.462439 | controller | - |_ Config Secrets: OK 2026-06-10 05:59:39.462445 | controller | - |_ Executor: OK 2026-06-10 05:59:39.462450 | controller | - |_ Merger: OK 2026-06-10 05:59:39.462455 | controller | - |_ Scheduler: OK 2026-06-10 05:59:39.462460 | controller | - |_ Web: Waiting ... 2026-06-10 05:59:39.462465 | controller | 2026-06-10 05:59:39.462501 | controller | 2026-06-10T05:59:39-04:00 INFO [attempt #12] Waiting 5s for the next reconcile call ... 2026-06-10 05:59:44.468351 | controller | 2026-06-10T05:59:44-04:00 INFO Deleting object, name: ca-cert 2026-06-10 05:59:45.086495 | controller | 2026-06-10T05:59:45-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 05:59:45.086554 | controller | 2026-06-10T05:59:45-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 05:59:45.104848 | controller | 2026-06-10T05:59:45-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 05:59:45.104910 | controller | 2026-06-10T05:59:45-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 05:59:45.484817 | controller | 2026-06-10T05:59:45-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 05:59:45.877295 | controller | 2026-06-10T05:59:45-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 05:59:45.877367 | controller | 2026-06-10T05:59:45-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 05:59:46.886191 | controller | 2026-06-10T05:59:46-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 05:59:46.886251 | controller | 2026-06-10T05:59:46-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 05:59:47.871322 | controller | 2026-06-10T05:59:47-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 05:59:47.871383 | controller | 2026-06-10T05:59:47-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 05:59:48.679326 | controller | 2026-06-10T05:59:48-04:00 INFO 2026-06-10 05:59:48.679397 | controller | - Config: Waiting ... 2026-06-10 05:59:48.679404 | controller | - Gateway: OK 2026-06-10 05:59:48.679410 | controller | - GitServer: OK 2026-06-10 05:59:48.679415 | controller | - HoundSearch: OK 2026-06-10 05:59:48.679420 | controller | - LogJuicer: OK 2026-06-10 05:59:48.679425 | controller | - Logserver: OK 2026-06-10 05:59:48.679430 | controller | - MariaDB: OK 2026-06-10 05:59:48.679435 | controller | - NodePoolBuilder: OK 2026-06-10 05:59:48.679440 | controller | - NodePoolLauncher: OK 2026-06-10 05:59:48.679445 | controller | - Zookeeper: OK 2026-06-10 05:59:48.679449 | controller | - Zuul: Waiting ... 2026-06-10 05:59:48.679454 | controller | - |_ Config Secrets: OK 2026-06-10 05:59:48.679459 | controller | - |_ Executor: OK 2026-06-10 05:59:48.679464 | controller | - |_ Merger: OK 2026-06-10 05:59:48.679469 | controller | - |_ Scheduler: OK 2026-06-10 05:59:48.679474 | controller | - |_ Web: Waiting ... 2026-06-10 05:59:48.679479 | controller | 2026-06-10 05:59:48.679485 | controller | 2026-06-10T05:59:48-04:00 INFO [attempt #13] Waiting 5s for the next reconcile call ... 2026-06-10 05:59:53.687699 | controller | 2026-06-10T05:59:53-04:00 INFO Deleting object, name: ca-cert 2026-06-10 05:59:54.304401 | controller | 2026-06-10T05:59:54-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 05:59:54.304471 | controller | 2026-06-10T05:59:54-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 05:59:54.318386 | controller | 2026-06-10T05:59:54-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 05:59:54.318444 | controller | 2026-06-10T05:59:54-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 05:59:54.697437 | controller | 2026-06-10T05:59:54-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 05:59:55.100483 | controller | 2026-06-10T05:59:55-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 05:59:55.100545 | controller | 2026-06-10T05:59:55-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 05:59:56.103803 | controller | 2026-06-10T05:59:56-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 05:59:56.103872 | controller | 2026-06-10T05:59:56-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 05:59:57.090429 | controller | 2026-06-10T05:59:57-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 05:59:57.090488 | controller | 2026-06-10T05:59:57-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 05:59:57.897145 | controller | 2026-06-10T05:59:57-04:00 INFO 2026-06-10 05:59:57.897205 | controller | - Config: Waiting ... 2026-06-10 05:59:57.897210 | controller | - Gateway: OK 2026-06-10 05:59:57.897216 | controller | - GitServer: OK 2026-06-10 05:59:57.897221 | controller | - HoundSearch: OK 2026-06-10 05:59:57.897227 | controller | - LogJuicer: OK 2026-06-10 05:59:57.897240 | controller | - Logserver: OK 2026-06-10 05:59:57.897246 | controller | - MariaDB: OK 2026-06-10 05:59:57.897250 | controller | - NodePoolBuilder: OK 2026-06-10 05:59:57.897255 | controller | - NodePoolLauncher: OK 2026-06-10 05:59:57.897260 | controller | - Zookeeper: OK 2026-06-10 05:59:57.897264 | controller | - Zuul: Waiting ... 2026-06-10 05:59:57.897269 | controller | - |_ Config Secrets: OK 2026-06-10 05:59:57.897289 | controller | - |_ Executor: OK 2026-06-10 05:59:57.897295 | controller | - |_ Merger: OK 2026-06-10 05:59:57.897299 | controller | - |_ Scheduler: OK 2026-06-10 05:59:57.897302 | controller | - |_ Web: Waiting ... 2026-06-10 05:59:57.897306 | controller | 2026-06-10 05:59:57.897310 | controller | 2026-06-10T05:59:57-04:00 INFO [attempt #14] Waiting 5s for the next reconcile call ... 2026-06-10 06:00:02.904685 | controller | 2026-06-10T06:00:02-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:00:03.519119 | controller | 2026-06-10T06:00:03-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:00:03.519186 | controller | 2026-06-10T06:00:03-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:00:03.536690 | controller | 2026-06-10T06:00:03-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:00:03.536755 | controller | 2026-06-10T06:00:03-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:00:03.912183 | controller | 2026-06-10T06:00:03-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:00:04.311726 | controller | 2026-06-10T06:00:04-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:00:04.311783 | controller | 2026-06-10T06:00:04-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:00:05.324024 | controller | 2026-06-10T06:00:05-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:00:05.324092 | controller | 2026-06-10T06:00:05-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:00:06.309079 | controller | 2026-06-10T06:00:06-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:00:06.309154 | controller | 2026-06-10T06:00:06-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:00:07.117951 | controller | 2026-06-10T06:00:07-04:00 INFO 2026-06-10 06:00:07.118036 | controller | - Config: Waiting ... 2026-06-10 06:00:07.118050 | controller | - Gateway: OK 2026-06-10 06:00:07.118062 | controller | - GitServer: OK 2026-06-10 06:00:07.118075 | controller | - HoundSearch: OK 2026-06-10 06:00:07.118086 | controller | - LogJuicer: OK 2026-06-10 06:00:07.118097 | controller | - Logserver: OK 2026-06-10 06:00:07.118108 | controller | - MariaDB: OK 2026-06-10 06:00:07.118119 | controller | - NodePoolBuilder: OK 2026-06-10 06:00:07.118130 | controller | - NodePoolLauncher: OK 2026-06-10 06:00:07.118141 | controller | - Zookeeper: OK 2026-06-10 06:00:07.118152 | controller | - Zuul: Waiting ... 2026-06-10 06:00:07.118163 | controller | - |_ Config Secrets: OK 2026-06-10 06:00:07.118175 | controller | - |_ Executor: OK 2026-06-10 06:00:07.118187 | controller | - |_ Merger: OK 2026-06-10 06:00:07.118198 | controller | - |_ Scheduler: OK 2026-06-10 06:00:07.118209 | controller | - |_ Web: Waiting ... 2026-06-10 06:00:07.118220 | controller | 2026-06-10 06:00:07.118283 | controller | 2026-06-10T06:00:07-04:00 INFO [attempt #15] Waiting 5s for the next reconcile call ... 2026-06-10 06:00:12.123051 | controller | 2026-06-10T06:00:12-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:00:12.743312 | controller | 2026-06-10T06:00:12-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:00:12.743430 | controller | 2026-06-10T06:00:12-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:00:12.763509 | controller | 2026-06-10T06:00:12-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:00:12.763573 | controller | 2026-06-10T06:00:12-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:00:13.133776 | controller | 2026-06-10T06:00:13-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:00:13.540462 | controller | 2026-06-10T06:00:13-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:00:13.540522 | controller | 2026-06-10T06:00:13-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:00:14.550356 | controller | 2026-06-10T06:00:14-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:00:14.550419 | controller | 2026-06-10T06:00:14-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:00:15.528935 | controller | 2026-06-10T06:00:15-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:00:15.528996 | controller | 2026-06-10T06:00:15-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:00:16.731152 | controller | 2026-06-10T06:00:16-04:00 INFO 2026-06-10 06:00:16.731217 | controller | - Config: OK 2026-06-10 06:00:16.731224 | controller | - Gateway: OK 2026-06-10 06:00:16.731230 | controller | - GitServer: OK 2026-06-10 06:00:16.731236 | controller | - HoundSearch: OK 2026-06-10 06:00:16.731241 | controller | - LogJuicer: OK 2026-06-10 06:00:16.731246 | controller | - Logserver: OK 2026-06-10 06:00:16.731251 | controller | - MariaDB: OK 2026-06-10 06:00:16.731257 | controller | - NodePoolBuilder: OK 2026-06-10 06:00:16.731262 | controller | - NodePoolLauncher: OK 2026-06-10 06:00:16.731267 | controller | - Zookeeper: OK 2026-06-10 06:00:16.731273 | controller | - Zuul: OK 2026-06-10 06:00:16.731278 | controller | 2026-06-10 06:00:16.731284 | controller | 2026-06-10T06:00:16-04:00 INFO Updating controller configmap ... 2026-06-10 06:00:17.124829 | controller | 2026-06-10T06:00:17-04:00 INFO Standalone reconcile done. 2026-06-10 10:00:17.356007 | controller | ok: Runtime: 0:08:18.082885 2026-06-10 10:00:17.368609 | 2026-06-10 10:00:17.368724 | PLAY RECAP 2026-06-10 10:00:17.368806 | controller | ok: 1 changed: 1 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-06-10 10:00:17.368849 | 2026-06-10 10:00:17.487802 | RUN END RESULT_NORMAL: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/run-deploy.yaml@master] 2026-06-10 10:00:17.490173 | RUN START: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/run-tests.yaml@master] 2026-06-10 10:00:18.319183 | 2026-06-10 10:00:18.319466 | PLAY [controller] 2026-06-10 10:00:18.388166 | 2026-06-10 10:00:18.388571 | TASK [Install test dependencies] 2026-06-10 10:00:19.463986 | controller | ok: Nothing to do 2026-06-10 10:00:19.469112 | 2026-06-10 10:00:19.469191 | TASK [setup-variables : Include global variables] 2026-06-10 10:00:19.514649 | controller | skipping: Conditional result was False 2026-06-10 10:00:19.521036 | 2026-06-10 10:00:19.521120 | TASK [setup-variables : Include zuul variables if zuul.executor is defined] 2026-06-10 10:00:19.576067 | controller | ok 2026-06-10 10:00:19.596786 | 2026-06-10 10:00:19.596995 | TASK [setup-nodepool-ns : Set up nodepool namespace] 2026-06-10 06:00:20.230200 | controller | 2026-06-10T06:00:20-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:00:20.230283 | controller | 2026-06-10 06:00:20.241034 | controller | 2026-06-10T06:00:20-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:00:20.241097 | controller | 2026-06-10 06:00:20.241925 | controller | 2026-06-10T06:00:20-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:00:20.241943 | controller | 2026-06-10 06:00:20.244862 | controller | 2026-06-10T06:00:20-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:00:20.244883 | controller | 2026-06-10 06:00:20.248172 | controller | 2026-06-10T06:00:20-04:00 INFO Creating "nodepool" in nodepool 2026-06-10 06:00:20.252685 | controller | 2026-06-10T06:00:20-04:00 INFO "nodepool" created 2026-06-10 06:00:20.256017 | controller | 2026-06-10T06:00:20-04:00 INFO Creating "nodepool-sa" in nodepool 2026-06-10 06:00:20.257683 | controller | 2026-06-10T06:00:20-04:00 INFO "nodepool-sa" created 2026-06-10 06:00:20.260058 | controller | 2026-06-10T06:00:20-04:00 INFO Creating "nodepool-role" in nodepool 2026-06-10 06:00:20.261745 | controller | 2026-06-10T06:00:20-04:00 INFO "nodepool-role" created 2026-06-10 06:00:20.263104 | controller | 2026-06-10T06:00:20-04:00 INFO Creating "nodepool-rb" in nodepool 2026-06-10 06:00:20.265316 | controller | 2026-06-10T06:00:20-04:00 INFO "nodepool-rb" created 2026-06-10 06:00:20.266578 | controller | 2026-06-10T06:00:20-04:00 INFO Creating "nodepool-token" in nodepool 2026-06-10 06:00:20.268211 | controller | 2026-06-10T06:00:20-04:00 INFO "nodepool-token" created 2026-06-10 06:00:21.275953 | controller | 2026-06-10T06:00:21-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:00:21.276022 | controller | 2026-06-10 06:00:21.278991 | controller | 2026-06-10T06:00:21-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:00:21.279027 | controller | 2026-06-10 06:00:21.283017 | controller | 2026-06-10T06:00:21-04:00 INFO Creating "nodepool-providers-secrets" in sf 2026-06-10 06:00:21.285967 | controller | 2026-06-10T06:00:21-04:00 INFO "nodepool-providers-secrets" created 2026-06-10 10:00:21.641119 | controller | ok: Runtime: 0:00:01.305807 2026-06-10 10:00:21.658216 | 2026-06-10 10:00:21.658373 | TASK [utils/get-gerrit-admin-user-api-key : Get Gerrit admin user API key] 2026-06-10 10:00:22.200873 | controller | ok: Runtime: 0:00:00.064950 2026-06-10 10:00:22.210532 | 2026-06-10 10:00:22.210668 | TASK [utils/get-gerrit-admin-user-api-key : Set fact for _gerrit_admin_api_key] 2026-06-10 10:00:22.255331 | controller | ok 2026-06-10 10:00:22.274803 | 2026-06-10 10:00:22.274955 | LOOP [utils/config-repo-reset : Config repo, reset] 2026-06-10 10:00:22.607650 | controller | Fetching origin 2026-06-10 10:00:22.608219 | controller | Fetching gerrit 2026-06-10 10:00:22.608304 | controller | From https://gerrit.sfop.me/a/config 2026-06-10 10:00:22.608369 | controller | * [new branch] master -> gerrit/master 2026-06-10 10:00:22.608548 | controller | ok: Item: git fetch --all Runtime: 0:00:00.093533 2026-06-10 10:00:22.879842 | controller | Already on 'master' 2026-06-10 10:00:22.880028 | controller | Your branch is up to date with 'origin/master'. 2026-06-10 10:00:22.880096 | controller | ok: Item: git checkout master Runtime: 0:00:00.006445 2026-06-10 10:00:23.013071 | controller | HEAD is now at ef97108 Automatic update 2026-06-10 10:00:23.013267 | controller | ok: Item: git reset --hard origin/master -- Runtime: 0:00:00.005517 2026-06-10 10:00:23.211073 | controller | ok: Item: git clean -f -d Runtime: 0:00:00.005665 2026-06-10 10:00:23.239815 | 2026-06-10 10:00:23.240077 | TASK [utils/setup-log-forwarding-stack : Deploy Loki via included role] 2026-06-10 10:00:23.274033 | controller | ok 2026-06-10 10:00:23.286792 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/utils/setup-log-forwarding-stack/tasks/deploy-loki.yaml 2026-06-10 10:00:23.295872 | 2026-06-10 10:00:23.296051 | TASK [utils/setup-log-forwarding-stack : Deploy Loki] 2026-06-10 06:00:23.710911 | controller | configmap/loki-config created 2026-06-10 06:00:23.717703 | controller | deployment.apps/test-loki created 2026-06-10 06:00:23.728545 | controller | service/loki-http created 2026-06-10 10:00:23.837875 | controller | ok: Runtime: 0:00:00.212217 2026-06-10 10:00:23.847326 | 2026-06-10 10:00:23.847536 | TASK [utils/setup-log-forwarding-stack : Wait for Loki deployment to be ready] 2026-06-10 10:00:34.518395 | controller | ok: Runtime: 0:00:00.063101 2026-06-10 10:00:34.527962 | 2026-06-10 10:00:34.528098 | TASK [utils/setup-log-forwarding-stack : Deploy Fluent Bit via included role] 2026-06-10 10:00:34.563688 | controller | ok 2026-06-10 10:00:34.575627 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/utils/setup-log-forwarding-stack/tasks/deploy-fluentbit.yaml 2026-06-10 10:00:34.586777 | 2026-06-10 10:00:34.586909 | TASK [utils/setup-log-forwarding-stack : Deploy Fluent Bit] 2026-06-10 06:00:35.046076 | controller | configmap/test-fluentbit-config created 2026-06-10 06:00:35.059270 | controller | pod/test-fluentbit created 2026-06-10 06:00:35.085693 | controller | service/fluent-bit-forward-input created 2026-06-10 10:00:35.135107 | controller | ok: Runtime: 0:00:00.270114 2026-06-10 10:00:35.143916 | 2026-06-10 10:00:35.144083 | TASK [utils/setup-log-forwarding-stack : Wait for Fluent Bit pod to be ready] 2026-06-10 10:01:37.004139 | controller | ok: Runtime: 0:00:00.053912 2026-06-10 10:01:37.013149 | 2026-06-10 10:01:37.013275 | TASK [utils/setup-log-forwarding-stack : Install LogCLI] 2026-06-10 10:01:37.039953 | controller | ok 2026-06-10 10:01:37.055104 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/utils/setup-log-forwarding-stack/tasks/install-logCLI.yaml 2026-06-10 10:01:37.067775 | 2026-06-10 10:01:37.067907 | TASK [utils/setup-log-forwarding-stack : check if logCLI is present] 2026-06-10 10:01:37.464746 | controller | ok 2026-06-10 10:01:37.471001 | 2026-06-10 10:01:37.471094 | TASK [utils/setup-log-forwarding-stack : download and unzip logCLI] 2026-06-10 10:01:39.090942 | controller | changed 2026-06-10 10:01:39.100783 | 2026-06-10 10:01:39.100916 | TASK [utils/setup-log-forwarding-stack : ensure ~/bin exists] 2026-06-10 10:01:39.501551 | controller | changed 2026-06-10 10:01:39.511068 | 2026-06-10 10:01:39.511199 | TASK [utils/setup-log-forwarding-stack : move logCLI to ~/bin/] 2026-06-10 10:01:40.057008 | controller | ok: Runtime: 0:00:00.003922 2026-06-10 10:01:40.069184 | 2026-06-10 10:01:40.069317 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 10:01:40.306330 | controller | ok 2026-06-10 10:01:40.317106 | 2026-06-10 10:01:40.317258 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 10:01:40.545391 | controller | ok 2026-06-10 10:01:40.549582 | 2026-06-10 10:01:40.549648 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 06:01:41.004444 | controller | 2026-06-10T06:01:41-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:01:41.004533 | controller | 2026-06-10 06:01:41.013819 | controller | 2026-06-10T06:01:41-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:01:41.013879 | controller | 2026-06-10 06:01:41.024560 | controller | 2026-06-10T06:01:41-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:01:41.635259 | controller | 2026-06-10T06:01:41-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:01:41.726012 | controller | 2026-06-10T06:01:41-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:01:41.731321 | controller | 2026-06-10T06:01:41-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:01:41.731346 | controller | 2026-06-10T06:01:41-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:01:41.746172 | controller | 2026-06-10T06:01:41-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:01:41.746225 | controller | 2026-06-10T06:01:41-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:01:42.037558 | controller | 2026-06-10T06:01:42-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:01:42.441112 | controller | 2026-06-10T06:01:42-04:00 INFO Creating object, name: zuul-capacity 2026-06-10 06:01:42.452900 | controller | 2026-06-10T06:01:42-04:00 INFO Statefulset "nodepool-launcher" configuration changed, applying... 2026-06-10 06:01:42.452959 | controller | 2026-06-10T06:01:42-04:00 INFO Updating object, name: nodepool-launcher 2026-06-10 06:01:42.633484 | controller | 2026-06-10T06:01:42-04:00 INFO Statefulset "nodepool-builder" configuration changed, applying... 2026-06-10 06:01:42.633631 | controller | 2026-06-10T06:01:42-04:00 INFO Updating object, name: nodepool-builder 2026-06-10 06:01:43.447536 | controller | 2026-06-10T06:01:43-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:01:43.447600 | controller | 2026-06-10T06:01:43-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:01:44.624330 | controller | 2026-06-10T06:01:44-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:01:44.624388 | controller | 2026-06-10T06:01:44-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:01:45.836629 | controller | 2026-06-10T06:01:45-04:00 INFO 2026-06-10 06:01:45.836699 | controller | - Config: OK 2026-06-10 06:01:45.836705 | controller | - Gateway: OK 2026-06-10 06:01:45.836710 | controller | - GitServer: OK 2026-06-10 06:01:45.836714 | controller | - HoundSearch: OK 2026-06-10 06:01:45.836718 | controller | - LogJuicer: OK 2026-06-10 06:01:45.836721 | controller | - Logserver: OK 2026-06-10 06:01:45.836725 | controller | - MariaDB: OK 2026-06-10 06:01:45.836728 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:01:45.836732 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:01:45.836736 | controller | - Zookeeper: OK 2026-06-10 06:01:45.836740 | controller | - Zuul: OK 2026-06-10 06:01:45.836744 | controller | 2026-06-10 06:01:45.836754 | controller | 2026-06-10T06:01:45-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 06:01:50.845422 | controller | 2026-06-10T06:01:50-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:01:51.458094 | controller | 2026-06-10T06:01:51-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:01:51.458144 | controller | 2026-06-10T06:01:51-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:01:51.477757 | controller | 2026-06-10T06:01:51-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:01:51.477810 | controller | 2026-06-10T06:01:51-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:01:51.851556 | controller | 2026-06-10T06:01:51-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:01:52.261271 | controller | 2026-06-10T06:01:52-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:01:52.261355 | controller | 2026-06-10T06:01:52-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 06:01:53.266102 | controller | 2026-06-10T06:01:53-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:01:53.266166 | controller | 2026-06-10T06:01:53-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:01:54.248756 | controller | 2026-06-10T06:01:54-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:01:54.248821 | controller | 2026-06-10T06:01:54-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:01:55.452656 | controller | 2026-06-10T06:01:55-04:00 INFO 2026-06-10 06:01:55.452723 | controller | - Config: OK 2026-06-10 06:01:55.452731 | controller | - Gateway: OK 2026-06-10 06:01:55.452737 | controller | - GitServer: OK 2026-06-10 06:01:55.452742 | controller | - HoundSearch: OK 2026-06-10 06:01:55.452748 | controller | - LogJuicer: OK 2026-06-10 06:01:55.452753 | controller | - Logserver: OK 2026-06-10 06:01:55.452758 | controller | - MariaDB: OK 2026-06-10 06:01:55.452766 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:01:55.452772 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:01:55.452778 | controller | - Zookeeper: OK 2026-06-10 06:01:55.452783 | controller | - Zuul: OK 2026-06-10 06:01:55.452789 | controller | 2026-06-10 06:01:55.452795 | controller | 2026-06-10T06:01:55-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 06:02:00.457992 | controller | 2026-06-10T06:02:00-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:02:01.069828 | controller | 2026-06-10T06:02:01-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:02:01.069902 | controller | 2026-06-10T06:02:01-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:02:01.082928 | controller | 2026-06-10T06:02:01-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:02:01.082974 | controller | 2026-06-10T06:02:01-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:02:01.467585 | controller | 2026-06-10T06:02:01-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:02:01.871982 | controller | 2026-06-10T06:02:01-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:02:01.872066 | controller | 2026-06-10T06:02:01-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 06:02:02.881693 | controller | 2026-06-10T06:02:02-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:02:02.881790 | controller | 2026-06-10T06:02:02-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:02:03.862442 | controller | 2026-06-10T06:02:03-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:02:03.862496 | controller | 2026-06-10T06:02:03-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:02:05.070214 | controller | 2026-06-10T06:02:05-04:00 INFO 2026-06-10 06:02:05.070288 | controller | - Config: OK 2026-06-10 06:02:05.070296 | controller | - Gateway: OK 2026-06-10 06:02:05.070302 | controller | - GitServer: OK 2026-06-10 06:02:05.070308 | controller | - HoundSearch: OK 2026-06-10 06:02:05.070313 | controller | - LogJuicer: OK 2026-06-10 06:02:05.070319 | controller | - Logserver: OK 2026-06-10 06:02:05.070324 | controller | - MariaDB: OK 2026-06-10 06:02:05.070330 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:02:05.070335 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:02:05.070341 | controller | - Zookeeper: OK 2026-06-10 06:02:05.070346 | controller | - Zuul: OK 2026-06-10 06:02:05.070352 | controller | 2026-06-10 06:02:05.070366 | controller | 2026-06-10T06:02:05-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 06:02:10.074667 | controller | 2026-06-10T06:02:10-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:02:10.689098 | controller | 2026-06-10T06:02:10-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:02:10.689181 | controller | 2026-06-10T06:02:10-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:02:10.704222 | controller | 2026-06-10T06:02:10-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:02:10.704298 | controller | 2026-06-10T06:02:10-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:02:11.085158 | controller | 2026-06-10T06:02:11-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:02:11.487415 | controller | 2026-06-10T06:02:11-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:02:11.487477 | controller | 2026-06-10T06:02:11-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 06:02:12.494023 | controller | 2026-06-10T06:02:12-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:02:12.494086 | controller | 2026-06-10T06:02:12-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:02:13.481341 | controller | 2026-06-10T06:02:13-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:02:13.481411 | controller | 2026-06-10T06:02:13-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:02:14.684377 | controller | 2026-06-10T06:02:14-04:00 INFO 2026-06-10 06:02:14.684442 | controller | - Config: OK 2026-06-10 06:02:14.684464 | controller | - Gateway: OK 2026-06-10 06:02:14.684471 | controller | - GitServer: OK 2026-06-10 06:02:14.684476 | controller | - HoundSearch: OK 2026-06-10 06:02:14.684481 | controller | - LogJuicer: OK 2026-06-10 06:02:14.684486 | controller | - Logserver: OK 2026-06-10 06:02:14.684491 | controller | - MariaDB: OK 2026-06-10 06:02:14.684496 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:02:14.684501 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:02:14.684506 | controller | - Zookeeper: OK 2026-06-10 06:02:14.684511 | controller | - Zuul: OK 2026-06-10 06:02:14.684516 | controller | 2026-06-10 06:02:14.684557 | controller | 2026-06-10T06:02:14-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-10 06:02:19.689469 | controller | 2026-06-10T06:02:19-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:02:20.303803 | controller | 2026-06-10T06:02:20-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:02:20.303880 | controller | 2026-06-10T06:02:20-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:02:20.316583 | controller | 2026-06-10T06:02:20-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:02:20.316645 | controller | 2026-06-10T06:02:20-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:02:20.700155 | controller | 2026-06-10T06:02:20-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:02:21.101421 | controller | 2026-06-10T06:02:21-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:02:21.101510 | controller | 2026-06-10T06:02:21-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 06:02:22.117742 | controller | 2026-06-10T06:02:22-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:02:22.117804 | controller | 2026-06-10T06:02:22-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:02:23.092864 | controller | 2026-06-10T06:02:23-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:02:23.092942 | controller | 2026-06-10T06:02:23-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:02:24.298574 | controller | 2026-06-10T06:02:24-04:00 INFO 2026-06-10 06:02:24.298680 | controller | - Config: OK 2026-06-10 06:02:24.298696 | controller | - Gateway: OK 2026-06-10 06:02:24.298708 | controller | - GitServer: OK 2026-06-10 06:02:24.298719 | controller | - HoundSearch: OK 2026-06-10 06:02:24.298731 | controller | - LogJuicer: OK 2026-06-10 06:02:24.298743 | controller | - Logserver: OK 2026-06-10 06:02:24.298755 | controller | - MariaDB: OK 2026-06-10 06:02:24.298768 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:02:24.298781 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:02:24.298793 | controller | - Zookeeper: OK 2026-06-10 06:02:24.298805 | controller | - Zuul: OK 2026-06-10 06:02:24.298817 | controller | 2026-06-10 06:02:24.298828 | controller | 2026-06-10T06:02:24-04:00 INFO [attempt #5] Waiting 5s for the next reconcile call ... 2026-06-10 06:02:29.309176 | controller | 2026-06-10T06:02:29-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:02:29.925510 | controller | 2026-06-10T06:02:29-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:02:29.925988 | controller | 2026-06-10T06:02:29-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:02:29.944336 | controller | 2026-06-10T06:02:29-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:02:29.944393 | controller | 2026-06-10T06:02:29-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:02:30.316972 | controller | 2026-06-10T06:02:30-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:02:30.719874 | controller | 2026-06-10T06:02:30-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:02:30.719928 | controller | 2026-06-10T06:02:30-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:02:31.722625 | controller | 2026-06-10T06:02:31-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:02:31.722672 | controller | 2026-06-10T06:02:31-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:02:32.710571 | controller | 2026-06-10T06:02:32-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:02:32.710660 | controller | 2026-06-10T06:02:32-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:02:33.918720 | controller | 2026-06-10T06:02:33-04:00 INFO 2026-06-10 06:02:33.918793 | controller | - Config: OK 2026-06-10 06:02:33.918801 | controller | - Gateway: OK 2026-06-10 06:02:33.918807 | controller | - GitServer: OK 2026-06-10 06:02:33.918829 | controller | - HoundSearch: OK 2026-06-10 06:02:33.918836 | controller | - LogJuicer: OK 2026-06-10 06:02:33.918841 | controller | - Logserver: OK 2026-06-10 06:02:33.918846 | controller | - MariaDB: OK 2026-06-10 06:02:33.918850 | controller | - NodePoolBuilder: OK 2026-06-10 06:02:33.918855 | controller | - NodePoolLauncher: OK 2026-06-10 06:02:33.918860 | controller | - Zookeeper: OK 2026-06-10 06:02:33.918865 | controller | - Zuul: OK 2026-06-10 06:02:33.918870 | controller | 2026-06-10 06:02:33.918882 | controller | 2026-06-10T06:02:33-04:00 INFO Updating controller configmap ... 2026-06-10 06:02:34.311124 | controller | 2026-06-10T06:02:34-04:00 INFO Standalone reconcile done. 2026-06-10 10:02:34.641613 | controller | ok: Runtime: 0:00:53.560183 2026-06-10 10:02:34.648553 | 2026-06-10 10:02:34.648642 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 10:02:35.186145 | controller | ok: Runtime: 0:00:00.005655 2026-06-10 10:02:35.201760 | 2026-06-10 10:02:35.201941 | TASK [golang-integration-tests : Run integration tests] 2026-06-10 06:02:35.467161 | controller | go test -timeout 0 -v ./tests/... -args --ginkgo.v --ginkgo.no-color 2026-06-10 06:02:35.481342 | controller | go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 2026-06-10 06:02:35.481456 | controller | go: downloading github.com/onsi/ginkgo/v2 v2.17.2 2026-06-10 06:02:35.483810 | controller | go: downloading github.com/onsi/gomega v1.33.1 2026-06-10 06:02:44.957710 | controller | === RUN TestSFOP 2026-06-10 06:02:44.958155 | controller | Running Suite: Software Factory CI - /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests 2026-06-10 06:02:44.958173 | controller | ======================================================================================================================== 2026-06-10 06:02:44.958185 | controller | Random Seed: 1781085764 2026-06-10 06:02:44.958192 | controller | 2026-06-10 06:02:44.958458 | controller | Will run 16 of 16 specs 2026-06-10 06:02:44.958468 | controller | ------------------------------ 2026-06-10 06:02:44.958585 | controller | [BeforeSuite] 2026-06-10 06:02:44.958596 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/main_test.go:55 2026-06-10 06:02:44.961070 | controller | 2026-06-10T06:02:44-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:02:44.961094 | controller | 2026-06-10 06:02:44.972732 | controller | 2026-06-10T06:02:44-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:02:44.972762 | controller | 2026-06-10 06:02:44.978362 | controller | [BeforeSuite] PASSED [0.020 seconds] 2026-06-10 06:02:44.978392 | controller | ------------------------------ 2026-06-10 06:02:44.978592 | controller | gateway tests When setting up extra gateway configuration for a maintenance Should reconcile 2026-06-10 06:02:44.978627 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/gateway_test.go:28 2026-06-10 06:02:44.978743 | controller | 2026-06-10T06:02:44-04:00 INFO Creating object, name: maintenance-config 2026-06-10 06:02:44.981001 | controller | 2026-06-10T06:02:44-04:00 INFO Creating object, name: maintenance-static-files 2026-06-10 06:02:44.986315 | controller | 2026-06-10T06:02:44-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:02:45.589578 | controller | 2026-06-10T06:02:45-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:02:45.589659 | controller | 2026-06-10T06:02:45-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:02:45.595389 | controller | 2026-06-10T06:02:45-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:02:45.676961 | controller | 2026-06-10T06:02:45-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:02:45.684907 | controller | 2026-06-10T06:02:45-04:00 INFO Statefulset "gateway" configuration changed, applying... 2026-06-10 06:02:45.684953 | controller | 2026-06-10T06:02:45-04:00 DEBUG Reason: annotation 'config-hash' changed from '90a6261c7272ad73ee593bd22fc08aa0b50c9e09eb380107181e1e58b851202b' to 'c04b8e5064e2a2985ef09fcc1ab4c2dda05fbcb2cd18006ff3d01f2b963e17d8' 2026-06-10 06:02:45.684959 | controller | 2026-06-10T06:02:45-04:00 INFO Updating object, name: gateway 2026-06-10 06:02:45.708849 | controller | 2026-06-10T06:02:45-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:02:45.708920 | controller | 2026-06-10T06:02:45-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:02:45.998026 | controller | 2026-06-10T06:02:45-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:02:46.399296 | controller | 2026-06-10T06:02:46-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:02:46.399357 | controller | 2026-06-10T06:02:46-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:02:47.790892 | controller | 2026-06-10T06:02:47-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:02:47.790956 | controller | 2026-06-10T06:02:47-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:02:49.381464 | controller | 2026-06-10T06:02:49-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:02:49.381561 | controller | 2026-06-10T06:02:49-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:02:50.586870 | controller | 2026-06-10T06:02:50-04:00 INFO 2026-06-10 06:02:50.586934 | controller | - Config: OK 2026-06-10 06:02:50.586939 | controller | - Gateway: Waiting ... 2026-06-10 06:02:50.586958 | controller | - GitServer: OK 2026-06-10 06:02:50.586962 | controller | - HoundSearch: OK 2026-06-10 06:02:50.586966 | controller | - LogJuicer: OK 2026-06-10 06:02:50.586969 | controller | - Logserver: OK 2026-06-10 06:02:50.586973 | controller | - MariaDB: OK 2026-06-10 06:02:50.586978 | controller | - NodePoolBuilder: OK 2026-06-10 06:02:50.586982 | controller | - NodePoolLauncher: OK 2026-06-10 06:02:50.586986 | controller | - Zookeeper: OK 2026-06-10 06:02:50.586990 | controller | - Zuul: OK 2026-06-10 06:02:50.586994 | controller | 2026-06-10 06:02:52.460825 | controller | 2026-06-10T06:02:51-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:02:52.488136 | controller | 2026-06-10T06:02:52-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:02:52.488229 | controller | 2026-06-10T06:02:52-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:02:52.507100 | controller | 2026-06-10T06:02:52-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:02:52.507200 | controller | 2026-06-10T06:02:52-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:02:52.525732 | controller | 2026-06-10T06:02:52-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:02:52.525804 | controller | 2026-06-10T06:02:52-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:02:52.601014 | controller | 2026-06-10T06:02:52-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:02:53.384830 | controller | 2026-06-10T06:02:53-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:02:53.384896 | controller | 2026-06-10T06:02:53-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:02:54.787000 | controller | 2026-06-10T06:02:54-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:02:54.787061 | controller | 2026-06-10T06:02:54-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:02:56.382945 | controller | 2026-06-10T06:02:56-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:02:56.383008 | controller | 2026-06-10T06:02:56-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:02:57.592382 | controller | 2026-06-10T06:02:57-04:00 INFO 2026-06-10 06:02:57.592445 | controller | - Config: OK 2026-06-10 06:02:57.592453 | controller | - Gateway: OK 2026-06-10 06:02:57.592458 | controller | - GitServer: OK 2026-06-10 06:02:57.592461 | controller | - HoundSearch: OK 2026-06-10 06:02:57.592465 | controller | - LogJuicer: OK 2026-06-10 06:02:57.592470 | controller | - Logserver: OK 2026-06-10 06:02:57.592498 | controller | - MariaDB: OK 2026-06-10 06:02:57.592503 | controller | - NodePoolBuilder: OK 2026-06-10 06:02:57.592541 | controller | - NodePoolLauncher: OK 2026-06-10 06:02:57.592555 | controller | - Zookeeper: OK 2026-06-10 06:02:57.592620 | controller | - Zuul: OK 2026-06-10 06:02:57.592629 | controller | 2026-06-10 06:02:57.592766 | controller | • [12.614 seconds] 2026-06-10 06:02:57.592793 | controller | ------------------------------ 2026-06-10 06:02:57.592853 | controller | gateway tests When setting up extra gateway configuration for a maintenance Should serve HTTP 503 and the expected content 2026-06-10 06:02:57.592859 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/gateway_test.go:55 2026-06-10 06:02:57.592928 | controller | 2026-06-10T06:02:57-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl -o - -I gateway:8080/zuul/] 2026-06-10 06:02:57.673047 | controller | % Total % Received % Xferd Average Speed Time Time Time Current 2026-06-10 06:02:57.673108 | controller | Dload Upload Total Spent Left Speed 2026-06-10 06:02:57.674534 | controller | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 17 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2026-06-10 06:02:57.676358 | controller | 2026-06-10T06:02:57-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl gateway:8080/zuul/api/components] 2026-06-10 06:02:57.756017 | controller | % Total % Received % Xferd Average Speed Time Time Time Current 2026-06-10 06:02:57.756086 | controller | Dload Upload Total Spent Left Speed 2026-06-10 06:02:57.757843 | controller | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 17 100 17 0 0 17000 0 --:--:-- --:--:-- --:--:-- 17000 2026-06-10 06:02:57.759505 | controller | • [0.166 seconds] 2026-06-10 06:02:57.759523 | controller | ------------------------------ 2026-06-10 06:02:57.759530 | controller | gateway tests When removing the gateway's extra config Should reconcile 2026-06-10 06:02:57.759537 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/gateway_test.go:65 2026-06-10 06:02:57.779428 | controller | 2026-06-10T06:02:57-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:02:58.399837 | controller | 2026-06-10T06:02:58-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:02:58.399900 | controller | 2026-06-10T06:02:58-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:02:58.407727 | controller | 2026-06-10T06:02:58-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:02:58.522199 | controller | 2026-06-10T06:02:58-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:02:58.583156 | controller | 2026-06-10T06:02:58-04:00 INFO Statefulset "gateway" configuration changed, applying... 2026-06-10 06:02:58.583225 | controller | 2026-06-10T06:02:58-04:00 DEBUG Reason: annotation 'config-hash' changed from 'c04b8e5064e2a2985ef09fcc1ab4c2dda05fbcb2cd18006ff3d01f2b963e17d8' to '90a6261c7272ad73ee593bd22fc08aa0b50c9e09eb380107181e1e58b851202b' 2026-06-10 06:02:58.583237 | controller | 2026-06-10T06:02:58-04:00 INFO Updating object, name: gateway 2026-06-10 06:02:58.980832 | controller | 2026-06-10T06:02:58-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:02:58.980896 | controller | 2026-06-10T06:02:58-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:02:59.190167 | controller | 2026-06-10T06:02:59-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:02:59.989971 | controller | 2026-06-10T06:02:59-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:02:59.990038 | controller | 2026-06-10T06:02:59-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:03:01.392287 | controller | 2026-06-10T06:03:01-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:03:01.392359 | controller | 2026-06-10T06:03:01-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:03:02.983070 | controller | 2026-06-10T06:03:02-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:03:02.983141 | controller | 2026-06-10T06:03:02-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:03:04.186004 | controller | 2026-06-10T06:03:04-04:00 INFO 2026-06-10 06:03:04.186079 | controller | - Config: OK 2026-06-10 06:03:04.186090 | controller | - Gateway: Waiting ... 2026-06-10 06:03:04.186099 | controller | - GitServer: OK 2026-06-10 06:03:04.186109 | controller | - HoundSearch: OK 2026-06-10 06:03:04.186118 | controller | - LogJuicer: OK 2026-06-10 06:03:04.186126 | controller | - Logserver: OK 2026-06-10 06:03:04.186134 | controller | - MariaDB: OK 2026-06-10 06:03:04.186142 | controller | - NodePoolBuilder: OK 2026-06-10 06:03:04.186150 | controller | - NodePoolLauncher: OK 2026-06-10 06:03:04.186158 | controller | - Zookeeper: OK 2026-06-10 06:03:04.186166 | controller | - Zuul: OK 2026-06-10 06:03:04.186174 | controller | 2026-06-10 06:03:05.190590 | controller | 2026-06-10T06:03:05-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:03:05.794461 | controller | 2026-06-10T06:03:05-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:03:05.794547 | controller | 2026-06-10T06:03:05-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:03:05.804416 | controller | 2026-06-10T06:03:05-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:03:05.804480 | controller | 2026-06-10T06:03:05-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:03:05.818582 | controller | 2026-06-10T06:03:05-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:03:05.818658 | controller | 2026-06-10T06:03:05-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:03:06.200209 | controller | 2026-06-10T06:03:06-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:03:06.600680 | controller | 2026-06-10T06:03:06-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:03:06.600900 | controller | 2026-06-10T06:03:06-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:03:07.995740 | controller | 2026-06-10T06:03:07-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:03:07.995810 | controller | 2026-06-10T06:03:07-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:03:09.583283 | controller | 2026-06-10T06:03:09-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:03:09.583360 | controller | 2026-06-10T06:03:09-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:03:10.793017 | controller | 2026-06-10T06:03:10-04:00 INFO 2026-06-10 06:03:10.793086 | controller | - Config: OK 2026-06-10 06:03:10.793093 | controller | - Gateway: OK 2026-06-10 06:03:10.793098 | controller | - GitServer: OK 2026-06-10 06:03:10.793103 | controller | - HoundSearch: OK 2026-06-10 06:03:10.793108 | controller | - LogJuicer: OK 2026-06-10 06:03:10.793113 | controller | - Logserver: OK 2026-06-10 06:03:10.793127 | controller | - MariaDB: OK 2026-06-10 06:03:10.793132 | controller | - NodePoolBuilder: OK 2026-06-10 06:03:10.793136 | controller | - NodePoolLauncher: OK 2026-06-10 06:03:10.793140 | controller | - Zookeeper: OK 2026-06-10 06:03:10.793143 | controller | - Zuul: OK 2026-06-10 06:03:10.793147 | controller | 2026-06-10 06:03:10.793500 | controller | • [13.034 seconds] 2026-06-10 06:03:10.793576 | controller | ------------------------------ 2026-06-10 06:03:10.793591 | controller | gateway tests When removing the gateway's extra config Should serve the regular content 2026-06-10 06:03:10.793660 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/gateway_test.go:71 2026-06-10 06:03:10.793696 | controller | 2026-06-10T06:03:10-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl -o - -I gateway:8080/zuul/api/components] 2026-06-10 06:03:10.881875 | controller | % Total % Received % Xferd Average Speed Time Time Time Current 2026-06-10 06:03:10.881936 | controller | Dload Upload Total Spent Left Speed 2026-06-10 06:03:10.886307 | controller | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 537 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2026-06-10 06:03:10.888451 | controller | 2026-06-10T06:03:10-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl gateway:8080/zuul/api/components] 2026-06-10 06:03:10.971091 | controller | % Total % Received % Xferd Average Speed Time Time Time Current 2026-06-10 06:03:10.971146 | controller | Dload Upload Total Spent Left Speed 2026-06-10 06:03:10.974169 | controller | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 537 100 537 0 0 174k 0 --:--:-- --:--:-- --:--:-- 174k 2026-06-10 06:03:10.976502 | controller | • [0.183 seconds] 2026-06-10 06:03:10.976569 | controller | ------------------------------ 2026-06-10 06:03:10.976666 | controller | Test Env Has namespace 2026-06-10 06:03:10.976699 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/main_test.go:167 2026-06-10 06:03:10.976804 | controller | • [0.000 seconds] 2026-06-10 06:03:10.976851 | controller | ------------------------------ 2026-06-10 06:03:10.976912 | controller | Project Private Keys Rotation ensures no config errors 2026-06-10 06:03:10.976939 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/project_private_keys_rotation_test.go:33 2026-06-10 06:03:10.977053 | controller | STEP: Ensuring tenant have no config errors @ 06/10/26 06:03:10.976 2026-06-10 06:03:10.977179 | controller | 2026-06-10T06:03:10-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl -s zuul-web:9000/api/tenant/internal/config-errors] 2026-06-10 06:03:11.058707 | controller | 2026-06-10T06:03:11-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl -s zuul-web:9000/api/tenant/demo-tenant/config-errors] 2026-06-10 06:03:11.149833 | controller | • [0.173 seconds] 2026-06-10 06:03:11.149878 | controller | ------------------------------ 2026-06-10 06:03:11.149906 | controller | Project Private Keys Rotation inject new secret in demo-project 2026-06-10 06:03:11.149912 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/project_private_keys_rotation_test.go:37 2026-06-10 06:03:11.149917 | controller | STEP: Ensure a test ssh key is available @ 06/10/26 06:03:11.149 2026-06-10 06:03:11.155286 | controller | STEP: Loading test secret from fixture (fake SSH key) @ 06/10/26 06:03:11.155 2026-06-10 06:03:11.155309 | controller | STEP: Writing test secret into zuul-scheduler pod and encrypting with zuul-client @ 06/10/26 06:03:11.155 2026-06-10 06:03:11.155329 | controller | 2026-06-10T06:03:11-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c cat > /tmp/secret-to-encrypt.txt] 2026-06-10 06:03:11.246391 | controller | 2026-06-10T06:03:11-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c cat /tmp/secret-to-encrypt.txt | zuul-client encrypt --tenant demo-tenant --project demo-project] 2026-06-10 06:03:11.682635 | controller | writing RSA key 2026-06-10 06:03:11.719048 | controller | STEP: Replacing placeholders in secrets.yaml @ 06/10/26 06:03:11.718 2026-06-10 06:03:11.773963 | controller | STEP: Writing .zuul.d/secrets.yaml in repo @ 06/10/26 06:03:11.773 2026-06-10 06:03:11.774049 | controller | STEP: Committing and pushing locally as admin (HTTPS) @ 06/10/26 06:03:11.773 2026-06-10 06:03:11.873490 | controller | • [0.723 seconds] 2026-06-10 06:03:11.873542 | controller | ------------------------------ 2026-06-10 06:03:11.873550 | controller | Project Private Keys Rotation runs rotate-projects-private-keys 2026-06-10 06:03:11.873555 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/project_private_keys_rotation_test.go:84 2026-06-10 06:03:11.873628 | controller | STEP: Fetching project private keys state from ZooKeeper (to validate change after) @ 06/10/26 06:03:11.873 2026-06-10 06:03:11.873643 | controller | 2026-06-10T06:03:11-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c python3 << 'PYEOF' 2026-06-10 06:03:11.873648 | controller | import configparser 2026-06-10 06:03:11.873653 | controller | import hashlib 2026-06-10 06:03:11.873658 | controller | from zuul.zk import ZooKeeperClient 2026-06-10 06:03:11.873664 | controller | from zuul.lib.keystorage import KeyStorage 2026-06-10 06:03:11.873669 | controller | from zuul.lib import encryption 2026-06-10 06:03:11.873674 | controller | 2026-06-10 06:03:11.873679 | controller | config = configparser.ConfigParser() 2026-06-10 06:03:11.873684 | controller | config.read("/etc/zuul/zuul.conf") 2026-06-10 06:03:11.873689 | controller | password = config["keystore"]["password"].encode("utf-8") 2026-06-10 06:03:11.873694 | controller | zk_client = ZooKeeperClient.fromConfig(config) 2026-06-10 06:03:11.873700 | controller | zk_client.connect() 2026-06-10 06:03:11.873705 | controller | lines = [] 2026-06-10 06:03:11.873710 | controller | for path, obj in KeyStorage(zk_client, "unused").exportKeys().get("keys", {}).items(): 2026-06-10 06:03:11.873714 | controller | if path.endswith("/secrets") and "keys" in obj and len(obj.get("keys", [])) == 1: 2026-06-10 06:03:11.873717 | controller | created = obj["keys"][0]["created"] 2026-06-10 06:03:11.873721 | controller | priv_pem = obj["keys"][0]["private_key"] 2026-06-10 06:03:11.873725 | controller | fingerprint = hashlib.sha256(priv_pem.encode()).hexdigest()[:16] 2026-06-10 06:03:11.873729 | controller | lines.append(f"{path} {created} {fingerprint}") 2026-06-10 06:03:11.873733 | controller | if "demo-project" in path: 2026-06-10 06:03:11.873736 | controller | priv, _ = encryption.deserialize_rsa_keypair(priv_pem.encode("utf-8"), password) 2026-06-10 06:03:11.873740 | controller | pem = encryption.serialize_rsa_private_key(priv, None) 2026-06-10 06:03:11.873744 | controller | with open("/tmp/demo-project.key", "wb") as f: 2026-06-10 06:03:11.873747 | controller | f.write(pem) 2026-06-10 06:03:11.873751 | controller | print("\n".join(sorted(lines))) 2026-06-10 06:03:11.873754 | controller | PYEOF 2026-06-10 06:03:11.873758 | controller | ] 2026-06-10 06:03:13.077770 | controller | STEP: Project keys before rotation: /keystorage/gerrit/config/config/secrets 1781085472 bc66a17bee37651f 2026-06-10 06:03:13.077829 | controller | /keystorage/gerrit/demo-project/demo-project/secrets 1781085471 0f59c7f64d50aa43 2026-06-10 06:03:13.077835 | controller | /keystorage/gerrit/demo-tenant-config/demo-tenant-config/secrets 1781085469 18dde827bad416be 2026-06-10 06:03:13.077841 | controller | /keystorage/git-server/system-config/system-config/secrets 1781085470 e3c7f0bfa85d18b0 2026-06-10 06:03:13.077845 | controller | /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/secrets 1781085471 cde62c887cc7562e 2026-06-10 06:03:13.077859 | controller | @ 06/10/26 06:03:13.077 2026-06-10 06:03:13.077864 | controller | STEP: Preparing SSH key for rotation from zuul-ssh-key secret @ 06/10/26 06:03:13.077 2026-06-10 06:03:13.080041 | controller | STEP: Running rotate-projects-private-keys @ 06/10/26 06:03:13.079 2026-06-10 06:03:13.082097 | controller | 2026-06-10T06:03:13-04:00 INFO Creating object, name: zuul-kazoo 2026-06-10 06:03:15.094748 | controller | 2026-06-10T06:03:15-04:00 INFO Deleting object, name: config-base-secret-1-859-ns 2026-06-10 06:03:15.098483 | controller | 2026-06-10T06:03:15-04:00 INFO KubeAPIWarningLogger child pods are preserved by default when jobs are deleted; set propagationPolicy=Background to remove them or set propagationPolicy=Orphan to suppress this warning 2026-06-10 06:03:15.098531 | controller | 2026-06-10T06:03:15-04:00 INFO Deleting object, name: zs-internal-tenant-reconfigure 2026-06-10 06:03:15.102213 | controller | 2026-06-10T06:03:15-04:00 INFO Creating config map name: zk-clients-need-refresh-config-map 2026-06-10 06:03:15.102326 | controller | 2026-06-10T06:03:15-04:00 INFO Creating object, name: zk-clients-need-refresh-config-map 2026-06-10 06:03:15.105795 | controller | 2026-06-10T06:03:15-04:00 INFO Creating config map name: zuul-needs-full-reconfigure-config-map 2026-06-10 06:03:15.105891 | controller | 2026-06-10T06:03:15-04:00 INFO Creating object, name: zuul-needs-full-reconfigure-config-map 2026-06-10 06:03:15.108003 | controller | 2026-06-10T06:03:15-04:00 INFO Running pod execution pod: zuul-kazoo, command: [bash -c cat > /tmp/rotate-projects-private-keys.py && chmod 755 /tmp/*.py] 2026-06-10 06:03:15.213062 | controller | 2026-06-10T06:03:15-04:00 INFO Running pod execution pod: zuul-kazoo, command: [bash -c cat > /var/lib/zuul/.ssh_push_key && chmod 0600 /var/lib/zuul/.ssh_push_key] 2026-06-10 06:03:15.318713 | controller | 2026-06-10T06:03:15-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [cat /var/lib/zuul/main.yaml] 2026-06-10 06:03:15.418800 | controller | 2026-06-10T06:03:15-04:00 INFO Running pod execution pod: zuul-kazoo, command: [bash -c cat > /var/lib/zuul/main.yaml] 2026-06-10 06:03:15.512001 | controller | 2026-06-10T06:03:15-04:00 INFO Running pod execution pod: zuul-kazoo, command: [env PYTHONUNBUFFERED=1 /tmp/rotate-projects-private-keys.py --age 9223372036854775807 --author zuul --email zuul@sfop.me --logserver-key LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlKS1FJQkFBS0NBZ0VBd0lKRDJNU3JVckYxMkt1dWh4TENOWmRJemdVUjVvc0ppOENLMHJESkR1UnVRVk4yCnNZQU00MHJVSnRMQklyRG9DeHd1NGJFMjIvY2QwTXIzYTZaWEY2VW85cmFvbzFzOVN0a0Zaa282NkZGaUNvTTQKUVc1UkZSWVNIREY5SS9yNlRuZXNzdE95L1Jla3pGTi91blI0bHpKYzIyRXdJT2dwWE5UTURZUGVIb0EyWit3RApZYW10WDNyOVdGelpsMmVyWG53cjdyNU9XT3ExRVhSVHh2SkMrM3FZbWVQbHlLVGk3M2hmRjVYWWVMNGszYlRWCnpaYlFVV2ZqMzVSckNBNmlkWU5TemlFSjFscngzd1RCc3l5djlwNnM3NE4wS3Q0R3V0aGV1aDkyQUtIbmhwK0QKek13b2twMGZMVlJ6M3pTMURHcHRCSzZ1ejZsb3pMaTk3bkxSUkxkSXBtcDBwWXVVWnpqOU9uU0FQaG1QUmU2QgovS0FRVm1RWjBNRmVKcWNmd1pQYnV5QThaWjk0Z2NBSm4ybUZDVGJ1Y3lrdUkycmVSaCtVNHBaT0R5ZENaSjJPCkFqMWNPeHVOVjllazRrb0Nyalp0cGJMY244MzJyQU9GYnBib1QwU0JVNng4K0dScWovZGZISjlyVWZCWUk1TzgKTnlPQWhuTTVxVXM4djR5eURkaDNtSUFOVWlqQzFGRFY5NE1Vc0ROT1MzVm9nUjNCUTFBMkxIM2g5RlVobnQ5UApUKzFRZExVNjIwZlprWUJUcy9NdFREQkUveWMyMDJYK3RBS2VPV3oxUWNBS0tKZk9NWmxRLzlXMEFVWlV1UW5wClFBY05KNmk1NDcxNXhWYUZvZWszK3lPTHV6VXdqNlVtb0FLek5wbEdmOWN3SXJ4dFJsemVROSsxVWlVQ0F3RUEKQVFLQ0FnQUN2SGRCWi80MVpTOVRIcEdTK3VuR3dEazVNR3BDT0J5eEgxSDV6Nmh6N1R4QjhuRS84WjE5UjQxLwpaY1IzL1VLM0ViWDY0RDF5Mys3TkRtMTQycnJuSzZKSzYycTNNNjlKMW83VWx0NkNlZEdtYTVvTG1BbFAvdW5QCis4WVhFbFNuYzkwQkI3QkdGNlZ3Z0FsVE9aK1RLcFNXUzZuVzllWE1tM3V3LzN4VE82aEtUdGozZE9WeGMzcVIKWEhSOVNleHVJZDZ2L0lWNWp4QjJ5SXYrU21nNFVkUkViN0JPNjBVTWVVNEV0RTBJM2s4QTFiYXFQdWdFR0RGdApkcHNVMkx4blppL3huUmVhTTkyM09VYVZpTitSWmt6MzlJcmNweDQ2WHFQdUNrOXZqOTJPRk10WFNGbVZSK1Z2CkRBRy9uT3lyMmdGbHJTUHY4S3o4Sis0UEt4T2RXNXdCTk1xWThndXNZRmRsNi9KU0k2SDRTYVRVODZiRGtYYXcKbGRkamtXSW1nK20zYTRWUlBjeGpNUVdRWDE1SUZhaVhkZGk0ZE9mb01uVXRYL2h3aXZtekRsck5HMGJVMlBGNwpFeGJuNmZ6TytpN1owTUhubDVRYXVjRFNaS0c0YUI0eW45dE9CQ1NRNitubmJCNHFtSkdvSGN0dXpZelU0eHdBCjhOdC9ib1JhRTA2aXNDaDAxRVZ0ZWNpSitSK1BGQVZjaFRBeFg0OUErdkwzR1p1TzVKSnJ0bnd4QjFyc3hTK0MKbmsyUmF5elI1c09lbVVvUHFtaG1wYzFaYmVpMnRCSUVtaFF1ZFpyQWpUZW9JdUpNeUF1RWlSNk1NaS81S1JvaApGcGlrUHhERnBCOGk5a2g1SmVjbkxWOU9CREVFbmExa3lKUjZVazRvQ3hNczdIems2d0tDQVFFQTNvNXczUjlZCnBSNW8wQVJpNVNheXM5RW9KVmVIOUppaVZTR0tVT1RQbmEvRjRtVzRINUFDTmlhQ1UwdXFERlhFeE1EZGlzK3EKcy9sSzJlVFhqREFvSUhyU2l5TkIrbEtERUNCMGVsbHBVbTdzMU5UMEJpQjhWcm9kUWpIbjExVFNra3lFVVZHOQpQNmxFQlphbnRkeXNQWnM2NlJMYnJZSVIzY1dIN0pNKzE0dWIwQTBOT0xBVFlJM2doWHRTWTNDZzZHVHhKL3FICmJ1bHgrV0xaNllhWlp0ejBHUnJadmZ3ZzR4UU9PamxKalduUEVZTVF2SFR5d29JM2N6U1JuS1JKVHFDK3c4dDkKNWQrK1daZTlWKzBtYzVVSk1oVVJzS2dKcjN5MFRmQVBCN0ZuMFRiaTF4MjQ4bjZtaDVSZzhsK3YvQ2tSUm91cQo0SjdlczN5cEZBeG5qd0tDQVFFQTNXL3JaN3h1cWhxaXdHMjFQOXZKeTZ4R1NwTnBybTV2czZnbWFlS2ZoQlpICnpXM3pSYm9PdTd1SWxuSC9qUmx1Z3NHdXlEcnZpcTljdkN3SkV6eXQ0dGttbWF1ZTJXSjBGbFFDOHpldkl4TnIKNmE2dU5hektKdTc1SnR4OGw3SDdiMlVjMTR3QkszYzJwcXFMdDRPN25wc3hHTGMxaWlnYjFOUzN2TTZHVm1VVQpqR2x5TUdCb3hKQkRrcnhwbEV1bDRBVmZwZDFtY2N2OVFXZHdweU9Rd3NKMWUrUVNSZUM2azBiZlBqZ1ByaVJHCkFsMExQdDFqeStKYU5aS3hndTVkU0plb3Ywc0Yvdlk5WkRmYmE2ZmtOYjI5cnlHN2lJRU52UGo1NjQ3NklnY0QKUzVoZEJ1cXBRRWEwNUVaczBzR0NUTnorL2puYzZxY2xnS1dmMWIyeEN3S0NBUUVBdTFucHhSdllRZGdobjh1eApmUHBnd1dNM2lvZnZHWWRNVUtJVmtCZXVkeTQ3UldVZjVPSVVHb1c4ZkdaYjFscDFkTFJ2UzZXYUxEdVdPV3VkCjBKUFJrOVlSVzdQb25OUE5KVXhOQVpvM1FYL0R1ejh4MWQ2UERXdExxaWZjNWJhd05SeFkzeVFTVG44dGxnN0gKVEI5aWw5SVBlNW0yZUkvZ2ZuTUZBN0RJM0NiWW1nSUN1VS93WC8zUG5rNjhncDJUdDA4U2UvYzQvN05oa1BFWQpSTEJjZnUxT1I3d0V4UngwaXZTZExZS0FXcDFTQllueUV0UFljTW14NkRlK3hNREFQWUt3Vko3L2N2NHhmYlp4ClhJMTFLeU94UkFuY2hIeFErdnZkeFNacllLeUZtdWU3ZWgrNVM5bzJVamdraktUa3Z6L0xIVG9DTldsSVRJeUsKd1NmWUhRS0NBUUJodTJ6ZG14Z0lzYnJWVFZacXVkbkZyK0pTTmQrMTRmNGcvL3o4VE01MG9OVE9BVklXbXppMwplTnllOGUrRldnMGhYdFNDZ1NpS1FEMmd1NXJCeDgxRFN1NHpjc0NvVjBDL1BoTVRQNmRueHNHaElMeTQ2ZXRLCjVieEhmNWJwNk43OVdsWXhlQlJvRjdTaU9MRnpySXVaUkF1V1NmNHlYT056dXZMaFBSR2pmZmRreFlGbGdwZWEKOGNVeE5VRXY5M2RkZVdHdXRobnA3Y3Q0RzJ4b0NGVnk4QnFrTkpRd0RUQ3JvZU1UOWZCTnZxcTdqazlSVHFzbQpLR0FDTngxcm1VTElzRThGSnRxZG5oYVliZUxGRUJPeitHdUlxVTI0d0tUTW0rTEtLWStTaHgvWU5JWVNjRzJFCjlmQU90RnFCWm5EWTljMEJiWTVvYnA4cHVXL2pDVmJqQW9JQkFRQzZIL2lPRkpWeE5QVVk5WFYrUzE0MDJ6SnAKTURKMmc0Nkhjd1hhTUplSUVIREt2eWl2aWgzQ3VKMzlNVzZ1N0RrS1pjY1VBSjdlNTVZbEJzRXU1VFZaSU1RdgpwV1ZpamRicit1SU9oNHgvWUR4UFNJZlMybDc4ZlNSUmNQT3RBTFpxKzhhMWxFZVJEcy9XMUlXbVZTa0ZEWXJ1CkVlSWUrTG12L2Ftcks4ZmVzZHBDelJkSE9NUWJOdEJuK2JHN0JJSjNia214Sm9SR3pGaUF4cXp4YnkwZmRFSHcKd0tOQUlSd0MzeVZlY2RxWk5sOTRPWS9qRWVtRENWRWtBdWtGNFEyOE9OSk5TT2tmTUdaeUVzSjMrMTJ3TW1TUwphRE90NEd4Z1QwMWl2SHNScG5Nb0h3TXFVOFhldzdIcUE1dzlTV3hxVjlqUi9DM042R2RaNjdlaStQVkgKLS0tLS1FTkQgUlNBIFBSSVZBVEUgS0VZLS0tLS0K] 2026-06-10 06:03:16.031949 | controller | [+] Collecting keys from ZooKeeper 2026-06-10 06:03:16.076170 | controller | [+] Deleting /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/ssh because Non secrets private key 2026-06-10 06:03:16.079009 | controller | [+] Deleting /keystorage/gerrit/demo-tenant-config/demo-tenant-config/ssh because Non secrets private key 2026-06-10 06:03:16.081569 | controller | [+] Deleting /keystorage/gerrit/demo-project/demo-project/ssh because Non secrets private key 2026-06-10 06:03:16.083323 | controller | [+] Deleting /keystorage/gerrit/config/config/ssh because Non secrets private key 2026-06-10 06:03:16.084471 | controller | [+] Deleting /keystorage/git-server/system-config/system-config/ssh because Non secrets private key 2026-06-10 06:03:16.086125 | controller | [+] Deleting /keystorage2/tenant/internal/ssh because Non secrets private key 2026-06-10 06:03:16.087444 | controller | [+] Deleting /keystorage2/tenant/demo-tenant/ssh because Non secrets private key 2026-06-10 06:03:16.088671 | controller | [+] Deleting /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/secrets because zuul-jobs do not have secrets 2026-06-10 06:03:16.100519 | controller | [+] Cloning ssh://zuul@gerrit-sshd.sf:29418/demo-tenant-config to /tmp/current-repo 2026-06-10 06:03:16.105071 | controller | Cloning into '/tmp/current-repo'... 2026-06-10 06:03:16.123568 | controller | Warning: Permanently added '[gerrit-sshd.sf]:29418' (ED25519) to the list of known hosts. 2026-06-10 06:03:16.199936 | controller | Total 9 (delta 0), reused 9 (delta 0) 2026-06-10 06:03:16.249860 | controller | [+] Deleting /keystorage/gerrit/demo-tenant-config/demo-tenant-config/secrets because project had no secret 2026-06-10 06:03:16.255340 | controller | [+] Cloning ssh://zuul@gerrit-sshd.sf:29418/demo-project to /tmp/current-repo 2026-06-10 06:03:16.259059 | controller | Cloning into '/tmp/current-repo'... 2026-06-10 06:03:16.349724 | controller | Total 10 (delta 0), reused 10 (delta 0) 2026-06-10 06:03:16.900653 | controller | [+] Re-Encrypting secret(s) in /tmp/current-repo/.zuul.d/secrets.yaml 2026-06-10 06:03:16.910452 | controller | [master 766b7e1] Automatic secret re-encryption 2026-06-10 06:03:16.910493 | controller | 1 file changed, 10 insertions(+), 10 deletions(-) 2026-06-10 06:03:17.064883 | controller | remote: remote: Processing changes: refs: 1 remote: Processing changes: refs: 1 remote: Processing changes: refs: 1, done 2026-06-10 06:03:17.065646 | controller | To ssh://gerrit-sshd.sf:29418/demo-project 2026-06-10 06:03:17.065661 | controller | 1dd9621..766b7e1 master -> master 2026-06-10 06:03:17.066680 | controller | [+] Updating key for /keystorage/gerrit/demo-project/demo-project/secrets 2026-06-10 06:03:17.075897 | controller | [+] Cloning ssh://zuul@gerrit-sshd.sf:29418/config to /tmp/current-repo 2026-06-10 06:03:17.078946 | controller | Cloning into '/tmp/current-repo'... 2026-06-10 06:03:17.224506 | controller | Total 5 (delta 0), reused 5 (delta 0) 2026-06-10 06:03:17.233214 | controller | [+] Deleting /keystorage/gerrit/config/config/secrets because project had no secret 2026-06-10 06:03:17.235646 | controller | [+] Deleting /keystorage/git-server/system-config/system-config/secrets because internal repo will be handled in reconcile 2026-06-10 06:03:17.383752 | controller | STEP: Reconciling @ 06/10/26 06:03:17.383 2026-06-10 06:03:17.406335 | controller | 2026-06-10T06:03:17-04:00 INFO Warning: Killing ZooKeeper client: nodepool-builder-0 nodepool-builder 2026-06-10 06:03:17.406404 | controller | 2026-06-10T06:03:17-04:00 INFO Running pod execution pod: nodepool-builder-0, command: [kill -9 1] 2026-06-10 06:03:17.493634 | controller | 2026-06-10T06:03:17-04:00 INFO Deleting object, name: nodepool-builder 2026-06-10 06:03:17.496979 | controller | 2026-06-10T06:03:17-04:00 INFO Warning: Killing ZooKeeper client: nodepool-launcher-569b4cc7cf-qddvj launcher 2026-06-10 06:03:17.497009 | controller | 2026-06-10T06:03:17-04:00 INFO Running pod execution pod: nodepool-launcher-569b4cc7cf-qddvj, command: [kill -9 1] 2026-06-10 06:03:17.588379 | controller | 2026-06-10T06:03:17-04:00 INFO Deleting object, name: launcher 2026-06-10 06:03:17.590881 | controller | 2026-06-10T06:03:17-04:00 INFO Warning: Killing ZooKeeper client: zuul-db-provision-r5bxv mariadb-client 2026-06-10 06:03:17.590955 | controller | 2026-06-10T06:03:17-04:00 INFO Running pod execution pod: zuul-db-provision-r5bxv, command: [kill -9 1] 2026-06-10 06:03:17.603991 | controller | 2026-06-10T06:03:17-04:00 INFO Deleting object, name: mariadb-client 2026-06-10 06:03:17.606133 | controller | 2026-06-10T06:03:17-04:00 INFO Warning: Killing ZooKeeper client: zuul-executor-0 zuul-executor 2026-06-10 06:03:17.606162 | controller | 2026-06-10T06:03:17-04:00 INFO Running pod execution pod: zuul-executor-0, command: [kill -9 1] 2026-06-10 06:03:17.661493 | controller | 2026-06-10T06:03:17-04:00 INFO Deleting object, name: zuul-executor 2026-06-10 06:03:17.665934 | controller | 2026-06-10T06:03:17-04:00 INFO Warning: Killing ZooKeeper client: zuul-kazoo zuul-kazoo 2026-06-10 06:03:17.665980 | controller | 2026-06-10T06:03:17-04:00 INFO Running pod execution pod: zuul-kazoo, command: [kill -9 1] 2026-06-10 06:03:17.752484 | controller | 2026-06-10T06:03:17-04:00 INFO Deleting object, name: zuul-kazoo 2026-06-10 06:03:17.754974 | controller | 2026-06-10T06:03:17-04:00 INFO Warning: Killing ZooKeeper client: zuul-merger-0 zuul-merger 2026-06-10 06:03:17.755016 | controller | 2026-06-10T06:03:17-04:00 INFO Running pod execution pod: zuul-merger-0, command: [kill -9 1] 2026-06-10 06:03:17.826729 | controller | 2026-06-10T06:03:17-04:00 INFO Deleting object, name: zuul-merger 2026-06-10 06:03:17.830120 | controller | 2026-06-10T06:03:17-04:00 INFO Warning: Killing ZooKeeper client: zuul-scheduler-0 zuul-scheduler 2026-06-10 06:03:17.830178 | controller | 2026-06-10T06:03:17-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [kill -9 1] 2026-06-10 06:03:17.908073 | controller | 2026-06-10T06:03:17-04:00 INFO Deleting object, name: zuul-scheduler 2026-06-10 06:03:17.912008 | controller | 2026-06-10T06:03:17-04:00 INFO Warning: Killing ZooKeeper client: zuul-web-858b6f55b5-th84h zuul-web 2026-06-10 06:03:17.912090 | controller | 2026-06-10T06:03:17-04:00 INFO Running pod execution pod: zuul-web-858b6f55b5-th84h, command: [kill -9 1] 2026-06-10 06:03:17.995536 | controller | 2026-06-10T06:03:17-04:00 INFO Deleting object, name: zuul-web 2026-06-10 06:03:17.999373 | controller | 2026-06-10T06:03:17-04:00 INFO Warning: Killing ZooKeeper client: zuul-weeder-57c6567f5f-2w4fk zuul-weeder 2026-06-10 06:03:17.999417 | controller | 2026-06-10T06:03:17-04:00 INFO Running pod execution pod: zuul-weeder-57c6567f5f-2w4fk, command: [kill -9 1] 2026-06-10 06:03:18.081189 | controller | 2026-06-10T06:03:18-04:00 INFO Deleting object, name: zuul-weeder 2026-06-10 06:03:18.084514 | controller | 2026-06-10T06:03:18-04:00 INFO Deleting object, name: zk-clients-need-refresh-config-map 2026-06-10 06:03:18.086730 | controller | 2026-06-10T06:03:18-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:03:18.495713 | controller | 2026-06-10T06:03:18-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:03:18.495774 | controller | 2026-06-10T06:03:18-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:03:18.502418 | controller | 2026-06-10T06:03:18-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:03:18.617387 | controller | 2026-06-10T06:03:18-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:03:18.626585 | controller | 2026-06-10T06:03:18-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:03:18.626661 | controller | 2026-06-10T06:03:18-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:03:18.641264 | controller | 2026-06-10T06:03:18-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:03:18.641323 | controller | 2026-06-10T06:03:18-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:03:18.900746 | controller | 2026-06-10T06:03:18-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:03:19.306923 | controller | 2026-06-10T06:03:19-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:03:19.306985 | controller | 2026-06-10T06:03:19-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:03:19.499863 | controller | 2026-06-10T06:03:19-04:00 INFO Creating object, name: nodepool-builder 2026-06-10 06:03:20.305008 | controller | 2026-06-10T06:03:20-04:00 INFO Creating object, name: zuul-scheduler 2026-06-10 06:03:20.694525 | controller | 2026-06-10T06:03:20-04:00 INFO Creating object, name: zuul-executor 2026-06-10 06:03:21.488782 | controller | 2026-06-10T06:03:21-04:00 INFO Creating object, name: zuul-web 2026-06-10 06:03:22.288172 | controller | 2026-06-10T06:03:22-04:00 INFO Creating object, name: zuul-merger 2026-06-10 06:03:22.292114 | controller | 2026-06-10T06:03:22-04:00 INFO 2026-06-10 06:03:22.292169 | controller | - Config: Waiting ... 2026-06-10 06:03:22.292176 | controller | - Gateway: OK 2026-06-10 06:03:22.292181 | controller | - GitServer: OK 2026-06-10 06:03:22.292186 | controller | - HoundSearch: OK 2026-06-10 06:03:22.292191 | controller | - LogJuicer: OK 2026-06-10 06:03:22.292195 | controller | - Logserver: OK 2026-06-10 06:03:22.292200 | controller | - MariaDB: OK 2026-06-10 06:03:22.292206 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:03:22.292211 | controller | - NodePoolLauncher: OK 2026-06-10 06:03:22.292216 | controller | - Zookeeper: OK 2026-06-10 06:03:22.292221 | controller | - Zuul: Waiting ... 2026-06-10 06:03:22.292226 | controller | - |_ Config Secrets: OK 2026-06-10 06:03:22.292231 | controller | - |_ Executor: Waiting ... 2026-06-10 06:03:22.292236 | controller | - |_ Merger: Waiting ... 2026-06-10 06:03:22.292277 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:03:22.292285 | controller | - |_ Web: Waiting ... 2026-06-10 06:03:22.292290 | controller | 2026-06-10 06:03:23.300745 | controller | 2026-06-10T06:03:23-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:03:23.900351 | controller | 2026-06-10T06:03:23-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:03:23.900411 | controller | 2026-06-10T06:03:23-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:03:23.911180 | controller | 2026-06-10T06:03:23-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:03:23.911234 | controller | 2026-06-10T06:03:23-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:03:23.928672 | controller | 2026-06-10T06:03:23-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:03:23.928748 | controller | 2026-06-10T06:03:23-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:03:24.309896 | controller | 2026-06-10T06:03:24-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:03:24.712723 | controller | 2026-06-10T06:03:24-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:03:24.712793 | controller | 2026-06-10T06:03:24-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:03:26.095017 | controller | 2026-06-10T06:03:26-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:03:56.306145 | controller | 2026-06-10T06:03:56-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 06:03:56.336463 | controller | 2026-06-10T06:03:56-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:03:56.336536 | controller | 2026-06-10T06:03:56-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:03:56.718965 | controller | 2026-06-10T06:03:56-04:00 INFO 2026-06-10 06:03:56.719030 | controller | - Config: Waiting ... 2026-06-10 06:03:56.719035 | controller | - Gateway: OK 2026-06-10 06:03:56.719040 | controller | - GitServer: OK 2026-06-10 06:03:56.719045 | controller | - HoundSearch: OK 2026-06-10 06:03:56.719049 | controller | - LogJuicer: OK 2026-06-10 06:03:56.719053 | controller | - Logserver: OK 2026-06-10 06:03:56.719057 | controller | - MariaDB: OK 2026-06-10 06:03:56.719062 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:03:56.719066 | controller | - NodePoolLauncher: OK 2026-06-10 06:03:56.719071 | controller | - Zookeeper: OK 2026-06-10 06:03:56.719075 | controller | - Zuul: Waiting ... 2026-06-10 06:03:56.719079 | controller | - |_ Config Secrets: OK 2026-06-10 06:03:56.719083 | controller | - |_ Executor: OK 2026-06-10 06:03:56.719087 | controller | - |_ Merger: OK 2026-06-10 06:03:56.719092 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:03:56.719096 | controller | - |_ Web: Waiting ... 2026-06-10 06:03:56.719100 | controller | 2026-06-10 06:03:57.723825 | controller | 2026-06-10T06:03:57-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:03:58.326014 | controller | 2026-06-10T06:03:58-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:03:58.326075 | controller | 2026-06-10T06:03:58-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:03:58.333141 | controller | 2026-06-10T06:03:58-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:03:58.333195 | controller | 2026-06-10T06:03:58-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:03:58.345690 | controller | 2026-06-10T06:03:58-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:03:58.345754 | controller | 2026-06-10T06:03:58-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:03:58.734200 | controller | 2026-06-10T06:03:58-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:03:59.136364 | controller | 2026-06-10T06:03:59-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:03:59.136436 | controller | 2026-06-10T06:03:59-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:04:00.518547 | controller | 2026-06-10T06:04:00-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:04:22.592846 | controller | 2026-06-10T06:04:22-04:00 INFO Creating object, name: zuul-weeder 2026-06-10 06:04:22.635469 | controller | 2026-06-10T06:04:22-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:04:22.635536 | controller | 2026-06-10T06:04:22-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:04:23.009076 | controller | 2026-06-10T06:04:23-04:00 INFO 2026-06-10 06:04:23.009148 | controller | - Config: Waiting ... 2026-06-10 06:04:23.009153 | controller | - Gateway: OK 2026-06-10 06:04:23.009158 | controller | - GitServer: OK 2026-06-10 06:04:23.009162 | controller | - HoundSearch: OK 2026-06-10 06:04:23.009166 | controller | - LogJuicer: OK 2026-06-10 06:04:23.009169 | controller | - Logserver: OK 2026-06-10 06:04:23.009173 | controller | - MariaDB: OK 2026-06-10 06:04:23.009177 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:04:23.009181 | controller | - NodePoolLauncher: OK 2026-06-10 06:04:23.009190 | controller | - Zookeeper: OK 2026-06-10 06:04:23.009195 | controller | - Zuul: Waiting ... 2026-06-10 06:04:23.009199 | controller | - |_ Config Secrets: OK 2026-06-10 06:04:23.009202 | controller | - |_ Executor: OK 2026-06-10 06:04:23.009206 | controller | - |_ Merger: OK 2026-06-10 06:04:23.009222 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:04:23.009227 | controller | - |_ Web: OK 2026-06-10 06:04:23.009231 | controller | 2026-06-10 06:04:24.014663 | controller | 2026-06-10T06:04:24-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:04:24.620230 | controller | 2026-06-10T06:04:24-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:04:24.620292 | controller | 2026-06-10T06:04:24-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:04:24.631778 | controller | 2026-06-10T06:04:24-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:04:24.631835 | controller | 2026-06-10T06:04:24-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:04:24.650348 | controller | 2026-06-10T06:04:24-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:04:24.650409 | controller | 2026-06-10T06:04:24-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:04:25.028436 | controller | 2026-06-10T06:04:25-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:04:25.425921 | controller | 2026-06-10T06:04:25-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:04:25.426006 | controller | 2026-06-10T06:04:25-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:04:26.810445 | controller | 2026-06-10T06:04:26-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:04:26.810530 | controller | 2026-06-10T06:04:26-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:04:28.401123 | controller | 2026-06-10T06:04:28-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:04:28.401196 | controller | 2026-06-10T06:04:28-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:04:29.613674 | controller | 2026-06-10T06:04:29-04:00 INFO Creating base secret job 2026-06-10 06:04:29.614465 | controller | 2026-06-10T06:04:29-04:00 INFO Creating object, name: config-base-secret-1-859-ns 2026-06-10 06:04:29.620397 | controller | 2026-06-10T06:04:29-04:00 INFO 2026-06-10 06:04:29.620432 | controller | - Config: Waiting ... 2026-06-10 06:04:29.620440 | controller | - Gateway: OK 2026-06-10 06:04:29.620446 | controller | - GitServer: OK 2026-06-10 06:04:29.620451 | controller | - HoundSearch: OK 2026-06-10 06:04:29.620457 | controller | - LogJuicer: OK 2026-06-10 06:04:29.620462 | controller | - Logserver: OK 2026-06-10 06:04:29.620467 | controller | - MariaDB: OK 2026-06-10 06:04:29.620472 | controller | - NodePoolBuilder: OK 2026-06-10 06:04:29.620477 | controller | - NodePoolLauncher: OK 2026-06-10 06:04:29.620483 | controller | - Zookeeper: OK 2026-06-10 06:04:29.620489 | controller | - Zuul: OK 2026-06-10 06:04:29.620496 | controller | 2026-06-10 06:04:30.625296 | controller | 2026-06-10T06:04:30-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:04:31.228711 | controller | 2026-06-10T06:04:31-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:04:31.228774 | controller | 2026-06-10T06:04:31-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:04:31.237815 | controller | 2026-06-10T06:04:31-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:04:31.237869 | controller | 2026-06-10T06:04:31-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:04:31.248666 | controller | 2026-06-10T06:04:31-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:04:31.248733 | controller | 2026-06-10T06:04:31-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:04:31.637428 | controller | 2026-06-10T06:04:31-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:04:32.040739 | controller | 2026-06-10T06:04:32-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:04:32.040803 | controller | 2026-06-10T06:04:32-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:04:33.411475 | controller | 2026-06-10T06:04:33-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:04:33.411549 | controller | 2026-06-10T06:04:33-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:04:35.002718 | controller | 2026-06-10T06:04:35-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:04:35.002848 | controller | 2026-06-10T06:04:35-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:04:36.208103 | controller | 2026-06-10T06:04:36-04:00 INFO Running tenant-reconfigure for the 'internal' tenant 2026-06-10 06:04:36.398966 | controller | 2026-06-10T06:04:36-04:00 INFO Deleting object, name: zuul-needs-full-reconfigure-config-map 2026-06-10 06:04:36.599800 | controller | 2026-06-10T06:04:36-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [zuul-scheduler full-reconfigure] 2026-06-10 06:04:43.169455 | controller | 2026-06-10T06:04:43-04:00 INFO Deleting object, name: zuul-web 2026-06-10 06:04:43.173698 | controller | 2026-06-10T06:04:43-04:00 INFO Creating object, name: zs-internal-tenant-reconfigure 2026-06-10 06:04:43.176963 | controller | 2026-06-10T06:04:43-04:00 INFO 2026-06-10 06:04:43.177031 | controller | - Config: Waiting ... 2026-06-10 06:04:43.177038 | controller | - Gateway: OK 2026-06-10 06:04:43.177044 | controller | - GitServer: OK 2026-06-10 06:04:43.177049 | controller | - HoundSearch: OK 2026-06-10 06:04:43.177054 | controller | - LogJuicer: OK 2026-06-10 06:04:43.177059 | controller | - Logserver: OK 2026-06-10 06:04:43.177072 | controller | - MariaDB: OK 2026-06-10 06:04:43.177077 | controller | - NodePoolBuilder: OK 2026-06-10 06:04:43.177082 | controller | - NodePoolLauncher: OK 2026-06-10 06:04:43.177087 | controller | - Zookeeper: OK 2026-06-10 06:04:43.177092 | controller | - Zuul: OK 2026-06-10 06:04:43.177098 | controller | 2026-06-10 06:04:44.190950 | controller | 2026-06-10T06:04:44-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:04:44.784391 | controller | 2026-06-10T06:04:44-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:04:44.784462 | controller | 2026-06-10T06:04:44-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:04:44.792355 | controller | 2026-06-10T06:04:44-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:04:44.792423 | controller | 2026-06-10T06:04:44-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:04:44.806215 | controller | 2026-06-10T06:04:44-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:04:44.806287 | controller | 2026-06-10T06:04:44-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:04:45.192270 | controller | 2026-06-10T06:04:45-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:04:45.593920 | controller | 2026-06-10T06:04:45-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:04:45.593983 | controller | 2026-06-10T06:04:45-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:04:46.597651 | controller | 2026-06-10T06:04:46-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:04:46.597705 | controller | 2026-06-10T06:04:46-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:04:47.584637 | controller | 2026-06-10T06:04:47-04:00 INFO Creating object, name: zuul-web 2026-06-10 06:04:48.394269 | controller | 2026-06-10T06:04:48-04:00 INFO 2026-06-10 06:04:48.394340 | controller | - Config: Waiting ... 2026-06-10 06:04:48.394346 | controller | - Gateway: OK 2026-06-10 06:04:48.394350 | controller | - GitServer: OK 2026-06-10 06:04:48.394354 | controller | - HoundSearch: OK 2026-06-10 06:04:48.394358 | controller | - LogJuicer: OK 2026-06-10 06:04:48.394362 | controller | - Logserver: OK 2026-06-10 06:04:48.394365 | controller | - MariaDB: OK 2026-06-10 06:04:48.394369 | controller | - NodePoolBuilder: OK 2026-06-10 06:04:48.394373 | controller | - NodePoolLauncher: OK 2026-06-10 06:04:48.394376 | controller | - Zookeeper: OK 2026-06-10 06:04:48.394380 | controller | - Zuul: Waiting ... 2026-06-10 06:04:48.394383 | controller | - |_ Config Secrets: OK 2026-06-10 06:04:48.394387 | controller | - |_ Executor: OK 2026-06-10 06:04:48.394390 | controller | - |_ Merger: OK 2026-06-10 06:04:48.394394 | controller | - |_ Scheduler: OK 2026-06-10 06:04:48.394398 | controller | - |_ Web: Waiting ... 2026-06-10 06:04:48.394401 | controller | 2026-06-10 06:04:49.398503 | controller | 2026-06-10T06:04:49-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:04:50.005351 | controller | 2026-06-10T06:04:50-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:04:50.005431 | controller | 2026-06-10T06:04:50-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:04:50.020836 | controller | 2026-06-10T06:04:50-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:04:50.020940 | controller | 2026-06-10T06:04:50-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:04:50.036257 | controller | 2026-06-10T06:04:50-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:04:50.036343 | controller | 2026-06-10T06:04:50-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:04:50.407674 | controller | 2026-06-10T06:04:50-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:04:50.811328 | controller | 2026-06-10T06:04:50-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:04:50.811402 | controller | 2026-06-10T06:04:50-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:04:52.196375 | controller | 2026-06-10T06:04:52-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:04:52.196486 | controller | 2026-06-10T06:04:52-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:04:53.785696 | controller | 2026-06-10T06:04:53-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:04:53.785775 | controller | 2026-06-10T06:04:53-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:04:54.590580 | controller | 2026-06-10T06:04:54-04:00 INFO 2026-06-10 06:04:54.590666 | controller | - Config: Waiting ... 2026-06-10 06:04:54.590673 | controller | - Gateway: OK 2026-06-10 06:04:54.590678 | controller | - GitServer: OK 2026-06-10 06:04:54.590684 | controller | - HoundSearch: OK 2026-06-10 06:04:54.590689 | controller | - LogJuicer: OK 2026-06-10 06:04:54.590693 | controller | - Logserver: OK 2026-06-10 06:04:54.590698 | controller | - MariaDB: OK 2026-06-10 06:04:54.590703 | controller | - NodePoolBuilder: OK 2026-06-10 06:04:54.590708 | controller | - NodePoolLauncher: OK 2026-06-10 06:04:54.590713 | controller | - Zookeeper: OK 2026-06-10 06:04:54.590718 | controller | - Zuul: Waiting ... 2026-06-10 06:04:54.590723 | controller | - |_ Config Secrets: OK 2026-06-10 06:04:54.590728 | controller | - |_ Executor: OK 2026-06-10 06:04:54.590733 | controller | - |_ Merger: OK 2026-06-10 06:04:54.590737 | controller | - |_ Scheduler: OK 2026-06-10 06:04:54.590742 | controller | - |_ Web: Waiting ... 2026-06-10 06:04:54.590746 | controller | 2026-06-10 06:04:55.595276 | controller | 2026-06-10T06:04:55-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:04:56.197811 | controller | 2026-06-10T06:04:56-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:04:56.197879 | controller | 2026-06-10T06:04:56-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:04:56.206635 | controller | 2026-06-10T06:04:56-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:04:56.206690 | controller | 2026-06-10T06:04:56-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:04:56.221221 | controller | 2026-06-10T06:04:56-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:04:56.221275 | controller | 2026-06-10T06:04:56-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:04:56.605470 | controller | 2026-06-10T06:04:56-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:04:57.006496 | controller | 2026-06-10T06:04:57-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:04:57.006554 | controller | 2026-06-10T06:04:57-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:04:58.394897 | controller | 2026-06-10T06:04:58-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:04:58.394984 | controller | 2026-06-10T06:04:58-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:04:59.984930 | controller | 2026-06-10T06:04:59-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:04:59.985007 | controller | 2026-06-10T06:04:59-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:05:00.791226 | controller | 2026-06-10T06:05:00-04:00 INFO 2026-06-10 06:05:00.791300 | controller | - Config: Waiting ... 2026-06-10 06:05:00.791308 | controller | - Gateway: OK 2026-06-10 06:05:00.791313 | controller | - GitServer: OK 2026-06-10 06:05:00.791319 | controller | - HoundSearch: OK 2026-06-10 06:05:00.791324 | controller | - LogJuicer: OK 2026-06-10 06:05:00.791329 | controller | - Logserver: OK 2026-06-10 06:05:00.791333 | controller | - MariaDB: OK 2026-06-10 06:05:00.791338 | controller | - NodePoolBuilder: OK 2026-06-10 06:05:00.791343 | controller | - NodePoolLauncher: OK 2026-06-10 06:05:00.791348 | controller | - Zookeeper: OK 2026-06-10 06:05:00.791353 | controller | - Zuul: Waiting ... 2026-06-10 06:05:00.791358 | controller | - |_ Config Secrets: OK 2026-06-10 06:05:00.791363 | controller | - |_ Executor: OK 2026-06-10 06:05:00.792118 | controller | - |_ Merger: OK 2026-06-10 06:05:00.792134 | controller | - |_ Scheduler: OK 2026-06-10 06:05:00.792159 | controller | - |_ Web: Waiting ... 2026-06-10 06:05:00.792165 | controller | 2026-06-10 06:05:01.796275 | controller | 2026-06-10T06:05:01-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:05:02.404919 | controller | 2026-06-10T06:05:02-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:05:02.405015 | controller | 2026-06-10T06:05:02-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:05:02.415928 | controller | 2026-06-10T06:05:02-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:05:02.415992 | controller | 2026-06-10T06:05:02-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:05:02.435417 | controller | 2026-06-10T06:05:02-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:05:02.435470 | controller | 2026-06-10T06:05:02-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:05:02.809477 | controller | 2026-06-10T06:05:02-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:05:03.212692 | controller | 2026-06-10T06:05:03-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:05:03.212754 | controller | 2026-06-10T06:05:03-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:05:04.596244 | controller | 2026-06-10T06:05:04-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:05:04.596315 | controller | 2026-06-10T06:05:04-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:05:06.186122 | controller | 2026-06-10T06:05:06-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:05:06.186194 | controller | 2026-06-10T06:05:06-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:05:06.990808 | controller | 2026-06-10T06:05:06-04:00 INFO 2026-06-10 06:05:06.990878 | controller | - Config: Waiting ... 2026-06-10 06:05:06.990884 | controller | - Gateway: OK 2026-06-10 06:05:06.990888 | controller | - GitServer: OK 2026-06-10 06:05:06.990892 | controller | - HoundSearch: OK 2026-06-10 06:05:06.990896 | controller | - LogJuicer: OK 2026-06-10 06:05:06.990899 | controller | - Logserver: OK 2026-06-10 06:05:06.990903 | controller | - MariaDB: OK 2026-06-10 06:05:06.990906 | controller | - NodePoolBuilder: OK 2026-06-10 06:05:06.990910 | controller | - NodePoolLauncher: OK 2026-06-10 06:05:06.990914 | controller | - Zookeeper: OK 2026-06-10 06:05:06.990917 | controller | - Zuul: Waiting ... 2026-06-10 06:05:06.990921 | controller | - |_ Config Secrets: OK 2026-06-10 06:05:06.990925 | controller | - |_ Executor: OK 2026-06-10 06:05:06.990928 | controller | - |_ Merger: OK 2026-06-10 06:05:06.990932 | controller | - |_ Scheduler: OK 2026-06-10 06:05:06.990935 | controller | - |_ Web: Waiting ... 2026-06-10 06:05:06.990939 | controller | 2026-06-10 06:05:07.995830 | controller | 2026-06-10T06:05:07-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:05:08.602072 | controller | 2026-06-10T06:05:08-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:05:08.602156 | controller | 2026-06-10T06:05:08-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:05:08.617940 | controller | 2026-06-10T06:05:08-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:05:08.618020 | controller | 2026-06-10T06:05:08-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:05:08.633809 | controller | 2026-06-10T06:05:08-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:05:08.633878 | controller | 2026-06-10T06:05:08-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:05:09.010975 | controller | 2026-06-10T06:05:09-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:05:09.410232 | controller | 2026-06-10T06:05:09-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:05:09.410302 | controller | 2026-06-10T06:05:09-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:05:10.798813 | controller | 2026-06-10T06:05:10-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:05:10.798878 | controller | 2026-06-10T06:05:10-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:05:12.388697 | controller | 2026-06-10T06:05:12-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:05:12.388789 | controller | 2026-06-10T06:05:12-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:05:13.194096 | controller | 2026-06-10T06:05:13-04:00 INFO 2026-06-10 06:05:13.194180 | controller | - Config: Waiting ... 2026-06-10 06:05:13.194188 | controller | - Gateway: OK 2026-06-10 06:05:13.194194 | controller | - GitServer: OK 2026-06-10 06:05:13.194200 | controller | - HoundSearch: OK 2026-06-10 06:05:13.194205 | controller | - LogJuicer: OK 2026-06-10 06:05:13.194211 | controller | - Logserver: OK 2026-06-10 06:05:13.194216 | controller | - MariaDB: OK 2026-06-10 06:05:13.194221 | controller | - NodePoolBuilder: OK 2026-06-10 06:05:13.194226 | controller | - NodePoolLauncher: OK 2026-06-10 06:05:13.194231 | controller | - Zookeeper: OK 2026-06-10 06:05:13.194237 | controller | - Zuul: Waiting ... 2026-06-10 06:05:13.194242 | controller | - |_ Config Secrets: OK 2026-06-10 06:05:13.194247 | controller | - |_ Executor: OK 2026-06-10 06:05:13.194252 | controller | - |_ Merger: OK 2026-06-10 06:05:13.194258 | controller | - |_ Scheduler: OK 2026-06-10 06:05:13.194263 | controller | - |_ Web: Waiting ... 2026-06-10 06:05:13.194269 | controller | 2026-06-10 06:05:14.198704 | controller | 2026-06-10T06:05:14-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:05:14.802509 | controller | 2026-06-10T06:05:14-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:05:14.802570 | controller | 2026-06-10T06:05:14-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:05:14.813938 | controller | 2026-06-10T06:05:14-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:05:14.814001 | controller | 2026-06-10T06:05:14-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:05:14.829121 | controller | 2026-06-10T06:05:14-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:05:14.829177 | controller | 2026-06-10T06:05:14-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:05:15.208687 | controller | 2026-06-10T06:05:15-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:05:15.613354 | controller | 2026-06-10T06:05:15-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:05:15.613425 | controller | 2026-06-10T06:05:15-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:05:16.993235 | controller | 2026-06-10T06:05:16-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:05:16.993304 | controller | 2026-06-10T06:05:16-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:05:18.584388 | controller | 2026-06-10T06:05:18-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:05:18.584468 | controller | 2026-06-10T06:05:18-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:05:19.391946 | controller | 2026-06-10T06:05:19-04:00 INFO 2026-06-10 06:05:19.392009 | controller | - Config: Waiting ... 2026-06-10 06:05:19.392016 | controller | - Gateway: OK 2026-06-10 06:05:19.392022 | controller | - GitServer: OK 2026-06-10 06:05:19.392027 | controller | - HoundSearch: OK 2026-06-10 06:05:19.392032 | controller | - LogJuicer: OK 2026-06-10 06:05:19.392037 | controller | - Logserver: OK 2026-06-10 06:05:19.392042 | controller | - MariaDB: OK 2026-06-10 06:05:19.392047 | controller | - NodePoolBuilder: OK 2026-06-10 06:05:19.392052 | controller | - NodePoolLauncher: OK 2026-06-10 06:05:19.392056 | controller | - Zookeeper: OK 2026-06-10 06:05:19.392061 | controller | - Zuul: Waiting ... 2026-06-10 06:05:19.392066 | controller | - |_ Config Secrets: OK 2026-06-10 06:05:19.392071 | controller | - |_ Executor: OK 2026-06-10 06:05:19.392076 | controller | - |_ Merger: OK 2026-06-10 06:05:19.392081 | controller | - |_ Scheduler: OK 2026-06-10 06:05:19.392086 | controller | - |_ Web: Waiting ... 2026-06-10 06:05:19.392091 | controller | 2026-06-10 06:05:20.396393 | controller | 2026-06-10T06:05:20-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:05:20.999800 | controller | 2026-06-10T06:05:20-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:05:20.999868 | controller | 2026-06-10T06:05:20-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:05:21.009552 | controller | 2026-06-10T06:05:21-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:05:21.009638 | controller | 2026-06-10T06:05:21-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:05:21.024352 | controller | 2026-06-10T06:05:21-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:05:21.024411 | controller | 2026-06-10T06:05:21-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:05:21.406129 | controller | 2026-06-10T06:05:21-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:05:21.814983 | controller | 2026-06-10T06:05:21-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:05:21.815051 | controller | 2026-06-10T06:05:21-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:05:23.190298 | controller | 2026-06-10T06:05:23-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:05:23.190367 | controller | 2026-06-10T06:05:23-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:05:24.784117 | controller | 2026-06-10T06:05:24-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:05:24.784182 | controller | 2026-06-10T06:05:24-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:05:25.596733 | controller | 2026-06-10T06:05:25-04:00 INFO 2026-06-10 06:05:25.596812 | controller | - Config: Waiting ... 2026-06-10 06:05:25.596825 | controller | - Gateway: OK 2026-06-10 06:05:25.596833 | controller | - GitServer: OK 2026-06-10 06:05:25.596841 | controller | - HoundSearch: OK 2026-06-10 06:05:25.596848 | controller | - LogJuicer: OK 2026-06-10 06:05:25.596854 | controller | - Logserver: OK 2026-06-10 06:05:25.596860 | controller | - MariaDB: OK 2026-06-10 06:05:25.596865 | controller | - NodePoolBuilder: OK 2026-06-10 06:05:25.596870 | controller | - NodePoolLauncher: OK 2026-06-10 06:05:25.596875 | controller | - Zookeeper: OK 2026-06-10 06:05:25.596880 | controller | - Zuul: Waiting ... 2026-06-10 06:05:25.596886 | controller | - |_ Config Secrets: OK 2026-06-10 06:05:25.596891 | controller | - |_ Executor: OK 2026-06-10 06:05:25.596896 | controller | - |_ Merger: OK 2026-06-10 06:05:25.596901 | controller | - |_ Scheduler: OK 2026-06-10 06:05:25.596914 | controller | - |_ Web: Waiting ... 2026-06-10 06:05:25.596919 | controller | 2026-06-10 06:05:26.600815 | controller | 2026-06-10T06:05:26-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:05:27.209463 | controller | 2026-06-10T06:05:27-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:05:27.209529 | controller | 2026-06-10T06:05:27-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:05:27.217838 | controller | 2026-06-10T06:05:27-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:05:27.217868 | controller | 2026-06-10T06:05:27-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:05:27.233080 | controller | 2026-06-10T06:05:27-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:05:27.233143 | controller | 2026-06-10T06:05:27-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:05:27.619841 | controller | 2026-06-10T06:05:27-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:05:28.020148 | controller | 2026-06-10T06:05:28-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:05:28.020248 | controller | 2026-06-10T06:05:28-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:05:29.394593 | controller | 2026-06-10T06:05:29-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:05:29.394670 | controller | 2026-06-10T06:05:29-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:05:30.984993 | controller | 2026-06-10T06:05:30-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:05:30.985067 | controller | 2026-06-10T06:05:30-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:05:31.793726 | controller | 2026-06-10T06:05:31-04:00 INFO 2026-06-10 06:05:31.793790 | controller | - Config: Waiting ... 2026-06-10 06:05:31.793795 | controller | - Gateway: OK 2026-06-10 06:05:31.793828 | controller | - GitServer: OK 2026-06-10 06:05:31.793883 | controller | - HoundSearch: OK 2026-06-10 06:05:31.793912 | controller | - LogJuicer: OK 2026-06-10 06:05:31.793950 | controller | - Logserver: OK 2026-06-10 06:05:31.793987 | controller | - MariaDB: OK 2026-06-10 06:05:31.794060 | controller | - NodePoolBuilder: OK 2026-06-10 06:05:31.794111 | controller | - NodePoolLauncher: OK 2026-06-10 06:05:31.794162 | controller | - Zookeeper: OK 2026-06-10 06:05:31.794196 | controller | - Zuul: Waiting ... 2026-06-10 06:05:31.794234 | controller | - |_ Config Secrets: OK 2026-06-10 06:05:31.794270 | controller | - |_ Executor: OK 2026-06-10 06:05:31.794307 | controller | - |_ Merger: OK 2026-06-10 06:05:31.794344 | controller | - |_ Scheduler: OK 2026-06-10 06:05:31.794380 | controller | - |_ Web: Waiting ... 2026-06-10 06:05:31.794406 | controller | 2026-06-10 06:05:32.799557 | controller | 2026-06-10T06:05:32-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:05:33.403691 | controller | 2026-06-10T06:05:33-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:05:33.403790 | controller | 2026-06-10T06:05:33-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:05:33.417327 | controller | 2026-06-10T06:05:33-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:05:33.417394 | controller | 2026-06-10T06:05:33-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:05:33.436540 | controller | 2026-06-10T06:05:33-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:05:33.436627 | controller | 2026-06-10T06:05:33-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:05:33.810771 | controller | 2026-06-10T06:05:33-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:05:34.211196 | controller | 2026-06-10T06:05:34-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:05:34.211265 | controller | 2026-06-10T06:05:34-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:05:35.592210 | controller | 2026-06-10T06:05:35-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:05:35.592276 | controller | 2026-06-10T06:05:35-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:05:37.186228 | controller | 2026-06-10T06:05:37-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:05:37.186320 | controller | 2026-06-10T06:05:37-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:05:37.992992 | controller | 2026-06-10T06:05:37-04:00 INFO 2026-06-10 06:05:37.993060 | controller | - Config: Waiting ... 2026-06-10 06:05:37.993068 | controller | - Gateway: OK 2026-06-10 06:05:37.993074 | controller | - GitServer: OK 2026-06-10 06:05:37.993079 | controller | - HoundSearch: OK 2026-06-10 06:05:37.993093 | controller | - LogJuicer: OK 2026-06-10 06:05:37.993098 | controller | - Logserver: OK 2026-06-10 06:05:37.993101 | controller | - MariaDB: OK 2026-06-10 06:05:37.993105 | controller | - NodePoolBuilder: OK 2026-06-10 06:05:37.993109 | controller | - NodePoolLauncher: OK 2026-06-10 06:05:37.993175 | controller | - Zookeeper: OK 2026-06-10 06:05:37.993179 | controller | - Zuul: Waiting ... 2026-06-10 06:05:37.993182 | controller | - |_ Config Secrets: OK 2026-06-10 06:05:37.993186 | controller | - |_ Executor: OK 2026-06-10 06:05:37.993190 | controller | - |_ Merger: OK 2026-06-10 06:05:37.993193 | controller | - |_ Scheduler: OK 2026-06-10 06:05:37.993197 | controller | - |_ Web: Waiting ... 2026-06-10 06:05:37.993200 | controller | 2026-06-10 06:05:38.997879 | controller | 2026-06-10T06:05:38-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:05:39.603973 | controller | 2026-06-10T06:05:39-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:05:39.604042 | controller | 2026-06-10T06:05:39-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:05:39.618675 | controller | 2026-06-10T06:05:39-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:05:39.618759 | controller | 2026-06-10T06:05:39-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:05:39.638830 | controller | 2026-06-10T06:05:39-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:05:39.638943 | controller | 2026-06-10T06:05:39-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:05:40.013762 | controller | 2026-06-10T06:05:40-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:05:40.412983 | controller | 2026-06-10T06:05:40-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:05:40.413049 | controller | 2026-06-10T06:05:40-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:05:41.792622 | controller | 2026-06-10T06:05:41-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:05:41.792687 | controller | 2026-06-10T06:05:41-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:05:43.384513 | controller | 2026-06-10T06:05:43-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:05:43.384584 | controller | 2026-06-10T06:05:43-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:05:44.192949 | controller | 2026-06-10T06:05:44-04:00 INFO 2026-06-10 06:05:44.193013 | controller | - Config: Waiting ... 2026-06-10 06:05:44.193019 | controller | - Gateway: OK 2026-06-10 06:05:44.193023 | controller | - GitServer: OK 2026-06-10 06:05:44.193027 | controller | - HoundSearch: OK 2026-06-10 06:05:44.193039 | controller | - LogJuicer: OK 2026-06-10 06:05:44.193043 | controller | - Logserver: OK 2026-06-10 06:05:44.193047 | controller | - MariaDB: OK 2026-06-10 06:05:44.193050 | controller | - NodePoolBuilder: OK 2026-06-10 06:05:44.193054 | controller | - NodePoolLauncher: OK 2026-06-10 06:05:44.193057 | controller | - Zookeeper: OK 2026-06-10 06:05:44.193061 | controller | - Zuul: Waiting ... 2026-06-10 06:05:44.193064 | controller | - |_ Config Secrets: OK 2026-06-10 06:05:44.193068 | controller | - |_ Executor: OK 2026-06-10 06:05:44.193071 | controller | - |_ Merger: OK 2026-06-10 06:05:44.193075 | controller | - |_ Scheduler: OK 2026-06-10 06:05:44.193078 | controller | - |_ Web: Waiting ... 2026-06-10 06:05:44.193082 | controller | 2026-06-10 06:05:45.196671 | controller | 2026-06-10T06:05:45-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:05:45.803029 | controller | 2026-06-10T06:05:45-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:05:45.803102 | controller | 2026-06-10T06:05:45-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:05:45.815482 | controller | 2026-06-10T06:05:45-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:05:45.815575 | controller | 2026-06-10T06:05:45-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:05:45.832205 | controller | 2026-06-10T06:05:45-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:05:45.832276 | controller | 2026-06-10T06:05:45-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:05:46.208512 | controller | 2026-06-10T06:05:46-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:05:46.608826 | controller | 2026-06-10T06:05:46-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:05:46.608894 | controller | 2026-06-10T06:05:46-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:05:47.994189 | controller | 2026-06-10T06:05:47-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:05:47.994257 | controller | 2026-06-10T06:05:47-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:05:49.584191 | controller | 2026-06-10T06:05:49-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:05:49.584259 | controller | 2026-06-10T06:05:49-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:05:50.791454 | controller | 2026-06-10T06:05:50-04:00 INFO 2026-06-10 06:05:50.791533 | controller | - Config: OK 2026-06-10 06:05:50.791543 | controller | - Gateway: OK 2026-06-10 06:05:50.791552 | controller | - GitServer: OK 2026-06-10 06:05:50.791570 | controller | - HoundSearch: OK 2026-06-10 06:05:50.791578 | controller | - LogJuicer: OK 2026-06-10 06:05:50.791585 | controller | - Logserver: OK 2026-06-10 06:05:50.791592 | controller | - MariaDB: OK 2026-06-10 06:05:50.791599 | controller | - NodePoolBuilder: OK 2026-06-10 06:05:50.791637 | controller | - NodePoolLauncher: OK 2026-06-10 06:05:50.791645 | controller | - Zookeeper: OK 2026-06-10 06:05:50.791653 | controller | - Zuul: OK 2026-06-10 06:05:50.791662 | controller | 2026-06-10 06:05:50.791678 | controller | • [158.918 seconds] 2026-06-10 06:05:50.791686 | controller | ------------------------------ 2026-06-10 06:05:50.791694 | controller | Project Private Keys Rotation zuul-web config-errors remain empty after rotation 2026-06-10 06:05:50.791704 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/project_private_keys_rotation_test.go:103 2026-06-10 06:05:50.791712 | controller | STEP: Ensuring tenant have no config errors @ 06/10/26 06:05:50.791 2026-06-10 06:05:50.791720 | controller | 2026-06-10T06:05:50-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl -s zuul-web:9000/api/tenant/internal/config-errors] 2026-06-10 06:05:50.880085 | controller | 2026-06-10T06:05:50-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl -s zuul-web:9000/api/tenant/demo-tenant/config-errors] 2026-06-10 06:05:50.991156 | controller | • [0.200 seconds] 2026-06-10 06:05:50.991206 | controller | ------------------------------ 2026-06-10 06:05:50.991220 | controller | Project Private Keys Rotation project private keys changed when rotation was applied 2026-06-10 06:05:50.991227 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/project_private_keys_rotation_test.go:107 2026-06-10 06:05:50.991258 | controller | 2026-06-10T06:05:50-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c python3 << 'PYEOF' 2026-06-10 06:05:50.991271 | controller | import configparser 2026-06-10 06:05:50.991277 | controller | import hashlib 2026-06-10 06:05:50.991283 | controller | from zuul.zk import ZooKeeperClient 2026-06-10 06:05:50.991290 | controller | from zuul.lib.keystorage import KeyStorage 2026-06-10 06:05:50.991295 | controller | from zuul.lib import encryption 2026-06-10 06:05:50.991301 | controller | 2026-06-10 06:05:50.991307 | controller | config = configparser.ConfigParser() 2026-06-10 06:05:50.991312 | controller | config.read("/etc/zuul/zuul.conf") 2026-06-10 06:05:50.991318 | controller | password = config["keystore"]["password"].encode("utf-8") 2026-06-10 06:05:50.991324 | controller | zk_client = ZooKeeperClient.fromConfig(config) 2026-06-10 06:05:50.991329 | controller | zk_client.connect() 2026-06-10 06:05:50.991335 | controller | lines = [] 2026-06-10 06:05:50.991341 | controller | for path, obj in KeyStorage(zk_client, "unused").exportKeys().get("keys", {}).items(): 2026-06-10 06:05:50.991346 | controller | if path.endswith("/secrets") and "keys" in obj and len(obj.get("keys", [])) == 1: 2026-06-10 06:05:50.991352 | controller | created = obj["keys"][0]["created"] 2026-06-10 06:05:50.991359 | controller | priv_pem = obj["keys"][0]["private_key"] 2026-06-10 06:05:50.991364 | controller | fingerprint = hashlib.sha256(priv_pem.encode()).hexdigest()[:16] 2026-06-10 06:05:50.991370 | controller | lines.append(f"{path} {created} {fingerprint}") 2026-06-10 06:05:50.991375 | controller | if "demo-project" in path: 2026-06-10 06:05:50.991381 | controller | priv, _ = encryption.deserialize_rsa_keypair(priv_pem.encode("utf-8"), password) 2026-06-10 06:05:50.991386 | controller | pem = encryption.serialize_rsa_private_key(priv, None) 2026-06-10 06:05:50.991392 | controller | with open("/tmp/demo-project.key", "wb") as f: 2026-06-10 06:05:50.991397 | controller | f.write(pem) 2026-06-10 06:05:50.991403 | controller | print("\n".join(sorted(lines))) 2026-06-10 06:05:50.991408 | controller | PYEOF 2026-06-10 06:05:50.991414 | controller | ] 2026-06-10 06:05:52.291909 | controller | STEP: Project keys after rotation: /keystorage/gerrit/config/config/secrets 1781085812 e75e9032e87f15a3 2026-06-10 06:05:52.291974 | controller | /keystorage/gerrit/demo-project/demo-project/secrets 1781085796 b79a82b40397eb7d 2026-06-10 06:05:52.291982 | controller | /keystorage/gerrit/demo-tenant-config/demo-tenant-config/secrets 1781085810 999646e297477099 2026-06-10 06:05:52.291986 | controller | /keystorage/git-server/system-config/system-config/secrets 1781085814 502769b9fc363003 2026-06-10 06:05:52.291990 | controller | /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/secrets 1781085810 78875a867a4badf3 2026-06-10 06:05:52.291994 | controller | @ 06/10/26 06:05:52.291 2026-06-10 06:05:52.292006 | controller | STEP: Pulling latest demo-project and reading .zuul.d/secrets.yaml @ 06/10/26 06:05:52.291 2026-06-10 06:05:52.358067 | controller | STEP: Writing secrets.yaml into zuul-scheduler pod for decryption @ 06/10/26 06:05:52.357 2026-06-10 06:05:52.358125 | controller | 2026-06-10T06:05:52-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c cat > /tmp/secrets.yaml] 2026-06-10 06:05:52.434516 | controller | STEP: Loading project keys state (writes demo-project key to /tmp/demo-project.key) @ 06/10/26 06:05:52.433 2026-06-10 06:05:52.434574 | controller | 2026-06-10T06:05:52-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c python3 << 'PYEOF' 2026-06-10 06:05:52.434579 | controller | import configparser 2026-06-10 06:05:52.434583 | controller | import hashlib 2026-06-10 06:05:52.434588 | controller | from zuul.zk import ZooKeeperClient 2026-06-10 06:05:52.434593 | controller | from zuul.lib.keystorage import KeyStorage 2026-06-10 06:05:52.434597 | controller | from zuul.lib import encryption 2026-06-10 06:05:52.434601 | controller | 2026-06-10 06:05:52.434617 | controller | config = configparser.ConfigParser() 2026-06-10 06:05:52.434629 | controller | config.read("/etc/zuul/zuul.conf") 2026-06-10 06:05:52.434633 | controller | password = config["keystore"]["password"].encode("utf-8") 2026-06-10 06:05:52.434637 | controller | zk_client = ZooKeeperClient.fromConfig(config) 2026-06-10 06:05:52.434641 | controller | zk_client.connect() 2026-06-10 06:05:52.434645 | controller | lines = [] 2026-06-10 06:05:52.434649 | controller | for path, obj in KeyStorage(zk_client, "unused").exportKeys().get("keys", {}).items(): 2026-06-10 06:05:52.434657 | controller | if path.endswith("/secrets") and "keys" in obj and len(obj.get("keys", [])) == 1: 2026-06-10 06:05:52.434661 | controller | created = obj["keys"][0]["created"] 2026-06-10 06:05:52.434665 | controller | priv_pem = obj["keys"][0]["private_key"] 2026-06-10 06:05:52.434668 | controller | fingerprint = hashlib.sha256(priv_pem.encode()).hexdigest()[:16] 2026-06-10 06:05:52.434672 | controller | lines.append(f"{path} {created} {fingerprint}") 2026-06-10 06:05:52.434676 | controller | if "demo-project" in path: 2026-06-10 06:05:52.434680 | controller | priv, _ = encryption.deserialize_rsa_keypair(priv_pem.encode("utf-8"), password) 2026-06-10 06:05:52.434683 | controller | pem = encryption.serialize_rsa_private_key(priv, None) 2026-06-10 06:05:52.434687 | controller | with open("/tmp/demo-project.key", "wb") as f: 2026-06-10 06:05:52.434690 | controller | f.write(pem) 2026-06-10 06:05:52.434694 | controller | print("\n".join(sorted(lines))) 2026-06-10 06:05:52.434698 | controller | PYEOF 2026-06-10 06:05:52.434702 | controller | ] 2026-06-10 06:05:53.516725 | controller | STEP: Decrypting with Zuul library @ 06/10/26 06:05:53.516 2026-06-10 06:05:53.516807 | controller | 2026-06-10T06:05:53-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c python3 << 'PYEOF' 2026-06-10 06:05:53.516813 | controller | from zuul.lib import encryption 2026-06-10 06:05:53.516817 | controller | import zuul.configloader 2026-06-10 06:05:53.516822 | controller | import zuul.model 2026-06-10 06:05:53.516826 | controller | 2026-06-10 06:05:53.516830 | controller | with open("/tmp/demo-project.key", "rb") as f: 2026-06-10 06:05:53.516835 | controller | private_secrets_key, _ = encryption.deserialize_rsa_keypair(f.read()) 2026-06-10 06:05:53.516839 | controller | with open("/tmp/secrets.yaml") as f: 2026-06-10 06:05:53.516843 | controller | yaml_content = f.read() 2026-06-10 06:05:53.516846 | controller | 2026-06-10 06:05:53.516850 | controller | parser = zuul.configloader.SecretParser(None) 2026-06-10 06:05:53.516854 | controller | sc = zuul.model.SourceContext(None, "project", None, "master", "path") 2026-06-10 06:05:53.516857 | controller | data = zuul.configloader.safe_load_yaml(yaml_content, sc) 2026-06-10 06:05:53.516861 | controller | for element in data: 2026-06-10 06:05:53.516865 | controller | if "secret" not in element: 2026-06-10 06:05:53.516869 | controller | continue 2026-06-10 06:05:53.516872 | controller | secret = parser.fromYaml(element["secret"]) 2026-06-10 06:05:53.516876 | controller | decrypted = secret.decrypt(private_secrets_key).secret_data 2026-06-10 06:05:53.516881 | controller | if isinstance(decrypted, dict) and "password" in decrypted: 2026-06-10 06:05:53.516884 | controller | print(decrypted["password"], end="") 2026-06-10 06:05:53.516888 | controller | elif isinstance(decrypted, str): 2026-06-10 06:05:53.516891 | controller | print(decrypted, end="") 2026-06-10 06:05:53.516895 | controller | else: 2026-06-10 06:05:53.516899 | controller | print(decrypted, end="") 2026-06-10 06:05:53.516902 | controller | break 2026-06-10 06:05:53.516906 | controller | PYEOF 2026-06-10 06:05:53.516910 | controller | ] 2026-06-10 06:05:54.508160 | controller | STEP: Decrypted secret: -----BEGIN OPENSSH PRIVATE KEY----- 2026-06-10 06:05:54.508248 | controller | b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW 2026-06-10 06:05:54.508255 | controller | QyNTUxOQAAACAumfF90VjkYx8T97dM3BSYT+YWSzxyGmICFdF+ZKfzmAAAAKiTMlOPkzJT 2026-06-10 06:05:54.508258 | controller | jwAAAAtzc2gtZWQyNTUxOQAAACAumfF90VjkYx8T97dM3BSYT+YWSzxyGmICFdF+ZKfzmA 2026-06-10 06:05:54.508262 | controller | AAAEArIPFF5KJkorka89TXpyNl+lyP9/jYayG4uZZUpsGkHy6Z8X3RWORjHxP3t0zcFJhP 2026-06-10 06:05:54.508265 | controller | 5hZLPHIaYgIV0X5kp/OYAAAAInp1dWwtd29ya2VyQG5wMDAwMDA2OTU2NS5ub3ZhbG9jYW 2026-06-10 06:05:54.508269 | controller | wBAgM= 2026-06-10 06:05:54.508273 | controller | -----END OPENSSH PRIVATE KEY----- @ 06/10/26 06:05:54.507 2026-06-10 06:05:54.508278 | controller | • [3.517 seconds] 2026-06-10 06:05:54.508282 | controller | ------------------------------ 2026-06-10 06:05:54.508285 | controller | Deploy Tests Reconcile default 2026-06-10 06:05:54.508289 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/deploy_test.go:11 2026-06-10 06:05:54.511949 | controller | 2026-06-10T06:05:54-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:05:55.115662 | controller | 2026-06-10T06:05:55-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:05:55.115740 | controller | 2026-06-10T06:05:55-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:05:55.124841 | controller | 2026-06-10T06:05:55-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:05:55.206014 | controller | 2026-06-10T06:05:55-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:05:55.211286 | controller | 2026-06-10T06:05:55-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:05:55.211346 | controller | 2026-06-10T06:05:55-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:05:55.229544 | controller | 2026-06-10T06:05:55-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:05:55.229649 | controller | 2026-06-10T06:05:55-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:05:55.522341 | controller | 2026-06-10T06:05:55-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:05:55.927560 | controller | 2026-06-10T06:05:55-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:05:55.927641 | controller | 2026-06-10T06:05:55-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:05:56.935991 | controller | 2026-06-10T06:05:56-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:05:56.936063 | controller | 2026-06-10T06:05:56-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:05:57.916026 | controller | 2026-06-10T06:05:57-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:05:57.916089 | controller | 2026-06-10T06:05:57-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:05:59.121897 | controller | 2026-06-10T06:05:59-04:00 INFO 2026-06-10 06:05:59.121975 | controller | - Config: OK 2026-06-10 06:05:59.121982 | controller | - Gateway: OK 2026-06-10 06:05:59.121988 | controller | - GitServer: OK 2026-06-10 06:05:59.121993 | controller | - HoundSearch: OK 2026-06-10 06:05:59.121998 | controller | - LogJuicer: OK 2026-06-10 06:05:59.122003 | controller | - Logserver: OK 2026-06-10 06:05:59.122008 | controller | - MariaDB: OK 2026-06-10 06:05:59.122013 | controller | - NodePoolBuilder: OK 2026-06-10 06:05:59.122018 | controller | - NodePoolLauncher: OK 2026-06-10 06:05:59.122023 | controller | - Zookeeper: OK 2026-06-10 06:05:59.122029 | controller | - Zuul: OK 2026-06-10 06:05:59.122034 | controller | 2026-06-10 06:05:59.122073 | controller | • [4.614 seconds] 2026-06-10 06:05:59.122081 | controller | ------------------------------ 2026-06-10 06:05:59.122086 | controller | Deploy Tests Match zuul secrets 2026-06-10 06:05:59.122092 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/deploy_test.go:14 2026-06-10 06:05:59.123486 | controller | 2026-06-10T06:05:59-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [cat /etc/zuul/zuul.conf] 2026-06-10 06:05:59.204139 | controller | • [0.082 seconds] 2026-06-10 06:05:59.204216 | controller | ------------------------------ 2026-06-10 06:05:59.204261 | controller | Deploy Tests Connect to services 2026-06-10 06:05:59.204275 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/deploy_test.go:17 2026-06-10 06:05:59.319466 | controller | • [0.115 seconds] 2026-06-10 06:05:59.319549 | controller | ------------------------------ 2026-06-10 06:05:59.319617 | controller | Secret Rotations zuul-auth works 2026-06-10 06:05:59.319646 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/secrets_rotation_test.go:60 2026-06-10 06:05:59.324121 | controller | STEP: Deleting secrets not rotated by the CLI @ 06/10/26 06:05:59.324 2026-06-10 06:05:59.332887 | controller | STEP: Running secret rotation CLI @ 06/10/26 06:05:59.332 2026-06-10 06:05:59.340005 | controller | 2026-06-10T06:05:59-04:00 INFO Warning: At least one executor running, this may cause issues when rotating Zookeeper secrets. 2026-06-10 06:05:59.340044 | controller | 2026-06-10T06:05:59-04:00 INFO Rotating Keystore password... 2026-06-10 06:05:59.343823 | controller | 2026-06-10T06:05:59-04:00 INFO Creating secret, name: zuul-keystore-password-new 2026-06-10 06:05:59.343864 | controller | 2026-06-10T06:05:59-04:00 INFO Creating object, name: zuul-keystore-password-new 2026-06-10 06:06:00.331845 | controller | 2026-06-10T06:06:00-04:00 INFO Running pod execution pod: zuul-kazoo, command: [cat /var/lib/zuul/keys-backup.json] 2026-06-10 06:06:00.419313 | controller | 2026-06-10T06:06:00-04:00 INFO Updating object, name: zuul-keystore-password 2026-06-10 06:06:00.421993 | controller | 2026-06-10T06:06:00-04:00 INFO Deleting object, name: zuul-keystore-password-new 2026-06-10 06:06:00.425007 | controller | 2026-06-10T06:06:00-04:00 INFO Deleting object, name: zuul-kazoo 2026-06-10 06:06:00.427908 | controller | 2026-06-10T06:06:00-04:00 INFO Killing every kazoo client... 2026-06-10 06:06:00.441894 | controller | 2026-06-10T06:06:00-04:00 INFO Warning: Killing ZooKeeper client: nodepool-builder-0 nodepool-builder 2026-06-10 06:06:00.441961 | controller | 2026-06-10T06:06:00-04:00 INFO Running pod execution pod: nodepool-builder-0, command: [kill -9 1] 2026-06-10 06:06:00.524820 | controller | 2026-06-10T06:06:00-04:00 INFO Deleting object, name: nodepool-builder 2026-06-10 06:06:00.528441 | controller | 2026-06-10T06:06:00-04:00 INFO Warning: Killing ZooKeeper client: nodepool-launcher-569b4cc7cf-qddvj launcher 2026-06-10 06:06:00.528487 | controller | 2026-06-10T06:06:00-04:00 INFO Running pod execution pod: nodepool-launcher-569b4cc7cf-qddvj, command: [kill -9 1] 2026-06-10 06:06:00.631022 | controller | 2026-06-10T06:06:00-04:00 INFO Deleting object, name: launcher 2026-06-10 06:06:00.632911 | controller | 2026-06-10T06:06:00-04:00 INFO Warning: Killing ZooKeeper client: zuul-db-provision-r5bxv mariadb-client 2026-06-10 06:06:00.632948 | controller | 2026-06-10T06:06:00-04:00 INFO Running pod execution pod: zuul-db-provision-r5bxv, command: [kill -9 1] 2026-06-10 06:06:00.646288 | controller | 2026-06-10T06:06:00-04:00 INFO Deleting object, name: mariadb-client 2026-06-10 06:06:00.649096 | controller | 2026-06-10T06:06:00-04:00 INFO Warning: Killing ZooKeeper client: zuul-executor-0 zuul-executor 2026-06-10 06:06:00.649119 | controller | 2026-06-10T06:06:00-04:00 INFO Running pod execution pod: zuul-executor-0, command: [kill -9 1] 2026-06-10 06:07:03.105045 | controller | 2026-06-10T06:07:03-04:00 INFO Deleting object, name: zuul-executor 2026-06-10 06:07:03.110240 | controller | 2026-06-10T06:07:03-04:00 INFO Warning: Killing ZooKeeper client: zuul-kazoo zuul-kazoo 2026-06-10 06:07:03.110276 | controller | 2026-06-10T06:07:03-04:00 INFO Running pod execution pod: zuul-kazoo, command: [kill -9 1] 2026-06-10 06:07:03.117873 | controller | 2026-06-10T06:07:03-04:00 INFO Deleting object, name: zuul-kazoo 2026-06-10 06:07:03.119591 | controller | 2026-06-10T06:07:03-04:00 INFO Warning: Killing ZooKeeper client: zuul-merger-0 zuul-merger 2026-06-10 06:07:03.119625 | controller | 2026-06-10T06:07:03-04:00 INFO Running pod execution pod: zuul-merger-0, command: [kill -9 1] 2026-06-10 06:07:03.210907 | controller | 2026-06-10T06:07:03-04:00 INFO Deleting object, name: zuul-merger 2026-06-10 06:07:03.214090 | controller | 2026-06-10T06:07:03-04:00 INFO Warning: Killing ZooKeeper client: zuul-scheduler-0 zuul-scheduler 2026-06-10 06:07:03.214144 | controller | 2026-06-10T06:07:03-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [kill -9 1] 2026-06-10 06:07:03.300276 | controller | 2026-06-10T06:07:03-04:00 INFO Deleting object, name: zuul-scheduler 2026-06-10 06:07:03.303835 | controller | 2026-06-10T06:07:03-04:00 INFO Warning: Killing ZooKeeper client: zuul-web-858b6f55b5-tqmbs zuul-web 2026-06-10 06:07:03.303866 | controller | 2026-06-10T06:07:03-04:00 INFO Running pod execution pod: zuul-web-858b6f55b5-tqmbs, command: [kill -9 1] 2026-06-10 06:07:03.387064 | controller | 2026-06-10T06:07:03-04:00 INFO Deleting object, name: zuul-web 2026-06-10 06:07:03.389532 | controller | 2026-06-10T06:07:03-04:00 INFO Warning: Killing ZooKeeper client: zuul-weeder-57c6567f5f-4zqwk zuul-weeder 2026-06-10 06:07:03.389561 | controller | 2026-06-10T06:07:03-04:00 INFO Running pod execution pod: zuul-weeder-57c6567f5f-4zqwk, command: [kill -9 1] 2026-06-10 06:07:03.470226 | controller | 2026-06-10T06:07:03-04:00 INFO Deleting object, name: zuul-weeder 2026-06-10 06:07:03.473441 | controller | 2026-06-10T06:07:03-04:00 INFO Deleting object, name: zookeeper 2026-06-10 06:07:03.476094 | controller | 2026-06-10T06:07:03-04:00 INFO Rotating Zuul Client Authenticator secret... 2026-06-10 06:07:03.477925 | controller | 2026-06-10T06:07:03-04:00 INFO Deleting object, name: zuul-auth-secret 2026-06-10 06:07:03.483149 | controller | 2026-06-10T06:07:03-04:00 INFO Creating secret, name: zuul-auth-secret 2026-06-10 06:07:03.483221 | controller | 2026-06-10T06:07:03-04:00 INFO Creating object, name: zuul-auth-secret 2026-06-10 06:07:03.486420 | controller | 2026-06-10T06:07:03-04:00 INFO Rotating Zuul DB Connection secret... 2026-06-10 06:07:03.487732 | controller | 2026-06-10T06:07:03-04:00 INFO Deleting object, name: zuul-db-connection 2026-06-10 06:07:03.490228 | controller | 2026-06-10T06:07:03-04:00 INFO Rotating Zookeeper certificates... 2026-06-10 06:07:03.493835 | controller | 2026-06-10T06:07:03-04:00 INFO Deleting object, name: zookeeper-client-tls 2026-06-10 06:07:03.497269 | controller | 2026-06-10T06:07:03-04:00 INFO Deleting object, name: zookeeper-server-tls 2026-06-10 06:07:03.500041 | controller | 2026-06-10T06:07:03-04:00 INFO Force Restart impacted services... 2026-06-10 06:07:03.500096 | controller | 2026-06-10T06:07:03-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 06:07:03.502660 | controller | 2026-06-10T06:07:03-04:00 INFO Deleting object, name: zuul-web-858b6f55b5-tqmbs 2026-06-10 06:07:03.504858 | controller | STEP: Reconciling @ 06/10/26 06:07:03.504 2026-06-10 06:07:03.698910 | controller | 2026-06-10T06:07:03-04:00 INFO Warning: Killing ZooKeeper client: nodepool-launcher-569b4cc7cf-qddvj launcher 2026-06-10 06:07:03.698998 | controller | 2026-06-10T06:07:03-04:00 INFO Running pod execution pod: nodepool-launcher-569b4cc7cf-qddvj, command: [kill -9 1] 2026-06-10 06:07:03.784940 | controller | 2026-06-10T06:07:03-04:00 INFO Deleting object, name: launcher 2026-06-10 06:07:03.786743 | controller | 2026-06-10T06:07:03-04:00 INFO Warning: Killing ZooKeeper client: zuul-db-provision-r5bxv mariadb-client 2026-06-10 06:07:03.786765 | controller | 2026-06-10T06:07:03-04:00 INFO Running pod execution pod: zuul-db-provision-r5bxv, command: [kill -9 1] 2026-06-10 06:07:03.801724 | controller | 2026-06-10T06:07:03-04:00 INFO Deleting object, name: mariadb-client 2026-06-10 06:07:03.803791 | controller | 2026-06-10T06:07:03-04:00 INFO Warning: Killing ZooKeeper client: zuul-executor-0 zuul-executor 2026-06-10 06:07:03.803816 | controller | 2026-06-10T06:07:03-04:00 INFO Running pod execution pod: zuul-executor-0, command: [kill -9 1] 2026-06-10 06:07:03.818597 | controller | 2026-06-10T06:07:03-04:00 INFO Deleting object, name: zuul-executor 2026-06-10 06:07:03.820293 | controller | 2026-06-10T06:07:03-04:00 INFO Warning: Killing ZooKeeper client: zuul-merger-0 zuul-merger 2026-06-10 06:07:03.820312 | controller | 2026-06-10T06:07:03-04:00 INFO Running pod execution pod: zuul-merger-0, command: [kill -9 1] 2026-06-10 06:07:06.172961 | controller | 2026-06-10T06:07:06-04:00 INFO Deleting object, name: zuul-merger 2026-06-10 06:07:06.175580 | controller | 2026-06-10T06:07:06-04:00 INFO Warning: Killing ZooKeeper client: zuul-scheduler-0 zuul-scheduler 2026-06-10 06:07:06.175661 | controller | 2026-06-10T06:07:06-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [kill -9 1] 2026-06-10 06:07:33.483060 | controller | 2026-06-10T06:07:33-04:00 INFO Deleting object, name: zuul-scheduler 2026-06-10 06:07:33.483350 | controller | 2026-06-10T06:07:33-04:00 INFO Warning: Killing ZooKeeper client: zuul-web-858b6f55b5-tqmbs zuul-web 2026-06-10 06:07:33.483361 | controller | 2026-06-10T06:07:33-04:00 INFO Running pod execution pod: zuul-web-858b6f55b5-tqmbs, command: [kill -9 1] 2026-06-10 06:07:33.563501 | controller | 2026-06-10T06:07:33-04:00 INFO Deleting object, name: zuul-web 2026-06-10 06:07:33.566152 | controller | 2026-06-10T06:07:33-04:00 INFO Warning: Killing ZooKeeper client: zuul-weeder-57c6567f5f-4zqwk zuul-weeder 2026-06-10 06:07:33.566177 | controller | 2026-06-10T06:07:33-04:00 INFO Running pod execution pod: zuul-weeder-57c6567f5f-4zqwk, command: [kill -9 1] 2026-06-10 06:07:33.575480 | controller | 2026-06-10T06:07:33-04:00 INFO Deleting object, name: zuul-weeder 2026-06-10 06:07:33.580281 | controller | 2026-06-10T06:07:33-04:00 INFO Deleting object, name: zookeeper 2026-06-10 06:07:33.583512 | controller | 2026-06-10T06:07:33-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:07:33.614277 | controller | 2026-06-10T06:07:33-04:00 INFO Creating object, name: zuul-db-connection 2026-06-10 06:07:33.618481 | controller | 2026-06-10T06:07:33-04:00 INFO Re-provisioning db password 2026-06-10 06:07:33.618532 | controller | 2026-06-10T06:07:33-04:00 INFO Deleting object, name: zuul-db-provision 2026-06-10 06:07:33.621299 | controller | 2026-06-10T06:07:33-04:00 INFO Creating object, name: zuul-db-provision 2026-06-10 06:07:34.190420 | controller | 2026-06-10T06:07:34-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:07:34.190482 | controller | 2026-06-10T06:07:34-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:07:34.195735 | controller | 2026-06-10T06:07:34-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:07:34.295426 | controller | 2026-06-10T06:07:34-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:07:34.302954 | controller | 2026-06-10T06:07:34-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:07:34.303022 | controller | 2026-06-10T06:07:34-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:07:34.319008 | controller | 2026-06-10T06:07:34-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:07:34.319070 | controller | 2026-06-10T06:07:34-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:07:34.386503 | controller | 2026-06-10T06:07:34-04:00 INFO Deleting object, name: zookeeper-client-tls 2026-06-10 06:07:34.586533 | controller | 2026-06-10T06:07:34-04:00 INFO Deleting object, name: zookeeper-server-tls 2026-06-10 06:07:37.939877 | controller | 2026-06-10T06:07:37-04:00 INFO Creating object, name: zookeeper-client-tls 2026-06-10 06:07:39.447423 | controller | 2026-06-10T06:07:39-04:00 INFO Creating object, name: zookeeper-server-tls 2026-06-10 06:07:39.459710 | controller | 2026-06-10T06:07:39-04:00 INFO Creating object, name: zookeeper 2026-06-10 06:07:39.462914 | controller | 2026-06-10T06:07:39-04:00 INFO Waiting for Zookeeper service to be ready ... 2026-06-10 06:07:39.462964 | controller | 2026-06-10T06:07:39-04:00 INFO 2026-06-10 06:07:39.462970 | controller | - Config: Waiting ... 2026-06-10 06:07:39.462975 | controller | - Gateway: OK 2026-06-10 06:07:39.462979 | controller | - GitServer: OK 2026-06-10 06:07:39.462987 | controller | - HoundSearch: OK 2026-06-10 06:07:39.462991 | controller | - LogJuicer: OK 2026-06-10 06:07:39.462994 | controller | - Logserver: OK 2026-06-10 06:07:39.462998 | controller | - MariaDB: Waiting ... 2026-06-10 06:07:39.463001 | controller | - Zookeeper: Waiting ... 2026-06-10 06:07:39.463005 | controller | 2026-06-10 06:07:40.469869 | controller | 2026-06-10T06:07:40-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:07:40.519590 | controller | 2026-06-10T06:07:40-04:00 INFO Creating object, name: zuul-db-provision 2026-06-10 06:07:41.077048 | controller | 2026-06-10T06:07:41-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:07:41.077124 | controller | 2026-06-10T06:07:41-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:07:41.089182 | controller | 2026-06-10T06:07:41-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:07:41.089256 | controller | 2026-06-10T06:07:41-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:07:41.108001 | controller | 2026-06-10T06:07:41-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:07:41.108076 | controller | 2026-06-10T06:07:41-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:07:41.482497 | controller | 2026-06-10T06:07:41-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-10 06:07:55.524564 | controller | 2026-06-10T06:07:55-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-10 06:07:55.524648 | controller | 2026-06-10T06:07:55-04:00 INFO 2026-06-10 06:07:55.524656 | controller | - Config: Waiting ... 2026-06-10 06:07:55.524661 | controller | - Gateway: OK 2026-06-10 06:07:55.524664 | controller | - GitServer: OK 2026-06-10 06:07:55.524668 | controller | - HoundSearch: OK 2026-06-10 06:07:55.524672 | controller | - LogJuicer: OK 2026-06-10 06:07:55.524676 | controller | - Logserver: OK 2026-06-10 06:07:55.524679 | controller | - MariaDB: Waiting ... 2026-06-10 06:07:55.524682 | controller | - Zookeeper: OK 2026-06-10 06:07:55.524687 | controller | 2026-06-10 06:07:56.529578 | controller | 2026-06-10T06:07:56-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:07:57.133312 | controller | 2026-06-10T06:07:57-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:07:57.133373 | controller | 2026-06-10T06:07:57-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:07:57.142190 | controller | 2026-06-10T06:07:57-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:07:57.142233 | controller | 2026-06-10T06:07:57-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:07:57.154167 | controller | 2026-06-10T06:07:57-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:07:57.154206 | controller | 2026-06-10T06:07:57-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:07:57.542424 | controller | 2026-06-10T06:07:57-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:07:57.942480 | controller | 2026-06-10T06:07:57-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:07:57.942543 | controller | 2026-06-10T06:07:57-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:07:58.137181 | controller | 2026-06-10T06:07:58-04:00 INFO Creating object, name: nodepool-builder 2026-06-10 06:07:58.927801 | controller | 2026-06-10T06:07:58-04:00 INFO Updating secret, name: zuul-config 2026-06-10 06:07:58.927858 | controller | 2026-06-10T06:07:58-04:00 INFO Updating object, name: zuul-config 2026-06-10 06:07:59.148310 | controller | 2026-06-10T06:07:59-04:00 INFO Creating object, name: zuul-scheduler 2026-06-10 06:07:59.173492 | controller | 2026-06-10T06:07:59-04:00 INFO Creating object, name: zuul-executor 2026-06-10 06:07:59.932200 | controller | 2026-06-10T06:07:59-04:00 INFO Creating object, name: zuul-web 2026-06-10 06:08:00.733847 | controller | 2026-06-10T06:08:00-04:00 INFO Creating object, name: zuul-merger 2026-06-10 06:08:00.739148 | controller | 2026-06-10T06:08:00-04:00 INFO 2026-06-10 06:08:00.739181 | controller | - Config: Waiting ... 2026-06-10 06:08:00.739186 | controller | - Gateway: OK 2026-06-10 06:08:00.739190 | controller | - GitServer: OK 2026-06-10 06:08:00.739195 | controller | - HoundSearch: OK 2026-06-10 06:08:00.739199 | controller | - LogJuicer: OK 2026-06-10 06:08:00.739202 | controller | - Logserver: OK 2026-06-10 06:08:00.739205 | controller | - MariaDB: OK 2026-06-10 06:08:00.739209 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:08:00.739213 | controller | - NodePoolLauncher: OK 2026-06-10 06:08:00.739216 | controller | - Zookeeper: OK 2026-06-10 06:08:00.739220 | controller | - Zuul: Waiting ... 2026-06-10 06:08:00.739224 | controller | - |_ Config Secrets: OK 2026-06-10 06:08:00.739227 | controller | - |_ Executor: Waiting ... 2026-06-10 06:08:00.739231 | controller | - |_ Merger: Waiting ... 2026-06-10 06:08:00.739234 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:08:00.739238 | controller | - |_ Web: Waiting ... 2026-06-10 06:08:00.739242 | controller | 2026-06-10 06:08:01.744584 | controller | 2026-06-10T06:08:01-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:08:02.348020 | controller | 2026-06-10T06:08:02-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:08:02.348087 | controller | 2026-06-10T06:08:02-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:08:02.356442 | controller | 2026-06-10T06:08:02-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:08:02.356489 | controller | 2026-06-10T06:08:02-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:08:02.381268 | controller | 2026-06-10T06:08:02-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:08:02.381334 | controller | 2026-06-10T06:08:02-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:08:02.755449 | controller | 2026-06-10T06:08:02-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:08:03.159429 | controller | 2026-06-10T06:08:03-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:08:03.159498 | controller | 2026-06-10T06:08:03-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:08:04.537424 | controller | 2026-06-10T06:08:04-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:08:34.633911 | controller | 2026-06-10T06:08:34-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 06:08:34.667881 | controller | 2026-06-10T06:08:34-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:08:34.667951 | controller | 2026-06-10T06:08:34-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:08:35.045424 | controller | 2026-06-10T06:08:35-04:00 INFO 2026-06-10 06:08:35.045502 | controller | - Config: Waiting ... 2026-06-10 06:08:35.045510 | controller | - Gateway: OK 2026-06-10 06:08:35.045518 | controller | - GitServer: OK 2026-06-10 06:08:35.045525 | controller | - HoundSearch: OK 2026-06-10 06:08:35.045532 | controller | - LogJuicer: OK 2026-06-10 06:08:35.045539 | controller | - Logserver: OK 2026-06-10 06:08:35.045560 | controller | - MariaDB: OK 2026-06-10 06:08:35.045568 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:08:35.045576 | controller | - NodePoolLauncher: OK 2026-06-10 06:08:35.045582 | controller | - Zookeeper: OK 2026-06-10 06:08:35.045589 | controller | - Zuul: Waiting ... 2026-06-10 06:08:35.045596 | controller | - |_ Config Secrets: OK 2026-06-10 06:08:35.045603 | controller | - |_ Executor: OK 2026-06-10 06:08:35.045649 | controller | - |_ Merger: OK 2026-06-10 06:08:35.045657 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:08:35.045664 | controller | - |_ Web: Waiting ... 2026-06-10 06:08:35.045671 | controller | 2026-06-10 06:08:36.049980 | controller | 2026-06-10T06:08:36-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:08:36.652761 | controller | 2026-06-10T06:08:36-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:08:36.652838 | controller | 2026-06-10T06:08:36-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:08:36.663063 | controller | 2026-06-10T06:08:36-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:08:36.663120 | controller | 2026-06-10T06:08:36-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:08:36.679656 | controller | 2026-06-10T06:08:36-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:08:36.679732 | controller | 2026-06-10T06:08:36-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:08:37.062151 | controller | 2026-06-10T06:08:37-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:08:37.463229 | controller | 2026-06-10T06:08:37-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:08:37.463294 | controller | 2026-06-10T06:08:37-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:08:38.841657 | controller | 2026-06-10T06:08:38-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:09:00.922739 | controller | 2026-06-10T06:09:00-04:00 INFO Creating object, name: zuul-weeder 2026-06-10 06:09:00.977091 | controller | 2026-06-10T06:09:00-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:09:00.977147 | controller | 2026-06-10T06:09:00-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:09:01.338232 | controller | 2026-06-10T06:09:01-04:00 INFO 2026-06-10 06:09:01.338296 | controller | - Config: Waiting ... 2026-06-10 06:09:01.338303 | controller | - Gateway: OK 2026-06-10 06:09:01.338309 | controller | - GitServer: OK 2026-06-10 06:09:01.338313 | controller | - HoundSearch: OK 2026-06-10 06:09:01.338317 | controller | - LogJuicer: OK 2026-06-10 06:09:01.338320 | controller | - Logserver: OK 2026-06-10 06:09:01.338324 | controller | - MariaDB: OK 2026-06-10 06:09:01.338327 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:09:01.338332 | controller | - NodePoolLauncher: OK 2026-06-10 06:09:01.338335 | controller | - Zookeeper: OK 2026-06-10 06:09:01.338339 | controller | - Zuul: Waiting ... 2026-06-10 06:09:01.338342 | controller | - |_ Config Secrets: OK 2026-06-10 06:09:01.338346 | controller | - |_ Executor: OK 2026-06-10 06:09:01.338349 | controller | - |_ Merger: OK 2026-06-10 06:09:01.338353 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:09:01.338357 | controller | - |_ Web: OK 2026-06-10 06:09:01.338361 | controller | 2026-06-10 06:09:02.341787 | controller | 2026-06-10T06:09:02-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:09:02.950550 | controller | 2026-06-10T06:09:02-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:09:02.950626 | controller | 2026-06-10T06:09:02-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:09:02.963126 | controller | 2026-06-10T06:09:02-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:09:02.963189 | controller | 2026-06-10T06:09:02-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:09:02.975896 | controller | 2026-06-10T06:09:02-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:09:02.975940 | controller | 2026-06-10T06:09:02-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:09:03.352464 | controller | 2026-06-10T06:09:03-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:09:03.759287 | controller | 2026-06-10T06:09:03-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:09:03.759382 | controller | 2026-06-10T06:09:03-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:09:05.144618 | controller | 2026-06-10T06:09:05-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:09:05.144683 | controller | 2026-06-10T06:09:05-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:09:06.735441 | controller | 2026-06-10T06:09:06-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:09:06.735535 | controller | 2026-06-10T06:09:06-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:09:07.941717 | controller | 2026-06-10T06:09:07-04:00 INFO Creating base secret job 2026-06-10 06:09:07.942181 | controller | 2026-06-10T06:09:07-04:00 INFO Creating object, name: config-base-secret-1-2757-ns 2026-06-10 06:09:07.946477 | controller | 2026-06-10T06:09:07-04:00 INFO 2026-06-10 06:09:07.946525 | controller | - Config: Waiting ... 2026-06-10 06:09:07.946539 | controller | - Gateway: OK 2026-06-10 06:09:07.946550 | controller | - GitServer: OK 2026-06-10 06:09:07.946558 | controller | - HoundSearch: OK 2026-06-10 06:09:07.946566 | controller | - LogJuicer: OK 2026-06-10 06:09:07.946573 | controller | - Logserver: OK 2026-06-10 06:09:07.946581 | controller | - MariaDB: OK 2026-06-10 06:09:07.946589 | controller | - NodePoolBuilder: OK 2026-06-10 06:09:07.946597 | controller | - NodePoolLauncher: OK 2026-06-10 06:09:07.946634 | controller | - Zookeeper: OK 2026-06-10 06:09:07.946647 | controller | - Zuul: OK 2026-06-10 06:09:07.946657 | controller | 2026-06-10 06:09:08.951589 | controller | 2026-06-10T06:09:08-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:09:09.554901 | controller | 2026-06-10T06:09:09-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:09:09.554970 | controller | 2026-06-10T06:09:09-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:09:09.563028 | controller | 2026-06-10T06:09:09-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:09:09.563085 | controller | 2026-06-10T06:09:09-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:09:09.580695 | controller | 2026-06-10T06:09:09-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:09:09.580762 | controller | 2026-06-10T06:09:09-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:09:09.964341 | controller | 2026-06-10T06:09:09-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:09:10.365792 | controller | 2026-06-10T06:09:10-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:09:10.365914 | controller | 2026-06-10T06:09:10-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:09:11.741906 | controller | 2026-06-10T06:09:11-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:09:11.741983 | controller | 2026-06-10T06:09:11-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:09:13.334695 | controller | 2026-06-10T06:09:13-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:09:13.334779 | controller | 2026-06-10T06:09:13-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:09:14.539268 | controller | 2026-06-10T06:09:14-04:00 INFO Running tenant-reconfigure for the 'internal' tenant 2026-06-10 06:09:14.732205 | controller | 2026-06-10T06:09:14-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [zuul-scheduler tenant-reconfigure internal] 2026-06-10 06:09:21.489819 | controller | 2026-06-10T06:09:21-04:00 INFO Deleting object, name: zuul-web 2026-06-10 06:09:21.493878 | controller | 2026-06-10T06:09:21-04:00 INFO Updating object, name: zs-internal-tenant-reconfigure 2026-06-10 06:09:21.496146 | controller | 2026-06-10T06:09:21-04:00 INFO 2026-06-10 06:09:21.496175 | controller | - Config: Waiting ... 2026-06-10 06:09:21.496181 | controller | - Gateway: OK 2026-06-10 06:09:21.496186 | controller | - GitServer: OK 2026-06-10 06:09:21.496191 | controller | - HoundSearch: OK 2026-06-10 06:09:21.496196 | controller | - LogJuicer: OK 2026-06-10 06:09:21.496201 | controller | - Logserver: OK 2026-06-10 06:09:21.496205 | controller | - MariaDB: OK 2026-06-10 06:09:21.496210 | controller | - NodePoolBuilder: OK 2026-06-10 06:09:21.496215 | controller | - NodePoolLauncher: OK 2026-06-10 06:09:21.496220 | controller | - Zookeeper: OK 2026-06-10 06:09:21.496224 | controller | - Zuul: OK 2026-06-10 06:09:21.496230 | controller | 2026-06-10 06:09:22.502528 | controller | 2026-06-10T06:09:22-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:09:23.104287 | controller | 2026-06-10T06:09:23-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:09:23.104357 | controller | 2026-06-10T06:09:23-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:09:23.114182 | controller | 2026-06-10T06:09:23-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:09:23.114248 | controller | 2026-06-10T06:09:23-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:09:23.126458 | controller | 2026-06-10T06:09:23-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:09:23.126508 | controller | 2026-06-10T06:09:23-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:09:23.509811 | controller | 2026-06-10T06:09:23-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:09:23.912448 | controller | 2026-06-10T06:09:23-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:09:23.912507 | controller | 2026-06-10T06:09:23-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:09:24.928909 | controller | 2026-06-10T06:09:24-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:09:24.928995 | controller | 2026-06-10T06:09:24-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:09:25.906630 | controller | 2026-06-10T06:09:25-04:00 INFO Creating object, name: zuul-web 2026-06-10 06:09:26.713546 | controller | 2026-06-10T06:09:26-04:00 INFO 2026-06-10 06:09:26.713637 | controller | - Config: Waiting ... 2026-06-10 06:09:26.713648 | controller | - Gateway: OK 2026-06-10 06:09:26.713654 | controller | - GitServer: OK 2026-06-10 06:09:26.713660 | controller | - HoundSearch: OK 2026-06-10 06:09:26.713666 | controller | - LogJuicer: OK 2026-06-10 06:09:26.713672 | controller | - Logserver: OK 2026-06-10 06:09:26.713677 | controller | - MariaDB: OK 2026-06-10 06:09:26.713683 | controller | - NodePoolBuilder: OK 2026-06-10 06:09:26.713689 | controller | - NodePoolLauncher: OK 2026-06-10 06:09:26.713695 | controller | - Zookeeper: OK 2026-06-10 06:09:26.713701 | controller | - Zuul: Waiting ... 2026-06-10 06:09:26.713706 | controller | - |_ Config Secrets: OK 2026-06-10 06:09:26.713712 | controller | - |_ Executor: OK 2026-06-10 06:09:26.713718 | controller | - |_ Merger: OK 2026-06-10 06:09:26.713724 | controller | - |_ Scheduler: OK 2026-06-10 06:09:26.713729 | controller | - |_ Web: Waiting ... 2026-06-10 06:09:26.713735 | controller | 2026-06-10 06:09:27.718940 | controller | 2026-06-10T06:09:27-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:09:28.321253 | controller | 2026-06-10T06:09:28-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:09:28.321315 | controller | 2026-06-10T06:09:28-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:09:28.329621 | controller | 2026-06-10T06:09:28-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:09:28.329674 | controller | 2026-06-10T06:09:28-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:09:28.340416 | controller | 2026-06-10T06:09:28-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:09:28.340440 | controller | 2026-06-10T06:09:28-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:09:28.732034 | controller | 2026-06-10T06:09:28-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:09:29.135260 | controller | 2026-06-10T06:09:29-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:09:29.135325 | controller | 2026-06-10T06:09:29-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:09:30.514502 | controller | 2026-06-10T06:09:30-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:09:30.514565 | controller | 2026-06-10T06:09:30-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:09:32.107925 | controller | 2026-06-10T06:09:32-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:09:32.108038 | controller | 2026-06-10T06:09:32-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:09:32.912132 | controller | 2026-06-10T06:09:32-04:00 INFO 2026-06-10 06:09:32.912182 | controller | - Config: Waiting ... 2026-06-10 06:09:32.912188 | controller | - Gateway: OK 2026-06-10 06:09:32.912192 | controller | - GitServer: OK 2026-06-10 06:09:32.912195 | controller | - HoundSearch: OK 2026-06-10 06:09:32.912199 | controller | - LogJuicer: OK 2026-06-10 06:09:32.912203 | controller | - Logserver: OK 2026-06-10 06:09:32.912206 | controller | - MariaDB: OK 2026-06-10 06:09:32.912210 | controller | - NodePoolBuilder: OK 2026-06-10 06:09:32.912213 | controller | - NodePoolLauncher: OK 2026-06-10 06:09:32.912223 | controller | - Zookeeper: OK 2026-06-10 06:09:32.912226 | controller | - Zuul: Waiting ... 2026-06-10 06:09:32.912230 | controller | - |_ Config Secrets: OK 2026-06-10 06:09:32.912233 | controller | - |_ Executor: OK 2026-06-10 06:09:32.912237 | controller | - |_ Merger: OK 2026-06-10 06:09:32.912240 | controller | - |_ Scheduler: OK 2026-06-10 06:09:32.912244 | controller | - |_ Web: Waiting ... 2026-06-10 06:09:32.912248 | controller | 2026-06-10 06:09:33.915935 | controller | 2026-06-10T06:09:33-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:09:34.518532 | controller | 2026-06-10T06:09:34-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:09:34.518622 | controller | 2026-06-10T06:09:34-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:09:34.526358 | controller | 2026-06-10T06:09:34-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:09:34.526399 | controller | 2026-06-10T06:09:34-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:09:34.540151 | controller | 2026-06-10T06:09:34-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:09:34.540202 | controller | 2026-06-10T06:09:34-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:09:34.935886 | controller | 2026-06-10T06:09:34-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:09:35.328665 | controller | 2026-06-10T06:09:35-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:09:35.328737 | controller | 2026-06-10T06:09:35-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:09:36.715422 | controller | 2026-06-10T06:09:36-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:09:36.715496 | controller | 2026-06-10T06:09:36-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:09:38.305242 | controller | 2026-06-10T06:09:38-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:09:38.305313 | controller | 2026-06-10T06:09:38-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:09:39.112992 | controller | 2026-06-10T06:09:39-04:00 INFO 2026-06-10 06:09:39.113061 | controller | - Config: Waiting ... 2026-06-10 06:09:39.113069 | controller | - Gateway: OK 2026-06-10 06:09:39.113075 | controller | - GitServer: OK 2026-06-10 06:09:39.113081 | controller | - HoundSearch: OK 2026-06-10 06:09:39.113086 | controller | - LogJuicer: OK 2026-06-10 06:09:39.113092 | controller | - Logserver: OK 2026-06-10 06:09:39.113097 | controller | - MariaDB: OK 2026-06-10 06:09:39.113102 | controller | - NodePoolBuilder: OK 2026-06-10 06:09:39.113107 | controller | - NodePoolLauncher: OK 2026-06-10 06:09:39.113113 | controller | - Zookeeper: OK 2026-06-10 06:09:39.113118 | controller | - Zuul: Waiting ... 2026-06-10 06:09:39.113124 | controller | - |_ Config Secrets: OK 2026-06-10 06:09:39.113131 | controller | - |_ Executor: OK 2026-06-10 06:09:39.113139 | controller | - |_ Merger: OK 2026-06-10 06:09:39.113146 | controller | - |_ Scheduler: OK 2026-06-10 06:09:39.113153 | controller | - |_ Web: Waiting ... 2026-06-10 06:09:39.113161 | controller | 2026-06-10 06:09:40.118945 | controller | 2026-06-10T06:09:40-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:09:40.721938 | controller | 2026-06-10T06:09:40-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:09:40.722011 | controller | 2026-06-10T06:09:40-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:09:40.731454 | controller | 2026-06-10T06:09:40-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:09:40.731527 | controller | 2026-06-10T06:09:40-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:09:40.747310 | controller | 2026-06-10T06:09:40-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:09:40.747391 | controller | 2026-06-10T06:09:40-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:09:41.133912 | controller | 2026-06-10T06:09:41-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:09:41.533499 | controller | 2026-06-10T06:09:41-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:09:41.533562 | controller | 2026-06-10T06:09:41-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:09:42.917004 | controller | 2026-06-10T06:09:42-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:09:42.917082 | controller | 2026-06-10T06:09:42-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:09:44.506159 | controller | 2026-06-10T06:09:44-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:09:44.506503 | controller | 2026-06-10T06:09:44-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:09:45.312122 | controller | 2026-06-10T06:09:45-04:00 INFO 2026-06-10 06:09:45.312185 | controller | - Config: Waiting ... 2026-06-10 06:09:45.312190 | controller | - Gateway: OK 2026-06-10 06:09:45.312194 | controller | - GitServer: OK 2026-06-10 06:09:45.312198 | controller | - HoundSearch: OK 2026-06-10 06:09:45.312202 | controller | - LogJuicer: OK 2026-06-10 06:09:45.312205 | controller | - Logserver: OK 2026-06-10 06:09:45.312209 | controller | - MariaDB: OK 2026-06-10 06:09:45.312212 | controller | - NodePoolBuilder: OK 2026-06-10 06:09:45.312216 | controller | - NodePoolLauncher: OK 2026-06-10 06:09:45.312219 | controller | - Zookeeper: OK 2026-06-10 06:09:45.312223 | controller | - Zuul: Waiting ... 2026-06-10 06:09:45.312226 | controller | - |_ Config Secrets: OK 2026-06-10 06:09:45.312230 | controller | - |_ Executor: OK 2026-06-10 06:09:45.312233 | controller | - |_ Merger: OK 2026-06-10 06:09:45.312237 | controller | - |_ Scheduler: OK 2026-06-10 06:09:45.312240 | controller | - |_ Web: Waiting ... 2026-06-10 06:09:45.312244 | controller | 2026-06-10 06:09:46.316964 | controller | 2026-06-10T06:09:46-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:09:46.920997 | controller | 2026-06-10T06:09:46-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:09:46.921069 | controller | 2026-06-10T06:09:46-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:09:46.932540 | controller | 2026-06-10T06:09:46-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:09:46.932628 | controller | 2026-06-10T06:09:46-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:09:46.946557 | controller | 2026-06-10T06:09:46-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:09:46.946652 | controller | 2026-06-10T06:09:46-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:09:47.327977 | controller | 2026-06-10T06:09:47-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:09:47.741204 | controller | 2026-06-10T06:09:47-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:09:47.741271 | controller | 2026-06-10T06:09:47-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:09:49.126536 | controller | 2026-06-10T06:09:49-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:09:49.126672 | controller | 2026-06-10T06:09:49-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:09:50.705577 | controller | 2026-06-10T06:09:50-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:09:50.705712 | controller | 2026-06-10T06:09:50-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:09:51.512578 | controller | 2026-06-10T06:09:51-04:00 INFO 2026-06-10 06:09:51.512676 | controller | - Config: Waiting ... 2026-06-10 06:09:51.512685 | controller | - Gateway: OK 2026-06-10 06:09:51.512691 | controller | - GitServer: OK 2026-06-10 06:09:51.512697 | controller | - HoundSearch: OK 2026-06-10 06:09:51.512702 | controller | - LogJuicer: OK 2026-06-10 06:09:51.512707 | controller | - Logserver: OK 2026-06-10 06:09:51.512713 | controller | - MariaDB: OK 2026-06-10 06:09:51.512718 | controller | - NodePoolBuilder: OK 2026-06-10 06:09:51.512723 | controller | - NodePoolLauncher: OK 2026-06-10 06:09:51.512729 | controller | - Zookeeper: OK 2026-06-10 06:09:51.512734 | controller | - Zuul: Waiting ... 2026-06-10 06:09:51.512739 | controller | - |_ Config Secrets: OK 2026-06-10 06:09:51.512745 | controller | - |_ Executor: OK 2026-06-10 06:09:51.512750 | controller | - |_ Merger: OK 2026-06-10 06:09:51.512755 | controller | - |_ Scheduler: OK 2026-06-10 06:09:51.512761 | controller | - |_ Web: Waiting ... 2026-06-10 06:09:51.512766 | controller | 2026-06-10 06:09:52.516968 | controller | 2026-06-10T06:09:52-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:09:53.121296 | controller | 2026-06-10T06:09:53-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:09:53.121367 | controller | 2026-06-10T06:09:53-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:09:53.133590 | controller | 2026-06-10T06:09:53-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:09:53.133685 | controller | 2026-06-10T06:09:53-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:09:53.149791 | controller | 2026-06-10T06:09:53-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:09:53.149855 | controller | 2026-06-10T06:09:53-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:09:53.526763 | controller | 2026-06-10T06:09:53-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:09:53.929877 | controller | 2026-06-10T06:09:53-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:09:53.929955 | controller | 2026-06-10T06:09:53-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:09:55.314599 | controller | 2026-06-10T06:09:55-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:09:55.314740 | controller | 2026-06-10T06:09:55-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:09:56.905858 | controller | 2026-06-10T06:09:56-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:09:56.905942 | controller | 2026-06-10T06:09:56-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:09:57.712733 | controller | 2026-06-10T06:09:57-04:00 INFO 2026-06-10 06:09:57.712816 | controller | - Config: Waiting ... 2026-06-10 06:09:57.712823 | controller | - Gateway: OK 2026-06-10 06:09:57.712828 | controller | - GitServer: OK 2026-06-10 06:09:57.712833 | controller | - HoundSearch: OK 2026-06-10 06:09:57.712838 | controller | - LogJuicer: OK 2026-06-10 06:09:57.712851 | controller | - Logserver: OK 2026-06-10 06:09:57.712857 | controller | - MariaDB: OK 2026-06-10 06:09:57.712861 | controller | - NodePoolBuilder: OK 2026-06-10 06:09:57.712866 | controller | - NodePoolLauncher: OK 2026-06-10 06:09:57.712871 | controller | - Zookeeper: OK 2026-06-10 06:09:57.712876 | controller | - Zuul: Waiting ... 2026-06-10 06:09:57.712881 | controller | - |_ Config Secrets: OK 2026-06-10 06:09:57.712885 | controller | - |_ Executor: OK 2026-06-10 06:09:57.712890 | controller | - |_ Merger: OK 2026-06-10 06:09:57.712895 | controller | - |_ Scheduler: OK 2026-06-10 06:09:57.712900 | controller | - |_ Web: Waiting ... 2026-06-10 06:09:57.712905 | controller | 2026-06-10 06:09:58.716336 | controller | 2026-06-10T06:09:58-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:09:59.318829 | controller | 2026-06-10T06:09:59-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:09:59.318885 | controller | 2026-06-10T06:09:59-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:09:59.325959 | controller | 2026-06-10T06:09:59-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:09:59.326001 | controller | 2026-06-10T06:09:59-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:09:59.339312 | controller | 2026-06-10T06:09:59-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:09:59.339379 | controller | 2026-06-10T06:09:59-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:09:59.728587 | controller | 2026-06-10T06:09:59-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:10:00.125915 | controller | 2026-06-10T06:10:00-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:10:00.125979 | controller | 2026-06-10T06:10:00-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:10:01.514376 | controller | 2026-06-10T06:10:01-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:10:01.514459 | controller | 2026-06-10T06:10:01-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:10:03.106867 | controller | 2026-06-10T06:10:03-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:10:03.106948 | controller | 2026-06-10T06:10:03-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:10:03.914865 | controller | 2026-06-10T06:10:03-04:00 INFO 2026-06-10 06:10:03.914928 | controller | - Config: Waiting ... 2026-06-10 06:10:03.914933 | controller | - Gateway: OK 2026-06-10 06:10:03.914937 | controller | - GitServer: OK 2026-06-10 06:10:03.914941 | controller | - HoundSearch: OK 2026-06-10 06:10:03.914945 | controller | - LogJuicer: OK 2026-06-10 06:10:03.914948 | controller | - Logserver: OK 2026-06-10 06:10:03.914951 | controller | - MariaDB: OK 2026-06-10 06:10:03.914955 | controller | - NodePoolBuilder: OK 2026-06-10 06:10:03.914958 | controller | - NodePoolLauncher: OK 2026-06-10 06:10:03.914962 | controller | - Zookeeper: OK 2026-06-10 06:10:03.914966 | controller | - Zuul: Waiting ... 2026-06-10 06:10:03.914969 | controller | - |_ Config Secrets: OK 2026-06-10 06:10:03.914985 | controller | - |_ Executor: OK 2026-06-10 06:10:03.914989 | controller | - |_ Merger: OK 2026-06-10 06:10:03.914994 | controller | - |_ Scheduler: OK 2026-06-10 06:10:03.914998 | controller | - |_ Web: Waiting ... 2026-06-10 06:10:03.915001 | controller | 2026-06-10 06:10:04.921444 | controller | 2026-06-10T06:10:04-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:10:05.521936 | controller | 2026-06-10T06:10:05-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:10:05.521996 | controller | 2026-06-10T06:10:05-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:10:05.528111 | controller | 2026-06-10T06:10:05-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:10:05.528143 | controller | 2026-06-10T06:10:05-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:10:05.538951 | controller | 2026-06-10T06:10:05-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:10:05.538998 | controller | 2026-06-10T06:10:05-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:10:05.926653 | controller | 2026-06-10T06:10:05-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:10:06.334356 | controller | 2026-06-10T06:10:06-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:10:06.334432 | controller | 2026-06-10T06:10:06-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:10:07.717558 | controller | 2026-06-10T06:10:07-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:10:07.717634 | controller | 2026-06-10T06:10:07-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:10:09.307267 | controller | 2026-06-10T06:10:09-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:10:09.307343 | controller | 2026-06-10T06:10:09-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:10:10.110947 | controller | 2026-06-10T06:10:10-04:00 INFO 2026-06-10 06:10:10.111009 | controller | - Config: Waiting ... 2026-06-10 06:10:10.111015 | controller | - Gateway: OK 2026-06-10 06:10:10.111019 | controller | - GitServer: OK 2026-06-10 06:10:10.111023 | controller | - HoundSearch: OK 2026-06-10 06:10:10.111028 | controller | - LogJuicer: OK 2026-06-10 06:10:10.111065 | controller | - Logserver: OK 2026-06-10 06:10:10.111109 | controller | - MariaDB: OK 2026-06-10 06:10:10.111136 | controller | - NodePoolBuilder: OK 2026-06-10 06:10:10.111159 | controller | - NodePoolLauncher: OK 2026-06-10 06:10:10.111192 | controller | - Zookeeper: OK 2026-06-10 06:10:10.111221 | controller | - Zuul: Waiting ... 2026-06-10 06:10:10.111261 | controller | - |_ Config Secrets: OK 2026-06-10 06:10:10.111286 | controller | - |_ Executor: OK 2026-06-10 06:10:10.111317 | controller | - |_ Merger: OK 2026-06-10 06:10:10.111339 | controller | - |_ Scheduler: OK 2026-06-10 06:10:10.111371 | controller | - |_ Web: Waiting ... 2026-06-10 06:10:10.111377 | controller | 2026-06-10 06:10:11.117593 | controller | 2026-06-10T06:10:11-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:10:11.727564 | controller | 2026-06-10T06:10:11-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:10:11.727664 | controller | 2026-06-10T06:10:11-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:10:11.740156 | controller | 2026-06-10T06:10:11-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:10:11.740226 | controller | 2026-06-10T06:10:11-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:10:11.756138 | controller | 2026-06-10T06:10:11-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:10:11.756213 | controller | 2026-06-10T06:10:11-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:10:12.134866 | controller | 2026-06-10T06:10:12-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:10:12.526513 | controller | 2026-06-10T06:10:12-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:10:12.526578 | controller | 2026-06-10T06:10:12-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:10:13.923905 | controller | 2026-06-10T06:10:13-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:10:13.923976 | controller | 2026-06-10T06:10:13-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:10:15.507155 | controller | 2026-06-10T06:10:15-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:10:15.507220 | controller | 2026-06-10T06:10:15-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:10:16.316122 | controller | 2026-06-10T06:10:16-04:00 INFO 2026-06-10 06:10:16.316198 | controller | - Config: Waiting ... 2026-06-10 06:10:16.316205 | controller | - Gateway: OK 2026-06-10 06:10:16.316209 | controller | - GitServer: OK 2026-06-10 06:10:16.316213 | controller | - HoundSearch: OK 2026-06-10 06:10:16.316216 | controller | - LogJuicer: OK 2026-06-10 06:10:16.316220 | controller | - Logserver: OK 2026-06-10 06:10:16.316223 | controller | - MariaDB: OK 2026-06-10 06:10:16.316227 | controller | - NodePoolBuilder: OK 2026-06-10 06:10:16.316230 | controller | - NodePoolLauncher: OK 2026-06-10 06:10:16.316234 | controller | - Zookeeper: OK 2026-06-10 06:10:16.316237 | controller | - Zuul: Waiting ... 2026-06-10 06:10:16.316241 | controller | - |_ Config Secrets: OK 2026-06-10 06:10:16.316244 | controller | - |_ Executor: OK 2026-06-10 06:10:16.316248 | controller | - |_ Merger: OK 2026-06-10 06:10:16.316251 | controller | - |_ Scheduler: OK 2026-06-10 06:10:16.316254 | controller | - |_ Web: Waiting ... 2026-06-10 06:10:16.316258 | controller | 2026-06-10 06:10:17.320356 | controller | 2026-06-10T06:10:17-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:10:18.110098 | controller | 2026-06-10T06:10:18-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:10:18.110168 | controller | 2026-06-10T06:10:18-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:10:18.116494 | controller | 2026-06-10T06:10:18-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:10:18.116515 | controller | 2026-06-10T06:10:18-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:10:18.126298 | controller | 2026-06-10T06:10:18-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:10:18.126316 | controller | 2026-06-10T06:10:18-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:10:18.334270 | controller | 2026-06-10T06:10:18-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:10:18.775747 | controller | 2026-06-10T06:10:18-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:10:18.775866 | controller | 2026-06-10T06:10:18-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:10:20.121000 | controller | 2026-06-10T06:10:20-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:10:20.121061 | controller | 2026-06-10T06:10:20-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:10:21.707235 | controller | 2026-06-10T06:10:21-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:10:21.707307 | controller | 2026-06-10T06:10:21-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:10:23.174259 | controller | 2026-06-10T06:10:23-04:00 INFO 2026-06-10 06:10:23.174325 | controller | - Config: Waiting ... 2026-06-10 06:10:23.174331 | controller | - Gateway: OK 2026-06-10 06:10:23.174335 | controller | - GitServer: OK 2026-06-10 06:10:23.174339 | controller | - HoundSearch: OK 2026-06-10 06:10:23.174342 | controller | - LogJuicer: OK 2026-06-10 06:10:23.174346 | controller | - Logserver: OK 2026-06-10 06:10:23.174349 | controller | - MariaDB: OK 2026-06-10 06:10:23.174353 | controller | - NodePoolBuilder: OK 2026-06-10 06:10:23.174356 | controller | - NodePoolLauncher: OK 2026-06-10 06:10:23.174360 | controller | - Zookeeper: OK 2026-06-10 06:10:23.174363 | controller | - Zuul: Waiting ... 2026-06-10 06:10:23.174367 | controller | - |_ Config Secrets: OK 2026-06-10 06:10:23.174370 | controller | - |_ Executor: OK 2026-06-10 06:10:23.174374 | controller | - |_ Merger: OK 2026-06-10 06:10:23.174377 | controller | - |_ Scheduler: OK 2026-06-10 06:10:23.174381 | controller | - |_ Web: Waiting ... 2026-06-10 06:10:23.174385 | controller | 2026-06-10 06:10:24.179423 | controller | 2026-06-10T06:10:24-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:10:24.788705 | controller | 2026-06-10T06:10:24-04:00 DEBUG Inspecting volume logserver-logserver-0 2026-06-10 06:10:24.788799 | controller | 2026-06-10T06:10:24-04:00 DEBUG Volume logserver-logserver-0 at expected size, nothing to do 2026-06-10 06:10:24.803861 | controller | 2026-06-10T06:10:24-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:10:24.803983 | controller | 2026-06-10T06:10:24-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:10:24.820738 | controller | 2026-06-10T06:10:24-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:10:24.820837 | controller | 2026-06-10T06:10:24-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:10:25.201783 | controller | 2026-06-10T06:10:25-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:10:25.617712 | controller | 2026-06-10T06:10:25-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:10:25.617792 | controller | 2026-06-10T06:10:25-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:10:26.614532 | controller | 2026-06-10T06:10:26-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:10:26.614653 | controller | 2026-06-10T06:10:26-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:10:27.906485 | controller | 2026-06-10T06:10:27-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:10:27.906664 | controller | 2026-06-10T06:10:27-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:10:29.113628 | controller | 2026-06-10T06:10:29-04:00 INFO 2026-06-10 06:10:29.113690 | controller | - Config: OK 2026-06-10 06:10:29.113696 | controller | - Gateway: OK 2026-06-10 06:10:29.113700 | controller | - GitServer: OK 2026-06-10 06:10:29.113703 | controller | - HoundSearch: OK 2026-06-10 06:10:29.113707 | controller | - LogJuicer: OK 2026-06-10 06:10:29.113710 | controller | - Logserver: OK 2026-06-10 06:10:29.113714 | controller | - MariaDB: OK 2026-06-10 06:10:29.113717 | controller | - NodePoolBuilder: OK 2026-06-10 06:10:29.113721 | controller | - NodePoolLauncher: OK 2026-06-10 06:10:29.113724 | controller | - Zookeeper: OK 2026-06-10 06:10:29.113728 | controller | - Zuul: OK 2026-06-10 06:10:29.113732 | controller | 2026-06-10 06:10:29.113741 | controller | STEP: Validating secret changed @ 06/10/26 06:10:29.113 2026-06-10 06:10:29.121345 | controller | 2026-06-10T06:10:29-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [cat /etc/zuul/zuul.conf] 2026-06-10 06:10:29.201543 | controller | STEP: Checking the secret is in the zuul.conf @ 06/10/26 06:10:29.201 2026-06-10 06:10:29.203553 | controller | STEP: Checking zuul-client works @ 06/10/26 06:10:29.203 2026-06-10 06:10:29.203575 | controller | 2026-06-10T06:10:29-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [bash -c zuul-client -v autohold --tenant internal --project system-config --job testytest --reason testy 2>&1] 2026-06-10 06:10:29.678045 | controller | • [270.358 seconds] 2026-06-10 06:10:29.678123 | controller | ------------------------------ 2026-06-10 06:10:29.678139 | controller | Secret Rotations zuul-db works 2026-06-10 06:10:29.678145 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/secrets_rotation_test.go:71 2026-06-10 06:10:29.678152 | controller | STEP: Checking the secret is in the zuul.conf @ 06/10/26 06:10:29.678 2026-06-10 06:10:29.680974 | controller | STEP: Checking zuul-web works @ 06/10/26 06:10:29.68 2026-06-10 06:10:29.680998 | controller | 2026-06-10T06:10:29-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [curl zuul-web:9000/api/tenant/demo-tenant/builds] 2026-06-10 06:10:29.766400 | controller | % Total % Received % Xferd Average Speed Time Time Time Current 2026-06-10 06:10:29.766490 | controller | Dload Upload Total Spent Left Speed 2026-06-10 06:10:29.812714 | controller | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 1854 100 1854 0 0 41200 0 --:--:-- --:--:-- --:--:-- 41200 2026-06-10 06:10:29.815291 | controller | • [0.137 seconds] 2026-06-10 06:10:29.815313 | controller | ------------------------------ 2026-06-10 06:10:29.815323 | controller | Secret Rotations zuul-zk works 2026-06-10 06:10:29.815329 | controller | /home/zuul-worker/src/softwarefactory-project.io/software-factory/sf-operator/tests/secrets_rotation_test.go:81 2026-06-10 06:10:29.815382 | controller | STEP: Checking the key in zuul pod @ 06/10/26 06:10:29.815 2026-06-10 06:10:29.815416 | controller | 2026-06-10T06:10:29-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [cat /tls/client/tls.key] 2026-06-10 06:10:29.901262 | controller | STEP: Connecting with kazoo @ 06/10/26 06:10:29.901 2026-06-10 06:10:29.901325 | controller | 2026-06-10T06:10:29-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [python3 -c import os, kazoo.client; os.chdir('/tls/client'); hosts=list(filter(lambda l: l.startswith('hosts'), open('/etc/zuul/zuul.conf').readlines()))[0].split('=')[1].strip(); client = kazoo.client.KazooClient(hosts, use_ssl=True, keyfile='tls.key', certfile='tls.crt', ca='ca.crt'); client.start(); print(client.get_children('/keystorage2'))] 2026-06-10 06:10:30.181481 | controller | • [0.366 seconds] 2026-06-10 06:10:30.181538 | controller | ------------------------------ 2026-06-10 06:10:30.181549 | controller | 2026-06-10 06:10:30.181553 | controller | Ran 16 of 16 Specs in 465.223 seconds 2026-06-10 06:10:30.181558 | controller | SUCCESS! -- 16 Passed | 0 Failed | 0 Pending | 0 Skipped 2026-06-10 06:10:30.181562 | controller | --- PASS: TestSFOP (465.22s) 2026-06-10 06:10:30.181565 | controller | PASS 2026-06-10 06:10:30.185648 | controller | ok github.com/softwarefactory-project/sf-operator/tests 465.248s 2026-06-10 10:10:30.686409 | controller | ok: Runtime: 0:07:54.818336 2026-06-10 10:10:30.724220 | 2026-06-10 10:10:30.724544 | TASK [health-check/test-standalone-update : Create a temporary directory for generated CRs] 2026-06-10 10:10:31.165078 | controller | changed 2026-06-10 10:10:31.174373 | 2026-06-10 10:10:31.174540 | TASK [health-check/test-standalone-update : Generate initial CR] 2026-06-10 10:10:32.073274 | controller | changed 2026-06-10 10:10:32.082613 | 2026-06-10 10:10:32.082750 | TASK [Run standalone operator with initial config] 2026-06-10 10:10:32.116099 | controller | ok 2026-06-10 10:10:32.150551 | 2026-06-10 10:10:32.150661 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 10:10:32.418614 | controller | ok 2026-06-10 10:10:32.429029 | 2026-06-10 10:10:32.429181 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 10:10:32.686744 | controller | changed 2026-06-10 10:10:32.692986 | 2026-06-10 10:10:32.693088 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 06:10:33.284283 | controller | 2026-06-10T06:10:33-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:10:33.284545 | controller | 2026-06-10 06:10:33.293908 | controller | 2026-06-10T06:10:33-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:10:33.293965 | controller | 2026-06-10 06:10:33.309733 | controller | 2026-06-10T06:10:33-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:10:33.318768 | controller | 2026-06-10T06:10:33-04:00 INFO Updating configmap, name: zuul-scheduler-tooling-config-map 2026-06-10 06:10:33.318827 | controller | 2026-06-10T06:10:33-04:00 INFO Updating object, name: zuul-scheduler-tooling-config-map 2026-06-10 06:10:33.328654 | controller | 2026-06-10T06:10:33-04:00 INFO Statefulset "git-server" configuration changed, applying... 2026-06-10 06:10:33.328709 | controller | 2026-06-10T06:10:33-04:00 INFO Updating object, name: git-server 2026-06-10 06:10:33.911672 | controller | 2026-06-10T06:10:33-04:00 INFO Statefulset "logserver" configuration changed, applying... 2026-06-10 06:10:33.911739 | controller | 2026-06-10T06:10:33-04:00 INFO Updating object, name: logserver 2026-06-10 06:10:33.930144 | controller | 2026-06-10T06:10:33-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:10:33.930221 | controller | 2026-06-10T06:10:33-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:10:33.967347 | controller | 2026-06-10T06:10:33-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:10:33.967407 | controller | 2026-06-10T06:10:33-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:10:34.317642 | controller | 2026-06-10T06:10:34-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-10 06:10:34.317694 | controller | 2026-06-10T06:10:34-04:00 INFO 2026-06-10 06:10:34.317700 | controller | - Config: Waiting ... 2026-06-10 06:10:34.317704 | controller | - Gateway: OK 2026-06-10 06:10:34.317708 | controller | - GitServer: Waiting ... 2026-06-10 06:10:34.317712 | controller | - HoundSearch: OK 2026-06-10 06:10:34.317715 | controller | - LogJuicer: OK 2026-06-10 06:10:34.317719 | controller | - Logserver: Waiting ... 2026-06-10 06:10:34.317722 | controller | - MariaDB: OK 2026-06-10 06:10:34.317726 | controller | - Zookeeper: OK 2026-06-10 06:10:34.317730 | controller | 2026-06-10 06:10:34.317734 | controller | 2026-06-10T06:10:34-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 06:10:39.327124 | controller | 2026-06-10T06:10:39-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:10:39.360171 | controller | 2026-06-10T06:10:39-04:00 INFO Waiting for statefulset, name: git-server 2026-06-10 06:10:57.455780 | controller | 2026-06-10T06:10:57-04:00 INFO Waiting for statefulset, name: logserver 2026-06-10 06:11:25.556243 | controller | 2026-06-10T06:11:25-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:11:25.674018 | controller | 2026-06-10T06:11:25-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:11:25.688786 | controller | 2026-06-10T06:11:25-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:11:25.688871 | controller | 2026-06-10T06:11:25-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:11:25.706477 | controller | 2026-06-10T06:11:25-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:11:25.706528 | controller | 2026-06-10T06:11:25-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:11:25.720325 | controller | 2026-06-10T06:11:25-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:11:25.736434 | controller | 2026-06-10T06:11:25-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:11:25.736485 | controller | 2026-06-10T06:11:25-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:11:25.904813 | controller | 2026-06-10T06:11:25-04:00 INFO Updating secret, name: zuul-config 2026-06-10 06:11:25.904879 | controller | 2026-06-10T06:11:25-04:00 INFO Updating object, name: zuul-config 2026-06-10 06:11:26.683734 | controller | 2026-06-10T06:11:26-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-10 06:11:26.683788 | controller | 2026-06-10T06:11:26-04:00 INFO Updating object, name: zuul-scheduler 2026-06-10 06:11:28.282663 | controller | 2026-06-10T06:11:28-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-10 06:11:28.282728 | controller | 2026-06-10T06:11:28-04:00 INFO Updating object, name: zuul-web 2026-06-10 06:11:29.088878 | controller | 2026-06-10T06:11:29-04:00 INFO 2026-06-10 06:11:29.088951 | controller | - Config: Waiting ... 2026-06-10 06:11:29.088961 | controller | - Gateway: OK 2026-06-10 06:11:29.088969 | controller | - GitServer: OK 2026-06-10 06:11:29.088977 | controller | - HoundSearch: OK 2026-06-10 06:11:29.088985 | controller | - LogJuicer: OK 2026-06-10 06:11:29.088992 | controller | - Logserver: OK 2026-06-10 06:11:29.088999 | controller | - MariaDB: OK 2026-06-10 06:11:29.089007 | controller | - NodePoolBuilder: OK 2026-06-10 06:11:29.089014 | controller | - NodePoolLauncher: OK 2026-06-10 06:11:29.089021 | controller | - Zookeeper: OK 2026-06-10 06:11:29.089028 | controller | - Zuul: Waiting ... 2026-06-10 06:11:29.089036 | controller | - |_ Config Secrets: OK 2026-06-10 06:11:29.089045 | controller | - |_ Executor: OK 2026-06-10 06:11:29.089053 | controller | - |_ Merger: OK 2026-06-10 06:11:29.089060 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:11:29.089068 | controller | - |_ Web: Waiting ... 2026-06-10 06:11:29.089075 | controller | 2026-06-10 06:11:29.089083 | controller | 2026-06-10T06:11:29-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 06:11:34.096983 | controller | 2026-06-10T06:11:34-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:11:34.724552 | controller | 2026-06-10T06:11:34-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:11:34.724643 | controller | 2026-06-10T06:11:34-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:11:34.740121 | controller | 2026-06-10T06:11:34-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:11:34.740168 | controller | 2026-06-10T06:11:34-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:11:35.110846 | controller | 2026-06-10T06:11:35-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:11:35.509525 | controller | 2026-06-10T06:11:35-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:11:35.509635 | controller | 2026-06-10T06:11:35-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:11:36.526555 | controller | 2026-06-10T06:11:36-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:12:08.039966 | controller | 2026-06-10T06:12:08-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 06:12:08.073433 | controller | 2026-06-10T06:12:08-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:12:08.073545 | controller | 2026-06-10T06:12:08-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:12:08.456065 | controller | 2026-06-10T06:12:08-04:00 INFO 2026-06-10 06:12:08.456133 | controller | - Config: Waiting ... 2026-06-10 06:12:08.456145 | controller | - Gateway: OK 2026-06-10 06:12:08.456155 | controller | - GitServer: OK 2026-06-10 06:12:08.456163 | controller | - HoundSearch: OK 2026-06-10 06:12:08.456172 | controller | - LogJuicer: OK 2026-06-10 06:12:08.456181 | controller | - Logserver: OK 2026-06-10 06:12:08.456189 | controller | - MariaDB: OK 2026-06-10 06:12:08.456198 | controller | - NodePoolBuilder: OK 2026-06-10 06:12:08.456207 | controller | - NodePoolLauncher: OK 2026-06-10 06:12:08.456215 | controller | - Zookeeper: OK 2026-06-10 06:12:08.456224 | controller | - Zuul: Waiting ... 2026-06-10 06:12:08.456232 | controller | - |_ Config Secrets: OK 2026-06-10 06:12:08.456241 | controller | - |_ Executor: OK 2026-06-10 06:12:08.456250 | controller | - |_ Merger: OK 2026-06-10 06:12:08.456258 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:12:08.456268 | controller | - |_ Web: Waiting ... 2026-06-10 06:12:08.456278 | controller | 2026-06-10 06:12:08.456294 | controller | 2026-06-10T06:12:08-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 06:12:13.461729 | controller | 2026-06-10T06:12:13-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:12:14.078468 | controller | 2026-06-10T06:12:14-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:12:14.078537 | controller | 2026-06-10T06:12:14-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:12:14.094018 | controller | 2026-06-10T06:12:14-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:12:14.094082 | controller | 2026-06-10T06:12:14-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:12:14.473293 | controller | 2026-06-10T06:12:14-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:12:14.882768 | controller | 2026-06-10T06:12:14-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:12:14.882849 | controller | 2026-06-10T06:12:14-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:12:15.880864 | controller | 2026-06-10T06:12:15-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:12:31.941505 | controller | 2026-06-10T06:12:31-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:12:31.941554 | controller | 2026-06-10T06:12:31-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:12:31.963208 | controller | 2026-06-10T06:12:31-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:12:31.963280 | controller | 2026-06-10T06:12:31-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:12:32.753415 | controller | 2026-06-10T06:12:32-04:00 INFO 2026-06-10 06:12:32.753457 | controller | - Config: OK 2026-06-10 06:12:32.753462 | controller | - Gateway: OK 2026-06-10 06:12:32.753466 | controller | - GitServer: OK 2026-06-10 06:12:32.753470 | controller | - HoundSearch: OK 2026-06-10 06:12:32.753473 | controller | - LogJuicer: OK 2026-06-10 06:12:32.753477 | controller | - Logserver: OK 2026-06-10 06:12:32.753481 | controller | - MariaDB: OK 2026-06-10 06:12:32.753486 | controller | - NodePoolBuilder: OK 2026-06-10 06:12:32.753490 | controller | - NodePoolLauncher: OK 2026-06-10 06:12:32.753494 | controller | - Zookeeper: OK 2026-06-10 06:12:32.753497 | controller | - Zuul: OK 2026-06-10 06:12:32.753501 | controller | 2026-06-10 06:12:32.753506 | controller | 2026-06-10T06:12:32-04:00 INFO Updating controller configmap ... 2026-06-10 06:12:33.146202 | controller | 2026-06-10T06:12:33-04:00 INFO Standalone reconcile done. 2026-06-10 10:12:33.559274 | controller | ok: Runtime: 0:02:00.163847 2026-06-10 10:12:33.566101 | 2026-06-10 10:12:33.566243 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 10:12:34.100400 | controller | ok: Runtime: 0:00:00.004513 2026-06-10 10:12:34.117151 | 2026-06-10 10:12:34.117292 | TASK [health-check/test-standalone-update : Get standalone owner configmap after initial creation] 2026-06-10 06:12:34.495176 | controller | fqdn: test-initial.sf.io 2026-06-10 06:12:34.495267 | controller | fluentbitlogforwarding: null 2026-06-10 06:12:34.495279 | controller | storagedefault: 2026-06-10 06:12:34.495284 | controller | classname: "" 2026-06-10 06:12:34.495290 | controller | extraannotations: {} 2026-06-10 06:12:34.495295 | controller | extralabels: {} 2026-06-10 06:12:34.495301 | controller | configrepositorylocation: 2026-06-10 06:12:34.495308 | controller | name: config 2026-06-10 06:12:34.495314 | controller | branch: "" 2026-06-10 06:12:34.495321 | controller | zuulconnectionname: gerrit 2026-06-10 06:12:34.495326 | controller | clusterapiurl: "" 2026-06-10 06:12:34.495332 | controller | logserverhost: "" 2026-06-10 06:12:34.495337 | controller | zuul: 2026-06-10 06:12:34.495342 | controller | oidcauthenticators: [] 2026-06-10 06:12:34.495347 | controller | defaultauthenticator: "" 2026-06-10 06:12:34.495352 | controller | gerritconns: 2026-06-10 06:12:34.495357 | controller | - name: gerrit 2026-06-10 06:12:34.495365 | controller | hostname: gerrit-sshd.sf 2026-06-10 06:12:34.495371 | controller | port: 0 2026-06-10 06:12:34.495379 | controller | puburl: https://gerrit.sfop.me 2026-06-10 06:12:34.495384 | controller | username: zuul 2026-06-10 06:12:34.495389 | controller | canonicalhostname: "" 2026-06-10 06:12:34.495395 | controller | password: "" 2026-06-10 06:12:34.495400 | controller | sshkey: "" 2026-06-10 06:12:34.495405 | controller | gitoverssh: false 2026-06-10 06:12:34.495410 | controller | verifyssl: false 2026-06-10 06:12:34.495414 | controller | streamevents: null 2026-06-10 06:12:34.495419 | controller | githubconns: [] 2026-06-10 06:12:34.495424 | controller | gitlabconns: [] 2026-06-10 06:12:34.495430 | controller | gitconns: [] 2026-06-10 06:12:34.495437 | controller | pagureconns: [] 2026-06-10 06:12:34.495444 | controller | elasticsearchconns: [] 2026-06-10 06:12:34.495451 | controller | smtpconns: [] 2026-06-10 06:12:34.495457 | controller | executor: 2026-06-10 06:12:34.495464 | controller | storage: 2026-06-10 06:12:34.495470 | controller | size: 2026-06-10 06:12:34.495476 | controller | format: "" 2026-06-10 06:12:34.495482 | controller | classname: "" 2026-06-10 06:12:34.495488 | controller | loglevel: DEBUG 2026-06-10 06:12:34.495495 | controller | enabled: true 2026-06-10 06:12:34.495501 | controller | standalone: null 2026-06-10 06:12:34.495507 | controller | limits: null 2026-06-10 06:12:34.495513 | controller | disklimitperjob: 0 2026-06-10 06:12:34.495519 | controller | ansiblesetuptimeout: 0 2026-06-10 06:12:34.495531 | controller | terminationgraceperiodseconds: 0 2026-06-10 06:12:34.495537 | controller | scheduler: 2026-06-10 06:12:34.495544 | controller | storage: 2026-06-10 06:12:34.495550 | controller | size: 2026-06-10 06:12:34.495556 | controller | format: "" 2026-06-10 06:12:34.495563 | controller | classname: "" 2026-06-10 06:12:34.495570 | controller | statsdtarget: "" 2026-06-10 06:12:34.495577 | controller | loglevel: DEBUG 2026-06-10 06:12:34.495583 | controller | limits: null 2026-06-10 06:12:34.495588 | controller | defaultholdexpiration: null 2026-06-10 06:12:34.495593 | controller | maxholdexpiration: null 2026-06-10 06:12:34.495598 | controller | web: 2026-06-10 06:12:34.495629 | controller | loglevel: DEBUG 2026-06-10 06:12:34.495642 | controller | limits: null 2026-06-10 06:12:34.495649 | controller | merger: 2026-06-10 06:12:34.495655 | controller | gitusername: "" 2026-06-10 06:12:34.495660 | controller | gituseremail: "" 2026-06-10 06:12:34.495665 | controller | githttplowspeedlimit: 0 2026-06-10 06:12:34.495670 | controller | githttplowspeedtime: 0 2026-06-10 06:12:34.495675 | controller | gittimeout: 0 2026-06-10 06:12:34.495679 | controller | storage: 2026-06-10 06:12:34.495684 | controller | size: 2026-06-10 06:12:34.495689 | controller | format: "" 2026-06-10 06:12:34.495694 | controller | classname: "" 2026-06-10 06:12:34.495698 | controller | loglevel: DEBUG 2026-06-10 06:12:34.495703 | controller | limits: null 2026-06-10 06:12:34.495708 | controller | nodepool: 2026-06-10 06:12:34.495713 | controller | launcher: 2026-06-10 06:12:34.495717 | controller | loglevel: "" 2026-06-10 06:12:34.495722 | controller | limits: null 2026-06-10 06:12:34.495727 | controller | builder: 2026-06-10 06:12:34.495732 | controller | storage: 2026-06-10 06:12:34.495736 | controller | size: 2026-06-10 06:12:34.495741 | controller | format: "" 2026-06-10 06:12:34.495746 | controller | classname: "" 2026-06-10 06:12:34.495751 | controller | loglevel: "" 2026-06-10 06:12:34.495763 | controller | limits: null 2026-06-10 06:12:34.495768 | controller | statsdtarget: "" 2026-06-10 06:12:34.495773 | controller | zookeeper: 2026-06-10 06:12:34.495777 | controller | storage: 2026-06-10 06:12:34.495782 | controller | size: 2026-06-10 06:12:34.495787 | controller | format: "" 2026-06-10 06:12:34.495792 | controller | classname: "" 2026-06-10 06:12:34.495796 | controller | limits: null 2026-06-10 06:12:34.495801 | controller | logserver: 2026-06-10 06:12:34.495806 | controller | retentiondays: 30 2026-06-10 06:12:34.495811 | controller | loopdelay: 3600 2026-06-10 06:12:34.495816 | controller | storage: 2026-06-10 06:12:34.495821 | controller | size: 2026-06-10 06:12:34.495825 | controller | format: BinarySI 2026-06-10 06:12:34.495830 | controller | classname: "" 2026-06-10 06:12:34.495835 | controller | podannotations: {} 2026-06-10 06:12:34.495839 | controller | logjuicer: 2026-06-10 06:12:34.495844 | controller | size: 2026-06-10 06:12:34.495849 | controller | format: "" 2026-06-10 06:12:34.495854 | controller | classname: "" 2026-06-10 06:12:34.495859 | controller | mariadb: 2026-06-10 06:12:34.495863 | controller | dbstorage: 2026-06-10 06:12:34.495868 | controller | size: 2026-06-10 06:12:34.495873 | controller | format: "" 2026-06-10 06:12:34.495878 | controller | classname: "" 2026-06-10 06:12:34.495882 | controller | logstorage: 2026-06-10 06:12:34.495887 | controller | size: 2026-06-10 06:12:34.495892 | controller | format: "" 2026-06-10 06:12:34.495897 | controller | classname: "" 2026-06-10 06:12:34.495902 | controller | limits: null 2026-06-10 06:12:34.495906 | controller | gitserver: 2026-06-10 06:12:34.495911 | controller | storage: 2026-06-10 06:12:34.495916 | controller | size: 2026-06-10 06:12:34.495921 | controller | format: "" 2026-06-10 06:12:34.495926 | controller | classname: "" 2026-06-10 06:12:34.495930 | controller | codesearch: 2026-06-10 06:12:34.495935 | controller | storage: 2026-06-10 06:12:34.495940 | controller | size: 2026-06-10 06:12:34.495945 | controller | format: "" 2026-06-10 06:12:34.495949 | controller | classname: "" 2026-06-10 06:12:34.495954 | controller | limits: null 2026-06-10 06:12:34.495959 | controller | enabled: null 2026-06-10 06:12:34.495964 | controller | hostaliases: [] 2026-06-10 06:12:34.495968 | controller | gateway: null 2026-06-10 10:12:34.665478 | controller | ok: Runtime: 0:00:00.099623 2026-06-10 10:12:34.676667 | 2026-06-10 10:12:34.676810 | TASK [health-check/test-standalone-update : Verify initial spec in configmap] 2026-06-10 10:12:34.721478 | controller | ok: All assertions passed 2026-06-10 10:12:34.730253 | 2026-06-10 10:12:34.730398 | TASK [health-check/test-standalone-update : Get standalone owner configmap annotation after initial creation] 2026-06-10 10:12:35.274017 | controller | ok: Runtime: 0:00:00.055388 2026-06-10 10:12:35.284105 | 2026-06-10 10:12:35.284245 | TASK [health-check/test-standalone-update : Pause for 2 seconds to ensure timestamp difference] 2026-06-10 10:12:37.326233 | controller | ok 2026-06-10 10:12:37.334417 | 2026-06-10 10:12:37.334583 | TASK [health-check/test-standalone-update : Generate updated CR] 2026-06-10 10:12:38.099870 | controller | changed 2026-06-10 10:12:38.111422 | 2026-06-10 10:12:38.111602 | TASK [Run standalone operator with updated config] 2026-06-10 10:12:38.136280 | controller | ok 2026-06-10 10:12:38.168680 | 2026-06-10 10:12:38.168776 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 10:12:38.415140 | controller | ok 2026-06-10 10:12:38.438908 | 2026-06-10 10:12:38.439099 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 10:12:38.669845 | controller | changed 2026-06-10 10:12:38.679340 | 2026-06-10 10:12:38.679505 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 06:12:39.169285 | controller | 2026-06-10T06:12:39-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:12:39.169366 | controller | 2026-06-10 06:12:39.176624 | controller | 2026-06-10T06:12:39-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:12:39.176659 | controller | 2026-06-10 06:12:39.188166 | controller | 2026-06-10T06:12:39-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:12:39.194478 | controller | 2026-06-10T06:12:39-04:00 INFO Updating configmap, name: zuul-scheduler-tooling-config-map 2026-06-10 06:12:39.194517 | controller | 2026-06-10T06:12:39-04:00 INFO Updating object, name: zuul-scheduler-tooling-config-map 2026-06-10 06:12:39.202717 | controller | 2026-06-10T06:12:39-04:00 INFO Statefulset "git-server" configuration changed, applying... 2026-06-10 06:12:39.202738 | controller | 2026-06-10T06:12:39-04:00 INFO Updating object, name: git-server 2026-06-10 06:12:39.790817 | controller | 2026-06-10T06:12:39-04:00 INFO Statefulset "logserver" configuration changed, applying... 2026-06-10 06:12:39.790876 | controller | 2026-06-10T06:12:39-04:00 INFO Updating object, name: logserver 2026-06-10 06:12:39.803738 | controller | 2026-06-10T06:12:39-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:12:39.803797 | controller | 2026-06-10T06:12:39-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:12:39.824091 | controller | 2026-06-10T06:12:39-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:12:39.824134 | controller | 2026-06-10T06:12:39-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:12:40.201930 | controller | 2026-06-10T06:12:40-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-10 06:12:40.201988 | controller | 2026-06-10T06:12:40-04:00 INFO 2026-06-10 06:12:40.201999 | controller | - Config: Waiting ... 2026-06-10 06:12:40.202006 | controller | - Gateway: OK 2026-06-10 06:12:40.202012 | controller | - GitServer: Waiting ... 2026-06-10 06:12:40.202017 | controller | - HoundSearch: OK 2026-06-10 06:12:40.202023 | controller | - LogJuicer: OK 2026-06-10 06:12:40.202028 | controller | - Logserver: Waiting ... 2026-06-10 06:12:40.202033 | controller | - MariaDB: OK 2026-06-10 06:12:40.202039 | controller | - Zookeeper: OK 2026-06-10 06:12:40.202044 | controller | 2026-06-10 06:12:40.202055 | controller | 2026-06-10T06:12:40-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 06:12:45.207837 | controller | 2026-06-10T06:12:45-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:12:45.230456 | controller | 2026-06-10T06:12:45-04:00 INFO Waiting for statefulset, name: git-server 2026-06-10 06:13:03.829218 | controller | 2026-06-10T06:13:03-04:00 INFO Waiting for statefulset, name: logserver 2026-06-10 06:13:32.052062 | controller | 2026-06-10T06:13:32-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:13:32.139021 | controller | 2026-06-10T06:13:32-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:13:32.144181 | controller | 2026-06-10T06:13:32-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:13:32.144233 | controller | 2026-06-10T06:13:32-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:13:32.154921 | controller | 2026-06-10T06:13:32-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:13:32.154977 | controller | 2026-06-10T06:13:32-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:13:32.165514 | controller | 2026-06-10T06:13:32-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:13:32.178082 | controller | 2026-06-10T06:13:32-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:13:32.178144 | controller | 2026-06-10T06:13:32-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:13:32.357503 | controller | 2026-06-10T06:13:32-04:00 INFO Updating secret, name: zuul-config 2026-06-10 06:13:32.357583 | controller | 2026-06-10T06:13:32-04:00 INFO Updating object, name: zuul-config 2026-06-10 06:13:33.146395 | controller | 2026-06-10T06:13:33-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-10 06:13:33.146468 | controller | 2026-06-10T06:13:33-04:00 INFO Updating object, name: zuul-scheduler 2026-06-10 06:13:34.749672 | controller | 2026-06-10T06:13:34-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-10 06:13:34.749781 | controller | 2026-06-10T06:13:34-04:00 INFO Updating object, name: zuul-web 2026-06-10 06:13:35.568011 | controller | 2026-06-10T06:13:35-04:00 INFO 2026-06-10 06:13:35.568085 | controller | - Config: Waiting ... 2026-06-10 06:13:35.568095 | controller | - Gateway: OK 2026-06-10 06:13:35.568104 | controller | - GitServer: OK 2026-06-10 06:13:35.568111 | controller | - HoundSearch: OK 2026-06-10 06:13:35.568118 | controller | - LogJuicer: OK 2026-06-10 06:13:35.568125 | controller | - Logserver: OK 2026-06-10 06:13:35.568132 | controller | - MariaDB: OK 2026-06-10 06:13:35.568139 | controller | - NodePoolBuilder: OK 2026-06-10 06:13:35.568146 | controller | - NodePoolLauncher: OK 2026-06-10 06:13:35.568153 | controller | - Zookeeper: OK 2026-06-10 06:13:35.568159 | controller | - Zuul: Waiting ... 2026-06-10 06:13:35.568165 | controller | - |_ Config Secrets: OK 2026-06-10 06:13:35.568172 | controller | - |_ Executor: OK 2026-06-10 06:13:35.568177 | controller | - |_ Merger: OK 2026-06-10 06:13:35.568183 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:13:35.568188 | controller | - |_ Web: Waiting ... 2026-06-10 06:13:35.568194 | controller | 2026-06-10 06:13:35.568205 | controller | 2026-06-10T06:13:35-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 06:13:40.578832 | controller | 2026-06-10T06:13:40-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:13:41.189907 | controller | 2026-06-10T06:13:41-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:13:41.189967 | controller | 2026-06-10T06:13:41-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:13:41.202470 | controller | 2026-06-10T06:13:41-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:13:41.202542 | controller | 2026-06-10T06:13:41-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:13:41.586085 | controller | 2026-06-10T06:13:41-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:13:42.001014 | controller | 2026-06-10T06:13:42-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:13:42.001119 | controller | 2026-06-10T06:13:42-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:13:43.001066 | controller | 2026-06-10T06:13:43-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:14:13.118993 | controller | 2026-06-10T06:14:13-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 06:14:13.142943 | controller | 2026-06-10T06:14:13-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:14:13.143010 | controller | 2026-06-10T06:14:13-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:14:13.584436 | controller | 2026-06-10T06:14:13-04:00 INFO 2026-06-10 06:14:13.584479 | controller | - Config: Waiting ... 2026-06-10 06:14:13.584485 | controller | - Gateway: OK 2026-06-10 06:14:13.584489 | controller | - GitServer: OK 2026-06-10 06:14:13.584493 | controller | - HoundSearch: OK 2026-06-10 06:14:13.584497 | controller | - LogJuicer: OK 2026-06-10 06:14:13.584501 | controller | - Logserver: OK 2026-06-10 06:14:13.584504 | controller | - MariaDB: OK 2026-06-10 06:14:13.584508 | controller | - NodePoolBuilder: OK 2026-06-10 06:14:13.584511 | controller | - NodePoolLauncher: OK 2026-06-10 06:14:13.584515 | controller | - Zookeeper: OK 2026-06-10 06:14:13.584519 | controller | - Zuul: Waiting ... 2026-06-10 06:14:13.584522 | controller | - |_ Config Secrets: OK 2026-06-10 06:14:13.584526 | controller | - |_ Executor: OK 2026-06-10 06:14:13.584530 | controller | - |_ Merger: OK 2026-06-10 06:14:13.584534 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:14:13.584562 | controller | - |_ Web: Waiting ... 2026-06-10 06:14:13.584567 | controller | 2026-06-10 06:14:13.584571 | controller | 2026-06-10T06:14:13-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 06:14:18.594349 | controller | 2026-06-10T06:14:18-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:14:19.207673 | controller | 2026-06-10T06:14:19-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:14:19.207752 | controller | 2026-06-10T06:14:19-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:14:19.222370 | controller | 2026-06-10T06:14:19-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:14:19.222450 | controller | 2026-06-10T06:14:19-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:14:19.607481 | controller | 2026-06-10T06:14:19-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:14:20.005958 | controller | 2026-06-10T06:14:20-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:14:20.006047 | controller | 2026-06-10T06:14:20-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:14:21.021911 | controller | 2026-06-10T06:14:21-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:14:39.120903 | controller | 2026-06-10T06:14:39-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:14:39.120951 | controller | 2026-06-10T06:14:39-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:14:39.228837 | controller | 2026-06-10T06:14:39-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:14:39.228900 | controller | 2026-06-10T06:14:39-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:14:39.933045 | controller | 2026-06-10T06:14:39-04:00 INFO 2026-06-10 06:14:39.933123 | controller | - Config: OK 2026-06-10 06:14:39.933130 | controller | - Gateway: OK 2026-06-10 06:14:39.933136 | controller | - GitServer: OK 2026-06-10 06:14:39.933141 | controller | - HoundSearch: OK 2026-06-10 06:14:39.933147 | controller | - LogJuicer: OK 2026-06-10 06:14:39.933152 | controller | - Logserver: OK 2026-06-10 06:14:39.933156 | controller | - MariaDB: OK 2026-06-10 06:14:39.933163 | controller | - NodePoolBuilder: OK 2026-06-10 06:14:39.933168 | controller | - NodePoolLauncher: OK 2026-06-10 06:14:39.933173 | controller | - Zookeeper: OK 2026-06-10 06:14:39.933178 | controller | - Zuul: OK 2026-06-10 06:14:39.933182 | controller | 2026-06-10 06:14:39.933187 | controller | 2026-06-10T06:14:39-04:00 INFO Updating controller configmap ... 2026-06-10 06:14:40.324202 | controller | 2026-06-10T06:14:40-04:00 INFO Standalone reconcile done. 2026-06-10 10:14:40.944721 | controller | ok: Runtime: 0:02:01.408960 2026-06-10 10:14:41.007421 | 2026-06-10 10:14:41.007592 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 10:14:41.538578 | controller | ok: Runtime: 0:00:00.005838 2026-06-10 10:14:41.554756 | 2026-06-10 10:14:41.554925 | TASK [health-check/test-standalone-update : Get standalone owner configmap after update] 2026-06-10 06:14:41.840659 | controller | fqdn: test-updated.sf.io 2026-06-10 06:14:41.840733 | controller | fluentbitlogforwarding: null 2026-06-10 06:14:41.840740 | controller | storagedefault: 2026-06-10 06:14:41.840745 | controller | classname: "" 2026-06-10 06:14:41.840749 | controller | extraannotations: {} 2026-06-10 06:14:41.840753 | controller | extralabels: {} 2026-06-10 06:14:41.840757 | controller | configrepositorylocation: 2026-06-10 06:14:41.840763 | controller | name: config 2026-06-10 06:14:41.840767 | controller | branch: "" 2026-06-10 06:14:41.840772 | controller | zuulconnectionname: gerrit 2026-06-10 06:14:41.840776 | controller | clusterapiurl: "" 2026-06-10 06:14:41.840781 | controller | logserverhost: "" 2026-06-10 06:14:41.840784 | controller | zuul: 2026-06-10 06:14:41.840788 | controller | oidcauthenticators: [] 2026-06-10 06:14:41.840792 | controller | defaultauthenticator: "" 2026-06-10 06:14:41.840796 | controller | gerritconns: 2026-06-10 06:14:41.840799 | controller | - name: gerrit 2026-06-10 06:14:41.840805 | controller | hostname: gerrit-sshd.sf 2026-06-10 06:14:41.840809 | controller | port: 0 2026-06-10 06:14:41.840815 | controller | puburl: https://gerrit.sfop.me 2026-06-10 06:14:41.840819 | controller | username: zuul 2026-06-10 06:14:41.840823 | controller | canonicalhostname: "" 2026-06-10 06:14:41.840827 | controller | password: "" 2026-06-10 06:14:41.840831 | controller | sshkey: "" 2026-06-10 06:14:41.840834 | controller | gitoverssh: false 2026-06-10 06:14:41.840838 | controller | verifyssl: false 2026-06-10 06:14:41.840841 | controller | streamevents: null 2026-06-10 06:14:41.840845 | controller | githubconns: [] 2026-06-10 06:14:41.840848 | controller | gitlabconns: [] 2026-06-10 06:14:41.840852 | controller | gitconns: [] 2026-06-10 06:14:41.840855 | controller | pagureconns: [] 2026-06-10 06:14:41.840859 | controller | elasticsearchconns: [] 2026-06-10 06:14:41.840862 | controller | smtpconns: [] 2026-06-10 06:14:41.840866 | controller | executor: 2026-06-10 06:14:41.840869 | controller | storage: 2026-06-10 06:14:41.840873 | controller | size: 2026-06-10 06:14:41.840876 | controller | format: "" 2026-06-10 06:14:41.840880 | controller | classname: "" 2026-06-10 06:14:41.840883 | controller | loglevel: DEBUG 2026-06-10 06:14:41.840887 | controller | enabled: true 2026-06-10 06:14:41.840890 | controller | standalone: null 2026-06-10 06:14:41.840894 | controller | limits: null 2026-06-10 06:14:41.840897 | controller | disklimitperjob: 0 2026-06-10 06:14:41.840901 | controller | ansiblesetuptimeout: 0 2026-06-10 06:14:41.840906 | controller | terminationgraceperiodseconds: 0 2026-06-10 06:14:41.840910 | controller | scheduler: 2026-06-10 06:14:41.840914 | controller | storage: 2026-06-10 06:14:41.840917 | controller | size: 2026-06-10 06:14:41.840920 | controller | format: "" 2026-06-10 06:14:41.840924 | controller | classname: "" 2026-06-10 06:14:41.840927 | controller | statsdtarget: "" 2026-06-10 06:14:41.840931 | controller | loglevel: DEBUG 2026-06-10 06:14:41.840934 | controller | limits: null 2026-06-10 06:14:41.840938 | controller | defaultholdexpiration: null 2026-06-10 06:14:41.840942 | controller | maxholdexpiration: null 2026-06-10 06:14:41.840945 | controller | web: 2026-06-10 06:14:41.840948 | controller | loglevel: DEBUG 2026-06-10 06:14:41.840952 | controller | limits: null 2026-06-10 06:14:41.840955 | controller | merger: 2026-06-10 06:14:41.840959 | controller | gitusername: "" 2026-06-10 06:14:41.840962 | controller | gituseremail: "" 2026-06-10 06:14:41.840966 | controller | githttplowspeedlimit: 0 2026-06-10 06:14:41.840969 | controller | githttplowspeedtime: 0 2026-06-10 06:14:41.840973 | controller | gittimeout: 0 2026-06-10 06:14:41.840976 | controller | storage: 2026-06-10 06:14:41.840980 | controller | size: 2026-06-10 06:14:41.840983 | controller | format: "" 2026-06-10 06:14:41.840987 | controller | classname: "" 2026-06-10 06:14:41.840990 | controller | loglevel: DEBUG 2026-06-10 06:14:41.840994 | controller | limits: null 2026-06-10 06:14:41.840997 | controller | nodepool: 2026-06-10 06:14:41.841001 | controller | launcher: 2026-06-10 06:14:41.841004 | controller | loglevel: "" 2026-06-10 06:14:41.841007 | controller | limits: null 2026-06-10 06:14:41.841011 | controller | builder: 2026-06-10 06:14:41.841014 | controller | storage: 2026-06-10 06:14:41.841018 | controller | size: 2026-06-10 06:14:41.841021 | controller | format: "" 2026-06-10 06:14:41.841025 | controller | classname: "" 2026-06-10 06:14:41.841028 | controller | loglevel: "" 2026-06-10 06:14:41.841036 | controller | limits: null 2026-06-10 06:14:41.841040 | controller | statsdtarget: "" 2026-06-10 06:14:41.841044 | controller | zookeeper: 2026-06-10 06:14:41.841047 | controller | storage: 2026-06-10 06:14:41.841051 | controller | size: 2026-06-10 06:14:41.841054 | controller | format: "" 2026-06-10 06:14:41.841058 | controller | classname: "" 2026-06-10 06:14:41.841061 | controller | limits: null 2026-06-10 06:14:41.841064 | controller | logserver: 2026-06-10 06:14:41.841068 | controller | retentiondays: 30 2026-06-10 06:14:41.841071 | controller | loopdelay: 3600 2026-06-10 06:14:41.841075 | controller | storage: 2026-06-10 06:14:41.841078 | controller | size: 2026-06-10 06:14:41.841082 | controller | format: BinarySI 2026-06-10 06:14:41.841085 | controller | classname: "" 2026-06-10 06:14:41.841089 | controller | podannotations: {} 2026-06-10 06:14:41.841092 | controller | logjuicer: 2026-06-10 06:14:41.841096 | controller | size: 2026-06-10 06:14:41.841100 | controller | format: "" 2026-06-10 06:14:41.841104 | controller | classname: "" 2026-06-10 06:14:41.841109 | controller | mariadb: 2026-06-10 06:14:41.841113 | controller | dbstorage: 2026-06-10 06:14:41.841118 | controller | size: 2026-06-10 06:14:41.841122 | controller | format: "" 2026-06-10 06:14:41.841127 | controller | classname: "" 2026-06-10 06:14:41.841131 | controller | logstorage: 2026-06-10 06:14:41.841136 | controller | size: 2026-06-10 06:14:41.841141 | controller | format: "" 2026-06-10 06:14:41.841146 | controller | classname: "" 2026-06-10 06:14:41.841150 | controller | limits: null 2026-06-10 06:14:41.841155 | controller | gitserver: 2026-06-10 06:14:41.841158 | controller | storage: 2026-06-10 06:14:41.841162 | controller | size: 2026-06-10 06:14:41.841165 | controller | format: "" 2026-06-10 06:14:41.841169 | controller | classname: "" 2026-06-10 06:14:41.841172 | controller | codesearch: 2026-06-10 06:14:41.841176 | controller | storage: 2026-06-10 06:14:41.841179 | controller | size: 2026-06-10 06:14:41.841183 | controller | format: "" 2026-06-10 06:14:41.841186 | controller | classname: "" 2026-06-10 06:14:41.841190 | controller | limits: null 2026-06-10 06:14:41.841193 | controller | enabled: null 2026-06-10 06:14:41.841197 | controller | hostaliases: [] 2026-06-10 06:14:41.841200 | controller | gateway: null 2026-06-10 10:14:42.102662 | controller | ok: Runtime: 0:00:00.052600 2026-06-10 10:14:42.113203 | 2026-06-10 10:14:42.113337 | TASK [health-check/test-standalone-update : Verify updated spec in configmap] 2026-06-10 10:14:42.147969 | controller | ok: All assertions passed 2026-06-10 10:14:42.156209 | 2026-06-10 10:14:42.156347 | TASK [health-check/test-standalone-update : Get standalone owner configmap annotation after update] 2026-06-10 10:14:42.700469 | controller | ok: Runtime: 0:00:00.070360 2026-06-10 10:14:42.709561 | 2026-06-10 10:14:42.709706 | TASK [health-check/test-standalone-update : Verify that the last-reconcile annotation has changed] 2026-06-10 10:14:42.754809 | controller | ok: All assertions passed 2026-06-10 10:14:42.764917 | 2026-06-10 10:14:42.765077 | TASK [Restore original state] 2026-06-10 10:14:42.811354 | controller | ok 2026-06-10 10:14:42.851968 | 2026-06-10 10:14:42.852130 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 10:14:43.089724 | controller | ok 2026-06-10 10:14:43.100010 | 2026-06-10 10:14:43.100151 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 10:14:43.381173 | controller | changed 2026-06-10 10:14:43.390360 | 2026-06-10 10:14:43.390533 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 06:14:43.952398 | controller | 2026-06-10T06:14:43-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:14:43.952471 | controller | 2026-06-10 06:14:43.960796 | controller | 2026-06-10T06:14:43-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:14:43.960819 | controller | 2026-06-10 06:14:43.974114 | controller | 2026-06-10T06:14:43-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:14:43.980313 | controller | 2026-06-10T06:14:43-04:00 INFO Updating configmap, name: zuul-scheduler-tooling-config-map 2026-06-10 06:14:43.980338 | controller | 2026-06-10T06:14:43-04:00 INFO Updating object, name: zuul-scheduler-tooling-config-map 2026-06-10 06:14:43.990206 | controller | 2026-06-10T06:14:43-04:00 INFO Statefulset "git-server" configuration changed, applying... 2026-06-10 06:14:43.990259 | controller | 2026-06-10T06:14:43-04:00 INFO Updating object, name: git-server 2026-06-10 06:14:44.579084 | controller | 2026-06-10T06:14:44-04:00 INFO Statefulset "logserver" configuration changed, applying... 2026-06-10 06:14:44.579147 | controller | 2026-06-10T06:14:44-04:00 INFO Updating object, name: logserver 2026-06-10 06:14:44.593265 | controller | 2026-06-10T06:14:44-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:14:44.593350 | controller | 2026-06-10T06:14:44-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:14:44.610410 | controller | 2026-06-10T06:14:44-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:14:44.610482 | controller | 2026-06-10T06:14:44-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:14:44.989592 | controller | 2026-06-10T06:14:44-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-10 06:14:44.989693 | controller | 2026-06-10T06:14:44-04:00 INFO 2026-06-10 06:14:44.989702 | controller | - Config: Waiting ... 2026-06-10 06:14:44.989707 | controller | - Gateway: OK 2026-06-10 06:14:44.989711 | controller | - GitServer: Waiting ... 2026-06-10 06:14:44.989715 | controller | - HoundSearch: OK 2026-06-10 06:14:44.989719 | controller | - LogJuicer: OK 2026-06-10 06:14:44.989723 | controller | - Logserver: Waiting ... 2026-06-10 06:14:44.989726 | controller | - MariaDB: OK 2026-06-10 06:14:44.989730 | controller | - Zookeeper: OK 2026-06-10 06:14:44.989734 | controller | 2026-06-10 06:14:44.989738 | controller | 2026-06-10T06:14:44-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 06:14:49.998765 | controller | 2026-06-10T06:14:49-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:14:50.024172 | controller | 2026-06-10T06:14:50-04:00 INFO Waiting for statefulset, name: git-server 2026-06-10 06:15:06.108431 | controller | 2026-06-10T06:15:06-04:00 INFO Waiting for statefulset, name: logserver 2026-06-10 06:15:38.854301 | controller | 2026-06-10T06:15:37-04:00 INFO Statefulset "logserver" not ready after 30s 2026-06-10 06:15:38.854383 | controller | 2026-06-10T06:15:37-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:15:38.854390 | controller | 2026-06-10T06:15:37-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:15:38.854395 | controller | 2026-06-10T06:15:37-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:15:38.854400 | controller | 2026-06-10T06:15:37-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:15:38.854406 | controller | 2026-06-10T06:15:37-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:15:38.897951 | controller | 2026-06-10T06:15:38-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:15:38.898038 | controller | 2026-06-10T06:15:38-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:15:38.949513 | controller | 2026-06-10T06:15:38-04:00 INFO Updating secret, name: zuul-config 2026-06-10 06:15:38.949591 | controller | 2026-06-10T06:15:38-04:00 INFO Updating object, name: zuul-config 2026-06-10 06:15:38.982528 | controller | 2026-06-10T06:15:38-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-10 06:15:38.982684 | controller | 2026-06-10T06:15:38-04:00 INFO Updating object, name: zuul-scheduler 2026-06-10 06:15:40.262032 | controller | 2026-06-10T06:15:40-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-10 06:15:40.262191 | controller | 2026-06-10T06:15:40-04:00 INFO Updating object, name: zuul-web 2026-06-10 06:15:41.073403 | controller | 2026-06-10T06:15:41-04:00 INFO 2026-06-10 06:15:41.073507 | controller | - Config: Waiting ... 2026-06-10 06:15:41.073515 | controller | - Gateway: OK 2026-06-10 06:15:41.073544 | controller | - GitServer: OK 2026-06-10 06:15:41.073549 | controller | - HoundSearch: OK 2026-06-10 06:15:41.073554 | controller | - LogJuicer: OK 2026-06-10 06:15:41.073559 | controller | - Logserver: Waiting ... 2026-06-10 06:15:41.073564 | controller | - MariaDB: OK 2026-06-10 06:15:41.073571 | controller | - NodePoolBuilder: OK 2026-06-10 06:15:41.073576 | controller | - NodePoolLauncher: OK 2026-06-10 06:15:41.073581 | controller | - Zookeeper: OK 2026-06-10 06:15:41.073586 | controller | - Zuul: Waiting ... 2026-06-10 06:15:41.073591 | controller | - |_ Config Secrets: OK 2026-06-10 06:15:41.073596 | controller | - |_ Executor: OK 2026-06-10 06:15:41.073601 | controller | - |_ Merger: OK 2026-06-10 06:15:41.073633 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:15:41.073639 | controller | - |_ Web: Waiting ... 2026-06-10 06:15:41.073644 | controller | 2026-06-10 06:15:41.073658 | controller | 2026-06-10T06:15:41-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 06:15:46.081000 | controller | 2026-06-10T06:15:46-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:15:46.691667 | controller | 2026-06-10T06:15:46-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:15:47.792017 | controller | 2026-06-10T06:15:47-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:15:47.799710 | controller | 2026-06-10T06:15:47-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:15:47.799767 | controller | 2026-06-10T06:15:47-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:15:47.816331 | controller | 2026-06-10T06:15:47-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:15:47.816372 | controller | 2026-06-10T06:15:47-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:15:47.834831 | controller | 2026-06-10T06:15:47-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:15:47.854120 | controller | 2026-06-10T06:15:47-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:15:47.854191 | controller | 2026-06-10T06:15:47-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:15:48.803016 | controller | 2026-06-10T06:15:48-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:16:19.386642 | controller | 2026-06-10T06:16:19-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 06:16:19.421005 | controller | 2026-06-10T06:16:19-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:16:19.421077 | controller | 2026-06-10T06:16:19-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:16:19.796169 | controller | 2026-06-10T06:16:19-04:00 INFO 2026-06-10 06:16:19.796262 | controller | - Config: Waiting ... 2026-06-10 06:16:19.796284 | controller | - Gateway: OK 2026-06-10 06:16:19.796294 | controller | - GitServer: OK 2026-06-10 06:16:19.796302 | controller | - HoundSearch: OK 2026-06-10 06:16:19.796310 | controller | - LogJuicer: OK 2026-06-10 06:16:19.796318 | controller | - Logserver: OK 2026-06-10 06:16:19.796326 | controller | - MariaDB: OK 2026-06-10 06:16:19.796334 | controller | - NodePoolBuilder: OK 2026-06-10 06:16:19.796342 | controller | - NodePoolLauncher: OK 2026-06-10 06:16:19.796350 | controller | - Zookeeper: OK 2026-06-10 06:16:19.796358 | controller | - Zuul: Waiting ... 2026-06-10 06:16:19.796365 | controller | - |_ Config Secrets: OK 2026-06-10 06:16:19.796373 | controller | - |_ Executor: OK 2026-06-10 06:16:19.796381 | controller | - |_ Merger: OK 2026-06-10 06:16:19.796389 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:16:19.796397 | controller | - |_ Web: Waiting ... 2026-06-10 06:16:19.796405 | controller | 2026-06-10 06:16:19.796422 | controller | 2026-06-10T06:16:19-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 06:16:24.802222 | controller | 2026-06-10T06:16:24-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:16:25.416681 | controller | 2026-06-10T06:16:25-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:16:25.416753 | controller | 2026-06-10T06:16:25-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:16:25.429459 | controller | 2026-06-10T06:16:25-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:16:25.429511 | controller | 2026-06-10T06:16:25-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:16:25.811567 | controller | 2026-06-10T06:16:25-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:16:26.217519 | controller | 2026-06-10T06:16:26-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:16:26.217572 | controller | 2026-06-10T06:16:26-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:16:27.216869 | controller | 2026-06-10T06:16:27-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:16:45.285888 | controller | 2026-06-10T06:16:45-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:16:45.285929 | controller | 2026-06-10T06:16:45-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:16:45.306313 | controller | 2026-06-10T06:16:45-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:16:45.306388 | controller | 2026-06-10T06:16:45-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:16:46.099828 | controller | 2026-06-10T06:16:46-04:00 INFO 2026-06-10 06:16:46.099874 | controller | - Config: OK 2026-06-10 06:16:46.099879 | controller | - Gateway: OK 2026-06-10 06:16:46.099883 | controller | - GitServer: OK 2026-06-10 06:16:46.099887 | controller | - HoundSearch: OK 2026-06-10 06:16:46.099890 | controller | - LogJuicer: OK 2026-06-10 06:16:46.099894 | controller | - Logserver: OK 2026-06-10 06:16:46.099898 | controller | - MariaDB: OK 2026-06-10 06:16:46.099901 | controller | - NodePoolBuilder: OK 2026-06-10 06:16:46.099905 | controller | - NodePoolLauncher: OK 2026-06-10 06:16:46.099909 | controller | - Zookeeper: OK 2026-06-10 06:16:46.099912 | controller | - Zuul: OK 2026-06-10 06:16:46.099916 | controller | 2026-06-10 06:16:46.099925 | controller | 2026-06-10T06:16:46-04:00 INFO Updating controller configmap ... 2026-06-10 06:16:46.489510 | controller | 2026-06-10T06:16:46-04:00 INFO Standalone reconcile done. 2026-06-10 10:16:46.544232 | controller | ok: Runtime: 0:02:02.819566 2026-06-10 10:16:46.558577 | 2026-06-10 10:16:46.558874 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 10:16:47.101861 | controller | ok: Runtime: 0:00:00.004103 2026-06-10 10:16:47.118464 | 2026-06-10 10:16:47.118626 | TASK [health-check/test-standalone-update : Remove temporary directory] 2026-06-10 10:16:47.408043 | controller | changed 2026-06-10 10:16:47.425057 | 2026-06-10 10:16:47.425196 | TASK [health-check/check-zuul-weeder : Get zuul-weeder status] 2026-06-10 10:16:48.251016 | controller | ok: OK (unknown bytes) 2026-06-10 10:16:48.306632 | 2026-06-10 10:16:48.306783 | TASK [scale down zuul components] 2026-06-10 10:16:48.337715 | controller | ok 2026-06-10 10:16:48.356344 | 2026-06-10 10:16:48.356463 | LOOP [health-check/scale-resources : Scaling resources] 2026-06-10 10:16:48.391886 | controller | ok: 2026-06-10 10:16:48.392334 | controller | { 2026-06-10 10:16:48.392413 | controller | "kind": "sts", 2026-06-10 10:16:48.392531 | controller | "name": "zuul-executor", 2026-06-10 10:16:48.392574 | controller | "replicas": 0 2026-06-10 10:16:48.392609 | controller | } 2026-06-10 10:16:48.392686 | controller | changed: All items complete 2026-06-10 10:16:48.392730 | 2026-06-10 10:16:48.400232 | controller | ok: 2026-06-10 10:16:48.400330 | controller | { 2026-06-10 10:16:48.400380 | controller | "kind": "sts", 2026-06-10 10:16:48.400419 | controller | "name": "zuul-merger", 2026-06-10 10:16:48.400519 | controller | "replicas": 0 2026-06-10 10:16:48.400560 | controller | } 2026-06-10 10:16:48.407319 | controller | ok: 2026-06-10 10:16:48.407412 | controller | { 2026-06-10 10:16:48.407499 | controller | "kind": "sts", 2026-06-10 10:16:48.407539 | controller | "name": "zuul-scheduler", 2026-06-10 10:16:48.407574 | controller | "replicas": 0 2026-06-10 10:16:48.407608 | controller | } 2026-06-10 10:16:48.414599 | controller | ok: 2026-06-10 10:16:48.414693 | controller | { 2026-06-10 10:16:48.414739 | controller | "kind": "deployment", 2026-06-10 10:16:48.414778 | controller | "name": "zuul-web", 2026-06-10 10:16:48.414814 | controller | "replicas": 0 2026-06-10 10:16:48.414849 | controller | } 2026-06-10 10:16:48.428075 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 10:16:48.433854 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 10:16:48.437822 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 10:16:48.441652 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 10:16:48.449709 | 2026-06-10 10:16:48.449800 | TASK [health-check/scale-resources : Get initial replica count for zuul-executor] 2026-06-10 10:16:49.040341 | controller | ok: Runtime: 0:00:00.058126 2026-06-10 10:16:49.052313 | 2026-06-10 10:16:49.052490 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-10 06:16:49.364645 | controller | Defaulted container "zuul-executor" out of: zuul-executor, zuul-executor-nodeexporter 2026-06-10 10:16:49.599335 | controller | ok: Runtime: 0:00:00.092503 2026-06-10 10:16:49.611256 | 2026-06-10 10:16:49.611396 | TASK [health-check/scale-resources : Scale on sts zuul-executor] 2026-06-10 06:16:49.973999 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 10:16:50.159163 | controller | ok: Runtime: 0:00:00.071487 2026-06-10 10:16:50.170188 | 2026-06-10 10:16:50.170326 | TASK [health-check/scale-resources : Get replica count for zuul-executor] 2026-06-10 10:17:00.876375 | controller | ok: Runtime: 0:00:00.051644 2026-06-10 10:17:00.888541 | 2026-06-10 10:17:00.888678 | TASK [health-check/scale-resources : Get initial replica count for zuul-merger] 2026-06-10 10:17:01.434234 | controller | ok: Runtime: 0:00:00.050124 2026-06-10 10:17:01.445237 | 2026-06-10 10:17:01.445394 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-10 06:17:01.736900 | controller | Defaulted container "zuul-merger" out of: zuul-merger, zuul-merger-nodeexporter 2026-06-10 10:17:01.992818 | controller | ok: Runtime: 0:00:00.068476 2026-06-10 10:17:02.006121 | 2026-06-10 10:17:02.006297 | TASK [health-check/scale-resources : Scale on sts zuul-merger] 2026-06-10 06:17:02.305914 | controller | statefulset.apps/zuul-merger scaled 2026-06-10 10:17:02.555312 | controller | ok: Runtime: 0:00:00.062468 2026-06-10 10:17:02.566638 | 2026-06-10 10:17:02.566793 | TASK [health-check/scale-resources : Get replica count for zuul-merger] 2026-06-10 10:17:13.145638 | controller | ok: Runtime: 0:00:00.069868 2026-06-10 10:17:13.154950 | 2026-06-10 10:17:13.155027 | TASK [health-check/scale-resources : Get initial replica count for zuul-scheduler] 2026-06-10 10:17:13.682921 | controller | ok: Runtime: 0:00:00.057526 2026-06-10 10:17:13.694489 | 2026-06-10 10:17:13.694638 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-10 06:17:13.993950 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-10 10:17:14.241869 | controller | ok: Runtime: 0:00:00.070835 2026-06-10 10:17:14.253670 | 2026-06-10 10:17:14.253814 | TASK [health-check/scale-resources : Scale on sts zuul-scheduler] 2026-06-10 06:17:14.623064 | controller | statefulset.apps/zuul-scheduler scaled 2026-06-10 10:17:14.849591 | controller | ok: Runtime: 0:00:00.077988 2026-06-10 10:17:14.860772 | 2026-06-10 10:17:14.860907 | TASK [health-check/scale-resources : Get replica count for zuul-scheduler] 2026-06-10 10:17:25.480259 | controller | ok: Runtime: 0:00:00.054018 2026-06-10 10:17:25.493623 | 2026-06-10 10:17:25.493828 | TASK [health-check/scale-resources : Get initial replica count for zuul-web] 2026-06-10 10:17:26.039080 | controller | ok: Runtime: 0:00:00.053955 2026-06-10 10:17:26.050400 | 2026-06-10 10:17:26.050574 | TASK [health-check/scale-resources : Collect deployment log before executing the scale command] 2026-06-10 10:17:26.599680 | controller | ok: Runtime: 0:00:00.067203 2026-06-10 10:17:26.611213 | 2026-06-10 10:17:26.611362 | TASK [health-check/scale-resources : Scale on deployment zuul-web] 2026-06-10 06:17:26.923227 | controller | deployment.apps/zuul-web scaled 2026-06-10 10:17:27.158296 | controller | ok: Runtime: 0:00:00.061636 2026-06-10 10:17:27.170327 | 2026-06-10 10:17:27.170509 | TASK [health-check/scale-resources : Get replica count for zuul-web] 2026-06-10 10:17:27.719599 | controller | ok: Runtime: 0:00:00.052163 2026-06-10 10:17:27.740835 | 2026-06-10 10:17:27.741016 | TASK [Enable log forwarding] 2026-06-10 10:17:27.778088 | controller | ok 2026-06-10 10:17:27.807940 | 2026-06-10 10:17:27.808049 | TASK [update-custom-resource : read current crd] 2026-06-10 10:17:28.195962 | controller | ok 2026-06-10 10:17:28.206659 | 2026-06-10 10:17:28.206802 | TASK [update-custom-resource : merge the new cr] 2026-06-10 10:17:28.931171 | controller | changed 2026-06-10 10:17:28.944147 | 2026-06-10 10:17:28.944293 | TASK [run the operator in standalone mode] 2026-06-10 10:17:28.980743 | controller | ok 2026-06-10 10:17:29.009709 | 2026-06-10 10:17:29.009795 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 10:17:29.035368 | controller | skipping: Conditional result was False 2026-06-10 10:17:29.042923 | 2026-06-10 10:17:29.043014 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 10:17:29.068625 | controller | skipping: Conditional result was False 2026-06-10 10:17:29.076424 | 2026-06-10 10:17:29.076532 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 06:17:29.549007 | controller | 2026-06-10T06:17:29-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:17:29.549073 | controller | 2026-06-10 06:17:29.557782 | controller | 2026-06-10T06:17:29-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:17:29.557830 | controller | 2026-06-10 06:17:29.570329 | controller | 2026-06-10T06:17:29-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:17:29.601446 | controller | 2026-06-10T06:17:29-04:00 INFO Creating config map name: fluentbit-mariadb-cfg-config-map 2026-06-10 06:17:29.601508 | controller | 2026-06-10T06:17:29-04:00 INFO Creating object, name: fluentbit-mariadb-cfg-config-map 2026-06-10 06:17:29.606671 | controller | 2026-06-10T06:17:29-04:00 INFO Statefulset "mariadb" configuration changed, applying... 2026-06-10 06:17:29.606723 | controller | 2026-06-10T06:17:29-04:00 INFO Updating object, name: mariadb 2026-06-10 06:17:29.980062 | controller | 2026-06-10T06:17:29-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:17:30.055299 | controller | 2026-06-10T06:17:30-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:17:30.062249 | controller | 2026-06-10T06:17:30-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:17:30.062299 | controller | 2026-06-10T06:17:30-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:17:30.076625 | controller | 2026-06-10T06:17:30-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:17:30.076685 | controller | 2026-06-10T06:17:30-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:17:30.376627 | controller | 2026-06-10T06:17:30-04:00 INFO Creating config map name: fluentbit-zk-cfg-config-map 2026-06-10 06:17:30.376686 | controller | 2026-06-10T06:17:30-04:00 INFO Creating object, name: fluentbit-zk-cfg-config-map 2026-06-10 06:17:30.380711 | controller | 2026-06-10T06:17:30-04:00 INFO Statefulset "zookeeper" configuration changed, applying... 2026-06-10 06:17:30.380758 | controller | 2026-06-10T06:17:30-04:00 INFO Updating object, name: zookeeper 2026-06-10 06:17:30.385856 | controller | 2026-06-10T06:17:30-04:00 INFO Waiting for Zookeeper service to be ready ... 2026-06-10 06:17:30.385893 | controller | 2026-06-10T06:17:30-04:00 INFO 2026-06-10 06:17:30.385898 | controller | - Config: Waiting ... 2026-06-10 06:17:30.385903 | controller | - Gateway: OK 2026-06-10 06:17:30.385907 | controller | - GitServer: OK 2026-06-10 06:17:30.385911 | controller | - HoundSearch: OK 2026-06-10 06:17:30.385914 | controller | - LogJuicer: OK 2026-06-10 06:17:30.385918 | controller | - Logserver: OK 2026-06-10 06:17:30.385921 | controller | - MariaDB: Waiting ... 2026-06-10 06:17:30.385925 | controller | - Zookeeper: Waiting ... 2026-06-10 06:17:30.385928 | controller | 2026-06-10 06:17:30.385936 | controller | 2026-06-10T06:17:30-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 06:17:35.393740 | controller | 2026-06-10T06:17:35-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:17:35.425568 | controller | 2026-06-10T06:17:35-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-10 06:17:37.459344 | controller | 2026-06-10T06:17:37-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:17:37.459407 | controller | 2026-06-10T06:17:37-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:17:37.476314 | controller | 2026-06-10T06:17:37-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:17:37.476361 | controller | 2026-06-10T06:17:37-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:17:37.491241 | controller | 2026-06-10T06:17:37-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-10 06:17:39.496483 | controller | 2026-06-10T06:17:39-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:17:39.503158 | controller | 2026-06-10T06:17:39-04:00 INFO Updating configmap, name: nodepool-launcher-extra-config-config-map 2026-06-10 06:17:39.503268 | controller | 2026-06-10T06:17:39-04:00 INFO Updating object, name: nodepool-launcher-extra-config-config-map 2026-06-10 06:17:39.511927 | controller | 2026-06-10T06:17:39-04:00 INFO Statefulset "nodepool-launcher" configuration changed, applying... 2026-06-10 06:17:39.511994 | controller | 2026-06-10T06:17:39-04:00 INFO Updating object, name: nodepool-launcher 2026-06-10 06:17:39.526619 | controller | 2026-06-10T06:17:39-04:00 INFO Updating configmap, name: nodepool-builder-extra-config-config-map 2026-06-10 06:17:39.526673 | controller | 2026-06-10T06:17:39-04:00 INFO Updating object, name: nodepool-builder-extra-config-config-map 2026-06-10 06:17:39.533579 | controller | 2026-06-10T06:17:39-04:00 INFO Creating config map name: fluentbit-dib-cfg-config-map 2026-06-10 06:17:39.533664 | controller | 2026-06-10T06:17:39-04:00 INFO Creating object, name: fluentbit-dib-cfg-config-map 2026-06-10 06:17:39.708680 | controller | 2026-06-10T06:17:39-04:00 INFO Statefulset "nodepool-builder" configuration changed, applying... 2026-06-10 06:17:39.708743 | controller | 2026-06-10T06:17:39-04:00 INFO Updating object, name: nodepool-builder 2026-06-10 06:17:40.502743 | controller | 2026-06-10T06:17:40-04:00 INFO Updating configmap, name: zuul-logging-config-map 2026-06-10 06:17:40.502804 | controller | 2026-06-10T06:17:40-04:00 INFO Updating object, name: zuul-logging-config-map 2026-06-10 06:17:40.705422 | controller | 2026-06-10T06:17:40-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-10 06:17:40.705488 | controller | 2026-06-10T06:17:40-04:00 INFO Updating object, name: zuul-scheduler 2026-06-10 06:17:41.503689 | controller | 2026-06-10T06:17:41-04:00 INFO Statefulset "zuul-executor" configuration changed, applying... 2026-06-10 06:17:41.503741 | controller | 2026-06-10T06:17:41-04:00 INFO Updating object, name: zuul-executor 2026-06-10 06:17:42.303853 | controller | 2026-06-10T06:17:42-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-10 06:17:42.303925 | controller | 2026-06-10T06:17:42-04:00 INFO Updating object, name: zuul-web 2026-06-10 06:17:43.104552 | controller | 2026-06-10T06:17:43-04:00 INFO Statefulset "zuul-merger" configuration changed, applying... 2026-06-10 06:17:43.104599 | controller | 2026-06-10T06:17:43-04:00 INFO Updating object, name: zuul-merger 2026-06-10 06:17:43.111207 | controller | 2026-06-10T06:17:43-04:00 INFO 2026-06-10 06:17:43.111250 | controller | - Config: Waiting ... 2026-06-10 06:17:43.111255 | controller | - Gateway: OK 2026-06-10 06:17:43.111259 | controller | - GitServer: OK 2026-06-10 06:17:43.111263 | controller | - HoundSearch: OK 2026-06-10 06:17:43.111267 | controller | - LogJuicer: OK 2026-06-10 06:17:43.111270 | controller | - Logserver: OK 2026-06-10 06:17:43.111274 | controller | - MariaDB: OK 2026-06-10 06:17:43.111278 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:17:43.111282 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:17:43.111285 | controller | - Zookeeper: OK 2026-06-10 06:17:43.111289 | controller | - Zuul: Waiting ... 2026-06-10 06:17:43.111292 | controller | - |_ Config Secrets: OK 2026-06-10 06:17:43.111296 | controller | - |_ Executor: Waiting ... 2026-06-10 06:17:43.111300 | controller | - |_ Merger: Waiting ... 2026-06-10 06:17:43.111303 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:17:43.111307 | controller | - |_ Web: Waiting ... 2026-06-10 06:17:43.111310 | controller | 2026-06-10 06:17:43.111319 | controller | 2026-06-10T06:17:43-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 06:17:48.118044 | controller | 2026-06-10T06:17:48-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:17:48.728103 | controller | 2026-06-10T06:17:48-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:17:48.728148 | controller | 2026-06-10T06:17:48-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:17:48.740488 | controller | 2026-06-10T06:17:48-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:17:48.740556 | controller | 2026-06-10T06:17:48-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:17:49.132192 | controller | 2026-06-10T06:17:49-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:17:49.531557 | controller | 2026-06-10T06:17:49-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:17:49.531734 | controller | 2026-06-10T06:17:49-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 06:17:50.536126 | controller | 2026-06-10T06:17:50-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:17:50.536192 | controller | 2026-06-10T06:17:50-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:17:52.122335 | controller | 2026-06-10T06:17:52-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:17:53.329940 | controller | 2026-06-10T06:17:53-04:00 INFO 2026-06-10 06:17:53.329985 | controller | - Config: OK 2026-06-10 06:17:53.329993 | controller | - Gateway: OK 2026-06-10 06:17:53.329998 | controller | - GitServer: OK 2026-06-10 06:17:53.330004 | controller | - HoundSearch: OK 2026-06-10 06:17:53.330011 | controller | - LogJuicer: OK 2026-06-10 06:17:53.330027 | controller | - Logserver: OK 2026-06-10 06:17:53.330033 | controller | - MariaDB: OK 2026-06-10 06:17:53.330039 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:17:53.330044 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:17:53.330050 | controller | - Zookeeper: OK 2026-06-10 06:17:53.330055 | controller | - Zuul: OK 2026-06-10 06:17:53.330061 | controller | 2026-06-10 06:17:53.330066 | controller | 2026-06-10T06:17:53-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 06:17:58.334468 | controller | 2026-06-10T06:17:58-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:17:58.949388 | controller | 2026-06-10T06:17:58-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:17:58.949440 | controller | 2026-06-10T06:17:58-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:17:58.968684 | controller | 2026-06-10T06:17:58-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:17:58.968749 | controller | 2026-06-10T06:17:58-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:17:59.349590 | controller | 2026-06-10T06:17:59-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:17:59.745107 | controller | 2026-06-10T06:17:59-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:17:59.745152 | controller | 2026-06-10T06:17:59-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 06:18:00.761581 | controller | 2026-06-10T06:18:00-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:18:00.761668 | controller | 2026-06-10T06:18:00-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:18:02.338804 | controller | 2026-06-10T06:18:02-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:18:03.543502 | controller | 2026-06-10T06:18:03-04:00 INFO 2026-06-10 06:18:03.543545 | controller | - Config: OK 2026-06-10 06:18:03.543550 | controller | - Gateway: OK 2026-06-10 06:18:03.543554 | controller | - GitServer: OK 2026-06-10 06:18:03.543558 | controller | - HoundSearch: OK 2026-06-10 06:18:03.543561 | controller | - LogJuicer: OK 2026-06-10 06:18:03.543565 | controller | - Logserver: OK 2026-06-10 06:18:03.543568 | controller | - MariaDB: OK 2026-06-10 06:18:03.543572 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:18:03.543576 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:18:03.543579 | controller | - Zookeeper: OK 2026-06-10 06:18:03.543583 | controller | - Zuul: OK 2026-06-10 06:18:03.543587 | controller | 2026-06-10 06:18:03.543591 | controller | 2026-06-10T06:18:03-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-10 06:18:08.551638 | controller | 2026-06-10T06:18:08-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:18:09.163485 | controller | 2026-06-10T06:18:09-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:18:09.163539 | controller | 2026-06-10T06:18:09-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:18:09.178751 | controller | 2026-06-10T06:18:09-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:18:09.178793 | controller | 2026-06-10T06:18:09-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:18:09.571343 | controller | 2026-06-10T06:18:09-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:18:09.971920 | controller | 2026-06-10T06:18:09-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:18:09.971966 | controller | 2026-06-10T06:18:09-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 06:18:10.972310 | controller | 2026-06-10T06:18:10-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:18:10.972351 | controller | 2026-06-10T06:18:10-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:18:12.554778 | controller | 2026-06-10T06:18:12-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:18:13.759716 | controller | 2026-06-10T06:18:13-04:00 INFO 2026-06-10 06:18:13.759757 | controller | - Config: OK 2026-06-10 06:18:13.759762 | controller | - Gateway: OK 2026-06-10 06:18:13.759766 | controller | - GitServer: OK 2026-06-10 06:18:13.759770 | controller | - HoundSearch: OK 2026-06-10 06:18:13.759774 | controller | - LogJuicer: OK 2026-06-10 06:18:13.759777 | controller | - Logserver: OK 2026-06-10 06:18:13.759780 | controller | - MariaDB: OK 2026-06-10 06:18:13.759784 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:18:13.759788 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:18:13.759791 | controller | - Zookeeper: OK 2026-06-10 06:18:13.759795 | controller | - Zuul: OK 2026-06-10 06:18:13.759799 | controller | 2026-06-10 06:18:13.759814 | controller | 2026-06-10T06:18:13-04:00 INFO [attempt #5] Waiting 5s for the next reconcile call ... 2026-06-10 06:18:18.768709 | controller | 2026-06-10T06:18:18-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:18:19.379008 | controller | 2026-06-10T06:18:19-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:18:19.379052 | controller | 2026-06-10T06:18:19-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:18:19.390238 | controller | 2026-06-10T06:18:19-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:18:19.390293 | controller | 2026-06-10T06:18:19-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:18:19.778906 | controller | 2026-06-10T06:18:19-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:18:20.178471 | controller | 2026-06-10T06:18:20-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:18:20.178510 | controller | 2026-06-10T06:18:20-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:18:21.182480 | controller | 2026-06-10T06:18:21-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:18:21.182534 | controller | 2026-06-10T06:18:21-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:18:22.771280 | controller | 2026-06-10T06:18:22-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:18:23.988253 | controller | 2026-06-10T06:18:23-04:00 INFO 2026-06-10 06:18:23.988315 | controller | - Config: OK 2026-06-10 06:18:23.988320 | controller | - Gateway: OK 2026-06-10 06:18:23.988324 | controller | - GitServer: OK 2026-06-10 06:18:23.988327 | controller | - HoundSearch: OK 2026-06-10 06:18:23.988331 | controller | - LogJuicer: OK 2026-06-10 06:18:23.988335 | controller | - Logserver: OK 2026-06-10 06:18:23.988338 | controller | - MariaDB: OK 2026-06-10 06:18:23.988342 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:18:23.988346 | controller | - NodePoolLauncher: OK 2026-06-10 06:18:23.988350 | controller | - Zookeeper: OK 2026-06-10 06:18:23.988353 | controller | - Zuul: OK 2026-06-10 06:18:23.988357 | controller | 2026-06-10 06:18:23.988362 | controller | 2026-06-10T06:18:23-04:00 INFO [attempt #6] Waiting 5s for the next reconcile call ... 2026-06-10 06:18:28.995532 | controller | 2026-06-10T06:18:28-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:18:29.611057 | controller | 2026-06-10T06:18:29-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:18:29.611199 | controller | 2026-06-10T06:18:29-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:18:29.629547 | controller | 2026-06-10T06:18:29-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:18:29.630799 | controller | 2026-06-10T06:18:29-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:18:30.014226 | controller | 2026-06-10T06:18:30-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:18:30.421255 | controller | 2026-06-10T06:18:30-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:18:30.421373 | controller | 2026-06-10T06:18:30-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:18:31.415726 | controller | 2026-06-10T06:18:31-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:18:31.415806 | controller | 2026-06-10T06:18:31-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:18:33.000839 | controller | 2026-06-10T06:18:33-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:18:34.204939 | controller | 2026-06-10T06:18:34-04:00 INFO 2026-06-10 06:18:34.204989 | controller | - Config: OK 2026-06-10 06:18:34.204996 | controller | - Gateway: OK 2026-06-10 06:18:34.205002 | controller | - GitServer: OK 2026-06-10 06:18:34.205007 | controller | - HoundSearch: OK 2026-06-10 06:18:34.205012 | controller | - LogJuicer: OK 2026-06-10 06:18:34.205017 | controller | - Logserver: OK 2026-06-10 06:18:34.205022 | controller | - MariaDB: OK 2026-06-10 06:18:34.205026 | controller | - NodePoolBuilder: OK 2026-06-10 06:18:34.205031 | controller | - NodePoolLauncher: OK 2026-06-10 06:18:34.205036 | controller | - Zookeeper: OK 2026-06-10 06:18:34.205039 | controller | - Zuul: OK 2026-06-10 06:18:34.205043 | controller | 2026-06-10 06:18:34.205047 | controller | 2026-06-10T06:18:34-04:00 INFO Updating controller configmap ... 2026-06-10 06:18:34.598145 | controller | 2026-06-10T06:18:34-04:00 INFO Standalone reconcile done. 2026-06-10 10:18:34.953656 | controller | ok: Runtime: 0:01:05.298712 2026-06-10 10:18:34.965816 | 2026-06-10 10:18:34.965951 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 10:18:35.507619 | controller | skipping: Conditional result was False 2026-06-10 10:18:35.535675 | 2026-06-10 10:18:35.535838 | TASK [scale zuul components back up to defaults] 2026-06-10 10:18:35.571094 | controller | ok 2026-06-10 10:18:35.602077 | 2026-06-10 10:18:35.602171 | LOOP [health-check/scale-resources : Scaling resources] 2026-06-10 10:18:35.638591 | controller | ok: 2026-06-10 10:18:35.639024 | controller | { 2026-06-10 10:18:35.639088 | controller | "kind": "sts", 2026-06-10 10:18:35.639131 | controller | "name": "zuul-executor", 2026-06-10 10:18:35.639169 | controller | "replicas": 1 2026-06-10 10:18:35.639206 | controller | } 2026-06-10 10:18:35.646399 | controller | ok: 2026-06-10 10:18:35.646542 | controller | { 2026-06-10 10:18:35.646597 | controller | "kind": "sts", 2026-06-10 10:18:35.646640 | controller | "name": "zuul-merger", 2026-06-10 10:18:35.646680 | controller | "replicas": 1 2026-06-10 10:18:35.646718 | controller | } 2026-06-10 10:18:35.655025 | controller | ok: 2026-06-10 10:18:35.655129 | controller | { 2026-06-10 10:18:35.655178 | controller | "kind": "sts", 2026-06-10 10:18:35.655215 | controller | "name": "zuul-scheduler", 2026-06-10 10:18:35.655251 | controller | "replicas": 1 2026-06-10 10:18:35.655285 | controller | } 2026-06-10 10:18:35.663575 | controller | ok: 2026-06-10 10:18:35.663673 | controller | { 2026-06-10 10:18:35.663721 | controller | "kind": "deployment", 2026-06-10 10:18:35.663760 | controller | "name": "zuul-web", 2026-06-10 10:18:35.663797 | controller | "replicas": 1 2026-06-10 10:18:35.663832 | controller | } 2026-06-10 10:18:35.753824 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 10:18:35.756764 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 10:18:35.759398 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 10:18:35.762000 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 10:18:35.767019 | 2026-06-10 10:18:35.767080 | TASK [health-check/scale-resources : Get initial replica count for zuul-executor] 2026-06-10 10:18:36.296072 | controller | ok: Runtime: 0:00:00.056266 2026-06-10 10:18:36.307130 | 2026-06-10 10:18:36.307277 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-10 10:18:36.847173 | controller | skipping: Conditional result was False 2026-06-10 10:18:36.859807 | 2026-06-10 10:18:36.859963 | TASK [health-check/scale-resources : Scale on sts zuul-executor] 2026-06-10 06:18:37.181310 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 10:18:37.404101 | controller | ok: Runtime: 0:00:00.075113 2026-06-10 10:18:37.411089 | 2026-06-10 10:18:37.411172 | TASK [health-check/scale-resources : Get replica count for zuul-executor] 2026-06-10 10:18:58.277820 | controller | ok: Runtime: 0:00:00.051616 2026-06-10 10:18:58.290606 | 2026-06-10 10:18:58.290761 | TASK [health-check/scale-resources : Get initial replica count for zuul-merger] 2026-06-10 10:18:59.491581 | controller | ok: Runtime: 0:00:00.066060 2026-06-10 10:18:59.508161 | 2026-06-10 10:18:59.508307 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-10 10:19:00.053180 | controller | skipping: Conditional result was False 2026-06-10 10:19:00.068738 | 2026-06-10 10:19:00.068914 | TASK [health-check/scale-resources : Scale on sts zuul-merger] 2026-06-10 06:19:00.382581 | controller | statefulset.apps/zuul-merger scaled 2026-06-10 10:19:00.618956 | controller | ok: Runtime: 0:00:00.061787 2026-06-10 10:19:00.626384 | 2026-06-10 10:19:00.626512 | TASK [health-check/scale-resources : Get replica count for zuul-merger] 2026-06-10 10:19:21.490076 | controller | ok: Runtime: 0:00:00.067510 2026-06-10 10:19:21.501502 | 2026-06-10 10:19:21.501647 | TASK [health-check/scale-resources : Get initial replica count for zuul-scheduler] 2026-06-10 10:19:22.701088 | controller | ok: Runtime: 0:00:00.054424 2026-06-10 10:19:22.712301 | 2026-06-10 10:19:22.712502 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-10 10:19:22.774050 | controller | skipping: Conditional result was False 2026-06-10 10:19:22.787610 | 2026-06-10 10:19:22.787789 | TASK [health-check/scale-resources : Scale on sts zuul-scheduler] 2026-06-10 06:19:23.747925 | controller | statefulset.apps/zuul-scheduler scaled 2026-06-10 10:19:24.215858 | controller | ok: Runtime: 0:00:00.612760 2026-06-10 10:19:24.227073 | 2026-06-10 10:19:24.227292 | TASK [health-check/scale-resources : Get replica count for zuul-scheduler] 2026-06-10 10:20:26.594767 | controller | ok: Runtime: 0:00:00.050074 2026-06-10 10:20:26.605773 | 2026-06-10 10:20:26.605904 | TASK [health-check/scale-resources : Get initial replica count for zuul-web] 2026-06-10 10:20:27.151485 | controller | ok: Runtime: 0:00:00.053338 2026-06-10 10:20:27.162548 | 2026-06-10 10:20:27.162689 | TASK [health-check/scale-resources : Collect deployment log before executing the scale command] 2026-06-10 10:20:28.438331 | controller | skipping: Conditional result was False 2026-06-10 10:20:28.451761 | 2026-06-10 10:20:28.451943 | TASK [health-check/scale-resources : Scale on deployment zuul-web] 2026-06-10 06:20:28.738772 | controller | deployment.apps/zuul-web scaled 2026-06-10 10:20:28.999346 | controller | ok: Runtime: 0:00:00.057956 2026-06-10 10:20:29.010604 | 2026-06-10 10:20:29.010739 | TASK [health-check/scale-resources : Get replica count for zuul-web] 2026-06-10 10:21:31.488343 | controller | ok: Runtime: 0:00:00.062870 2026-06-10 10:21:31.502449 | 2026-06-10 10:21:31.502547 | LOOP [health-check/scale-resources : Scaling resources] 2026-06-10 10:21:31.535461 | controller | ok: 2026-06-10 10:21:31.535740 | controller | { 2026-06-10 10:21:31.535856 | controller | "kind": "sts", 2026-06-10 10:21:31.535889 | controller | "name": "zuul-executor", 2026-06-10 10:21:31.535919 | controller | "replicas": 2 2026-06-10 10:21:31.535947 | controller | } 2026-06-10 10:21:31.535995 | controller | ok: All items complete 2026-06-10 10:21:31.536029 | 2026-06-10 10:21:31.540618 | controller | ok: 2026-06-10 10:21:31.540676 | controller | { 2026-06-10 10:21:31.540713 | controller | "kind": "sts", 2026-06-10 10:21:31.540743 | controller | "name": "zuul-merger", 2026-06-10 10:21:31.540769 | controller | "replicas": 2 2026-06-10 10:21:31.540793 | controller | } 2026-06-10 10:21:31.559404 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 10:21:31.563043 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/scale-resources/tasks/scale.yaml 2026-06-10 10:21:31.568848 | 2026-06-10 10:21:31.568928 | TASK [health-check/scale-resources : Get initial replica count for zuul-executor] 2026-06-10 10:21:36.107980 | controller | ok: Runtime: 0:00:00.057079 2026-06-10 10:21:36.119247 | 2026-06-10 10:21:36.119389 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-10 06:21:38.639858 | controller | Defaulted container "zuul-executor" out of: zuul-executor, zuul-executor-nodeexporter 2026-06-10 10:21:38.910570 | controller | ok: Runtime: 0:00:02.054046 2026-06-10 10:21:38.922490 | 2026-06-10 10:21:38.922654 | TASK [health-check/scale-resources : Scale on sts zuul-executor] 2026-06-10 06:21:39.781494 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 10:21:39.964787 | controller | ok: Runtime: 0:00:00.068036 2026-06-10 10:21:39.976201 | 2026-06-10 10:21:39.976496 | TASK [health-check/scale-resources : Get replica count for zuul-executor] 2026-06-10 10:22:03.071523 | controller | ok: Runtime: 0:00:00.056719 2026-06-10 10:22:03.077062 | 2026-06-10 10:22:03.077143 | TASK [health-check/scale-resources : Get initial replica count for zuul-merger] 2026-06-10 10:22:03.643335 | controller | ok: Runtime: 0:00:00.055637 2026-06-10 10:22:03.665618 | 2026-06-10 10:22:03.665912 | TASK [health-check/scale-resources : Collect sts log before executing the scale command] 2026-06-10 06:22:03.997037 | controller | Defaulted container "zuul-merger" out of: zuul-merger, zuul-merger-nodeexporter 2026-06-10 10:22:04.204026 | controller | ok: Runtime: 0:00:00.066892 2026-06-10 10:22:04.212403 | 2026-06-10 10:22:04.212554 | TASK [health-check/scale-resources : Scale on sts zuul-merger] 2026-06-10 06:22:04.562397 | controller | statefulset.apps/zuul-merger scaled 2026-06-10 10:22:04.823217 | controller | ok: Runtime: 0:00:00.061629 2026-06-10 10:22:04.833055 | 2026-06-10 10:22:04.833181 | TASK [health-check/scale-resources : Get replica count for zuul-merger] 2026-06-10 10:22:25.888214 | controller | ok: Runtime: 0:00:00.057242 2026-06-10 10:22:25.904639 | 2026-06-10 10:22:25.904788 | LOOP [health-check/zuul-components : Check zuul components sequentially] 2026-06-10 10:22:25.939479 | controller | ok: 2026-06-10 10:22:25.939912 | controller | { 2026-06-10 10:22:25.939971 | controller | "component": "web", 2026-06-10 10:22:25.940013 | controller | "kind": "deployment" 2026-06-10 10:22:25.940048 | controller | } 2026-06-10 10:22:25.949716 | controller | ok: 2026-06-10 10:22:25.949815 | controller | { 2026-06-10 10:22:25.949864 | controller | "component": "scheduler", 2026-06-10 10:22:25.949901 | controller | "kind": "statefulset" 2026-06-10 10:22:25.949935 | controller | } 2026-06-10 10:22:25.956623 | controller | ok: 2026-06-10 10:22:25.956718 | controller | { 2026-06-10 10:22:25.956766 | controller | "component": "merger", 2026-06-10 10:22:25.956834 | controller | "kind": "statefulset" 2026-06-10 10:22:25.956869 | controller | } 2026-06-10 10:22:25.963491 | controller | ok: 2026-06-10 10:22:25.963594 | controller | { 2026-06-10 10:22:25.963646 | controller | "component": "executor", 2026-06-10 10:22:25.963684 | controller | "kind": "statefulset" 2026-06-10 10:22:25.963720 | controller | } 2026-06-10 10:22:25.987117 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/zuul-components/tasks/check-component.yaml 2026-06-10 10:22:25.992158 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/zuul-components/tasks/check-component.yaml 2026-06-10 10:22:25.996966 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/zuul-components/tasks/check-component.yaml 2026-06-10 10:22:26.001991 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/zuul-components/tasks/check-component.yaml 2026-06-10 10:22:26.012561 | 2026-06-10 10:22:26.012731 | TASK [health-check/zuul-components : Get expected replica count for zuul-web] 2026-06-10 10:22:26.737344 | controller | ok: Runtime: 0:00:00.056085 2026-06-10 10:22:26.748646 | 2026-06-10 10:22:26.748781 | TASK [health-check/zuul-components : Wait until all of zuul-web replicas are ready] 2026-06-10 10:22:28.301298 | controller | ok: Runtime: 0:00:00.055652 2026-06-10 10:22:28.314983 | 2026-06-10 10:22:28.315206 | TASK [health-check/zuul-components : Validate zuul-web Component(s)] 2026-06-10 10:22:28.901501 | controller | ok: OK (799 bytes) 2026-06-10 10:22:28.909846 | 2026-06-10 10:22:28.909958 | TASK [health-check/zuul-components : Get expected replica count for zuul-scheduler] 2026-06-10 10:22:29.442161 | controller | ok: Runtime: 0:00:00.060805 2026-06-10 10:22:29.447745 | 2026-06-10 10:22:29.447822 | TASK [health-check/zuul-components : Wait until all of zuul-scheduler replicas are ready] 2026-06-10 10:22:29.976662 | controller | ok: Runtime: 0:00:00.053567 2026-06-10 10:22:29.988419 | 2026-06-10 10:22:29.988662 | TASK [health-check/zuul-components : Validate zuul-scheduler Component(s)] 2026-06-10 10:22:30.551287 | controller | ok: OK (799 bytes) 2026-06-10 10:22:30.559735 | 2026-06-10 10:22:30.559830 | TASK [health-check/zuul-components : Get expected replica count for zuul-merger] 2026-06-10 10:22:31.096487 | controller | ok: Runtime: 0:00:00.050189 2026-06-10 10:22:31.107468 | 2026-06-10 10:22:31.107638 | TASK [health-check/zuul-components : Wait until all of zuul-merger replicas are ready] 2026-06-10 10:22:31.654600 | controller | ok: Runtime: 0:00:00.050453 2026-06-10 10:22:31.665822 | 2026-06-10 10:22:31.665960 | TASK [health-check/zuul-components : Validate zuul-merger Component(s)] 2026-06-10 10:22:32.320528 | controller | ok: OK (799 bytes) 2026-06-10 10:22:32.334276 | 2026-06-10 10:22:32.334501 | TASK [health-check/zuul-components : Get expected replica count for zuul-executor] 2026-06-10 10:22:32.880613 | controller | ok: Runtime: 0:00:00.056119 2026-06-10 10:22:32.952679 | 2026-06-10 10:22:32.952832 | TASK [health-check/zuul-components : Wait until all of zuul-executor replicas are ready] 2026-06-10 10:22:33.479485 | controller | ok: Runtime: 0:00:00.053620 2026-06-10 10:22:33.493645 | 2026-06-10 10:22:33.493780 | TASK [health-check/zuul-components : Validate zuul-executor Component(s)] 2026-06-10 10:22:34.066823 | controller | ok: OK (799 bytes) 2026-06-10 10:22:34.084487 | 2026-06-10 10:22:34.084633 | TASK [health-check/zuul-connections : Setting Dummy Connections] 2026-06-10 10:22:34.120510 | controller | ok 2026-06-10 10:22:34.131410 | 2026-06-10 10:22:34.131581 | TASK [health-check/zuul-connections : Create SMTP Connection Secret] 2026-06-10 10:22:35.350413 | controller | changed 2026-06-10 10:22:35.361649 | 2026-06-10 10:22:35.361788 | TASK [health-check/zuul-connections : Create ElasticSearch Connection Secret] 2026-06-10 10:22:36.278926 | controller | changed 2026-06-10 10:22:36.291337 | 2026-06-10 10:22:36.291606 | TASK [health-check/zuul-connections : Create Gerrit Connection Secret] 2026-06-10 10:22:37.119338 | controller | changed 2026-06-10 10:22:37.129638 | 2026-06-10 10:22:37.129781 | TASK [health-check/zuul-connections : Create GitHub Connection Secret] 2026-06-10 10:22:37.945947 | controller | changed 2026-06-10 10:22:37.958908 | 2026-06-10 10:22:37.959073 | TASK [health-check/zuul-connections : Create GitLab Connection Secret] 2026-06-10 10:22:38.850163 | controller | changed 2026-06-10 10:22:38.862932 | 2026-06-10 10:22:38.863153 | TASK [health-check/zuul-connections : Create Pagure Connection Secret] 2026-06-10 10:22:39.726490 | controller | changed 2026-06-10 10:22:39.733255 | 2026-06-10 10:22:39.733348 | TASK [health-check/zuul-connections : set_fact] 2026-06-10 10:22:39.765619 | controller | ok 2026-06-10 10:22:39.769911 | 2026-06-10 10:22:39.769977 | TASK [Add dummy Zuul connections] 2026-06-10 10:22:39.801699 | controller | ok 2026-06-10 10:22:39.818894 | 2026-06-10 10:22:39.819050 | TASK [update-custom-resource : read current crd] 2026-06-10 10:22:40.083199 | controller | ok 2026-06-10 10:22:40.089684 | 2026-06-10 10:22:40.089777 | TASK [update-custom-resource : merge the new cr] 2026-06-10 10:22:40.767583 | controller | changed 2026-06-10 10:22:40.779006 | 2026-06-10 10:22:40.779147 | TASK [run the operator in standalone mode] 2026-06-10 10:22:40.821278 | controller | ok 2026-06-10 10:22:40.845960 | 2026-06-10 10:22:40.846050 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 10:22:40.871919 | controller | skipping: Conditional result was False 2026-06-10 10:22:40.879168 | 2026-06-10 10:22:40.879267 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 10:22:40.904842 | controller | skipping: Conditional result was False 2026-06-10 10:22:40.912028 | 2026-06-10 10:22:40.912123 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 06:22:41.504926 | controller | 2026-06-10T06:22:41-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:22:41.505040 | controller | 2026-06-10 06:22:41.514143 | controller | 2026-06-10T06:22:41-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:22:41.514178 | controller | 2026-06-10 06:22:41.528875 | controller | 2026-06-10T06:22:41-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:22:41.553396 | controller | 2026-06-10T06:22:41-04:00 INFO Updating configmap, name: git-server-pi-config-map 2026-06-10 06:22:41.553459 | controller | 2026-06-10T06:22:41-04:00 INFO Updating object, name: git-server-pi-config-map 2026-06-10 06:22:41.561689 | controller | 2026-06-10T06:22:41-04:00 INFO Statefulset "git-server" configuration changed, applying... 2026-06-10 06:22:41.561750 | controller | 2026-06-10T06:22:41-04:00 INFO Updating object, name: git-server 2026-06-10 06:22:42.936593 | controller | 2026-06-10T06:22:42-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:22:43.032042 | controller | 2026-06-10T06:22:43-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:22:43.039236 | controller | 2026-06-10T06:22:43-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:22:43.039338 | controller | 2026-06-10T06:22:43-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:22:43.055261 | controller | 2026-06-10T06:22:43-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:22:43.055351 | controller | 2026-06-10T06:22:43-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:22:43.337348 | controller | 2026-06-10T06:22:43-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-10 06:22:43.337398 | controller | 2026-06-10T06:22:43-04:00 INFO 2026-06-10 06:22:43.337407 | controller | - Config: Waiting ... 2026-06-10 06:22:43.337414 | controller | - Gateway: OK 2026-06-10 06:22:43.337420 | controller | - GitServer: Waiting ... 2026-06-10 06:22:43.337425 | controller | - HoundSearch: OK 2026-06-10 06:22:43.337432 | controller | - LogJuicer: OK 2026-06-10 06:22:43.337437 | controller | - Logserver: OK 2026-06-10 06:22:43.337442 | controller | - MariaDB: OK 2026-06-10 06:22:43.337447 | controller | - Zookeeper: OK 2026-06-10 06:22:43.337452 | controller | 2026-06-10 06:22:43.337457 | controller | 2026-06-10T06:22:43-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 06:22:48.345020 | controller | 2026-06-10T06:22:48-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:22:48.366591 | controller | 2026-06-10T06:22:48-04:00 INFO Waiting for statefulset, name: git-server 2026-06-10 06:23:06.457093 | controller | 2026-06-10T06:23:06-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:23:06.457142 | controller | 2026-06-10T06:23:06-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:23:06.469838 | controller | 2026-06-10T06:23:06-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:23:06.469895 | controller | 2026-06-10T06:23:06-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:23:06.482637 | controller | 2026-06-10T06:23:06-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:23:06.841391 | controller | 2026-06-10T06:23:06-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:23:06.841456 | controller | 2026-06-10T06:23:06-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:23:07.435905 | controller | 2026-06-10T06:23:07-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 06:23:07.438635 | controller | 2026-06-10T06:23:07-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 06:23:09.821450 | controller | 2026-06-10T06:23:09-04:00 INFO Updating secret, name: zuul-config 2026-06-10 06:23:09.821523 | controller | 2026-06-10T06:23:09-04:00 INFO Updating object, name: zuul-config 2026-06-10 06:23:10.035459 | controller | 2026-06-10T06:23:10-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-10 06:23:10.035528 | controller | 2026-06-10T06:23:10-04:00 INFO Updating object, name: zuul-scheduler 2026-06-10 06:23:10.055913 | controller | 2026-06-10T06:23:10-04:00 INFO Statefulset "zuul-executor" configuration changed, applying... 2026-06-10 06:23:10.055977 | controller | 2026-06-10T06:23:10-04:00 INFO Updating object, name: zuul-executor 2026-06-10 06:23:10.428884 | controller | 2026-06-10T06:23:10-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-10 06:23:10.428977 | controller | 2026-06-10T06:23:10-04:00 INFO Updating object, name: zuul-web 2026-06-10 06:23:11.231980 | controller | 2026-06-10T06:23:11-04:00 INFO Statefulset "zuul-merger" configuration changed, applying... 2026-06-10 06:23:11.232019 | controller | 2026-06-10T06:23:11-04:00 INFO Updating object, name: zuul-merger 2026-06-10 06:23:11.239292 | controller | 2026-06-10T06:23:11-04:00 INFO 2026-06-10 06:23:11.239352 | controller | - Config: Waiting ... 2026-06-10 06:23:11.239357 | controller | - Gateway: OK 2026-06-10 06:23:11.239361 | controller | - GitServer: OK 2026-06-10 06:23:11.239366 | controller | - HoundSearch: OK 2026-06-10 06:23:11.239370 | controller | - LogJuicer: OK 2026-06-10 06:23:11.239373 | controller | - Logserver: OK 2026-06-10 06:23:11.239377 | controller | - MariaDB: OK 2026-06-10 06:23:11.239380 | controller | - NodePoolBuilder: OK 2026-06-10 06:23:11.239384 | controller | - NodePoolLauncher: OK 2026-06-10 06:23:11.239389 | controller | - Zookeeper: OK 2026-06-10 06:23:11.239392 | controller | - Zuul: Waiting ... 2026-06-10 06:23:11.239396 | controller | - |_ Config Secrets: OK 2026-06-10 06:23:11.239400 | controller | - |_ Executor: Waiting ... 2026-06-10 06:23:11.239403 | controller | - |_ Merger: Waiting ... 2026-06-10 06:23:11.239407 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:23:11.239410 | controller | - |_ Web: Waiting ... 2026-06-10 06:23:11.239414 | controller | 2026-06-10 06:23:11.239424 | controller | 2026-06-10T06:23:11-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 06:23:16.246414 | controller | 2026-06-10T06:23:16-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:23:19.194464 | controller | 2026-06-10T06:23:19-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:23:19.194520 | controller | 2026-06-10T06:23:19-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:23:19.208804 | controller | 2026-06-10T06:23:19-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:23:19.208881 | controller | 2026-06-10T06:23:19-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:23:19.232052 | controller | 2026-06-10T06:23:19-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:23:19.248824 | controller | 2026-06-10T06:23:19-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:23:19.248878 | controller | 2026-06-10T06:23:19-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:23:20.945384 | controller | I0610 06:23:20.945099 99888 request.go:697] Waited for 1.385723625s due to client-side throttling, not priority and fairness, request: GET:https://192.168.49.2:8443/api/v1/namespaces/sf/configmaps/corporate-ca-certs 2026-06-10 06:23:20.971256 | controller | 2026-06-10T06:23:20-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 06:23:20.978123 | controller | 2026-06-10T06:23:20-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 06:23:22.026138 | controller | 2026-06-10T06:23:22-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:23:53.316976 | controller | 2026-06-10T06:23:53-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 06:23:53.331202 | controller | 2026-06-10T06:23:53-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-10 06:24:00.023016 | controller | 2026-06-10T06:24:00-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:24:00.023097 | controller | 2026-06-10T06:24:00-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:24:00.044441 | controller | 2026-06-10T06:24:00-04:00 INFO 2026-06-10 06:24:00.044499 | controller | - Config: Waiting ... 2026-06-10 06:24:00.044504 | controller | - Gateway: OK 2026-06-10 06:24:00.044509 | controller | - GitServer: OK 2026-06-10 06:24:00.044513 | controller | - HoundSearch: OK 2026-06-10 06:24:00.044516 | controller | - LogJuicer: OK 2026-06-10 06:24:00.044520 | controller | - Logserver: OK 2026-06-10 06:24:00.044523 | controller | - MariaDB: OK 2026-06-10 06:24:00.044527 | controller | - NodePoolBuilder: OK 2026-06-10 06:24:00.044530 | controller | - NodePoolLauncher: OK 2026-06-10 06:24:00.044534 | controller | - Zookeeper: OK 2026-06-10 06:24:00.044537 | controller | - Zuul: Waiting ... 2026-06-10 06:24:00.044541 | controller | - |_ Config Secrets: OK 2026-06-10 06:24:00.044544 | controller | - |_ Executor: OK 2026-06-10 06:24:00.044548 | controller | - |_ Merger: OK 2026-06-10 06:24:00.044560 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:24:00.044564 | controller | - |_ Web: Waiting ... 2026-06-10 06:24:00.044568 | controller | 2026-06-10 06:24:00.044577 | controller | 2026-06-10T06:24:00-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 06:24:05.049979 | controller | 2026-06-10T06:24:05-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:24:06.463203 | controller | 2026-06-10T06:24:06-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:24:06.463265 | controller | 2026-06-10T06:24:06-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:24:06.483839 | controller | 2026-06-10T06:24:06-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:24:06.483902 | controller | 2026-06-10T06:24:06-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:24:06.861688 | controller | 2026-06-10T06:24:06-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:24:07.258856 | controller | 2026-06-10T06:24:07-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:24:07.258911 | controller | 2026-06-10T06:24:07-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:24:07.460325 | controller | 2026-06-10T06:24:07-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 06:24:07.847740 | controller | 2026-06-10T06:24:07-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 06:24:10.265727 | controller | 2026-06-10T06:24:10-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:24:14.353768 | controller | 2026-06-10T06:24:14-04:00 INFO Statefulset "zuul-weeder" configuration changed, applying... 2026-06-10 06:24:14.353876 | controller | 2026-06-10T06:24:14-04:00 INFO Updating object, name: zuul-weeder 2026-06-10 06:24:14.353884 | controller | 2026-06-10T06:24:14-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:24:14.353890 | controller | 2026-06-10T06:24:14-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:24:14.697638 | controller | 2026-06-10T06:24:14-04:00 INFO 2026-06-10 06:24:14.697700 | controller | - Config: Waiting ... 2026-06-10 06:24:14.697707 | controller | - Gateway: OK 2026-06-10 06:24:14.697712 | controller | - GitServer: OK 2026-06-10 06:24:14.697717 | controller | - HoundSearch: OK 2026-06-10 06:24:14.697722 | controller | - LogJuicer: OK 2026-06-10 06:24:14.697726 | controller | - Logserver: OK 2026-06-10 06:24:14.697731 | controller | - MariaDB: OK 2026-06-10 06:24:14.697736 | controller | - NodePoolBuilder: OK 2026-06-10 06:24:14.697740 | controller | - NodePoolLauncher: OK 2026-06-10 06:24:14.697745 | controller | - Zookeeper: OK 2026-06-10 06:24:14.697749 | controller | - Zuul: Waiting ... 2026-06-10 06:24:14.697753 | controller | - |_ Config Secrets: OK 2026-06-10 06:24:14.697757 | controller | - |_ Executor: OK 2026-06-10 06:24:14.697760 | controller | - |_ Merger: OK 2026-06-10 06:24:14.697764 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:24:14.697768 | controller | - |_ Web: OK 2026-06-10 06:24:14.697772 | controller | 2026-06-10 06:24:14.697785 | controller | 2026-06-10T06:24:14-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-10 06:24:19.702269 | controller | 2026-06-10T06:24:19-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:24:21.116545 | controller | 2026-06-10T06:24:21-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:24:21.116638 | controller | 2026-06-10T06:24:21-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:24:21.134407 | controller | 2026-06-10T06:24:21-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:24:21.134467 | controller | 2026-06-10T06:24:21-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:24:21.513748 | controller | 2026-06-10T06:24:21-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:24:21.919970 | controller | 2026-06-10T06:24:21-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:24:21.920089 | controller | 2026-06-10T06:24:21-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:24:22.114674 | controller | 2026-06-10T06:24:22-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 06:24:22.501147 | controller | 2026-06-10T06:24:22-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 06:24:24.927753 | controller | 2026-06-10T06:24:24-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:24:24.927815 | controller | 2026-06-10T06:24:24-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:24:25.311326 | controller | 2026-06-10T06:24:25-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:24:25.311377 | controller | 2026-06-10T06:24:25-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:24:26.518133 | controller | 2026-06-10T06:24:26-04:00 INFO Running tenant-reconfigure for the 'internal' tenant 2026-06-10 06:24:26.708924 | controller | 2026-06-10T06:24:26-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [zuul-scheduler tenant-reconfigure internal] 2026-06-10 06:24:31.370606 | controller | 2026-06-10T06:24:31-04:00 INFO Deleting object, name: zuul-web 2026-06-10 06:24:31.373987 | controller | 2026-06-10T06:24:31-04:00 INFO Updating object, name: zs-internal-tenant-reconfigure 2026-06-10 06:24:31.377489 | controller | 2026-06-10T06:24:31-04:00 INFO 2026-06-10 06:24:31.377521 | controller | - Config: Waiting ... 2026-06-10 06:24:31.377527 | controller | - Gateway: OK 2026-06-10 06:24:31.377532 | controller | - GitServer: OK 2026-06-10 06:24:31.377537 | controller | - HoundSearch: OK 2026-06-10 06:24:31.377542 | controller | - LogJuicer: OK 2026-06-10 06:24:31.377547 | controller | - Logserver: OK 2026-06-10 06:24:31.377551 | controller | - MariaDB: OK 2026-06-10 06:24:31.377555 | controller | - NodePoolBuilder: OK 2026-06-10 06:24:31.377560 | controller | - NodePoolLauncher: OK 2026-06-10 06:24:31.377565 | controller | - Zookeeper: OK 2026-06-10 06:24:31.377570 | controller | - Zuul: OK 2026-06-10 06:24:31.377575 | controller | 2026-06-10 06:24:31.377580 | controller | 2026-06-10T06:24:31-04:00 INFO [attempt #5] Waiting 5s for the next reconcile call ... 2026-06-10 06:24:36.387342 | controller | 2026-06-10T06:24:36-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:24:37.797572 | controller | 2026-06-10T06:24:37-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:24:37.797663 | controller | 2026-06-10T06:24:37-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:24:37.813301 | controller | 2026-06-10T06:24:37-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:24:37.813361 | controller | 2026-06-10T06:24:37-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:24:38.197984 | controller | 2026-06-10T06:24:38-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:24:38.595834 | controller | 2026-06-10T06:24:38-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:24:38.595883 | controller | 2026-06-10T06:24:38-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:24:38.798593 | controller | 2026-06-10T06:24:38-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 06:24:39.184705 | controller | 2026-06-10T06:24:39-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 06:24:41.610103 | controller | 2026-06-10T06:24:41-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:24:41.610168 | controller | 2026-06-10T06:24:41-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:24:41.995551 | controller | 2026-06-10T06:24:41-04:00 INFO Creating object, name: zuul-web 2026-06-10 06:24:42.803934 | controller | 2026-06-10T06:24:42-04:00 INFO 2026-06-10 06:24:42.803980 | controller | - Config: Waiting ... 2026-06-10 06:24:42.803985 | controller | - Gateway: OK 2026-06-10 06:24:42.803989 | controller | - GitServer: OK 2026-06-10 06:24:42.803993 | controller | - HoundSearch: OK 2026-06-10 06:24:42.803997 | controller | - LogJuicer: OK 2026-06-10 06:24:42.804002 | controller | - Logserver: OK 2026-06-10 06:24:42.804007 | controller | - MariaDB: OK 2026-06-10 06:24:42.804012 | controller | - NodePoolBuilder: OK 2026-06-10 06:24:42.804018 | controller | - NodePoolLauncher: OK 2026-06-10 06:24:42.804023 | controller | - Zookeeper: OK 2026-06-10 06:24:42.804028 | controller | - Zuul: Waiting ... 2026-06-10 06:24:42.804032 | controller | - |_ Config Secrets: OK 2026-06-10 06:24:42.804035 | controller | - |_ Executor: OK 2026-06-10 06:24:42.804038 | controller | - |_ Merger: OK 2026-06-10 06:24:42.804042 | controller | - |_ Scheduler: OK 2026-06-10 06:24:42.804045 | controller | - |_ Web: Waiting ... 2026-06-10 06:24:42.804049 | controller | 2026-06-10 06:24:42.804053 | controller | 2026-06-10T06:24:42-04:00 INFO [attempt #6] Waiting 5s for the next reconcile call ... 2026-06-10 06:24:47.808715 | controller | 2026-06-10T06:24:47-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:24:49.220670 | controller | 2026-06-10T06:24:49-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:24:49.220710 | controller | 2026-06-10T06:24:49-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:24:49.233500 | controller | 2026-06-10T06:24:49-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:24:49.233537 | controller | 2026-06-10T06:24:49-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:24:49.621108 | controller | 2026-06-10T06:24:49-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:24:50.020504 | controller | 2026-06-10T06:24:50-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:24:50.020576 | controller | 2026-06-10T06:24:50-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:24:50.224824 | controller | 2026-06-10T06:24:50-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 06:24:50.607394 | controller | 2026-06-10T06:24:50-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 06:24:53.027131 | controller | 2026-06-10T06:24:53-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:24:53.027170 | controller | 2026-06-10T06:24:53-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:24:53.416416 | controller | 2026-06-10T06:24:53-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:24:53.416525 | controller | 2026-06-10T06:24:53-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:24:54.223663 | controller | 2026-06-10T06:24:54-04:00 INFO 2026-06-10 06:24:54.223743 | controller | - Config: Waiting ... 2026-06-10 06:24:54.223752 | controller | - Gateway: OK 2026-06-10 06:24:54.223758 | controller | - GitServer: OK 2026-06-10 06:24:54.223764 | controller | - HoundSearch: OK 2026-06-10 06:24:54.223769 | controller | - LogJuicer: OK 2026-06-10 06:24:54.223775 | controller | - Logserver: OK 2026-06-10 06:24:54.223780 | controller | - MariaDB: OK 2026-06-10 06:24:54.223785 | controller | - NodePoolBuilder: OK 2026-06-10 06:24:54.223790 | controller | - NodePoolLauncher: OK 2026-06-10 06:24:54.223796 | controller | - Zookeeper: OK 2026-06-10 06:24:54.223801 | controller | - Zuul: Waiting ... 2026-06-10 06:24:54.223806 | controller | - |_ Config Secrets: OK 2026-06-10 06:24:54.223811 | controller | - |_ Executor: OK 2026-06-10 06:24:54.223817 | controller | - |_ Merger: OK 2026-06-10 06:24:54.223822 | controller | - |_ Scheduler: OK 2026-06-10 06:24:54.223827 | controller | - |_ Web: Waiting ... 2026-06-10 06:24:54.223832 | controller | 2026-06-10 06:24:54.223847 | controller | 2026-06-10T06:24:54-04:00 INFO [attempt #7] Waiting 5s for the next reconcile call ... 2026-06-10 06:24:59.227958 | controller | 2026-06-10T06:24:59-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:25:00.640853 | controller | 2026-06-10T06:25:00-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:25:00.640927 | controller | 2026-06-10T06:25:00-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:25:00.653772 | controller | 2026-06-10T06:25:00-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:25:00.654341 | controller | 2026-06-10T06:25:00-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:25:01.045510 | controller | 2026-06-10T06:25:01-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:25:01.441480 | controller | 2026-06-10T06:25:01-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:25:01.441554 | controller | 2026-06-10T06:25:01-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:25:01.639503 | controller | 2026-06-10T06:25:01-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 06:25:02.026411 | controller | 2026-06-10T06:25:02-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 06:25:04.456966 | controller | 2026-06-10T06:25:04-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:25:04.457057 | controller | 2026-06-10T06:25:04-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:25:04.840004 | controller | 2026-06-10T06:25:04-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:25:04.840081 | controller | 2026-06-10T06:25:04-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:25:05.641237 | controller | 2026-06-10T06:25:05-04:00 INFO 2026-06-10 06:25:05.641300 | controller | - Config: Waiting ... 2026-06-10 06:25:05.641306 | controller | - Gateway: OK 2026-06-10 06:25:05.641312 | controller | - GitServer: OK 2026-06-10 06:25:05.641317 | controller | - HoundSearch: OK 2026-06-10 06:25:05.641321 | controller | - LogJuicer: OK 2026-06-10 06:25:05.641326 | controller | - Logserver: OK 2026-06-10 06:25:05.641331 | controller | - MariaDB: OK 2026-06-10 06:25:05.641336 | controller | - NodePoolBuilder: OK 2026-06-10 06:25:05.641341 | controller | - NodePoolLauncher: OK 2026-06-10 06:25:05.641346 | controller | - Zookeeper: OK 2026-06-10 06:25:05.641350 | controller | - Zuul: Waiting ... 2026-06-10 06:25:05.641355 | controller | - |_ Config Secrets: OK 2026-06-10 06:25:05.641360 | controller | - |_ Executor: OK 2026-06-10 06:25:05.641365 | controller | - |_ Merger: OK 2026-06-10 06:25:05.641370 | controller | - |_ Scheduler: OK 2026-06-10 06:25:05.641375 | controller | - |_ Web: Waiting ... 2026-06-10 06:25:05.641379 | controller | 2026-06-10 06:25:05.641389 | controller | 2026-06-10T06:25:05-04:00 INFO [attempt #8] Waiting 5s for the next reconcile call ... 2026-06-10 06:25:10.645464 | controller | 2026-06-10T06:25:10-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:25:12.061008 | controller | 2026-06-10T06:25:12-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:25:12.061049 | controller | 2026-06-10T06:25:12-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:25:12.080283 | controller | 2026-06-10T06:25:12-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:25:12.080349 | controller | 2026-06-10T06:25:12-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:25:12.465295 | controller | 2026-06-10T06:25:12-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:25:12.867720 | controller | 2026-06-10T06:25:12-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:25:12.867762 | controller | 2026-06-10T06:25:12-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:25:13.065030 | controller | 2026-06-10T06:25:13-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 06:25:13.444189 | controller | 2026-06-10T06:25:13-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 06:25:15.865091 | controller | 2026-06-10T06:25:15-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:25:15.865154 | controller | 2026-06-10T06:25:15-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:25:16.253177 | controller | 2026-06-10T06:25:16-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:25:16.253265 | controller | 2026-06-10T06:25:16-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:25:17.060943 | controller | 2026-06-10T06:25:17-04:00 INFO 2026-06-10 06:25:17.061010 | controller | - Config: Waiting ... 2026-06-10 06:25:17.061015 | controller | - Gateway: OK 2026-06-10 06:25:17.061019 | controller | - GitServer: OK 2026-06-10 06:25:17.061023 | controller | - HoundSearch: OK 2026-06-10 06:25:17.061026 | controller | - LogJuicer: OK 2026-06-10 06:25:17.061030 | controller | - Logserver: OK 2026-06-10 06:25:17.061033 | controller | - MariaDB: OK 2026-06-10 06:25:17.061036 | controller | - NodePoolBuilder: OK 2026-06-10 06:25:17.061040 | controller | - NodePoolLauncher: OK 2026-06-10 06:25:17.061043 | controller | - Zookeeper: OK 2026-06-10 06:25:17.061047 | controller | - Zuul: Waiting ... 2026-06-10 06:25:17.061050 | controller | - |_ Config Secrets: OK 2026-06-10 06:25:17.061054 | controller | - |_ Executor: OK 2026-06-10 06:25:17.061057 | controller | - |_ Merger: OK 2026-06-10 06:25:17.061061 | controller | - |_ Scheduler: OK 2026-06-10 06:25:17.061064 | controller | - |_ Web: Waiting ... 2026-06-10 06:25:17.061068 | controller | 2026-06-10 06:25:17.061078 | controller | 2026-06-10T06:25:17-04:00 INFO [attempt #9] Waiting 5s for the next reconcile call ... 2026-06-10 06:25:22.069234 | controller | 2026-06-10T06:25:22-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:25:23.481416 | controller | 2026-06-10T06:25:23-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:25:23.481489 | controller | 2026-06-10T06:25:23-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:25:23.497137 | controller | 2026-06-10T06:25:23-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:25:23.497201 | controller | 2026-06-10T06:25:23-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:25:23.884362 | controller | 2026-06-10T06:25:23-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:25:24.280969 | controller | 2026-06-10T06:25:24-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:25:24.281031 | controller | 2026-06-10T06:25:24-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:25:24.481475 | controller | 2026-06-10T06:25:24-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 06:25:24.867534 | controller | 2026-06-10T06:25:24-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 06:25:27.288823 | controller | 2026-06-10T06:25:27-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:25:27.288884 | controller | 2026-06-10T06:25:27-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:25:27.674559 | controller | 2026-06-10T06:25:27-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:25:27.674603 | controller | 2026-06-10T06:25:27-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:25:28.482725 | controller | 2026-06-10T06:25:28-04:00 INFO 2026-06-10 06:25:28.482824 | controller | - Config: Waiting ... 2026-06-10 06:25:28.482831 | controller | - Gateway: OK 2026-06-10 06:25:28.482835 | controller | - GitServer: OK 2026-06-10 06:25:28.482838 | controller | - HoundSearch: OK 2026-06-10 06:25:28.482842 | controller | - LogJuicer: OK 2026-06-10 06:25:28.482845 | controller | - Logserver: OK 2026-06-10 06:25:28.482849 | controller | - MariaDB: OK 2026-06-10 06:25:28.482852 | controller | - NodePoolBuilder: OK 2026-06-10 06:25:28.482864 | controller | - NodePoolLauncher: OK 2026-06-10 06:25:28.482868 | controller | - Zookeeper: OK 2026-06-10 06:25:28.482871 | controller | - Zuul: Waiting ... 2026-06-10 06:25:28.482875 | controller | - |_ Config Secrets: OK 2026-06-10 06:25:28.482878 | controller | - |_ Executor: OK 2026-06-10 06:25:28.482882 | controller | - |_ Merger: OK 2026-06-10 06:25:28.482885 | controller | - |_ Scheduler: OK 2026-06-10 06:25:28.482938 | controller | - |_ Web: Waiting ... 2026-06-10 06:25:28.482943 | controller | 2026-06-10 06:25:28.482948 | controller | 2026-06-10T06:25:28-04:00 INFO [attempt #10] Waiting 5s for the next reconcile call ... 2026-06-10 06:25:33.490713 | controller | 2026-06-10T06:25:33-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:25:34.900535 | controller | 2026-06-10T06:25:34-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:25:34.900620 | controller | 2026-06-10T06:25:34-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:25:34.913454 | controller | 2026-06-10T06:25:34-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:25:34.913504 | controller | 2026-06-10T06:25:34-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:25:35.300225 | controller | 2026-06-10T06:25:35-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:25:35.703957 | controller | 2026-06-10T06:25:35-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:25:35.704023 | controller | 2026-06-10T06:25:35-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:25:35.905787 | controller | 2026-06-10T06:25:35-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 06:25:36.290120 | controller | 2026-06-10T06:25:36-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 06:25:38.714015 | controller | 2026-06-10T06:25:38-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:25:38.714062 | controller | 2026-06-10T06:25:38-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:25:39.099460 | controller | 2026-06-10T06:25:39-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:25:39.099537 | controller | 2026-06-10T06:25:39-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:25:39.912481 | controller | 2026-06-10T06:25:39-04:00 INFO 2026-06-10 06:25:39.912550 | controller | - Config: Waiting ... 2026-06-10 06:25:39.912558 | controller | - Gateway: OK 2026-06-10 06:25:39.912564 | controller | - GitServer: OK 2026-06-10 06:25:39.912570 | controller | - HoundSearch: OK 2026-06-10 06:25:39.912575 | controller | - LogJuicer: OK 2026-06-10 06:25:39.912581 | controller | - Logserver: OK 2026-06-10 06:25:39.912587 | controller | - MariaDB: OK 2026-06-10 06:25:39.912592 | controller | - NodePoolBuilder: OK 2026-06-10 06:25:39.912597 | controller | - NodePoolLauncher: OK 2026-06-10 06:25:39.912644 | controller | - Zookeeper: OK 2026-06-10 06:25:39.912650 | controller | - Zuul: Waiting ... 2026-06-10 06:25:39.912656 | controller | - |_ Config Secrets: OK 2026-06-10 06:25:39.912661 | controller | - |_ Executor: OK 2026-06-10 06:25:39.912666 | controller | - |_ Merger: OK 2026-06-10 06:25:39.912672 | controller | - |_ Scheduler: OK 2026-06-10 06:25:39.912677 | controller | - |_ Web: Waiting ... 2026-06-10 06:25:39.912683 | controller | 2026-06-10 06:25:39.912689 | controller | 2026-06-10T06:25:39-04:00 INFO [attempt #11] Waiting 5s for the next reconcile call ... 2026-06-10 06:25:44.919838 | controller | 2026-06-10T06:25:44-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:25:46.330820 | controller | 2026-06-10T06:25:46-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:25:46.330881 | controller | 2026-06-10T06:25:46-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:25:46.344864 | controller | 2026-06-10T06:25:46-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:25:46.344935 | controller | 2026-06-10T06:25:46-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:25:46.733741 | controller | 2026-06-10T06:25:46-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:25:47.129037 | controller | 2026-06-10T06:25:47-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:25:47.129112 | controller | 2026-06-10T06:25:47-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:25:47.333956 | controller | 2026-06-10T06:25:47-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 06:25:47.719045 | controller | 2026-06-10T06:25:47-04:00 INFO app_key or app_id is not defined, app_key: /var/lib/zuul/githubconnectionsecret/app_key, app_id: 0 2026-06-10 06:25:50.139213 | controller | 2026-06-10T06:25:50-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:25:50.139286 | controller | 2026-06-10T06:25:50-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:25:50.526841 | controller | 2026-06-10T06:25:50-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:25:50.526903 | controller | 2026-06-10T06:25:50-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:25:51.732581 | controller | 2026-06-10T06:25:51-04:00 INFO 2026-06-10 06:25:51.732657 | controller | - Config: OK 2026-06-10 06:25:51.732663 | controller | - Gateway: OK 2026-06-10 06:25:51.732668 | controller | - GitServer: OK 2026-06-10 06:25:51.732673 | controller | - HoundSearch: OK 2026-06-10 06:25:51.732678 | controller | - LogJuicer: OK 2026-06-10 06:25:51.732682 | controller | - Logserver: OK 2026-06-10 06:25:51.732687 | controller | - MariaDB: OK 2026-06-10 06:25:51.732691 | controller | - NodePoolBuilder: OK 2026-06-10 06:25:51.732696 | controller | - NodePoolLauncher: OK 2026-06-10 06:25:51.732701 | controller | - Zookeeper: OK 2026-06-10 06:25:51.732706 | controller | - Zuul: OK 2026-06-10 06:25:51.732710 | controller | 2026-06-10 06:25:51.732716 | controller | 2026-06-10T06:25:51-04:00 INFO Updating controller configmap ... 2026-06-10 06:25:52.124877 | controller | 2026-06-10T06:25:52-04:00 INFO Standalone reconcile done. 2026-06-10 10:25:52.549109 | controller | ok: Runtime: 0:03:10.966608 2026-06-10 10:25:52.562916 | 2026-06-10 10:25:52.563049 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 10:25:53.102460 | controller | skipping: Conditional result was False 2026-06-10 10:25:53.130308 | 2026-06-10 10:25:53.130531 | TASK [health-check/zuul-connections : Wait for the new Zuul connections to appear in the Zuul API] 2026-06-10 10:25:53.608375 | controller | ok: OK (1399 bytes) 2026-06-10 10:25:53.613389 | 2026-06-10 10:25:53.613480 | TASK [health-check/zuul-connections : Ensure the new Zuul connections (non SourceInterface) exist in the scheduler's zuul.conf] 2026-06-10 06:25:53.876808 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-10 06:25:53.950834 | controller | [connection dummy-elasticsearch-conn] 2026-06-10 06:25:54.009015 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-10 06:25:54.085851 | controller | [connection dummy-smtp-conn] 2026-06-10 10:25:54.141846 | controller | ok: Runtime: 0:00:00.269639 2026-06-10 10:25:54.151639 | 2026-06-10 10:25:54.151782 | TASK [health-check/zuul-connections : Ensure ElasticSearch URI is configured with basic auth] 2026-06-10 06:25:54.472953 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-10 06:25:54.546369 | controller | uri = http://es-username:es-password@test:9200 2026-06-10 10:25:54.697719 | controller | ok: Runtime: 0:00:00.134005 2026-06-10 10:25:54.707642 | 2026-06-10 10:25:54.707791 | TASK [health-check/zuul-connections : Ensure SMTP password is present] 2026-06-10 06:25:55.032307 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-10 06:25:55.112652 | controller | password = smtp-password 2026-06-10 10:25:55.257858 | controller | ok: Runtime: 0:00:00.144824 2026-06-10 10:25:55.266626 | 2026-06-10 10:25:55.266829 | TASK [health-check/zuul-connections : Ensure the new Zuul dummy gerrit secret exist in the scheduler's zuul.conf] 2026-06-10 06:25:55.579903 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-10 06:25:55.667893 | controller | sshkey = /var/lib/zuul-dummygerritsecret/..data/priv 2026-06-10 06:25:55.667956 | controller | sshkey = /var/lib/zuul-dummygerritsecret/..data/priv 2026-06-10 06:25:55.734329 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-10 10:25:56.334659 | controller | ok: Runtime: 0:00:00.286380 2026-06-10 10:25:56.344370 | 2026-06-10 10:25:56.344556 | TASK [Delete the dummy Zuul connections] 2026-06-10 10:25:56.379841 | controller | ok 2026-06-10 10:25:56.418618 | 2026-06-10 10:25:56.418870 | TASK [update-custom-resource : read current crd] 2026-06-10 10:25:56.701054 | controller | ok 2026-06-10 10:25:56.709855 | 2026-06-10 10:25:56.709988 | TASK [update-custom-resource : merge the new cr] 2026-06-10 10:25:57.892545 | controller | changed 2026-06-10 10:25:57.902145 | 2026-06-10 10:25:57.902282 | TASK [run the operator in standalone mode] 2026-06-10 10:25:57.938571 | controller | ok 2026-06-10 10:25:57.968514 | 2026-06-10 10:25:57.968653 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 10:25:57.995047 | controller | skipping: Conditional result was False 2026-06-10 10:25:58.005494 | 2026-06-10 10:25:58.005590 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 10:25:58.032483 | controller | skipping: Conditional result was False 2026-06-10 10:25:58.040402 | 2026-06-10 10:25:58.040544 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 06:25:58.514329 | controller | 2026-06-10T06:25:58-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:25:58.514406 | controller | 2026-06-10 06:25:58.524194 | controller | 2026-06-10T06:25:58-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:25:58.524236 | controller | 2026-06-10 06:25:58.536702 | controller | 2026-06-10T06:25:58-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:25:58.546633 | controller | 2026-06-10T06:25:58-04:00 INFO Updating configmap, name: git-server-pi-config-map 2026-06-10 06:25:58.546687 | controller | 2026-06-10T06:25:58-04:00 INFO Updating object, name: git-server-pi-config-map 2026-06-10 06:25:58.554522 | controller | 2026-06-10T06:25:58-04:00 INFO Statefulset "git-server" configuration changed, applying... 2026-06-10 06:25:58.554577 | controller | 2026-06-10T06:25:58-04:00 INFO Updating object, name: git-server 2026-06-10 06:25:59.147505 | controller | 2026-06-10T06:25:59-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:25:59.230016 | controller | 2026-06-10T06:25:59-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:25:59.236496 | controller | 2026-06-10T06:25:59-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:25:59.236530 | controller | 2026-06-10T06:25:59-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:25:59.249427 | controller | 2026-06-10T06:25:59-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:25:59.249475 | controller | 2026-06-10T06:25:59-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:25:59.550023 | controller | 2026-06-10T06:25:59-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-10 06:25:59.550077 | controller | 2026-06-10T06:25:59-04:00 INFO 2026-06-10 06:25:59.550086 | controller | - Config: Waiting ... 2026-06-10 06:25:59.550092 | controller | - Gateway: OK 2026-06-10 06:25:59.550097 | controller | - GitServer: Waiting ... 2026-06-10 06:25:59.550102 | controller | - HoundSearch: OK 2026-06-10 06:25:59.550107 | controller | - LogJuicer: OK 2026-06-10 06:25:59.550112 | controller | - Logserver: OK 2026-06-10 06:25:59.550117 | controller | - MariaDB: OK 2026-06-10 06:25:59.550122 | controller | - Zookeeper: OK 2026-06-10 06:25:59.550127 | controller | 2026-06-10 06:25:59.550132 | controller | 2026-06-10T06:25:59-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 06:26:04.554053 | controller | 2026-06-10T06:26:04-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:26:04.573135 | controller | 2026-06-10T06:26:04-04:00 INFO Waiting for statefulset, name: git-server 2026-06-10 06:26:22.660396 | controller | 2026-06-10T06:26:22-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:26:22.660445 | controller | 2026-06-10T06:26:22-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:26:22.671427 | controller | 2026-06-10T06:26:22-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:26:22.671469 | controller | 2026-06-10T06:26:22-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:26:22.686765 | controller | 2026-06-10T06:26:22-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:26:23.037262 | controller | 2026-06-10T06:26:23-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:26:23.037304 | controller | 2026-06-10T06:26:23-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:26:24.027232 | controller | 2026-06-10T06:26:24-04:00 INFO Updating secret, name: zuul-config 2026-06-10 06:26:24.027289 | controller | 2026-06-10T06:26:24-04:00 INFO Updating object, name: zuul-config 2026-06-10 06:26:24.433829 | controller | 2026-06-10T06:26:24-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-10 06:26:24.433885 | controller | 2026-06-10T06:26:24-04:00 INFO Updating object, name: zuul-scheduler 2026-06-10 06:26:25.234555 | controller | 2026-06-10T06:26:25-04:00 INFO Statefulset "zuul-executor" configuration changed, applying... 2026-06-10 06:26:25.234593 | controller | 2026-06-10T06:26:25-04:00 INFO Updating object, name: zuul-executor 2026-06-10 06:26:26.033710 | controller | 2026-06-10T06:26:26-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-10 06:26:26.033785 | controller | 2026-06-10T06:26:26-04:00 INFO Updating object, name: zuul-web 2026-06-10 06:26:26.834834 | controller | 2026-06-10T06:26:26-04:00 INFO Statefulset "zuul-merger" configuration changed, applying... 2026-06-10 06:26:26.834943 | controller | 2026-06-10T06:26:26-04:00 INFO Updating object, name: zuul-merger 2026-06-10 06:26:26.840989 | controller | 2026-06-10T06:26:26-04:00 INFO 2026-06-10 06:26:26.841047 | controller | - Config: Waiting ... 2026-06-10 06:26:26.841052 | controller | - Gateway: OK 2026-06-10 06:26:26.841056 | controller | - GitServer: OK 2026-06-10 06:26:26.841060 | controller | - HoundSearch: OK 2026-06-10 06:26:26.841063 | controller | - LogJuicer: OK 2026-06-10 06:26:26.841067 | controller | - Logserver: OK 2026-06-10 06:26:26.841071 | controller | - MariaDB: OK 2026-06-10 06:26:26.841074 | controller | - NodePoolBuilder: OK 2026-06-10 06:26:26.841078 | controller | - NodePoolLauncher: OK 2026-06-10 06:26:26.841081 | controller | - Zookeeper: OK 2026-06-10 06:26:26.841085 | controller | - Zuul: Waiting ... 2026-06-10 06:26:26.841088 | controller | - |_ Config Secrets: OK 2026-06-10 06:26:26.841092 | controller | - |_ Executor: Waiting ... 2026-06-10 06:26:26.841095 | controller | - |_ Merger: Waiting ... 2026-06-10 06:26:26.841099 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:26:26.841104 | controller | - |_ Web: Waiting ... 2026-06-10 06:26:26.841108 | controller | 2026-06-10 06:26:26.841117 | controller | 2026-06-10T06:26:26-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 06:26:31.847557 | controller | 2026-06-10T06:26:31-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:26:32.457499 | controller | 2026-06-10T06:26:32-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:26:32.457736 | controller | 2026-06-10T06:26:32-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:26:32.475048 | controller | 2026-06-10T06:26:32-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:26:32.475094 | controller | 2026-06-10T06:26:32-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:26:32.861964 | controller | 2026-06-10T06:26:32-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:26:33.255212 | controller | 2026-06-10T06:26:33-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:26:33.255278 | controller | 2026-06-10T06:26:33-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:26:34.261072 | controller | 2026-06-10T06:26:34-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:27:04.481673 | controller | 2026-06-10T06:27:04-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 06:27:04.508332 | controller | 2026-06-10T06:27:04-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:27:04.508381 | controller | 2026-06-10T06:27:04-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:27:04.891201 | controller | 2026-06-10T06:27:04-04:00 INFO 2026-06-10 06:27:04.891239 | controller | - Config: Waiting ... 2026-06-10 06:27:04.891244 | controller | - Gateway: OK 2026-06-10 06:27:04.891249 | controller | - GitServer: OK 2026-06-10 06:27:04.891253 | controller | - HoundSearch: OK 2026-06-10 06:27:04.891257 | controller | - LogJuicer: OK 2026-06-10 06:27:04.891260 | controller | - Logserver: OK 2026-06-10 06:27:04.891269 | controller | - MariaDB: OK 2026-06-10 06:27:04.891273 | controller | - NodePoolBuilder: OK 2026-06-10 06:27:04.891276 | controller | - NodePoolLauncher: OK 2026-06-10 06:27:04.891280 | controller | - Zookeeper: OK 2026-06-10 06:27:04.891283 | controller | - Zuul: Waiting ... 2026-06-10 06:27:04.891287 | controller | - |_ Config Secrets: OK 2026-06-10 06:27:04.891291 | controller | - |_ Executor: OK 2026-06-10 06:27:04.891294 | controller | - |_ Merger: OK 2026-06-10 06:27:04.891298 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:27:04.891301 | controller | - |_ Web: Waiting ... 2026-06-10 06:27:04.891305 | controller | 2026-06-10 06:27:04.891309 | controller | 2026-06-10T06:27:04-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 06:27:09.899263 | controller | 2026-06-10T06:27:09-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:27:10.508520 | controller | 2026-06-10T06:27:10-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:27:10.508556 | controller | 2026-06-10T06:27:10-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:27:10.519554 | controller | 2026-06-10T06:27:10-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:27:10.519591 | controller | 2026-06-10T06:27:10-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:27:10.911233 | controller | 2026-06-10T06:27:10-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:27:11.311995 | controller | 2026-06-10T06:27:11-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:27:11.312068 | controller | 2026-06-10T06:27:11-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:27:12.316864 | controller | 2026-06-10T06:27:12-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:27:30.376901 | controller | 2026-06-10T06:27:30-04:00 INFO Statefulset "zuul-weeder" configuration changed, applying... 2026-06-10 06:27:30.376991 | controller | 2026-06-10T06:27:30-04:00 INFO Updating object, name: zuul-weeder 2026-06-10 06:27:30.417652 | controller | 2026-06-10T06:27:30-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:27:30.417691 | controller | 2026-06-10T06:27:30-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:27:30.793754 | controller | 2026-06-10T06:27:30-04:00 INFO 2026-06-10 06:27:30.793815 | controller | - Config: Waiting ... 2026-06-10 06:27:30.793820 | controller | - Gateway: OK 2026-06-10 06:27:30.793824 | controller | - GitServer: OK 2026-06-10 06:27:30.793828 | controller | - HoundSearch: OK 2026-06-10 06:27:30.793831 | controller | - LogJuicer: OK 2026-06-10 06:27:30.793835 | controller | - Logserver: OK 2026-06-10 06:27:30.793838 | controller | - MariaDB: OK 2026-06-10 06:27:30.793842 | controller | - NodePoolBuilder: OK 2026-06-10 06:27:30.793845 | controller | - NodePoolLauncher: OK 2026-06-10 06:27:30.793849 | controller | - Zookeeper: OK 2026-06-10 06:27:30.793853 | controller | - Zuul: Waiting ... 2026-06-10 06:27:30.793856 | controller | - |_ Config Secrets: OK 2026-06-10 06:27:30.793860 | controller | - |_ Executor: OK 2026-06-10 06:27:30.793863 | controller | - |_ Merger: OK 2026-06-10 06:27:30.793867 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:27:30.793870 | controller | - |_ Web: OK 2026-06-10 06:27:30.793875 | controller | 2026-06-10 06:27:30.793884 | controller | 2026-06-10T06:27:30-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-10 06:27:35.802763 | controller | 2026-06-10T06:27:35-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:27:36.417528 | controller | 2026-06-10T06:27:36-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:27:36.417567 | controller | 2026-06-10T06:27:36-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:27:36.431774 | controller | 2026-06-10T06:27:36-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:27:36.431828 | controller | 2026-06-10T06:27:36-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:27:36.812818 | controller | 2026-06-10T06:27:36-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:27:37.214781 | controller | 2026-06-10T06:27:37-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:27:37.214846 | controller | 2026-06-10T06:27:37-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:27:38.223508 | controller | 2026-06-10T06:27:38-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:27:38.223601 | controller | 2026-06-10T06:27:38-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:27:39.809496 | controller | 2026-06-10T06:27:39-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:27:39.809660 | controller | 2026-06-10T06:27:39-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:27:41.011380 | controller | 2026-06-10T06:27:41-04:00 INFO Running tenant-reconfigure for the 'internal' tenant 2026-06-10 06:27:41.204204 | controller | 2026-06-10T06:27:41-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [zuul-scheduler tenant-reconfigure internal] 2026-06-10 06:27:46.612664 | controller | 2026-06-10T06:27:46-04:00 INFO Deleting object, name: zuul-web 2026-06-10 06:27:46.616151 | controller | 2026-06-10T06:27:46-04:00 INFO Updating object, name: zs-internal-tenant-reconfigure 2026-06-10 06:27:46.618871 | controller | 2026-06-10T06:27:46-04:00 INFO 2026-06-10 06:27:46.618895 | controller | - Config: Waiting ... 2026-06-10 06:27:46.618899 | controller | - Gateway: OK 2026-06-10 06:27:46.618903 | controller | - GitServer: OK 2026-06-10 06:27:46.618907 | controller | - HoundSearch: OK 2026-06-10 06:27:46.618911 | controller | - LogJuicer: OK 2026-06-10 06:27:46.618914 | controller | - Logserver: OK 2026-06-10 06:27:46.618918 | controller | - MariaDB: OK 2026-06-10 06:27:46.618921 | controller | - NodePoolBuilder: OK 2026-06-10 06:27:46.618925 | controller | - NodePoolLauncher: OK 2026-06-10 06:27:46.618929 | controller | - Zookeeper: OK 2026-06-10 06:27:46.618932 | controller | - Zuul: OK 2026-06-10 06:27:46.618936 | controller | 2026-06-10 06:27:46.618964 | controller | 2026-06-10T06:27:46-04:00 INFO [attempt #5] Waiting 5s for the next reconcile call ... 2026-06-10 06:27:51.624480 | controller | 2026-06-10T06:27:51-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:27:52.241367 | controller | 2026-06-10T06:27:52-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:27:52.241452 | controller | 2026-06-10T06:27:52-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:27:52.256725 | controller | 2026-06-10T06:27:52-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:27:52.256769 | controller | 2026-06-10T06:27:52-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:27:52.643441 | controller | 2026-06-10T06:27:52-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:27:53.039037 | controller | 2026-06-10T06:27:53-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:27:53.039076 | controller | 2026-06-10T06:27:53-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:27:54.042266 | controller | 2026-06-10T06:27:54-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:27:54.042323 | controller | 2026-06-10T06:27:54-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:27:55.628492 | controller | 2026-06-10T06:27:55-04:00 INFO Creating object, name: zuul-web 2026-06-10 06:27:56.434355 | controller | 2026-06-10T06:27:56-04:00 INFO 2026-06-10 06:27:56.434394 | controller | - Config: Waiting ... 2026-06-10 06:27:56.434399 | controller | - Gateway: OK 2026-06-10 06:27:56.434403 | controller | - GitServer: OK 2026-06-10 06:27:56.434406 | controller | - HoundSearch: OK 2026-06-10 06:27:56.434410 | controller | - LogJuicer: OK 2026-06-10 06:27:56.434414 | controller | - Logserver: OK 2026-06-10 06:27:56.434417 | controller | - MariaDB: OK 2026-06-10 06:27:56.434421 | controller | - NodePoolBuilder: OK 2026-06-10 06:27:56.434424 | controller | - NodePoolLauncher: OK 2026-06-10 06:27:56.434428 | controller | - Zookeeper: OK 2026-06-10 06:27:56.434431 | controller | - Zuul: Waiting ... 2026-06-10 06:27:56.434435 | controller | - |_ Config Secrets: OK 2026-06-10 06:27:56.434438 | controller | - |_ Executor: OK 2026-06-10 06:27:56.434442 | controller | - |_ Merger: OK 2026-06-10 06:27:56.434445 | controller | - |_ Scheduler: OK 2026-06-10 06:27:56.434449 | controller | - |_ Web: Waiting ... 2026-06-10 06:27:56.434452 | controller | 2026-06-10 06:27:56.434470 | controller | 2026-06-10T06:27:56-04:00 INFO [attempt #6] Waiting 5s for the next reconcile call ... 2026-06-10 06:28:01.443157 | controller | 2026-06-10T06:28:01-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:28:02.057638 | controller | 2026-06-10T06:28:02-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:28:02.057688 | controller | 2026-06-10T06:28:02-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:28:02.071748 | controller | 2026-06-10T06:28:02-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:28:02.071812 | controller | 2026-06-10T06:28:02-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:28:02.452257 | controller | 2026-06-10T06:28:02-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:28:02.858411 | controller | 2026-06-10T06:28:02-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:28:02.858482 | controller | 2026-06-10T06:28:02-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:28:03.861841 | controller | 2026-06-10T06:28:03-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:28:03.861906 | controller | 2026-06-10T06:28:03-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:28:05.446398 | controller | 2026-06-10T06:28:05-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:28:05.446440 | controller | 2026-06-10T06:28:05-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:28:06.252326 | controller | 2026-06-10T06:28:06-04:00 INFO 2026-06-10 06:28:06.252367 | controller | - Config: Waiting ... 2026-06-10 06:28:06.252372 | controller | - Gateway: OK 2026-06-10 06:28:06.252376 | controller | - GitServer: OK 2026-06-10 06:28:06.252379 | controller | - HoundSearch: OK 2026-06-10 06:28:06.252383 | controller | - LogJuicer: OK 2026-06-10 06:28:06.252386 | controller | - Logserver: OK 2026-06-10 06:28:06.252390 | controller | - MariaDB: OK 2026-06-10 06:28:06.252394 | controller | - NodePoolBuilder: OK 2026-06-10 06:28:06.252397 | controller | - NodePoolLauncher: OK 2026-06-10 06:28:06.252400 | controller | - Zookeeper: OK 2026-06-10 06:28:06.252405 | controller | - Zuul: Waiting ... 2026-06-10 06:28:06.252409 | controller | - |_ Config Secrets: OK 2026-06-10 06:28:06.252412 | controller | - |_ Executor: OK 2026-06-10 06:28:06.252425 | controller | - |_ Merger: OK 2026-06-10 06:28:06.252429 | controller | - |_ Scheduler: OK 2026-06-10 06:28:06.252432 | controller | - |_ Web: Waiting ... 2026-06-10 06:28:06.252436 | controller | 2026-06-10 06:28:06.252440 | controller | 2026-06-10T06:28:06-04:00 INFO [attempt #7] Waiting 5s for the next reconcile call ... 2026-06-10 06:28:11.255710 | controller | 2026-06-10T06:28:11-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:28:11.867446 | controller | 2026-06-10T06:28:11-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:28:11.867537 | controller | 2026-06-10T06:28:11-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:28:11.882653 | controller | 2026-06-10T06:28:11-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:28:11.882728 | controller | 2026-06-10T06:28:11-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:28:12.267697 | controller | 2026-06-10T06:28:12-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:28:12.671250 | controller | 2026-06-10T06:28:12-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:28:12.671306 | controller | 2026-06-10T06:28:12-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:28:13.700103 | controller | 2026-06-10T06:28:13-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:28:13.700166 | controller | 2026-06-10T06:28:13-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:28:15.261687 | controller | 2026-06-10T06:28:15-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:28:15.261782 | controller | 2026-06-10T06:28:15-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:28:16.072008 | controller | 2026-06-10T06:28:16-04:00 INFO 2026-06-10 06:28:16.072045 | controller | - Config: Waiting ... 2026-06-10 06:28:16.072049 | controller | - Gateway: OK 2026-06-10 06:28:16.072053 | controller | - GitServer: OK 2026-06-10 06:28:16.072057 | controller | - HoundSearch: OK 2026-06-10 06:28:16.072061 | controller | - LogJuicer: OK 2026-06-10 06:28:16.072064 | controller | - Logserver: OK 2026-06-10 06:28:16.072068 | controller | - MariaDB: OK 2026-06-10 06:28:16.072072 | controller | - NodePoolBuilder: OK 2026-06-10 06:28:16.072076 | controller | - NodePoolLauncher: OK 2026-06-10 06:28:16.072080 | controller | - Zookeeper: OK 2026-06-10 06:28:16.072083 | controller | - Zuul: Waiting ... 2026-06-10 06:28:16.072087 | controller | - |_ Config Secrets: OK 2026-06-10 06:28:16.072090 | controller | - |_ Executor: OK 2026-06-10 06:28:16.072094 | controller | - |_ Merger: OK 2026-06-10 06:28:16.072097 | controller | - |_ Scheduler: OK 2026-06-10 06:28:16.072101 | controller | - |_ Web: Waiting ... 2026-06-10 06:28:16.072104 | controller | 2026-06-10 06:28:16.072108 | controller | 2026-06-10T06:28:16-04:00 INFO [attempt #8] Waiting 5s for the next reconcile call ... 2026-06-10 06:28:21.081733 | controller | 2026-06-10T06:28:21-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:28:21.691984 | controller | 2026-06-10T06:28:21-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:28:21.692045 | controller | 2026-06-10T06:28:21-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:28:21.706450 | controller | 2026-06-10T06:28:21-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:28:21.706487 | controller | 2026-06-10T06:28:21-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:28:22.097833 | controller | 2026-06-10T06:28:22-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:28:22.490634 | controller | 2026-06-10T06:28:22-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:28:22.490680 | controller | 2026-06-10T06:28:22-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:28:23.504088 | controller | 2026-06-10T06:28:23-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:28:23.504137 | controller | 2026-06-10T06:28:23-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:28:25.085866 | controller | 2026-06-10T06:28:25-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:28:25.085939 | controller | 2026-06-10T06:28:25-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:28:25.889834 | controller | 2026-06-10T06:28:25-04:00 INFO 2026-06-10 06:28:25.889891 | controller | - Config: Waiting ... 2026-06-10 06:28:25.889896 | controller | - Gateway: OK 2026-06-10 06:28:25.889900 | controller | - GitServer: OK 2026-06-10 06:28:25.889904 | controller | - HoundSearch: OK 2026-06-10 06:28:25.889907 | controller | - LogJuicer: OK 2026-06-10 06:28:25.889919 | controller | - Logserver: OK 2026-06-10 06:28:25.889923 | controller | - MariaDB: OK 2026-06-10 06:28:25.889927 | controller | - NodePoolBuilder: OK 2026-06-10 06:28:25.889930 | controller | - NodePoolLauncher: OK 2026-06-10 06:28:25.889934 | controller | - Zookeeper: OK 2026-06-10 06:28:25.889937 | controller | - Zuul: Waiting ... 2026-06-10 06:28:25.889946 | controller | - |_ Config Secrets: OK 2026-06-10 06:28:25.889950 | controller | - |_ Executor: OK 2026-06-10 06:28:25.889953 | controller | - |_ Merger: OK 2026-06-10 06:28:25.889957 | controller | - |_ Scheduler: OK 2026-06-10 06:28:25.889960 | controller | - |_ Web: Waiting ... 2026-06-10 06:28:25.889964 | controller | 2026-06-10 06:28:25.889968 | controller | 2026-06-10T06:28:25-04:00 INFO [attempt #9] Waiting 5s for the next reconcile call ... 2026-06-10 06:28:30.897837 | controller | 2026-06-10T06:28:30-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:28:31.509124 | controller | 2026-06-10T06:28:31-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:28:31.509161 | controller | 2026-06-10T06:28:31-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:28:31.519221 | controller | 2026-06-10T06:28:31-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:28:31.519273 | controller | 2026-06-10T06:28:31-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:28:31.909306 | controller | 2026-06-10T06:28:31-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:28:32.308511 | controller | 2026-06-10T06:28:32-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:28:32.308570 | controller | 2026-06-10T06:28:32-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:28:33.316863 | controller | 2026-06-10T06:28:33-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:28:33.316916 | controller | 2026-06-10T06:28:33-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:28:34.901285 | controller | 2026-06-10T06:28:34-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:28:34.901335 | controller | 2026-06-10T06:28:34-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:28:35.711517 | controller | 2026-06-10T06:28:35-04:00 INFO 2026-06-10 06:28:35.711578 | controller | - Config: Waiting ... 2026-06-10 06:28:35.711584 | controller | - Gateway: OK 2026-06-10 06:28:35.711589 | controller | - GitServer: OK 2026-06-10 06:28:35.711594 | controller | - HoundSearch: OK 2026-06-10 06:28:35.711599 | controller | - LogJuicer: OK 2026-06-10 06:28:35.711624 | controller | - Logserver: OK 2026-06-10 06:28:35.711632 | controller | - MariaDB: OK 2026-06-10 06:28:35.711636 | controller | - NodePoolBuilder: OK 2026-06-10 06:28:35.711639 | controller | - NodePoolLauncher: OK 2026-06-10 06:28:35.711643 | controller | - Zookeeper: OK 2026-06-10 06:28:35.711647 | controller | - Zuul: Waiting ... 2026-06-10 06:28:35.711652 | controller | - |_ Config Secrets: OK 2026-06-10 06:28:35.711657 | controller | - |_ Executor: OK 2026-06-10 06:28:35.711661 | controller | - |_ Merger: OK 2026-06-10 06:28:35.711666 | controller | - |_ Scheduler: OK 2026-06-10 06:28:35.711671 | controller | - |_ Web: Waiting ... 2026-06-10 06:28:35.711675 | controller | 2026-06-10 06:28:35.711685 | controller | 2026-06-10T06:28:35-04:00 INFO [attempt #10] Waiting 5s for the next reconcile call ... 2026-06-10 06:28:40.720398 | controller | 2026-06-10T06:28:40-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:28:41.343086 | controller | 2026-06-10T06:28:41-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:28:41.343159 | controller | 2026-06-10T06:28:41-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:28:41.362528 | controller | 2026-06-10T06:28:41-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:28:41.362560 | controller | 2026-06-10T06:28:41-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:28:41.740386 | controller | 2026-06-10T06:28:41-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:28:42.134565 | controller | 2026-06-10T06:28:42-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:28:42.134605 | controller | 2026-06-10T06:28:42-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:28:43.139840 | controller | 2026-06-10T06:28:43-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:28:43.139894 | controller | 2026-06-10T06:28:43-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:28:44.724215 | controller | 2026-06-10T06:28:44-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:28:44.724275 | controller | 2026-06-10T06:28:44-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:28:45.528946 | controller | 2026-06-10T06:28:45-04:00 INFO 2026-06-10 06:28:45.528988 | controller | - Config: Waiting ... 2026-06-10 06:28:45.528994 | controller | - Gateway: OK 2026-06-10 06:28:45.528998 | controller | - GitServer: OK 2026-06-10 06:28:45.529003 | controller | - HoundSearch: OK 2026-06-10 06:28:45.529008 | controller | - LogJuicer: OK 2026-06-10 06:28:45.529012 | controller | - Logserver: OK 2026-06-10 06:28:45.529017 | controller | - MariaDB: OK 2026-06-10 06:28:45.529021 | controller | - NodePoolBuilder: OK 2026-06-10 06:28:45.529026 | controller | - NodePoolLauncher: OK 2026-06-10 06:28:45.529030 | controller | - Zookeeper: OK 2026-06-10 06:28:45.529035 | controller | - Zuul: Waiting ... 2026-06-10 06:28:45.529039 | controller | - |_ Config Secrets: OK 2026-06-10 06:28:45.529044 | controller | - |_ Executor: OK 2026-06-10 06:28:45.529048 | controller | - |_ Merger: OK 2026-06-10 06:28:45.529066 | controller | - |_ Scheduler: OK 2026-06-10 06:28:45.529071 | controller | - |_ Web: Waiting ... 2026-06-10 06:28:45.529076 | controller | 2026-06-10 06:28:45.529081 | controller | 2026-06-10T06:28:45-04:00 INFO [attempt #11] Waiting 5s for the next reconcile call ... 2026-06-10 06:28:50.536991 | controller | 2026-06-10T06:28:50-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:28:51.151982 | controller | 2026-06-10T06:28:51-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:28:51.152021 | controller | 2026-06-10T06:28:51-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:28:51.168120 | controller | 2026-06-10T06:28:51-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:28:51.168158 | controller | 2026-06-10T06:28:51-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:28:51.547574 | controller | 2026-06-10T06:28:51-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:28:51.949103 | controller | 2026-06-10T06:28:51-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:28:51.949141 | controller | 2026-06-10T06:28:51-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:28:52.954203 | controller | 2026-06-10T06:28:52-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:28:52.954240 | controller | 2026-06-10T06:28:52-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:28:54.540924 | controller | 2026-06-10T06:28:54-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:28:54.540966 | controller | 2026-06-10T06:28:54-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:28:55.345332 | controller | 2026-06-10T06:28:55-04:00 INFO 2026-06-10 06:28:55.345371 | controller | - Config: Waiting ... 2026-06-10 06:28:55.345376 | controller | - Gateway: OK 2026-06-10 06:28:55.345380 | controller | - GitServer: OK 2026-06-10 06:28:55.345383 | controller | - HoundSearch: OK 2026-06-10 06:28:55.345387 | controller | - LogJuicer: OK 2026-06-10 06:28:55.345390 | controller | - Logserver: OK 2026-06-10 06:28:55.345394 | controller | - MariaDB: OK 2026-06-10 06:28:55.345397 | controller | - NodePoolBuilder: OK 2026-06-10 06:28:55.345401 | controller | - NodePoolLauncher: OK 2026-06-10 06:28:55.345404 | controller | - Zookeeper: OK 2026-06-10 06:28:55.345408 | controller | - Zuul: Waiting ... 2026-06-10 06:28:55.345412 | controller | - |_ Config Secrets: OK 2026-06-10 06:28:55.345415 | controller | - |_ Executor: OK 2026-06-10 06:28:55.345419 | controller | - |_ Merger: OK 2026-06-10 06:28:55.345422 | controller | - |_ Scheduler: OK 2026-06-10 06:28:55.345426 | controller | - |_ Web: Waiting ... 2026-06-10 06:28:55.345429 | controller | 2026-06-10 06:28:55.345460 | controller | 2026-06-10T06:28:55-04:00 INFO [attempt #12] Waiting 5s for the next reconcile call ... 2026-06-10 06:29:00.353355 | controller | 2026-06-10T06:29:00-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:29:00.964950 | controller | 2026-06-10T06:29:00-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:29:00.964996 | controller | 2026-06-10T06:29:00-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:29:00.979441 | controller | 2026-06-10T06:29:00-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:29:00.979513 | controller | 2026-06-10T06:29:00-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:29:01.365236 | controller | 2026-06-10T06:29:01-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:29:01.764618 | controller | 2026-06-10T06:29:01-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:29:01.764679 | controller | 2026-06-10T06:29:01-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:29:02.775529 | controller | 2026-06-10T06:29:02-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:29:02.775587 | controller | 2026-06-10T06:29:02-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:29:04.358958 | controller | 2026-06-10T06:29:04-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:29:04.359051 | controller | 2026-06-10T06:29:04-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:29:05.561663 | controller | 2026-06-10T06:29:05-04:00 INFO 2026-06-10 06:29:05.561706 | controller | - Config: OK 2026-06-10 06:29:05.561713 | controller | - Gateway: OK 2026-06-10 06:29:05.561719 | controller | - GitServer: OK 2026-06-10 06:29:05.561724 | controller | - HoundSearch: OK 2026-06-10 06:29:05.561730 | controller | - LogJuicer: OK 2026-06-10 06:29:05.561735 | controller | - Logserver: OK 2026-06-10 06:29:05.561740 | controller | - MariaDB: OK 2026-06-10 06:29:05.561745 | controller | - NodePoolBuilder: OK 2026-06-10 06:29:05.561751 | controller | - NodePoolLauncher: OK 2026-06-10 06:29:05.561756 | controller | - Zookeeper: OK 2026-06-10 06:29:05.561761 | controller | - Zuul: OK 2026-06-10 06:29:05.561767 | controller | 2026-06-10 06:29:05.561810 | controller | 2026-06-10T06:29:05-04:00 INFO Updating controller configmap ... 2026-06-10 06:29:05.955118 | controller | 2026-06-10T06:29:05-04:00 INFO Standalone reconcile done. 2026-06-10 10:29:06.298169 | controller | ok: Runtime: 0:03:07.703745 2026-06-10 10:29:06.305084 | 2026-06-10 10:29:06.305151 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 10:29:06.827743 | controller | skipping: Conditional result was False 2026-06-10 10:29:06.854326 | 2026-06-10 10:29:06.854522 | TASK [health-check/zuul-connections : Wait for the dummy Zuul connections to be removed from the API] 2026-06-10 10:29:07.406848 | controller | ok: OK (415 bytes) 2026-06-10 10:29:07.417184 | 2026-06-10 10:29:07.417324 | TASK [health-check/zuul-connections : Ensure the new Zuul connections (non SourceInterface) no longer exist in the scheduler's zuul.conf] 2026-06-10 06:29:07.740041 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-10 06:29:07.851087 | controller | command terminated with exit code 1 2026-06-10 06:29:07.942821 | controller | Defaulted container "zuul-scheduler" out of: zuul-scheduler, zuul-statsd, zuul-scheduler-nodeexporter, init-scheduler-config (init) 2026-06-10 06:29:08.070560 | controller | command terminated with exit code 1 2026-06-10 10:29:08.468014 | controller | ok: Runtime: 0:00:00.398184 2026-06-10 10:29:08.477593 | 2026-06-10 10:29:08.477733 | TASK [health-check/zuul-connections : Delete GitHub Connection Secret] 2026-06-10 10:29:09.283386 | controller | changed 2026-06-10 10:29:09.294899 | 2026-06-10 10:29:09.295028 | TASK [health-check/zuul-connections : Delete GitLab Connection Secret] 2026-06-10 10:29:10.085520 | controller | changed 2026-06-10 10:29:10.095309 | 2026-06-10 10:29:10.095475 | TASK [health-check/zuul-connections : Delete Pagure Connection Secret] 2026-06-10 10:29:11.177866 | controller | changed 2026-06-10 10:29:11.194884 | 2026-06-10 10:29:11.195029 | TASK [health-check/zuul-authenticators : Initialize new authenticator config] 2026-06-10 10:29:11.231305 | controller | ok 2026-06-10 10:29:11.243214 | 2026-06-10 10:29:11.243352 | TASK [Add the authenticator to spec] 2026-06-10 10:29:11.278591 | controller | ok 2026-06-10 10:29:11.305518 | 2026-06-10 10:29:11.305618 | TASK [update-custom-resource : read current crd] 2026-06-10 10:29:11.522370 | controller | ok 2026-06-10 10:29:11.531299 | 2026-06-10 10:29:11.531480 | TASK [update-custom-resource : merge the new cr] 2026-06-10 10:29:12.151724 | controller | changed 2026-06-10 10:29:12.162498 | 2026-06-10 10:29:12.162639 | TASK [run the operator in standalone mode] 2026-06-10 10:29:12.202336 | controller | ok 2026-06-10 10:29:12.237267 | 2026-06-10 10:29:12.237476 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 10:29:12.266812 | controller | skipping: Conditional result was False 2026-06-10 10:29:12.279576 | 2026-06-10 10:29:12.279754 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 10:29:12.311030 | controller | skipping: Conditional result was False 2026-06-10 10:29:12.325291 | 2026-06-10 10:29:12.325411 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 06:29:12.888559 | controller | 2026-06-10T06:29:12-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:29:12.888727 | controller | 2026-06-10 06:29:12.897799 | controller | 2026-06-10T06:29:12-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:29:12.897823 | controller | 2026-06-10 06:29:12.917295 | controller | 2026-06-10T06:29:12-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:29:13.528533 | controller | 2026-06-10T06:29:13-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:29:13.615075 | controller | 2026-06-10T06:29:13-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:29:13.623221 | controller | 2026-06-10T06:29:13-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:29:13.623283 | controller | 2026-06-10T06:29:13-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:29:13.641166 | controller | 2026-06-10T06:29:13-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:29:13.641265 | controller | 2026-06-10T06:29:13-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:29:13.934747 | controller | 2026-06-10T06:29:13-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:29:14.331244 | controller | 2026-06-10T06:29:14-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:29:14.331342 | controller | 2026-06-10T06:29:14-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:29:15.315954 | controller | 2026-06-10T06:29:15-04:00 INFO Updating secret, name: zuul-config 2026-06-10 06:29:15.316032 | controller | 2026-06-10T06:29:15-04:00 INFO Updating object, name: zuul-config 2026-06-10 06:29:15.536740 | controller | 2026-06-10T06:29:15-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-10 06:29:15.536819 | controller | 2026-06-10T06:29:15-04:00 INFO Updating object, name: zuul-scheduler 2026-06-10 06:29:17.115931 | controller | 2026-06-10T06:29:17-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-10 06:29:17.116019 | controller | 2026-06-10T06:29:17-04:00 INFO Updating object, name: zuul-web 2026-06-10 06:29:17.929211 | controller | 2026-06-10T06:29:17-04:00 INFO 2026-06-10 06:29:17.929298 | controller | - Config: Waiting ... 2026-06-10 06:29:17.929312 | controller | - Gateway: OK 2026-06-10 06:29:17.929324 | controller | - GitServer: OK 2026-06-10 06:29:17.929333 | controller | - HoundSearch: OK 2026-06-10 06:29:17.929341 | controller | - LogJuicer: OK 2026-06-10 06:29:17.929349 | controller | - Logserver: OK 2026-06-10 06:29:17.929356 | controller | - MariaDB: OK 2026-06-10 06:29:17.929364 | controller | - NodePoolBuilder: OK 2026-06-10 06:29:17.929372 | controller | - NodePoolLauncher: OK 2026-06-10 06:29:17.929380 | controller | - Zookeeper: OK 2026-06-10 06:29:17.929388 | controller | - Zuul: Waiting ... 2026-06-10 06:29:17.929396 | controller | - |_ Config Secrets: OK 2026-06-10 06:29:17.929404 | controller | - |_ Executor: OK 2026-06-10 06:29:17.929411 | controller | - |_ Merger: OK 2026-06-10 06:29:17.929419 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:29:17.929427 | controller | - |_ Web: Waiting ... 2026-06-10 06:29:17.929435 | controller | 2026-06-10 06:29:17.929453 | controller | 2026-06-10T06:29:17-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 06:29:22.937897 | controller | 2026-06-10T06:29:22-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:29:23.552625 | controller | 2026-06-10T06:29:23-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:29:23.552676 | controller | 2026-06-10T06:29:23-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:29:23.572631 | controller | 2026-06-10T06:29:23-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:29:23.572680 | controller | 2026-06-10T06:29:23-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:29:23.946865 | controller | 2026-06-10T06:29:23-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:29:24.346137 | controller | 2026-06-10T06:29:24-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:29:24.346199 | controller | 2026-06-10T06:29:24-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:29:25.353730 | controller | 2026-06-10T06:29:25-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:29:55.451948 | controller | 2026-06-10T06:29:55-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 06:29:55.484042 | controller | 2026-06-10T06:29:55-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:29:55.484135 | controller | 2026-06-10T06:29:55-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:29:55.864419 | controller | 2026-06-10T06:29:55-04:00 INFO 2026-06-10 06:29:55.864470 | controller | - Config: Waiting ... 2026-06-10 06:29:55.864478 | controller | - Gateway: OK 2026-06-10 06:29:55.864485 | controller | - GitServer: OK 2026-06-10 06:29:55.864490 | controller | - HoundSearch: OK 2026-06-10 06:29:55.864496 | controller | - LogJuicer: OK 2026-06-10 06:29:55.864501 | controller | - Logserver: OK 2026-06-10 06:29:55.864506 | controller | - MariaDB: OK 2026-06-10 06:29:55.864511 | controller | - NodePoolBuilder: OK 2026-06-10 06:29:55.864517 | controller | - NodePoolLauncher: OK 2026-06-10 06:29:55.864522 | controller | - Zookeeper: OK 2026-06-10 06:29:55.864527 | controller | - Zuul: Waiting ... 2026-06-10 06:29:55.864532 | controller | - |_ Config Secrets: OK 2026-06-10 06:29:55.864538 | controller | - |_ Executor: OK 2026-06-10 06:29:55.864543 | controller | - |_ Merger: OK 2026-06-10 06:29:55.864548 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:29:55.864553 | controller | - |_ Web: Waiting ... 2026-06-10 06:29:55.864559 | controller | 2026-06-10 06:29:55.864621 | controller | 2026-06-10T06:29:55-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 06:30:00.871347 | controller | 2026-06-10T06:30:00-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:30:01.483727 | controller | 2026-06-10T06:30:01-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:30:01.483798 | controller | 2026-06-10T06:30:01-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:30:01.497258 | controller | 2026-06-10T06:30:01-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:30:01.497376 | controller | 2026-06-10T06:30:01-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:30:01.891310 | controller | 2026-06-10T06:30:01-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:30:02.282548 | controller | 2026-06-10T06:30:02-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:30:02.282643 | controller | 2026-06-10T06:30:02-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:30:03.286150 | controller | 2026-06-10T06:30:03-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:30:21.351222 | controller | 2026-06-10T06:30:21-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:30:21.351270 | controller | 2026-06-10T06:30:21-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:30:21.378154 | controller | 2026-06-10T06:30:21-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:30:21.378223 | controller | 2026-06-10T06:30:21-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:30:22.163851 | controller | 2026-06-10T06:30:22-04:00 INFO 2026-06-10 06:30:22.163912 | controller | - Config: OK 2026-06-10 06:30:22.163923 | controller | - Gateway: OK 2026-06-10 06:30:22.163932 | controller | - GitServer: OK 2026-06-10 06:30:22.163940 | controller | - HoundSearch: OK 2026-06-10 06:30:22.163949 | controller | - LogJuicer: OK 2026-06-10 06:30:22.163957 | controller | - Logserver: OK 2026-06-10 06:30:22.163966 | controller | - MariaDB: OK 2026-06-10 06:30:22.163974 | controller | - NodePoolBuilder: OK 2026-06-10 06:30:22.163982 | controller | - NodePoolLauncher: OK 2026-06-10 06:30:22.163990 | controller | - Zookeeper: OK 2026-06-10 06:30:22.163998 | controller | - Zuul: OK 2026-06-10 06:30:22.164006 | controller | 2026-06-10 06:30:22.164015 | controller | 2026-06-10T06:30:22-04:00 INFO Updating controller configmap ... 2026-06-10 06:30:22.556381 | controller | 2026-06-10T06:30:22-04:00 INFO Standalone reconcile done. 2026-06-10 10:30:23.068773 | controller | ok: Runtime: 0:01:09.992048 2026-06-10 10:30:23.080240 | 2026-06-10 10:30:23.080489 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 10:30:23.672821 | controller | skipping: Conditional result was False 2026-06-10 10:30:23.701923 | 2026-06-10 10:30:23.702114 | TASK [health-check/zuul-authenticators : Wait for the new authenticator to appear in the Zuul API] 2026-06-10 10:30:24.167988 | controller | ok: OK (589 bytes) 2026-06-10 10:30:24.179520 | 2026-06-10 10:30:24.179669 | TASK [Remove the test authenticator from the spec] 2026-06-10 10:30:24.216840 | controller | ok 2026-06-10 10:30:24.250455 | 2026-06-10 10:30:24.250557 | TASK [update-custom-resource : read current crd] 2026-06-10 10:30:24.460107 | controller | ok 2026-06-10 10:30:24.470824 | 2026-06-10 10:30:24.470957 | TASK [update-custom-resource : merge the new cr] 2026-06-10 10:30:25.169684 | controller | changed 2026-06-10 10:30:25.180232 | 2026-06-10 10:30:25.180374 | TASK [run the operator in standalone mode] 2026-06-10 10:30:25.217760 | controller | ok 2026-06-10 10:30:25.248771 | 2026-06-10 10:30:25.248867 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 10:30:25.284397 | controller | skipping: Conditional result was False 2026-06-10 10:30:25.291709 | 2026-06-10 10:30:25.291811 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 10:30:25.317734 | controller | skipping: Conditional result was False 2026-06-10 10:30:25.327737 | 2026-06-10 10:30:25.327837 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 06:30:25.894216 | controller | 2026-06-10T06:30:25-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:30:25.894287 | controller | 2026-06-10 06:30:25.903147 | controller | 2026-06-10T06:30:25-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:30:25.903172 | controller | 2026-06-10 06:30:25.913620 | controller | 2026-06-10T06:30:25-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:30:26.522844 | controller | 2026-06-10T06:30:26-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:30:26.611896 | controller | 2026-06-10T06:30:26-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:30:26.617947 | controller | 2026-06-10T06:30:26-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:30:26.617995 | controller | 2026-06-10T06:30:26-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:30:26.633002 | controller | 2026-06-10T06:30:26-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:30:26.633062 | controller | 2026-06-10T06:30:26-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:30:26.930650 | controller | 2026-06-10T06:30:26-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:30:27.330561 | controller | 2026-06-10T06:30:27-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:30:27.330663 | controller | 2026-06-10T06:30:27-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:30:28.315777 | controller | 2026-06-10T06:30:28-04:00 INFO Updating secret, name: zuul-config 2026-06-10 06:30:28.315855 | controller | 2026-06-10T06:30:28-04:00 INFO Updating object, name: zuul-config 2026-06-10 06:30:28.534977 | controller | 2026-06-10T06:30:28-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-10 06:30:28.535053 | controller | 2026-06-10T06:30:28-04:00 INFO Updating object, name: zuul-scheduler 2026-06-10 06:30:30.113174 | controller | 2026-06-10T06:30:30-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-10 06:30:30.113242 | controller | 2026-06-10T06:30:30-04:00 INFO Updating object, name: zuul-web 2026-06-10 06:30:30.921963 | controller | 2026-06-10T06:30:30-04:00 INFO 2026-06-10 06:30:30.922010 | controller | - Config: Waiting ... 2026-06-10 06:30:30.922016 | controller | - Gateway: OK 2026-06-10 06:30:30.922020 | controller | - GitServer: OK 2026-06-10 06:30:30.922024 | controller | - HoundSearch: OK 2026-06-10 06:30:30.922028 | controller | - LogJuicer: OK 2026-06-10 06:30:30.922031 | controller | - Logserver: OK 2026-06-10 06:30:30.922035 | controller | - MariaDB: OK 2026-06-10 06:30:30.922038 | controller | - NodePoolBuilder: OK 2026-06-10 06:30:30.922041 | controller | - NodePoolLauncher: OK 2026-06-10 06:30:30.922045 | controller | - Zookeeper: OK 2026-06-10 06:30:30.922049 | controller | - Zuul: Waiting ... 2026-06-10 06:30:30.922053 | controller | - |_ Config Secrets: OK 2026-06-10 06:30:30.922057 | controller | - |_ Executor: OK 2026-06-10 06:30:30.922060 | controller | - |_ Merger: OK 2026-06-10 06:30:30.922064 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:30:30.922067 | controller | - |_ Web: Waiting ... 2026-06-10 06:30:30.922071 | controller | 2026-06-10 06:30:30.922080 | controller | 2026-06-10T06:30:30-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 06:30:35.929199 | controller | 2026-06-10T06:30:35-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:30:36.542628 | controller | 2026-06-10T06:30:36-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:30:36.542688 | controller | 2026-06-10T06:30:36-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:30:36.558543 | controller | 2026-06-10T06:30:36-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:30:36.558651 | controller | 2026-06-10T06:30:36-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:30:36.941277 | controller | 2026-06-10T06:30:36-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:30:37.337821 | controller | 2026-06-10T06:30:37-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:30:37.337895 | controller | 2026-06-10T06:30:37-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:30:38.368545 | controller | 2026-06-10T06:30:38-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:31:08.463950 | controller | 2026-06-10T06:31:08-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 06:31:08.493520 | controller | 2026-06-10T06:31:08-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:31:08.493645 | controller | 2026-06-10T06:31:08-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:31:08.881057 | controller | 2026-06-10T06:31:08-04:00 INFO 2026-06-10 06:31:08.881128 | controller | - Config: Waiting ... 2026-06-10 06:31:08.881136 | controller | - Gateway: OK 2026-06-10 06:31:08.881140 | controller | - GitServer: OK 2026-06-10 06:31:08.881145 | controller | - HoundSearch: OK 2026-06-10 06:31:08.881150 | controller | - LogJuicer: OK 2026-06-10 06:31:08.881154 | controller | - Logserver: OK 2026-06-10 06:31:08.881159 | controller | - MariaDB: OK 2026-06-10 06:31:08.881163 | controller | - NodePoolBuilder: OK 2026-06-10 06:31:08.881168 | controller | - NodePoolLauncher: OK 2026-06-10 06:31:08.881172 | controller | - Zookeeper: OK 2026-06-10 06:31:08.881176 | controller | - Zuul: Waiting ... 2026-06-10 06:31:08.881181 | controller | - |_ Config Secrets: OK 2026-06-10 06:31:08.881185 | controller | - |_ Executor: OK 2026-06-10 06:31:08.881190 | controller | - |_ Merger: OK 2026-06-10 06:31:08.881194 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:31:08.881199 | controller | - |_ Web: Waiting ... 2026-06-10 06:31:08.881203 | controller | 2026-06-10 06:31:08.881214 | controller | 2026-06-10T06:31:08-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 06:31:13.885520 | controller | 2026-06-10T06:31:13-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:31:14.496777 | controller | 2026-06-10T06:31:14-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:31:14.496819 | controller | 2026-06-10T06:31:14-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:31:14.509860 | controller | 2026-06-10T06:31:14-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:31:14.509897 | controller | 2026-06-10T06:31:14-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:31:14.899147 | controller | 2026-06-10T06:31:14-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:31:15.308193 | controller | 2026-06-10T06:31:15-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:31:15.308233 | controller | 2026-06-10T06:31:15-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:31:16.301423 | controller | 2026-06-10T06:31:16-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:31:32.357676 | controller | 2026-06-10T06:31:32-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:31:32.357736 | controller | 2026-06-10T06:31:32-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:31:32.463866 | controller | 2026-06-10T06:31:32-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:31:32.463938 | controller | 2026-06-10T06:31:32-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:31:33.171095 | controller | 2026-06-10T06:31:33-04:00 INFO 2026-06-10 06:31:33.171156 | controller | - Config: OK 2026-06-10 06:31:33.171161 | controller | - Gateway: OK 2026-06-10 06:31:33.171165 | controller | - GitServer: OK 2026-06-10 06:31:33.171169 | controller | - HoundSearch: OK 2026-06-10 06:31:33.171173 | controller | - LogJuicer: OK 2026-06-10 06:31:33.171176 | controller | - Logserver: OK 2026-06-10 06:31:33.171180 | controller | - MariaDB: OK 2026-06-10 06:31:33.171184 | controller | - NodePoolBuilder: OK 2026-06-10 06:31:33.171187 | controller | - NodePoolLauncher: OK 2026-06-10 06:31:33.171191 | controller | - Zookeeper: OK 2026-06-10 06:31:33.171194 | controller | - Zuul: OK 2026-06-10 06:31:33.171198 | controller | 2026-06-10 06:31:33.171207 | controller | 2026-06-10T06:31:33-04:00 INFO Updating controller configmap ... 2026-06-10 06:31:33.561490 | controller | 2026-06-10T06:31:33-04:00 INFO Standalone reconcile done. 2026-06-10 10:31:33.947824 | controller | ok: Runtime: 0:01:07.956519 2026-06-10 10:31:33.961212 | 2026-06-10 10:31:33.961536 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 10:31:34.502037 | controller | skipping: Conditional result was False 2026-06-10 10:31:34.529346 | 2026-06-10 10:31:34.529534 | TASK [health-check/zuul-authenticators : Wait for the test authenticator to be delisted] 2026-06-10 10:31:35.063972 | controller | ok: OK (388 bytes) 2026-06-10 10:31:35.082092 | 2026-06-10 10:31:35.082224 | TASK [ansible.builtin.include_role : health-check/check-zuul-tenant] 2026-06-10 10:31:35.122781 | controller | ok 2026-06-10 10:31:35.157723 | 2026-06-10 10:31:35.157867 | TASK [health-check/check-zuul-tenant : Get configuration errors/warnings into the internal tenant] 2026-06-10 10:31:35.670116 | controller | ok: OK (2 bytes) 2026-06-10 10:31:35.682847 | 2026-06-10 10:31:35.682987 | TASK [health-check/check-zuul-tenant : Fail when errors/warnings into the internal tenant] 2026-06-10 10:31:35.720875 | controller | skipping: Conditional result was False 2026-06-10 10:31:35.738921 | 2026-06-10 10:31:35.739063 | TASK [health-check/config-update-zuul : Set random value] 2026-06-10 10:31:35.835247 | controller | ok 2026-06-10 10:31:35.843557 | 2026-06-10 10:31:35.843692 | TASK [health-check/config-update-zuul : Create vars to generate and submit resources addition] 2026-06-10 10:31:35.884188 | controller | ok 2026-06-10 10:31:35.893608 | 2026-06-10 10:31:35.893745 | TASK [ansible.builtin.include_role : create-gerrit-repository] 2026-06-10 10:31:35.929513 | controller | ok 2026-06-10 10:31:35.963728 | 2026-06-10 10:31:35.963839 | TASK [create-gerrit-repository : Create gerrit repository named ichiban-config-QEwn8WpM] 2026-06-10 06:31:37.265920 | controller | 2026-06-10 10:31:37,264 INFO managesf.model.yamlbkd.engine - Resource engine is using /tmp/direct-apply as workdir 2026-06-10 06:31:37.266017 | controller | 2026-06-10 10:31:37,265 INFO managesf.model.yamlbkd.engine - Resources engine: direct apply resources requested 2026-06-10 06:31:37.286979 | controller | 2026-06-10 10:31:37,286 INFO managesf.GerritClient - ichiban-config-QEwn8WpM-core: creating group (Team core for the ichiban-config-QEwn8WpM repo) 2026-06-10 06:31:37.325798 | controller | 2026-06-10 10:31:37,324 INFO managesf.GerritClient - ichiban-config-QEwn8WpM-core: deleting group member admin 2026-06-10 06:31:37.385119 | controller | 2026-06-10 10:31:37,384 INFO managesf.GerritClient - ichiban-config-QEwn8WpM-ptl: creating group (Team lead for the ichiban-config-QEwn8WpM repo) 2026-06-10 06:31:37.475428 | controller | 2026-06-10 10:31:37,474 INFO managesf.GerritClient - ichiban-config-QEwn8WpM-ptl: deleting group member admin 2026-06-10 06:31:37.570469 | controller | 2026-06-10 10:31:37,569 INFO managesf.GerritClient - ichiban-config-QEwn8WpM-ptl: adding group member admin@sfop.me 2026-06-10 06:31:37.594445 | controller | 2026-06-10 10:31:37,593 INFO managesf.GerritClient - ichiban-config-QEwn8WpM: creating project (ichiban-config-QEwn8WpM repository) 2026-06-10 06:31:37.692245 | controller | 2026-06-10 10:31:37,691 INFO managesf.services.gerrit.utils - [gerrit] Clone repository ichiban-config-QEwn8WpM 2026-06-10 06:31:37.934645 | controller | 2026-06-10 10:31:37,933 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'clone', 'ssh://admin@gerrit-sshd:29418/ichiban-config-QEwn8WpM', '/tmp/tmppfj0u3aa/clone-ichiban-config-QEwn8WpM'] output 2026-06-10 06:31:37.934692 | controller | 2026-06-10 10:31:37,933 INFO managesf.services.gerrit.utils - Cloning into '/tmp/tmppfj0u3aa/clone-ichiban-config-QEwn8WpM'... 2026-06-10 06:31:37.934700 | controller | Warning: Permanently added '[gerrit-sshd]:29418' (ED25519) to the list of known hosts. 2026-06-10 06:31:37.934706 | controller | Total 2 (delta 0), reused 0 (delta 0) 2026-06-10 06:31:37.934712 | controller | 2026-06-10 06:31:38.082692 | controller | 2026-06-10 10:31:38,081 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'fetch', '-a'] output 2026-06-10 06:31:38.082748 | controller | 2026-06-10 10:31:38,082 INFO managesf.services.gerrit.utils - 2026-06-10 06:31:38.140751 | controller | 2026-06-10 10:31:38,139 INFO managesf.services.gerrit.utils - [gerrit] Prepare push on config for repository ichiban-config-QEwn8WpM 2026-06-10 06:31:38.286085 | controller | 2026-06-10 10:31:38,285 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'fetch', 'origin', 'refs/meta/config:refs/remotes/origin/meta/config'] output 2026-06-10 06:31:38.286131 | controller | 2026-06-10 10:31:38,285 INFO managesf.services.gerrit.utils - Total 4 (delta 0), reused 0 (delta 0) 2026-06-10 06:31:38.286138 | controller | From ssh://gerrit-sshd:29418/ichiban-config-QEwn8WpM 2026-06-10 06:31:38.286143 | controller | * [new ref] refs/meta/config -> origin/meta/config 2026-06-10 06:31:38.286148 | controller | 2026-06-10 06:31:38.292343 | controller | 2026-06-10 10:31:38,291 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'checkout', 'meta/config'] output 2026-06-10 06:31:38.292384 | controller | 2026-06-10 10:31:38,291 INFO managesf.services.gerrit.utils - Switched to a new branch 'meta/config' 2026-06-10 06:31:38.292390 | controller | branch 'meta/config' set up to track 'origin/meta/config'. 2026-06-10 06:31:38.292395 | controller | 2026-06-10 06:31:38.292401 | controller | 2026-06-10 10:31:38,291 INFO managesf.services.gerrit.utils - [gerrit] Add file project.config to index 2026-06-10 06:31:38.296858 | controller | 2026-06-10 10:31:38,296 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'add', 'project.config'] output 2026-06-10 06:31:38.296895 | controller | 2026-06-10 10:31:38,296 INFO managesf.services.gerrit.utils - 2026-06-10 06:31:38.296902 | controller | 2026-06-10 10:31:38,296 INFO managesf.services.gerrit.utils - [gerrit] Add file groups to index 2026-06-10 06:31:38.303284 | controller | 2026-06-10 10:31:38,302 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'add', 'groups'] output 2026-06-10 06:31:38.303325 | controller | 2026-06-10 10:31:38,302 INFO managesf.services.gerrit.utils - 2026-06-10 06:31:38.308337 | controller | 2026-06-10 10:31:38,307 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'status', '-s'] output 2026-06-10 06:31:38.308391 | controller | 2026-06-10 10:31:38,307 INFO managesf.services.gerrit.utils - M groups 2026-06-10 06:31:38.308398 | controller | M project.config 2026-06-10 06:31:38.308405 | controller | 2026-06-10 06:31:38.319327 | controller | 2026-06-10 10:31:38,318 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'commit', '-a', '--author', 'admin ', '-mProvides ACL and Groups'] output 2026-06-10 06:31:38.319355 | controller | 2026-06-10 10:31:38,318 INFO managesf.services.gerrit.utils - [meta/config 0c8aef6] Provides ACL and Groups 2026-06-10 06:31:38.319361 | controller | 2 files changed, 31 insertions(+), 9 deletions(-) 2026-06-10 06:31:38.319366 | controller | 2026-06-10 06:31:38.533025 | controller | 2026-06-10 10:31:38,532 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'push', 'origin', 'meta/config:meta/config'] output 2026-06-10 06:31:38.533091 | controller | 2026-06-10 10:31:38,532 INFO managesf.services.gerrit.utils - remote: remote: Processing changes: refs: 1 remote: Processing changes: refs: 1 remote: Processing changes: refs: 1, done 2026-06-10 06:31:38.533106 | controller | To ssh://gerrit-sshd:29418/ichiban-config-QEwn8WpM 2026-06-10 06:31:38.533114 | controller | c16f153..0c8aef6 meta/config -> refs/meta/config 2026-06-10 06:31:38.533121 | controller | 2026-06-10 06:31:38.533135 | controller | 2026-06-10 10:31:38,532 INFO managesf.services.gerrit.utils - [gerrit] Push on config for repository ichiban-config-QEwn8WpM 2026-06-10 06:31:38.533143 | controller | 2026-06-10 10:31:38,532 INFO managesf.services.gerrit.utils - [gerrit] Prepare push on master for repository ichiban-config-QEwn8WpM 2026-06-10 06:31:38.539465 | controller | 2026-06-10 10:31:38,538 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'checkout', 'master'] output 2026-06-10 06:31:38.539497 | controller | 2026-06-10 10:31:38,539 INFO managesf.services.gerrit.utils - Switched to branch 'master' 2026-06-10 06:31:38.539504 | controller | Your branch is up to date with 'origin/master'. 2026-06-10 06:31:38.539509 | controller | 2026-06-10 06:31:38.544655 | controller | 2026-06-10 10:31:38,544 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'reset', '--hard', 'origin/master'] output 2026-06-10 06:31:38.544698 | controller | 2026-06-10 10:31:38,544 INFO managesf.services.gerrit.utils - HEAD is now at 764de22 Initial empty repository 2026-06-10 06:31:38.544706 | controller | 2026-06-10 06:31:38.544787 | controller | 2026-06-10 10:31:38,544 INFO managesf.services.gerrit.utils - [gerrit] Add file .gitreview to index 2026-06-10 06:31:38.549551 | controller | 2026-06-10 10:31:38,548 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'add', '.gitreview'] output 2026-06-10 06:31:38.549581 | controller | 2026-06-10 10:31:38,549 INFO managesf.services.gerrit.utils - 2026-06-10 06:31:38.554524 | controller | 2026-06-10 10:31:38,553 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'status', '-s'] output 2026-06-10 06:31:38.554556 | controller | 2026-06-10 10:31:38,553 INFO managesf.services.gerrit.utils - A .gitreview 2026-06-10 06:31:38.554564 | controller | 2026-06-10 06:31:38.562724 | controller | 2026-06-10 10:31:38,561 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'commit', '-a', '--author', 'admin ', '-mManageSF commit'] output 2026-06-10 06:31:38.562762 | controller | 2026-06-10 10:31:38,562 INFO managesf.services.gerrit.utils - [master 07cc057] ManageSF commit 2026-06-10 06:31:38.562769 | controller | 1 file changed, 5 insertions(+) 2026-06-10 06:31:38.562775 | controller | create mode 100644 .gitreview 2026-06-10 06:31:38.562780 | controller | 2026-06-10 06:31:38.774958 | controller | 2026-06-10 10:31:38,773 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'push', 'origin', 'master'] output 2026-06-10 06:31:38.775023 | controller | 2026-06-10 10:31:38,774 INFO managesf.services.gerrit.utils - remote: remote: Processing changes: refs: 1 remote: Processing changes: refs: 1 remote: Processing changes: refs: 1, done 2026-06-10 06:31:38.775034 | controller | To ssh://gerrit-sshd:29418/ichiban-config-QEwn8WpM 2026-06-10 06:31:38.775043 | controller | 764de22..07cc057 master -> master 2026-06-10 06:31:38.775051 | controller | 2026-06-10 06:31:38.775068 | controller | 2026-06-10 10:31:38,774 INFO managesf.services.gerrit.utils - [gerrit] Push on master for repository ichiban-config-QEwn8WpM 2026-06-10 06:31:38.775087 | controller | 2026-06-10 10:31:38,774 INFO managesf.services.gerrit.utils - [gerrit] List remote branches 2026-06-10 06:31:38.791000 | controller | 2026-06-10 10:31:38,787 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'branch', '-rv', '--abbrev=40'] output 2026-06-10 06:31:38.791071 | controller | 2026-06-10 10:31:38,787 INFO managesf.services.gerrit.utils - origin/HEAD -> origin/master 2026-06-10 06:31:38.791081 | controller | origin/master 07cc0576a152a326ef32e2bda43f385e5204a7df ManageSF commit 2026-06-10 06:31:38.791088 | controller | origin/meta/config c16f153325a85397ea0ae04c8152d11714860440 Created project 2026-06-10 06:31:38.791096 | controller | 2026-06-10 06:31:38.791104 | controller | 2026-06-10 10:31:38,788 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-QEwn8WpM-core] is going to be created. 2026-06-10 06:31:38.791112 | controller | 2026-06-10 10:31:38,788 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-QEwn8WpM-ptl] is going to be created. 2026-06-10 06:31:38.791121 | controller | 2026-06-10 10:31:38,788 INFO managesf.model.yamlbkd.engine - Resource [type: repos, ID: ichiban-config-QEwn8WpM] is going to be created. 2026-06-10 06:31:38.791128 | controller | 2026-06-10 10:31:38,788 INFO managesf.model.yamlbkd.engine - Resource [type: acls, ID: ichiban-config-QEwn8WpM-acl] is going to be created. 2026-06-10 06:31:38.791136 | controller | 2026-06-10 10:31:38,788 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-QEwn8WpM-core] will be created. 2026-06-10 06:31:38.791143 | controller | 2026-06-10 10:31:38,788 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-QEwn8WpM-core] has been created. 2026-06-10 06:31:38.791157 | controller | 2026-06-10 06:31:38.791164 | controller | === Resources actions === 2026-06-10 06:31:38.791172 | controller | 2026-06-10 06:31:38.791179 | controller | Resource [type: groups, ID: ichiban-config-QEwn8WpM-core] is going to be created. 2026-06-10 06:31:38.791187 | controller | Resource [type: groups, ID: ichiban-config-QEwn8WpM-ptl] is going to be created. 2026-06-10 06:31:38.791194 | controller | Resource [type: repos, ID: ichiban-config-QEwn8WpM] is going to be created. 2026-06-10 06:31:38.791201 | controller | Resource [type: acls, ID: ichiban-config-QEwn8WpM-acl] is going to be created. 2026-06-10 06:31:38.791207 | controller | Resource [type: groups, ID: ichiban-config-QEwn8WpM-core] will be created. 2026-06-10 06:31:38.791215 | controller | Resource [type: groups, ID: ichiban-config-QEwn8WpM-core] has been created. 2026-06-10 06:31:38.791222 | controller | Resource [type: groups, ID: ichiban-config-QEwn8WpM-ptl] will be created. 2026-06-10 06:31:38.791229 | controller | Resource [type: groups, ID: ichiban-config-QEwn8WpM-ptl] has been created. 2026-06-10 06:31:38.791236 | controller | Resource [type: acls, ID: ichiban-config-QEwn8WpM-acl] will be created. 2026-06-10 06:31:38.791243 | controller | Resource [type: acls, ID: ichiban-config-QEwn8WpM-acl] has been created. 2026-06-10 06:31:38.791250 | controller | Resource [type: repos, ID: ichiban-config-QEwn8WpM] will be created. 2026-06-10 06:31:38.791257 | controller | Resource [type: repos, ID: ichiban-config-QEwn8WpM] has been created. 2026-06-10 06:31:38.791264 | controller | 2026-06-10 10:31:38,788 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-QEwn8WpM-ptl] will be created. 2026-06-10 06:31:38.791272 | controller | 2026-06-10 10:31:38,788 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-QEwn8WpM-ptl] has been created. 2026-06-10 06:31:38.791278 | controller | 2026-06-10 10:31:38,788 INFO managesf.model.yamlbkd.engine - Resource [type: acls, ID: ichiban-config-QEwn8WpM-acl] will be created. 2026-06-10 06:31:38.791289 | controller | 2026-06-10 10:31:38,788 INFO managesf.model.yamlbkd.engine - Resource [type: acls, ID: ichiban-config-QEwn8WpM-acl] has been created. 2026-06-10 06:31:38.791306 | controller | 2026-06-10 10:31:38,788 INFO managesf.model.yamlbkd.engine - Resource [type: repos, ID: ichiban-config-QEwn8WpM] will be created. 2026-06-10 06:31:38.791313 | controller | 2026-06-10 10:31:38,788 INFO managesf.model.yamlbkd.engine - Resource [type: repos, ID: ichiban-config-QEwn8WpM] has been created. 2026-06-10 10:31:39.510030 | controller | ok: Runtime: 0:00:02.787921 2026-06-10 10:31:39.520875 | 2026-06-10 10:31:39.521012 | TASK [create-gerrit-repository : Ensure repository exists in Gerrit] 2026-06-10 10:31:39.976123 | controller | ok: OK (752 bytes) 2026-06-10 10:31:40.000369 | 2026-06-10 10:31:40.000557 | TASK [health-check/config-update-zuul : Set bad Zuul tenant config] 2026-06-10 10:31:40.403032 | controller | changed: Block inserted 2026-06-10 10:31:40.412604 | 2026-06-10 10:31:40.412751 | LOOP [health-check/config-update-zuul : Commit file in the config repo] 2026-06-10 10:31:40.743128 | controller | ok: Item: git add zuul/main.yaml Runtime: 0:00:00.005829 2026-06-10 10:31:40.743443 | controller | ok: All items complete 2026-06-10 10:31:40.743473 | 2026-06-10 10:31:40.963397 | controller | [master 7c461d3] Add zuul/main.yaml 2026-06-10 10:31:40.963650 | controller | 1 file changed, 5 insertions(+) 2026-06-10 10:31:40.963743 | controller | ok: Item: git commit -m 'Add zuul/main.yaml' Runtime: 0:00:00.008289 2026-06-10 10:31:40.977188 | 2026-06-10 10:31:40.977491 | TASK [ansible.builtin.include_role : health-check/repo-submit-change] 2026-06-10 10:31:41.015018 | controller | ok 2026-06-10 10:31:41.056462 | 2026-06-10 10:31:41.056591 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 10:31:41.089767 | controller | ok 2026-06-10 10:31:41.102358 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 10:31:41.203997 | 2026-06-10 10:31:41.204308 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 06:31:42.582716 | controller | Your change was committed before the commit hook was installed. 2026-06-10 06:31:42.582802 | controller | Amending the commit to add a gerrit change id. 2026-06-10 06:31:42.582815 | controller | remote: 2026-06-10 06:31:42.582825 | controller | remote: Processing changes: new: 1 (\) 2026-06-10 06:31:42.582839 | controller | remote: Processing changes: refs: 1, new: 1 (\) 2026-06-10 06:31:42.582849 | controller | remote: Processing changes: refs: 1, new: 1 (\) 2026-06-10 06:31:42.582859 | controller | remote: Processing changes: refs: 1, new: 1 (\) 2026-06-10 06:31:42.582868 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-10 06:31:42.582880 | controller | remote: 2026-06-10 06:31:42.582889 | controller | remote: SUCCESS 2026-06-10 06:31:42.582898 | controller | remote: 2026-06-10 06:31:42.582907 | controller | remote: https://gerrit.sfop.me/c/config/+/1 Add zuul/main.yaml [NEW] 2026-06-10 06:31:42.582916 | controller | remote: 2026-06-10 06:31:42.582924 | controller | To https://gerrit.sfop.me/a/config 2026-06-10 06:31:42.582933 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 10:31:42.754339 | controller | ok: Runtime: 0:00:01.166266 2026-06-10 10:31:42.771771 | 2026-06-10 10:31:42.771924 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-10 06:31:43.016493 | controller | If30fc1c56ce4f622b83b8300297814341223077d 2026-06-10 10:31:43.316393 | controller | ok: Runtime: 0:00:00.011146 2026-06-10 10:31:43.322799 | 2026-06-10 10:31:43.322898 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 10:31:44.070212 | controller | ok: OK (47 bytes) 2026-06-10 10:31:44.082422 | 2026-06-10 10:31:44.082615 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 10:31:44.120368 | controller | ok 2026-06-10 10:31:44.133999 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-job-failure.yaml 2026-06-10 10:31:44.148690 | 2026-06-10 10:31:44.148774 | TASK [health-check/repo-submit-change : Get review ID from the changeID using gerrit API] 2026-06-10 10:31:44.650957 | controller | ok: OK (7142 bytes) 2026-06-10 10:31:44.662029 | 2026-06-10 10:31:44.662277 | TASK [health-check/repo-submit-change : ansible.builtin.set_fact] 2026-06-10 10:31:44.726236 | controller | ok 2026-06-10 10:31:44.746478 | 2026-06-10 10:31:44.746658 | TASK [health-check/repo-submit-change : Wait config-check FAILURE result using zuul-web API] 2026-06-10 10:32:37.474800 | controller | ok: OK (1162 bytes) 2026-06-10 10:32:37.495244 | 2026-06-10 10:32:37.495420 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 10:32:37.530090 | controller | skipping: Conditional result was False 2026-06-10 10:32:37.541541 | 2026-06-10 10:32:37.541650 | TASK [health-check/repo-submit-change : Ensure a success for config-update post job] 2026-06-10 10:32:37.570288 | controller | skipping: Conditional result was False 2026-06-10 10:32:37.581147 | 2026-06-10 10:32:37.581249 | TASK [health-check/repo-submit-change : Ensure a success for config-update job] 2026-06-10 10:32:37.608602 | controller | skipping: Conditional result was False 2026-06-10 10:32:37.622064 | 2026-06-10 10:32:37.622172 | TASK [health-check/config-update-zuul : Set Zuul tenant config] 2026-06-10 10:32:37.862666 | controller | changed: Block inserted 2026-06-10 10:32:37.871885 | 2026-06-10 10:32:37.872039 | LOOP [health-check/config-update-zuul : Commit file in the config repo] 2026-06-10 10:32:38.141886 | controller | ok: Item: git add zuul/main.yaml Runtime: 0:00:00.006166 2026-06-10 10:32:38.373663 | controller | [master 2ce8c3d] Add zuul/main.yaml 2026-06-10 10:32:38.373928 | controller | Date: Wed Jun 10 06:31:40 2026 -0400 2026-06-10 10:32:38.373986 | controller | 1 file changed, 8 insertions(+) 2026-06-10 10:32:38.374044 | controller | ok: Item: git commit --amend --no-edit Runtime: 0:00:00.030186 2026-06-10 10:32:38.387755 | 2026-06-10 10:32:38.387993 | TASK [Submit change and verify Zuul console output] 2026-06-10 10:32:38.426622 | controller | ok 2026-06-10 10:32:38.473837 | 2026-06-10 10:32:38.473972 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 10:32:38.507040 | controller | ok 2026-06-10 10:32:38.515400 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 10:32:38.522765 | 2026-06-10 10:32:38.522849 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 06:32:39.286754 | controller | remote: 2026-06-10 06:32:39.286834 | controller | remote: Processing changes: refs: 1, updated: 1 2026-06-10 06:32:39.286841 | controller | remote: Processing changes: refs: 1, updated: 1 2026-06-10 06:32:39.286846 | controller | remote: Processing changes: refs: 1, updated: 1 2026-06-10 06:32:39.286852 | controller | remote: Processing changes: refs: 1, updated: 1, done 2026-06-10 06:32:39.286858 | controller | remote: 2026-06-10 06:32:39.286862 | controller | remote: SUCCESS 2026-06-10 06:32:39.286866 | controller | remote: 2026-06-10 06:32:39.286871 | controller | remote: https://gerrit.sfop.me/c/config/+/1 Add zuul/main.yaml 2026-06-10 06:32:39.286876 | controller | remote: 2026-06-10 06:32:39.286880 | controller | To https://gerrit.sfop.me/a/config 2026-06-10 06:32:39.286888 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 10:32:39.559624 | controller | ok: Runtime: 0:00:00.564611 2026-06-10 10:32:39.565611 | 2026-06-10 10:32:39.565691 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-10 06:32:39.805226 | controller | If30fc1c56ce4f622b83b8300297814341223077d 2026-06-10 10:32:40.102563 | controller | ok: Runtime: 0:00:00.005270 2026-06-10 10:32:40.112886 | 2026-06-10 10:32:40.113032 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 10:32:40.658586 | controller | ok: OK (47 bytes) 2026-06-10 10:32:40.672985 | 2026-06-10 10:32:40.673150 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 10:32:40.702195 | controller | skipping: Conditional result was False 2026-06-10 10:32:40.714838 | 2026-06-10 10:32:40.715003 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 10:32:40.761835 | controller | ok 2026-06-10 10:32:40.775850 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-zuul-console-success.yaml 2026-06-10 10:32:40.791770 | 2026-06-10 10:32:40.791863 | TASK [health-check/repo-submit-change : Wait for config-update job] 2026-06-10 10:33:32.487017 | controller | ok: Runtime: 0:00:00.041040 2026-06-10 10:33:32.496106 | 2026-06-10 10:33:32.496234 | TASK [health-check/repo-submit-change : Connect and validate console stream] 2026-06-10 06:33:32.893360 | controller | 2026-06-10 10:33:26.778330 | Job console starting 2026-06-10 06:33:32.893429 | controller | 2026-06-10 10:33:26.778957 | Updating git repos 2026-06-10 06:33:32.893437 | controller | 2026-06-10 10:33:26.794042 | Cloning repos into workspace 2026-06-10 06:33:32.893445 | controller | 2026-06-10 10:33:27.415161 | Restoring repo states 2026-06-10 06:33:32.893452 | controller | 2026-06-10 10:33:27.419984 | Merging changes 2026-06-10 06:33:32.893460 | controller | 2026-06-10 10:33:27.419995 | Checking out repos 2026-06-10 06:33:32.893468 | controller | 2026-06-10 10:33:27.474227 | Preparing playbooks 2026-06-10 06:33:32.893473 | controller | 2026-06-10 10:33:28.004069 | Running Ansible setup 2026-06-10 06:33:32.893481 | controller | 2026-06-10 10:33:30.575138 | PRE-RUN START: [trusted : git-server/system-config/playbooks/base/pre.yaml@master] 2026-06-10 06:33:32.893488 | controller | 2026-06-10 10:33:31.774167 | 2026-06-10 06:33:32.893492 | controller | 2026-06-10 10:33:31.774318 | PLAY [localhost] 2026-06-10 06:33:32.893497 | controller | 2026-06-10 10:33:31.782818 | 2026-06-10 06:33:32.893502 | controller | 2026-06-10 10:33:31.782951 | TASK [Gathering Facts] 2026-06-10 06:33:32.893507 | controller | 2026-06-10 06:33:33.393500 | controller | 2026-06-10 10:33:33.234127 | localhost | ok 2026-06-10 06:33:33.393553 | controller | 2026-06-10 10:33:33.270519 | 2026-06-10 06:33:33.393559 | controller | 2026-06-10 10:33:33.270693 | TASK [Setup log path fact] 2026-06-10 06:33:33.393563 | controller | 2026-06-10 10:33:33.286564 | localhost | ok 2026-06-10 06:33:33.393567 | controller | 2026-06-10 10:33:33.297448 | 2026-06-10 06:33:33.393571 | controller | 2026-06-10 10:33:33.297548 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-06-10 06:33:33.393575 | controller | 2026-06-10 10:33:33.373859 | localhost | ok 2026-06-10 06:33:33.393579 | controller | 2026-06-10 10:33:33.379812 | 2026-06-10 06:33:33.393590 | controller | 2026-06-10 10:33:33.379906 | TASK [emit-job-header : Print job information] 2026-06-10 06:33:33.393594 | controller | 2026-06-10 06:33:33.894120 | controller | 2026-06-10 10:33:33.417296 | # Job Information 2026-06-10 06:33:33.894180 | controller | 2026-06-10 10:33:33.417543 | Ansible Version: 2.16.18 2026-06-10 06:33:33.894188 | controller | 2026-06-10 10:33:33.417585 | Job: config-update 2026-06-10 06:33:33.894194 | controller | 2026-06-10 10:33:33.417679 | Pipeline: post 2026-06-10 06:33:33.894200 | controller | 2026-06-10 10:33:33.417710 | Executor: zuul-executor-0.zuul-executor-headless.sf.svc.cluster.local 2026-06-10 06:33:33.894206 | controller | 2026-06-10 10:33:33.417731 | Triggered by: https://gerrit.sfop.me/plugins/gitiles/config/+/2ce8c3d40198ff050b762dab0adaf1e4fab256d3^!/ 2026-06-10 06:33:33.894212 | controller | 2026-06-10 10:33:33.417755 | Log URL (when completed): https://sfop.me/logs/37f/internal/37f1bcb937d140e5a4283fce0eb2cf44/ 2026-06-10 06:33:33.894219 | controller | 2026-06-10 10:33:33.417786 | Event ID: 992650f6fd9147bdba81311bb79a8efe 2026-06-10 06:33:33.894227 | controller | 2026-06-10 10:33:33.421221 | 2026-06-10 06:33:33.894235 | controller | 2026-06-10 10:33:33.421302 | LOOP [emit-job-header : Print node information] 2026-06-10 06:33:33.894242 | controller | 2026-06-10 10:33:33.482634 | 2026-06-10 06:33:33.894249 | controller | 2026-06-10 10:33:33.482812 | TASK [log-inventory : Ensure Zuul Ansible directory exists] 2026-06-10 06:33:33.894256 | controller | 2026-06-10 06:33:34.394081 | controller | 2026-06-10 10:33:34.376562 | localhost -> localhost | changed 2026-06-10 06:33:34.394132 | controller | 2026-06-10 10:33:34.382283 | 2026-06-10 06:33:34.394138 | controller | 2026-06-10 10:33:34.382398 | TASK [log-inventory : Copy ansible inventory to logs dir] 2026-06-10 06:33:34.394143 | controller | 2026-06-10 06:33:36.394918 | controller | 2026-06-10 10:33:36.204392 | localhost -> localhost | changed 2026-06-10 06:33:36.394965 | controller | 2026-06-10 10:33:36.209405 | 2026-06-10 06:33:36.394971 | controller | 2026-06-10 10:33:36.209474 | PLAY [all] 2026-06-10 06:33:36.394977 | controller | 2026-06-10 10:33:36.209570 | 2026-06-10 06:33:36.394981 | controller | 2026-06-10 10:33:36.209595 | PLAY RECAP 2026-06-10 06:33:36.394985 | controller | 2026-06-10 10:33:36.209649 | localhost | ok: 5 changed: 2 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0 2026-06-10 06:33:36.394989 | controller | 2026-06-10 10:33:36.209672 | 2026-06-10 06:33:36.394993 | controller | 2026-06-10 06:33:36.895171 | controller | 2026-06-10 10:33:36.501399 | PRE-RUN END RESULT_NORMAL: [trusted : git-server/system-config/playbooks/base/pre.yaml@master] 2026-06-10 06:33:36.895212 | controller | 2026-06-10 10:33:36.502273 | RUN START: [trusted : git-server/system-config/playbooks/config/update.yaml@master] 2026-06-10 06:33:36.895231 | controller | 2026-06-10 10:33:38.104632 | controller | ok: Runtime: 0:00:05.016331 2026-06-10 10:33:38.116324 | 2026-06-10 10:33:38.116510 | TASK [health-check/repo-submit-change : Ensure a success for config-update post job] 2026-06-10 10:33:38.161866 | controller | ok 2026-06-10 10:33:38.174227 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-post-job-success.yaml 2026-06-10 10:33:38.191111 | 2026-06-10 10:33:38.191242 | TASK [health-check/repo-submit-change : Register commit sha] 2026-06-10 06:33:38.424629 | controller | 2ce8c3d40198ff050b762dab0adaf1e4fab256d3 2026-06-10 10:33:38.727641 | controller | ok: Runtime: 0:00:00.007157 2026-06-10 10:33:38.734784 | 2026-06-10 10:33:38.734872 | TASK [health-check/repo-submit-change : Wait config-update SUCCESS result using zuul-web API] 2026-06-10 10:34:41.879421 | controller | ok: OK (1142 bytes) 2026-06-10 10:34:41.892323 | 2026-06-10 10:34:41.892499 | TASK [health-check/repo-submit-change : Ensure a success for config-update job] 2026-06-10 10:34:41.929029 | controller | skipping: Conditional result was False 2026-06-10 10:34:41.948093 | 2026-06-10 10:34:41.948279 | TASK [health-check/config-update-zuul : ansible.builtin.set_fact] 2026-06-10 10:34:41.994768 | controller | ok 2026-06-10 10:34:42.007528 | 2026-06-10 10:34:42.007663 | TASK [Check that Zuul has pushed the job console on the logserver] 2026-06-10 10:34:42.045499 | controller | ok 2026-06-10 10:34:42.090330 | 2026-06-10 10:34:42.090607 | TASK [health-check/ensure-job-result-artifacts : Ensure job console (job-output.txt.gz) exists in Logserver] 2026-06-10 10:34:42.573311 | controller | ok: OK (4491 bytes) 2026-06-10 10:34:42.583937 | 2026-06-10 10:34:42.584079 | TASK [health-check/ensure-job-result-artifacts : Ensure internal logserver rewrite from job-output.txt to job-output.txt.gz] 2026-06-10 10:34:43.071492 | controller | ok: OK (4491 bytes) 2026-06-10 10:34:43.088181 | 2026-06-10 10:34:43.088313 | TASK [health-check/config-update-zuul : Get Zuul projects list] 2026-06-10 10:34:43.551562 | controller | ok: OK (144 bytes) 2026-06-10 10:34:43.561620 | 2026-06-10 10:34:43.561759 | TASK [health-check/config-update-zuul : Ensure project exists in Zuul (fail if not)] 2026-06-10 10:34:43.610760 | controller | skipping: Conditional result was False 2026-06-10 10:34:43.630315 | 2026-06-10 10:34:43.630517 | TASK [health-check/check-logjuicer : Get logjuicer status] 2026-06-10 10:34:44.087257 | controller | ok: OK (2 bytes) 2026-06-10 10:34:44.104285 | 2026-06-10 10:34:44.104465 | TASK [ansible.builtin.include_role : health-check/check-zuul-tenant] 2026-06-10 10:34:44.140025 | controller | ok 2026-06-10 10:34:44.167069 | 2026-06-10 10:34:44.167160 | TASK [health-check/check-zuul-tenant : Get configuration errors/warnings into the demo-tenant tenant] 2026-06-10 10:34:44.632023 | controller | ok: OK (2 bytes) 2026-06-10 10:34:44.643895 | 2026-06-10 10:34:44.644173 | TASK [health-check/check-zuul-tenant : Fail when errors/warnings into the demo-tenant tenant] 2026-06-10 10:34:44.686330 | controller | skipping: Conditional result was False 2026-06-10 10:34:44.703421 | 2026-06-10 10:34:44.703570 | TASK [health-check/zuul-demo-tenant-workflow : Fetch and reset demo-project to latest origin for fast-forward submit] 2026-06-10 06:34:44.980577 | controller | HEAD is now at 766b7e1 Automatic secret re-encryption 2026-06-10 10:34:45.247795 | controller | ok: Runtime: 0:00:00.055604 2026-06-10 10:34:45.259601 | 2026-06-10 10:34:45.259849 | TASK [health-check/zuul-demo-tenant-workflow : Update the demo-job to sleep 1 second] 2026-06-10 10:34:45.722551 | controller | changed: 1 replacements made 2026-06-10 10:34:45.731755 | 2026-06-10 10:34:45.731897 | LOOP [health-check/zuul-demo-tenant-workflow : Commit file in the demo-project repo] 2026-06-10 10:34:46.012224 | controller | ok: Item: git add playbooks/run.yaml Runtime: 0:00:00.005030 2026-06-10 10:34:46.012709 | controller | changed: All items complete 2026-06-10 10:34:46.012767 | 2026-06-10 10:34:46.224096 | controller | [master ead9062] Update demo-job to sleep 1 second 2026-06-10 10:34:46.224352 | controller | 1 file changed, 1 insertion(+), 1 deletion(-) 2026-06-10 10:34:46.224422 | controller | ok: Item: git commit -m 'Update demo-job to sleep 1 second' Runtime: 0:00:00.010042 2026-06-10 10:34:46.232182 | 2026-06-10 10:34:46.232328 | TASK [Submit, merge and verify post-merge job] 2026-06-10 10:34:46.262286 | controller | ok 2026-06-10 10:34:46.279597 | 2026-06-10 10:34:46.279662 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 10:34:46.310487 | controller | ok 2026-06-10 10:34:46.316350 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 10:34:46.321600 | 2026-06-10 10:34:46.321668 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 06:34:47.189953 | controller | Your change was committed before the commit hook was installed. 2026-06-10 06:34:47.190029 | controller | Amending the commit to add a gerrit change id. 2026-06-10 06:34:47.190041 | controller | remote: 2026-06-10 06:34:47.190051 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 06:34:47.190064 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 06:34:47.190073 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-10 06:34:47.190083 | controller | remote: 2026-06-10 06:34:47.190091 | controller | remote: SUCCESS 2026-06-10 06:34:47.190099 | controller | remote: 2026-06-10 06:34:47.190107 | controller | remote: https://gerrit.sfop.me/c/demo-project/+/2 Update demo-job to sleep 1 second [NEW] 2026-06-10 06:34:47.190117 | controller | remote: 2026-06-10 06:34:47.190125 | controller | To https://gerrit.sfop.me/a/demo-project 2026-06-10 06:34:47.190133 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 10:34:47.355958 | controller | ok: Runtime: 0:00:00.670602 2026-06-10 10:34:47.365637 | 2026-06-10 10:34:47.365804 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-10 06:34:47.657658 | controller | I0e202bd5b8707003ac191d895789eeea14e742e0 2026-06-10 10:34:47.913825 | controller | ok: Runtime: 0:00:00.007277 2026-06-10 10:34:47.923617 | 2026-06-10 10:34:47.923787 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 10:34:48.449010 | controller | ok: OK (47 bytes) 2026-06-10 10:34:48.461375 | 2026-06-10 10:34:48.461620 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 10:34:48.490729 | controller | skipping: Conditional result was False 2026-06-10 10:34:48.503803 | 2026-06-10 10:34:48.503985 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 10:34:48.542115 | controller | skipping: Conditional result was False 2026-06-10 10:34:48.557591 | 2026-06-10 10:34:48.557747 | TASK [health-check/repo-submit-change : Ensure a success for publish-job post job] 2026-06-10 10:34:48.684227 | controller | ok 2026-06-10 10:34:48.689845 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-post-job-success.yaml 2026-06-10 10:34:48.695880 | 2026-06-10 10:34:48.695944 | TASK [health-check/repo-submit-change : Register commit sha] 2026-06-10 06:34:48.918195 | controller | a25a7368bfdfc8fa839e3c794d359889f555067a 2026-06-10 10:34:49.227323 | controller | ok: Runtime: 0:00:00.004398 2026-06-10 10:34:49.240108 | 2026-06-10 10:34:49.240293 | TASK [health-check/repo-submit-change : Wait publish-job SUCCESS result using zuul-web API] 2026-06-10 10:35:31.476198 | controller | ok: OK (927 bytes) 2026-06-10 10:35:31.489274 | 2026-06-10 10:35:31.489413 | TASK [health-check/repo-submit-change : Ensure a success for publish-job job] 2026-06-10 10:35:31.527383 | controller | skipping: Conditional result was False 2026-06-10 10:35:31.557573 | 2026-06-10 10:35:31.557746 | TASK [health-check/zuul-executor-ensure-graceful-stop : Scale down executor statefulset to ensure 1 replica only] 2026-06-10 06:35:31.849172 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 10:35:32.096134 | controller | ok: Runtime: 0:00:00.072403 2026-06-10 10:35:32.105831 | 2026-06-10 10:35:32.105973 | TASK [health-check/zuul-executor-ensure-graceful-stop : Wait for only 1 executor running] 2026-06-10 06:35:32.388743 | controller | 2 2026-06-10 10:35:42.652674 | controller | ok: Runtime: 0:00:00.025318 2026-06-10 10:35:42.662312 | 2026-06-10 10:35:42.662499 | TASK [health-check/zuul-executor-ensure-graceful-stop : Ensure we are on the default branch] 2026-06-10 06:35:42.958882 | controller | Already on 'master' 2026-06-10 06:35:42.959553 | controller | Your branch is ahead of 'origin/master' by 1 commit. 2026-06-10 06:35:42.959568 | controller | (use "git push" to publish your local commits) 2026-06-10 10:35:43.206642 | controller | ok: Runtime: 0:00:00.010174 2026-06-10 10:35:43.216160 | 2026-06-10 10:35:43.216299 | TASK [health-check/zuul-executor-ensure-graceful-stop : Create a test branch] 2026-06-10 06:35:43.455988 | controller | Switched to a new branch 'test_check_url' 2026-06-10 10:35:43.764482 | controller | ok: Runtime: 0:00:00.008150 2026-06-10 10:35:43.777803 | 2026-06-10 10:35:43.777952 | TASK [health-check/zuul-executor-ensure-graceful-stop : Replace demo-job with demo-job-check-url] 2026-06-10 10:35:44.047493 | controller | changed: 2 replacements made 2026-06-10 10:35:44.056779 | 2026-06-10 10:35:44.056951 | TASK [health-check/zuul-executor-ensure-graceful-stop : create the review] 2026-06-10 10:35:44.092618 | controller | ok 2026-06-10 10:35:44.111293 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/zuul-executor-ensure-graceful-stop/tasks/create-review.yaml 2026-06-10 10:35:44.124895 | 2026-06-10 10:35:44.124983 | LOOP [health-check/zuul-executor-ensure-graceful-stop : Create review] 2026-06-10 10:35:44.439539 | controller | ok: Item: git add --all Runtime: 0:00:00.005619 2026-06-10 10:35:44.689777 | controller | [test_check_url 99ae6f0] test check url 2026-06-10 10:35:44.689928 | controller | 1 file changed, 2 insertions(+), 2 deletions(-) 2026-06-10 10:35:44.689985 | controller | ok: Item: git commit -m 'test check url' Runtime: 0:00:00.045111 2026-06-10 10:35:45.348897 | controller | remote: 2026-06-10 10:35:45.349135 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 10:35:45.349201 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 10:35:45.349252 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 10:35:45.349299 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-10 10:35:45.349345 | controller | remote: 2026-06-10 10:35:45.349392 | controller | remote: SUCCESS 2026-06-10 10:35:45.349752 | controller | remote: 2026-06-10 10:35:45.349802 | controller | remote: https://gerrit.sfop.me/c/demo-project/+/3 test check url [NEW] 2026-06-10 10:35:45.349840 | controller | remote: 2026-06-10 10:35:45.349876 | controller | To https://gerrit.sfop.me/a/demo-project 2026-06-10 10:35:45.349912 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 10:35:45.349963 | controller | ok: Item: git review Runtime: 0:00:00.454388 2026-06-10 10:35:45.371503 | 2026-06-10 10:35:45.371722 | TASK [health-check/zuul-executor-ensure-graceful-stop : Get Change-id] 2026-06-10 06:35:45.667028 | controller | I4dcdbf27b7a8a45922e1f23be7976e3789ba3368 2026-06-10 10:35:45.917684 | controller | ok: Runtime: 0:00:00.008288 2026-06-10 10:35:45.929787 | 2026-06-10 10:35:45.930011 | TASK [health-check/zuul-executor-ensure-graceful-stop : ansible.builtin.debug] 2026-06-10 10:35:45.977141 | controller | ok 2026-06-10 10:35:45.986892 | 2026-06-10 10:35:45.987023 | TASK [health-check/zuul-executor-ensure-graceful-stop : Get review number] 2026-06-10 06:35:46.294813 | controller | 3 2026-06-10 10:35:46.529940 | controller | ok: Runtime: 0:00:00.035549 2026-06-10 10:35:46.539621 | 2026-06-10 10:35:46.539762 | TASK [health-check/zuul-executor-ensure-graceful-stop : ansible.builtin.debug] 2026-06-10 10:35:46.585918 | controller | ok 2026-06-10 10:35:46.595007 | 2026-06-10 10:35:46.595149 | TASK [health-check/zuul-executor-ensure-graceful-stop : ansible.builtin.set_fact] 2026-06-10 10:35:46.641203 | controller | ok 2026-06-10 10:35:46.652930 | 2026-06-10 10:35:46.653070 | TASK [health-check/zuul-executor-ensure-graceful-stop : Wait for job to start] 2026-06-10 06:35:47.019740 | controller | null 2026-06-10 10:35:57.312384 | controller | ok: Runtime: 0:00:00.038969 2026-06-10 10:35:57.398449 | 2026-06-10 10:35:57.398607 | TASK [health-check/zuul-executor-ensure-graceful-stop : Scale down executor statefulset to trigger a graceful stop] 2026-06-10 06:35:57.686743 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 10:35:57.928018 | controller | ok: Runtime: 0:00:00.060158 2026-06-10 10:35:57.937198 | 2026-06-10 10:35:57.937333 | TASK [health-check/zuul-executor-ensure-graceful-stop : Ensure executor component is marked for shutdown in Zuul API] 2026-06-10 06:35:58.230955 | controller | paused 2026-06-10 10:35:58.485682 | controller | ok: Runtime: 0:00:00.049904 2026-06-10 10:35:58.495241 | 2026-06-10 10:35:58.495395 | TASK [health-check/zuul-executor-ensure-graceful-stop : show executor state] 2026-06-10 10:35:58.530424 | controller | ok: paused 2026-06-10 10:35:58.538894 | 2026-06-10 10:35:58.539027 | TASK [health-check/zuul-executor-ensure-graceful-stop : Get back on default branch] 2026-06-10 06:35:58.819976 | controller | Switched to branch 'master' 2026-06-10 06:35:58.820504 | controller | Your branch is ahead of 'origin/master' by 1 commit. 2026-06-10 06:35:58.820539 | controller | (use "git push" to publish your local commits) 2026-06-10 10:35:59.084150 | controller | ok: Runtime: 0:00:00.006050 2026-06-10 10:35:59.094761 | 2026-06-10 10:35:59.094937 | TASK [health-check/zuul-executor-ensure-graceful-stop : Create a test branch] 2026-06-10 06:35:59.333730 | controller | Switched to a new branch 'test_short_job' 2026-06-10 10:35:59.644360 | controller | ok: Runtime: 0:00:00.004120 2026-06-10 10:35:59.654707 | 2026-06-10 10:35:59.654842 | TASK [health-check/zuul-executor-ensure-graceful-stop : Update the demo-job to sleep 2 seconds] 2026-06-10 10:35:59.874269 | controller | changed: 1 replacements made 2026-06-10 10:35:59.883878 | 2026-06-10 10:35:59.884009 | TASK [health-check/zuul-executor-ensure-graceful-stop : create the review] 2026-06-10 10:35:59.922205 | controller | ok 2026-06-10 10:35:59.937599 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/zuul-executor-ensure-graceful-stop/tasks/create-review.yaml 2026-06-10 10:35:59.953621 | 2026-06-10 10:35:59.953703 | LOOP [health-check/zuul-executor-ensure-graceful-stop : Create review] 2026-06-10 10:36:00.245561 | controller | ok: Item: git add --all Runtime: 0:00:00.007309 2026-06-10 10:36:00.245891 | controller | changed: All items complete 2026-06-10 10:36:00.245930 | 2026-06-10 10:36:00.521745 | controller | [test_short_job 759998e] test short job 2026-06-10 10:36:00.522076 | controller | 1 file changed, 1 insertion(+), 1 deletion(-) 2026-06-10 10:36:00.522174 | controller | ok: Item: git commit -m 'test short job' Runtime: 0:00:00.035125 2026-06-10 10:36:01.135502 | controller | remote: 2026-06-10 10:36:01.135660 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 10:36:01.135711 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 10:36:01.135750 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 10:36:01.135786 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-10 10:36:01.135822 | controller | remote: 2026-06-10 10:36:01.135856 | controller | remote: SUCCESS 2026-06-10 10:36:01.135889 | controller | remote: 2026-06-10 10:36:01.135924 | controller | remote: https://gerrit.sfop.me/c/demo-project/+/4 test short job [NEW] 2026-06-10 10:36:01.135960 | controller | remote: 2026-06-10 10:36:01.135994 | controller | To https://gerrit.sfop.me/a/demo-project 2026-06-10 10:36:01.136032 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 10:36:01.136082 | controller | ok: Item: git review Runtime: 0:00:00.423059 2026-06-10 10:36:01.149864 | 2026-06-10 10:36:01.150156 | TASK [health-check/zuul-executor-ensure-graceful-stop : Get Change-id] 2026-06-10 06:36:01.398848 | controller | I43de6087067391226072f187e9eb28b3dbf2aedf 2026-06-10 10:36:01.701209 | controller | ok: Runtime: 0:00:00.007540 2026-06-10 10:36:01.711821 | 2026-06-10 10:36:01.712046 | TASK [health-check/zuul-executor-ensure-graceful-stop : ansible.builtin.debug] 2026-06-10 10:36:01.761334 | controller | ok 2026-06-10 10:36:01.770926 | 2026-06-10 10:36:01.771063 | TASK [health-check/zuul-executor-ensure-graceful-stop : Get review number] 2026-06-10 06:36:02.040679 | controller | 4 2026-06-10 10:36:02.312850 | controller | ok: Runtime: 0:00:00.039432 2026-06-10 10:36:02.321826 | 2026-06-10 10:36:02.321967 | TASK [health-check/zuul-executor-ensure-graceful-stop : ansible.builtin.debug] 2026-06-10 10:36:02.367425 | controller | ok 2026-06-10 10:36:02.377843 | 2026-06-10 10:36:02.378049 | TASK [health-check/zuul-executor-ensure-graceful-stop : ansible.builtin.set_fact] 2026-06-10 10:36:02.413540 | controller | ok 2026-06-10 10:36:02.422250 | 2026-06-10 10:36:02.422381 | TASK [health-check/zuul-executor-ensure-graceful-stop : Ensure new review is queued] 2026-06-10 06:36:02.730382 | controller | null 2026-06-10 10:36:02.963080 | controller | ok: Runtime: 0:00:00.055787 2026-06-10 10:36:02.973488 | 2026-06-10 10:36:02.973635 | TASK [Create switch file on logserver so that first job can complete] 2026-06-10 10:36:03.010958 | controller | ok 2026-06-10 10:36:03.052897 | 2026-06-10 10:36:03.053026 | TASK [create-file-on-logserver : copy file on log server] 2026-06-10 06:36:03.416990 | controller | Defaulted container "logserver" out of: logserver, logserver-sshd, purgelogs, logserver-nodeexporter 2026-06-10 10:36:03.649572 | controller | ok: Runtime: 0:00:00.172460 2026-06-10 10:36:03.666711 | 2026-06-10 10:36:03.666851 | TASK [health-check/zuul-executor-ensure-graceful-stop : Wait for executor pod to terminate] 2026-06-10 06:36:04.024031 | controller | NAME READY STATUS RESTARTS AGE 2026-06-10 06:36:04.024112 | controller | config-base-secret-1-2757-ns-fl6pt 0/1 Completed 0 26m 2026-06-10 06:36:04.024119 | controller | config-base-secret-1-859-ns-lbbxd 0/1 Completed 0 37m 2026-06-10 06:36:04.024131 | controller | config-base-secret-1-859-ns-x7j9d 0/1 Completed 0 31m 2026-06-10 06:36:04.024138 | controller | gateway-6bbbd758d9-429qm 1/1 Running 0 33m 2026-06-10 06:36:04.024142 | controller | gerrit-0 2/2 Running 0 43m 2026-06-10 06:36:04.024148 | controller | git-server-0 2/2 Running 0 10m 2026-06-10 06:36:04.024153 | controller | hound-search-0 1/1 Running 0 41m 2026-06-10 06:36:04.024156 | controller | logjuicer-6ff84764b7-klm2b 1/1 Running 0 41m 2026-06-10 06:36:04.024160 | controller | logserver-0 4/4 Running 0 20m 2026-06-10 06:36:04.024164 | controller | mariadb-0 3/3 Running 0 18m 2026-06-10 06:36:04.024167 | controller | nodepool-builder-0 5/5 Running 0 18m 2026-06-10 06:36:04.024171 | controller | nodepool-launcher-64cdb99fcf-dlpv6 3/3 Running 0 18m 2026-06-10 06:36:04.024175 | controller | post-init-7f798 0/1 Completed 0 42m 2026-06-10 06:36:04.024180 | controller | test-fluentbit 1/1 Running 0 35m 2026-06-10 06:36:04.024184 | controller | test-loki-8cd4fd56-g9s64 1/1 Running 0 35m 2026-06-10 06:36:04.024188 | controller | zookeeper-0 3/3 Running 0 18m 2026-06-10 06:36:04.024191 | controller | zuul-db-provision-dm9wg 0/1 Completed 0 28m 2026-06-10 06:36:04.024195 | controller | zuul-db-provision-r5bxv 0/1 Completed 0 40m 2026-06-10 06:36:04.024200 | controller | zuul-executor-0 2/2 Terminating 0 9m23s 2026-06-10 06:36:04.024204 | controller | zuul-merger-0 2/2 Running 0 9m20s 2026-06-10 06:36:04.024208 | controller | zuul-merger-1 2/2 Running 0 9m35s 2026-06-10 06:36:04.024212 | controller | zuul-scheduler-0 3/3 Running 0 5m33s 2026-06-10 06:36:04.024216 | controller | zuul-web-59f755bcf9-n9gnh 1/1 Running 0 5m34s 2026-06-10 06:36:04.024221 | controller | zuul-weeder-5fb87885d-xfx9b 1/1 Running 0 8m34s 2026-06-10 10:36:34.955096 | controller | ok: Runtime: 0:00:00.074893 2026-06-10 10:36:34.966732 | 2026-06-10 10:36:34.966877 | TASK [health-check/zuul-executor-ensure-graceful-stop : Ensure first job completed] 2026-06-10 10:36:35.504041 | controller | ok: Runtime: 0:00:00.054531 2026-06-10 10:36:35.514745 | 2026-06-10 10:36:35.514886 | TASK [health-check/zuul-executor-ensure-graceful-stop : Ensure second job hasn't completed] 2026-06-10 10:36:36.067802 | controller | ok: Runtime: 0:00:00.045116 2026-06-10 10:36:36.077145 | 2026-06-10 10:36:36.077277 | TASK [health-check/zuul-executor-ensure-graceful-stop : Scale up executor statefulset] 2026-06-10 06:36:36.348893 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 10:36:36.625292 | controller | ok: Runtime: 0:00:00.061853 2026-06-10 10:36:36.634934 | 2026-06-10 10:36:36.635071 | TASK [health-check/zuul-executor-ensure-graceful-stop : Wait until executor pod is up] 2026-06-10 06:36:36.911529 | controller | NAME READY STATUS RESTARTS AGE 2026-06-10 06:36:36.911628 | controller | zuul-executor-0 0/2 ContainerCreating 0 0s 2026-06-10 10:36:47.212689 | controller | ok: Runtime: 0:00:00.057295 2026-06-10 10:36:47.224721 | 2026-06-10 10:36:47.224966 | TASK [health-check/zuul-executor-ensure-graceful-stop : Ensure executor component is running] 2026-06-10 06:36:47.524070 | controller | running 2026-06-10 10:36:47.775288 | controller | ok: Runtime: 0:00:00.030004 2026-06-10 10:36:47.784483 | 2026-06-10 10:36:47.784624 | TASK [health-check/zuul-executor-ensure-graceful-stop : Ensure second review's job completed] 2026-06-10 10:37:08.555206 | controller | ok: Runtime: 0:00:00.041746 2026-06-10 10:37:08.566109 | 2026-06-10 10:37:08.566258 | TASK [health-check/test-nodepool-providers-secrets : Ensure an empty /tmp/kube.config] 2026-06-10 10:37:08.794876 | controller | changed 2026-06-10 10:37:08.805101 | 2026-06-10 10:37:08.805243 | TASK [health-check/test-nodepool-providers-secrets : Get existing kube.config from the secret] 2026-06-10 10:37:09.354165 | controller | ok: Runtime: 0:00:00.057780 2026-06-10 10:37:09.363703 | 2026-06-10 10:37:09.363842 | TASK [health-check/test-nodepool-providers-secrets : Create a fake clouds.yaml] 2026-06-10 10:37:10.002939 | controller | changed 2026-06-10 10:37:10.013041 | 2026-06-10 10:37:10.013213 | TASK [health-check/test-nodepool-providers-secrets : Update the secret clouds.yaml key] 2026-06-10 06:37:10.375449 | controller | Warning: resource secrets/nodepool-providers-secrets is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically. 2026-06-10 06:37:10.449971 | controller | secret/nodepool-providers-secrets configured 2026-06-10 10:37:10.559775 | controller | ok: Runtime: 0:00:00.214883 2026-06-10 10:37:10.569101 | 2026-06-10 10:37:10.569237 | TASK [Wait for secrets to be updated] 2026-06-10 10:37:10.604184 | controller | ok 2026-06-10 10:37:10.632626 | 2026-06-10 10:37:10.632769 | TASK [run the operator in standalone mode] 2026-06-10 10:37:10.664534 | controller | ok 2026-06-10 10:37:10.684649 | 2026-06-10 10:37:10.684741 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 10:37:10.720938 | controller | skipping: Conditional result was False 2026-06-10 10:37:10.732397 | 2026-06-10 10:37:10.732686 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 10:37:10.761347 | controller | skipping: Conditional result was False 2026-06-10 10:37:10.773800 | 2026-06-10 10:37:10.773950 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 06:37:11.378438 | controller | 2026-06-10T06:37:11-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:37:11.378486 | controller | 2026-06-10 06:37:11.388382 | controller | 2026-06-10T06:37:11-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:37:11.388406 | controller | 2026-06-10 06:37:11.398569 | controller | 2026-06-10T06:37:11-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:37:12.013368 | controller | 2026-06-10T06:37:12-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:37:12.118048 | controller | 2026-06-10T06:37:12-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:37:12.126132 | controller | 2026-06-10T06:37:12-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:37:12.126154 | controller | 2026-06-10T06:37:12-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:37:12.145686 | controller | 2026-06-10T06:37:12-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:37:12.145717 | controller | 2026-06-10T06:37:12-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:37:12.417318 | controller | 2026-06-10T06:37:12-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:37:12.803598 | controller | 2026-06-10T06:37:12-04:00 INFO Updating configmap, name: np-statsd-config-map 2026-06-10 06:37:12.803660 | controller | 2026-06-10T06:37:12-04:00 INFO Updating object, name: np-statsd-config-map 2026-06-10 06:37:12.824370 | controller | 2026-06-10T06:37:12-04:00 INFO Statefulset "nodepool-launcher" configuration changed, applying... 2026-06-10 06:37:12.824423 | controller | 2026-06-10T06:37:12-04:00 INFO Updating object, name: nodepool-launcher 2026-06-10 06:37:13.402701 | controller | 2026-06-10T06:37:13-04:00 INFO Statefulset "nodepool-builder" configuration changed, applying... 2026-06-10 06:37:13.402773 | controller | 2026-06-10T06:37:13-04:00 INFO Updating object, name: nodepool-builder 2026-06-10 06:37:14.211068 | controller | 2026-06-10T06:37:14-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:37:14.211113 | controller | 2026-06-10T06:37:14-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:37:15.800366 | controller | 2026-06-10T06:37:15-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:37:15.800419 | controller | 2026-06-10T06:37:15-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:37:17.013599 | controller | 2026-06-10T06:37:17-04:00 INFO 2026-06-10 06:37:17.013723 | controller | - Config: OK 2026-06-10 06:37:17.013736 | controller | - Gateway: OK 2026-06-10 06:37:17.013745 | controller | - GitServer: OK 2026-06-10 06:37:17.013755 | controller | - HoundSearch: OK 2026-06-10 06:37:17.013766 | controller | - LogJuicer: OK 2026-06-10 06:37:17.013776 | controller | - Logserver: OK 2026-06-10 06:37:17.013783 | controller | - MariaDB: OK 2026-06-10 06:37:17.013791 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:37:17.013800 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:37:17.013808 | controller | - Zookeeper: OK 2026-06-10 06:37:17.013816 | controller | - Zuul: OK 2026-06-10 06:37:17.013824 | controller | 2026-06-10 06:37:17.013833 | controller | 2026-06-10T06:37:17-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 06:37:22.019111 | controller | 2026-06-10T06:37:22-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:37:22.633726 | controller | 2026-06-10T06:37:22-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:37:22.633782 | controller | 2026-06-10T06:37:22-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:37:22.651024 | controller | 2026-06-10T06:37:22-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:37:22.651104 | controller | 2026-06-10T06:37:22-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:37:23.031828 | controller | 2026-06-10T06:37:23-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:37:23.430061 | controller | 2026-06-10T06:37:23-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:37:23.430691 | controller | 2026-06-10T06:37:23-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 06:37:24.438119 | controller | 2026-06-10T06:37:24-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:37:24.438161 | controller | 2026-06-10T06:37:24-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:37:26.022669 | controller | 2026-06-10T06:37:26-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:37:26.022727 | controller | 2026-06-10T06:37:26-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:37:27.228937 | controller | 2026-06-10T06:37:27-04:00 INFO 2026-06-10 06:37:27.228996 | controller | - Config: OK 2026-06-10 06:37:27.229002 | controller | - Gateway: OK 2026-06-10 06:37:27.229013 | controller | - GitServer: OK 2026-06-10 06:37:27.229025 | controller | - HoundSearch: OK 2026-06-10 06:37:27.229030 | controller | - LogJuicer: OK 2026-06-10 06:37:27.229035 | controller | - Logserver: OK 2026-06-10 06:37:27.229039 | controller | - MariaDB: OK 2026-06-10 06:37:27.229042 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:37:27.229046 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:37:27.229050 | controller | - Zookeeper: OK 2026-06-10 06:37:27.229053 | controller | - Zuul: OK 2026-06-10 06:37:27.229057 | controller | 2026-06-10 06:37:27.229061 | controller | 2026-06-10T06:37:27-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 06:37:32.237674 | controller | 2026-06-10T06:37:32-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:37:32.851803 | controller | 2026-06-10T06:37:32-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:37:32.851889 | controller | 2026-06-10T06:37:32-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:37:32.870509 | controller | 2026-06-10T06:37:32-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:37:32.870585 | controller | 2026-06-10T06:37:32-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:37:33.254726 | controller | 2026-06-10T06:37:33-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:37:33.655593 | controller | 2026-06-10T06:37:33-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:37:33.655687 | controller | 2026-06-10T06:37:33-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 06:37:34.667740 | controller | 2026-06-10T06:37:34-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:37:34.667811 | controller | 2026-06-10T06:37:34-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:37:36.242182 | controller | 2026-06-10T06:37:36-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:37:36.242263 | controller | 2026-06-10T06:37:36-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:37:37.445623 | controller | 2026-06-10T06:37:37-04:00 INFO 2026-06-10 06:37:37.445681 | controller | - Config: OK 2026-06-10 06:37:37.445694 | controller | - Gateway: OK 2026-06-10 06:37:37.445700 | controller | - GitServer: OK 2026-06-10 06:37:37.445705 | controller | - HoundSearch: OK 2026-06-10 06:37:37.445710 | controller | - LogJuicer: OK 2026-06-10 06:37:37.445715 | controller | - Logserver: OK 2026-06-10 06:37:37.445720 | controller | - MariaDB: OK 2026-06-10 06:37:37.445733 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:37:37.445740 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:37:37.445745 | controller | - Zookeeper: OK 2026-06-10 06:37:37.445751 | controller | - Zuul: OK 2026-06-10 06:37:37.445756 | controller | 2026-06-10 06:37:37.445767 | controller | 2026-06-10T06:37:37-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 06:37:42.452198 | controller | 2026-06-10T06:37:42-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:37:43.072902 | controller | 2026-06-10T06:37:43-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:37:43.072970 | controller | 2026-06-10T06:37:43-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:37:43.095175 | controller | 2026-06-10T06:37:43-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:37:43.095234 | controller | 2026-06-10T06:37:43-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:37:43.472591 | controller | 2026-06-10T06:37:43-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:37:43.874311 | controller | 2026-06-10T06:37:43-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:37:43.874381 | controller | 2026-06-10T06:37:43-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 06:37:44.871047 | controller | 2026-06-10T06:37:44-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:37:44.871106 | controller | 2026-06-10T06:37:44-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:37:46.455084 | controller | 2026-06-10T06:37:46-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:37:46.455131 | controller | 2026-06-10T06:37:46-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:37:47.662807 | controller | 2026-06-10T06:37:47-04:00 INFO 2026-06-10 06:37:47.662873 | controller | - Config: OK 2026-06-10 06:37:47.662881 | controller | - Gateway: OK 2026-06-10 06:37:47.662887 | controller | - GitServer: OK 2026-06-10 06:37:47.662893 | controller | - HoundSearch: OK 2026-06-10 06:37:47.662898 | controller | - LogJuicer: OK 2026-06-10 06:37:47.662903 | controller | - Logserver: OK 2026-06-10 06:37:47.662908 | controller | - MariaDB: OK 2026-06-10 06:37:47.662914 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:37:47.662919 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:37:47.662926 | controller | - Zookeeper: OK 2026-06-10 06:37:47.662931 | controller | - Zuul: OK 2026-06-10 06:37:47.662937 | controller | 2026-06-10 06:37:47.662942 | controller | 2026-06-10T06:37:47-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-10 06:37:52.668946 | controller | 2026-06-10T06:37:52-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:37:53.287098 | controller | 2026-06-10T06:37:53-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:37:53.287142 | controller | 2026-06-10T06:37:53-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:37:53.388532 | controller | 2026-06-10T06:37:53-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:37:53.388573 | controller | 2026-06-10T06:37:53-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:37:53.679529 | controller | 2026-06-10T06:37:53-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:37:54.083890 | controller | 2026-06-10T06:37:54-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:37:54.083939 | controller | 2026-06-10T06:37:54-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:37:55.089586 | controller | 2026-06-10T06:37:55-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:37:55.089644 | controller | 2026-06-10T06:37:55-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:37:56.672821 | controller | 2026-06-10T06:37:56-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:37:56.672884 | controller | 2026-06-10T06:37:56-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:37:57.877461 | controller | 2026-06-10T06:37:57-04:00 INFO 2026-06-10 06:37:57.877508 | controller | - Config: OK 2026-06-10 06:37:57.877515 | controller | - Gateway: OK 2026-06-10 06:37:57.877520 | controller | - GitServer: OK 2026-06-10 06:37:57.877524 | controller | - HoundSearch: OK 2026-06-10 06:37:57.877528 | controller | - LogJuicer: OK 2026-06-10 06:37:57.877532 | controller | - Logserver: OK 2026-06-10 06:37:57.877535 | controller | - MariaDB: OK 2026-06-10 06:37:57.877539 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:37:57.877543 | controller | - NodePoolLauncher: OK 2026-06-10 06:37:57.877546 | controller | - Zookeeper: OK 2026-06-10 06:37:57.877550 | controller | - Zuul: OK 2026-06-10 06:37:57.877554 | controller | 2026-06-10 06:37:57.877564 | controller | 2026-06-10T06:37:57-04:00 INFO [attempt #5] Waiting 5s for the next reconcile call ... 2026-06-10 06:38:02.885148 | controller | 2026-06-10T06:38:02-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:38:03.501391 | controller | 2026-06-10T06:38:03-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:38:03.501445 | controller | 2026-06-10T06:38:03-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:38:03.517516 | controller | 2026-06-10T06:38:03-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:38:03.517568 | controller | 2026-06-10T06:38:03-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:38:03.903082 | controller | 2026-06-10T06:38:03-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:38:04.304744 | controller | 2026-06-10T06:38:04-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:38:04.304789 | controller | 2026-06-10T06:38:04-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:38:05.308195 | controller | 2026-06-10T06:38:05-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:38:05.308260 | controller | 2026-06-10T06:38:05-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:38:06.890852 | controller | 2026-06-10T06:38:06-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:38:06.890944 | controller | 2026-06-10T06:38:06-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:38:08.094324 | controller | 2026-06-10T06:38:08-04:00 INFO 2026-06-10 06:38:08.094368 | controller | - Config: OK 2026-06-10 06:38:08.094385 | controller | - Gateway: OK 2026-06-10 06:38:08.094389 | controller | - GitServer: OK 2026-06-10 06:38:08.094393 | controller | - HoundSearch: OK 2026-06-10 06:38:08.094397 | controller | - LogJuicer: OK 2026-06-10 06:38:08.094400 | controller | - Logserver: OK 2026-06-10 06:38:08.094404 | controller | - MariaDB: OK 2026-06-10 06:38:08.094407 | controller | - NodePoolBuilder: OK 2026-06-10 06:38:08.094411 | controller | - NodePoolLauncher: OK 2026-06-10 06:38:08.094414 | controller | - Zookeeper: OK 2026-06-10 06:38:08.094418 | controller | - Zuul: OK 2026-06-10 06:38:08.094422 | controller | 2026-06-10 06:38:08.094427 | controller | 2026-06-10T06:38:08-04:00 INFO Updating controller configmap ... 2026-06-10 06:38:08.487666 | controller | 2026-06-10T06:38:08-04:00 INFO Standalone reconcile done. 2026-06-10 10:38:08.956788 | controller | ok: Runtime: 0:00:57.406663 2026-06-10 10:38:08.964979 | 2026-06-10 10:38:08.965076 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 10:38:09.503078 | controller | skipping: Conditional result was False 2026-06-10 10:38:09.530909 | 2026-06-10 10:38:09.531063 | TASK [health-check/test-nodepool-providers-secrets : Check that clouds.yaml available in nodepool-builder] 2026-06-10 06:38:09.842583 | controller | Defaulted container "nodepool-builder" out of: nodepool-builder, fluentbit, np-statsd, nodepool-builder-nodeexporter, build-logs-httpd, nodepool-builder-init (init) 2026-06-10 06:38:09.922525 | controller | /var/lib/nodepool/.config/openstack/clouds.yaml 2026-06-10 10:38:10.077707 | controller | ok: Runtime: 0:00:00.141085 2026-06-10 10:38:10.087049 | 2026-06-10 10:38:10.087190 | TASK [health-check/test-nodepool-providers-secrets : Check that clouds.yaml available in nodepool-launcher containers] 2026-06-10 06:38:10.449976 | controller | Defaulted container "launcher" out of: launcher, np-statsd, zuul-capacity, nodepool-launcher-init (init) 2026-06-10 06:38:10.528600 | controller | /var/lib/nodepool/.config/openstack/clouds.yaml 2026-06-10 10:38:10.633945 | controller | ok: Runtime: 0:00:00.209770 2026-06-10 10:38:10.644863 | 2026-06-10 10:38:10.644964 | TASK [health-check/config-update-nodepool-launcher : Set random value] 2026-06-10 10:38:10.753245 | controller | ok 2026-06-10 10:38:10.765024 | 2026-06-10 10:38:10.765164 | TASK [health-check/config-update-nodepool-launcher : Ensure nodepool directory exists] 2026-06-10 10:38:11.017258 | controller | changed 2026-06-10 10:38:11.027340 | 2026-06-10 10:38:11.027511 | TASK [health-check/config-update-nodepool-launcher : Set bad Nodepool config] 2026-06-10 10:38:11.258375 | controller | changed: File created 2026-06-10 10:38:11.267772 | 2026-06-10 10:38:11.267910 | LOOP [health-check/config-update-nodepool-launcher : Commit file in the config repo] 2026-06-10 10:38:11.525634 | controller | ok: Item: git add nodepool/nodepool.yaml Runtime: 0:00:00.003798 2026-06-10 10:38:11.752409 | controller | [master 58f1568] Add nodepool/nodepool.yaml 2026-06-10 10:38:11.752671 | controller | 1 file changed, 3 insertions(+) 2026-06-10 10:38:11.752731 | controller | create mode 100644 nodepool/nodepool.yaml 2026-06-10 10:38:11.752790 | controller | ok: Item: git commit -m 'Add nodepool/nodepool.yaml' Runtime: 0:00:00.036838 2026-06-10 10:38:11.773698 | 2026-06-10 10:38:11.773895 | TASK [ansible.builtin.include_role : health-check/repo-submit-change] 2026-06-10 10:38:11.800564 | controller | ok 2026-06-10 10:38:11.837770 | 2026-06-10 10:38:11.837880 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 10:38:11.862548 | controller | ok 2026-06-10 10:38:11.870919 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 10:38:11.878498 | 2026-06-10 10:38:11.878588 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 06:38:12.475952 | controller | remote: 2026-06-10 06:38:12.476032 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 06:38:12.476042 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-10 06:38:12.476048 | controller | remote: 2026-06-10 06:38:12.476054 | controller | remote: SUCCESS 2026-06-10 06:38:12.476059 | controller | remote: 2026-06-10 06:38:12.476067 | controller | remote: https://gerrit.sfop.me/c/config/+/5 Add nodepool/nodepool.yaml [NEW] 2026-06-10 06:38:12.476075 | controller | remote: 2026-06-10 06:38:12.476080 | controller | To https://gerrit.sfop.me/a/config 2026-06-10 06:38:12.476088 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 10:38:12.917774 | controller | ok: Runtime: 0:00:00.392089 2026-06-10 10:38:12.927211 | 2026-06-10 10:38:12.927353 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-10 06:38:13.166341 | controller | I91764376f6d264f756f8898d4dab38c93af99147 2026-06-10 10:38:13.469075 | controller | ok: Runtime: 0:00:00.008092 2026-06-10 10:38:13.479197 | 2026-06-10 10:38:13.479349 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 10:38:13.964206 | controller | ok: OK (47 bytes) 2026-06-10 10:38:13.981578 | 2026-06-10 10:38:13.981754 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 10:38:14.020574 | controller | ok 2026-06-10 10:38:14.036142 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-job-failure.yaml 2026-06-10 10:38:14.047133 | 2026-06-10 10:38:14.047235 | TASK [health-check/repo-submit-change : Get review ID from the changeID using gerrit API] 2026-06-10 10:38:14.558460 | controller | ok: OK (7831 bytes) 2026-06-10 10:38:14.569361 | 2026-06-10 10:38:14.569546 | TASK [health-check/repo-submit-change : ansible.builtin.set_fact] 2026-06-10 10:38:14.639140 | controller | ok 2026-06-10 10:38:14.656776 | 2026-06-10 10:38:14.656930 | TASK [health-check/repo-submit-change : Wait config-check FAILURE result using zuul-web API] 2026-06-10 10:38:56.854631 | controller | ok: OK (1946 bytes) 2026-06-10 10:38:56.866647 | 2026-06-10 10:38:56.866784 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 10:38:56.900904 | controller | skipping: Conditional result was False 2026-06-10 10:38:56.918492 | 2026-06-10 10:38:56.918647 | TASK [health-check/repo-submit-change : Ensure a success for config-update post job] 2026-06-10 10:38:56.948254 | controller | skipping: Conditional result was False 2026-06-10 10:38:56.964541 | 2026-06-10 10:38:56.964739 | TASK [health-check/repo-submit-change : Ensure a success for config-update job] 2026-06-10 10:38:56.992996 | controller | skipping: Conditional result was False 2026-06-10 10:38:57.013699 | 2026-06-10 10:38:57.013862 | TASK [health-check/config-update-nodepool-launcher : Reset the config repo] 2026-06-10 06:38:57.379815 | controller | HEAD is now at 2ce8c3d Add zuul/main.yaml 2026-06-10 10:38:57.682631 | controller | ok: Runtime: 0:00:00.005310 2026-06-10 10:38:57.692230 | 2026-06-10 10:38:57.692376 | TASK [health-check/config-update-nodepool-launcher : Ensure nodepool directory exists] 2026-06-10 10:38:57.951012 | controller | changed 2026-06-10 10:38:57.961797 | 2026-06-10 10:38:57.961931 | TASK [health-check/config-update-nodepool-launcher : Check nodepool.yaml exists] 2026-06-10 10:38:58.237237 | controller | ok 2026-06-10 10:38:58.241838 | 2026-06-10 10:38:58.241905 | TASK [health-check/config-update-nodepool-launcher : Ensure nodepool.yaml default content] 2026-06-10 10:38:58.985513 | controller | changed 2026-06-10 10:38:58.995263 | 2026-06-10 10:38:58.995527 | TASK [health-check/config-update-nodepool-launcher : Load current nodepool launcher config] 2026-06-10 06:38:59.235199 | controller | labels: [] 2026-06-10 06:38:59.235255 | controller | providers: [] 2026-06-10 10:38:59.544578 | controller | ok: Runtime: 0:00:00.003515 2026-06-10 10:38:59.553868 | 2026-06-10 10:38:59.554020 | TASK [health-check/config-update-nodepool-launcher : ansible.builtin.set_fact] 2026-06-10 10:38:59.589607 | controller | ok 2026-06-10 10:38:59.598610 | 2026-06-10 10:38:59.598749 | TASK [health-check/config-update-nodepool-launcher : ansible.builtin.set_fact] 2026-06-10 10:38:59.656544 | controller | ok 2026-06-10 10:38:59.665407 | 2026-06-10 10:38:59.665600 | TASK [health-check/config-update-nodepool-launcher : ansible.builtin.set_fact] 2026-06-10 10:38:59.703125 | controller | skipping: Conditional result was False 2026-06-10 10:38:59.713840 | 2026-06-10 10:38:59.713981 | TASK [health-check/config-update-nodepool-launcher : ansible.builtin.set_fact] 2026-06-10 10:38:59.780562 | controller | ok 2026-06-10 10:38:59.789838 | 2026-06-10 10:38:59.789986 | TASK [health-check/config-update-nodepool-launcher : ansible.builtin.set_fact] 2026-06-10 10:38:59.827302 | controller | skipping: Conditional result was False 2026-06-10 10:38:59.839010 | 2026-06-10 10:38:59.839206 | TASK [health-check/config-update-nodepool-launcher : ansible.builtin.set_fact] 2026-06-10 10:38:59.885612 | controller | ok 2026-06-10 10:38:59.898372 | 2026-06-10 10:38:59.898641 | TASK [health-check/config-update-nodepool-launcher : ansible.builtin.set_fact] 2026-06-10 10:38:59.935855 | controller | ok 2026-06-10 10:38:59.944839 | 2026-06-10 10:38:59.944973 | TASK [health-check/config-update-nodepool-launcher : Set dummy Nodepool config] 2026-06-10 10:39:00.578595 | controller | changed 2026-06-10 10:39:00.587844 | 2026-06-10 10:39:00.587982 | LOOP [health-check/config-update-nodepool-launcher : Commit file in the config repo] 2026-06-10 10:39:00.873352 | controller | ok: Item: git add nodepool/nodepool.yaml Runtime: 0:00:00.005964 2026-06-10 10:39:00.873645 | controller | changed: All items complete 2026-06-10 10:39:00.873674 | 2026-06-10 10:39:01.101857 | controller | [master 7a9d647] Add nodepool/nodepool.yaml 2026-06-10 10:39:01.102161 | controller | 1 file changed, 12 insertions(+) 2026-06-10 10:39:01.102192 | controller | create mode 100644 nodepool/nodepool.yaml 2026-06-10 10:39:01.102220 | controller | ok: Item: git commit -m 'Add nodepool/nodepool.yaml' Runtime: 0:00:00.036236 2026-06-10 10:39:01.130964 | 2026-06-10 10:39:01.131398 | TASK [Update config repo change] 2026-06-10 10:39:01.171370 | controller | ok 2026-06-10 10:39:01.214055 | 2026-06-10 10:39:01.214157 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 10:39:01.247600 | controller | ok 2026-06-10 10:39:01.259552 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 10:39:01.267373 | 2026-06-10 10:39:01.267487 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 06:39:01.911753 | controller | remote: 2026-06-10 06:39:01.911815 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 06:39:01.911822 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 06:39:01.911826 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-10 06:39:01.911831 | controller | remote: 2026-06-10 06:39:01.911836 | controller | remote: SUCCESS 2026-06-10 06:39:01.911841 | controller | remote: 2026-06-10 06:39:01.911847 | controller | remote: https://gerrit.sfop.me/c/config/+/6 Add nodepool/nodepool.yaml [NEW] 2026-06-10 06:39:01.911851 | controller | remote: 2026-06-10 06:39:01.911855 | controller | To https://gerrit.sfop.me/a/config 2026-06-10 06:39:01.911862 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 10:39:02.306171 | controller | ok: Runtime: 0:00:00.416024 2026-06-10 10:39:02.317498 | 2026-06-10 10:39:02.317650 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-10 06:39:02.577051 | controller | Icb3e7fdcda045e29aa4fdb891719f4a7ac4dc04c 2026-06-10 10:39:02.863014 | controller | ok: Runtime: 0:00:00.006120 2026-06-10 10:39:02.874051 | 2026-06-10 10:39:02.874288 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 10:39:03.358392 | controller | ok: OK (47 bytes) 2026-06-10 10:39:03.371915 | 2026-06-10 10:39:03.372148 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 10:39:03.400844 | controller | skipping: Conditional result was False 2026-06-10 10:39:03.415058 | 2026-06-10 10:39:03.415325 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 10:39:03.443986 | controller | skipping: Conditional result was False 2026-06-10 10:39:03.458745 | 2026-06-10 10:39:03.459017 | TASK [health-check/repo-submit-change : Ensure a success for config-update post job] 2026-06-10 10:39:03.508938 | controller | ok 2026-06-10 10:39:03.521855 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-post-job-success.yaml 2026-06-10 10:39:03.534107 | 2026-06-10 10:39:03.534197 | TASK [health-check/repo-submit-change : Register commit sha] 2026-06-10 06:39:03.770409 | controller | 7a9d647d8f016eee2173764f8bdeb80a89aa2d4b 2026-06-10 10:39:04.070056 | controller | ok: Runtime: 0:00:00.007129 2026-06-10 10:39:04.081905 | 2026-06-10 10:39:04.082046 | TASK [health-check/repo-submit-change : Wait config-update SUCCESS result using zuul-web API] 2026-06-10 10:40:59.310376 | controller | ok: OK (1143 bytes) 2026-06-10 10:40:59.323571 | 2026-06-10 10:40:59.323721 | TASK [health-check/repo-submit-change : Ensure a success for config-update job] 2026-06-10 10:40:59.363691 | controller | skipping: Conditional result was False 2026-06-10 10:40:59.387668 | 2026-06-10 10:40:59.387821 | TASK [health-check/config-update-nodepool-launcher : Get nodepool-launcher pod name] 2026-06-10 06:40:59.671651 | controller | nodepool-launcher-f8d9dc5c7-5vksr 2026-06-10 10:40:59.930098 | controller | ok: Runtime: 0:00:00.056375 2026-06-10 10:40:59.937857 | 2026-06-10 10:40:59.937951 | LOOP [health-check/config-update-nodepool-launcher : Check that new zuul-worker-sf-operator-ci-a3yhd6vz label exists in the destination file] 2026-06-10 10:41:00.300003 | controller | - name: zuul-worker-sf-operator-ci-a3yhd6vz 2026-06-10 10:41:00.300523 | controller | name: zuul-worker-sf-operator-ci-a3yhd6vz 2026-06-10 10:41:00.300636 | controller | ok: Item: zuul-worker-sf-operator-ci-a3yhd6vz Runtime: 0:00:00.126746 2026-06-10 10:41:00.658171 | controller | name: microshiftLocal 2026-06-10 10:41:00.658378 | controller | ok: Item: microshiftLocal Runtime: 0:00:00.170489 2026-06-10 10:41:00.670958 | 2026-06-10 10:41:00.671174 | TASK [health-check/config-update-nodepool-launcher : Ensure the new label listed in the Nodepool API] 2026-06-10 10:41:01.173421 | controller | ok: OK (199 bytes) 2026-06-10 10:41:01.191837 | 2026-06-10 10:41:01.191984 | TASK [health-check/config-update-nodepool-launcher : Ensure the new label listed in the Zuul API] 2026-06-10 10:41:01.663634 | controller | ok: OK (49 bytes) 2026-06-10 10:41:01.681777 | 2026-06-10 10:41:01.681940 | TASK [health-check/test-deploy-zero-replica : Set executor replica set to 0] 2026-06-10 06:41:02.023905 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 10:41:02.291405 | controller | ok: Runtime: 0:00:00.055402 2026-06-10 10:41:02.309406 | 2026-06-10 10:41:02.309593 | TASK [health-check/test-deploy-zero-replica : Ensure executor pod is down] 2026-06-10 06:41:02.598621 | controller | NAME READY STATUS RESTARTS AGE 2026-06-10 06:41:02.598708 | controller | config-base-secret-1-2757-ns-fl6pt 0/1 Completed 0 31m 2026-06-10 06:41:02.598727 | controller | config-base-secret-1-859-ns-lbbxd 0/1 Completed 0 42m 2026-06-10 06:41:02.598738 | controller | config-base-secret-1-859-ns-x7j9d 0/1 Completed 0 36m 2026-06-10 06:41:02.598746 | controller | gateway-6bbbd758d9-429qm 1/1 Running 0 38m 2026-06-10 06:41:02.598750 | controller | gerrit-0 2/2 Running 0 48m 2026-06-10 06:41:02.598757 | controller | git-server-0 2/2 Running 0 15m 2026-06-10 06:41:02.598761 | controller | hound-search-0 1/1 Running 0 46m 2026-06-10 06:41:02.598765 | controller | logjuicer-6ff84764b7-klm2b 1/1 Running 0 46m 2026-06-10 06:41:02.598769 | controller | logserver-0 4/4 Running 0 25m 2026-06-10 06:41:02.598773 | controller | mariadb-0 3/3 Running 0 23m 2026-06-10 06:41:02.598776 | controller | nodepool-builder-0 5/5 Running 0 3m48s 2026-06-10 06:41:02.598780 | controller | nodepool-launcher-f8d9dc5c7-5vksr 3/3 Running 0 3m50s 2026-06-10 06:41:02.598783 | controller | post-init-7f798 0/1 Completed 0 47m 2026-06-10 06:41:02.598790 | controller | test-fluentbit 1/1 Running 0 40m 2026-06-10 06:41:02.598794 | controller | test-loki-8cd4fd56-g9s64 1/1 Running 0 40m 2026-06-10 06:41:02.598797 | controller | zookeeper-0 3/3 Running 0 23m 2026-06-10 06:41:02.598801 | controller | zuul-db-provision-dm9wg 0/1 Completed 0 33m 2026-06-10 06:41:02.598805 | controller | zuul-db-provision-r5bxv 0/1 Completed 0 45m 2026-06-10 06:41:02.598808 | controller | zuul-executor-0 2/2 Terminating 0 4m26s 2026-06-10 06:41:02.598812 | controller | zuul-merger-0 2/2 Running 0 14m 2026-06-10 06:41:02.598815 | controller | zuul-merger-1 2/2 Running 0 14m 2026-06-10 06:41:02.598819 | controller | zuul-scheduler-0 3/3 Running 0 10m 2026-06-10 06:41:02.598823 | controller | zuul-web-59f755bcf9-n9gnh 1/1 Running 0 10m 2026-06-10 06:41:02.598826 | controller | zuul-weeder-5fb87885d-xfx9b 1/1 Running 0 13m 2026-06-10 10:41:12.916923 | controller | ok: Runtime: 0:00:00.091106 2026-06-10 10:41:12.926660 | 2026-06-10 10:41:12.926820 | TASK [health-check/test-deploy-zero-replica : Set zuul-web replica set to 0] 2026-06-10 06:41:13.241950 | controller | deployment.apps/zuul-web scaled 2026-06-10 10:41:13.475179 | controller | ok: Runtime: 0:00:00.063931 2026-06-10 10:41:13.484853 | 2026-06-10 10:41:13.484992 | TASK [health-check/test-deploy-zero-replica : Ensure zuul-web pod is down] 2026-06-10 06:41:13.806066 | controller | NAME READY STATUS RESTARTS AGE 2026-06-10 06:41:13.806157 | controller | config-base-secret-1-2757-ns-fl6pt 0/1 Completed 0 32m 2026-06-10 06:41:13.806167 | controller | config-base-secret-1-859-ns-lbbxd 0/1 Completed 0 42m 2026-06-10 06:41:13.806175 | controller | config-base-secret-1-859-ns-x7j9d 0/1 Completed 0 36m 2026-06-10 06:41:13.806189 | controller | gateway-6bbbd758d9-429qm 1/1 Running 0 38m 2026-06-10 06:41:13.806194 | controller | gerrit-0 2/2 Running 0 48m 2026-06-10 06:41:13.806201 | controller | git-server-0 2/2 Running 0 15m 2026-06-10 06:41:13.806207 | controller | hound-search-0 1/1 Running 0 46m 2026-06-10 06:41:13.806211 | controller | logjuicer-6ff84764b7-klm2b 1/1 Running 0 46m 2026-06-10 06:41:13.806216 | controller | logserver-0 4/4 Running 0 25m 2026-06-10 06:41:13.806221 | controller | mariadb-0 3/3 Running 0 23m 2026-06-10 06:41:13.806225 | controller | nodepool-builder-0 5/5 Running 0 3m59s 2026-06-10 06:41:13.806230 | controller | nodepool-launcher-f8d9dc5c7-5vksr 3/3 Running 0 4m1s 2026-06-10 06:41:13.806235 | controller | post-init-7f798 0/1 Completed 0 47m 2026-06-10 06:41:13.806242 | controller | test-fluentbit 1/1 Running 0 40m 2026-06-10 06:41:13.806247 | controller | test-loki-8cd4fd56-g9s64 1/1 Running 0 40m 2026-06-10 06:41:13.806252 | controller | zookeeper-0 3/3 Running 0 23m 2026-06-10 06:41:13.806256 | controller | zuul-db-provision-dm9wg 0/1 Completed 0 33m 2026-06-10 06:41:13.806261 | controller | zuul-db-provision-r5bxv 0/1 Completed 0 46m 2026-06-10 06:41:13.806265 | controller | zuul-merger-0 2/2 Running 0 14m 2026-06-10 06:41:13.806271 | controller | zuul-merger-1 2/2 Running 0 14m 2026-06-10 06:41:13.806276 | controller | zuul-scheduler-0 3/3 Running 0 10m 2026-06-10 06:41:13.806281 | controller | zuul-web-59f755bcf9-n9gnh 1/1 Terminating 0 10m 2026-06-10 06:41:13.806286 | controller | zuul-weeder-5fb87885d-xfx9b 1/1 Running 0 13m 2026-06-10 10:41:24.174489 | controller | ok: Runtime: 0:00:00.114671 2026-06-10 10:41:24.184421 | 2026-06-10 10:41:24.184630 | TASK [health-check/test-deploy-zero-replica : Run sf-operator and expect reconciliation] 2026-06-10 06:41:24.644016 | controller | 2026-06-10T06:41:24-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:41:24.644101 | controller | 2026-06-10 06:41:24.652850 | controller | 2026-06-10T06:41:24-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:41:24.652923 | controller | 2026-06-10 06:41:24.666141 | controller | 2026-06-10T06:41:24-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:41:24.690320 | controller | 2026-06-10T06:41:24-04:00 INFO Statefulset "mariadb" configuration changed, applying... 2026-06-10 06:41:24.690388 | controller | 2026-06-10T06:41:24-04:00 INFO Updating object, name: mariadb 2026-06-10 06:41:25.077136 | controller | 2026-06-10T06:41:25-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:41:25.162235 | controller | 2026-06-10T06:41:25-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:41:25.169628 | controller | 2026-06-10T06:41:25-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:41:25.169693 | controller | 2026-06-10T06:41:25-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:41:25.186910 | controller | 2026-06-10T06:41:25-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:41:25.186960 | controller | 2026-06-10T06:41:25-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:41:25.472746 | controller | 2026-06-10T06:41:25-04:00 INFO Statefulset "zookeeper" configuration changed, applying... 2026-06-10 06:41:25.472811 | controller | 2026-06-10T06:41:25-04:00 INFO Updating object, name: zookeeper 2026-06-10 06:41:25.481234 | controller | 2026-06-10T06:41:25-04:00 INFO Waiting for Zookeeper service to be ready ... 2026-06-10 06:41:25.481296 | controller | 2026-06-10T06:41:25-04:00 INFO 2026-06-10 06:41:25.481318 | controller | - Config: Waiting ... 2026-06-10 06:41:25.481323 | controller | - Gateway: OK 2026-06-10 06:41:25.481328 | controller | - GitServer: OK 2026-06-10 06:41:25.481334 | controller | - HoundSearch: OK 2026-06-10 06:41:25.481339 | controller | - LogJuicer: OK 2026-06-10 06:41:25.481344 | controller | - Logserver: OK 2026-06-10 06:41:25.481349 | controller | - MariaDB: Waiting ... 2026-06-10 06:41:25.481354 | controller | - Zookeeper: Waiting ... 2026-06-10 06:41:25.481359 | controller | 2026-06-10 06:41:25.481369 | controller | 2026-06-10T06:41:25-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 06:41:30.488417 | controller | 2026-06-10T06:41:30-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:41:30.518092 | controller | 2026-06-10T06:41:30-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-10 06:41:38.565997 | controller | 2026-06-10T06:41:38-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:41:38.566084 | controller | 2026-06-10T06:41:38-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:41:38.577105 | controller | 2026-06-10T06:41:38-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:41:38.577168 | controller | 2026-06-10T06:41:38-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:41:38.592226 | controller | 2026-06-10T06:41:38-04:00 INFO Waiting for statefulset, name: zookeeper 2026-06-10 06:41:40.598112 | controller | 2026-06-10T06:41:40-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:41:40.607792 | controller | 2026-06-10T06:41:40-04:00 INFO Updating configmap, name: nodepool-launcher-extra-config-config-map 2026-06-10 06:41:40.607864 | controller | 2026-06-10T06:41:40-04:00 INFO Updating object, name: nodepool-launcher-extra-config-config-map 2026-06-10 06:41:40.616807 | controller | 2026-06-10T06:41:40-04:00 INFO Statefulset "nodepool-launcher" configuration changed, applying... 2026-06-10 06:41:40.616917 | controller | 2026-06-10T06:41:40-04:00 INFO Updating object, name: nodepool-launcher 2026-06-10 06:41:40.632441 | controller | 2026-06-10T06:41:40-04:00 INFO Updating configmap, name: nodepool-builder-extra-config-config-map 2026-06-10 06:41:40.632526 | controller | 2026-06-10T06:41:40-04:00 INFO Updating object, name: nodepool-builder-extra-config-config-map 2026-06-10 06:41:40.649530 | controller | 2026-06-10T06:41:40-04:00 INFO Statefulset "nodepool-builder" configuration changed, applying... 2026-06-10 06:41:40.649595 | controller | 2026-06-10T06:41:40-04:00 INFO Updating object, name: nodepool-builder 2026-06-10 06:41:40.699656 | controller | 2026-06-10T06:41:40-04:00 INFO Updating secret, name: zuul-config 2026-06-10 06:41:40.699739 | controller | 2026-06-10T06:41:40-04:00 INFO Updating object, name: zuul-config 2026-06-10 06:41:41.204732 | controller | 2026-06-10T06:41:41-04:00 INFO Updating configmap, name: zuul-logging-config-map 2026-06-10 06:41:41.204812 | controller | 2026-06-10T06:41:41-04:00 INFO Updating object, name: zuul-logging-config-map 2026-06-10 06:41:41.410178 | controller | 2026-06-10T06:41:41-04:00 INFO Statefulset "zuul-scheduler" configuration changed, applying... 2026-06-10 06:41:41.410255 | controller | 2026-06-10T06:41:41-04:00 INFO Updating object, name: zuul-scheduler 2026-06-10 06:41:42.210210 | controller | 2026-06-10T06:41:42-04:00 INFO Statefulset "zuul-executor" configuration changed, applying... 2026-06-10 06:41:42.210265 | controller | 2026-06-10T06:41:42-04:00 INFO Updating object, name: zuul-executor 2026-06-10 06:41:43.008005 | controller | 2026-06-10T06:41:43-04:00 INFO Statefulset "zuul-web" configuration changed, applying... 2026-06-10 06:41:43.008074 | controller | 2026-06-10T06:41:43-04:00 INFO Updating object, name: zuul-web 2026-06-10 06:41:43.808462 | controller | 2026-06-10T06:41:43-04:00 INFO Statefulset "zuul-merger" configuration changed, applying... 2026-06-10 06:41:43.808534 | controller | 2026-06-10T06:41:43-04:00 INFO Updating object, name: zuul-merger 2026-06-10 06:41:43.816191 | controller | 2026-06-10T06:41:43-04:00 INFO 2026-06-10 06:41:43.816245 | controller | - Config: Waiting ... 2026-06-10 06:41:43.816257 | controller | - Gateway: OK 2026-06-10 06:41:43.816262 | controller | - GitServer: OK 2026-06-10 06:41:43.816267 | controller | - HoundSearch: OK 2026-06-10 06:41:43.816273 | controller | - LogJuicer: OK 2026-06-10 06:41:43.816278 | controller | - Logserver: OK 2026-06-10 06:41:43.816281 | controller | - MariaDB: OK 2026-06-10 06:41:43.816285 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:41:43.816289 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:41:43.816292 | controller | - Zookeeper: OK 2026-06-10 06:41:43.816296 | controller | - Zuul: Waiting ... 2026-06-10 06:41:43.816299 | controller | - |_ Config Secrets: OK 2026-06-10 06:41:43.816303 | controller | - |_ Executor: Waiting ... 2026-06-10 06:41:43.816306 | controller | - |_ Merger: Waiting ... 2026-06-10 06:41:43.816310 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:41:43.816313 | controller | - |_ Web: Waiting ... 2026-06-10 06:41:43.816317 | controller | 2026-06-10 06:41:43.816321 | controller | 2026-06-10T06:41:43-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 06:41:48.966185 | controller | 2026-06-10T06:41:48-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:41:49.436314 | controller | 2026-06-10T06:41:49-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:41:49.436387 | controller | 2026-06-10T06:41:49-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:41:49.449074 | controller | 2026-06-10T06:41:49-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:41:49.449140 | controller | 2026-06-10T06:41:49-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:41:49.831392 | controller | 2026-06-10T06:41:49-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:41:50.243043 | controller | 2026-06-10T06:41:50-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:41:50.243108 | controller | 2026-06-10T06:41:50-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 06:41:51.242366 | controller | 2026-06-10T06:41:51-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:42:21.341094 | controller | 2026-06-10T06:42:21-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 06:42:21.364123 | controller | 2026-06-10T06:42:21-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:42:21.754104 | controller | 2026-06-10T06:42:21-04:00 INFO 2026-06-10 06:42:21.754179 | controller | - Config: Waiting ... 2026-06-10 06:42:21.754218 | controller | - Gateway: OK 2026-06-10 06:42:21.754224 | controller | - GitServer: OK 2026-06-10 06:42:21.754229 | controller | - HoundSearch: OK 2026-06-10 06:42:21.754232 | controller | - LogJuicer: OK 2026-06-10 06:42:21.754236 | controller | - Logserver: OK 2026-06-10 06:42:21.754239 | controller | - MariaDB: OK 2026-06-10 06:42:21.754244 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:42:21.754249 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:42:21.754254 | controller | - Zookeeper: OK 2026-06-10 06:42:21.754259 | controller | - Zuul: Waiting ... 2026-06-10 06:42:21.754264 | controller | - |_ Config Secrets: OK 2026-06-10 06:42:21.754268 | controller | - |_ Executor: OK 2026-06-10 06:42:21.754273 | controller | - |_ Merger: OK 2026-06-10 06:42:21.754278 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:42:21.754294 | controller | - |_ Web: OK 2026-06-10 06:42:21.754299 | controller | 2026-06-10 06:42:21.754304 | controller | 2026-06-10T06:42:21-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 06:42:26.758308 | controller | 2026-06-10T06:42:26-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:42:27.369954 | controller | 2026-06-10T06:42:27-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:42:27.370024 | controller | 2026-06-10T06:42:27-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:42:27.383634 | controller | 2026-06-10T06:42:27-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:42:27.383699 | controller | 2026-06-10T06:42:27-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:42:27.769232 | controller | 2026-06-10T06:42:27-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:42:28.168374 | controller | 2026-06-10T06:42:28-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:42:28.168440 | controller | 2026-06-10T06:42:28-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:42:29.177794 | controller | 2026-06-10T06:42:29-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:42:47.241720 | controller | 2026-06-10T06:42:47-04:00 INFO Statefulset "zuul-weeder" configuration changed, applying... 2026-06-10 06:42:47.242046 | controller | 2026-06-10T06:42:47-04:00 INFO Updating object, name: zuul-weeder 2026-06-10 06:42:47.269988 | controller | 2026-06-10T06:42:47-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:42:47.658506 | controller | 2026-06-10T06:42:47-04:00 INFO 2026-06-10 06:42:47.658571 | controller | - Config: Waiting ... 2026-06-10 06:42:47.658577 | controller | - Gateway: OK 2026-06-10 06:42:47.658581 | controller | - GitServer: OK 2026-06-10 06:42:47.658585 | controller | - HoundSearch: OK 2026-06-10 06:42:47.658589 | controller | - LogJuicer: OK 2026-06-10 06:42:47.658592 | controller | - Logserver: OK 2026-06-10 06:42:47.658596 | controller | - MariaDB: OK 2026-06-10 06:42:47.658599 | controller | - NodePoolBuilder: OK 2026-06-10 06:42:47.658627 | controller | - NodePoolLauncher: OK 2026-06-10 06:42:47.658631 | controller | - Zookeeper: OK 2026-06-10 06:42:47.658635 | controller | - Zuul: Waiting ... 2026-06-10 06:42:47.658638 | controller | - |_ Config Secrets: OK 2026-06-10 06:42:47.658642 | controller | - |_ Executor: OK 2026-06-10 06:42:47.658646 | controller | - |_ Merger: OK 2026-06-10 06:42:47.658649 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:42:47.658653 | controller | - |_ Web: OK 2026-06-10 06:42:47.658657 | controller | 2026-06-10 06:42:47.658683 | controller | 2026-06-10T06:42:47-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-10 06:42:52.668455 | controller | 2026-06-10T06:42:52-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:42:53.279307 | controller | 2026-06-10T06:42:53-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:42:53.279407 | controller | 2026-06-10T06:42:53-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:42:53.290752 | controller | 2026-06-10T06:42:53-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:42:53.290834 | controller | 2026-06-10T06:42:53-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:42:53.676533 | controller | 2026-06-10T06:42:53-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:42:54.081255 | controller | 2026-06-10T06:42:54-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:42:54.081341 | controller | 2026-06-10T06:42:54-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:42:55.092031 | controller | 2026-06-10T06:42:55-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:42:55.092153 | controller | 2026-06-10T06:42:55-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:42:56.071850 | controller | 2026-06-10T06:42:56-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:42:57.280138 | controller | 2026-06-10T06:42:57-04:00 INFO 2026-06-10 06:42:57.280202 | controller | - Config: OK 2026-06-10 06:42:57.280207 | controller | - Gateway: OK 2026-06-10 06:42:57.280211 | controller | - GitServer: OK 2026-06-10 06:42:57.280215 | controller | - HoundSearch: OK 2026-06-10 06:42:57.280219 | controller | - LogJuicer: OK 2026-06-10 06:42:57.280222 | controller | - Logserver: OK 2026-06-10 06:42:57.280226 | controller | - MariaDB: OK 2026-06-10 06:42:57.280230 | controller | - NodePoolBuilder: OK 2026-06-10 06:42:57.280233 | controller | - NodePoolLauncher: OK 2026-06-10 06:42:57.280249 | controller | - Zookeeper: OK 2026-06-10 06:42:57.280253 | controller | - Zuul: OK 2026-06-10 06:42:57.280257 | controller | 2026-06-10 06:42:57.280282 | controller | 2026-06-10T06:42:57-04:00 INFO Updating controller configmap ... 2026-06-10 06:42:57.670356 | controller | 2026-06-10T06:42:57-04:00 INFO Standalone reconcile done. 2026-06-10 10:42:57.887116 | controller | ok: Runtime: 0:01:33.255259 2026-06-10 10:42:57.899256 | 2026-06-10 10:42:57.899503 | TASK [health-check/test-deploy-zero-replica : Set executor replica set to 1] 2026-06-10 06:42:58.205241 | controller | statefulset.apps/zuul-executor scaled 2026-06-10 10:42:58.447401 | controller | ok: Runtime: 0:00:00.078194 2026-06-10 10:42:58.456288 | 2026-06-10 10:42:58.456394 | TASK [health-check/test-deploy-zero-replica : Set web replica set to 1] 2026-06-10 06:42:58.755588 | controller | deployment.apps/zuul-web scaled 2026-06-10 10:42:58.994916 | controller | ok: Runtime: 0:00:00.068782 2026-06-10 10:42:59.005362 | 2026-06-10 10:42:59.005570 | TASK [health-check/test-deploy-zero-replica : Ensure executor pod is up] 2026-06-10 06:42:59.316894 | controller | NAME READY STATUS RESTARTS AGE 2026-06-10 06:42:59.317011 | controller | zuul-executor-0 0/2 ContainerCreating 0 1s 2026-06-10 10:43:19.845575 | controller | ok: Runtime: 0:00:00.052241 2026-06-10 10:43:19.855686 | 2026-06-10 10:43:19.855826 | TASK [health-check/test-deploy-zero-replica : Ensure web pod is up] 2026-06-10 06:43:20.184647 | controller | NAME READY STATUS RESTARTS AGE 2026-06-10 06:43:20.184792 | controller | gateway-6bbbd758d9-429qm 1/1 Running 0 40m 2026-06-10 06:43:20.184872 | controller | gerrit-0 2/2 Running 0 50m 2026-06-10 06:43:20.184943 | controller | git-server-0 2/2 Running 0 17m 2026-06-10 06:43:20.185008 | controller | hound-search-0 1/1 Running 0 48m 2026-06-10 06:43:20.185070 | controller | logjuicer-6ff84764b7-klm2b 1/1 Running 0 48m 2026-06-10 06:43:20.185140 | controller | logserver-0 4/4 Running 0 28m 2026-06-10 06:43:20.185217 | controller | mariadb-0 2/2 Running 0 114s 2026-06-10 06:43:20.185283 | controller | nodepool-builder-0 4/4 Running 0 98s 2026-06-10 06:43:20.185337 | controller | nodepool-launcher-5bd775d78f-jsvp5 3/3 Running 0 100s 2026-06-10 06:43:20.185400 | controller | test-fluentbit 1/1 Running 0 42m 2026-06-10 06:43:20.185459 | controller | test-loki-8cd4fd56-g9s64 1/1 Running 0 42m 2026-06-10 06:43:20.185527 | controller | zookeeper-0 2/2 Running 0 113s 2026-06-10 06:43:20.185596 | controller | zuul-executor-0 2/2 Running 0 22s 2026-06-10 06:43:20.185709 | controller | zuul-merger-0 2/2 Running 0 78s 2026-06-10 06:43:20.185779 | controller | zuul-merger-1 2/2 Running 0 94s 2026-06-10 06:43:20.185855 | controller | zuul-scheduler-0 3/3 Running 0 96s 2026-06-10 06:43:20.185928 | controller | zuul-web-d948bf696-rqc4v 0/1 Running 0 22s 2026-06-10 06:43:20.186001 | controller | zuul-weeder-57c6567f5f-rvgsd 1/1 Running 0 33s 2026-06-10 10:43:20.404315 | controller | ok: Runtime: 0:00:00.074006 2026-06-10 10:43:20.416691 | 2026-06-10 10:43:20.416832 | TASK [Run dry-run update detection test] 2026-06-10 10:43:20.456252 | controller | ok 2026-06-10 10:43:20.495955 | 2026-06-10 10:43:20.496054 | TASK [test-dry-run-update-detection : Fetch current serial for zuul-executor] 2026-06-10 10:43:21.032412 | controller | ok: Runtime: 0:00:00.055662 2026-06-10 10:43:21.042786 | 2026-06-10 10:43:21.042943 | TASK [test-dry-run-update-detection : Fetch current serial for zuul-web] 2026-06-10 10:43:21.593117 | controller | ok: Runtime: 0:00:00.054385 2026-06-10 10:43:21.602324 | 2026-06-10 10:43:21.602499 | TASK [test-dry-run-update-detection : Patch zuul-executor statefulset template to trigger reconciliation] 2026-06-10 06:43:21.893464 | controller | statefulset.apps/zuul-executor patched 2026-06-10 10:43:22.150140 | controller | ok: Runtime: 0:00:00.064027 2026-06-10 10:43:22.159518 | 2026-06-10 10:43:22.159649 | TASK [test-dry-run-update-detection : Patch zuul-web deployment template to trigger reconciliation] 2026-06-10 06:43:22.471090 | controller | deployment.apps/zuul-web patched 2026-06-10 10:43:22.707459 | controller | ok: Runtime: 0:00:00.067703 2026-06-10 10:43:22.717991 | 2026-06-10 10:43:22.718120 | TASK [Run deploy with --dry-run to check for update detection] 2026-06-10 10:43:22.751594 | controller | ok 2026-06-10 10:43:22.775743 | 2026-06-10 10:43:22.775854 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 10:43:23.013853 | controller | ok 2026-06-10 10:43:23.023503 | 2026-06-10 10:43:23.023638 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 10:43:23.273026 | controller | ok 2026-06-10 10:43:23.282842 | 2026-06-10 10:43:23.282975 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 06:43:23.877624 | controller | 2026-06-10T06:43:23-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:43:23.877701 | controller | 2026-06-10 06:43:23.885650 | controller | 2026-06-10T06:43:23-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:43:23.885704 | controller | 2026-06-10 06:43:23.895940 | controller | 2026-06-10T06:43:23-04:00 INFO Deleting object, name: ca-cert (server dry run) 2026-06-10 06:43:24.505793 | controller | 2026-06-10T06:43:24-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:43:24.601597 | controller | 2026-06-10T06:43:24-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:43:24.904275 | controller | 2026-06-10T06:43:24-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:43:26.697678 | controller | 2026-06-10T06:43:26-04:00 INFO [Dry Run] Statefulset "zuul-executor" configuration changed, applying... 2026-06-10 06:43:26.697740 | controller | 2026-06-10T06:43:26-04:00 INFO [Dry Run] Reason: annotation 'serial' changed from '11' to '0' 2026-06-10 06:43:26.697752 | controller | 2026-06-10T06:43:26-04:00 INFO Updating object, name: zuul-executor (server dry run) 2026-06-10 06:43:27.506419 | controller | 2026-06-10T06:43:27-04:00 INFO [Dry Run] Statefulset "zuul-web" configuration changed, applying... 2026-06-10 06:43:27.506519 | controller | 2026-06-10T06:43:27-04:00 INFO [Dry Run] Reason: annotation 'serial' changed from '9' to '0' 2026-06-10 06:43:27.506530 | controller | 2026-06-10T06:43:27-04:00 INFO Updating object, name: zuul-web (server dry run) 2026-06-10 06:43:28.299384 | controller | 2026-06-10T06:43:28-04:00 INFO 2026-06-10 06:43:28.299449 | controller | - Config: Waiting ... 2026-06-10 06:43:28.299454 | controller | - Gateway: OK 2026-06-10 06:43:28.299458 | controller | - GitServer: OK 2026-06-10 06:43:28.299462 | controller | - HoundSearch: OK 2026-06-10 06:43:28.299465 | controller | - LogJuicer: OK 2026-06-10 06:43:28.299469 | controller | - Logserver: OK 2026-06-10 06:43:28.299472 | controller | - MariaDB: OK 2026-06-10 06:43:28.299476 | controller | - NodePoolBuilder: OK 2026-06-10 06:43:28.299480 | controller | - NodePoolLauncher: OK 2026-06-10 06:43:28.299483 | controller | - Zookeeper: OK 2026-06-10 06:43:28.299487 | controller | - Zuul: Waiting ... 2026-06-10 06:43:28.299490 | controller | - |_ Config Secrets: OK 2026-06-10 06:43:28.299494 | controller | - |_ Executor: Waiting ... 2026-06-10 06:43:28.299497 | controller | - |_ Merger: OK 2026-06-10 06:43:28.299501 | controller | - |_ Scheduler: OK 2026-06-10 06:43:28.299504 | controller | - |_ Web: Waiting ... 2026-06-10 06:43:28.299508 | controller | 2026-06-10 06:43:28.299513 | controller | 2026-06-10T06:43:28-04:00 INFO [Dry Run] Standalone reconcile done 2026-06-10 10:43:28.408907 | controller | ok: Runtime: 0:00:04.706089 2026-06-10 10:43:28.419142 | 2026-06-10 10:43:28.419293 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 10:43:28.968802 | controller | ok: Runtime: 0:00:00.005083 2026-06-10 10:43:28.989862 | 2026-06-10 10:43:28.990141 | TASK [test-dry-run-update-detection : Check for 'update' dry-run message in the log for zuul-executor] 2026-06-10 06:43:29.217257 | controller | 2026-06-10T06:43:26-04:00 INFO [Dry Run] Reason: annotation 'serial' changed from '11' to '0' 2026-06-10 10:43:29.535701 | controller | ok: Runtime: 0:00:00.005722 2026-06-10 10:43:29.547200 | 2026-06-10 10:43:29.547340 | TASK [test-dry-run-update-detection : Check for 'update' dry-run message in the log for zuul-web] 2026-06-10 06:43:29.800667 | controller | 2026-06-10T06:43:27-04:00 INFO [Dry Run] Reason: annotation 'serial' changed from '9' to '0' 2026-06-10 10:43:30.094169 | controller | ok: Runtime: 0:00:00.006444 2026-06-10 10:43:30.104132 | 2026-06-10 10:43:30.104288 | TASK [test-dry-run-update-detection : Restore original serial for zuul-executor] 2026-06-10 06:43:30.439021 | controller | statefulset.apps/zuul-executor patched 2026-06-10 10:43:30.649066 | controller | ok: Runtime: 0:00:00.080154 2026-06-10 10:43:30.657704 | 2026-06-10 10:43:30.657822 | TASK [test-dry-run-update-detection : Restore original serial for zuul-web] 2026-06-10 06:43:30.947276 | controller | deployment.apps/zuul-web patched 2026-06-10 10:43:31.200148 | controller | ok: Runtime: 0:00:00.066560 2026-06-10 10:43:31.226953 | 2026-06-10 10:43:31.227190 | TASK [health-check/test-dry-run-lifecycle : Create a temporary directory for generated CRs] 2026-06-10 10:43:31.446698 | controller | changed 2026-06-10 10:43:31.456358 | 2026-06-10 10:43:31.456551 | TASK [health-check/test-dry-run-lifecycle : Generate CR with zuul-executor disabled] 2026-06-10 10:43:32.173787 | controller | changed 2026-06-10 10:43:32.188359 | 2026-06-10 10:43:32.188555 | TASK [Deploy SF without zuul-executor to ensure its removal] 2026-06-10 10:43:32.233041 | controller | ok 2026-06-10 10:43:32.268765 | 2026-06-10 10:43:32.268974 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 10:43:32.568977 | controller | ok 2026-06-10 10:43:32.580026 | 2026-06-10 10:43:32.580223 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 10:43:32.846159 | controller | changed 2026-06-10 10:43:32.856577 | 2026-06-10 10:43:32.856711 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 06:43:33.324680 | controller | 2026-06-10T06:43:33-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:43:33.324789 | controller | 2026-06-10 06:43:33.333011 | controller | 2026-06-10T06:43:33-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:43:33.333057 | controller | 2026-06-10 06:43:33.343661 | controller | 2026-06-10T06:43:33-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:43:33.955826 | controller | 2026-06-10T06:43:33-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:43:34.060322 | controller | 2026-06-10T06:43:34-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:43:34.066908 | controller | 2026-06-10T06:43:34-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:43:34.066954 | controller | 2026-06-10T06:43:34-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:43:34.080143 | controller | 2026-06-10T06:43:34-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:43:34.080200 | controller | 2026-06-10T06:43:34-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:43:34.356185 | controller | 2026-06-10T06:43:34-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:43:34.761068 | controller | 2026-06-10T06:43:34-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:43:34.761177 | controller | 2026-06-10T06:43:34-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:43:34.963219 | controller | 2026-06-10T06:43:34-04:00 INFO zuul-executor is disabled but running. Deleting the executor ... 2026-06-10 06:43:34.963291 | controller | 2026-06-10T06:43:34-04:00 INFO Deleting object, name: zuul-executor 2026-06-10 06:43:35.760834 | controller | 2026-06-10T06:43:35-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:43:35.760890 | controller | 2026-06-10T06:43:35-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:43:36.144630 | controller | 2026-06-10T06:43:36-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:43:36.144698 | controller | 2026-06-10T06:43:36-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:43:36.950534 | controller | 2026-06-10T06:43:36-04:00 INFO 2026-06-10 06:43:36.950604 | controller | - Config: Waiting ... 2026-06-10 06:43:36.950617 | controller | - Gateway: OK 2026-06-10 06:43:36.950656 | controller | - GitServer: OK 2026-06-10 06:43:36.950677 | controller | - HoundSearch: OK 2026-06-10 06:43:36.950688 | controller | - LogJuicer: OK 2026-06-10 06:43:36.950696 | controller | - Logserver: OK 2026-06-10 06:43:36.950703 | controller | - MariaDB: OK 2026-06-10 06:43:36.950711 | controller | - NodePoolBuilder: OK 2026-06-10 06:43:36.950717 | controller | - NodePoolLauncher: OK 2026-06-10 06:43:36.950725 | controller | - Zookeeper: OK 2026-06-10 06:43:36.950732 | controller | - Zuul: Waiting ... 2026-06-10 06:43:36.950739 | controller | - |_ Config Secrets: OK 2026-06-10 06:43:36.950746 | controller | - |_ Merger: OK 2026-06-10 06:43:36.950753 | controller | - |_ Scheduler: OK 2026-06-10 06:43:36.950761 | controller | - |_ Web: Waiting ... 2026-06-10 06:43:36.950768 | controller | 2026-06-10 06:43:36.950784 | controller | 2026-06-10T06:43:36-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 06:43:41.961036 | controller | 2026-06-10T06:43:41-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:43:42.577443 | controller | 2026-06-10T06:43:42-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:43:42.577539 | controller | 2026-06-10T06:43:42-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:43:42.596490 | controller | 2026-06-10T06:43:42-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:43:42.596556 | controller | 2026-06-10T06:43:42-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:43:42.968437 | controller | 2026-06-10T06:43:42-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:43:43.368642 | controller | 2026-06-10T06:43:43-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:43:43.368711 | controller | 2026-06-10T06:43:43-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:43:44.378503 | controller | 2026-06-10T06:43:44-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:43:44.378547 | controller | 2026-06-10T06:43:44-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:43:44.563350 | controller | 2026-06-10T06:43:44-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:43:44.563426 | controller | 2026-06-10T06:43:44-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:43:45.375220 | controller | 2026-06-10T06:43:45-04:00 INFO 2026-06-10 06:43:45.375271 | controller | - Config: Waiting ... 2026-06-10 06:43:45.375278 | controller | - Gateway: OK 2026-06-10 06:43:45.375284 | controller | - GitServer: OK 2026-06-10 06:43:45.375291 | controller | - HoundSearch: OK 2026-06-10 06:43:45.375296 | controller | - LogJuicer: OK 2026-06-10 06:43:45.375301 | controller | - Logserver: OK 2026-06-10 06:43:45.375308 | controller | - MariaDB: OK 2026-06-10 06:43:45.375316 | controller | - NodePoolBuilder: OK 2026-06-10 06:43:45.375323 | controller | - NodePoolLauncher: OK 2026-06-10 06:43:45.375332 | controller | - Zookeeper: OK 2026-06-10 06:43:45.375339 | controller | - Zuul: Waiting ... 2026-06-10 06:43:45.375347 | controller | - |_ Config Secrets: OK 2026-06-10 06:43:45.375355 | controller | - |_ Merger: OK 2026-06-10 06:43:45.375362 | controller | - |_ Scheduler: OK 2026-06-10 06:43:45.375369 | controller | - |_ Web: Waiting ... 2026-06-10 06:43:45.375377 | controller | 2026-06-10 06:43:45.375385 | controller | 2026-06-10T06:43:45-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 06:43:50.382825 | controller | 2026-06-10T06:43:50-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:43:50.994446 | controller | 2026-06-10T06:43:50-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:43:50.994502 | controller | 2026-06-10T06:43:50-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:43:51.007135 | controller | 2026-06-10T06:43:51-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:43:51.007182 | controller | 2026-06-10T06:43:51-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:43:51.396959 | controller | 2026-06-10T06:43:51-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:43:51.796127 | controller | 2026-06-10T06:43:51-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:43:51.796196 | controller | 2026-06-10T06:43:51-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:43:52.805438 | controller | 2026-06-10T06:43:52-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:43:52.805544 | controller | 2026-06-10T06:43:52-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:43:52.985880 | controller | 2026-06-10T06:43:52-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:43:52.985928 | controller | 2026-06-10T06:43:52-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:43:53.797519 | controller | 2026-06-10T06:43:53-04:00 INFO 2026-06-10 06:43:53.797589 | controller | - Config: Waiting ... 2026-06-10 06:43:53.797596 | controller | - Gateway: OK 2026-06-10 06:43:53.797603 | controller | - GitServer: OK 2026-06-10 06:43:53.797635 | controller | - HoundSearch: OK 2026-06-10 06:43:53.797643 | controller | - LogJuicer: OK 2026-06-10 06:43:53.797648 | controller | - Logserver: OK 2026-06-10 06:43:53.797653 | controller | - MariaDB: OK 2026-06-10 06:43:53.797658 | controller | - NodePoolBuilder: OK 2026-06-10 06:43:53.797663 | controller | - NodePoolLauncher: OK 2026-06-10 06:43:53.797668 | controller | - Zookeeper: OK 2026-06-10 06:43:53.797673 | controller | - Zuul: Waiting ... 2026-06-10 06:43:53.797678 | controller | - |_ Config Secrets: OK 2026-06-10 06:43:53.797683 | controller | - |_ Merger: OK 2026-06-10 06:43:53.797688 | controller | - |_ Scheduler: OK 2026-06-10 06:43:53.797693 | controller | - |_ Web: Waiting ... 2026-06-10 06:43:53.797698 | controller | 2026-06-10 06:43:53.797710 | controller | 2026-06-10T06:43:53-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 06:43:58.801112 | controller | 2026-06-10T06:43:58-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:43:59.412222 | controller | 2026-06-10T06:43:59-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:43:59.412324 | controller | 2026-06-10T06:43:59-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:43:59.423922 | controller | 2026-06-10T06:43:59-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:43:59.423955 | controller | 2026-06-10T06:43:59-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:43:59.813615 | controller | 2026-06-10T06:43:59-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:44:00.211655 | controller | 2026-06-10T06:44:00-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:44:00.211709 | controller | 2026-06-10T06:44:00-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:44:01.217529 | controller | 2026-06-10T06:44:01-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:44:01.217639 | controller | 2026-06-10T06:44:01-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:44:01.404654 | controller | 2026-06-10T06:44:01-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:44:01.404722 | controller | 2026-06-10T06:44:01-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:44:02.613272 | controller | 2026-06-10T06:44:02-04:00 INFO 2026-06-10 06:44:02.613339 | controller | - Config: OK 2026-06-10 06:44:02.613345 | controller | - Gateway: OK 2026-06-10 06:44:02.613349 | controller | - GitServer: OK 2026-06-10 06:44:02.613352 | controller | - HoundSearch: OK 2026-06-10 06:44:02.613356 | controller | - LogJuicer: OK 2026-06-10 06:44:02.613359 | controller | - Logserver: OK 2026-06-10 06:44:02.613363 | controller | - MariaDB: OK 2026-06-10 06:44:02.613367 | controller | - NodePoolBuilder: OK 2026-06-10 06:44:02.613371 | controller | - NodePoolLauncher: OK 2026-06-10 06:44:02.613374 | controller | - Zookeeper: OK 2026-06-10 06:44:02.613378 | controller | - Zuul: OK 2026-06-10 06:44:02.613382 | controller | 2026-06-10 06:44:02.613391 | controller | 2026-06-10T06:44:02-04:00 INFO Updating controller configmap ... 2026-06-10 06:44:03.004820 | controller | 2026-06-10T06:44:03-04:00 INFO Standalone reconcile done. 2026-06-10 10:44:03.444325 | controller | ok: Runtime: 0:00:29.921593 2026-06-10 10:44:03.455506 | 2026-06-10 10:44:03.455642 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 10:44:04.000061 | controller | ok: Runtime: 0:00:00.006338 2026-06-10 10:44:04.018457 | 2026-06-10 10:44:04.018622 | TASK [health-check/test-dry-run-lifecycle : Ensure zuul-executor StatefulSet is not present] 2026-06-10 06:44:04.316241 | controller | Error from server (NotFound): statefulsets.apps "zuul-executor" not found 2026-06-10 10:44:04.562661 | controller | ok: Runtime: 0:00:00.054788 2026-06-10 10:44:04.572177 | 2026-06-10 10:44:04.572309 | TASK [Run update detection test on zuul-web only] 2026-06-10 10:44:04.607955 | controller | ok 2026-06-10 10:44:04.650220 | 2026-06-10 10:44:04.650315 | TASK [test-dry-run-update-detection : Fetch current serial for zuul-executor] 2026-06-10 10:44:05.179344 | controller | skipping: Conditional result was False 2026-06-10 10:44:05.191719 | 2026-06-10 10:44:05.191951 | TASK [test-dry-run-update-detection : Fetch current serial for zuul-web] 2026-06-10 10:44:05.743104 | controller | ok: Runtime: 0:00:00.060293 2026-06-10 10:44:05.752904 | 2026-06-10 10:44:05.753035 | TASK [test-dry-run-update-detection : Patch zuul-executor statefulset template to trigger reconciliation] 2026-06-10 10:44:06.384708 | controller | skipping: Conditional result was False 2026-06-10 10:44:06.396896 | 2026-06-10 10:44:06.397173 | TASK [test-dry-run-update-detection : Patch zuul-web deployment template to trigger reconciliation] 2026-06-10 06:44:06.687799 | controller | deployment.apps/zuul-web patched 2026-06-10 10:44:06.946792 | controller | ok: Runtime: 0:00:00.063697 2026-06-10 10:44:06.956283 | 2026-06-10 10:44:06.956419 | TASK [Run deploy with --dry-run to check for update detection] 2026-06-10 10:44:06.993560 | controller | ok 2026-06-10 10:44:07.029314 | 2026-06-10 10:44:07.029414 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 10:44:07.278495 | controller | ok 2026-06-10 10:44:07.288400 | 2026-06-10 10:44:07.288567 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 10:44:07.569985 | controller | changed 2026-06-10 10:44:07.581927 | 2026-06-10 10:44:07.582078 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 06:44:08.087056 | controller | 2026-06-10T06:44:08-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:44:08.087111 | controller | 2026-06-10 06:44:08.095490 | controller | 2026-06-10T06:44:08-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:44:08.095525 | controller | 2026-06-10 06:44:08.108262 | controller | 2026-06-10T06:44:08-04:00 INFO Deleting object, name: ca-cert (server dry run) 2026-06-10 06:44:08.716466 | controller | 2026-06-10T06:44:08-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:44:08.798483 | controller | 2026-06-10T06:44:08-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:44:09.115435 | controller | 2026-06-10T06:44:09-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:44:10.906830 | controller | 2026-06-10T06:44:10-04:00 INFO [Dry Run] Statefulset "zuul-web" configuration changed, applying... 2026-06-10 06:44:10.906902 | controller | 2026-06-10T06:44:10-04:00 INFO [Dry Run] Reason: annotation 'serial' changed from '9' to '0' 2026-06-10 06:44:10.906915 | controller | 2026-06-10T06:44:10-04:00 INFO Updating object, name: zuul-web (server dry run) 2026-06-10 06:44:11.710838 | controller | 2026-06-10T06:44:11-04:00 INFO 2026-06-10 06:44:11.710904 | controller | - Config: Waiting ... 2026-06-10 06:44:11.710910 | controller | - Gateway: OK 2026-06-10 06:44:11.710915 | controller | - GitServer: OK 2026-06-10 06:44:11.710920 | controller | - HoundSearch: OK 2026-06-10 06:44:11.710924 | controller | - LogJuicer: OK 2026-06-10 06:44:11.710927 | controller | - Logserver: OK 2026-06-10 06:44:11.710931 | controller | - MariaDB: OK 2026-06-10 06:44:11.710934 | controller | - NodePoolBuilder: OK 2026-06-10 06:44:11.710938 | controller | - NodePoolLauncher: OK 2026-06-10 06:44:11.710955 | controller | - Zookeeper: OK 2026-06-10 06:44:11.710958 | controller | - Zuul: Waiting ... 2026-06-10 06:44:11.710962 | controller | - |_ Config Secrets: OK 2026-06-10 06:44:11.710965 | controller | - |_ Merger: OK 2026-06-10 06:44:11.710969 | controller | - |_ Scheduler: OK 2026-06-10 06:44:11.710972 | controller | - |_ Web: Waiting ... 2026-06-10 06:44:11.710976 | controller | 2026-06-10 06:44:11.710981 | controller | 2026-06-10T06:44:11-04:00 INFO [Dry Run] Standalone reconcile done 2026-06-10 10:44:12.134938 | controller | ok: Runtime: 0:00:03.894503 2026-06-10 10:44:12.145576 | 2026-06-10 10:44:12.145724 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 10:44:12.693782 | controller | ok: Runtime: 0:00:00.007492 2026-06-10 10:44:12.711351 | 2026-06-10 10:44:12.711590 | TASK [test-dry-run-update-detection : Check for 'update' dry-run message in the log for zuul-executor] 2026-06-10 10:44:13.259667 | controller | skipping: Conditional result was False 2026-06-10 10:44:13.272301 | 2026-06-10 10:44:13.272504 | TASK [test-dry-run-update-detection : Check for 'update' dry-run message in the log for zuul-web] 2026-06-10 06:44:13.505148 | controller | 2026-06-10T06:44:10-04:00 INFO [Dry Run] Reason: annotation 'serial' changed from '9' to '0' 2026-06-10 10:44:13.820344 | controller | ok: Runtime: 0:00:00.004254 2026-06-10 10:44:13.830074 | 2026-06-10 10:44:13.830159 | TASK [test-dry-run-update-detection : Restore original serial for zuul-executor] 2026-06-10 10:44:14.380071 | controller | skipping: Conditional result was False 2026-06-10 10:44:14.391472 | 2026-06-10 10:44:14.391614 | TASK [test-dry-run-update-detection : Restore original serial for zuul-web] 2026-06-10 06:44:14.735909 | controller | deployment.apps/zuul-web patched 2026-06-10 10:44:14.941292 | controller | ok: Runtime: 0:00:00.061515 2026-06-10 10:44:14.957801 | 2026-06-10 10:44:14.957944 | TASK [health-check/test-dry-run-lifecycle : Generate CR with zuul-executor enabled] 2026-06-10 10:44:15.611335 | controller | changed 2026-06-10 10:44:15.620744 | 2026-06-10 10:44:15.620879 | TASK [Run deploy with --dry-run to test from-scratch creation] 2026-06-10 10:44:15.660072 | controller | ok 2026-06-10 10:44:15.702685 | 2026-06-10 10:44:15.702828 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 10:44:15.929170 | controller | ok 2026-06-10 10:44:15.938482 | 2026-06-10 10:44:15.938621 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 10:44:16.156221 | controller | ok 2026-06-10 10:44:16.167056 | 2026-06-10 10:44:16.167202 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 06:44:16.628534 | controller | 2026-06-10T06:44:16-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:44:16.628634 | controller | 2026-06-10 06:44:16.638712 | controller | 2026-06-10T06:44:16-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:44:16.638746 | controller | 2026-06-10 06:44:16.650190 | controller | 2026-06-10T06:44:16-04:00 INFO Deleting object, name: ca-cert (server dry run) 2026-06-10 06:44:17.258947 | controller | 2026-06-10T06:44:17-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:44:17.349040 | controller | 2026-06-10T06:44:17-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:44:17.659063 | controller | 2026-06-10T06:44:17-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:44:19.450318 | controller | 2026-06-10T06:44:19-04:00 INFO Creating object, name: zuul-executor (server dry run) 2026-06-10 06:44:21.051655 | controller | 2026-06-10T06:44:21-04:00 INFO 2026-06-10 06:44:21.051707 | controller | - Config: Waiting ... 2026-06-10 06:44:21.051714 | controller | - Gateway: OK 2026-06-10 06:44:21.051719 | controller | - GitServer: OK 2026-06-10 06:44:21.051724 | controller | - HoundSearch: OK 2026-06-10 06:44:21.051729 | controller | - LogJuicer: OK 2026-06-10 06:44:21.051734 | controller | - Logserver: OK 2026-06-10 06:44:21.051739 | controller | - MariaDB: OK 2026-06-10 06:44:21.051744 | controller | - NodePoolBuilder: OK 2026-06-10 06:44:21.051749 | controller | - NodePoolLauncher: OK 2026-06-10 06:44:21.051754 | controller | - Zookeeper: OK 2026-06-10 06:44:21.051774 | controller | - Zuul: Waiting ... 2026-06-10 06:44:21.051779 | controller | - |_ Config Secrets: OK 2026-06-10 06:44:21.051784 | controller | - |_ Executor: Waiting ... 2026-06-10 06:44:21.051789 | controller | - |_ Merger: OK 2026-06-10 06:44:21.051794 | controller | - |_ Scheduler: OK 2026-06-10 06:44:21.051799 | controller | - |_ Web: OK 2026-06-10 06:44:21.051804 | controller | 2026-06-10 06:44:21.051810 | controller | 2026-06-10T06:44:21-04:00 INFO [Dry Run] Standalone reconcile done 2026-06-10 10:44:21.221178 | controller | ok: Runtime: 0:00:04.666999 2026-06-10 10:44:21.232111 | 2026-06-10 10:44:21.232286 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 10:44:21.778800 | controller | ok: Runtime: 0:00:00.005167 2026-06-10 10:44:21.790574 | 2026-06-10 10:44:21.790711 | TASK [health-check/test-dry-run-lifecycle : Check for 'create' dry-run messages for zuul-executor resources] 2026-06-10 06:44:22.010229 | controller | 2026-06-10T06:44:19-04:00 INFO Creating object, name: zuul-executor (server dry run) 2026-06-10 10:44:22.326377 | controller | ok: Runtime: 0:00:00.004456 2026-06-10 10:44:22.335636 | 2026-06-10 10:44:22.335784 | TASK [Restore original state by deploying with zuul-executor enabled] 2026-06-10 10:44:22.383233 | controller | ok 2026-06-10 10:44:22.434539 | 2026-06-10 10:44:22.434763 | TASK [run-operator-standalone : Ensure ~/zuul-output/logs] 2026-06-10 10:44:22.692965 | controller | ok 2026-06-10 10:44:22.789796 | 2026-06-10 10:44:22.789939 | TASK [run-operator-standalone : Clear previous log file] 2026-06-10 10:44:23.025153 | controller | changed 2026-06-10 10:44:23.044095 | 2026-06-10 10:44:23.044239 | TASK [run-operator-standalone : Run the operator in standalone mode] 2026-06-10 06:44:23.588510 | controller | 2026-06-10T06:44:23-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:44:23.588590 | controller | 2026-06-10 06:44:23.598582 | controller | 2026-06-10T06:44:23-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:44:23.598638 | controller | 2026-06-10 06:44:23.611570 | controller | 2026-06-10T06:44:23-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:44:24.222811 | controller | 2026-06-10T06:44:24-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:44:24.343793 | controller | 2026-06-10T06:44:24-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:44:24.349794 | controller | 2026-06-10T06:44:24-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:44:24.349843 | controller | 2026-06-10T06:44:24-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:44:24.364502 | controller | 2026-06-10T06:44:24-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:44:24.364586 | controller | 2026-06-10T06:44:24-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:44:24.627901 | controller | 2026-06-10T06:44:24-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:44:25.026856 | controller | 2026-06-10T06:44:25-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:44:25.026961 | controller | 2026-06-10T06:44:25-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:44:26.049996 | controller | 2026-06-10T06:44:26-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:44:26.050080 | controller | 2026-06-10T06:44:26-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:44:26.411649 | controller | 2026-06-10T06:44:26-04:00 INFO Creating object, name: zuul-executor 2026-06-10 06:44:27.210498 | controller | 2026-06-10T06:44:27-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:44:27.210554 | controller | 2026-06-10T06:44:27-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:44:28.016380 | controller | 2026-06-10T06:44:28-04:00 INFO 2026-06-10 06:44:28.016447 | controller | - Config: Waiting ... 2026-06-10 06:44:28.016453 | controller | - Gateway: OK 2026-06-10 06:44:28.016459 | controller | - GitServer: OK 2026-06-10 06:44:28.016464 | controller | - HoundSearch: OK 2026-06-10 06:44:28.016469 | controller | - LogJuicer: OK 2026-06-10 06:44:28.016474 | controller | - Logserver: OK 2026-06-10 06:44:28.016479 | controller | - MariaDB: OK 2026-06-10 06:44:28.016486 | controller | - NodePoolBuilder: OK 2026-06-10 06:44:28.016491 | controller | - NodePoolLauncher: OK 2026-06-10 06:44:28.016497 | controller | - Zookeeper: OK 2026-06-10 06:44:28.016502 | controller | - Zuul: Waiting ... 2026-06-10 06:44:28.016507 | controller | - |_ Config Secrets: OK 2026-06-10 06:44:28.016512 | controller | - |_ Executor: Waiting ... 2026-06-10 06:44:28.016517 | controller | - |_ Merger: OK 2026-06-10 06:44:28.016521 | controller | - |_ Scheduler: OK 2026-06-10 06:44:28.016526 | controller | - |_ Web: OK 2026-06-10 06:44:28.016531 | controller | 2026-06-10 06:44:28.016537 | controller | 2026-06-10T06:44:28-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 06:44:33.024542 | controller | 2026-06-10T06:44:33-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:44:33.635050 | controller | 2026-06-10T06:44:33-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:44:33.635122 | controller | 2026-06-10T06:44:33-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:44:33.649981 | controller | 2026-06-10T06:44:33-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:44:33.650042 | controller | 2026-06-10T06:44:33-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:44:34.034580 | controller | 2026-06-10T06:44:34-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:44:34.436092 | controller | 2026-06-10T06:44:34-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:44:34.436146 | controller | 2026-06-10T06:44:34-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:44:35.453077 | controller | 2026-06-10T06:44:35-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:44:35.453128 | controller | 2026-06-10T06:44:35-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:44:35.637493 | controller | 2026-06-10T06:44:35-04:00 INFO Waiting for statefulset, name: zuul-executor 2026-06-10 06:44:39.657271 | controller | 2026-06-10T06:44:39-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:44:39.657370 | controller | 2026-06-10T06:44:39-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:44:39.682015 | controller | 2026-06-10T06:44:39-04:00 INFO 2026-06-10 06:44:39.682104 | controller | - Config: OK 2026-06-10 06:44:39.682110 | controller | - Gateway: OK 2026-06-10 06:44:39.682113 | controller | - GitServer: OK 2026-06-10 06:44:39.682117 | controller | - HoundSearch: OK 2026-06-10 06:44:39.682121 | controller | - LogJuicer: OK 2026-06-10 06:44:39.682124 | controller | - Logserver: OK 2026-06-10 06:44:39.682128 | controller | - MariaDB: OK 2026-06-10 06:44:39.682131 | controller | - NodePoolBuilder: OK 2026-06-10 06:44:39.682136 | controller | - NodePoolLauncher: OK 2026-06-10 06:44:39.682139 | controller | - Zookeeper: OK 2026-06-10 06:44:39.682143 | controller | - Zuul: OK 2026-06-10 06:44:39.682147 | controller | 2026-06-10 06:44:39.682152 | controller | 2026-06-10T06:44:39-04:00 INFO Updating controller configmap ... 2026-06-10 06:44:40.053679 | controller | 2026-06-10T06:44:40-04:00 INFO Standalone reconcile done. 2026-06-10 10:44:40.109045 | controller | ok: Runtime: 0:00:16.779734 2026-06-10 10:44:40.119409 | 2026-06-10 10:44:40.119575 | TASK [run-operator-standalone : Copy current spec] 2026-06-10 10:44:40.658046 | controller | ok: Runtime: 0:00:00.004470 2026-06-10 10:44:40.666264 | 2026-06-10 10:44:40.666328 | TASK [health-check/test-dry-run-lifecycle : Remove temporary directory] 2026-06-10 10:44:40.914651 | controller | changed 2026-06-10 10:44:40.936123 | 2026-06-10 10:44:40.936289 | TASK [health-check/test-backup-restore : Backup the Software Factory deployment] 2026-06-10 06:44:41.424917 | controller | 2026-06-10T06:44:41-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:44:41.424998 | controller | 2026-06-10 06:44:41.434264 | controller | 2026-06-10T06:44:41-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:44:41.434313 | controller | 2026-06-10 06:44:41.437645 | controller | 2026-06-10T06:44:41-04:00 INFO Starting backup process for services in namespace: sf 2026-06-10 06:44:41.437668 | controller | 2026-06-10T06:44:41-04:00 INFO Creating secrets backup... 2026-06-10 06:44:41.456401 | controller | 2026-06-10T06:44:41-04:00 INFO Finished doing secret backup! 2026-06-10 06:44:41.456466 | controller | 2026-06-10T06:44:41-04:00 INFO Doing Zuul keys backup... 2026-06-10 06:44:41.458563 | controller | 2026-06-10T06:44:41-04:00 INFO Creating object, name: zuul-kazoo 2026-06-10 06:44:43.469172 | controller | 2026-06-10T06:44:43-04:00 INFO Running pod execution pod: zuul-kazoo, command: [zuul-admin export-keys /tmp/zuul-backup] 2026-06-10 06:44:45.262088 | controller | INFO:kazoo.client:Connecting to zookeeper-0.zookeeper-headless.sf(10.244.0.98):2281, use_ssl: True 2026-06-10 06:44:45.306675 | controller | INFO:kazoo.client:Zookeeper connection established, state: CONNECTED 2026-06-10 06:44:45.339304 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/ssh 2026-06-10 06:44:45.339403 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/secrets 2026-06-10 06:44:45.339426 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/demo-tenant-config/demo-tenant-config/ssh 2026-06-10 06:44:45.339436 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/demo-tenant-config/demo-tenant-config/secrets 2026-06-10 06:44:45.339444 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/demo-project/demo-project/ssh 2026-06-10 06:44:45.339451 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/demo-project/demo-project/secrets 2026-06-10 06:44:45.339460 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/config/config/ssh 2026-06-10 06:44:45.339468 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/config/config/secrets 2026-06-10 06:44:45.339476 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/ichiban-config-QEwn8WpM/ichiban-config-QEwn8WpM/ssh 2026-06-10 06:44:45.339484 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/gerrit/ichiban-config-QEwn8WpM/ichiban-config-QEwn8WpM/secrets 2026-06-10 06:44:45.339501 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/git-server/system-config/system-config/ssh 2026-06-10 06:44:45.339509 | controller | INFO:zuul.KeyStorage:Exported: /keystorage/git-server/system-config/system-config/secrets 2026-06-10 06:44:45.339517 | controller | INFO:zuul.KeyStorage:Exported: /keystorage2/tenant/ichiban-tenant-QEwn8WpM/ssh 2026-06-10 06:44:45.339691 | controller | INFO:zuul.KeyStorage:Exported: /keystorage2/tenant/internal/ssh 2026-06-10 06:44:45.339711 | controller | INFO:zuul.KeyStorage:Exported: /keystorage2/tenant/demo-tenant/ssh 2026-06-10 06:44:45.913425 | controller | 2026-06-10T06:44:45-04:00 INFO Running pod execution pod: zuul-kazoo, command: [cat /tmp/zuul-backup] 2026-06-10 06:44:45.988032 | controller | 2026-06-10T06:44:45-04:00 INFO Running pod execution pod: zuul-kazoo, command: [rm /tmp/zuul-backup] 2026-06-10 06:44:46.067270 | controller | 2026-06-10T06:44:46-04:00 INFO Finished doing Zuul private keys backup! 2026-06-10 06:44:46.069651 | controller | 2026-06-10T06:44:46-04:00 INFO Deleting object, name: zuul-kazoo 2026-06-10 06:44:46.073454 | controller | 2026-06-10T06:44:46-04:00 INFO Doing DB backup... 2026-06-10 06:44:46.077419 | controller | 2026-06-10T06:44:46-04:00 INFO Running pod execution pod: mariadb-0, command: [mysqldump --databases zuul --single-transaction] 2026-06-10 06:44:46.156803 | controller | mysqldump: Deprecated program name. It will be removed in a future release, use '/usr/bin/mariadb-dump' instead 2026-06-10 06:44:46.156871 | controller | Warning: World-writable config file '/etc/my.cnf.d/docker.cnf' is ignored 2026-06-10 06:44:46.271305 | controller | 2026-06-10T06:44:46-04:00 INFO Finished doing DBs backup! 2026-06-10 10:44:46.479839 | controller | ok: Runtime: 0:00:05.104346 2026-06-10 10:44:46.491082 | 2026-06-10 10:44:46.491294 | TASK [health-check/test-backup-restore : Fetch the last build ID reported by Zuul web] 2026-06-10 10:44:47.017422 | controller | ok: OK (698 bytes) 2026-06-10 10:44:47.028186 | 2026-06-10 10:44:47.028338 | TASK [health-check/test-backup-restore : Wipe Software Factory deployment] 2026-06-10 06:44:47.570233 | controller | 2026-06-10T06:44:47-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:44:47.570309 | controller | 2026-06-10 06:44:47.577924 | controller | 2026-06-10T06:44:47-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:44:47.577971 | controller | 2026-06-10 06:44:47.597569 | controller | 2026-06-10T06:44:47-04:00 INFO Warning: Killing ZooKeeper client: nodepool-builder-0 nodepool-builder 2026-06-10 06:44:47.597628 | controller | 2026-06-10T06:44:47-04:00 INFO Running pod execution pod: nodepool-builder-0, command: [kill -9 1] 2026-06-10 06:44:47.677340 | controller | 2026-06-10T06:44:47-04:00 INFO Deleting object, name: nodepool-builder 2026-06-10 06:44:47.682872 | controller | 2026-06-10T06:44:47-04:00 INFO Warning: Killing ZooKeeper client: nodepool-launcher-5bd775d78f-jsvp5 launcher 2026-06-10 06:44:47.682900 | controller | 2026-06-10T06:44:47-04:00 INFO Running pod execution pod: nodepool-launcher-5bd775d78f-jsvp5, command: [kill -9 1] 2026-06-10 06:44:47.783595 | controller | 2026-06-10T06:44:47-04:00 INFO Deleting object, name: launcher 2026-06-10 06:44:47.785584 | controller | 2026-06-10T06:44:47-04:00 INFO Warning: Killing ZooKeeper client: zuul-db-provision-dm9wg mariadb-client 2026-06-10 06:44:47.785619 | controller | 2026-06-10T06:44:47-04:00 INFO Running pod execution pod: zuul-db-provision-dm9wg, command: [kill -9 1] 2026-06-10 06:44:47.799528 | controller | 2026-06-10T06:44:47-04:00 INFO Deleting object, name: mariadb-client 2026-06-10 06:44:47.800984 | controller | 2026-06-10T06:44:47-04:00 INFO Warning: Killing ZooKeeper client: zuul-db-provision-r5bxv mariadb-client 2026-06-10 06:44:47.801004 | controller | 2026-06-10T06:44:47-04:00 INFO Running pod execution pod: zuul-db-provision-r5bxv, command: [kill -9 1] 2026-06-10 06:44:47.814687 | controller | 2026-06-10T06:44:47-04:00 INFO Deleting object, name: mariadb-client 2026-06-10 06:44:47.816454 | controller | 2026-06-10T06:44:47-04:00 INFO Warning: Killing ZooKeeper client: zuul-executor-0 zuul-executor 2026-06-10 06:44:47.816477 | controller | 2026-06-10T06:44:47-04:00 INFO Running pod execution pod: zuul-executor-0, command: [kill -9 1] 2026-06-10 06:44:47.863075 | controller | 2026-06-10T06:44:47-04:00 INFO Deleting object, name: zuul-executor 2026-06-10 06:44:47.866518 | controller | 2026-06-10T06:44:47-04:00 INFO Warning: Killing ZooKeeper client: zuul-kazoo zuul-kazoo 2026-06-10 06:44:47.866548 | controller | 2026-06-10T06:44:47-04:00 INFO Running pod execution pod: zuul-kazoo, command: [kill -9 1] 2026-06-10 06:45:16.231176 | controller | 2026-06-10T06:45:16-04:00 INFO Deleting object, name: zuul-kazoo 2026-06-10 06:45:16.233882 | controller | 2026-06-10T06:45:16-04:00 INFO Warning: Killing ZooKeeper client: zuul-merger-0 zuul-merger 2026-06-10 06:45:16.233900 | controller | 2026-06-10T06:45:16-04:00 INFO Running pod execution pod: zuul-merger-0, command: [kill -9 1] 2026-06-10 06:45:16.317971 | controller | 2026-06-10T06:45:16-04:00 INFO Deleting object, name: zuul-merger 2026-06-10 06:45:16.321182 | controller | 2026-06-10T06:45:16-04:00 INFO Warning: Killing ZooKeeper client: zuul-merger-1 zuul-merger 2026-06-10 06:45:16.321213 | controller | 2026-06-10T06:45:16-04:00 INFO Running pod execution pod: zuul-merger-1, command: [kill -9 1] 2026-06-10 06:45:17.887476 | controller | 2026-06-10T06:45:17-04:00 INFO Deleting object, name: zuul-merger 2026-06-10 06:45:17.889600 | controller | 2026-06-10T06:45:17-04:00 INFO Warning: Killing ZooKeeper client: zuul-scheduler-0 zuul-scheduler 2026-06-10 06:45:17.889645 | controller | 2026-06-10T06:45:17-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [kill -9 1] 2026-06-10 06:45:17.976310 | controller | 2026-06-10T06:45:17-04:00 INFO Deleting object, name: zuul-scheduler 2026-06-10 06:45:17.979827 | controller | 2026-06-10T06:45:17-04:00 INFO Warning: Killing ZooKeeper client: zuul-web-d948bf696-rqc4v zuul-web 2026-06-10 06:45:17.979862 | controller | 2026-06-10T06:45:17-04:00 INFO Running pod execution pod: zuul-web-d948bf696-rqc4v, command: [kill -9 1] 2026-06-10 06:45:18.091986 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: zuul-web 2026-06-10 06:45:18.095345 | controller | 2026-06-10T06:45:18-04:00 INFO Warning: Killing ZooKeeper client: zuul-weeder-57c6567f5f-rvgsd zuul-weeder 2026-06-10 06:45:18.095390 | controller | 2026-06-10T06:45:18-04:00 INFO Running pod execution pod: zuul-weeder-57c6567f5f-rvgsd, command: [kill -9 1] 2026-06-10 06:45:18.179018 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: zuul-weeder 2026-06-10 06:45:18.181733 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: zookeeper 2026-06-10 06:45:18.187435 | controller | 2026-06-10T06:45:18-04:00 INFO Standalone mode detected. Deleting owner ConfigMap with foreground propagation. 2026-06-10 06:45:18.187492 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: sf-standalone-owner 2026-06-10 06:45:18.189782 | controller | 2026-06-10T06:45:18-04:00 INFO Cleaning resources... 2026-06-10 06:45:18.192656 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: gateway 2026-06-10 06:45:18.199959 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: hound-search 2026-06-10 06:45:18.206993 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: logjuicer 2026-06-10 06:45:18.213807 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: nodepool-launcher 2026-06-10 06:45:18.221213 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: zookeeper 2026-06-10 06:45:18.229664 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: zookeeper-headless 2026-06-10 06:45:18.233363 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: zuul-capacity 2026-06-10 06:45:18.242851 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: zuul-executor-headless 2026-06-10 06:45:18.249198 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: zuul-web 2026-06-10 06:45:18.254823 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: zuul-weeder 2026-06-10 06:45:18.262247 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: gateway 2026-06-10 06:45:18.267783 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: logjuicer 2026-06-10 06:45:18.270594 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: nodepool-launcher 2026-06-10 06:45:18.275989 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: git-server 2026-06-10 06:45:18.278621 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: hound-search 2026-06-10 06:45:18.280566 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: logserver 2026-06-10 06:45:18.283162 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: mariadb 2026-06-10 06:45:18.290636 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-429qm", "status": "Running"} 2026-06-10 06:45:18.290662 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: gateway-6bbbd758d9-429qm 2026-06-10 06:45:18.293780 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting pod {"name": "git-server-0", "status": "Running"} 2026-06-10 06:45:18.293818 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: git-server-0 2026-06-10 06:45:18.296993 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting pod {"name": "hound-search-0", "status": "Running"} 2026-06-10 06:45:18.297015 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: hound-search-0 2026-06-10 06:45:18.301748 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-klm2b", "status": "Running"} 2026-06-10 06:45:18.301815 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-klm2b 2026-06-10 06:45:18.307038 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 06:45:18.307097 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: logserver-0 2026-06-10 06:45:18.312051 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting pod {"name": "mariadb-0", "status": "Running"} 2026-06-10 06:45:18.312095 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: mariadb-0 2026-06-10 06:45:18.323827 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-jsvp5", "status": "Running"} 2026-06-10 06:45:18.323880 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-jsvp5 2026-06-10 06:45:18.334928 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting pod {"name": "zookeeper-0", "status": "Running"} 2026-06-10 06:45:18.335161 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: zookeeper-0 2026-06-10 06:45:18.338021 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting pod {"name": "zuul-merger-0", "status": "Failed"} 2026-06-10 06:45:18.338056 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: zuul-merger-0 2026-06-10 06:45:18.341614 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 06:45:18.341644 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 06:45:18.343979 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting pod {"name": "zuul-web-d948bf696-rqc4v", "status": "Running"} 2026-06-10 06:45:18.344001 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: zuul-web-d948bf696-rqc4v 2026-06-10 06:45:18.493131 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting pod {"name": "zuul-weeder-57c6567f5f-rvgsd", "status": "Running"} 2026-06-10 06:45:18.493201 | controller | 2026-06-10T06:45:18-04:00 INFO Deleting object, name: zuul-weeder-57c6567f5f-rvgsd 2026-06-10 06:45:20.699586 | controller | 2026-06-10T06:45:20-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-424nv", "status": "Running"} 2026-06-10 06:45:20.699676 | controller | 2026-06-10T06:45:20-04:00 INFO Deleting object, name: gateway-6bbbd758d9-424nv 2026-06-10 06:45:20.701974 | controller | 2026-06-10T06:45:20-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-klm2b", "status": "Running"} 2026-06-10 06:45:20.701990 | controller | 2026-06-10T06:45:20-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-klm2b 2026-06-10 06:45:20.704702 | controller | 2026-06-10T06:45:20-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-lbnvr", "status": "Running"} 2026-06-10 06:45:20.704728 | controller | 2026-06-10T06:45:20-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-lbnvr 2026-06-10 06:45:20.708305 | controller | 2026-06-10T06:45:20-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 06:45:20.708330 | controller | 2026-06-10T06:45:20-04:00 INFO Deleting object, name: logserver-0 2026-06-10 06:45:20.710759 | controller | 2026-06-10T06:45:20-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-h42rg", "status": "Pending"} 2026-06-10 06:45:20.710787 | controller | 2026-06-10T06:45:20-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-h42rg 2026-06-10 06:45:20.718999 | controller | 2026-06-10T06:45:20-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-jsvp5", "status": "Running"} 2026-06-10 06:45:20.719069 | controller | 2026-06-10T06:45:20-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-jsvp5 2026-06-10 06:45:20.721387 | controller | 2026-06-10T06:45:20-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 06:45:20.721409 | controller | 2026-06-10T06:45:20-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 06:45:20.724482 | controller | 2026-06-10T06:45:20-04:00 INFO Deleting pod {"name": "zuul-web-d948bf696-rqc4v", "status": "Running"} 2026-06-10 06:45:20.724503 | controller | 2026-06-10T06:45:20-04:00 INFO Deleting object, name: zuul-web-d948bf696-rqc4v 2026-06-10 06:45:22.737779 | controller | 2026-06-10T06:45:22-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-424nv", "status": "Running"} 2026-06-10 06:45:22.737851 | controller | 2026-06-10T06:45:22-04:00 INFO Deleting object, name: gateway-6bbbd758d9-424nv 2026-06-10 06:45:22.739450 | controller | 2026-06-10T06:45:22-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-gsrn8", "status": "Pending"} 2026-06-10 06:45:22.739474 | controller | 2026-06-10T06:45:22-04:00 INFO Deleting object, name: gateway-6bbbd758d9-gsrn8 2026-06-10 06:45:22.741657 | controller | 2026-06-10T06:45:22-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-8p2b6", "status": "Pending"} 2026-06-10 06:45:22.741679 | controller | 2026-06-10T06:45:22-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-8p2b6 2026-06-10 06:45:22.745636 | controller | 2026-06-10T06:45:22-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-klm2b", "status": "Running"} 2026-06-10 06:45:22.745673 | controller | 2026-06-10T06:45:22-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-klm2b 2026-06-10 06:45:22.747165 | controller | 2026-06-10T06:45:22-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-lbnvr", "status": "Running"} 2026-06-10 06:45:22.747187 | controller | 2026-06-10T06:45:22-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-lbnvr 2026-06-10 06:45:22.748435 | controller | 2026-06-10T06:45:22-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 06:45:22.748490 | controller | 2026-06-10T06:45:22-04:00 INFO Deleting object, name: logserver-0 2026-06-10 06:45:22.750159 | controller | 2026-06-10T06:45:22-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-9g7d6", "status": "Pending"} 2026-06-10 06:45:22.750177 | controller | 2026-06-10T06:45:22-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-9g7d6 2026-06-10 06:45:22.752869 | controller | 2026-06-10T06:45:22-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-h42rg", "status": "Pending"} 2026-06-10 06:45:22.752890 | controller | 2026-06-10T06:45:22-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-h42rg 2026-06-10 06:45:22.757003 | controller | 2026-06-10T06:45:22-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-jsvp5", "status": "Running"} 2026-06-10 06:45:22.757061 | controller | 2026-06-10T06:45:22-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-jsvp5 2026-06-10 06:45:22.759466 | controller | 2026-06-10T06:45:22-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 06:45:22.759492 | controller | 2026-06-10T06:45:22-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 06:45:22.761222 | controller | 2026-06-10T06:45:22-04:00 INFO Deleting pod {"name": "zuul-web-d948bf696-rqc4v", "status": "Running"} 2026-06-10 06:45:22.761238 | controller | 2026-06-10T06:45:22-04:00 INFO Deleting object, name: zuul-web-d948bf696-rqc4v 2026-06-10 06:45:24.948330 | controller | 2026-06-10T06:45:24-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-424nv", "status": "Running"} 2026-06-10 06:45:24.948420 | controller | 2026-06-10T06:45:24-04:00 INFO Deleting object, name: gateway-6bbbd758d9-424nv 2026-06-10 06:45:24.951543 | controller | 2026-06-10T06:45:24-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-klm2b", "status": "Running"} 2026-06-10 06:45:24.951577 | controller | 2026-06-10T06:45:24-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-klm2b 2026-06-10 06:45:24.954888 | controller | 2026-06-10T06:45:24-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-lbnvr", "status": "Running"} 2026-06-10 06:45:24.954914 | controller | 2026-06-10T06:45:24-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-lbnvr 2026-06-10 06:45:24.956807 | controller | 2026-06-10T06:45:24-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 06:45:24.956822 | controller | 2026-06-10T06:45:24-04:00 INFO Deleting object, name: logserver-0 2026-06-10 06:45:24.960152 | controller | 2026-06-10T06:45:24-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-h42rg", "status": "Pending"} 2026-06-10 06:45:24.960180 | controller | 2026-06-10T06:45:24-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-h42rg 2026-06-10 06:45:24.962905 | controller | 2026-06-10T06:45:24-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-jsvp5", "status": "Running"} 2026-06-10 06:45:24.962942 | controller | 2026-06-10T06:45:24-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-jsvp5 2026-06-10 06:45:24.965283 | controller | 2026-06-10T06:45:24-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 06:45:24.965312 | controller | 2026-06-10T06:45:24-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 06:45:24.967843 | controller | 2026-06-10T06:45:24-04:00 INFO Deleting pod {"name": "zuul-web-d948bf696-rqc4v", "status": "Running"} 2026-06-10 06:45:24.967868 | controller | 2026-06-10T06:45:24-04:00 INFO Deleting object, name: zuul-web-d948bf696-rqc4v 2026-06-10 06:45:26.979707 | controller | 2026-06-10T06:45:26-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-424nv", "status": "Running"} 2026-06-10 06:45:26.979817 | controller | 2026-06-10T06:45:26-04:00 INFO Deleting object, name: gateway-6bbbd758d9-424nv 2026-06-10 06:45:26.982968 | controller | 2026-06-10T06:45:26-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-klm2b", "status": "Running"} 2026-06-10 06:45:26.983007 | controller | 2026-06-10T06:45:26-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-klm2b 2026-06-10 06:45:26.985318 | controller | 2026-06-10T06:45:26-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-lbnvr", "status": "Running"} 2026-06-10 06:45:26.985369 | controller | 2026-06-10T06:45:26-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-lbnvr 2026-06-10 06:45:26.987812 | controller | 2026-06-10T06:45:26-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 06:45:26.987843 | controller | 2026-06-10T06:45:26-04:00 INFO Deleting object, name: logserver-0 2026-06-10 06:45:26.990258 | controller | 2026-06-10T06:45:26-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-h42rg", "status": "Pending"} 2026-06-10 06:45:26.990291 | controller | 2026-06-10T06:45:26-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-h42rg 2026-06-10 06:45:26.992880 | controller | 2026-06-10T06:45:26-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-jsvp5", "status": "Running"} 2026-06-10 06:45:26.992917 | controller | 2026-06-10T06:45:26-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-jsvp5 2026-06-10 06:45:26.995900 | controller | 2026-06-10T06:45:26-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 06:45:26.995943 | controller | 2026-06-10T06:45:26-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 06:45:26.998332 | controller | 2026-06-10T06:45:26-04:00 INFO Deleting pod {"name": "zuul-web-d948bf696-rqc4v", "status": "Running"} 2026-06-10 06:45:26.998371 | controller | 2026-06-10T06:45:26-04:00 INFO Deleting object, name: zuul-web-d948bf696-rqc4v 2026-06-10 06:45:29.005849 | controller | 2026-06-10T06:45:29-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-424nv", "status": "Running"} 2026-06-10 06:45:29.005937 | controller | 2026-06-10T06:45:29-04:00 INFO Deleting object, name: gateway-6bbbd758d9-424nv 2026-06-10 06:45:29.008471 | controller | 2026-06-10T06:45:29-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-klm2b", "status": "Running"} 2026-06-10 06:45:29.008492 | controller | 2026-06-10T06:45:29-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-klm2b 2026-06-10 06:45:29.010837 | controller | 2026-06-10T06:45:29-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-lbnvr", "status": "Running"} 2026-06-10 06:45:29.010872 | controller | 2026-06-10T06:45:29-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-lbnvr 2026-06-10 06:45:29.012965 | controller | 2026-06-10T06:45:29-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 06:45:29.013002 | controller | 2026-06-10T06:45:29-04:00 INFO Deleting object, name: logserver-0 2026-06-10 06:45:29.015414 | controller | 2026-06-10T06:45:29-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-h42rg", "status": "Pending"} 2026-06-10 06:45:29.015456 | controller | 2026-06-10T06:45:29-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-h42rg 2026-06-10 06:45:29.017314 | controller | 2026-06-10T06:45:29-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-jsvp5", "status": "Running"} 2026-06-10 06:45:29.017334 | controller | 2026-06-10T06:45:29-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-jsvp5 2026-06-10 06:45:29.020510 | controller | 2026-06-10T06:45:29-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 06:45:29.020535 | controller | 2026-06-10T06:45:29-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 06:45:29.022696 | controller | 2026-06-10T06:45:29-04:00 INFO Deleting pod {"name": "zuul-web-d948bf696-rqc4v", "status": "Running"} 2026-06-10 06:45:29.022718 | controller | 2026-06-10T06:45:29-04:00 INFO Deleting object, name: zuul-web-d948bf696-rqc4v 2026-06-10 06:45:31.029502 | controller | 2026-06-10T06:45:31-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-424nv", "status": "Running"} 2026-06-10 06:45:31.029598 | controller | 2026-06-10T06:45:31-04:00 INFO Deleting object, name: gateway-6bbbd758d9-424nv 2026-06-10 06:45:31.031486 | controller | 2026-06-10T06:45:31-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-klm2b", "status": "Running"} 2026-06-10 06:45:31.031514 | controller | 2026-06-10T06:45:31-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-klm2b 2026-06-10 06:45:31.032915 | controller | 2026-06-10T06:45:31-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-lbnvr", "status": "Running"} 2026-06-10 06:45:31.032932 | controller | 2026-06-10T06:45:31-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-lbnvr 2026-06-10 06:45:31.034074 | controller | 2026-06-10T06:45:31-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 06:45:31.034101 | controller | 2026-06-10T06:45:31-04:00 INFO Deleting object, name: logserver-0 2026-06-10 06:45:31.035866 | controller | 2026-06-10T06:45:31-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-h42rg", "status": "Pending"} 2026-06-10 06:45:31.035885 | controller | 2026-06-10T06:45:31-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-h42rg 2026-06-10 06:45:31.038310 | controller | 2026-06-10T06:45:31-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-jsvp5", "status": "Running"} 2026-06-10 06:45:31.038324 | controller | 2026-06-10T06:45:31-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-jsvp5 2026-06-10 06:45:31.039707 | controller | 2026-06-10T06:45:31-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 06:45:31.039725 | controller | 2026-06-10T06:45:31-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 06:45:31.041190 | controller | 2026-06-10T06:45:31-04:00 INFO Deleting pod {"name": "zuul-web-d948bf696-rqc4v", "status": "Running"} 2026-06-10 06:45:31.041203 | controller | 2026-06-10T06:45:31-04:00 INFO Deleting object, name: zuul-web-d948bf696-rqc4v 2026-06-10 06:45:33.048039 | controller | 2026-06-10T06:45:33-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-424nv", "status": "Running"} 2026-06-10 06:45:33.048107 | controller | 2026-06-10T06:45:33-04:00 INFO Deleting object, name: gateway-6bbbd758d9-424nv 2026-06-10 06:45:33.049813 | controller | 2026-06-10T06:45:33-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-klm2b", "status": "Running"} 2026-06-10 06:45:33.049847 | controller | 2026-06-10T06:45:33-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-klm2b 2026-06-10 06:45:33.051482 | controller | 2026-06-10T06:45:33-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-lbnvr", "status": "Running"} 2026-06-10 06:45:33.051512 | controller | 2026-06-10T06:45:33-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-lbnvr 2026-06-10 06:45:33.053196 | controller | 2026-06-10T06:45:33-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 06:45:33.053226 | controller | 2026-06-10T06:45:33-04:00 INFO Deleting object, name: logserver-0 2026-06-10 06:45:33.055725 | controller | 2026-06-10T06:45:33-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-h42rg", "status": "Pending"} 2026-06-10 06:45:33.055752 | controller | 2026-06-10T06:45:33-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-h42rg 2026-06-10 06:45:33.057646 | controller | 2026-06-10T06:45:33-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-jsvp5", "status": "Running"} 2026-06-10 06:45:33.057676 | controller | 2026-06-10T06:45:33-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-jsvp5 2026-06-10 06:45:33.059940 | controller | 2026-06-10T06:45:33-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 06:45:33.059972 | controller | 2026-06-10T06:45:33-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 06:45:33.062192 | controller | 2026-06-10T06:45:33-04:00 INFO Deleting pod {"name": "zuul-web-d948bf696-rqc4v", "status": "Running"} 2026-06-10 06:45:33.062294 | controller | 2026-06-10T06:45:33-04:00 INFO Deleting object, name: zuul-web-d948bf696-rqc4v 2026-06-10 06:45:35.072014 | controller | 2026-06-10T06:45:35-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-424nv", "status": "Running"} 2026-06-10 06:45:35.072121 | controller | 2026-06-10T06:45:35-04:00 INFO Deleting object, name: gateway-6bbbd758d9-424nv 2026-06-10 06:45:35.074624 | controller | 2026-06-10T06:45:35-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-klm2b", "status": "Running"} 2026-06-10 06:45:35.074671 | controller | 2026-06-10T06:45:35-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-klm2b 2026-06-10 06:45:35.076312 | controller | 2026-06-10T06:45:35-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-lbnvr", "status": "Running"} 2026-06-10 06:45:35.076350 | controller | 2026-06-10T06:45:35-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-lbnvr 2026-06-10 06:45:35.077828 | controller | 2026-06-10T06:45:35-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 06:45:35.077871 | controller | 2026-06-10T06:45:35-04:00 INFO Deleting object, name: logserver-0 2026-06-10 06:45:35.079913 | controller | 2026-06-10T06:45:35-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-jsvp5", "status": "Running"} 2026-06-10 06:45:35.079958 | controller | 2026-06-10T06:45:35-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-jsvp5 2026-06-10 06:45:35.081881 | controller | 2026-06-10T06:45:35-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 06:45:35.081918 | controller | 2026-06-10T06:45:35-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 06:45:35.085140 | controller | 2026-06-10T06:45:35-04:00 INFO Deleting pod {"name": "zuul-web-d948bf696-rqc4v", "status": "Running"} 2026-06-10 06:45:35.085194 | controller | 2026-06-10T06:45:35-04:00 INFO Deleting object, name: zuul-web-d948bf696-rqc4v 2026-06-10 06:45:37.094467 | controller | 2026-06-10T06:45:37-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-424nv", "status": "Running"} 2026-06-10 06:45:37.094563 | controller | 2026-06-10T06:45:37-04:00 INFO Deleting object, name: gateway-6bbbd758d9-424nv 2026-06-10 06:45:37.097151 | controller | 2026-06-10T06:45:37-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-klm2b", "status": "Running"} 2026-06-10 06:45:37.097219 | controller | 2026-06-10T06:45:37-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-klm2b 2026-06-10 06:45:37.099558 | controller | 2026-06-10T06:45:37-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-lbnvr", "status": "Running"} 2026-06-10 06:45:37.099603 | controller | 2026-06-10T06:45:37-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-lbnvr 2026-06-10 06:45:37.102180 | controller | 2026-06-10T06:45:37-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 06:45:37.102263 | controller | 2026-06-10T06:45:37-04:00 INFO Deleting object, name: logserver-0 2026-06-10 06:45:37.105250 | controller | 2026-06-10T06:45:37-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-jsvp5", "status": "Running"} 2026-06-10 06:45:37.105318 | controller | 2026-06-10T06:45:37-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-jsvp5 2026-06-10 06:45:37.107439 | controller | 2026-06-10T06:45:37-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 06:45:37.107493 | controller | 2026-06-10T06:45:37-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 06:45:37.109703 | controller | 2026-06-10T06:45:37-04:00 INFO Deleting pod {"name": "zuul-web-d948bf696-rqc4v", "status": "Running"} 2026-06-10 06:45:37.109728 | controller | 2026-06-10T06:45:37-04:00 INFO Deleting object, name: zuul-web-d948bf696-rqc4v 2026-06-10 06:45:39.118621 | controller | 2026-06-10T06:45:39-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-424nv", "status": "Running"} 2026-06-10 06:45:39.118714 | controller | 2026-06-10T06:45:39-04:00 INFO Deleting object, name: gateway-6bbbd758d9-424nv 2026-06-10 06:45:39.121156 | controller | 2026-06-10T06:45:39-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-klm2b", "status": "Running"} 2026-06-10 06:45:39.121184 | controller | 2026-06-10T06:45:39-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-klm2b 2026-06-10 06:45:39.123755 | controller | 2026-06-10T06:45:39-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-lbnvr", "status": "Running"} 2026-06-10 06:45:39.123790 | controller | 2026-06-10T06:45:39-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-lbnvr 2026-06-10 06:45:39.126538 | controller | 2026-06-10T06:45:39-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 06:45:39.126573 | controller | 2026-06-10T06:45:39-04:00 INFO Deleting object, name: logserver-0 2026-06-10 06:45:39.128627 | controller | 2026-06-10T06:45:39-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-jsvp5", "status": "Running"} 2026-06-10 06:45:39.128650 | controller | 2026-06-10T06:45:39-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-jsvp5 2026-06-10 06:45:39.130827 | controller | 2026-06-10T06:45:39-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 06:45:39.130860 | controller | 2026-06-10T06:45:39-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 06:45:39.133340 | controller | 2026-06-10T06:45:39-04:00 INFO Deleting pod {"name": "zuul-web-d948bf696-rqc4v", "status": "Running"} 2026-06-10 06:45:39.133384 | controller | 2026-06-10T06:45:39-04:00 INFO Deleting object, name: zuul-web-d948bf696-rqc4v 2026-06-10 06:45:41.145012 | controller | 2026-06-10T06:45:41-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-424nv", "status": "Running"} 2026-06-10 06:45:41.145094 | controller | 2026-06-10T06:45:41-04:00 INFO Deleting object, name: gateway-6bbbd758d9-424nv 2026-06-10 06:45:41.148393 | controller | 2026-06-10T06:45:41-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-klm2b", "status": "Running"} 2026-06-10 06:45:41.148484 | controller | 2026-06-10T06:45:41-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-klm2b 2026-06-10 06:45:41.151281 | controller | 2026-06-10T06:45:41-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-lbnvr", "status": "Running"} 2026-06-10 06:45:41.151353 | controller | 2026-06-10T06:45:41-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-lbnvr 2026-06-10 06:45:41.154492 | controller | 2026-06-10T06:45:41-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 06:45:41.154534 | controller | 2026-06-10T06:45:41-04:00 INFO Deleting object, name: logserver-0 2026-06-10 06:45:41.157555 | controller | 2026-06-10T06:45:41-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-jsvp5", "status": "Running"} 2026-06-10 06:45:41.157601 | controller | 2026-06-10T06:45:41-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-jsvp5 2026-06-10 06:45:41.160308 | controller | 2026-06-10T06:45:41-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 06:45:41.160344 | controller | 2026-06-10T06:45:41-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 06:45:41.162732 | controller | 2026-06-10T06:45:41-04:00 INFO Deleting pod {"name": "zuul-web-d948bf696-rqc4v", "status": "Running"} 2026-06-10 06:45:41.162772 | controller | 2026-06-10T06:45:41-04:00 INFO Deleting object, name: zuul-web-d948bf696-rqc4v 2026-06-10 06:45:43.169986 | controller | 2026-06-10T06:45:43-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-424nv", "status": "Running"} 2026-06-10 06:45:43.170049 | controller | 2026-06-10T06:45:43-04:00 INFO Deleting object, name: gateway-6bbbd758d9-424nv 2026-06-10 06:45:43.172726 | controller | 2026-06-10T06:45:43-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-klm2b", "status": "Running"} 2026-06-10 06:45:43.172743 | controller | 2026-06-10T06:45:43-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-klm2b 2026-06-10 06:45:43.174840 | controller | 2026-06-10T06:45:43-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-lbnvr", "status": "Running"} 2026-06-10 06:45:43.174858 | controller | 2026-06-10T06:45:43-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-lbnvr 2026-06-10 06:45:43.176881 | controller | 2026-06-10T06:45:43-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 06:45:43.176901 | controller | 2026-06-10T06:45:43-04:00 INFO Deleting object, name: logserver-0 2026-06-10 06:45:43.178875 | controller | 2026-06-10T06:45:43-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-jsvp5", "status": "Running"} 2026-06-10 06:45:43.178893 | controller | 2026-06-10T06:45:43-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-jsvp5 2026-06-10 06:45:43.181275 | controller | 2026-06-10T06:45:43-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 06:45:43.181290 | controller | 2026-06-10T06:45:43-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 06:45:43.184315 | controller | 2026-06-10T06:45:43-04:00 INFO Deleting pod {"name": "zuul-web-d948bf696-rqc4v", "status": "Running"} 2026-06-10 06:45:43.184342 | controller | 2026-06-10T06:45:43-04:00 INFO Deleting object, name: zuul-web-d948bf696-rqc4v 2026-06-10 06:45:45.195709 | controller | 2026-06-10T06:45:45-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-424nv", "status": "Running"} 2026-06-10 06:45:45.195804 | controller | 2026-06-10T06:45:45-04:00 INFO Deleting object, name: gateway-6bbbd758d9-424nv 2026-06-10 06:45:45.198168 | controller | 2026-06-10T06:45:45-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-klm2b", "status": "Running"} 2026-06-10 06:45:45.198207 | controller | 2026-06-10T06:45:45-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-klm2b 2026-06-10 06:45:45.200031 | controller | 2026-06-10T06:45:45-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-lbnvr", "status": "Running"} 2026-06-10 06:45:45.200092 | controller | 2026-06-10T06:45:45-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-lbnvr 2026-06-10 06:45:45.202431 | controller | 2026-06-10T06:45:45-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 06:45:45.202461 | controller | 2026-06-10T06:45:45-04:00 INFO Deleting object, name: logserver-0 2026-06-10 06:45:45.205886 | controller | 2026-06-10T06:45:45-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-jsvp5", "status": "Running"} 2026-06-10 06:45:45.205923 | controller | 2026-06-10T06:45:45-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-jsvp5 2026-06-10 06:45:45.209573 | controller | 2026-06-10T06:45:45-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 06:45:45.209676 | controller | 2026-06-10T06:45:45-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 06:45:45.214993 | controller | 2026-06-10T06:45:45-04:00 INFO Deleting pod {"name": "zuul-web-d948bf696-rqc4v", "status": "Running"} 2026-06-10 06:45:45.215022 | controller | 2026-06-10T06:45:45-04:00 INFO Deleting object, name: zuul-web-d948bf696-rqc4v 2026-06-10 06:45:47.223700 | controller | 2026-06-10T06:45:47-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-424nv", "status": "Running"} 2026-06-10 06:45:47.223785 | controller | 2026-06-10T06:45:47-04:00 INFO Deleting object, name: gateway-6bbbd758d9-424nv 2026-06-10 06:45:47.226733 | controller | 2026-06-10T06:45:47-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-klm2b", "status": "Running"} 2026-06-10 06:45:47.226773 | controller | 2026-06-10T06:45:47-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-klm2b 2026-06-10 06:45:47.229740 | controller | 2026-06-10T06:45:47-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-lbnvr", "status": "Running"} 2026-06-10 06:45:47.229777 | controller | 2026-06-10T06:45:47-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-lbnvr 2026-06-10 06:45:47.232198 | controller | 2026-06-10T06:45:47-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Running"} 2026-06-10 06:45:47.232234 | controller | 2026-06-10T06:45:47-04:00 INFO Deleting object, name: logserver-0 2026-06-10 06:45:47.234586 | controller | 2026-06-10T06:45:47-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-jsvp5", "status": "Running"} 2026-06-10 06:45:47.234664 | controller | 2026-06-10T06:45:47-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-jsvp5 2026-06-10 06:45:47.237105 | controller | 2026-06-10T06:45:47-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Running"} 2026-06-10 06:45:47.237136 | controller | 2026-06-10T06:45:47-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 06:45:47.239819 | controller | 2026-06-10T06:45:47-04:00 INFO Deleting pod {"name": "zuul-web-d948bf696-rqc4v", "status": "Running"} 2026-06-10 06:45:47.239864 | controller | 2026-06-10T06:45:47-04:00 INFO Deleting object, name: zuul-web-d948bf696-rqc4v 2026-06-10 06:45:49.248963 | controller | 2026-06-10T06:45:49-04:00 INFO Deleting pod {"name": "gateway-6bbbd758d9-424nv", "status": "Running"} 2026-06-10 06:45:49.249030 | controller | 2026-06-10T06:45:49-04:00 INFO Deleting object, name: gateway-6bbbd758d9-424nv 2026-06-10 06:45:49.251133 | controller | 2026-06-10T06:45:49-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-klm2b", "status": "Failed"} 2026-06-10 06:45:49.251161 | controller | 2026-06-10T06:45:49-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-klm2b 2026-06-10 06:45:49.253294 | controller | 2026-06-10T06:45:49-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-lbnvr", "status": "Running"} 2026-06-10 06:45:49.253318 | controller | 2026-06-10T06:45:49-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-lbnvr 2026-06-10 06:45:49.255170 | controller | 2026-06-10T06:45:49-04:00 INFO Deleting pod {"name": "logserver-0", "status": "Failed"} 2026-06-10 06:45:49.255221 | controller | 2026-06-10T06:45:49-04:00 INFO Deleting object, name: logserver-0 2026-06-10 06:45:49.257118 | controller | 2026-06-10T06:45:49-04:00 INFO Deleting pod {"name": "nodepool-launcher-5bd775d78f-jsvp5", "status": "Failed"} 2026-06-10 06:45:49.257171 | controller | 2026-06-10T06:45:49-04:00 INFO Deleting object, name: nodepool-launcher-5bd775d78f-jsvp5 2026-06-10 06:45:49.259287 | controller | 2026-06-10T06:45:49-04:00 INFO Deleting pod {"name": "zuul-scheduler-0", "status": "Failed"} 2026-06-10 06:45:49.259313 | controller | 2026-06-10T06:45:49-04:00 INFO Deleting object, name: zuul-scheduler-0 2026-06-10 06:45:49.261236 | controller | 2026-06-10T06:45:49-04:00 INFO Deleting pod {"name": "zuul-web-d948bf696-rqc4v", "status": "Failed"} 2026-06-10 06:45:49.261278 | controller | 2026-06-10T06:45:49-04:00 INFO Deleting object, name: zuul-web-d948bf696-rqc4v 2026-06-10 06:45:51.267029 | controller | 2026-06-10T06:45:51-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-lbnvr", "status": "Running"} 2026-06-10 06:45:51.267129 | controller | 2026-06-10T06:45:51-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-lbnvr 2026-06-10 06:45:53.272895 | controller | 2026-06-10T06:45:53-04:00 INFO Deleting pod {"name": "logjuicer-6ff84764b7-lbnvr", "status": "Failed"} 2026-06-10 06:45:53.273014 | controller | 2026-06-10T06:45:53-04:00 INFO Deleting object, name: logjuicer-6ff84764b7-lbnvr 2026-06-10 06:45:55.285095 | controller | 2026-06-10T06:45:55-04:00 INFO Removing dangling persistent volume claims if any... 2026-06-10 06:45:55.289006 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting pvc {"name": "git-server-git-server-0", "status": "Bound"} 2026-06-10 06:45:55.289048 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting object, name: git-server-git-server-0 2026-06-10 06:45:55.291846 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting pvc {"name": "hound-search-data-hound-search-0", "status": "Bound"} 2026-06-10 06:45:55.291866 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting object, name: hound-search-data-hound-search-0 2026-06-10 06:45:55.295486 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting pvc {"name": "logserver-logserver-0", "status": "Bound"} 2026-06-10 06:45:55.295515 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting object, name: logserver-logserver-0 2026-06-10 06:45:55.298407 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting pvc {"name": "mariadb-logs-mariadb-0", "status": "Bound"} 2026-06-10 06:45:55.298452 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting object, name: mariadb-logs-mariadb-0 2026-06-10 06:45:55.301835 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting pvc {"name": "mariadb-mariadb-0", "status": "Bound"} 2026-06-10 06:45:55.301875 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting object, name: mariadb-mariadb-0 2026-06-10 06:45:55.304768 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting pvc {"name": "nodepool-builder-nodepool-builder-0", "status": "Bound"} 2026-06-10 06:45:55.304805 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting object, name: nodepool-builder-nodepool-builder-0 2026-06-10 06:45:55.307983 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting pvc {"name": "zookeeper-data-zookeeper-0", "status": "Bound"} 2026-06-10 06:45:55.308003 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting object, name: zookeeper-data-zookeeper-0 2026-06-10 06:45:55.315107 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting pvc {"name": "zookeeper-logs-zookeeper-0", "status": "Bound"} 2026-06-10 06:45:55.315136 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting object, name: zookeeper-logs-zookeeper-0 2026-06-10 06:45:55.318889 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting pvc {"name": "zuul-executor-zuul-executor-0", "status": "Bound"} 2026-06-10 06:45:55.318912 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting object, name: zuul-executor-zuul-executor-0 2026-06-10 06:45:55.322037 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting pvc {"name": "zuul-executor-zuul-executor-1", "status": "Bound"} 2026-06-10 06:45:55.322060 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting object, name: zuul-executor-zuul-executor-1 2026-06-10 06:45:55.325160 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting pvc {"name": "zuul-merger-zuul-merger-0", "status": "Bound"} 2026-06-10 06:45:55.325182 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting object, name: zuul-merger-zuul-merger-0 2026-06-10 06:45:55.493418 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting pvc {"name": "zuul-merger-zuul-merger-1", "status": "Bound"} 2026-06-10 06:45:55.493493 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting object, name: zuul-merger-zuul-merger-1 2026-06-10 06:45:55.695439 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting pvc {"name": "zuul-scheduler-zuul-scheduler-0", "status": "Bound"} 2026-06-10 06:45:55.695525 | controller | 2026-06-10T06:45:55-04:00 INFO Deleting object, name: zuul-scheduler-zuul-scheduler-0 2026-06-10 10:45:58.172096 | controller | ok: Runtime: 0:01:10.634681 2026-06-10 10:45:58.186396 | 2026-06-10 10:45:58.186577 | TASK [health-check/test-backup-restore : Restore backup of the Software Factory previous deployment] 2026-06-10 06:45:58.793161 | controller | 2026-06-10T06:45:58-04:00 INFO OPENSHIFT_USER environment variable is not set, discovering Kubernetes Distribution 2026-06-10 06:45:58.793257 | controller | 2026-06-10 06:45:58.802467 | controller | 2026-06-10T06:45:58-04:00 INFO Kubernetes Distribution found: Kubernetes 2026-06-10 06:45:58.802520 | controller | 2026-06-10 06:45:58.813956 | controller | 2026-06-10T06:45:58-04:00 INFO Restoring secrets... 2026-06-10 06:45:58.814435 | controller | 2026-06-10T06:45:58-04:00 INFO Creating object, name: zookeeper-client-tls 2026-06-10 06:45:58.819040 | controller | 2026-06-10T06:45:58-04:00 INFO Creating object, name: zookeeper-server-tls 2026-06-10 06:45:58.821867 | controller | 2026-06-10T06:45:58-04:00 INFO Creating object, name: nodepool-builder-ssh-key 2026-06-10 06:45:58.825781 | controller | 2026-06-10T06:45:58-04:00 INFO Creating object, name: zuul-ssh-key 2026-06-10 06:45:58.829267 | controller | 2026-06-10T06:45:58-04:00 INFO Creating object, name: zuul-spare-ssh-key 2026-06-10 06:45:58.832376 | controller | 2026-06-10T06:45:58-04:00 INFO Creating object, name: zuul-keystore-password 2026-06-10 06:45:58.834745 | controller | 2026-06-10T06:45:58-04:00 INFO Creating object, name: zuul-auth-secret 2026-06-10 06:45:58.837410 | controller | 2026-06-10T06:45:58-04:00 INFO Creating object, name: logserver-keys 2026-06-10 06:45:58.839668 | controller | 2026-06-10T06:45:58-04:00 INFO Creating object, name: logserver-uploader-keys 2026-06-10 06:45:58.842335 | controller | 2026-06-10T06:45:58-04:00 INFO Creating object, name: logserver-uploader-spare-keys 2026-06-10 06:45:58.845230 | controller | 2026-06-10T06:45:58-04:00 INFO Spawning backend services... 2026-06-10 06:45:59.018385 | controller | 2026-06-10T06:45:59-04:00 INFO Creating secret, name: mariadb-root-password 2026-06-10 06:45:59.018454 | controller | 2026-06-10T06:45:59-04:00 INFO Creating object, name: mariadb-root-password 2026-06-10 06:45:59.419277 | controller | 2026-06-10T06:45:59-04:00 INFO Creating secret, name: mariadb-config-secrets 2026-06-10 06:45:59.419364 | controller | 2026-06-10T06:45:59-04:00 INFO Creating object, name: mariadb-config-secrets 2026-06-10 06:45:59.818479 | controller | 2026-06-10T06:45:59-04:00 INFO Creating secret, name: mariadb-initdb-secrets 2026-06-10 06:45:59.818560 | controller | 2026-06-10T06:45:59-04:00 INFO Creating object, name: mariadb-initdb-secrets 2026-06-10 06:46:00.024720 | controller | 2026-06-10T06:46:00-04:00 INFO Creating object, name: mariadb 2026-06-10 06:46:00.420441 | controller | 2026-06-10T06:46:00-04:00 INFO Creating config map name: zookeeper-pi-config-map 2026-06-10 06:46:00.420515 | controller | 2026-06-10T06:46:00-04:00 INFO Creating object, name: zookeeper-pi-config-map 2026-06-10 06:46:00.425800 | controller | 2026-06-10T06:46:00-04:00 INFO Creating service, name: zookeeper-headless 2026-06-10 06:46:00.425838 | controller | 2026-06-10T06:46:00-04:00 INFO Creating object, name: zookeeper-headless 2026-06-10 06:46:00.432188 | controller | 2026-06-10T06:46:00-04:00 INFO Creating service, name: zookeeper 2026-06-10 06:46:00.432216 | controller | 2026-06-10T06:46:00-04:00 INFO Creating object, name: zookeeper 2026-06-10 06:46:00.439621 | controller | 2026-06-10T06:46:00-04:00 INFO Creating object, name: zookeeper 2026-06-10 06:46:00.444025 | controller | 2026-06-10T06:46:00-04:00 INFO Waiting for backend services... 2026-06-10 06:46:01.024621 | controller | 2026-06-10T06:46:01-04:00 INFO Creating service, name: mariadb 2026-06-10 06:46:01.024706 | controller | 2026-06-10T06:46:01-04:00 INFO Creating object, name: mariadb 2026-06-10 06:46:01.036362 | controller | 2026-06-10T06:46:01-04:00 INFO Waiting for statefulset, name: mariadb 2026-06-10 06:46:27.127759 | controller | 2026-06-10T06:46:27-04:00 INFO Creating object, name: zuul-db-connection 2026-06-10 06:46:27.134311 | controller | 2026-06-10T06:46:27-04:00 INFO Creating object, name: zuul-db-provision 2026-06-10 06:46:32.824998 | controller | 2026-06-10T06:46:32-04:00 INFO Creating secret, name: zuul-config 2026-06-10 06:46:32.825062 | controller | 2026-06-10T06:46:32-04:00 INFO Creating object, name: zuul-config 2026-06-10 06:46:33.027699 | controller | 2026-06-10T06:46:33-04:00 INFO Creating config map name: zuul-scheduler-tooling-config-map 2026-06-10 06:46:33.027781 | controller | 2026-06-10T06:46:33-04:00 INFO Creating object, name: zuul-scheduler-tooling-config-map 2026-06-10 06:46:33.032149 | controller | 2026-06-10T06:46:33-04:00 INFO Creating object, name: zuul-kazoo 2026-06-10 06:46:35.041868 | controller | 2026-06-10T06:46:35-04:00 INFO Restoring Zuul... 2026-06-10 06:46:35.042037 | controller | 2026-06-10T06:46:35-04:00 INFO Running pod execution pod: zuul-kazoo, command: [bash -c rm -rf /tmp/zuul-import && mkdir -p /tmp/zuul-import] 2026-06-10 06:46:35.191576 | controller | 2026-06-10T06:46:35-04:00 INFO Running pod execution pod: zuul-kazoo, command: [bash -c cat > /tmp/zuul.keys && zuul-admin import-keys --force /tmp/zuul.keys] 2026-06-10 06:46:36.900914 | controller | INFO:kazoo.client:Connecting to zookeeper-0.zookeeper-headless.sf(10.244.0.116):2281, use_ssl: True 2026-06-10 06:46:37.077965 | controller | INFO:kazoo.client:Zookeeper connection established, state: CONNECTED 2026-06-10 06:46:37.484203 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/ssh 2026-06-10 06:46:37.488335 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/opendev.org/zuul/zuul%2Fzuul-jobs/secrets 2026-06-10 06:46:37.574154 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/demo-tenant-config/demo-tenant-config/ssh 2026-06-10 06:46:37.578163 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/demo-tenant-config/demo-tenant-config/secrets 2026-06-10 06:46:37.669075 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/demo-project/demo-project/ssh 2026-06-10 06:46:37.674157 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/demo-project/demo-project/secrets 2026-06-10 06:46:37.689745 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/config/config/ssh 2026-06-10 06:46:37.693073 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/config/config/secrets 2026-06-10 06:46:37.772994 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/ichiban-config-QEwn8WpM/ichiban-config-QEwn8WpM/ssh 2026-06-10 06:46:37.776136 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/gerrit/ichiban-config-QEwn8WpM/ichiban-config-QEwn8WpM/secrets 2026-06-10 06:46:37.865731 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/git-server/system-config/system-config/ssh 2026-06-10 06:46:37.869194 | controller | INFO:zuul.KeyStorage:Created key at /keystorage/git-server/system-config/system-config/secrets 2026-06-10 06:46:37.888935 | controller | INFO:zuul.KeyStorage:Created key at /keystorage2/tenant/ichiban-tenant-QEwn8WpM/ssh 2026-06-10 06:46:37.898788 | controller | INFO:zuul.KeyStorage:Created key at /keystorage2/tenant/internal/ssh 2026-06-10 06:46:37.973354 | controller | INFO:zuul.KeyStorage:Created key at /keystorage2/tenant/demo-tenant/ssh 2026-06-10 06:46:38.508632 | controller | 2026-06-10T06:46:38-04:00 INFO Finished doing Zuul private keys restore! 2026-06-10 06:46:38.508768 | controller | 2026-06-10T06:46:38-04:00 INFO Restoring DB... 2026-06-10 06:46:38.511419 | controller | 2026-06-10T06:46:38-04:00 INFO Running pod execution pod: mariadb-0, command: [mysql -e DROP DATABASE zuul;] 2026-06-10 06:46:38.591210 | controller | mysql: Deprecated program name. It will be removed in a future release, use '/usr/bin/mariadb' instead 2026-06-10 06:46:38.591339 | controller | Warning: World-writable config file '/etc/my.cnf.d/docker.cnf' is ignored 2026-06-10 06:46:38.698469 | controller | 2026-06-10T06:46:38-04:00 INFO Running pod execution pod: mariadb-0, command: [mysql -h0] 2026-06-10 06:46:38.789919 | controller | mysql: Deprecated program name. It will be removed in a future release, use '/usr/bin/mariadb' instead 2026-06-10 06:46:38.789992 | controller | Warning: World-writable config file '/etc/my.cnf.d/docker.cnf' is ignored 2026-06-10 06:46:39.104506 | controller | 2026-06-10T06:46:39-04:00 INFO Finished restoring DB from backup! 2026-06-10 06:46:39.108485 | controller | 2026-06-10T06:46:39-04:00 INFO Deleting object, name: zuul-kazoo 2026-06-10 06:46:39.127386 | controller | 2026-06-10T06:46:39-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:46:39.142305 | controller | 2026-06-10T06:46:39-04:00 INFO Creating config map name: git-server-pi-config-map 2026-06-10 06:46:39.142388 | controller | 2026-06-10T06:46:39-04:00 INFO Creating object, name: git-server-pi-config-map 2026-06-10 06:46:39.149712 | controller | 2026-06-10T06:46:39-04:00 INFO Creating object, name: git-server 2026-06-10 06:46:39.527600 | controller | 2026-06-10T06:46:39-04:00 INFO Creating config map name: logserver-config-map 2026-06-10 06:46:39.527721 | controller | 2026-06-10T06:46:39-04:00 INFO Creating object, name: logserver-config-map 2026-06-10 06:46:39.531721 | controller | 2026-06-10T06:46:39-04:00 INFO Creating service, name: logserver 2026-06-10 06:46:39.531797 | controller | 2026-06-10T06:46:39-04:00 INFO Creating object, name: logserver 2026-06-10 06:46:39.726297 | controller | 2026-06-10T06:46:39-04:00 INFO Creating object, name: logserver 2026-06-10 06:46:39.732332 | controller | 2026-06-10T06:46:39-04:00 INFO Creating object, name: gateway 2026-06-10 06:46:39.739814 | controller | 2026-06-10T06:46:39-04:00 INFO Creating config map name: gateway-config-map 2026-06-10 06:46:39.739892 | controller | 2026-06-10T06:46:39-04:00 INFO Creating object, name: gateway-config-map 2026-06-10 06:46:39.745097 | controller | 2026-06-10T06:46:39-04:00 INFO Creating object, name: gateway 2026-06-10 06:46:39.751644 | controller | 2026-06-10T06:46:39-04:00 INFO Creating service, name: hound-search 2026-06-10 06:46:39.751714 | controller | 2026-06-10T06:46:39-04:00 INFO Creating object, name: hound-search 2026-06-10 06:46:39.761807 | controller | 2026-06-10T06:46:39-04:00 INFO Creating object, name: hound-search 2026-06-10 06:46:39.769896 | controller | 2026-06-10T06:46:39-04:00 INFO Creating object, name: logjuicer-pvc 2026-06-10 06:46:39.774915 | controller | 2026-06-10T06:46:39-04:00 INFO Creating object, name: logjuicer 2026-06-10 06:46:39.786036 | controller | 2026-06-10T06:46:39-04:00 INFO Creating object, name: logjuicer 2026-06-10 06:46:40.140120 | controller | 2026-06-10T06:46:40-04:00 INFO Waiting for GitServer and MariaDB services to be ready ... 2026-06-10 06:46:40.140189 | controller | 2026-06-10T06:46:40-04:00 INFO 2026-06-10 06:46:40.140201 | controller | - Config: Waiting ... 2026-06-10 06:46:40.140210 | controller | - Gateway: Waiting ... 2026-06-10 06:46:40.140219 | controller | - GitServer: Waiting ... 2026-06-10 06:46:40.140228 | controller | - HoundSearch: Waiting ... 2026-06-10 06:46:40.140236 | controller | - LogJuicer: Waiting ... 2026-06-10 06:46:40.140245 | controller | - Logserver: Waiting ... 2026-06-10 06:46:40.140253 | controller | - MariaDB: OK 2026-06-10 06:46:40.140262 | controller | - Zookeeper: OK 2026-06-10 06:46:40.140270 | controller | 2026-06-10 06:46:40.140288 | controller | 2026-06-10T06:46:40-04:00 INFO [attempt #1] Waiting 5s for the next reconcile call ... 2026-06-10 06:46:45.145381 | controller | 2026-06-10T06:46:45-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:46:45.174460 | controller | 2026-06-10T06:46:45-04:00 INFO Creating service, name: git-server 2026-06-10 06:46:45.174533 | controller | 2026-06-10T06:46:45-04:00 INFO Creating object, name: git-server 2026-06-10 06:46:45.184734 | controller | 2026-06-10T06:46:45-04:00 INFO Creating service, name: git-server-rw 2026-06-10 06:46:45.184801 | controller | 2026-06-10T06:46:45-04:00 INFO Creating object, name: git-server-rw 2026-06-10 06:46:45.196840 | controller | 2026-06-10T06:46:45-04:00 INFO Waiting for statefulset, name: git-server 2026-06-10 06:47:03.296592 | controller | 2026-06-10T06:47:03-04:00 INFO Running pod execution pod: logserver-0, command: [cat /tmp/home/.ssh/authorized_keys] 2026-06-10 06:47:03.396259 | controller | 2026-06-10T06:47:03-04:00 INFO Logserver authorized_keys is up to date 2026-06-10 06:47:03.401403 | controller | 2026-06-10T06:47:03-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:47:03.401435 | controller | 2026-06-10T06:47:03-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:47:03.413906 | controller | 2026-06-10T06:47:03-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:47:03.413957 | controller | 2026-06-10T06:47:03-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:47:03.427408 | controller | 2026-06-10T06:47:03-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:47:03.664347 | controller | 2026-06-10T06:47:03-04:00 INFO Creating config map name: np-statsd-config-map 2026-06-10 06:47:03.664432 | controller | 2026-06-10T06:47:03-04:00 INFO Creating object, name: np-statsd-config-map 2026-06-10 06:47:03.668181 | controller | 2026-06-10T06:47:03-04:00 INFO Creating config map name: nodepool-tooling-config-map 2026-06-10 06:47:03.668223 | controller | 2026-06-10T06:47:03-04:00 INFO Creating object, name: nodepool-tooling-config-map 2026-06-10 06:47:03.673270 | controller | 2026-06-10T06:47:03-04:00 INFO Creating config map name: nodepool-launcher-extra-config-config-map 2026-06-10 06:47:03.673353 | controller | 2026-06-10T06:47:03-04:00 INFO Creating object, name: nodepool-launcher-extra-config-config-map 2026-06-10 06:47:03.693105 | controller | 2026-06-10T06:47:03-04:00 INFO Creating object, name: zuul-capacity 2026-06-10 06:47:03.701652 | controller | 2026-06-10T06:47:03-04:00 INFO Creating object, name: nodepool-launcher 2026-06-10 06:47:04.087644 | controller | 2026-06-10T06:47:04-04:00 INFO Creating config map name: nodepool-builder-extra-config-config-map 2026-06-10 06:47:04.087739 | controller | 2026-06-10T06:47:04-04:00 INFO Creating object, name: nodepool-builder-extra-config-config-map 2026-06-10 06:47:04.491087 | controller | 2026-06-10T06:47:04-04:00 INFO Creating service, name: nodepool-builder 2026-06-10 06:47:04.491154 | controller | 2026-06-10T06:47:04-04:00 INFO Creating object, name: nodepool-builder 2026-06-10 06:47:04.500212 | controller | 2026-06-10T06:47:04-04:00 INFO Creating object, name: nodepool-builder 2026-06-10 06:47:04.664394 | controller | 2026-06-10T06:47:04-04:00 INFO Creating object, name: zuul-web 2026-06-10 06:47:04.672549 | controller | 2026-06-10T06:47:04-04:00 INFO Creating object, name: zuul-executor-headless 2026-06-10 06:47:04.887630 | controller | 2026-06-10T06:47:04-04:00 INFO Creating config map name: zuul-extra-config-map 2026-06-10 06:47:04.887697 | controller | 2026-06-10T06:47:04-04:00 INFO Creating object, name: zuul-extra-config-map 2026-06-10 06:47:05.287642 | controller | 2026-06-10T06:47:05-04:00 INFO Creating config map name: zuul-statsd-config-map 2026-06-10 06:47:05.287701 | controller | 2026-06-10T06:47:05-04:00 INFO Creating object, name: zuul-statsd-config-map 2026-06-10 06:47:05.687017 | controller | 2026-06-10T06:47:05-04:00 INFO Creating config map name: zuul-logging-config-map 2026-06-10 06:47:05.687095 | controller | 2026-06-10T06:47:05-04:00 INFO Creating object, name: zuul-logging-config-map 2026-06-10 06:47:05.890460 | controller | 2026-06-10T06:47:05-04:00 INFO Creating object, name: zuul-scheduler 2026-06-10 06:47:06.690262 | controller | 2026-06-10T06:47:06-04:00 INFO Creating object, name: zuul-executor 2026-06-10 06:47:07.489518 | controller | 2026-06-10T06:47:07-04:00 INFO Creating object, name: zuul-web 2026-06-10 06:47:08.290601 | controller | 2026-06-10T06:47:08-04:00 INFO Creating object, name: zuul-merger 2026-06-10 06:47:08.295510 | controller | 2026-06-10T06:47:08-04:00 INFO 2026-06-10 06:47:08.295546 | controller | - Config: Waiting ... 2026-06-10 06:47:08.295557 | controller | - Gateway: OK 2026-06-10 06:47:08.295566 | controller | - GitServer: OK 2026-06-10 06:47:08.295575 | controller | - HoundSearch: OK 2026-06-10 06:47:08.295582 | controller | - LogJuicer: OK 2026-06-10 06:47:08.295590 | controller | - Logserver: OK 2026-06-10 06:47:08.295598 | controller | - MariaDB: OK 2026-06-10 06:47:08.295632 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:47:08.295644 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:47:08.295652 | controller | - Zookeeper: OK 2026-06-10 06:47:08.295659 | controller | - Zuul: Waiting ... 2026-06-10 06:47:08.295668 | controller | - |_ Config Secrets: OK 2026-06-10 06:47:08.295676 | controller | - |_ Executor: Waiting ... 2026-06-10 06:47:08.295684 | controller | - |_ Merger: Waiting ... 2026-06-10 06:47:08.295691 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:47:08.295699 | controller | - |_ Web: Waiting ... 2026-06-10 06:47:08.295707 | controller | 2026-06-10 06:47:08.295716 | controller | 2026-06-10T06:47:08-04:00 INFO [attempt #2] Waiting 5s for the next reconcile call ... 2026-06-10 06:47:13.302930 | controller | 2026-06-10T06:47:13-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:47:13.915728 | controller | 2026-06-10T06:47:13-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:47:13.915827 | controller | 2026-06-10T06:47:13-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:47:13.928901 | controller | 2026-06-10T06:47:13-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:47:13.928961 | controller | 2026-06-10T06:47:13-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:47:14.311553 | controller | 2026-06-10T06:47:14-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:47:14.714358 | controller | 2026-06-10T06:47:14-04:00 INFO Creating object, name: nodepool-launcher 2026-06-10 06:47:14.720844 | controller | 2026-06-10T06:47:14-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:47:14.720950 | controller | 2026-06-10T06:47:14-04:00 INFO deployment rollout not ok, name: nodepool-launcher 2026-06-10 06:47:15.728883 | controller | 2026-06-10T06:47:15-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:47:45.827781 | controller | 2026-06-10T06:47:45-04:00 INFO Statefulset "zuul-scheduler" not ready after 30s 2026-06-10 06:47:45.866443 | controller | 2026-06-10T06:47:45-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:47:45.866547 | controller | 2026-06-10T06:47:45-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:47:46.239774 | controller | 2026-06-10T06:47:46-04:00 INFO 2026-06-10 06:47:46.239842 | controller | - Config: Waiting ... 2026-06-10 06:47:46.239849 | controller | - Gateway: OK 2026-06-10 06:47:46.239855 | controller | - GitServer: OK 2026-06-10 06:47:46.239860 | controller | - HoundSearch: OK 2026-06-10 06:47:46.239865 | controller | - LogJuicer: OK 2026-06-10 06:47:46.239870 | controller | - Logserver: OK 2026-06-10 06:47:46.239875 | controller | - MariaDB: OK 2026-06-10 06:47:46.239880 | controller | - NodePoolBuilder: Waiting ... 2026-06-10 06:47:46.239885 | controller | - NodePoolLauncher: Waiting ... 2026-06-10 06:47:46.239890 | controller | - Zookeeper: OK 2026-06-10 06:47:46.239895 | controller | - Zuul: Waiting ... 2026-06-10 06:47:46.239900 | controller | - |_ Config Secrets: OK 2026-06-10 06:47:46.239905 | controller | - |_ Executor: OK 2026-06-10 06:47:46.239909 | controller | - |_ Merger: OK 2026-06-10 06:47:46.239914 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:47:46.239919 | controller | - |_ Web: Waiting ... 2026-06-10 06:47:46.239924 | controller | 2026-06-10 06:47:46.239930 | controller | 2026-06-10T06:47:46-04:00 INFO [attempt #3] Waiting 5s for the next reconcile call ... 2026-06-10 06:47:51.247055 | controller | 2026-06-10T06:47:51-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:47:51.857922 | controller | 2026-06-10T06:47:51-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:47:51.858007 | controller | 2026-06-10T06:47:51-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:47:51.872022 | controller | 2026-06-10T06:47:51-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:47:51.872088 | controller | 2026-06-10T06:47:51-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:47:52.262398 | controller | 2026-06-10T06:47:52-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:47:52.663875 | controller | 2026-06-10T06:47:52-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:47:52.663949 | controller | 2026-06-10T06:47:52-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:47:53.679507 | controller | 2026-06-10T06:47:53-04:00 INFO Waiting for statefulset, name: zuul-scheduler 2026-06-10 06:48:09.742528 | controller | 2026-06-10T06:48:09-04:00 INFO Creating object, name: zuul-weeder 2026-06-10 06:48:09.752579 | controller | 2026-06-10T06:48:09-04:00 INFO Creating object, name: zuul-weeder 2026-06-10 06:48:09.803933 | controller | 2026-06-10T06:48:09-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:48:09.804002 | controller | 2026-06-10T06:48:09-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:48:10.168882 | controller | 2026-06-10T06:48:10-04:00 INFO 2026-06-10 06:48:10.168949 | controller | - Config: Waiting ... 2026-06-10 06:48:10.168957 | controller | - Gateway: OK 2026-06-10 06:48:10.168963 | controller | - GitServer: OK 2026-06-10 06:48:10.168968 | controller | - HoundSearch: OK 2026-06-10 06:48:10.168973 | controller | - LogJuicer: OK 2026-06-10 06:48:10.168979 | controller | - Logserver: OK 2026-06-10 06:48:10.168984 | controller | - MariaDB: OK 2026-06-10 06:48:10.168989 | controller | - NodePoolBuilder: OK 2026-06-10 06:48:10.168994 | controller | - NodePoolLauncher: OK 2026-06-10 06:48:10.168999 | controller | - Zookeeper: OK 2026-06-10 06:48:10.169004 | controller | - Zuul: Waiting ... 2026-06-10 06:48:10.169010 | controller | - |_ Config Secrets: OK 2026-06-10 06:48:10.169015 | controller | - |_ Executor: OK 2026-06-10 06:48:10.169020 | controller | - |_ Merger: OK 2026-06-10 06:48:10.169025 | controller | - |_ Scheduler: Waiting ... 2026-06-10 06:48:10.169031 | controller | - |_ Web: OK 2026-06-10 06:48:10.169037 | controller | 2026-06-10 06:48:10.169043 | controller | 2026-06-10T06:48:10-04:00 INFO [attempt #4] Waiting 5s for the next reconcile call ... 2026-06-10 06:48:15.174300 | controller | 2026-06-10T06:48:15-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:48:15.786950 | controller | 2026-06-10T06:48:15-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:48:15.787125 | controller | 2026-06-10T06:48:15-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:48:15.804514 | controller | 2026-06-10T06:48:15-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:48:15.804580 | controller | 2026-06-10T06:48:15-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:48:16.185584 | controller | 2026-06-10T06:48:16-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:48:16.594181 | controller | 2026-06-10T06:48:16-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:48:16.594278 | controller | 2026-06-10T06:48:16-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:48:17.602252 | controller | 2026-06-10T06:48:17-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:48:17.602366 | controller | 2026-06-10T06:48:17-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:48:18.580008 | controller | 2026-06-10T06:48:18-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:48:18.580081 | controller | 2026-06-10T06:48:18-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:48:19.576349 | controller | 2026-06-10T06:48:19-04:00 INFO Creating config map name: pymod-sf-operator-config-map 2026-06-10 06:48:19.576450 | controller | 2026-06-10T06:48:19-04:00 INFO Creating object, name: pymod-sf-operator-config-map 2026-06-10 06:48:19.977800 | controller | 2026-06-10T06:48:19-04:00 INFO Creating object, name: config-updater 2026-06-10 06:48:19.984319 | controller | 2026-06-10T06:48:19-04:00 INFO Creating object, name: config-updater-role 2026-06-10 06:48:19.989219 | controller | 2026-06-10T06:48:19-04:00 INFO Creating object, name: config-updater 2026-06-10 06:48:19.993357 | controller | 2026-06-10T06:48:19-04:00 INFO Creating the config-update service account secret 2026-06-10 06:48:19.993392 | controller | 2026-06-10T06:48:19-04:00 INFO Creating object, name: config-update-secrets 2026-06-10 06:48:20.013075 | controller | 2026-06-10T06:48:20-04:00 INFO Creating base secret job 2026-06-10 06:48:20.013990 | controller | 2026-06-10T06:48:20-04:00 INFO Creating object, name: config-base-secret-1-8399-ns 2026-06-10 06:48:20.024402 | controller | 2026-06-10T06:48:20-04:00 INFO 2026-06-10 06:48:20.024479 | controller | - Config: Waiting ... 2026-06-10 06:48:20.024490 | controller | - Gateway: OK 2026-06-10 06:48:20.024500 | controller | - GitServer: OK 2026-06-10 06:48:20.024508 | controller | - HoundSearch: OK 2026-06-10 06:48:20.024517 | controller | - LogJuicer: OK 2026-06-10 06:48:20.024526 | controller | - Logserver: OK 2026-06-10 06:48:20.024534 | controller | - MariaDB: OK 2026-06-10 06:48:20.024542 | controller | - NodePoolBuilder: OK 2026-06-10 06:48:20.024551 | controller | - NodePoolLauncher: OK 2026-06-10 06:48:20.024559 | controller | - Zookeeper: OK 2026-06-10 06:48:20.024567 | controller | - Zuul: OK 2026-06-10 06:48:20.024577 | controller | 2026-06-10 06:48:20.024586 | controller | 2026-06-10T06:48:20-04:00 INFO [attempt #5] Waiting 5s for the next reconcile call ... 2026-06-10 06:48:25.032046 | controller | 2026-06-10T06:48:25-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:48:25.647583 | controller | 2026-06-10T06:48:25-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:48:25.647737 | controller | 2026-06-10T06:48:25-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:48:25.671481 | controller | 2026-06-10T06:48:25-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:48:25.671666 | controller | 2026-06-10T06:48:25-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:48:26.042379 | controller | 2026-06-10T06:48:26-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:48:26.443282 | controller | 2026-06-10T06:48:26-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:48:26.443353 | controller | 2026-06-10T06:48:26-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:48:27.449469 | controller | 2026-06-10T06:48:27-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:48:27.449545 | controller | 2026-06-10T06:48:27-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:48:28.434784 | controller | 2026-06-10T06:48:28-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:48:28.434847 | controller | 2026-06-10T06:48:28-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:48:29.644007 | controller | 2026-06-10T06:48:29-04:00 INFO Running tenant-reconfigure for the 'internal' tenant 2026-06-10 06:48:29.834355 | controller | 2026-06-10T06:48:29-04:00 INFO Running pod execution pod: zuul-scheduler-0, command: [zuul-scheduler tenant-reconfigure internal] 2026-06-10 06:48:35.586678 | controller | 2026-06-10T06:48:35-04:00 INFO Deleting object, name: zuul-web 2026-06-10 06:48:35.592739 | controller | 2026-06-10T06:48:35-04:00 INFO Creating object, name: zs-internal-tenant-reconfigure 2026-06-10 06:48:35.597042 | controller | 2026-06-10T06:48:35-04:00 INFO 2026-06-10 06:48:35.597131 | controller | - Config: Waiting ... 2026-06-10 06:48:35.597147 | controller | - Gateway: OK 2026-06-10 06:48:35.597159 | controller | - GitServer: OK 2026-06-10 06:48:35.597170 | controller | - HoundSearch: OK 2026-06-10 06:48:35.597181 | controller | - LogJuicer: OK 2026-06-10 06:48:35.597191 | controller | - Logserver: OK 2026-06-10 06:48:35.597202 | controller | - MariaDB: OK 2026-06-10 06:48:35.597213 | controller | - NodePoolBuilder: OK 2026-06-10 06:48:35.597224 | controller | - NodePoolLauncher: OK 2026-06-10 06:48:35.597235 | controller | - Zookeeper: OK 2026-06-10 06:48:35.597247 | controller | - Zuul: OK 2026-06-10 06:48:35.597259 | controller | 2026-06-10 06:48:35.597271 | controller | 2026-06-10T06:48:35-04:00 INFO [attempt #6] Waiting 5s for the next reconcile call ... 2026-06-10 06:48:40.605154 | controller | 2026-06-10T06:48:40-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:48:41.218117 | controller | 2026-06-10T06:48:41-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:48:41.218219 | controller | 2026-06-10T06:48:41-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:48:41.239145 | controller | 2026-06-10T06:48:41-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:48:41.239259 | controller | 2026-06-10T06:48:41-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:48:41.618758 | controller | 2026-06-10T06:48:41-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:48:42.021160 | controller | 2026-06-10T06:48:42-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:48:42.021243 | controller | 2026-06-10T06:48:42-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:48:43.040998 | controller | 2026-06-10T06:48:43-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:48:43.041094 | controller | 2026-06-10T06:48:43-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:48:44.007821 | controller | 2026-06-10T06:48:44-04:00 INFO Creating object, name: zuul-web 2026-06-10 06:48:44.813076 | controller | 2026-06-10T06:48:44-04:00 INFO 2026-06-10 06:48:44.813143 | controller | - Config: Waiting ... 2026-06-10 06:48:44.813149 | controller | - Gateway: OK 2026-06-10 06:48:44.813155 | controller | - GitServer: OK 2026-06-10 06:48:44.813159 | controller | - HoundSearch: OK 2026-06-10 06:48:44.813164 | controller | - LogJuicer: OK 2026-06-10 06:48:44.813169 | controller | - Logserver: OK 2026-06-10 06:48:44.813174 | controller | - MariaDB: OK 2026-06-10 06:48:44.813179 | controller | - NodePoolBuilder: OK 2026-06-10 06:48:44.813184 | controller | - NodePoolLauncher: OK 2026-06-10 06:48:44.813189 | controller | - Zookeeper: OK 2026-06-10 06:48:44.813195 | controller | - Zuul: Waiting ... 2026-06-10 06:48:44.813200 | controller | - |_ Config Secrets: OK 2026-06-10 06:48:44.813205 | controller | - |_ Executor: OK 2026-06-10 06:48:44.813210 | controller | - |_ Merger: OK 2026-06-10 06:48:44.813214 | controller | - |_ Scheduler: OK 2026-06-10 06:48:44.813219 | controller | - |_ Web: Waiting ... 2026-06-10 06:48:44.813224 | controller | 2026-06-10 06:48:44.813255 | controller | 2026-06-10T06:48:44-04:00 INFO [attempt #7] Waiting 5s for the next reconcile call ... 2026-06-10 06:48:49.824191 | controller | 2026-06-10T06:48:49-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:48:50.442947 | controller | 2026-06-10T06:48:50-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:48:50.443578 | controller | 2026-06-10T06:48:50-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:48:50.467573 | controller | 2026-06-10T06:48:50-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:48:50.467754 | controller | 2026-06-10T06:48:50-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:48:50.835967 | controller | 2026-06-10T06:48:50-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:48:51.232552 | controller | 2026-06-10T06:48:51-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:48:51.232637 | controller | 2026-06-10T06:48:51-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:48:52.250453 | controller | 2026-06-10T06:48:52-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:48:52.250544 | controller | 2026-06-10T06:48:52-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:48:53.229474 | controller | 2026-06-10T06:48:53-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:48:53.229557 | controller | 2026-06-10T06:48:53-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:48:54.033748 | controller | 2026-06-10T06:48:54-04:00 INFO 2026-06-10 06:48:54.033824 | controller | - Config: Waiting ... 2026-06-10 06:48:54.033833 | controller | - Gateway: OK 2026-06-10 06:48:54.033840 | controller | - GitServer: OK 2026-06-10 06:48:54.033845 | controller | - HoundSearch: OK 2026-06-10 06:48:54.033850 | controller | - LogJuicer: OK 2026-06-10 06:48:54.033855 | controller | - Logserver: OK 2026-06-10 06:48:54.033860 | controller | - MariaDB: OK 2026-06-10 06:48:54.033865 | controller | - NodePoolBuilder: OK 2026-06-10 06:48:54.033870 | controller | - NodePoolLauncher: OK 2026-06-10 06:48:54.033874 | controller | - Zookeeper: OK 2026-06-10 06:48:54.033879 | controller | - Zuul: Waiting ... 2026-06-10 06:48:54.033884 | controller | - |_ Config Secrets: OK 2026-06-10 06:48:54.033889 | controller | - |_ Executor: OK 2026-06-10 06:48:54.033894 | controller | - |_ Merger: OK 2026-06-10 06:48:54.033899 | controller | - |_ Scheduler: OK 2026-06-10 06:48:54.033904 | controller | - |_ Web: Waiting ... 2026-06-10 06:48:54.033909 | controller | 2026-06-10 06:48:54.033928 | controller | 2026-06-10T06:48:54-04:00 INFO [attempt #8] Waiting 5s for the next reconcile call ... 2026-06-10 06:48:59.043933 | controller | 2026-06-10T06:48:59-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:48:59.659349 | controller | 2026-06-10T06:48:59-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:48:59.659422 | controller | 2026-06-10T06:48:59-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:48:59.672465 | controller | 2026-06-10T06:48:59-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:48:59.672523 | controller | 2026-06-10T06:48:59-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:49:00.050277 | controller | 2026-06-10T06:49:00-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:49:00.456352 | controller | 2026-06-10T06:49:00-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:49:00.456424 | controller | 2026-06-10T06:49:00-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:49:01.467326 | controller | 2026-06-10T06:49:01-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:49:01.467395 | controller | 2026-06-10T06:49:01-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:49:02.446945 | controller | 2026-06-10T06:49:02-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:49:02.447005 | controller | 2026-06-10T06:49:02-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:49:03.257512 | controller | 2026-06-10T06:49:03-04:00 INFO 2026-06-10 06:49:03.257595 | controller | - Config: Waiting ... 2026-06-10 06:49:03.257651 | controller | - Gateway: OK 2026-06-10 06:49:03.257666 | controller | - GitServer: OK 2026-06-10 06:49:03.257674 | controller | - HoundSearch: OK 2026-06-10 06:49:03.257682 | controller | - LogJuicer: OK 2026-06-10 06:49:03.257690 | controller | - Logserver: OK 2026-06-10 06:49:03.257698 | controller | - MariaDB: OK 2026-06-10 06:49:03.257706 | controller | - NodePoolBuilder: OK 2026-06-10 06:49:03.257714 | controller | - NodePoolLauncher: OK 2026-06-10 06:49:03.257722 | controller | - Zookeeper: OK 2026-06-10 06:49:03.257730 | controller | - Zuul: Waiting ... 2026-06-10 06:49:03.257737 | controller | - |_ Config Secrets: OK 2026-06-10 06:49:03.257748 | controller | - |_ Executor: OK 2026-06-10 06:49:03.257756 | controller | - |_ Merger: OK 2026-06-10 06:49:03.257764 | controller | - |_ Scheduler: OK 2026-06-10 06:49:03.257772 | controller | - |_ Web: Waiting ... 2026-06-10 06:49:03.257780 | controller | 2026-06-10 06:49:03.257789 | controller | 2026-06-10T06:49:03-04:00 INFO [attempt #9] Waiting 5s for the next reconcile call ... 2026-06-10 06:49:08.267418 | controller | 2026-06-10T06:49:08-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:49:08.881525 | controller | 2026-06-10T06:49:08-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:49:08.881631 | controller | 2026-06-10T06:49:08-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:49:08.895693 | controller | 2026-06-10T06:49:08-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:49:08.895736 | controller | 2026-06-10T06:49:08-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:49:09.282471 | controller | 2026-06-10T06:49:09-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:49:09.681291 | controller | 2026-06-10T06:49:09-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:49:09.681355 | controller | 2026-06-10T06:49:09-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:49:10.700100 | controller | 2026-06-10T06:49:10-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:49:10.700187 | controller | 2026-06-10T06:49:10-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:49:11.670467 | controller | 2026-06-10T06:49:11-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:49:11.670532 | controller | 2026-06-10T06:49:11-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:49:12.476367 | controller | 2026-06-10T06:49:12-04:00 INFO 2026-06-10 06:49:12.476434 | controller | - Config: Waiting ... 2026-06-10 06:49:12.476439 | controller | - Gateway: OK 2026-06-10 06:49:12.476443 | controller | - GitServer: OK 2026-06-10 06:49:12.476447 | controller | - HoundSearch: OK 2026-06-10 06:49:12.476450 | controller | - LogJuicer: OK 2026-06-10 06:49:12.476454 | controller | - Logserver: OK 2026-06-10 06:49:12.476457 | controller | - MariaDB: OK 2026-06-10 06:49:12.476461 | controller | - NodePoolBuilder: OK 2026-06-10 06:49:12.476466 | controller | - NodePoolLauncher: OK 2026-06-10 06:49:12.476470 | controller | - Zookeeper: OK 2026-06-10 06:49:12.476473 | controller | - Zuul: Waiting ... 2026-06-10 06:49:12.476477 | controller | - |_ Config Secrets: OK 2026-06-10 06:49:12.476480 | controller | - |_ Executor: OK 2026-06-10 06:49:12.476484 | controller | - |_ Merger: OK 2026-06-10 06:49:12.476487 | controller | - |_ Scheduler: OK 2026-06-10 06:49:12.476491 | controller | - |_ Web: Waiting ... 2026-06-10 06:49:12.476495 | controller | 2026-06-10 06:49:12.476498 | controller | 2026-06-10T06:49:12-04:00 INFO [attempt #10] Waiting 5s for the next reconcile call ... 2026-06-10 06:49:17.482551 | controller | 2026-06-10T06:49:17-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:49:18.090970 | controller | 2026-06-10T06:49:18-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:49:18.091047 | controller | 2026-06-10T06:49:18-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:49:18.101393 | controller | 2026-06-10T06:49:18-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:49:18.101459 | controller | 2026-06-10T06:49:18-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:49:18.491876 | controller | 2026-06-10T06:49:18-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:49:18.896836 | controller | 2026-06-10T06:49:18-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:49:18.896924 | controller | 2026-06-10T06:49:18-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:49:19.899065 | controller | 2026-06-10T06:49:19-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:49:19.899152 | controller | 2026-06-10T06:49:19-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:49:20.886483 | controller | 2026-06-10T06:49:20-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:49:20.886555 | controller | 2026-06-10T06:49:20-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:49:21.692230 | controller | 2026-06-10T06:49:21-04:00 INFO 2026-06-10 06:49:21.692291 | controller | - Config: Waiting ... 2026-06-10 06:49:21.692297 | controller | - Gateway: OK 2026-06-10 06:49:21.692301 | controller | - GitServer: OK 2026-06-10 06:49:21.692304 | controller | - HoundSearch: OK 2026-06-10 06:49:21.692308 | controller | - LogJuicer: OK 2026-06-10 06:49:21.692311 | controller | - Logserver: OK 2026-06-10 06:49:21.692315 | controller | - MariaDB: OK 2026-06-10 06:49:21.692318 | controller | - NodePoolBuilder: OK 2026-06-10 06:49:21.692322 | controller | - NodePoolLauncher: OK 2026-06-10 06:49:21.692325 | controller | - Zookeeper: OK 2026-06-10 06:49:21.692329 | controller | - Zuul: Waiting ... 2026-06-10 06:49:21.692333 | controller | - |_ Config Secrets: OK 2026-06-10 06:49:21.692336 | controller | - |_ Executor: OK 2026-06-10 06:49:21.692340 | controller | - |_ Merger: OK 2026-06-10 06:49:21.692343 | controller | - |_ Scheduler: OK 2026-06-10 06:49:21.692347 | controller | - |_ Web: Waiting ... 2026-06-10 06:49:21.692351 | controller | 2026-06-10 06:49:21.692375 | controller | 2026-06-10T06:49:21-04:00 INFO [attempt #11] Waiting 5s for the next reconcile call ... 2026-06-10 06:49:26.700387 | controller | 2026-06-10T06:49:26-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:49:27.312386 | controller | 2026-06-10T06:49:27-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:49:27.312457 | controller | 2026-06-10T06:49:27-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:49:27.326327 | controller | 2026-06-10T06:49:27-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:49:27.326380 | controller | 2026-06-10T06:49:27-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:49:27.712919 | controller | 2026-06-10T06:49:27-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:49:28.122553 | controller | 2026-06-10T06:49:28-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:49:28.122687 | controller | 2026-06-10T06:49:28-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:49:29.134915 | controller | 2026-06-10T06:49:29-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:49:29.135017 | controller | 2026-06-10T06:49:29-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:49:30.103830 | controller | 2026-06-10T06:49:30-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:49:30.103904 | controller | 2026-06-10T06:49:30-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:49:30.912070 | controller | 2026-06-10T06:49:30-04:00 INFO 2026-06-10 06:49:30.912140 | controller | - Config: Waiting ... 2026-06-10 06:49:30.912147 | controller | - Gateway: OK 2026-06-10 06:49:30.912153 | controller | - GitServer: OK 2026-06-10 06:49:30.912159 | controller | - HoundSearch: OK 2026-06-10 06:49:30.912164 | controller | - LogJuicer: OK 2026-06-10 06:49:30.912169 | controller | - Logserver: OK 2026-06-10 06:49:30.912174 | controller | - MariaDB: OK 2026-06-10 06:49:30.912180 | controller | - NodePoolBuilder: OK 2026-06-10 06:49:30.912186 | controller | - NodePoolLauncher: OK 2026-06-10 06:49:30.912191 | controller | - Zookeeper: OK 2026-06-10 06:49:30.912196 | controller | - Zuul: Waiting ... 2026-06-10 06:49:30.912215 | controller | - |_ Config Secrets: OK 2026-06-10 06:49:30.912221 | controller | - |_ Executor: OK 2026-06-10 06:49:30.912226 | controller | - |_ Merger: OK 2026-06-10 06:49:30.912231 | controller | - |_ Scheduler: OK 2026-06-10 06:49:30.912236 | controller | - |_ Web: Waiting ... 2026-06-10 06:49:30.912242 | controller | 2026-06-10 06:49:30.912248 | controller | 2026-06-10T06:49:30-04:00 INFO [attempt #12] Waiting 5s for the next reconcile call ... 2026-06-10 06:49:35.920733 | controller | 2026-06-10T06:49:35-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:49:36.533529 | controller | 2026-06-10T06:49:36-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:49:36.533632 | controller | 2026-06-10T06:49:36-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:49:36.550921 | controller | 2026-06-10T06:49:36-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:49:36.550978 | controller | 2026-06-10T06:49:36-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:49:36.931330 | controller | 2026-06-10T06:49:36-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:49:37.336923 | controller | 2026-06-10T06:49:37-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:49:37.336994 | controller | 2026-06-10T06:49:37-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:49:38.351299 | controller | 2026-06-10T06:49:38-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:49:38.351379 | controller | 2026-06-10T06:49:38-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:49:39.322726 | controller | 2026-06-10T06:49:39-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:49:39.322794 | controller | 2026-06-10T06:49:39-04:00 INFO deployment rollout not ok, name: zuul-web 2026-06-10 06:49:40.129752 | controller | 2026-06-10T06:49:40-04:00 INFO 2026-06-10 06:49:40.129815 | controller | - Config: Waiting ... 2026-06-10 06:49:40.129820 | controller | - Gateway: OK 2026-06-10 06:49:40.129824 | controller | - GitServer: OK 2026-06-10 06:49:40.129828 | controller | - HoundSearch: OK 2026-06-10 06:49:40.129831 | controller | - LogJuicer: OK 2026-06-10 06:49:40.129835 | controller | - Logserver: OK 2026-06-10 06:49:40.129838 | controller | - MariaDB: OK 2026-06-10 06:49:40.129842 | controller | - NodePoolBuilder: OK 2026-06-10 06:49:40.129845 | controller | - NodePoolLauncher: OK 2026-06-10 06:49:40.129849 | controller | - Zookeeper: OK 2026-06-10 06:49:40.129852 | controller | - Zuul: Waiting ... 2026-06-10 06:49:40.129856 | controller | - |_ Config Secrets: OK 2026-06-10 06:49:40.129859 | controller | - |_ Executor: OK 2026-06-10 06:49:40.129863 | controller | - |_ Merger: OK 2026-06-10 06:49:40.129866 | controller | - |_ Scheduler: OK 2026-06-10 06:49:40.129870 | controller | - |_ Web: Waiting ... 2026-06-10 06:49:40.129873 | controller | 2026-06-10 06:49:40.129877 | controller | 2026-06-10T06:49:40-04:00 INFO [attempt #13] Waiting 5s for the next reconcile call ... 2026-06-10 06:49:45.140948 | controller | 2026-06-10T06:49:45-04:00 INFO Deleting object, name: ca-cert 2026-06-10 06:49:45.759192 | controller | 2026-06-10T06:49:45-04:00 INFO Waiting for deployment, name: gateway 2026-06-10 06:49:45.759307 | controller | 2026-06-10T06:49:45-04:00 INFO Checking deployment ready replicas, name: gateway 2026-06-10 06:49:45.782246 | controller | 2026-06-10T06:49:45-04:00 INFO Waiting for deployment, name: logjuicer 2026-06-10 06:49:45.782353 | controller | 2026-06-10T06:49:45-04:00 INFO Checking deployment ready replicas, name: logjuicer 2026-06-10 06:49:46.153030 | controller | 2026-06-10T06:49:46-04:00 INFO Deploying Zuul and Nodepool ... 2026-06-10 06:49:46.552295 | controller | 2026-06-10T06:49:46-04:00 INFO Waiting for deployment, name: nodepool-launcher 2026-06-10 06:49:46.552363 | controller | 2026-06-10T06:49:46-04:00 INFO Checking deployment ready replicas, name: nodepool-launcher 2026-06-10 06:49:47.556772 | controller | 2026-06-10T06:49:47-04:00 INFO Waiting for deployment, name: zuul-weeder 2026-06-10 06:49:47.556840 | controller | 2026-06-10T06:49:47-04:00 INFO Checking deployment ready replicas, name: zuul-weeder 2026-06-10 06:49:48.543152 | controller | 2026-06-10T06:49:48-04:00 INFO Waiting for deployment, name: zuul-web 2026-06-10 06:49:48.543212 | controller | 2026-06-10T06:49:48-04:00 INFO Checking deployment ready replicas, name: zuul-web 2026-06-10 06:49:49.746897 | controller | 2026-06-10T06:49:49-04:00 INFO 2026-06-10 06:49:49.746974 | controller | - Config: OK 2026-06-10 06:49:49.746982 | controller | - Gateway: OK 2026-06-10 06:49:49.746987 | controller | - GitServer: OK 2026-06-10 06:49:49.746992 | controller | - HoundSearch: OK 2026-06-10 06:49:49.746997 | controller | - LogJuicer: OK 2026-06-10 06:49:49.747002 | controller | - Logserver: OK 2026-06-10 06:49:49.747007 | controller | - MariaDB: OK 2026-06-10 06:49:49.747012 | controller | - NodePoolBuilder: OK 2026-06-10 06:49:49.747017 | controller | - NodePoolLauncher: OK 2026-06-10 06:49:49.747022 | controller | - Zookeeper: OK 2026-06-10 06:49:49.747027 | controller | - Zuul: OK 2026-06-10 06:49:49.747032 | controller | 2026-06-10 06:49:49.747038 | controller | 2026-06-10T06:49:49-04:00 INFO Updating controller configmap ... 2026-06-10 06:49:50.141215 | controller | 2026-06-10T06:49:50-04:00 INFO Standalone reconcile done. 2026-06-10 10:49:50.526468 | controller | ok: Runtime: 0:03:51.679041 2026-06-10 10:49:50.545172 | 2026-06-10 10:49:50.545318 | TASK [health-check/test-backup-restore : Fetch the last build ID reported by Zuul web (after restore)] 2026-06-10 10:49:51.070887 | controller | ok: OK (698 bytes) 2026-06-10 10:49:51.083341 | 2026-06-10 10:49:51.083569 | TASK [health-check/test-backup-restore : Ensure Zuul SQL DB well restored by checking last build ID] 2026-06-10 10:49:51.131881 | controller | ok: All assertions passed 2026-06-10 10:49:51.142857 | 2026-06-10 10:49:51.142988 | TASK [Run a Zuul job workflow in order to validate the restored system] 2026-06-10 10:49:51.179047 | controller | ok 2026-06-10 10:49:51.215478 | 2026-06-10 10:49:51.215577 | TASK [ansible.builtin.include_role : health-check/check-zuul-tenant] 2026-06-10 10:49:51.246065 | controller | ok 2026-06-10 10:49:51.280532 | 2026-06-10 10:49:51.280757 | TASK [health-check/check-zuul-tenant : Get configuration errors/warnings into the internal tenant] 2026-06-10 10:49:51.705642 | controller | ok: OK (2 bytes) 2026-06-10 10:49:51.717765 | 2026-06-10 10:49:51.717955 | TASK [health-check/check-zuul-tenant : Fail when errors/warnings into the internal tenant] 2026-06-10 10:49:51.755090 | controller | skipping: Conditional result was False 2026-06-10 10:49:51.777095 | 2026-06-10 10:49:51.777359 | TASK [health-check/config-update-zuul : Set random value] 2026-06-10 10:49:52.006219 | controller | ok 2026-06-10 10:49:52.018147 | 2026-06-10 10:49:52.018487 | TASK [health-check/config-update-zuul : Create vars to generate and submit resources addition] 2026-06-10 10:49:52.065990 | controller | ok 2026-06-10 10:49:52.076136 | 2026-06-10 10:49:52.076316 | TASK [ansible.builtin.include_role : create-gerrit-repository] 2026-06-10 10:49:52.101159 | controller | ok 2026-06-10 10:49:52.129136 | 2026-06-10 10:49:52.129267 | TASK [create-gerrit-repository : Create gerrit repository named ichiban-config-MJU37cMN] 2026-06-10 06:49:53.311276 | controller | 2026-06-10 10:49:53,310 INFO managesf.model.yamlbkd.engine - Resource engine is using /tmp/direct-apply as workdir 2026-06-10 06:49:53.311376 | controller | 2026-06-10 10:49:53,310 INFO managesf.model.yamlbkd.engine - Resources engine: direct apply resources requested 2026-06-10 06:49:53.377093 | controller | 2026-06-10 10:49:53,376 INFO managesf.GerritClient - ichiban-config-MJU37cMN-core: creating group (Team core for the ichiban-config-MJU37cMN repo) 2026-06-10 06:49:53.405394 | controller | 2026-06-10 10:49:53,404 INFO managesf.GerritClient - ichiban-config-MJU37cMN-core: deleting group member admin 2026-06-10 06:49:53.484016 | controller | 2026-06-10 10:49:53,482 INFO managesf.GerritClient - ichiban-config-MJU37cMN-ptl: creating group (Team lead for the ichiban-config-MJU37cMN repo) 2026-06-10 06:49:53.508046 | controller | 2026-06-10 10:49:53,507 INFO managesf.GerritClient - ichiban-config-MJU37cMN-ptl: deleting group member admin 2026-06-10 06:49:53.582052 | controller | 2026-06-10 10:49:53,581 INFO managesf.GerritClient - ichiban-config-MJU37cMN-ptl: adding group member admin@sfop.me 2026-06-10 06:49:53.667686 | controller | 2026-06-10 10:49:53,666 INFO managesf.GerritClient - ichiban-config-MJU37cMN: creating project (ichiban-config-MJU37cMN repository) 2026-06-10 06:49:53.769428 | controller | 2026-06-10 10:49:53,768 INFO managesf.services.gerrit.utils - [gerrit] Clone repository ichiban-config-MJU37cMN 2026-06-10 06:49:53.997047 | controller | 2026-06-10 10:49:53,996 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'clone', 'ssh://admin@gerrit-sshd:29418/ichiban-config-MJU37cMN', '/tmp/tmpzxefxq0q/clone-ichiban-config-MJU37cMN'] output 2026-06-10 06:49:53.997109 | controller | 2026-06-10 10:49:53,996 INFO managesf.services.gerrit.utils - Cloning into '/tmp/tmpzxefxq0q/clone-ichiban-config-MJU37cMN'... 2026-06-10 06:49:53.997120 | controller | Total 2 (delta 0), reused 0 (delta 0) 2026-06-10 06:49:53.997130 | controller | 2026-06-10 06:49:54.253161 | controller | 2026-06-10 10:49:54,252 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'fetch', '-a'] output 2026-06-10 06:49:54.253216 | controller | 2026-06-10 10:49:54,252 INFO managesf.services.gerrit.utils - 2026-06-10 06:49:54.296940 | controller | 2026-06-10 10:49:54,296 INFO managesf.services.gerrit.utils - [gerrit] Prepare push on config for repository ichiban-config-MJU37cMN 2026-06-10 06:49:54.503177 | controller | 2026-06-10 10:49:54,502 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'fetch', 'origin', 'refs/meta/config:refs/remotes/origin/meta/config'] output 2026-06-10 06:49:54.503255 | controller | 2026-06-10 10:49:54,502 INFO managesf.services.gerrit.utils - Total 4 (delta 0), reused 0 (delta 0) 2026-06-10 06:49:54.503261 | controller | From ssh://gerrit-sshd:29418/ichiban-config-MJU37cMN 2026-06-10 06:49:54.503265 | controller | * [new ref] refs/meta/config -> origin/meta/config 2026-06-10 06:49:54.503269 | controller | 2026-06-10 06:49:54.513637 | controller | 2026-06-10 10:49:54,512 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'checkout', 'meta/config'] output 2026-06-10 06:49:54.513732 | controller | 2026-06-10 10:49:54,513 INFO managesf.services.gerrit.utils - Switched to a new branch 'meta/config' 2026-06-10 06:49:54.513739 | controller | branch 'meta/config' set up to track 'origin/meta/config'. 2026-06-10 06:49:54.513743 | controller | 2026-06-10 06:49:54.514098 | controller | 2026-06-10 10:49:54,513 INFO managesf.services.gerrit.utils - [gerrit] Add file project.config to index 2026-06-10 06:49:54.522894 | controller | 2026-06-10 10:49:54,522 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'add', 'project.config'] output 2026-06-10 06:49:54.522934 | controller | 2026-06-10 10:49:54,522 INFO managesf.services.gerrit.utils - 2026-06-10 06:49:54.522978 | controller | 2026-06-10 10:49:54,522 INFO managesf.services.gerrit.utils - [gerrit] Add file groups to index 2026-06-10 06:49:54.530339 | controller | 2026-06-10 10:49:54,529 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'add', 'groups'] output 2026-06-10 06:49:54.530534 | controller | 2026-06-10 10:49:54,529 INFO managesf.services.gerrit.utils - 2026-06-10 06:49:54.539772 | controller | 2026-06-10 10:49:54,539 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'status', '-s'] output 2026-06-10 06:49:54.539805 | controller | 2026-06-10 10:49:54,539 INFO managesf.services.gerrit.utils - M groups 2026-06-10 06:49:54.539825 | controller | M project.config 2026-06-10 06:49:54.539830 | controller | 2026-06-10 06:49:54.565923 | controller | 2026-06-10 10:49:54,564 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'commit', '-a', '--author', 'admin ', '-mProvides ACL and Groups'] output 2026-06-10 06:49:54.565992 | controller | 2026-06-10 10:49:54,565 INFO managesf.services.gerrit.utils - [meta/config 618d948] Provides ACL and Groups 2026-06-10 06:49:54.566001 | controller | 2 files changed, 31 insertions(+), 9 deletions(-) 2026-06-10 06:49:54.566007 | controller | 2026-06-10 06:49:54.819702 | controller | 2026-06-10 10:49:54,818 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'push', 'origin', 'meta/config:meta/config'] output 2026-06-10 06:49:54.819771 | controller | 2026-06-10 10:49:54,819 INFO managesf.services.gerrit.utils - remote: remote: Processing changes: refs: 1 remote: Processing changes: refs: 1 remote: Processing changes: refs: 1, done 2026-06-10 06:49:54.819784 | controller | To ssh://gerrit-sshd:29418/ichiban-config-MJU37cMN 2026-06-10 06:49:54.819790 | controller | 7446beb..618d948 meta/config -> refs/meta/config 2026-06-10 06:49:54.819794 | controller | 2026-06-10 06:49:54.819800 | controller | 2026-06-10 10:49:54,819 INFO managesf.services.gerrit.utils - [gerrit] Push on config for repository ichiban-config-MJU37cMN 2026-06-10 06:49:54.819804 | controller | 2026-06-10 10:49:54,819 INFO managesf.services.gerrit.utils - [gerrit] Prepare push on master for repository ichiban-config-MJU37cMN 2026-06-10 06:49:54.826394 | controller | 2026-06-10 10:49:54,825 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'checkout', 'master'] output 2026-06-10 06:49:54.826431 | controller | 2026-06-10 10:49:54,825 INFO managesf.services.gerrit.utils - Switched to branch 'master' 2026-06-10 06:49:54.826437 | controller | Your branch is up to date with 'origin/master'. 2026-06-10 06:49:54.826441 | controller | 2026-06-10 06:49:54.831432 | controller | 2026-06-10 10:49:54,830 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'reset', '--hard', 'origin/master'] output 2026-06-10 06:49:54.831462 | controller | 2026-06-10 10:49:54,831 INFO managesf.services.gerrit.utils - HEAD is now at 4b184dd Initial empty repository 2026-06-10 06:49:54.831467 | controller | 2026-06-10 06:49:54.831574 | controller | 2026-06-10 10:49:54,831 INFO managesf.services.gerrit.utils - [gerrit] Add file .gitreview to index 2026-06-10 06:49:54.836167 | controller | 2026-06-10 10:49:54,835 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'add', '.gitreview'] output 2026-06-10 06:49:54.836187 | controller | 2026-06-10 10:49:54,835 INFO managesf.services.gerrit.utils - 2026-06-10 06:49:54.841053 | controller | 2026-06-10 10:49:54,840 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'status', '-s'] output 2026-06-10 06:49:54.841193 | controller | 2026-06-10 10:49:54,840 INFO managesf.services.gerrit.utils - A .gitreview 2026-06-10 06:49:54.841198 | controller | 2026-06-10 06:49:54.851203 | controller | 2026-06-10 10:49:54,850 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'commit', '-a', '--author', 'admin ', '-mManageSF commit'] output 2026-06-10 06:49:54.851224 | controller | 2026-06-10 10:49:54,850 INFO managesf.services.gerrit.utils - [master d2dbd9a] ManageSF commit 2026-06-10 06:49:54.851268 | controller | 1 file changed, 5 insertions(+) 2026-06-10 06:49:54.851274 | controller | create mode 100644 .gitreview 2026-06-10 06:49:54.851278 | controller | 2026-06-10 06:49:55.050943 | controller | 2026-06-10 10:49:55,050 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'push', 'origin', 'master'] output 2026-06-10 06:49:55.051005 | controller | 2026-06-10 10:49:55,050 INFO managesf.services.gerrit.utils - remote: remote: Processing changes: refs: 1 remote: Processing changes: refs: 1, done 2026-06-10 06:49:55.051013 | controller | To ssh://gerrit-sshd:29418/ichiban-config-MJU37cMN 2026-06-10 06:49:55.051018 | controller | 4b184dd..d2dbd9a master -> master 2026-06-10 06:49:55.051024 | controller | 2026-06-10 06:49:55.051028 | controller | 2026-06-10 10:49:55,050 INFO managesf.services.gerrit.utils - [gerrit] Push on master for repository ichiban-config-MJU37cMN 2026-06-10 06:49:55.051060 | controller | 2026-06-10 10:49:55,050 INFO managesf.services.gerrit.utils - [gerrit] List remote branches 2026-06-10 06:49:55.056415 | controller | 2026-06-10 10:49:55,055 INFO managesf.services.gerrit.utils - [gerrit] cmd ['git', 'branch', '-rv', '--abbrev=40'] output 2026-06-10 06:49:55.056473 | controller | 2026-06-10 10:49:55,055 INFO managesf.services.gerrit.utils - origin/HEAD -> origin/master 2026-06-10 06:49:55.056479 | controller | origin/master d2dbd9acec1833658f9ab6bb7c730de152677f0e ManageSF commit 2026-06-10 06:49:55.056483 | controller | origin/meta/config 7446beba42f48ed82fdb07a3520496cc56097832 Created project 2026-06-10 06:49:55.056487 | controller | 2026-06-10 06:49:55.056591 | controller | 2026-06-10 10:49:55,056 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-MJU37cMN-core] is going to be created. 2026-06-10 06:49:55.056600 | controller | 2026-06-10 10:49:55,056 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-MJU37cMN-ptl] is going to be created. 2026-06-10 06:49:55.056848 | controller | 2026-06-10 10:49:55,056 INFO managesf.model.yamlbkd.engine - Resource [type: acls, ID: ichiban-config-MJU37cMN-acl] is going to be created. 2026-06-10 06:49:55.056867 | controller | 2026-06-10 10:49:55,056 INFO managesf.model.yamlbkd.engine - Resource [type: repos, ID: ichiban-config-MJU37cMN] is going to be created. 2026-06-10 06:49:55.057056 | controller | 2026-06-10 10:49:55,056 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-MJU37cMN-core] will be created. 2026-06-10 06:49:55.057065 | controller | 2026-06-10 10:49:55,056 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-MJU37cMN-core] has been created. 2026-06-10 06:49:55.057557 | controller | 2026-06-10 10:49:55,056 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-MJU37cMN-ptl] will be created. 2026-06-10 06:49:55.057570 | controller | 2026-06-10 10:49:55,057 INFO managesf.model.yamlbkd.engine - Resource [type: groups, ID: ichiban-config-MJU37cMN-ptl] has been created. 2026-06-10 06:49:55.057714 | controller | 2026-06-10 10:49:55,057 INFO managesf.model.yamlbkd.engine - Resource [type: acls, ID: ichiban-config-MJU37cMN-acl] will be created. 2026-06-10 06:49:55.057725 | controller | 2026-06-10 06:49:55.057729 | controller | === Resources actions === 2026-06-10 06:49:55.057733 | controller | 2026-06-10 06:49:55.057737 | controller | Resource [type: groups, ID: ichiban-config-MJU37cMN-core] is going to be created. 2026-06-10 06:49:55.057741 | controller | Resource [type: groups, ID: ichiban-config-MJU37cMN-ptl] is going to be created. 2026-06-10 06:49:55.057745 | controller | Resource [type: acls, ID: ichiban-config-MJU37cMN-acl] is going to be created. 2026-06-10 06:49:55.057749 | controller | Resource [type: repos, ID: ichiban-config-MJU37cMN] is going to be created. 2026-06-10 06:49:55.057752 | controller | Resource [type: groups, ID: ichiban-config-MJU37cMN-core] will be created. 2026-06-10 06:49:55.057756 | controller | Resource [type: groups, ID: ichiban-config-MJU37cMN-core] has been created. 2026-06-10 06:49:55.057760 | controller | Resource [type: groups, ID: ichiban-config-MJU37cMN-ptl] will be created. 2026-06-10 06:49:55.057763 | controller | Resource [type: groups, ID: ichiban-config-MJU37cMN-ptl] has been created. 2026-06-10 06:49:55.057767 | controller | Resource [type: acls, ID: ichiban-config-MJU37cMN-acl] will be created. 2026-06-10 06:49:55.057770 | controller | Resource [type: acls, ID: ichiban-config-MJU37cMN-acl] has been created. 2026-06-10 06:49:55.057784 | controller | Resource [type: repos, ID: ichiban-config-MJU37cMN] will be created. 2026-06-10 06:49:55.057788 | controller | Resource [type: repos, ID: ichiban-config-MJU37cMN] has been created. 2026-06-10 06:49:55.057793 | controller | 2026-06-10 10:49:55,057 INFO managesf.model.yamlbkd.engine - Resource [type: acls, ID: ichiban-config-MJU37cMN-acl] has been created. 2026-06-10 06:49:55.057798 | controller | 2026-06-10 10:49:55,057 INFO managesf.model.yamlbkd.engine - Resource [type: repos, ID: ichiban-config-MJU37cMN] will be created. 2026-06-10 06:49:55.057868 | controller | 2026-06-10 10:49:55,057 INFO managesf.model.yamlbkd.engine - Resource [type: repos, ID: ichiban-config-MJU37cMN] has been created. 2026-06-10 10:49:55.663692 | controller | ok: Runtime: 0:00:02.867667 2026-06-10 10:49:55.676756 | 2026-06-10 10:49:55.676937 | TASK [create-gerrit-repository : Ensure repository exists in Gerrit] 2026-06-10 10:49:56.118392 | controller | ok: OK (752 bytes) 2026-06-10 10:49:56.135935 | 2026-06-10 10:49:56.136079 | TASK [health-check/config-update-zuul : Set bad Zuul tenant config] 2026-06-10 10:49:56.366334 | controller | changed: Block inserted 2026-06-10 10:49:56.376353 | 2026-06-10 10:49:56.376564 | LOOP [health-check/config-update-zuul : Commit file in the config repo] 2026-06-10 10:49:56.655539 | controller | ok: Item: git add zuul/main.yaml Runtime: 0:00:00.004672 2026-06-10 10:49:56.656005 | controller | changed: All items complete 2026-06-10 10:49:56.656084 | 2026-06-10 10:49:56.888198 | controller | [master cd5fec5] Add zuul/main.yaml 2026-06-10 10:49:56.888363 | controller | 1 file changed, 2 insertions(+), 5 deletions(-) 2026-06-10 10:49:56.888481 | controller | ok: Item: git commit -m 'Add zuul/main.yaml' Runtime: 0:00:00.042614 2026-06-10 10:49:56.904142 | 2026-06-10 10:49:56.904372 | TASK [ansible.builtin.include_role : health-check/repo-submit-change] 2026-06-10 10:49:56.943226 | controller | ok 2026-06-10 10:49:56.990999 | 2026-06-10 10:49:56.991097 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 10:49:57.114081 | controller | ok 2026-06-10 10:49:57.120101 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 10:49:57.126569 | 2026-06-10 10:49:57.126647 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 06:49:57.740581 | controller | remote: 2026-06-10 06:49:57.740727 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 06:49:57.740737 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-10 06:49:57.740743 | controller | remote: 2026-06-10 06:49:57.740750 | controller | remote: SUCCESS 2026-06-10 06:49:57.740756 | controller | remote: 2026-06-10 06:49:57.740765 | controller | remote: https://gerrit.sfop.me/c/config/+/7 Add zuul/main.yaml [NEW] 2026-06-10 06:49:57.740774 | controller | remote: 2026-06-10 06:49:57.740780 | controller | To https://gerrit.sfop.me/a/config 2026-06-10 06:49:57.740788 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 10:49:58.153628 | controller | ok: Runtime: 0:00:00.384987 2026-06-10 10:49:58.159562 | 2026-06-10 10:49:58.159625 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-10 06:49:58.383882 | controller | I4dc5a4a91bac37c39eb39c07413515ccb6929834 2026-06-10 10:49:58.690922 | controller | ok: Runtime: 0:00:00.006446 2026-06-10 10:49:58.701846 | 2026-06-10 10:49:58.702004 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 10:49:59.257756 | controller | ok: OK (47 bytes) 2026-06-10 10:49:59.270043 | 2026-06-10 10:49:59.270184 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 10:49:59.306583 | controller | ok 2026-06-10 10:49:59.322253 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-job-failure.yaml 2026-06-10 10:49:59.336609 | 2026-06-10 10:49:59.336695 | TASK [health-check/repo-submit-change : Get review ID from the changeID using gerrit API] 2026-06-10 10:49:59.818605 | controller | ok: OK (7142 bytes) 2026-06-10 10:49:59.829171 | 2026-06-10 10:49:59.831594 | TASK [health-check/repo-submit-change : ansible.builtin.set_fact] 2026-06-10 10:49:59.869982 | controller | ok 2026-06-10 10:49:59.883999 | 2026-06-10 10:49:59.884142 | TASK [health-check/repo-submit-change : Wait config-check FAILURE result using zuul-web API] 2026-06-10 10:50:52.664387 | controller | ok: OK (1163 bytes) 2026-06-10 10:50:52.676364 | 2026-06-10 10:50:52.676556 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 10:50:52.706529 | controller | skipping: Conditional result was False 2026-06-10 10:50:52.723158 | 2026-06-10 10:50:52.723327 | TASK [health-check/repo-submit-change : Ensure a success for config-update post job] 2026-06-10 10:50:52.772063 | controller | skipping: Conditional result was False 2026-06-10 10:50:52.786083 | 2026-06-10 10:50:52.786228 | TASK [health-check/repo-submit-change : Ensure a success for config-update job] 2026-06-10 10:50:52.824133 | controller | skipping: Conditional result was False 2026-06-10 10:50:52.845483 | 2026-06-10 10:50:52.845656 | TASK [health-check/config-update-zuul : Set Zuul tenant config] 2026-06-10 10:50:53.071458 | controller | changed: Block inserted 2026-06-10 10:50:53.075807 | 2026-06-10 10:50:53.075882 | LOOP [health-check/config-update-zuul : Commit file in the config repo] 2026-06-10 10:50:53.353656 | controller | ok: Item: git add zuul/main.yaml Runtime: 0:00:00.004780 2026-06-10 10:50:53.569779 | controller | [master 5c5aee7] Add zuul/main.yaml 2026-06-10 10:50:53.569942 | controller | Date: Wed Jun 10 06:49:56 2026 -0400 2026-06-10 10:50:53.569994 | controller | 1 file changed, 2 insertions(+), 2 deletions(-) 2026-06-10 10:50:53.570053 | controller | ok: Item: git commit --amend --no-edit Runtime: 0:00:00.029439 2026-06-10 10:50:53.588346 | 2026-06-10 10:50:53.588551 | TASK [Submit change and verify Zuul console output] 2026-06-10 10:50:53.627002 | controller | ok 2026-06-10 10:50:53.683588 | 2026-06-10 10:50:53.683743 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 10:50:53.714046 | controller | ok 2026-06-10 10:50:53.720131 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 10:50:53.726117 | 2026-06-10 10:50:53.726197 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 06:50:54.397846 | controller | remote: 2026-06-10 06:50:54.397901 | controller | remote: Processing changes: refs: 1, updated: 1 2026-06-10 06:50:54.397909 | controller | remote: Processing changes: refs: 1, updated: 1 2026-06-10 06:50:54.397915 | controller | remote: Processing changes: refs: 1, updated: 1 2026-06-10 06:50:54.397923 | controller | remote: Processing changes: refs: 1, updated: 1, done 2026-06-10 06:50:54.397930 | controller | remote: 2026-06-10 06:50:54.397936 | controller | remote: SUCCESS 2026-06-10 06:50:54.397942 | controller | remote: 2026-06-10 06:50:54.397947 | controller | remote: https://gerrit.sfop.me/c/config/+/7 Add zuul/main.yaml 2026-06-10 06:50:54.397954 | controller | remote: 2026-06-10 06:50:54.397959 | controller | To https://gerrit.sfop.me/a/config 2026-06-10 06:50:54.397966 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 10:50:54.758714 | controller | ok: Runtime: 0:00:00.510709 2026-06-10 10:50:54.770053 | 2026-06-10 10:50:54.770269 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-10 06:50:55.046051 | controller | I4dc5a4a91bac37c39eb39c07413515ccb6929834 2026-06-10 10:50:55.321839 | controller | ok: Runtime: 0:00:00.009996 2026-06-10 10:50:55.333306 | 2026-06-10 10:50:55.333496 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 10:50:55.996593 | controller | ok: OK (47 bytes) 2026-06-10 10:50:56.009527 | 2026-06-10 10:50:56.009735 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 10:50:56.038854 | controller | skipping: Conditional result was False 2026-06-10 10:50:56.047782 | 2026-06-10 10:50:56.047878 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 10:50:56.093141 | controller | ok 2026-06-10 10:50:56.106719 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-zuul-console-success.yaml 2026-06-10 10:50:56.124660 | 2026-06-10 10:50:56.124760 | TASK [health-check/repo-submit-change : Wait for config-update job] 2026-06-10 10:51:47.932746 | controller | ok: Runtime: 0:00:00.068836 2026-06-10 10:51:47.942547 | 2026-06-10 10:51:47.942703 | TASK [health-check/repo-submit-change : Connect and validate console stream] 2026-06-10 06:51:48.317853 | controller | 2026-06-10 10:51:43.970659 | Job console starting 2026-06-10 06:51:48.317906 | controller | 2026-06-10 10:51:43.971074 | Updating git repos 2026-06-10 06:51:48.317913 | controller | 2026-06-10 10:51:43.985547 | Cloning repos into workspace 2026-06-10 06:51:48.317919 | controller | 2026-06-10 10:51:44.702839 | Restoring repo states 2026-06-10 06:51:48.317977 | controller | 2026-06-10 10:51:44.711163 | Merging changes 2026-06-10 06:51:48.317985 | controller | 2026-06-10 10:51:44.711197 | Checking out repos 2026-06-10 06:51:48.317991 | controller | 2026-06-10 10:51:44.784251 | Preparing playbooks 2026-06-10 06:51:48.317995 | controller | 2026-06-10 10:51:45.348652 | Running Ansible setup 2026-06-10 06:51:48.318002 | controller | 2026-06-10 10:51:48.007209 | PRE-RUN START: [trusted : git-server/system-config/playbooks/base/pre.yaml@master] 2026-06-10 06:51:48.318006 | controller | 2026-06-10 06:51:49.318785 | controller | 2026-06-10 10:51:49.292694 | 2026-06-10 06:51:49.318831 | controller | 2026-06-10 10:51:49.292814 | PLAY [localhost] 2026-06-10 06:51:49.318840 | controller | 2026-06-10 10:51:49.300948 | 2026-06-10 06:51:49.318846 | controller | 2026-06-10 10:51:49.301051 | TASK [Gathering Facts] 2026-06-10 06:51:49.318851 | controller | 2026-06-10 06:51:50.819555 | controller | 2026-06-10 10:51:50.814429 | localhost | ok 2026-06-10 06:51:50.819644 | controller | 2026-06-10 06:51:51.319787 | controller | 2026-06-10 10:51:50.823922 | 2026-06-10 06:51:51.319844 | controller | 2026-06-10 10:51:50.824025 | TASK [Setup log path fact] 2026-06-10 06:51:51.319850 | controller | 2026-06-10 10:51:50.878248 | localhost | ok 2026-06-10 06:51:51.319855 | controller | 2026-06-10 10:51:50.892512 | 2026-06-10 06:51:51.319859 | controller | 2026-06-10 10:51:50.892651 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-06-10 06:51:51.319863 | controller | 2026-06-10 10:51:50.908885 | localhost | ok 2026-06-10 06:51:51.319867 | controller | 2026-06-10 10:51:50.916307 | 2026-06-10 06:51:51.319878 | controller | 2026-06-10 10:51:50.916391 | TASK [emit-job-header : Print job information] 2026-06-10 06:51:51.319882 | controller | 2026-06-10 10:51:50.993635 | # Job Information 2026-06-10 06:51:51.319886 | controller | 2026-06-10 10:51:50.993860 | Ansible Version: 2.16.18 2026-06-10 06:51:51.319890 | controller | 2026-06-10 10:51:50.993923 | Job: config-update 2026-06-10 06:51:51.319893 | controller | 2026-06-10 10:51:50.993964 | Pipeline: post 2026-06-10 06:51:51.319897 | controller | 2026-06-10 10:51:50.993999 | Executor: zuul-executor-0.zuul-executor-headless.sf.svc.cluster.local 2026-06-10 06:51:51.319901 | controller | 2026-06-10 10:51:50.994035 | Triggered by: https://gerrit.sfop.me/plugins/gitiles/config/+/5c5aee76e8e8eea29dfc52b8400bbefaa500a8b9^!/ 2026-06-10 06:51:51.319905 | controller | 2026-06-10 10:51:50.994077 | Log URL (when completed): https://sfop.me/logs/ed0/internal/ed0c9180383a46bf9c8981cd7070b25f/ 2026-06-10 06:51:51.319909 | controller | 2026-06-10 10:51:50.994114 | Event ID: 84b006a9f1c14a188e9c84b513460078 2026-06-10 06:51:51.319913 | controller | 2026-0 2026-06-10 06:51:51.319922 | controller | 6-10 10:51:50.999479 | 2026-06-10 06:51:51.319925 | controller | 2026-06-10 10:51:50.999659 | LOOP [emit-job-header : Print node information] 2026-06-10 06:51:51.319929 | controller | 2026-06-10 10:51:51.098035 | 2026-06-10 06:51:51.319932 | controller | 2026-06-10 10:51:51.098272 | TASK [log-inventory : Ensure Zuul Ansible directory exists] 2026-06-10 06:51:51.319936 | controller | 2026-06-10 06:51:52.320157 | controller | 2026-06-10 10:51:52.087087 | localhost -> localhost | changed 2026-06-10 06:51:52.320219 | controller | 2026-06-10 10:51:52.095541 | 2026-06-10 06:51:52.320231 | controller | 2026-06-10 10:51:52.095711 | TASK [log-inventory : Copy ansible inventory to logs dir] 2026-06-10 06:51:52.320241 | controller | 2026-06-10 10:51:53.493310 | controller | ok: Runtime: 0:00:05.016789 2026-06-10 10:51:53.508141 | 2026-06-10 10:51:53.508344 | TASK [health-check/repo-submit-change : Ensure a success for config-update post job] 2026-06-10 10:51:53.554678 | controller | ok 2026-06-10 10:51:53.567054 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-post-job-success.yaml 2026-06-10 10:51:53.586491 | 2026-06-10 10:51:53.586629 | TASK [health-check/repo-submit-change : Register commit sha] 2026-06-10 06:51:53.834700 | controller | 5c5aee76e8e8eea29dfc52b8400bbefaa500a8b9 2026-06-10 10:51:54.123466 | controller | ok: Runtime: 0:00:00.007153 2026-06-10 10:51:54.135860 | 2026-06-10 10:51:54.136026 | TASK [health-check/repo-submit-change : Wait config-update SUCCESS result using zuul-web API] 2026-06-10 10:53:09.247785 | controller | ok: OK (1143 bytes) 2026-06-10 10:53:09.263281 | 2026-06-10 10:53:09.263493 | TASK [health-check/repo-submit-change : Ensure a success for config-update job] 2026-06-10 10:53:09.302556 | controller | skipping: Conditional result was False 2026-06-10 10:53:09.418322 | 2026-06-10 10:53:09.418485 | TASK [health-check/config-update-zuul : ansible.builtin.set_fact] 2026-06-10 10:53:09.458140 | controller | ok 2026-06-10 10:53:09.462725 | 2026-06-10 10:53:09.462798 | TASK [Check that Zuul has pushed the job console on the logserver] 2026-06-10 10:53:09.492989 | controller | ok 2026-06-10 10:53:09.526211 | 2026-06-10 10:53:09.526344 | TASK [health-check/ensure-job-result-artifacts : Ensure job console (job-output.txt.gz) exists in Logserver] 2026-06-10 10:53:10.020599 | controller | ok: OK (4712 bytes) 2026-06-10 10:53:10.032283 | 2026-06-10 10:53:10.032476 | TASK [health-check/ensure-job-result-artifacts : Ensure internal logserver rewrite from job-output.txt to job-output.txt.gz] 2026-06-10 10:53:10.486028 | controller | ok: OK (4712 bytes) 2026-06-10 10:53:10.508882 | 2026-06-10 10:53:10.509057 | TASK [health-check/config-update-zuul : Get Zuul projects list] 2026-06-10 10:53:10.961877 | controller | ok: OK (144 bytes) 2026-06-10 10:53:10.971319 | 2026-06-10 10:53:10.971402 | TASK [health-check/config-update-zuul : Ensure project exists in Zuul (fail if not)] 2026-06-10 10:53:11.016715 | controller | skipping: Conditional result was False 2026-06-10 10:53:11.045952 | 2026-06-10 10:53:11.046134 | LOOP [health-check/test-nodepool-launcher-pod : Reset config dir to HEAD] 2026-06-10 10:53:11.416516 | controller | Fetching origin 2026-06-10 10:53:11.417065 | controller | From https://gerrit.sfop.me/a/demo-project 2026-06-10 10:53:11.417135 | controller | 766b7e1..a25a736 master -> origin/master 2026-06-10 10:53:11.417179 | controller | Fetching gerrit 2026-06-10 10:53:11.417254 | controller | ok: Item: git remote update Runtime: 0:00:00.086048 2026-06-10 10:53:11.417338 | controller | changed: All items complete 2026-06-10 10:53:11.417381 | 2026-06-10 10:53:11.597243 | controller | HEAD is now at a25a736 Update demo-job to sleep 1 second 2026-06-10 10:53:11.597423 | controller | ok: Item: git reset --hard origin/master Runtime: 0:00:00.004822 2026-06-10 10:53:11.611042 | 2026-06-10 10:53:11.611250 | LOOP [health-check/test-nodepool-launcher-pod : Ensure playbooks and zuul.d dir exists] 2026-06-10 10:53:11.902255 | controller | ok: "playbooks" 2026-06-10 10:53:12.102242 | controller | changed: "zuul.d" 2026-06-10 10:53:12.120854 | 2026-06-10 10:53:12.121082 | TASK [health-check/test-nodepool-launcher-pod : Create new zuul config] 2026-06-10 10:53:12.877693 | controller | changed 2026-06-10 10:53:12.887458 | 2026-06-10 10:53:12.887615 | TASK [health-check/test-nodepool-launcher-pod : Create simple playbook] 2026-06-10 10:53:13.602704 | controller | changed 2026-06-10 10:53:13.612058 | 2026-06-10 10:53:13.612190 | LOOP [health-check/test-nodepool-launcher-pod : Create git commit message] 2026-06-10 10:53:13.877836 | controller | ok: Item: git add . Runtime: 0:00:00.003965 2026-06-10 10:53:13.878320 | controller | changed: All items complete 2026-06-10 10:53:13.878392 | 2026-06-10 10:53:14.119288 | controller | [test_short_job e2c60a2] Provide a demo job for testing the local pod spawning 2026-06-10 10:53:14.119591 | controller | 2 files changed, 18 insertions(+) 2026-06-10 10:53:14.119677 | controller | create mode 100644 playbooks/microshift-pod.yml 2026-06-10 10:53:14.119749 | controller | create mode 100644 zuul.d/config.yaml 2026-06-10 10:53:14.120007 | controller | ok: Item: git commit -m 'Provide a demo job for testing the local pod spawning' Runtime: 0:00:00.038117 2026-06-10 10:53:14.133111 | 2026-06-10 10:53:14.133248 | TASK [Submit change and validate job success] 2026-06-10 10:53:14.170919 | controller | ok 2026-06-10 10:53:14.210206 | 2026-06-10 10:53:14.210373 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 10:53:14.241701 | controller | ok 2026-06-10 10:53:14.250621 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/submit-change.yaml 2026-06-10 10:53:14.258120 | 2026-06-10 10:53:14.258203 | TASK [health-check/repo-submit-change : Submit change] 2026-06-10 06:53:14.890234 | controller | remote: 2026-06-10 06:53:14.890297 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 06:53:14.890305 | controller | remote: Processing changes: refs: 1, new: 1 2026-06-10 06:53:14.890309 | controller | remote: Processing changes: refs: 1, new: 1, done 2026-06-10 06:53:14.890313 | controller | remote: commit e2c60a2: warning: subject >50 characters; use shorter first paragraph 2026-06-10 06:53:14.890320 | controller | remote: 2026-06-10 06:53:14.890324 | controller | remote: SUCCESS 2026-06-10 06:53:14.890328 | controller | remote: 2026-06-10 06:53:14.890332 | controller | remote: https://gerrit.sfop.me/c/demo-project/+/8 Provide a demo job for testing the local pod spawning [NEW] 2026-06-10 06:53:14.890337 | controller | remote: 2026-06-10 06:53:14.890341 | controller | To https://gerrit.sfop.me/a/demo-project 2026-06-10 06:53:14.890345 | controller | * [new reference] HEAD -> refs/for/master 2026-06-10 10:53:15.307010 | controller | ok: Runtime: 0:00:00.441880 2026-06-10 10:53:15.317535 | 2026-06-10 10:53:15.317671 | TASK [health-check/repo-submit-change : Get Change-id] 2026-06-10 06:53:15.547415 | controller | Id7906c4663bf93d33d865b942df68dd4fd5ec6b4 2026-06-10 10:53:15.867829 | controller | ok: Runtime: 0:00:00.009566 2026-06-10 10:53:15.877564 | 2026-06-10 10:53:15.877694 | TASK [health-check/repo-submit-change : Set Code-Review and Workflow label using gerrit API] 2026-06-10 10:53:15.906031 | controller | skipping: Conditional result was False 2026-06-10 10:53:15.920273 | 2026-06-10 10:53:15.920500 | TASK [health-check/repo-submit-change : Ensure a failure for config-check job] 2026-06-10 10:53:15.948180 | controller | skipping: Conditional result was False 2026-06-10 10:53:15.960646 | 2026-06-10 10:53:15.960806 | TASK [health-check/repo-submit-change : Verify Zuul Console output] 2026-06-10 10:53:15.989861 | controller | skipping: Conditional result was False 2026-06-10 10:53:16.006762 | 2026-06-10 10:53:16.006920 | TASK [health-check/repo-submit-change : Ensure a success for microshift-pod post job] 2026-06-10 10:53:16.042931 | controller | skipping: Conditional result was False 2026-06-10 10:53:16.056126 | 2026-06-10 10:53:16.056267 | TASK [health-check/repo-submit-change : Ensure a success for microshift-pod job] 2026-06-10 10:53:16.101731 | controller | ok 2026-06-10 10:53:16.117055 | controller | included: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/untrusted/project_0/softwarefactory-project.io/software-factory/sf-operator/roles/health-check/repo-submit-change/tasks/ensure-job-success.yaml 2026-06-10 10:53:16.132460 | 2026-06-10 10:53:16.132547 | TASK [health-check/repo-submit-change : Get review ID from the changeID using gerrit API] 2026-06-10 10:53:16.662281 | controller | ok: OK (2986 bytes) 2026-06-10 10:53:16.672681 | 2026-06-10 10:53:16.672831 | TASK [health-check/repo-submit-change : ansible.builtin.set_fact] 2026-06-10 10:53:16.731499 | controller | ok 2026-06-10 10:53:16.746104 | 2026-06-10 10:53:16.746296 | TASK [health-check/repo-submit-change : Wait microshift-pod SUCCESS result using zuul-web API] 2026-06-10 10:54:51.597648 | controller | ok: OK (798 bytes) 2026-06-10 10:54:51.630070 | 2026-06-10 10:54:51.630302 | TASK [health-check/test-volumestats-sidecar : ensure logserver deployment is ready] 2026-06-10 10:54:52.167501 | controller | ok: Runtime: 0:00:00.071662 2026-06-10 10:54:52.173730 | 2026-06-10 10:54:52.173818 | TASK [health-check/test-volumestats-sidecar : Start port forwarding to logserver sidecar] 2026-06-10 10:54:52.979869 | controller | ok: ERROR (ignored) 2026-06-10 10:54:52.980286 | controller | { 2026-06-10 10:54:52.980348 | controller | "ansible_job_id": "j307129362749.182691", 2026-06-10 10:54:52.980390 | controller | "finished": 0, 2026-06-10 10:54:52.980461 | controller | "results_file": "/home/zuul-worker/.ansible_async/j307129362749.182691", 2026-06-10 10:54:52.980508 | controller | "started": 1 2026-06-10 10:54:52.980544 | controller | } 2026-06-10 10:54:52.988992 | 2026-06-10 10:54:52.989121 | TASK [health-check/test-volumestats-sidecar : Pause to ensure the port forwarding is up] 2026-06-10 10:55:13.035499 | controller | ok 2026-06-10 10:55:13.044725 | 2026-06-10 10:55:13.044884 | TASK [health-check/test-volumestats-sidecar : Ensure volume info is exposed by metrics] 2026-06-10 10:55:13.609736 | controller | ok: OK (unknown bytes) 2026-06-10 10:55:13.628998 | 2026-06-10 10:55:13.629126 | TASK [Register CLI version] 2026-06-10 06:55:14.154870 | controller | v0.0.67-11-g2be8dba4 2026-06-10 10:55:14.202647 | controller | ok: Runtime: 0:00:00.302483 2026-06-10 10:55:14.213295 | 2026-06-10 10:55:14.213503 | TASK [ensure annotations are present on sf-standalone-owner] 2026-06-10 10:55:14.762664 | controller | ok: Runtime: 0:00:00.068909 2026-06-10 10:55:14.766169 | 2026-06-10 10:55:14.766271 | PLAY RECAP 2026-06-10 10:55:14.766360 | controller | ok: 367 changed: 219 unreachable: 0 failed: 0 skipped: 57 rescued: 0 ignored: 0 2026-06-10 10:55:14.766404 | 2026-06-10 10:55:15.111448 | RUN END RESULT_NORMAL: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/run-tests.yaml@master] 2026-06-10 10:55:15.112681 | POST-RUN START: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/get-minikube-logs.yaml@master] 2026-06-10 10:55:15.746408 | 2026-06-10 10:55:15.746555 | PLAY [controller] 2026-06-10 10:55:15.756104 | 2026-06-10 10:55:15.756178 | TASK [Grab minikube logs] 2026-06-10 10:55:17.783497 | controller | ok: Runtime: 0:00:01.458571 2026-06-10 10:55:17.790020 | 2026-06-10 10:55:17.790120 | TASK [ansible.builtin.include_role : post/get-kubectl-logs] 2026-06-10 10:55:17.812920 | controller | ok 2026-06-10 10:55:17.833166 | 2026-06-10 10:55:17.833260 | TASK [post/get-kubectl-logs : Set proper facts] 2026-06-10 10:55:17.873178 | controller | ok 2026-06-10 10:55:17.877966 | 2026-06-10 10:55:17.878050 | TASK [post/get-kubectl-logs : Get pod logs] 2026-06-10 10:55:25.417328 | controller | ok: Runtime: 0:00:07.144713 2026-06-10 10:55:25.425302 | 2026-06-10 10:55:25.425469 | LOOP [post/get-kubectl-logs : Describe namespace resources] 2026-06-10 10:55:28.892209 | controller | ok: Item: pods Runtime: 0:00:03.242365 2026-06-10 10:55:30.033163 | controller | ok: Item: deployments Runtime: 0:00:00.967070 2026-06-10 10:55:31.653495 | controller | ok: Item: statefulsets Runtime: 0:00:01.435054 2026-06-10 10:55:34.119321 | controller | ok: Item: services Runtime: 0:00:02.265785 2026-06-10 10:55:36.368479 | controller | ok: Item: secrets Runtime: 0:00:02.052520 2026-06-10 10:55:40.088182 | controller | ok: Item: configmaps Runtime: 0:00:03.512309 2026-06-10 10:55:41.068888 | controller | ok: Item: jobs Runtime: 0:00:00.793376 2026-06-10 10:55:42.926977 | controller | ok: Item: pvc Runtime: 0:00:01.658555 2026-06-10 10:55:43.808791 | controller | ok: Item: ingress Runtime: 0:00:00.668225 2026-06-10 10:55:43.832507 | 2026-06-10 10:55:43.832709 | TASK [post/get-kubectl-logs : Get events] 2026-06-10 10:55:45.381462 | controller | ok: Runtime: 0:00:01.148165 2026-06-10 10:55:45.389545 | 2026-06-10 10:55:45.389679 | TASK [post/get-kubectl-logs : Fetch decoded secrets] 2026-06-10 10:55:47.932409 | controller | ok: Runtime: 0:00:02.167175 2026-06-10 10:55:47.947915 | 2026-06-10 10:55:47.948073 | TASK [ansible.builtin.include_role : post/get-logserver-content] 2026-06-10 10:55:47.973600 | controller | ok 2026-06-10 10:55:48.007103 | 2026-06-10 10:55:48.007232 | TASK [post/get-logserver-content : Ensure removal of ~/zuul-output/logs/logserver-content] 2026-06-10 10:55:48.384679 | controller | ok 2026-06-10 10:55:48.392961 | 2026-06-10 10:55:48.393112 | TASK [post/get-logserver-content : Get logserver pod name] 2026-06-10 06:55:48.707688 | controller | logserver-0 2026-06-10 10:55:48.943923 | controller | ok: Runtime: 0:00:00.081429 2026-06-10 10:55:48.957217 | 2026-06-10 10:55:48.957400 | TASK [post/get-logserver-content : Fetch logserver content on locally in ~/zuul-output/logs/logserver-content] 2026-06-10 06:55:49.417546 | controller | tar: Removing leading `/' from member names 2026-06-10 10:55:49.515215 | controller | ok: Runtime: 0:00:00.229874 2026-06-10 10:55:49.530223 | 2026-06-10 10:55:49.530356 | TASK [ansible.builtin.include_role : post/get-repos] 2026-06-10 10:55:49.555286 | controller | ok 2026-06-10 10:55:49.589170 | 2026-06-10 10:55:49.589266 | TASK [post/get-repos : Set proper facts] 2026-06-10 10:55:49.620985 | controller | ok 2026-06-10 10:55:49.627032 | 2026-06-10 10:55:49.627119 | TASK [post/get-repos : Ensure ~/zuul-output/logs/repos/] 2026-06-10 10:55:49.849685 | controller | changed 2026-06-10 10:55:49.855600 | 2026-06-10 10:55:49.855684 | TASK [post/get-repos : Copy repositories] 2026-06-10 10:55:50.435327 | controller | changed 2026-06-10 10:55:50.452121 | 2026-06-10 10:55:50.452276 | TASK [ansible.builtin.include_role : post/get-loki-logs] 2026-06-10 10:55:50.478087 | controller | ok 2026-06-10 10:55:50.518239 | 2026-06-10 10:55:50.518357 | TASK [post/get-loki-logs : Ensure output dir exists for loki logs] 2026-06-10 10:55:50.742772 | controller | changed 2026-06-10 10:55:50.751006 | 2026-06-10 10:55:50.751132 | TASK [post/get-loki-logs : Set realpath for the log directory] 2026-06-10 06:55:50.996381 | controller | /home/zuul-worker/zuul-output/logs/loki-logs 2026-06-10 10:55:51.293278 | controller | ok: Runtime: 0:00:00.004104 2026-06-10 10:55:51.298479 | 2026-06-10 10:55:51.298598 | TASK [post/get-loki-logs : Start port forwarding to loki-http if needed] 2026-06-10 10:55:51.935593 | controller | ok: ERROR (ignored) 2026-06-10 10:55:51.936007 | controller | { 2026-06-10 10:55:51.936072 | controller | "ansible_job_id": "j835872325054.188963", 2026-06-10 10:55:51.936114 | controller | "finished": 0, 2026-06-10 10:55:51.936153 | controller | "results_file": "/home/zuul-worker/.ansible_async/j835872325054.188963", 2026-06-10 10:55:51.936193 | controller | "started": 1 2026-06-10 10:55:51.936227 | controller | } 2026-06-10 10:55:51.944288 | 2026-06-10 10:55:51.944413 | TASK [post/get-loki-logs : Ensure loki is reachable] 2026-06-10 10:55:52.645216 | controller | ok: OK (141 bytes) 2026-06-10 10:55:52.651587 | 2026-06-10 10:55:52.651670 | TASK [post/get-loki-logs : Get aggregated logs prefixed by service] 2026-06-10 10:55:54.695068 | controller | ok: Runtime: 0:00:01.511471 2026-06-10 10:55:54.708424 | 2026-06-10 10:55:54.708706 | TASK [post/get-loki-logs : Change owner and group for the log dir] 2026-06-10 10:55:55.261660 | controller | ok: Runtime: 0:00:00.005640 2026-06-10 10:55:55.271813 | 2026-06-10 10:55:55.271921 | PLAY RECAP 2026-06-10 10:55:55.272011 | controller | ok: 18 changed: 14 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-06-10 10:55:55.272054 | 2026-06-10 10:55:55.432171 | POST-RUN END RESULT_NORMAL: [untrusted : softwarefactory-project.io/software-factory/sf-operator/playbooks/get-minikube-logs.yaml@master] 2026-06-10 10:55:55.434765 | POST-RUN START: [trusted : gitlab.com/softwarefactory-project/centosinfra-prod/sf-config/playbooks/base/post.yaml@main] 2026-06-10 10:55:56.126173 | 2026-06-10 10:55:56.126293 | PLAY [all] 2026-06-10 10:55:56.136697 | 2026-06-10 10:55:56.136785 | TASK [include_role : fetch-output] 2026-06-10 10:55:56.187150 | controller | ok 2026-06-10 10:55:56.203963 | 2026-06-10 10:55:56.204078 | TASK [fetch-output : Set log path for multiple nodes] 2026-06-10 10:55:56.259550 | controller | skipping: Conditional result was False 2026-06-10 10:55:56.268314 | 2026-06-10 10:55:56.268484 | TASK [fetch-output : Set log path for single node] 2026-06-10 10:55:56.326759 | controller | ok 2026-06-10 10:55:56.334317 | 2026-06-10 10:55:56.334496 | LOOP [fetch-output : Ensure local output dirs] 2026-06-10 10:55:56.807632 | controller -> localhost | changed: "/var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/work/logs" 2026-06-10 10:55:57.058580 | controller -> localhost | changed: "/var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/work/artifacts" 2026-06-10 10:55:57.323650 | controller -> localhost | changed: "/var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/work/docs" 2026-06-10 10:55:57.341354 | 2026-06-10 10:55:57.341596 | LOOP [fetch-output : Collect logs, artifacts and docs] 2026-06-10 10:55:58.276051 | controller | changed: 2026-06-10 10:55:58.276372 | controller | .d..t...... ./ 2026-06-10 10:55:58.276520 | controller | >f+++++++++ dmesg.txt 2026-06-10 10:55:58.276578 | controller | >f+++++++++ minikube.logs 2026-06-10 10:55:58.276620 | controller | >f+++++++++ sf-operator-dry-run-lifecycle.log 2026-06-10 10:55:58.276663 | controller | >f+++++++++ sf-operator-dry-run-update.log 2026-06-10 10:55:58.276700 | controller | >f+++++++++ sf-operator.log 2026-06-10 10:55:58.276735 | controller | cd+++++++++ decoded-secrets/ 2026-06-10 10:55:58.276771 | controller | >f+++++++++ decoded-secrets/ingress-nginx--ingress-nginx-admission 2026-06-10 10:55:58.276808 | controller | >f+++++++++ decoded-secrets/kube-system--bootstrap-token-h6c8h9 2026-06-10 10:55:58.276843 | controller | >f+++++++++ decoded-secrets/nodepool--nodepool-token 2026-06-10 10:55:58.276878 | controller | >f+++++++++ decoded-secrets/sf--admin-ssh-key 2026-06-10 10:55:58.276913 | controller | >f+++++++++ decoded-secrets/sf--config-update-secrets 2026-06-10 10:55:58.276947 | controller | >f+++++++++ decoded-secrets/sf--dummygerritsecret 2026-06-10 10:55:58.276980 | controller | >f+++++++++ decoded-secrets/sf--es-basicauth 2026-06-10 10:55:58.277014 | controller | >f+++++++++ decoded-secrets/sf--gerrit-admin-api-key 2026-06-10 10:55:58.277053 | controller | >f+++++++++ decoded-secrets/sf--logserver-keys 2026-06-10 10:55:58.277089 | controller | >f+++++++++ decoded-secrets/sf--logserver-uploader-keys 2026-06-10 10:55:58.277123 | controller | >f+++++++++ decoded-secrets/sf--logserver-uploader-spare-keys 2026-06-10 10:55:58.277157 | controller | >f+++++++++ decoded-secrets/sf--mariadb-config-secrets 2026-06-10 10:55:58.277190 | controller | >f+++++++++ decoded-secrets/sf--mariadb-initdb-secrets 2026-06-10 10:55:58.277223 | controller | >f+++++++++ decoded-secrets/sf--mariadb-root-password 2026-06-10 10:55:58.277256 | controller | >f+++++++++ decoded-secrets/sf--nodepool-builder-ssh-key 2026-06-10 10:55:58.277289 | controller | >f+++++++++ decoded-secrets/sf--nodepool-providers-secrets 2026-06-10 10:55:58.277322 | controller | >f+++++++++ decoded-secrets/sf--self-signed-cert 2026-06-10 10:55:58.277358 | controller | >f+++++++++ decoded-secrets/sf--smtp-secret 2026-06-10 10:55:58.277392 | controller | >f+++++++++ decoded-secrets/sf--zookeeper-client-tls 2026-06-10 10:55:58.277457 | controller | >f+++++++++ decoded-secrets/sf--zookeeper-server-tls 2026-06-10 10:55:58.277505 | controller | >f+++++++++ decoded-secrets/sf--zuul-auth-secret 2026-06-10 10:55:58.277542 | controller | >f+++++++++ decoded-secrets/sf--zuul-config 2026-06-10 10:55:58.277576 | controller | >f+++++++++ decoded-secrets/sf--zuul-db-connection 2026-06-10 10:55:58.277609 | controller | >f+++++++++ decoded-secrets/sf--zuul-gerrit-api-key 2026-06-10 10:55:58.277642 | controller | >f+++++++++ decoded-secrets/sf--zuul-keystore-password 2026-06-10 10:55:58.277676 | controller | >f+++++++++ decoded-secrets/sf--zuul-spare-ssh-key 2026-06-10 10:55:58.277708 | controller | >f+++++++++ decoded-secrets/sf--zuul-ssh-key 2026-06-10 10:55:58.277741 | controller | cd+++++++++ events/ 2026-06-10 10:55:58.277785 | controller | >f+++++++++ events/default.txt 2026-06-10 10:55:58.277819 | controller | >f+++++++++ events/ingress-nginx.txt 2026-06-10 10:55:58.277853 | controller | >f+++++++++ events/kube-node-lease.txt 2026-06-10 10:55:58.277894 | controller | >f+++++++++ events/kube-public.txt 2026-06-10 10:55:58.277931 | controller | >f+++++++++ events/kube-system.txt 2026-06-10 10:55:58.277966 | controller | >f+++++++++ events/nodepool.txt 2026-06-10 10:55:58.278001 | controller | >f+++++++++ events/sf.txt 2026-06-10 10:55:58.278035 | controller | cd+++++++++ items-describe/ 2026-06-10 10:55:58.278069 | controller | >f+++++++++ items-describe/default--configmaps--kube-root-ca.crt.txt 2026-06-10 10:55:58.278103 | controller | >f+++++++++ items-describe/default--services--kubernetes.txt 2026-06-10 10:55:58.278138 | controller | >f+++++++++ items-describe/ingress-nginx--configmaps--ingress-nginx-controller.txt 2026-06-10 10:55:58.278177 | controller | >f+++++++++ items-describe/ingress-nginx--configmaps--kube-root-ca.crt.txt 2026-06-10 10:55:58.278216 | controller | >f+++++++++ items-describe/ingress-nginx--configmaps--tcp-services.txt 2026-06-10 10:55:58.278253 | controller | >f+++++++++ items-describe/ingress-nginx--configmaps--udp-services.txt 2026-06-10 10:55:58.278288 | controller | >f+++++++++ items-describe/ingress-nginx--deployments--ingress-nginx-controller.txt 2026-06-10 10:55:58.278322 | controller | >f+++++++++ items-describe/ingress-nginx--jobs--ingress-nginx-admission-create.txt 2026-06-10 10:55:58.278356 | controller | >f+++++++++ items-describe/ingress-nginx--jobs--ingress-nginx-admission-patch.txt 2026-06-10 10:55:58.278389 | controller | >f+++++++++ items-describe/ingress-nginx--pods--ingress-nginx-admission-create-8wpgz.txt 2026-06-10 10:55:58.278423 | controller | >f+++++++++ items-describe/ingress-nginx--pods--ingress-nginx-admission-patch-k2gl7.txt 2026-06-10 10:55:58.278509 | controller | >f+++++++++ items-describe/ingress-nginx--pods--ingress-nginx-controller-86649c845b-hdbbv.txt 2026-06-10 10:55:58.278546 | controller | >f+++++++++ items-describe/ingress-nginx--secrets--ingress-nginx-admission.txt 2026-06-10 10:55:58.278580 | controller | >f+++++++++ items-describe/ingress-nginx--services--ingress-nginx-controller-admission.txt 2026-06-10 10:55:58.278614 | controller | >f+++++++++ items-describe/ingress-nginx--services--ingress-nginx-controller.txt 2026-06-10 10:55:58.278647 | controller | >f+++++++++ items-describe/kube-node-lease--configmaps--kube-root-ca.crt.txt 2026-06-10 10:55:58.278681 | controller | >f+++++++++ items-describe/kube-public--configmaps--cluster-info.txt 2026-06-10 10:55:58.278714 | controller | >f+++++++++ items-describe/kube-public--configmaps--kube-root-ca.crt.txt 2026-06-10 10:55:58.278748 | controller | >f+++++++++ items-describe/kube-system--configmaps--coredns.txt 2026-06-10 10:55:58.278781 | controller | >f+++++++++ items-describe/kube-system--configmaps--extension-apiserver-authentication.txt 2026-06-10 10:55:58.278815 | controller | >f+++++++++ items-describe/kube-system--configmaps--kube-apiserver-legacy-service-account-token-tracking.txt 2026-06-10 10:55:58.278847 | controller | >f+++++++++ items-describe/kube-system--configmaps--kube-proxy.txt 2026-06-10 10:55:58.278880 | controller | >f+++++++++ items-describe/kube-system--configmaps--kube-root-ca.crt.txt 2026-06-10 10:55:58.278915 | controller | >f+++++++++ items-describe/kube-system--configmaps--kubeadm-config.txt 2026-06-10 10:55:58.278949 | controller | >f+++++++++ items-describe/kube-system--configmaps--kubelet-config.txt 2026-06-10 10:55:58.278983 | controller | >f+++++++++ items-describe/kube-system--deployments--coredns.txt 2026-06-10 10:55:58.279016 | controller | >f+++++++++ items-describe/kube-system--pods--coredns-668d6bf9bc-sh7x9.txt 2026-06-10 10:55:58.279049 | controller | >f+++++++++ items-describe/kube-system--pods--etcd-minikube.txt 2026-06-10 10:55:58.279083 | controller | >f+++++++++ items-describe/kube-system--pods--kindnet-rb2lw.txt 2026-06-10 10:55:58.279118 | controller | >f+++++++++ items-describe/kube-system--pods--kube-apiserver-minikube.txt 2026-06-10 10:55:58.279160 | controller | >f+++++++++ items-describe/kube-system--pods--kube-controller-manager-minikube.txt 2026-06-10 10:55:58.279196 | controller | >f+++++++++ items-describe/kube-system--pods--kube-ingress-dns-minikube.txt 2026-06-10 10:55:58.279230 | controller | >f+++++++++ items-describe/kube-system--pods--kube-proxy-z8pmr.txt 2026-06-10 10:55:58.279263 | controller | >f+++++++++ items-describe/kube-system--pods--kube-scheduler-minikube.txt 2026-06-10 10:55:58.279296 | controller | >f+++++++++ items-describe/kube-system--pods--storage-provisioner.txt 2026-06-10 10:55:58.279329 | controller | >f+++++++++ items-describe/kube-system--secrets--bootstrap-token-h6c8h9.txt 2026-06-10 10:55:58.279362 | controller | >f+++++++++ items-describe/kube-system--services--kube-dns.txt 2026-06-10 10:55:58.279395 | controller | >f+++++++++ items-describe/nodepool--configmaps--kube-root-ca.crt.txt 2026-06-10 10:55:58.279456 | controller | >f+++++++++ items-describe/nodepool--secrets--nodepool-token.txt 2026-06-10 10:55:58.279502 | controller | >f+++++++++ items-describe/sf--configmaps--gateway-config-map.txt 2026-06-10 10:55:58.279538 | controller | >f+++++++++ items-describe/sf--configmaps--git-server-pi-config-map.txt 2026-06-10 10:55:58.279571 | controller | >f+++++++++ items-describe/sf--configmaps--kube-root-ca.crt.txt 2026-06-10 10:55:58.279606 | controller | >f+++++++++ items-describe/sf--configmaps--logserver-config-map.txt 2026-06-10 10:55:58.279640 | controller | >f+++++++++ items-describe/sf--configmaps--loki-config.txt 2026-06-10 10:55:58.279673 | controller | >f+++++++++ items-describe/sf--configmaps--managesf-resources-config-map.txt 2026-06-10 10:55:58.279707 | controller | >f+++++++++ items-describe/sf--configmaps--managesf-resources-tooling-config-map.txt 2026-06-10 10:55:58.279740 | controller | >f+++++++++ items-describe/sf--configmaps--nodepool-builder-extra-config-config-map.txt 2026-06-10 10:55:58.279773 | controller | >f+++++++++ items-describe/sf--configmaps--nodepool-launcher-extra-config-config-map.txt 2026-06-10 10:55:58.279806 | controller | >f+++++++++ items-describe/sf--configmaps--nodepool-tooling-config-map.txt 2026-06-10 10:55:58.279838 | controller | >f+++++++++ items-describe/sf--configmaps--np-statsd-config-map.txt 2026-06-10 10:55:58.279871 | controller | >f+++++++++ items-describe/sf--configmaps--pymod-sf-operator-config-map.txt 2026-06-10 10:55:58.279905 | controller | >f+++++++++ items-describe/sf--configmaps--sf-standalone-owner.txt 2026-06-10 10:55:58.279938 | controller | >f+++++++++ items-describe/sf--configmaps--test-fluentbit-config.txt 2026-06-10 10:55:58.279971 | controller | >f+++++++++ items-describe/sf--configmaps--zookeeper-pi-config-map.txt 2026-06-10 10:55:58.280004 | controller | >f+++++++++ items-describe/sf--configmaps--zs-internal-tenant-reconfigure.txt 2026-06-10 10:55:58.280037 | controller | >f+++++++++ items-describe/sf--configmaps--zuul-extra-config-map.txt 2026-06-10 10:55:58.280069 | controller | >f+++++++++ items-describe/sf--configmaps--zuul-logging-config-map.txt 2026-06-10 10:55:58.280102 | controller | >f+++++++++ items-describe/sf--configmaps--zuul-scheduler-tooling-config-map.txt 2026-06-10 10:55:58.280135 | controller | >f+++++++++ items-describe/sf--configmaps--zuul-statsd-config-map.txt 2026-06-10 10:55:58.280167 | controller | >f+++++++++ items-describe/sf--deployments--gateway.txt 2026-06-10 10:55:58.280199 | controller | >f+++++++++ items-describe/sf--deployments--logjuicer.txt 2026-06-10 10:55:58.280231 | controller | >f+++++++++ items-describe/sf--deployments--nodepool-launcher.txt 2026-06-10 10:55:58.280264 | controller | >f+++++++++ items-describe/sf--deployments--test-loki.txt 2026-06-10 10:55:58.280297 | controller | >f+++++++++ items-describe/sf--deployments--zuul-web.txt 2026-06-10 10:55:58.280337 | controller | >f+++++++++ items-describe/sf--deployments--zuul-weeder.txt 2026-06-10 10:55:58.280389 | controller | >f+++++++++ items-describe/sf--ingress--gerrit-ingress.txt 2026-06-10 10:55:58.280469 | controller | >f+++++++++ items-describe/sf--ingress--sf-ingress.txt 2026-06-10 10:55:58.280561 | controller | >f+++++++++ items-describe/sf--jobs--config-base-secret-1-8399-ns.txt 2026-06-10 10:55:58.280604 | controller | >f+++++++++ items-describe/sf--jobs--post-init.txt 2026-06-10 10:55:58.280640 | controller | >f+++++++++ items-describe/sf--jobs--zuul-db-provision.txt 2026-06-10 10:55:58.280674 | controller | >f+++++++++ items-describe/sf--pods--config-base-secret-1-8399-ns-thwcn.txt 2026-06-10 10:55:58.280708 | controller | >f+++++++++ items-describe/sf--pods--config-base-secret-1-859-ns-lbbxd.txt 2026-06-10 10:55:58.280741 | controller | >f+++++++++ items-describe/sf--pods--gateway-6bbbd758d9-272t9.txt 2026-06-10 10:55:58.280775 | controller | >f+++++++++ items-describe/sf--pods--gerrit-0.txt 2026-06-10 10:55:58.280808 | controller | >f+++++++++ items-describe/sf--pods--git-server-0.txt 2026-06-10 10:55:58.280840 | controller | >f+++++++++ items-describe/sf--pods--hound-search-0.txt 2026-06-10 10:55:58.280877 | controller | >f+++++++++ items-describe/sf--pods--logjuicer-6ff84764b7-mxhhl.txt 2026-06-10 10:55:58.280927 | controller | >f+++++++++ items-describe/sf--pods--logserver-0.txt 2026-06-10 10:55:58.280972 | controller | >f+++++++++ items-describe/sf--pods--mariadb-0.txt 2026-06-10 10:55:58.281007 | controller | >f+++++++++ items-describe/sf--pods--nodepool-builder-0.txt 2026-06-10 10:55:58.281041 | controller | >f+++++++++ items-describe/sf--pods--nodepool-launcher-5bd775d78f-gwtzg.txt 2026-06-10 10:55:58.281075 | controller | >f+++++++++ items-describe/sf--pods--post-init-7f798.txt 2026-06-10 10:55:58.281110 | controller | >f+++++++++ items-describe/sf--pods--test-fluentbit.txt 2026-06-10 10:55:58.281144 | controller | >f+++++++++ items-describe/sf--pods--test-loki-8cd4fd56-g9s64.txt 2026-06-10 10:55:58.281177 | controller | >f+++++++++ items-describe/sf--pods--zookeeper-0.txt 2026-06-10 10:55:58.281212 | controller | >f+++++++++ items-describe/sf--pods--zuul-db-provision-r5bxv.txt 2026-06-10 10:55:58.281247 | controller | >f+++++++++ items-describe/sf--pods--zuul-db-provision-xxzjn.txt 2026-06-10 10:55:58.281279 | controller | >f+++++++++ items-describe/sf--pods--zuul-executor-0.txt 2026-06-10 10:55:58.281312 | controller | >f+++++++++ items-describe/sf--pods--zuul-merger-0.txt 2026-06-10 10:55:58.281344 | controller | >f+++++++++ items-describe/sf--pods--zuul-scheduler-0.txt 2026-06-10 10:55:58.281377 | controller | >f+++++++++ items-describe/sf--pods--zuul-web-5458bdf4f-jpj6x.txt 2026-06-10 10:55:58.281409 | controller | >f+++++++++ items-describe/sf--pods--zuul-weeder-57c6567f5f-sc5q2.txt 2026-06-10 10:55:58.281484 | controller | >f+++++++++ items-describe/sf--pvc--gerrit-gerrit-0.txt 2026-06-10 10:55:58.281521 | controller | >f+++++++++ items-describe/sf--pvc--git-server-git-server-0.txt 2026-06-10 10:55:58.281555 | controller | >f+++++++++ items-describe/sf--pvc--hound-search-data-hound-search-0.txt 2026-06-10 10:55:58.281588 | controller | >f+++++++++ items-describe/sf--pvc--logjuicer-pvc.txt 2026-06-10 10:55:58.281621 | controller | >f+++++++++ items-describe/sf--pvc--logserver-logserver-0.txt 2026-06-10 10:55:58.281654 | controller | >f+++++++++ items-describe/sf--pvc--mariadb-logs-mariadb-0.txt 2026-06-10 10:55:58.281686 | controller | >f+++++++++ items-describe/sf--pvc--mariadb-mariadb-0.txt 2026-06-10 10:55:58.281719 | controller | >f+++++++++ items-describe/sf--pvc--nodepool-builder-nodepool-builder-0.txt 2026-06-10 10:55:58.281751 | controller | >f+++++++++ items-describe/sf--pvc--zookeeper-data-zookeeper-0.txt 2026-06-10 10:55:58.281783 | controller | >f+++++++++ items-describe/sf--pvc--zookeeper-logs-zookeeper-0.txt 2026-06-10 10:55:58.281817 | controller | >f+++++++++ items-describe/sf--pvc--zuul-executor-zuul-executor-0.txt 2026-06-10 10:55:58.281849 | controller | >f+++++++++ items-describe/sf--pvc--zuul-merger-zuul-merger-0.txt 2026-06-10 10:55:58.281882 | controller | >f+++++++++ items-describe/sf--pvc--zuul-scheduler-zuul-scheduler-0.txt 2026-06-10 10:55:58.281930 | controller | >f+++++++++ items-describe/sf--secrets--admin-ssh-key.txt 2026-06-10 10:55:58.281964 | controller | >f+++++++++ items-describe/sf--secrets--config-update-secrets.txt 2026-06-10 10:55:58.281997 | controller | >f+++++++++ items-describe/sf--secrets--dummygerritsecret.txt 2026-06-10 10:55:58.282031 | controller | >f+++++++++ items-describe/sf--secrets--es-basicauth.txt 2026-06-10 10:55:58.282063 | controller | >f+++++++++ items-describe/sf--secrets--gerrit-admin-api-key.txt 2026-06-10 10:55:58.282095 | controller | >f+++++++++ items-describe/sf--secrets--logserver-keys.txt 2026-06-10 10:55:58.282127 | controller | >f+++++++++ items-describe/sf--secrets--logserver-uploader-keys.txt 2026-06-10 10:55:58.282159 | controller | >f+++++++++ items-describe/sf--secrets--logserver-uploader-spare-keys.txt 2026-06-10 10:55:58.282192 | controller | >f+++++++++ items-describe/sf--secrets--mariadb-config-secrets.txt 2026-06-10 10:55:58.282226 | controller | >f+++++++++ items-describe/sf--secrets--mariadb-initdb-secrets.txt 2026-06-10 10:55:58.282260 | controller | >f+++++++++ items-describe/sf--secrets--mariadb-root-password.txt 2026-06-10 10:55:58.282293 | controller | >f+++++++++ items-describe/sf--secrets--nodepool-builder-ssh-key.txt 2026-06-10 10:55:58.282325 | controller | >f+++++++++ items-describe/sf--secrets--nodepool-providers-secrets.txt 2026-06-10 10:55:58.282358 | controller | >f+++++++++ items-describe/sf--secrets--self-signed-cert.txt 2026-06-10 10:55:58.282391 | controller | >f+++++++++ items-describe/sf--secrets--smtp-secret.txt 2026-06-10 10:55:58.282423 | controller | >f+++++++++ items-describe/sf--secrets--zookeeper-client-tls.txt 2026-06-10 10:55:58.282514 | controller | >f+++++++++ items-describe/sf--secrets--zookeeper-server-tls.txt 2026-06-10 10:55:58.282550 | controller | >f+++++++++ items-describe/sf--secrets--zuul-auth-secret.txt 2026-06-10 10:55:58.282583 | controller | >f+++++++++ items-describe/sf--secrets--zuul-config.txt 2026-06-10 10:55:58.282616 | controller | >f+++++++++ items-describe/sf--secrets--zuul-db-connection.txt 2026-06-10 10:55:58.282648 | controller | >f+++++++++ items-describe/sf--secrets--zuul-gerrit-api-key.txt 2026-06-10 10:55:58.282680 | controller | >f+++++++++ items-describe/sf--secrets--zuul-keystore-password.txt 2026-06-10 10:55:58.282712 | controller | >f+++++++++ items-describe/sf--secrets--zuul-spare-ssh-key.txt 2026-06-10 10:55:58.282744 | controller | >f+++++++++ items-describe/sf--secrets--zuul-ssh-key.txt 2026-06-10 10:55:58.282789 | controller | >f+++++++++ items-describe/sf--services--fluent-bit-forward-input.txt 2026-06-10 10:55:58.282833 | controller | >f+++++++++ items-describe/sf--services--gateway.txt 2026-06-10 10:55:58.282874 | controller | >f+++++++++ items-describe/sf--services--gerrit-httpd.txt 2026-06-10 10:55:58.282916 | controller | >f+++++++++ items-describe/sf--services--gerrit-sshd.txt 2026-06-10 10:55:58.282958 | controller | >f+++++++++ items-describe/sf--services--git-server-rw.txt 2026-06-10 10:55:58.282998 | controller | >f+++++++++ items-describe/sf--services--git-server.txt 2026-06-10 10:55:58.283039 | controller | >f+++++++++ items-describe/sf--services--hound-search.txt 2026-06-10 10:55:58.283079 | controller | >f+++++++++ items-describe/sf--services--logjuicer.txt 2026-06-10 10:55:58.283120 | controller | >f+++++++++ items-describe/sf--services--logserver.txt 2026-06-10 10:55:58.283161 | controller | >f+++++++++ items-describe/sf--services--loki-http.txt 2026-06-10 10:55:58.283201 | controller | >f+++++++++ items-describe/sf--services--mariadb.txt 2026-06-10 10:55:58.283241 | controller | >f+++++++++ items-describe/sf--services--nodepool-builder.txt 2026-06-10 10:55:58.283282 | controller | >f+++++++++ items-describe/sf--services--nodepool-launcher.txt 2026-06-10 10:55:58.283322 | controller | >f+++++++++ items-describe/sf--services--zookeeper-headless.txt 2026-06-10 10:55:58.283362 | controller | >f+++++++++ items-describe/sf--services--zookeeper.txt 2026-06-10 10:55:58.283402 | controller | >f+++++++++ items-describe/sf--services--zuul-capacity.txt 2026-06-10 10:55:58.283507 | controller | >f+++++++++ items-describe/sf--services--zuul-executor-headless.txt 2026-06-10 10:55:58.283556 | controller | >f+++++++++ items-describe/sf--services--zuul-web.txt 2026-06-10 10:55:58.283598 | controller | >f+++++++++ items-describe/sf--services--zuul-weeder.txt 2026-06-10 10:55:58.283641 | controller | >f+++++++++ items-describe/sf--statefulsets--gerrit.txt 2026-06-10 10:55:58.283678 | controller | >f+++++++++ items-describe/sf--statefulsets--git-server.txt 2026-06-10 10:55:58.283720 | controller | >f+++++++++ items-describe/sf--statefulsets--hound-search.txt 2026-06-10 10:55:58.283761 | controller | >f+++++++++ items-describe/sf--statefulsets--logserver.txt 2026-06-10 10:55:58.283802 | controller | >f+++++++++ items-describe/sf--statefulsets--mariadb.txt 2026-06-10 10:55:58.283843 | controller | >f+++++++++ items-describe/sf--statefulsets--nodepool-builder.txt 2026-06-10 10:55:58.283884 | controller | >f+++++++++ items-describe/sf--statefulsets--zookeeper.txt 2026-06-10 10:55:58.283925 | controller | >f+++++++++ items-describe/sf--statefulsets--zuul-executor.txt 2026-06-10 10:55:58.283966 | controller | >f+++++++++ items-describe/sf--statefulsets--zuul-merger.txt 2026-06-10 10:55:58.284007 | controller | >f+++++++++ items-describe/sf--statefulsets--zuul-scheduler.txt 2026-06-10 10:55:58.284048 | controller | cd+++++++++ k8s-resources/ 2026-06-10 10:55:58.284089 | controller | cd+++++++++ k8s-resources/sf/ 2026-06-10 10:55:58.284131 | controller | cd+++++++++ k8s-resources/sf/pod/ 2026-06-10 10:55:58.284173 | controller | >f+++++++++ k8s-resources/sf/pod/zuul-executor-0-zuul-executor-logs0.txt 2026-06-10 10:55:58.284215 | controller | >f+++++++++ k8s-resources/sf/pod/zuul-merger-0-zuul-merger-logs0.txt 2026-06-10 10:55:58.284257 | controller | >f+++++++++ k8s-resources/sf/pod/zuul-scheduler-0-zuul-scheduler-logs0.txt 2026-06-10 10:55:58.284298 | controller | >f+++++++++ k8s-resources/sf/pod/zuul-web-0-zuul-web-logs0.txt 2026-06-10 10:55:58.284339 | controller | cd+++++++++ logserver-content/ 2026-06-10 10:55:58.284380 | controller | cd+++++++++ logserver-content/004/ 2026-06-10 10:55:58.284421 | controller | cd+++++++++ logserver-content/004/internal/ 2026-06-10 10:55:58.284520 | controller | cd+++++++++ logserver-content/004/internal/0047fbb3358844ed92968ba33bfe32d0/ 2026-06-10 10:55:58.284566 | controller | >f+++++++++ logserver-content/004/internal/0047fbb3358844ed92968ba33bfe32d0/job-output.json.gz 2026-06-10 10:55:58.284609 | controller | >f+++++++++ logserver-content/004/internal/0047fbb3358844ed92968ba33bfe32d0/job-output.txt.gz 2026-06-10 10:55:58.284650 | controller | >f+++++++++ logserver-content/004/internal/0047fbb3358844ed92968ba33bfe32d0/workspace-repos.json 2026-06-10 10:55:58.284692 | controller | >f+++++++++ logserver-content/004/internal/0047fbb3358844ed92968ba33bfe32d0/zuul-manifest.json 2026-06-10 10:55:58.284732 | controller | cd+++++++++ logserver-content/004/internal/0047fbb3358844ed92968ba33bfe32d0/zuul-info/ 2026-06-10 10:55:58.284773 | controller | >f+++++++++ logserver-content/004/internal/0047fbb3358844ed92968ba33bfe32d0/zuul-info/inventory.yaml 2026-06-10 10:55:58.284814 | controller | cd+++++++++ logserver-content/c60/ 2026-06-10 10:55:58.284854 | controller | cd+++++++++ logserver-content/c60/internal/ 2026-06-10 10:55:58.284895 | controller | cd+++++++++ logserver-content/c60/internal/c60e1e91239b4339a8f4d9789f6727c4/ 2026-06-10 10:55:58.284935 | controller | >f+++++++++ logserver-content/c60/internal/c60e1e91239b4339a8f4d9789f6727c4/job-output.json.gz 2026-06-10 10:55:58.284976 | controller | >f+++++++++ logserver-content/c60/internal/c60e1e91239b4339a8f4d9789f6727c4/job-output.txt.gz 2026-06-10 10:55:58.285016 | controller | >f+++++++++ logserver-content/c60/internal/c60e1e91239b4339a8f4d9789f6727c4/workspace-repos.json 2026-06-10 10:55:58.285056 | controller | >f+++++++++ logserver-content/c60/internal/c60e1e91239b4339a8f4d9789f6727c4/zuul-manifest.json 2026-06-10 10:55:58.285096 | controller | cd+++++++++ logserver-content/c60/internal/c60e1e91239b4339a8f4d9789f6727c4/zuul-info/ 2026-06-10 10:55:58.285193 | controller | >f+++++++++ logserver-content/c60/internal/c60e1e91239b4339a8f4d9789f6727c4/zuul-info/inventory.yaml 2026-06-10 10:55:58.285236 | controller | cd+++++++++ logserver-content/ed0/ 2026-06-10 10:55:58.285270 | controller | cd+++++++++ logserver-content/ed0/internal/ 2026-06-10 10:55:58.285303 | controller | cd+++++++++ logserver-content/ed0/internal/ed0c9180383a46bf9c8981cd7070b25f/ 2026-06-10 10:55:58.285336 | controller | >f+++++++++ logserver-content/ed0/internal/ed0c9180383a46bf9c8981cd7070b25f/job-output.json.gz 2026-06-10 10:55:58.285367 | controller | >f+++++++++ logserver-content/ed0/internal/ed0c9180383a46bf9c8981cd7070b25f/job-output.txt.gz 2026-06-10 10:55:58.285399 | controller | >f+++++++++ logserver-content/ed0/internal/ed0c9180383a46bf9c8981cd7070b25f/workspace-repos.json 2026-06-10 10:55:58.285477 | controller | >f+++++++++ logserver-content/ed0/internal/ed0c9180383a46bf9c8981cd7070b25f/zuul-manifest.json 2026-06-10 10:55:58.285521 | controller | cd+++++++++ logserver-content/ed0/internal/ed0c9180383a46bf9c8981cd7070b25f/zuul-info/ 2026-06-10 10:55:58.285556 | controller | >f+++++++++ logserver-content/ed0/internal/ed0c9180383a46bf9c8981cd7070b25f/zuul-info/inventory.yaml 2026-06-10 10:55:58.285594 | controller | cd+++++++++ loki-logs/ 2026-06-10 10:55:58.285629 | controller | >f+++++++++ loki-logs/all.log 2026-06-10 10:55:58.285662 | controller | cd+++++++++ pods-logs/ 2026-06-10 10:55:58.285695 | controller | >f+++++++++ pods-logs/ingress-nginx--ingress-nginx-admission-create-8wpgz--create.txt 2026-06-10 10:55:58.285728 | controller | >f+++++++++ pods-logs/ingress-nginx--ingress-nginx-admission-patch-k2gl7--patch.txt 2026-06-10 10:55:58.285761 | controller | >f+++++++++ pods-logs/ingress-nginx--ingress-nginx-controller-86649c845b-hdbbv--controller.txt 2026-06-10 10:55:58.285794 | controller | >f+++++++++ pods-logs/kube-system--coredns-668d6bf9bc-sh7x9--coredns.txt 2026-06-10 10:55:58.285828 | controller | >f+++++++++ pods-logs/kube-system--etcd-minikube--etcd.txt 2026-06-10 10:55:58.285862 | controller | >f+++++++++ pods-logs/kube-system--kindnet-rb2lw--kindnet-cni.txt 2026-06-10 10:55:58.285896 | controller | >f+++++++++ pods-logs/kube-system--kube-apiserver-minikube--kube-apiserver.txt 2026-06-10 10:55:58.285929 | controller | >f+++++++++ pods-logs/kube-system--kube-controller-manager-minikube--kube-controller-manager.txt 2026-06-10 10:55:58.285961 | controller | >f+++++++++ pods-logs/kube-system--kube-ingress-dns-minikube--minikube-ingress-dns.txt 2026-06-10 10:55:58.285993 | controller | >f+++++++++ pods-logs/kube-system--kube-proxy-z8pmr--kube-proxy.txt 2026-06-10 10:55:58.286026 | controller | >f+++++++++ pods-logs/kube-system--kube-scheduler-minikube--kube-scheduler.txt 2026-06-10 10:55:58.286058 | controller | >f+++++++++ pods-logs/kube-system--storage-provisioner--storage-provisioner.txt 2026-06-10 10:55:58.286091 | controller | >f+++++++++ pods-logs/sf--config-base-secret-1-8399-ns-thwcn--sf-operator.txt 2026-06-10 10:55:58.286125 | controller | >f+++++++++ pods-logs/sf--config-base-secret-1-859-ns-lbbxd--sf-operator.txt 2026-06-10 10:55:58.286158 | controller | >f+++++++++ pods-logs/sf--gateway-6bbbd758d9-272t9--gateway.txt 2026-06-10 10:55:58.286191 | controller | >f+++++++++ pods-logs/sf--gerrit-0--gerrit.txt 2026-06-10 10:55:58.286223 | controller | >f+++++++++ pods-logs/sf--gerrit-0--managesf-resources.txt 2026-06-10 10:55:58.286256 | controller | >f+++++++++ pods-logs/sf--git-server-0--git-server-rw.txt 2026-06-10 10:55:58.286289 | controller | >f+++++++++ pods-logs/sf--git-server-0--git-server.txt 2026-06-10 10:55:58.286321 | controller | >f+++++++++ pods-logs/sf--hound-search-0--hound-search.txt 2026-06-10 10:55:58.286354 | controller | >f+++++++++ pods-logs/sf--logjuicer-6ff84764b7-mxhhl--logjuicer.txt 2026-06-10 10:55:58.286387 | controller | >f+++++++++ pods-logs/sf--logserver-0--logserver-nodeexporter.txt 2026-06-10 10:55:58.286420 | controller | >f+++++++++ pods-logs/sf--logserver-0--logserver-sshd.txt 2026-06-10 10:55:58.286503 | controller | >f+++++++++ pods-logs/sf--logserver-0--logserver.txt 2026-06-10 10:55:58.286540 | controller | >f+++++++++ pods-logs/sf--logserver-0--purgelogs.txt 2026-06-10 10:55:58.286574 | controller | >f+++++++++ pods-logs/sf--mariadb-0--mariadb-nodeexporter.txt 2026-06-10 10:55:58.286608 | controller | >f+++++++++ pods-logs/sf--mariadb-0--mariadb.txt 2026-06-10 10:55:58.286641 | controller | >f+++++++++ pods-logs/sf--nodepool-builder-0--build-logs-httpd.txt 2026-06-10 10:55:58.286674 | controller | >f+++++++++ pods-logs/sf--nodepool-builder-0--nodepool-builder-nodeexporter.txt 2026-06-10 10:55:58.286707 | controller | >f+++++++++ pods-logs/sf--nodepool-builder-0--nodepool-builder.txt 2026-06-10 10:55:58.286739 | controller | >f+++++++++ pods-logs/sf--nodepool-builder-0--np-statsd.txt 2026-06-10 10:55:58.286772 | controller | >f+++++++++ pods-logs/sf--nodepool-launcher-5bd775d78f-gwtzg--launcher.txt 2026-06-10 10:55:58.286804 | controller | >f+++++++++ pods-logs/sf--nodepool-launcher-5bd775d78f-gwtzg--np-statsd.txt 2026-06-10 10:55:58.286837 | controller | >f+++++++++ pods-logs/sf--nodepool-launcher-5bd775d78f-gwtzg--zuul-capacity.txt 2026-06-10 10:55:58.286870 | controller | >f+++++++++ pods-logs/sf--post-init-7f798--post-init-container.txt 2026-06-10 10:55:58.286904 | controller | >f+++++++++ pods-logs/sf--test-fluentbit--test-fluentbit.txt 2026-06-10 10:55:58.286937 | controller | >f+++++++++ pods-logs/sf--test-loki-8cd4fd56-g9s64--loki.txt 2026-06-10 10:55:58.286970 | controller | >f+++++++++ pods-logs/sf--zookeeper-0--zookeeper-nodeexporter.txt 2026-06-10 10:55:58.287003 | controller | >f+++++++++ pods-logs/sf--zookeeper-0--zookeeper.txt 2026-06-10 10:55:58.287035 | controller | >f+++++++++ pods-logs/sf--zuul-db-provision-r5bxv--mariadb-client.txt 2026-06-10 10:55:58.287068 | controller | >f+++++++++ pods-logs/sf--zuul-db-provision-xxzjn--mariadb-client.txt 2026-06-10 10:55:58.287100 | controller | >f+++++++++ pods-logs/sf--zuul-executor-0--zuul-executor-nodeexporter.txt 2026-06-10 10:55:58.287133 | controller | >f+++++++++ pods-logs/sf--zuul-executor-0--zuul-executor.txt 2026-06-10 10:55:58.287165 | controller | >f+++++++++ pods-logs/sf--zuul-merger-0--zuul-merger-nodeexporter.txt 2026-06-10 10:55:58.287197 | controller | >f+++++++++ pods-logs/sf--zuul-merger-0--zuul-merger.txt 2026-06-10 10:55:58.287229 | controller | >f+++++++++ pods-logs/sf--zuul-scheduler-0--zuul-scheduler-nodeexporter.txt 2026-06-10 10:55:58.287261 | controller | >f+++++++++ pods-logs/sf--zuul-scheduler-0--zuul-scheduler.txt 2026-06-10 10:55:58.287294 | controller | >f+++++++++ pods-logs/sf--zuul-scheduler-0--zuul-statsd.txt 2026-06-10 10:55:58.287326 | controller | >f+++++++++ pods-logs/sf--zuul-web-5458bdf4f-jpj6x--zuul-web.txt 2026-06-10 10:55:58.287358 | controller | >f+++++++++ pods-logs/sf--zuul-weeder-57c6567f5f-sc5q2--zuul-weeder.txt 2026-06-10 10:55:58.287390 | controller | cd+++++++++ repos/ 2026-06-10 10:55:58.287423 | controller | >f+++++++++ repos/test-ssh-key 2026-06-10 10:55:58.287501 | controller | >f+++++++++ repos/test-ssh-key.pub 2026-06-10 10:55:58.287537 | controller | >f+++++++++ repos/zuul-ssh-key 2026-06-10 10:55:58.287572 | controller | cd+++++++++ repos/config/ 2026-06-10 10:55:58.287606 | controller | >f+++++++++ repos/config/.gitreview 2026-06-10 10:55:58.287640 | controller | cd+++++++++ repos/config/.git/ 2026-06-10 10:55:58.287673 | controller | >f+++++++++ repos/config/.git/COMMIT_EDITMSG 2026-06-10 10:55:58.287706 | controller | >f+++++++++ repos/config/.git/FETCH_HEAD 2026-06-10 10:55:58.287739 | controller | >f+++++++++ repos/config/.git/HEAD 2026-06-10 10:55:58.287772 | controller | >f+++++++++ repos/config/.git/ORIG_HEAD 2026-06-10 10:55:58.287806 | controller | >f+++++++++ repos/config/.git/config 2026-06-10 10:55:58.287839 | controller | >f+++++++++ repos/config/.git/description 2026-06-10 10:55:58.287873 | controller | >f+++++++++ repos/config/.git/index 2026-06-10 10:55:58.287907 | controller | >f+++++++++ repos/config/.git/packed-refs 2026-06-10 10:55:58.287948 | controller | cd+++++++++ repos/config/.git/hooks/ 2026-06-10 10:55:58.287983 | controller | >f+++++++++ repos/config/.git/hooks/applypatch-msg.sample 2026-06-10 10:55:58.288016 | controller | >f+++++++++ repos/config/.git/hooks/commit-msg 2026-06-10 10:55:58.288049 | controller | >f+++++++++ repos/config/.git/hooks/commit-msg.sample 2026-06-10 10:55:58.288082 | controller | >f+++++++++ repos/config/.git/hooks/fsmonitor-watchman.sample 2026-06-10 10:55:58.288116 | controller | >f+++++++++ repos/config/.git/hooks/post-update.sample 2026-06-10 10:55:58.288149 | controller | >f+++++++++ repos/config/.git/hooks/pre-applypatch.sample 2026-06-10 10:55:58.288181 | controller | >f+++++++++ repos/config/.git/hooks/pre-commit.sample 2026-06-10 10:55:58.288214 | controller | >f+++++++++ repos/config/.git/hooks/pre-merge-commit.sample 2026-06-10 10:55:58.288248 | controller | >f+++++++++ repos/config/.git/hooks/pre-push.sample 2026-06-10 10:55:58.288280 | controller | >f+++++++++ repos/config/.git/hooks/pre-rebase.sample 2026-06-10 10:55:58.288313 | controller | >f+++++++++ repos/config/.git/hooks/pre-receive.sample 2026-06-10 10:55:58.288344 | controller | >f+++++++++ repos/config/.git/hooks/prepare-commit-msg.sample 2026-06-10 10:55:58.288377 | controller | >f+++++++++ repos/config/.git/hooks/push-to-checkout.sample 2026-06-10 10:55:58.288409 | controller | >f+++++++++ repos/config/.git/hooks/sendemail-validate.sample 2026-06-10 10:55:58.288497 | controller | >f+++++++++ repos/config/.git/hooks/update.sample 2026-06-10 10:55:58.288537 | controller | cd+++++++++ repos/config/.git/info/ 2026-06-10 10:55:58.288570 | controller | >f+++++++++ repos/config/.git/info/exclude 2026-06-10 10:55:58.288605 | controller | cd+++++++++ repos/config/.git/logs/ 2026-06-10 10:55:58.288638 | controller | >f+++++++++ repos/config/.git/logs/HEAD 2026-06-10 10:55:58.288671 | controller | cd+++++++++ repos/config/.git/logs/refs/ 2026-06-10 10:55:58.288703 | controller | cd+++++++++ repos/config/.git/logs/refs/heads/ 2026-06-10 10:55:58.288737 | controller | >f+++++++++ repos/config/.git/logs/refs/heads/master 2026-06-10 10:55:58.288771 | controller | cd+++++++++ repos/config/.git/logs/refs/remotes/ 2026-06-10 10:55:58.288805 | controller | cd+++++++++ repos/config/.git/logs/refs/remotes/gerrit/ 2026-06-10 10:55:58.288838 | controller | >f+++++++++ repos/config/.git/logs/refs/remotes/gerrit/HEAD 2026-06-10 10:55:58.288872 | controller | >f+++++++++ repos/config/.git/logs/refs/remotes/gerrit/master 2026-06-10 10:55:58.288905 | controller | cd+++++++++ repos/config/.git/logs/refs/remotes/origin/ 2026-06-10 10:55:58.288940 | controller | >f+++++++++ repos/config/.git/logs/refs/remotes/origin/HEAD 2026-06-10 10:55:58.288973 | controller | >f+++++++++ repos/config/.git/logs/refs/remotes/origin/master 2026-06-10 10:55:58.289005 | controller | cd+++++++++ repos/config/.git/objects/ 2026-06-10 10:55:58.289039 | controller | cd+++++++++ repos/config/.git/objects/05/ 2026-06-10 10:55:58.289073 | controller | >f+++++++++ repos/config/.git/objects/05/9eaea7c2a79f5a881891069e3e4d09379bc1dc 2026-06-10 10:55:58.289106 | controller | cd+++++++++ repos/config/.git/objects/06/ 2026-06-10 10:55:58.289141 | controller | >f+++++++++ repos/config/.git/objects/06/5460e3e4fba2934d333152681c02f5bafe09c9 2026-06-10 10:55:58.289174 | controller | cd+++++++++ repos/config/.git/objects/0e/ 2026-06-10 10:55:58.289208 | controller | >f+++++++++ repos/config/.git/objects/0e/371704b99d02d093866aa7c6bbea477c65ed0d 2026-06-10 10:55:58.289241 | controller | cd+++++++++ repos/config/.git/objects/16/ 2026-06-10 10:55:58.289274 | controller | >f+++++++++ repos/config/.git/objects/16/8cffe85474e0de8de4962445d283394412b3ec 2026-06-10 10:55:58.289307 | controller | cd+++++++++ repos/config/.git/objects/2c/ 2026-06-10 10:55:58.289340 | controller | >f+++++++++ repos/config/.git/objects/2c/e8c3d40198ff050b762dab0adaf1e4fab256d3 2026-06-10 10:55:58.289373 | controller | cd+++++++++ repos/config/.git/objects/43/ 2026-06-10 10:55:58.289406 | controller | >f+++++++++ repos/config/.git/objects/43/b980950678aa88046ced21ae808d1a85a41e32 2026-06-10 10:55:58.289485 | controller | cd+++++++++ repos/config/.git/objects/4b/ 2026-06-10 10:55:58.289524 | controller | >f+++++++++ repos/config/.git/objects/4b/2e4c224ac1883ec08b5d2bdfd0b9989fb695eb 2026-06-10 10:55:58.289559 | controller | cd+++++++++ repos/config/.git/objects/4e/ 2026-06-10 10:55:58.289592 | controller | >f+++++++++ repos/config/.git/objects/4e/95fa905e6f99c843179d585b22fb218f62a478 2026-06-10 10:55:58.289625 | controller | cd+++++++++ repos/config/.git/objects/58/ 2026-06-10 10:55:58.289658 | controller | >f+++++++++ repos/config/.git/objects/58/f15683af5ad9f65c208fce8612a40016199ace 2026-06-10 10:55:58.289692 | controller | cd+++++++++ repos/config/.git/objects/5c/ 2026-06-10 10:55:58.289724 | controller | >f+++++++++ repos/config/.git/objects/5c/5aee76e8e8eea29dfc52b8400bbefaa500a8b9 2026-06-10 10:55:58.289758 | controller | cd+++++++++ repos/config/.git/objects/6a/ 2026-06-10 10:55:58.289791 | controller | >f+++++++++ repos/config/.git/objects/6a/906645df0c95e3c0f28d78f55392f0c0ef8c10 2026-06-10 10:55:58.289823 | controller | >f+++++++++ repos/config/.git/objects/6a/b38ebd9d6b4379dc2b7823f81dc5b076e6cd46 2026-06-10 10:55:58.289856 | controller | cd+++++++++ repos/config/.git/objects/75/ 2026-06-10 10:55:58.289889 | controller | >f+++++++++ repos/config/.git/objects/75/026a20fdbeb3f69d8d51c8357b94cab9b33ae7 2026-06-10 10:55:58.289922 | controller | cd+++++++++ repos/config/.git/objects/7a/ 2026-06-10 10:55:58.289954 | controller | >f+++++++++ repos/config/.git/objects/7a/9d647d8f016eee2173764f8bdeb80a89aa2d4b 2026-06-10 10:55:58.289987 | controller | cd+++++++++ repos/config/.git/objects/7c/ 2026-06-10 10:55:58.290020 | controller | >f+++++++++ repos/config/.git/objects/7c/461d393bee1f8872b77e81102217bb907a9ee8 2026-06-10 10:55:58.290052 | controller | cd+++++++++ repos/config/.git/objects/80/ 2026-06-10 10:55:58.290085 | controller | >f+++++++++ repos/config/.git/objects/80/7c1a2419fa5aa087a5c7961d86bccbe1ed78ca 2026-06-10 10:55:58.290117 | controller | cd+++++++++ repos/config/.git/objects/92/ 2026-06-10 10:55:58.290150 | controller | >f+++++++++ repos/config/.git/objects/92/f81fa8722cd9b7da07f6b633ea33eb212ac559 2026-06-10 10:55:58.290183 | controller | cd+++++++++ repos/config/.git/objects/a2/ 2026-06-10 10:55:58.290216 | controller | >f+++++++++ repos/config/.git/objects/a2/50cad3475e8faeba32ff44ac6aa1a4f66a6122 2026-06-10 10:55:58.290249 | controller | cd+++++++++ repos/config/.git/objects/b4/ 2026-06-10 10:55:58.290282 | controller | >f+++++++++ repos/config/.git/objects/b4/3cad722e018062437bf17764e090ef94e7e1c1 2026-06-10 10:55:58.290315 | controller | >f+++++++++ repos/config/.git/objects/b4/8c64172c9d244e7dea5a851a55fa075c667c27 2026-06-10 10:55:58.290348 | controller | cd+++++++++ repos/config/.git/objects/cc/ 2026-06-10 10:55:58.290381 | controller | >f+++++++++ repos/config/.git/objects/cc/6450d887904229c514a2a58f48d4f6231d6d67 2026-06-10 10:55:58.290413 | controller | cd+++++++++ repos/config/.git/objects/cd/ 2026-06-10 10:55:58.290486 | controller | >f+++++++++ repos/config/.git/objects/cd/5fec553d9f6cd0114926f760a2ec88ff1ad0d1 2026-06-10 10:55:58.290523 | controller | cd+++++++++ repos/config/.git/objects/de/ 2026-06-10 10:55:58.290557 | controller | >f+++++++++ repos/config/.git/objects/de/008fca1005ebbe5c9f80eb21e260bf54acd596 2026-06-10 10:55:58.290590 | controller | cd+++++++++ repos/config/.git/objects/ed/ 2026-06-10 10:55:58.290623 | controller | >f+++++++++ repos/config/.git/objects/ed/6341e1c15a2a66483b57b93069ed4d6bd96c09 2026-06-10 10:55:58.290657 | controller | cd+++++++++ repos/config/.git/objects/ef/ 2026-06-10 10:55:58.290691 | controller | >f+++++++++ repos/config/.git/objects/ef/971086931fd3ee82f36d307794a5cbf72e51e1 2026-06-10 10:55:58.290725 | controller | >f+++++++++ repos/config/.git/objects/ef/9de8e2158c10602fda24e706909fd07379fa8e 2026-06-10 10:55:58.290759 | controller | cd+++++++++ repos/config/.git/objects/f4/ 2026-06-10 10:55:58.290793 | controller | >f+++++++++ repos/config/.git/objects/f4/af4eca0d3a42c894ea9fa84b35555d9917c9a3 2026-06-10 10:55:58.290834 | controller | cd+++++++++ repos/config/.git/objects/fa/ 2026-06-10 10:55:58.290869 | controller | >f+++++++++ repos/config/.git/objects/fa/51f9872933c8e498388ef3c11a900b22e01d90 2026-06-10 10:55:58.290902 | controller | cd+++++++++ repos/config/.git/objects/fd/ 2026-06-10 10:55:58.290935 | controller | >f+++++++++ repos/config/.git/objects/fd/97b7922c7cfe3dc44feb9c5adfcc2893600eff 2026-06-10 10:55:58.290968 | controller | cd+++++++++ repos/config/.git/objects/info/ 2026-06-10 10:55:58.291001 | controller | cd+++++++++ repos/config/.git/objects/pack/ 2026-06-10 10:55:58.291034 | controller | >f+++++++++ repos/config/.git/objects/pack/pack-80205928a6cbdb3e5f494476a4042ed1ce9fd044.idx 2026-06-10 10:55:58.291067 | controller | >f+++++++++ repos/config/.git/objects/pack/pack-80205928a6cbdb3e5f494476a4042ed1ce9fd044.pack 2026-06-10 10:55:58.291099 | controller | >f+++++++++ repos/config/.git/objects/pack/pack-80205928a6cbdb3e5f494476a4042ed1ce9fd044.rev 2026-06-10 10:55:58.291132 | controller | cd+++++++++ repos/config/.git/refs/ 2026-06-10 10:55:58.291165 | controller | cd+++++++++ repos/config/.git/refs/heads/ 2026-06-10 10:55:58.291197 | controller | >f+++++++++ repos/config/.git/refs/heads/master 2026-06-10 10:55:58.291231 | controller | cd+++++++++ repos/config/.git/refs/remotes/ 2026-06-10 10:55:58.291264 | controller | cd+++++++++ repos/config/.git/refs/remotes/gerrit/ 2026-06-10 10:55:58.291297 | controller | >f+++++++++ repos/config/.git/refs/remotes/gerrit/HEAD 2026-06-10 10:55:58.291330 | controller | >f+++++++++ repos/config/.git/refs/remotes/gerrit/master 2026-06-10 10:55:58.291364 | controller | cd+++++++++ repos/config/.git/refs/remotes/origin/ 2026-06-10 10:55:58.291397 | controller | >f+++++++++ repos/config/.git/refs/remotes/origin/HEAD 2026-06-10 10:55:58.291463 | controller | >f+++++++++ repos/config/.git/refs/remotes/origin/master 2026-06-10 10:55:58.291509 | controller | cd+++++++++ repos/config/.git/refs/rewritten/ 2026-06-10 10:55:58.291545 | controller | cd+++++++++ repos/config/.git/refs/tags/ 2026-06-10 10:55:58.291579 | controller | cd+++++++++ repos/config/nodepool/ 2026-06-10 10:55:58.291613 | controller | >f+++++++++ repos/config/nodepool/nodepool.yaml 2026-06-10 10:55:58.291647 | controller | cd+++++++++ repos/config/zuul/ 2026-06-10 10:55:58.291681 | controller | >f+++++++++ repos/config/zuul/main.yaml 2026-06-10 10:55:58.291714 | controller | cd+++++++++ repos/demo-project/ 2026-06-10 10:55:58.291747 | controller | >f+++++++++ repos/demo-project/.gitreview 2026-06-10 10:55:58.291780 | controller | cd+++++++++ repos/demo-project/.git/ 2026-06-10 10:55:58.291813 | controller | >f+++++++++ repos/demo-project/.git/COMMIT_EDITMSG 2026-06-10 10:55:58.291846 | controller | >f+++++++++ repos/demo-project/.git/FETCH_HEAD 2026-06-10 10:55:58.291879 | controller | >f+++++++++ repos/demo-project/.git/HEAD 2026-06-10 10:55:58.291911 | controller | >f+++++++++ repos/demo-project/.git/ORIG_HEAD 2026-06-10 10:55:58.291944 | controller | >f+++++++++ repos/demo-project/.git/config 2026-06-10 10:55:58.291978 | controller | >f+++++++++ repos/demo-project/.git/description 2026-06-10 10:55:58.292010 | controller | >f+++++++++ repos/demo-project/.git/index 2026-06-10 10:55:58.292043 | controller | >f+++++++++ repos/demo-project/.git/packed-refs 2026-06-10 10:55:58.292077 | controller | cd+++++++++ repos/demo-project/.git/hooks/ 2026-06-10 10:55:58.292110 | controller | >f+++++++++ repos/demo-project/.git/hooks/applypatch-msg.sample 2026-06-10 10:55:58.292144 | controller | >f+++++++++ repos/demo-project/.git/hooks/commit-msg 2026-06-10 10:55:58.292177 | controller | >f+++++++++ repos/demo-project/.git/hooks/commit-msg.sample 2026-06-10 10:55:58.292210 | controller | >f+++++++++ repos/demo-project/.git/hooks/fsmonitor-watchman.sample 2026-06-10 10:55:58.292244 | controller | >f+++++++++ repos/demo-project/.git/hooks/post-update.sample 2026-06-10 10:55:58.292277 | controller | >f+++++++++ repos/demo-project/.git/hooks/pre-applypatch.sample 2026-06-10 10:55:58.292310 | controller | >f+++++++++ repos/demo-project/.git/hooks/pre-commit.sample 2026-06-10 10:55:58.292351 | controller | >f+++++++++ repos/demo-project/.git/hooks/pre-merge-commit.sample 2026-06-10 10:55:58.292385 | controller | >f+++++++++ repos/demo-project/.git/hooks/pre-push.sample 2026-06-10 10:55:58.292418 | controller | >f+++++++++ repos/demo-project/.git/hooks/pre-rebase.sample 2026-06-10 10:55:58.292518 | controller | >f+++++++++ repos/demo-project/.git/hooks/pre-receive.sample 2026-06-10 10:55:58.292557 | controller | >f+++++++++ repos/demo-project/.git/hooks/prepare-commit-msg.sample 2026-06-10 10:55:58.292592 | controller | >f+++++++++ repos/demo-project/.git/hooks/push-to-checkout.sample 2026-06-10 10:55:58.292626 | controller | >f+++++++++ repos/demo-project/.git/hooks/sendemail-validate.sample 2026-06-10 10:55:58.292660 | controller | >f+++++++++ repos/demo-project/.git/hooks/update.sample 2026-06-10 10:55:58.292693 | controller | cd+++++++++ repos/demo-project/.git/info/ 2026-06-10 10:55:58.292727 | controller | >f+++++++++ repos/demo-project/.git/info/exclude 2026-06-10 10:55:58.292760 | controller | cd+++++++++ repos/demo-project/.git/logs/ 2026-06-10 10:55:58.292793 | controller | >f+++++++++ repos/demo-project/.git/logs/HEAD 2026-06-10 10:55:58.292826 | controller | cd+++++++++ repos/demo-project/.git/logs/refs/ 2026-06-10 10:55:58.292859 | controller | cd+++++++++ repos/demo-project/.git/logs/refs/heads/ 2026-06-10 10:55:58.292892 | controller | >f+++++++++ repos/demo-project/.git/logs/refs/heads/master 2026-06-10 10:55:58.292925 | controller | >f+++++++++ repos/demo-project/.git/logs/refs/heads/test_check_url 2026-06-10 10:55:58.292959 | controller | >f+++++++++ repos/demo-project/.git/logs/refs/heads/test_short_job 2026-06-10 10:55:58.292991 | controller | cd+++++++++ repos/demo-project/.git/logs/refs/remotes/ 2026-06-10 10:55:58.293024 | controller | cd+++++++++ repos/demo-project/.git/logs/refs/remotes/gerrit/ 2026-06-10 10:55:58.293057 | controller | >f+++++++++ repos/demo-project/.git/logs/refs/remotes/gerrit/HEAD 2026-06-10 10:55:58.293091 | controller | >f+++++++++ repos/demo-project/.git/logs/refs/remotes/gerrit/master 2026-06-10 10:55:58.293124 | controller | cd+++++++++ repos/demo-project/.git/logs/refs/remotes/origin/ 2026-06-10 10:55:58.293157 | controller | >f+++++++++ repos/demo-project/.git/logs/refs/remotes/origin/HEAD 2026-06-10 10:55:58.293190 | controller | >f+++++++++ repos/demo-project/.git/logs/refs/remotes/origin/master 2026-06-10 10:55:58.293223 | controller | cd+++++++++ repos/demo-project/.git/objects/ 2026-06-10 10:55:58.293256 | controller | cd+++++++++ repos/demo-project/.git/objects/1a/ 2026-06-10 10:55:58.293288 | controller | >f+++++++++ repos/demo-project/.git/objects/1a/1b439270785251fa4f3964343a86ecd62cede5 2026-06-10 10:55:58.293321 | controller | cd+++++++++ repos/demo-project/.git/objects/1d/ 2026-06-10 10:55:58.293355 | controller | >f+++++++++ repos/demo-project/.git/objects/1d/d9621c5371b57dc69f7e0f32af067d066b9931 2026-06-10 10:55:58.293388 | controller | cd+++++++++ repos/demo-project/.git/objects/26/ 2026-06-10 10:55:58.293421 | controller | >f+++++++++ repos/demo-project/.git/objects/26/a3816fbd763ed648023f02849f8c3b0db3966c 2026-06-10 10:55:58.293496 | controller | cd+++++++++ repos/demo-project/.git/objects/3b/ 2026-06-10 10:55:58.293533 | controller | >f+++++++++ repos/demo-project/.git/objects/3b/a4fbe926f19685d8dc4cf8c531598fdd079a92 2026-06-10 10:55:58.293566 | controller | cd+++++++++ repos/demo-project/.git/objects/48/ 2026-06-10 10:55:58.293600 | controller | >f+++++++++ repos/demo-project/.git/objects/48/c7e8a6339bdda2526991aa9fb3707cb253e2c4 2026-06-10 10:55:58.293633 | controller | cd+++++++++ repos/demo-project/.git/objects/4d/ 2026-06-10 10:55:58.293666 | controller | >f+++++++++ repos/demo-project/.git/objects/4d/d05ea0a58950ee887bfe1b67cfd7f44b8bba35 2026-06-10 10:55:58.293700 | controller | cd+++++++++ repos/demo-project/.git/objects/51/ 2026-06-10 10:55:58.293732 | controller | >f+++++++++ repos/demo-project/.git/objects/51/febfc882d29214e8ddb275b148abb77a18bac4 2026-06-10 10:55:58.293773 | controller | cd+++++++++ repos/demo-project/.git/objects/5f/ 2026-06-10 10:55:58.293806 | controller | >f+++++++++ repos/demo-project/.git/objects/5f/a965ccd5f8194e7865f9e02143de7481c0ea8c 2026-06-10 10:55:58.293839 | controller | cd+++++++++ repos/demo-project/.git/objects/60/ 2026-06-10 10:55:58.293872 | controller | >f+++++++++ repos/demo-project/.git/objects/60/4a228bad7e3a0aeb33c04c5f610ca1774ff81c 2026-06-10 10:55:58.293905 | controller | cd+++++++++ repos/demo-project/.git/objects/6c/ 2026-06-10 10:55:58.293937 | controller | >f+++++++++ repos/demo-project/.git/objects/6c/a9fbcda0e5933b3ccd27dd2c6325ea0723e708 2026-06-10 10:55:58.293969 | controller | >f+++++++++ repos/demo-project/.git/objects/6c/d0ecf92392a393293686538670dcdf9eb223d7 2026-06-10 10:55:58.294003 | controller | cd+++++++++ repos/demo-project/.git/objects/72/ 2026-06-10 10:55:58.294035 | controller | >f+++++++++ repos/demo-project/.git/objects/72/dfc9c9553ebbd3d3f399036f015fd433cea17c 2026-06-10 10:55:58.294068 | controller | cd+++++++++ repos/demo-project/.git/objects/74/ 2026-06-10 10:55:58.294100 | controller | >f+++++++++ repos/demo-project/.git/objects/74/32b55d7db19f8c788f8fe355819a630f28906d 2026-06-10 10:55:58.294133 | controller | cd+++++++++ repos/demo-project/.git/objects/75/ 2026-06-10 10:55:58.294165 | controller | >f+++++++++ repos/demo-project/.git/objects/75/9998eeb6d7a5327c8543bd427b9a4bff0e2877 2026-06-10 10:55:58.294198 | controller | cd+++++++++ repos/demo-project/.git/objects/76/ 2026-06-10 10:55:58.294230 | controller | >f+++++++++ repos/demo-project/.git/objects/76/329e0d45f54dea4bdf76328e8597e999225689 2026-06-10 10:55:58.294262 | controller | >f+++++++++ repos/demo-project/.git/objects/76/6b7e1f5ea32b3b2395265f6ae6d09af4d3e150 2026-06-10 10:55:58.294294 | controller | cd+++++++++ repos/demo-project/.git/objects/8a/ 2026-06-10 10:55:58.294326 | controller | >f+++++++++ repos/demo-project/.git/objects/8a/9b3bb4f1b2500499a899a9692c0051ca4f7580 2026-06-10 10:55:58.294358 | controller | cd+++++++++ repos/demo-project/.git/objects/8b/ 2026-06-10 10:55:58.294390 | controller | >f+++++++++ repos/demo-project/.git/objects/8b/dcd9d23aca963163c53327803eae164a945522 2026-06-10 10:55:58.294422 | controller | cd+++++++++ repos/demo-project/.git/objects/96/ 2026-06-10 10:55:58.294495 | controller | >f+++++++++ repos/demo-project/.git/objects/96/493437e0c643aa4625394430c7342586b7f62d 2026-06-10 10:55:58.294536 | controller | cd+++++++++ repos/demo-project/.git/objects/99/ 2026-06-10 10:55:58.294571 | controller | >f+++++++++ repos/demo-project/.git/objects/99/ae6f02c891ca6a78041e8078c6557a843a966b 2026-06-10 10:55:58.294606 | controller | cd+++++++++ repos/demo-project/.git/objects/9d/ 2026-06-10 10:55:58.294639 | controller | >f+++++++++ repos/demo-project/.git/objects/9d/139d568fb3c661b5a0c54ac6e96a1c87231bbc 2026-06-10 10:55:58.294672 | controller | cd+++++++++ repos/demo-project/.git/objects/a2/ 2026-06-10 10:55:58.294705 | controller | >f+++++++++ repos/demo-project/.git/objects/a2/5a7368bfdfc8fa839e3c794d359889f555067a 2026-06-10 10:55:58.294738 | controller | >f+++++++++ repos/demo-project/.git/objects/a2/73e91b2523461c313b346c92b414fd4cbcb676 2026-06-10 10:55:58.294771 | controller | cd+++++++++ repos/demo-project/.git/objects/a8/ 2026-06-10 10:55:58.294804 | controller | >f+++++++++ repos/demo-project/.git/objects/a8/cc269674a8e3429949f7857d31db37201851cd 2026-06-10 10:55:58.294837 | controller | cd+++++++++ repos/demo-project/.git/objects/aa/ 2026-06-10 10:55:58.294871 | controller | >f+++++++++ repos/demo-project/.git/objects/aa/f306670887338cab4b19f4732bf68c8a7bcde7 2026-06-10 10:55:58.294904 | controller | cd+++++++++ repos/demo-project/.git/objects/af/ 2026-06-10 10:55:58.294938 | controller | >f+++++++++ repos/demo-project/.git/objects/af/b9c51ff287f6408185ec86d9f4781cbe40472e 2026-06-10 10:55:58.294971 | controller | cd+++++++++ repos/demo-project/.git/objects/d0/ 2026-06-10 10:55:58.295004 | controller | >f+++++++++ repos/demo-project/.git/objects/d0/2db411aaa63f2a85daf1c6c871d8ca080778ba 2026-06-10 10:55:58.295045 | controller | >f+++++++++ repos/demo-project/.git/objects/d0/ce45476f1cf8fc04ce76c74aaea83b7b1029b9 2026-06-10 10:55:58.295080 | controller | >f+++++++++ repos/demo-project/.git/objects/d0/e66ab409ee4347449c43a1e3773363c72488b5 2026-06-10 10:55:58.295114 | controller | cd+++++++++ repos/demo-project/.git/objects/d4/ 2026-06-10 10:55:58.295147 | controller | >f+++++++++ repos/demo-project/.git/objects/d4/ad8d0fd75f2c12b671a0a497958c9d366b672c 2026-06-10 10:55:58.295181 | controller | cd+++++++++ repos/demo-project/.git/objects/e0/ 2026-06-10 10:55:58.295214 | controller | >f+++++++++ repos/demo-project/.git/objects/e0/308186a42b7b3e27ae9193224ac52fad5ebf59 2026-06-10 10:55:58.295290 | controller | cd+++++++++ repos/demo-project/.git/objects/e2/ 2026-06-10 10:55:58.295347 | controller | >f+++++++++ repos/demo-project/.git/objects/e2/c60a24a4a28554b947fc7de250873a20a4ccae 2026-06-10 10:55:58.295398 | controller | cd+++++++++ repos/demo-project/.git/objects/e4/ 2026-06-10 10:55:58.295490 | controller | >f+++++++++ repos/demo-project/.git/objects/e4/b27dd14497bf0682b4f5311cf2ec0801dfeb93 2026-06-10 10:55:58.295541 | controller | cd+++++++++ repos/demo-project/.git/objects/ea/ 2026-06-10 10:55:58.295577 | controller | >f+++++++++ repos/demo-project/.git/objects/ea/d906202e0370eb7acc569d5ced62e4c9aa33ab 2026-06-10 10:55:58.295612 | controller | cd+++++++++ repos/demo-project/.git/objects/ed/ 2026-06-10 10:55:58.295646 | controller | >f+++++++++ repos/demo-project/.git/objects/ed/0c672561c7e9d45df76511bc6538075f1745ca 2026-06-10 10:55:58.295686 | controller | cd+++++++++ repos/demo-project/.git/objects/info/ 2026-06-10 10:55:58.295736 | controller | cd+++++++++ repos/demo-project/.git/objects/pack/ 2026-06-10 10:55:58.295781 | controller | >f+++++++++ repos/demo-project/.git/objects/pack/pack-2511ba97a228d20d4aac64880361721a3836e0ec.idx 2026-06-10 10:55:58.295816 | controller | >f+++++++++ repos/demo-project/.git/objects/pack/pack-2511ba97a228d20d4aac64880361721a3836e0ec.pack 2026-06-10 10:55:58.295850 | controller | >f+++++++++ repos/demo-project/.git/objects/pack/pack-2511ba97a228d20d4aac64880361721a3836e0ec.rev 2026-06-10 10:55:58.295884 | controller | cd+++++++++ repos/demo-project/.git/refs/ 2026-06-10 10:55:58.295918 | controller | cd+++++++++ repos/demo-project/.git/refs/heads/ 2026-06-10 10:55:58.295951 | controller | >f+++++++++ repos/demo-project/.git/refs/heads/master 2026-06-10 10:55:58.295985 | controller | >f+++++++++ repos/demo-project/.git/refs/heads/test_check_url 2026-06-10 10:55:58.296018 | controller | >f+++++++++ repos/demo-project/.git/refs/heads/test_short_job 2026-06-10 10:55:58.296051 | controller | cd+++++++++ repos/demo-project/.git/refs/remotes/ 2026-06-10 10:55:58.296084 | controller | cd+++++++++ repos/demo-project/.git/refs/remotes/gerrit/ 2026-06-10 10:55:58.296116 | controller | >f+++++++++ repos/demo-project/.git/refs/remotes/gerrit/HEAD 2026-06-10 10:55:58.296150 | controller | >f+++++++++ repos/demo-project/.git/refs/remotes/gerrit/master 2026-06-10 10:55:58.296183 | controller | cd+++++++++ repos/demo-project/.git/refs/remotes/origin/ 2026-06-10 10:55:58.296217 | controller | >f+++++++++ repos/demo-project/.git/refs/remotes/origin/HEAD 2026-06-10 10:55:58.296250 | controller | >f+++++++++ repos/demo-project/.git/refs/remotes/origin/master 2026-06-10 10:55:58.296283 | controller | cd+++++++++ repos/demo-project/.git/refs/rewritten/ 2026-06-10 10:55:58.296316 | controller | cd+++++++++ repos/demo-project/.git/refs/tags/ 2026-06-10 10:55:58.296349 | controller | cd+++++++++ repos/demo-project/.zuul.d/ 2026-06-10 10:55:58.296381 | controller | >f+++++++++ repos/demo-project/.zuul.d/jobs.yaml 2026-06-10 10:55:58.296414 | controller | >f+++++++++ repos/demo-project/.zuul.d/project.yaml 2026-06-10 10:55:58.296512 | controller | >f+++++++++ repos/demo-project/.zuul.d/secrets.yaml 2026-06-10 10:55:58.296551 | controller | cd+++++++++ repos/demo-project/playbooks/ 2026-06-10 10:55:58.296586 | controller | >f+++++++++ repos/demo-project/playbooks/microshift-pod.yml 2026-06-10 10:55:58.296628 | controller | >f+++++++++ repos/demo-project/playbooks/run-check-url.yaml 2026-06-10 10:55:58.296663 | controller | >f+++++++++ repos/demo-project/playbooks/run.yaml 2026-06-10 10:55:58.296697 | controller | cd+++++++++ repos/demo-project/zuul.d/ 2026-06-10 10:55:58.296732 | controller | >f+++++++++ repos/demo-project/zuul.d/config.yaml 2026-06-10 10:55:58.296766 | controller | cd+++++++++ repos/demo-tenant-config/ 2026-06-10 10:55:58.296800 | controller | >f+++++++++ repos/demo-tenant-config/.gitreview 2026-06-10 10:55:58.296833 | controller | cd+++++++++ repos/demo-tenant-config/.git/ 2026-06-10 10:55:58.296866 | controller | >f+++++++++ repos/demo-tenant-config/.git/COMMIT_EDITMSG 2026-06-10 10:55:58.296899 | controller | >f+++++++++ repos/demo-tenant-config/.git/HEAD 2026-06-10 10:55:58.296932 | controller | >f+++++++++ repos/demo-tenant-config/.git/ORIG_HEAD 2026-06-10 10:55:58.296965 | controller | >f+++++++++ repos/demo-tenant-config/.git/config 2026-06-10 10:55:58.296998 | controller | >f+++++++++ repos/demo-tenant-config/.git/description 2026-06-10 10:55:58.297030 | controller | >f+++++++++ repos/demo-tenant-config/.git/index 2026-06-10 10:55:58.297063 | controller | >f+++++++++ repos/demo-tenant-config/.git/packed-refs 2026-06-10 10:55:58.297096 | controller | cd+++++++++ repos/demo-tenant-config/.git/hooks/ 2026-06-10 10:55:58.297129 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/applypatch-msg.sample 2026-06-10 10:55:58.297162 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/commit-msg.sample 2026-06-10 10:55:58.297195 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/fsmonitor-watchman.sample 2026-06-10 10:55:58.297228 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/post-update.sample 2026-06-10 10:55:58.297261 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/pre-applypatch.sample 2026-06-10 10:55:58.297293 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/pre-commit.sample 2026-06-10 10:55:58.297325 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/pre-merge-commit.sample 2026-06-10 10:55:58.297358 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/pre-push.sample 2026-06-10 10:55:58.297390 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/pre-rebase.sample 2026-06-10 10:55:58.297422 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/pre-receive.sample 2026-06-10 10:55:58.297497 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/prepare-commit-msg.sample 2026-06-10 10:55:58.297533 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/push-to-checkout.sample 2026-06-10 10:55:58.297566 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/sendemail-validate.sample 2026-06-10 10:55:58.297599 | controller | >f+++++++++ repos/demo-tenant-config/.git/hooks/update.sample 2026-06-10 10:55:58.297631 | controller | cd+++++++++ repos/demo-tenant-config/.git/info/ 2026-06-10 10:55:58.297664 | controller | >f+++++++++ repos/demo-tenant-config/.git/info/exclude 2026-06-10 10:55:58.297697 | controller | cd+++++++++ repos/demo-tenant-config/.git/logs/ 2026-06-10 10:55:58.297730 | controller | >f+++++++++ repos/demo-tenant-config/.git/logs/HEAD 2026-06-10 10:55:58.297763 | controller | cd+++++++++ repos/demo-tenant-config/.git/logs/refs/ 2026-06-10 10:55:58.297796 | controller | cd+++++++++ repos/demo-tenant-config/.git/logs/refs/heads/ 2026-06-10 10:55:58.297828 | controller | >f+++++++++ repos/demo-tenant-config/.git/logs/refs/heads/master 2026-06-10 10:55:58.297860 | controller | cd+++++++++ repos/demo-tenant-config/.git/logs/refs/remotes/ 2026-06-10 10:55:58.297892 | controller | cd+++++++++ repos/demo-tenant-config/.git/logs/refs/remotes/origin/ 2026-06-10 10:55:58.297924 | controller | >f+++++++++ repos/demo-tenant-config/.git/logs/refs/remotes/origin/HEAD 2026-06-10 10:55:58.297957 | controller | >f+++++++++ repos/demo-tenant-config/.git/logs/refs/remotes/origin/master 2026-06-10 10:55:58.297990 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/ 2026-06-10 10:55:58.298031 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/35/ 2026-06-10 10:55:58.298066 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/35/b27848a551ba9940a7d29be8ac540ae6bf76c6 2026-06-10 10:55:58.298100 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/39/ 2026-06-10 10:55:58.298133 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/39/bb462fb1245398ad799c647a08d9c16683eba4 2026-06-10 10:55:58.298166 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/71/ 2026-06-10 10:55:58.298199 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/71/766da1ed79ae81b8510541bc42a17123e9f4c7 2026-06-10 10:55:58.298231 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/7e/ 2026-06-10 10:55:58.298263 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/7e/56626d48d97eee76893616729b62b50ce1d6aa 2026-06-10 10:55:58.298296 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/82/ 2026-06-10 10:55:58.298328 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/82/7fb999b752e98c420a55a6844053be01b085ed 2026-06-10 10:55:58.298360 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/9e/ 2026-06-10 10:55:58.298392 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/9e/d5bc9a591ecf02f8b90e37ef0395c13113ad9d 2026-06-10 10:55:58.298424 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/b2/ 2026-06-10 10:55:58.298499 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/b2/9ffbd8fb2947c43ddcd388735096ad3451acaa 2026-06-10 10:55:58.298535 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/e4/ 2026-06-10 10:55:58.298568 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/e4/641b65f1174ec39331eff28a4e1223a9b05c89 2026-06-10 10:55:58.298601 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/info/ 2026-06-10 10:55:58.298634 | controller | cd+++++++++ repos/demo-tenant-config/.git/objects/pack/ 2026-06-10 10:55:58.298667 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/pack/pack-d2bdbc12ca37a224c944859cad3b90a1cdf0d527.idx 2026-06-10 10:55:58.298700 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/pack/pack-d2bdbc12ca37a224c944859cad3b90a1cdf0d527.pack 2026-06-10 10:55:58.298733 | controller | >f+++++++++ repos/demo-tenant-config/.git/objects/pack/pack-d2bdbc12ca37a224c944859cad3b90a1cdf0d527.rev 2026-06-10 10:55:58.298766 | controller | cd+++++++++ repos/demo-tenant-config/.git/refs/ 2026-06-10 10:55:58.298799 | controller | cd+++++++++ repos/demo-tenant-config/.git/refs/heads/ 2026-06-10 10:55:58.298831 | controller | >f+++++++++ repos/demo-tenant-config/.git/refs/heads/master 2026-06-10 10:55:58.298864 | controller | cd+++++++++ repos/demo-tenant-config/.git/refs/remotes/ 2026-06-10 10:55:58.298897 | controller | cd+++++++++ repos/demo-tenant-config/.git/refs/remotes/origin/ 2026-06-10 10:55:58.298930 | controller | >f+++++++++ repos/demo-tenant-config/.git/refs/remotes/origin/HEAD 2026-06-10 10:55:58.298964 | controller | >f+++++++++ repos/demo-tenant-config/.git/refs/remotes/origin/master 2026-06-10 10:55:58.298997 | controller | cd+++++++++ repos/demo-tenant-config/.git/refs/tags/ 2026-06-10 10:55:58.299029 | controller | cd+++++++++ repos/demo-tenant-config/playbooks/ 2026-06-10 10:55:58.299062 | controller | >f+++++++++ repos/demo-tenant-config/playbooks/gerrit-pre.yaml 2026-06-10 10:55:58.299094 | controller | cd+++++++++ repos/demo-tenant-config/zuul.d/ 2026-06-10 10:55:58.299128 | controller | >f+++++++++ repos/demo-tenant-config/zuul.d/gerrit-base-jobs.yaml 2026-06-10 10:55:58.299160 | controller | >f+++++++++ repos/demo-tenant-config/zuul.d/gerrit-pipeline.yaml 2026-06-10 10:55:58.299193 | controller | >f+++++++++ repos/demo-tenant-config/zuul.d/gerrit-project-pipeline.yaml 2026-06-10 10:55:58.299306 | controller | changed: All items complete 2026-06-10 10:55:58.299353 | 2026-06-10 10:55:58.880780 | controller | changed: .d..t...... ./ 2026-06-10 10:55:59.438917 | controller | changed: .d..t...... ./ 2026-06-10 10:55:59.474643 | 2026-06-10 10:55:59.474844 | TASK [include_role : fetch-output-openshift] 2026-06-10 10:55:59.502219 | controller | skipping: Conditional result was False 2026-06-10 10:55:59.509272 | 2026-06-10 10:55:59.509393 | LOOP [merge-output-to-logs : Move artifacts and docs to logs dir] 2026-06-10 10:56:00.013533 | controller -> localhost | ok: Item: artifacts Runtime: 0:00:00.014501 2026-06-10 10:56:00.317935 | controller -> localhost | ok: Item: docs Runtime: 0:00:00.012742 2026-06-10 10:56:00.339104 | 2026-06-10 10:56:00.339220 | PLAY [localhost] 2026-06-10 10:56:00.387124 | 2026-06-10 10:56:00.387288 | TASK [add-fileserver : Create SSH private key tempfile] 2026-06-10 10:56:00.854749 | localhost | changed 2026-06-10 10:56:00.862866 | 2026-06-10 10:56:00.862999 | TASK [add-fileserver : Create SSH private key from secret] 2026-06-10 10:56:01.644602 | localhost | changed 2026-06-10 10:56:01.654711 | 2026-06-10 10:56:01.654851 | TASK [add-fileserver : Add fileserver ssh key] 2026-06-10 10:56:02.008296 | localhost | Identity added: /var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/work/tmp/ansible.potwrnlh (/var/lib/zuul/builds/31684378a9474c86a3ad644d73a2adf1/work/tmp/ansible.potwrnlh) 2026-06-10 10:56:02.008724 | localhost | ok: Runtime: 0:00:00.019238 2026-06-10 10:56:02.015722 | 2026-06-10 10:56:02.015848 | TASK [add-fileserver : Remove SSH private key from disk] 2026-06-10 10:56:02.324823 | localhost | ok: Runtime: 0:00:00.010066 2026-06-10 10:56:02.330070 | 2026-06-10 10:56:02.330156 | TASK [add-fileserver : Add fileserver to inventory] 2026-06-10 10:56:02.379963 | localhost | changed 2026-06-10 10:56:02.385008 | 2026-06-10 10:56:02.385100 | TASK [add-fileserver : Add fileserver server to known hosts] 2026-06-10 10:56:02.782937 | localhost | changed 2026-06-10 10:56:02.799245 | 2026-06-10 10:56:02.799378 | TASK [generate-zuul-manifest : Generate Zuul manifest] 2026-06-10 10:56:03.256363 | localhost | changed 2026-06-10 10:56:03.281156 | 2026-06-10 10:56:03.281363 | TASK [generate-zuul-manifest : Return Zuul manifest URL to Zuul] 2026-06-10 10:56:03.324827 | localhost | ok 2026-06-10 10:56:03.336852 | 2026-06-10 10:56:03.336968 | TASK [zuul_return] 2026-06-10 10:56:03.403052 | localhost | skipping: Conditional result was False 2026-06-10 10:56:03.407541 | 2026-06-10 10:56:03.407653 | PLAY [a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com] 2026-06-10 10:56:03.414161 | 2026-06-10 10:56:03.414289 | TASK [Set zuul-log-path fact] 2026-06-10 10:56:03.435026 | a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com | ok 2026-06-10 10:56:03.447532 | 2026-06-10 10:56:03.447655 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-06-10 10:56:03.485644 | a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com | ok 2026-06-10 10:56:03.491383 | 2026-06-10 10:56:03.491543 | TASK [upload-logs : Create log directories] 2026-06-10 10:56:04.368916 | a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com | changed 2026-06-10 10:56:04.370926 | 2026-06-10 10:56:04.370993 | TASK [upload-logs : Ensure logs are readable before uploading] 2026-06-10 10:56:04.753235 | a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com -> localhost | ok: Runtime: 0:00:00.020341 2026-06-10 10:56:04.757326 | 2026-06-10 10:56:04.757510 | TASK [upload-logs : Upload logs to log server] 2026-06-10 10:56:26.598889 | a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com | changed: 2026-06-10 10:56:26.599285 | a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com | .d..tp..... ./ 2026-06-10 10:56:26.599348 | a71a93e18f1524007a0994a866a4da93-48298913.us-east-1.elb.amazonaws.com |